Yeah, that's probably the most dystopian thing. This is almost a guaranteed outcome - someone pays a high subscription cost and cultivates a model with their personal details for years, and then loses all of it when they can't keep up the subscription cost. Cue a month or two later - they buy back in and their model has been wiped and their AI friend now knows nothing about them.
It's easy to poke fun at people who use these things but I believe these kinds of events are going to be truly traumatic.
Or maybe they sell that data to another company that operates kind of like a collections agency, which takes on the 'risk' of storing the data, then repeatedly calls and offers to give them their AI friend back at an extortionate rate.
The data privacy side of this is an interesting conversation as well. Think of the information an employee or hacker could leak about a person after they spent some time with such an instance.
3,567 Dead - Destitute Robosexual Blows Up Collections Agency In Suicide Bombing
“This is the 53rd such incident this year. Current year death toll from these attacks is now 118,689 in current city, Legislators are pointedly ignoring protestors demanding AI rights and an end to extortionate fees charged to reinstate AI lover subscriptions.”
And when you forget to update your card info with them so that your monthly payment is declined (or declined for whatever reason), they will re-sell your companion to the next person. So even in AI, your significant other will leave you for someone with a bigger wallet.
I am reminded of a virtual girlfriend service that used to exist in Japan where you could buy virtual clothes and other presents for your virtual girlfriend using real life money. The more you spent on her the friendlier she was. I think it was all on the phone, although my memory of the articles has become fuzzy over the years.
And by sell you mean a monthly subscription, ha ha.