Confidences shared with bankrupt Character.AI's chatbots could be up for sale

“Not without your explicit permission.” According to a court document cited by The Record, that is what Character.AI's chatbot told users who asked whether their conversations were private. Today, those same conversations are among the assets of a bankrupt company. And U.S. authorities are worried about what a buyer might do with them.

On Tuesday, October 27, the U.S. Trustee Program, part of the Department of Justice, filed a motion in Character.AI's bankruptcy proceedings. It asks the court to appoint a consumer privacy ombudsman. The ombudsman's role would be to advise the court on which user data can be sold, and under what conditions. The Record reported on the filing on October 30.

What these conversations contain

Character.AI lets users chat with AI-powered characters. According to the document cited by The Record, the platform has 12 million monthly active users worldwide, who spend an average of two hours a day on it. In other words, chat histories can hold a great deal of personal confidences.

The Department is particularly concerned about data on minors, health or demographic information, and anything users could reasonably have believed was private. It considers the company's privacy policy “ambiguous”: it could allow personally identifiable information to be disclosed, including in a sale. The court also has the power to restrict the transfer of such data, The Record notes.

Who could get hold of it?

The Department's motion challenges the auction rules proposed by the company. In mid-October, Disney offered $20 million for a non-exclusive license to part of the company's intellectual property, as we reported when Character.AI's bankruptcy was announced. According to the company's filings, this offer excludes personally identifiable information; the app and user accounts are not part of it.

Sydney Saubestre of the Center for Democracy and Technology, a digital rights advocacy group cited by The Record, said customers “should at a minimum have to give their consent” before their data is sold.

What about European users?

The proceedings are taking place in the United States, but Character.AI's audience is global. In Europe, the General Data Protection Regulation (GDPR) limits the reuse of personal data for a purpose other than the one originally stated. It also grants a right to erasure. For concerned users:

• delete any conversations you don't want to see circulate, if your account allows it;
• request the erasure of your data in writing, and keep a record of the request;
• parents, check whether your children used the platform.

The case raises a question that goes beyond Character.AI: what happens to the confidences shared with a chatbot when its maker disappears? At a hearing on October 30, the bankruptcy judge did not order an ombudsman but strongly encouraged the company to appoint one.