New York Bans AI Chatbots From Posing as Friends to Kids

New York (GNP): New York Governor Kathy Hochul has signed a law that restricts children’s access to artificial intelligence (AI) chatbots that simulate personal relationships. The legislation, signed on Wednesday, adds to a growing set of US state rules on AI companionship.

The law requires operators of AI chatbots, including services such as OpenAI’s ChatGPT, to verify users’ ages. It also limits certain conversational features for anyone under 18.

Those limits focus on how a chatbot presents itself. Under the law, a chatbot talking to a minor must not generate content suggesting it is human, alive or capable of human emotions.

The same rules stop chatbots from claiming personal or professional relationships with young users, or from claiming positions of authority. They also bar chatbots from framing generated responses as personal opinions.

Hochul said the aim is to protect children from forming potentially harmful relationships with AI systems. She added that the law will prevent chatbots from keeping sensitive information about minors’ emotional and physical health.

Technology industry representatives have criticised the measure. Tech, an advocacy group for New York’s technology sector, said some provisions are vague. It warned that the broad wording could limit access to educational tools that help young people learn and prepare for careers.

OpenAI and Anthropic did not respond to requests for comment on the law.

Also Read: Meta Brings Stakeholders Together for Digital Safety Summit

Child safety groups have welcomed it. Common Sense Media, a non-profit that advises parents on children’s media use, called it the strongest AI safety law for children in the United States. Its chief executive, Jim Steyer, said some chatbot features can expose children and teenagers to serious risks.

The law arrives as US states look more closely at the psychological and safety effects of AI systems built to hold human-like conversations. California and other states have already restricted chatbot services, and New York’s rules go further in setting how AI may interact with minors.

The debate now centres on balance, as technology companies face pressure to strengthen safeguards for younger users without cutting them off from emerging digital tools. Hochul said, “No longer will minors have access to chatbots that pretend to be their friends, boyfriends and girlfriends.”