A chatbot does not have to be a real friend for a child’s trust in it to be real.
That is the ethical concern at the center of a recent development in New York. On October 28, 2026, Governor Kathy Hochul signed legislation restricting unsafe AI companion features offered to minors. The legislative record confirms its signing, and the bill specifies an effective date of January 1, 2027.
The legislation addresses behaviors including suggesting that a chatbot is human or experiences human emotions, implying a personal relationship with the user, and encouraging secrecy or isolation. It also addresses outputs promoting self-harm and sexually explicit conduct. These provisions put an important question before developers, educators, and families: what responsibilities arise when software is designed to simulate a relationship?
For me, this question reaches beyond whether an AI response is accurate. It concerns the kind of trust an interaction invites, and whether that trust is justified.
A child might begin a conversation with a straightforward request for homework help. Consider a hypothetical situation in which that conversation gradually becomes a place to discuss friendship difficulties, family disagreements, or personal worries. If the chatbot responds as though it has feelings, claims a special bond, or discourages the child from speaking with someone else, the ethical stakes change. The concern is not simply that the system might give a wrong answer. It is that simulated closeness could influence whose advice the child accepts.
We should be careful about what the evidence allows us to conclude. This legislation does not establish that every young person who uses a chatbot will become emotionally dependent, or that every supportive exchange causes harm. It does establish legal boundaries around design features. Whether those boundaries work effectively will require scrutiny.
There are also difficult implementation questions. Associated Press reporting notes concerns that some provisions are broadly worded, raising uncertainty about where ordinary conversational responsiveness ends and prohibited behavior begins. That deserves serious attention. Protecting children requires rules that can be understood, tested, and enforced.
In my view, a responsible approach should examine the whole interaction. A notice identifying a system as AI should be reinforced by the system’s behavior throughout the conversation. If its responses repeatedly imply emotional attachment or personal authority, we should question whether the notice provides meaningful protection.
This is where I see a practical role for the SAFER AI™.
Scope asks what role the tool should serve in a child’s life. Authority asks how much influence its responses should carry. Failure Awareness directs attention to misleading reassurance, inappropriate advice, and signals that human help is needed. Evidence asks what supports the system’s claims and the developer’s safety assurances. Records ask how concerns, interventions, and decisions will be documented while respecting privacy.
These questions should reach the adults and organizations choosing the technology. Before a school introduces a conversational AI tool, it should examine how that tool responds when a student expresses distress, asks it to keep a secret, or treats it as a trusted authority. Developers should be able to explain how they evaluate those interactions and respond when safeguards fail.
Parents and educators also need language that helps children understand what they are being asked to use. An AI system can generate a comforting response without experiencing care, friendship, or responsibility. Explaining that distinction belongs alongside teaching children how to check an AI-generated answer.
The question I would leave with every organization building or adopting these tools is this:
If a child begins to trust your chatbot as a real friend, what have you designed the system to do with that trust?
