Millions of students now turn to AI chatbots every day. They use them for homework help, emotional support, and even personal advice. But most students have little idea where those conversations end up. They may also not know who can access them or whether their data is used to train AI systems.
That quiet unease is now turning into policy. Microsoft has signed a sweeping, legally binding agreement with the American Federation of Teachers (AFT).
The agreement sets rules for how Microsoft handles student and educator data in its AI tools. It is also reshaping the conversation about AI use in classrooms across the United States.
What Microsoft Actually Agreed To
The agreement, announced jointly by Microsoft and AFT president Randi Weingarten, goes further than most existing tech industry privacy pledges. Under its terms, Microsoft will not use student or educator data to train its AI systems, with one narrow exception carved out for data that could help protect student safety. The company has also committed to never selling that data, using it for advertising, or repurposing it for product development.
Perhaps the most notable clause bans AI companions or features explicitly “designed to foster emotional attachment or dependency.” Microsoft also agreed that its tools should not be designed to keep students engaged longer than necessary. The move responds to growing concerns about AI products using engagement tactics similar to social media apps.
What sets this agreement apart from a typical corporate privacy policy is enforceability. It’s legally binding and includes provisions for third-party audits to verify compliance. AFT officials say that structure alone could make it a template other districts and unions push for nationally.
Why Google Is the Big Question Mark
Google is the dominant provider of education technology in American schools, powering everything from Classroom to Docs to, increasingly, Gemini. That makes its silence on this issue hard to ignore. Weingarten said she personally reached out to Google ahead of the announcement, hoping the company would sign on as well. According to the AFT, Google has not responded to requests for comment on whether it intends to offer similar protections.
The timing adds to the pressure. Google recently expanded AI access within Classroom. Gemini and NotebookLM are now available to schools that choose to enable them.
Earlier this year, Google restricted Gemini access to students aged 18 and older. The change marks a shift in how deeply AI is being integrated into everyday school tools. It also comes as Microsoft makes new binding privacy commitments public.
Two other major AI companies, OpenAI and Anthropic, say they are in active discussions with the AFT. The talks focus on similar safety and privacy arrangements.
Google has not announced a comparable agreement with the AFT. This is notable given Google’s significant presence in classroom technology and school infrastructure.
School Districts Aren’t Waiting Around
While tech companies negotiate, some of the largest school systems in the country are moving independently. New York City and Los Angeles have both implemented yearlong moratoriums on certain AI tools while they conduct extensive audits of their education technology contracts. The audits are focused squarely on data privacy, transparency, and accountability, the same themes at the heart of the Microsoft-AFT deal.
The scope of these bans differs by district. In New York City, the restriction applies to all elementary and middle school students. Los Angeles has gone further, extending its moratorium through high school. These local decisions reflect a broader trend: districts are no longer content to rely solely on vendor assurances and are instead building their own oversight mechanisms before allowing AI tools deeper into classrooms.
The Bigger Privacy Picture
Concerns about student data extend well beyond any single company’s policy. Reporting on AI tools in education has found meaningful gaps in how different platforms handle sensitive information. Some tools retain deeply personal content, like an essay describing a student’s traumatic experience, without clear limits on how long that data is kept or who can access it. Age requirements also vary widely: some platforms require parental permission for teen users, while others do not require it at all. Several major AI providers reserve the right to share collected data with “affiliates,” which can include federal authorities under certain circumstances.
For teachers and administrators, this patchwork of policies creates real risk. Something as routine as asking a chatbot to draft an email about a student’s grade can run afoul of federal privacy law under FERPA if sensitive information is entered into a tool without adequate safeguards. And schools rarely rely on just one or two AI products. Thousands of AI-driven edtech tools, many from smaller vendors with far less scrutiny than Microsoft or Google, are already in use across classrooms, often without the kind of formal privacy review larger platforms are now facing.
What This Means Going Forward
The Microsoft-AFT agreement matters less as a single company’s policy update and more as a potential industry benchmark. If it holds up under audit and gains traction with other unions and districts, it could push the rest of the AI industry toward similar commitments, whether they come voluntarily or under public pressure.
For now, one question remains. Will Google and smaller EdTech vendors follow Microsoft’s lead?
These companies provide much of the software used in American classrooms. Until stronger protections are adopted, much of the responsibility for protecting student data will remain with teachers and school districts.
They must navigate a fast-moving technology landscape. Privacy protections also remain uneven and continue to evolve.
