Adams-Era Bot Advised Users to Break Four Laws in First Three Months, City Considered This a Beta Period
Bohiney Magazine | The London Prat
NYC’s $600,000 AI Chatbot Told Landlords They Could Discriminate and Businesses Could Steal Tips, Mamdani Shuts It Down, Promises Better Chatbot
NEW YORK, NY — Mayor Zohran Mamdani announced this week the termination of New York City’s official AI chatbot, a $600,000 municipal customer service tool deployed by the previous administration that, according to joint reporting by The Markup and The City, confidently advised users to commit at least four violations of city law, including telling landlords they could discriminate against Section 8 voucher holders, informing businesses they could refuse cash payments despite a 2020 city law to the contrary, and getting the minimum wage wrong.
The bot was described by former Mayor Adams at its launch as so impressive that “people are going to come and watch what we’re doing in New York City.” People did not come. The bot continued providing incorrect legal advice to anyone who asked.
The Technology Assessment
The chatbot was built at a cost of nearly $600,000, deployed to help business owners navigate city regulations, and operated for approximately two years during which it told an unknown number of business owners that they could do things they cannot do. The city added disclaimers. The disclaimers said users should not treat the bot’s responses as legal advice. Users who had already been told by the bot that tip theft was acceptable were not retrospectively notified.
Mayor Mamdani, upon learning of the chatbot, identified it as a budget savings opportunity and announced its termination at his “Adams Budget Crisis” press conference. He also promised to build “the best chatbot system on the globe,” a statement that, given the performance of the previous chatbot system, represents both a low bar and an ambitious vision simultaneously. Britain’s MIT coverage of LLMs suggests the technology is capable of remarkable things. New York City’s experience suggests it is also capable of telling a restaurant owner that stealing from employees is fine.
The Broader Lesson
The episode illustrates a pattern in municipal AI deployment: the press conference announcing the chatbot is excellent; the chatbot itself is not. AI is now capable of saying sorry without meaning it, which is more than the previous NYC chatbot did — it never apologized, it just kept giving advice. The new chatbot, when deployed, will presumably do better. The bar has been clearly established at “do not advise people to break the law.” Officials say this is achievable.
SOURCE: https://bohiney.com/mouse-wiggle-now-counts-as-innovation-data/