FTC Cracks Down on AI Model’s AI Detection Claims

The Federal Trade Commission (FTC) issued a proposed order prohibiting Workado, LLC, from advertising the accuracy of its artificial intelligence (AI) detection products unless it possesses competent and reliable evidence demonstrating that the model performs as depicted.

Workado marketed its AI Content Detector, a tool using an AI model itself, as a product that could determine whether online content was developed using generative AI technology, like ChatGPT, or if it was written by a human. Workado promoted the AI Content Detector as “98 percent” accurate in detecting whether text was written using generative AI technology. The company claimed that AI Content Detector was developed using a wide range of material, including blog posts and Wikipedia entries, to make it more accurate for the average user.

However, independent testing showed the accuracy rate on general-purpose content was just 53 percent, according to the FTC’s administrative complaint. The FTC alleged that the AI model powering the AI Content Detector was only trained or fine-tuned to effectively classify academic content. In other words, Workado’s product did “no better than a coin toss,” according to Chris Mufarrige, Director of the FTC’s Bureau of Consumer Protection, who also stated, “Misleading claims about AI undermine competition by making it harder for legitimate providers of AI-related products to reach consumers.”

The proposed order prohibits Workado from engaging in similar false, misleading, or unsupported advertising in the future. Under the proposed order, Workado:

• Cannot make any representations about the effectiveness of any covered product unless it is not misleading, and the company has “competent and reliable” evidence to support the claim at the time it is made;
• Must retain any evidence it uses to support such efficacy claims;
• Must email eligible consumers about the consent order and settlement with the Commission; and
• Must submit four annual compliance reports to the FTC beginning one year after the order is issued.

Although the proposed order settling the complaint does not include a monetary component, each future violation of a consent order may result in a civil penalty of up to $53,088.

The Commission voted 3-0 to issue the administrative complaint and to accept the consent agreement. The FTC is currently governed by only three Republican commissioners; since taking office in January, President Donald Trump appointed Republican Commissioner Andrew Ferguson as FTC chair, former-Chair Lina Khan resigned, President Trump abruptly fired the FTC’s two remaining Democratic commissioners (Commissioners Slaughter and Bedoya, who are currently fighting their dismissal) and the U.S. Senate has confirmed Mark Meador to fill its third Republican seat. For more information on the firing of Commissioners Slaughter and Bedoya, see Crowell & Moring’s client alert: Trump Fires the FTC’s Two Democratic Commissioners.

On January 17, 2025, before he was appointed FTC Chair, Commissioner Ferguson issued a concurring and dissenting statement to the Khan-led FTC’s Staff Report on AI Partnerships & Investments 6(b) Study, advocating for a balanced approach to analyzing AI’s effects on competition by stating: “On the one hand, the Commission must not charge headlong to regulate AI. Such regulation could strangle this nascent technology in its cradle, or move the development of the technology to foreign states hostile to our national interests. On the other hand, the Commission must remain a vigilant competition watchman, ensuring that Big Tech incumbents do not control AI innovators in order to blunt any potential competitive threats.”

FTC Commissioner Melissa Holyoak recently confirmed that AI was one of the agency’s top priorities under the new administration. During her keynote address on April 22 at the IAPP's Global Privacy Summit in Washington, D.C., Commissioner Holyoak said that the FTC would “aggressively root out AI-powered frauds and scams and stop companies from making false or unsubstantiated representations that harm consumers” though she also iterated that flexibility is needed to avoid “misguided enforcement actions or excessive regulation” that could stifle innovation and competition in the emerging field.

In September 2024, as part of an enforcement sweep called Operation AI Comply, the FTC revealed a number of enforcement actions aimed at cracking down on the use of AI to “supercharge” harmful and deceptive business practices. The cases include one against DoNotPay, which claimed its “AI Lawyer” was “the world's first robot lawyer,” though the FTC alleged that it failed to live up to its claims. Also, the FTC announced an order against Sitejabber for its AI-enabled review platform that misrepresented consumer ratings and reviews: FTC Announces Final Order Against AI-Enabled Review Platform Sitejabber for Misrepresenting Consumer Ratings and Reviews.

As AI products and related performance claims are likely to continue to remain a focus for the FTC, businesses should continue to exercise diligence when making performance claims about consumer-facing AI models.