The FDA Is Already Outsourcing Drug and Food Analysis to Error-Plagued AI Chatbot

In case you haven't had enough about artificial intelligence, the US Food and Drug Administration is now outsourcing its oversight duties to a large language model (LLM.)

In an article published in the Journal of the American Medical Association (JAMA), FDA bureaucrats Marty Makary and Vinay Prasad — the latter of whom is a noted critic of COVID mask mandates and vaccine boosters — laid out a five-point list of priorities that the federal agency is hoping to tackle. Among them is "unleashing AI" via a plan to use chatbots to "radically increase efficiency in the review process."

This includes reviewing documents, drug applications "that often exceed 500,000 pages," and data formatting. This goes hand in hand with other priorities to "accelerate cures" by delivering "rapid or instant reviews" of drug products, and deliver "healthier food to children" by quickly approving "novel food dyes that use natural ingredients."

On top of drugs, the FDA also wants to blow the lid off of AI reviews and medical device evaluation in order to "fuel innovation." That's a clever way to say "open the flood gates" for pharmaceutical superprofits, not coincidentally a major talking point for PhRMA, the drug industry's lobbying group.

Probably conscious of this criticism, the article's coauthors specifically say they will "guard against a cozy relationship" with drug industry moguls. While this remains to be seen, the flipside under Health Secretary RFK Jr — a notorious friend to the alternative medicine industry — is hardly much better.

Regardless, experts aren't impressed with the AI agenda.

"How is this guarding the agency ‘against a cozy relationship’ with industry?" Dr. Reshma Ramachandran, director of a major research and policy initiative at Yale, asked the New York Times. The FDA's priorities "read as though they’re straight out of PhRMA’s playbook," she added.

As far as the AI goes, the FDA had already been testing an "AI-assisted scientific review pilot" back on May 8, the JAMA article notes. Last week, that was revealed to be "Elsa," an LLM running on GovCloud, a platform developed by Amazon for the US intelligence community.

Though evidence suggests LLM chatbots are becoming worse at summarizing scientific research as time goes on, the government has already been using Elsa to read reports, summarize unintended drug harms, compare product labels, and generate code.

Whether or not Elsa was responsible for RFK Jr's "Make America Healthy Again" dossier, which cited AI-hallucinated studies, remains to be seen. Still, the sloppified report could be a harrowing glimpse of things to come as the agency looks to offload its critical oversight work to an untested chatbot.

Elsa's rollout comes after over 2,000 of the FDA's 10,000 workers have been sent packing under the Trump administration.

That process began first under Elon Musk's DOGE, where device inspectors responsible for regulating his Neuralink device were axed. Musk's moves were followed by broader cuts under RFK Jr, who recently ousted the CDC's entire panel of vaccine advisors before installing his own vaccine-skeptic picks.

More on AI: Apple Quietly Working on AI Agent to "Replica" a Human Doctor

The post The FDA Is Already Outsourcing Drug and Food Analysis to Error-Plagued AI Chatbot appeared first on Futurism.