Avoiding a Cautionary Tale: Policy Considerations for Artificial Intelligence in Health Care

On November 8, 2023, the Senate Health, Education, Labor and Pensions (HELP) Committee Subcommittee on Primary Health and Retirement Security discussed the impact of artificial intelligence (AI) on the healthcare sector in the Committee’s second AI hearing in nine days. The hearing comes as the White House and Congressional leaders seek to quickly respond to AI threats, mitigate its dangers, and harness its potential for American industry. Senators discussed the recent Executive Order issued by the White House to guide AI regulation and innovation across all sectors, including in the health and human services sectors.

Opening Statements

Subcommittee Chairman Ed Markey (D-MA) introduced to the Committee the importance of AI’s role in healthcare by hearkening back to social media’s rise in popularity, noting that large social media companies “made big promises” for innovation, democracy, and community, but instead “unleashed” critical complications, particularly for youth. Senator Markey also observed that unregulated AI could lead to the next pandemic, “supercharge” inequalities in healthcare systems, and jeopardize privacy, among other risks. Stating that Congress cannot make the mistake of allowing big tech to self-regulate, which puts “profit over people every time,” Markey mentioned three of his bills on AI, including two he has introduced with bipartisan cosponsors: the Children and Teens’ Online Privacy Protection Act (COPPA 2.0) with Senator Bill Cassidy (R-LA), the AI Biosecurity Risk Assessment Act, with Senator Ted Budd (R-NC) and Securing Gene Synthesis Act, co-introduced with Representative Anna Eshoo (D-CA).

Ranking Member of the Subcommittee Roger Marshall (R-KS) recognized AI’s potential to revolutionize healthcare, with benefits including better standards of care, timely access to care, and discovery of innovative treatments. Senator Marshall also expressed concern about potential AI applications for biosecurity issues, noting the risk of bioterrorism in particular. He urged the implementation of guardrails that maintain innovation and American leadership in the space. He also endorsed Senator Cassidy’s whitepaper exploring a Congressional framework for the future of AI in healthcare.

Expert Witness Testimony

Ms. Christine Huberty, Supervising Attorney, Greater Wisconsin Agency on Aging Resources

Telling a story of a patient failed by AI-directed care, Huberty’s testimony focused on algorithms that decide a patient’s recovery timeline, which she argued do not prioritize medical recommendations. Huberty shared stories of insurance companies using AI-directed care to deny patients continued care, saddling them with the prospect of mounting out-of-pocket costs and worsening conditions. Huberty also noted that AI-directed care also harms healthcare centers because of the resulting administrative burdens associated with denial appeals. As a result, some healthcare centers refuse to treat patients whose insurance uses predictive technology. Noting it is “unrealistic” to eliminate AI from the healthcare system, Huberty urged the committee to address the specific algorithmic issue she identified. Dr. Thomas Inglesby, Director, Johns Hopkins Center for Health Security

In his statement, Dr. Inglesby warned of high consequence risks that require priority legislative attention, including the potential for AI to simplify the creation of dangerous viruses that are now extinct or only exist in research labs. He also noted the potential for AI to enable, accelerate, or simplify the creation of new biological constructs such as viruses that could start a new pandemic. Inglesby maintained that last week’s AI Executive Order (EO) launched strong actions to address biosecurity risks, adding that private AI developers have taken similar measures, but noted more action is needed. He recommended three immediate steps:

1. 1. Provide HHS with the authority and resources to require anyone purchasing synthetic nucleic acids in the U.S. to purchase only from a nucleic acid provider conducting sequence and customer screening, irrespective of funding source.
   2. Commission rapid risk assessment to identify whether the EO signed last week will adequately address biologic risks or whether additional Congressional action is needed.
   3. Require entities developing products with significant dual use risks to evaluate and red team models, identify and address risks; Congress should also task an agency with auditing the dual use models.

Dr. Kenneth D. Mandl, Harvard Professor and Director, Computational Health Informatics Program, Boston Children’s Hospital

Dr. Mandl’s testimony focused on the use of Large Language Models (LLMs), which he said will transform healthcare delivery sooner than previously anticipated. Mandl expressed the concern that AI is being adopted in healthcare without guardrails and suggested this lack of guardrails may lead to “complex and distorted” clinical work flows.  

Dr. Keith Sale, Vice President and Chief Physician Executive of Ambulatory Services, The University of Kansas Health System

Dr. Sale said the most impactful result of AI in healthcare might be on physician burnout. According to Sale, burnout resulting from transcription, documentation, and other administrative burdens impacts patient care, which can be profoundly improved by AI. Dr. Sale said Congress should focus its regulatory effort, on privacy and adherence to Health Insurance Portability and Accountability Act (HIPAA) guidelines, data security, and traceability of AI-generated information.

Hearing Highlights

Reducing clinicians’ administrative burdens

In his opening remarks and questions to witnesses, Senator Markey highlighted safety concerns related to worker liability and AI output. Senator Markey asked about the danger of using health AI to monitor administrative tasks and diagnostic tasks in clinical settings. Dr. Mandl emphasized that the clinician workforce should understand the functionalities and limitations of AI tools, including ergonomics and workflow issues. Dr. Mandl also urged developers to design algorithms focused on improving administrative burdens and physician-patient relationships while maintaining safety He also encouraged Congress to create and implement legal and ethical safeguards to protect health care workers from liability.

Protecting patients’ privacy

Patient privacy and data security was of top priority to the Senators present. Senator Maggie Hassan (D-NH) highlighted the need for AI standards in medical settings to protect privacy, asking how Congress could help set these standards. Senator John Hickenlooper (D-CO) questioned how to use real world data while also establishing guardrails that promote data security. Dr. Mandl, to both of these questions, underscored the need to continuously reevaluate and maintain algorithms in use in order to ensure that AI is improving the value of care and acting as intended. He also urged health system actors to implement standards widely to avoid individual company monopolization of AI innovation.

Preventing AI-related biosecurity risk

Several Senators stressed the biosecurity risk related to the use of AI in healthcare, from preventing bad actors from spreading highly transmissible pathogens to how to use AI to prevent biosecurity failures. Senator Tina Smith (D-MN) asked Dr. Inglesby about the use of AI in Covid-19 vaccine development and about the lessons learned from using AI to prevent biosecurity risks. Dr. Inglesby noted that AI can improve the speed, precision, and efficiency in vaccine and drug development, making vaccines and treatments more powerful. However, he warned that these processes can be accidentally or intentionally used to create uncontrollable biological weapons and recommended that Congress conduct government audits of biotech companies.

Senators Hassan and Hickenlooper also expressed interest in the use of AI in pathogenic outbreak response. Senator Hassan asked how AI algorithms could identify antimicrobial resistance to help public health authorities identify and respond to outbreaks more quickly, while Senator Hickenlooper highlighted the importance of preventing actors from spreading highly transmissible pathogens, asking how risk would be assessed without federal guardrails.  

Preventing bias and ensuring accountability

Senator Ben Ray Luján (D-NM) raised concerns about AI perpetuating existing racial and ethnic biases in health care, specifically regarding bias issues in data sets that are used to train AI systems. Dr. Mandl responded that collecting representative data is essential to prevent bias and added that interoperability across the health system, in additional to diverse staff to conduct reinforcement learning, is needed in order to obtain representative data to train AI algorithms.

Conclusion

Congress and the Administration are working to keep pace with rapidly developing innovation in AI and in health care. Crowell & Moring, LLP will continue to federal government efforts to regulate AI. Our lawyers and public policy professionals are available to advise clients wishing to play an active role in current policy debates or seeking to navigate AI-related concerns in government contracts, employment law, intellectual property, privacy, healthcare, antitrust, or other strategic areas.