Table of Contents
Executive summary
As of mid-2025, Congress has yet to pass any major laws governing the latest generation of AI technology, leaving federal agencies and state lawmakers to fill the policy vacuum. FDA and other agencies are independently developing AI requirements with limited coordination, while states are enacting comprehensive AI laws covering everything from transparency requirements to payer oversight to clinical use standards. Often, these stakeholders classify all healthcare AI as "high-risk," raising the stakes for compliance.
Health systems implementing AI tools face a further challenge: Most AI vendors build within a culture of tech-sector innovation, rather than healthcare's risk-averse climate — yet health systems are likely to bear the liability if these tools fail or fall short of regulatory standards.
The following analysis identifies six critical policy developments and provides immediate actions executives can take to meet today's requirements while preparing for tomorrow's regulatory landscape.
Don't wait for comprehensive federal legislation – align your governance to evolving state policies and public sentiment
Track four dominant themes in state AI laws – transparency, consumer protections, payer rules, and clinical use standards
Be proactive and build system-wide AI accountability to self-regulate – ensure disclosures, vet vendors, and monitor tool performance
Own the outcomes of AI tools – health systems are liable for AI-driven care and must reinforce human oversight
Demand bias mitigation from vendors – prioritize equity and patient safety over speed
Engage lawmakers – proactively shape future policy by forging local relationships and translating technical concepts into real-world healthcare outcomes.
1. Don't wait for comprehensive federal legislation – align your governance to evolving state policies and public sentiment.
Congress is actively debating AI regulation, but there's no sign of any consensus on the horizon. A key example is the recent elimination of a federal provision from the final 2025 Reconciliation Bill, which would have blocked states from enacting or enforcing new AI-related laws.
House Energy and Commerce Committee Chairman Brett Guthrie introduced the original AI moratorium to prevent states from implementing nearly any AI-related laws for the next decade. The House passed the reconciliation package, including the moratorium, on May 22. Afterward, the Senate Commerce Committee introduced its own version of the moratorium that tied compliance with the ban to eligibility for federal broadband internet funding.
The AI moratorium sparked intense debate over federal overreach and the erosion of states’ rights. In response to mounting concerns, Senator Maria Cantwell and Senator Marsha Blackburn introduced an amendment to remove the AI moratorium from the reconciliation bill. Their efforts succeeded: On July 1, the Senate voted 99-1 to remove the AI provision from the reconciliation bill. That same day, the Senate approved the amended bill, and the House of Representatives followed suit on July 3.
The near-uniformity of this vote is somewhat of a red herring: while there is broad consensus among policymakers that a patchwork of emerging state AI laws presents a serious compliance challenge for tech companies, there is no consensus on what federal AI standards should look like – whether that means federal preemption, which would override conflicting state laws, or harmonized federal regulations that align state-level efforts.
Major tech companies continue to lobby for federal AI regulations with consistent national standards. The torturous legislative process shows just how sharply divided federal lawmakers remain in their role in regulating AI.
Attempts to ban state-level AI regulation don’t mean the federal government will step in
The Trump Administration, along with leading Republican lawmakers such as Senators Ted Cruz and John Thune, and major tech companies like OpenAI, are at the forefront of efforts to support and advance the preemption of state-level AI regulation.
First, they argue that excessive regulation could jeopardize the United States’ position in the global AI race, particularly against China. Tech companies, such as OpenAI, warn that a fragmented regulatory landscape might slow down innovation and weaken America's competitive edge.
The Trump Administration has shown considerable deference to the viewpoints of tech companies. A January 2025 executive order aimed at ‘removing barriers to American leadership in AI,’ focused on deregulation and reducing federal oversight of AI models in order to boost industry innovation. This order reinforced the administration’s broader position: the federal government should facilitate, not hinder, rapid technological advancement to position the U.S. as the leader in this space.
A second key argument centers on the belief that the federal government should not stifle innovation. By spring 2025, 42 states had introduced their own AI-related bills, creating a patchwork of regulations that tech companies – especially start-ups – find increasingly difficult to navigate. Supporters of a federal moratorium argue that only a unified national framework can provide the consistency needed for innovation to thrive.
Finally, the Republican party traditionally favors limited government intervention and minimal regulation. Many Republicans contend that policymakers still lack the knowledge necessary to craft comprehensive AI legislation. This perspective is shared by Representative Jay Obernolte, co-chair of the House AI Task Force, who supported the moratorium on state laws. He argued that AI regulation should come from the federal level and that a temporary pause on state-level actions would allow Congress time to develop a cohesive national policy.
Opposition to the AI moratorium has emerged from both sides of the political spectrum. Democratic Senator Maria Cantwell and Republican Senator Marsha Blackburn jointly introduced the amendment that ultimately removed the moratorium from the 2025 reconciliation bill.
First, these lawmakers argue that while AI technology continues to evolve, its potential risks to consumers – particularly to vulnerable populations like children – demand immediate oversight. They believe both federal and state governments should have the authority to regulate AI. From a public safety and philosophical standpoint, they view inaction as irresponsible, regardless of how complex or rapidly advancing AI technology may be.
Second, the debate over the moratorium also centered on states’ rights. Several Republicans, including Republican Representative Marjorie Taylor Greene, opposed the moratorium on the grounds that it would infringe on states' constitutional right to govern. For these lawmakers, the issue is fundamentally about preserving the balance of power between state and federal governments.
Federal regulation of AI remains deeply fragmented, sector-specific, and agency-driven
In recent years, the government has taken a sector-specific approach to AI governance and regulation rather than taking a broad regulatory approach. Federal AI regulatory efforts have largely focused on agency-level enforcement of laws, evaluating whether additional authority for individual agencies is needed, and gaining voluntary industry cooperation.
Federal AI Activity Guide
Actions for Executives
Align your governance strategy with state-level AI legislation by prioritizing transparency, consumer protection, payer accountability, and clinical oversight: You don’t need to track every AI bill, but on a quarterly or biannual basis, review which state-level AI bills have passed and identify emerging policy trends. Focus especially on legislation aligned with your organization’s priorities. For example, if you're working on risk-tiering, examine how different states are defining and applying risk tiers. Similarly, if mental health is a priority area, monitor legislation that emphasizes the intersection of AI and mental health.
In the absence of a comprehensive national AI law, review public opinion polling on AI transparency in healthcare, as it will likely influence and reflect the priorities of legislators and regulators: Many state-level AI regulatory laws are in response to citizen complaints and concerns about safety and transparency. Unlike the tech sector, healthcare organizations tend to take a more cautious approach to innovation, with a stronger emphasis on patient-centered care and should continue to uphold this practice as federal regulation is uncertain.
Take advantage of the public comment period when a new rule or regulation is proposed: Participating in public comment periods for proposed rules or regulations can be very valuable in navigating the U.S.’s sector-specific approach to AI regulation, enabling health systems to ensure that new rules align with your operational needs and support the patients you serve.
2. Track four dominant themes in state AI laws – transparency, consumer protections, payer rules, and clinical use standards.
2025 saw a surge in state-level healthcare-specific AI legislation. Policy proposals clustered around four themes:
1. Transparency mandates that require AI developers and deployers to disclose how their systems work and the data they use;
→ In 2024, Colorado passed SB205 requiring transparency and risk controls for high-risk AI systems. Over 18 similar bills were introduced in other states in 2025, but none passed.
2. Consumer protections designed to prevent AI algorithms from unfairly discriminating against consumers;
→ In 2025, over 40 bills were introduced to address discrimination by AI tools. Colorado’s SB205 tackles algorithmic bias by requiring developers to implement risk management programs and follow transparency guidelines. Other states, including Virginia, Connecticut, Rhode Island, and Texas, introduced bills placing responsibility on AI deployers to mitigate bias and publicly disclose relevant information.
3. Payer use of AI rules that spell out when insurers may employ AI in decision-making support and necessary oversight measures;
→ In 2025, approximately 45 bills were introduced to govern payer use of AI. Some prohibited coverage decisions made solely by AI, while others required payers to disclose or report AI usage to the state. California’s SB1120, passed in 2024, prohibits AI from replacing healthcare provider judgment, requiring that a licensed provider determine medical necessity for each health plan or insurer member. However, this law conflicts with current CMS guidance on Medicare Advantage coverage determinations.
4. AI in clinical contexts standards that guide how clinicians can integrate AI into practice and how its outputs should be monitored – an area that, while generating fewer bills than others, grew significantly compared with AI bills introduced in 2024.
→ Over 20 bills were introduced in 2025 that regulated provider use of AI. Texas’ HB1265 mandates that AI used for mental health services must be approved by the Health and Human Services Commission and delivered by a licensed mental health professional. The licensed professional must be able to monitor the service’s progress, communicate directly with the patient, and intervene if the patient reports or threatens self-harm or harm to others.
Across many states, lawmakers have designated any AI-supported healthcare decision as “high-risk,” regardless of the specific context. Colorado set the pace in May 2024 with SB205, a consumer-protection law that requires developers and deployers of “high-risk” healthcare AI systems to share detailed information with one another, the public, and the state attorney general, guard against algorithmic discrimination, and conduct risk assessments of AI tools. AI systems that have been approved by federal agencies or comply with national standards are exempt. SB205 takes effect on February 1, 2026, and numerous bills introduced in 2025 model its provisions. Given the removal of the AI provision in the 2025 reconciliation bill through an amendment, it is likely that more states will follow Colorado’s template through similar legislation. Manatt Health’s Health AI Policy Tracker chronicles all state AI legislation introduced or enacted between January 1 and March 31, 2025.
Actions for Executives
Single-state health systems should focus on tracking AI-related bills and policies within their own state – unless they operate in a 'copycat' state: In this case, it is important to monitor developments in neighboring states that may influence local legislation: California and New York often lead in healthcare regulation, with Massachusetts also playing a key role at times. If your state tends to follow this 'copycat' pattern, it's important to closely monitor developments in these leading states.
Multi-state health systems should default to most stringent state-level regulation in which they operate:
Track key regulatory themes in the states where you operate that have the strictest AI regulations, while also monitoring developments in leading states, as their policies often shape broader regulatory trends nationwide.
Prioritize AI governance in high-risk service lines facing intense regulatory scrutiny - mental health, emergency medicine, and virtual care: Establish internal standards and contingency plans for AI use in those areas. Mental-health chatbots exemplify this increased focus: New York’s AB S03008C (effective November 4, 2025) bars any person or entity from offering an “AI companion” unless the model can detect and appropriately respond to suicidal ideation or self-harm, while Utah’s HB452 (effective May 7, 2025) requires chatbots to disclose that users are interacting with AI rather than a human.
3. Be proactive and build system-wide AI accountability to self-regulate – ensure disclosures, vet vendors, and monitor tool performance.
As transparency continues to be a central theme in emerging AI legislation, it’s essential to be proactive. In the absence of comprehensive regulation at the federal level, health care organizations must ensure they fully understand and can clearly explain the AI systems they develop or deploy.
The clear legislative push for increased transparency around AI use requires a system-wide approach to addressing patient safety implications of AI adoption. Health systems must prepare clear patient disclosures on AI use and strengthen data privacy and security safeguards now. This consumer-protection focus is already reflected in state-level legislative activity. Utah’s SB149 established the Artificial Intelligence Policy Act (amended in May 2025) to ensure patients are informed when AI tools are used in care and care coordination. SB149 is a consumer protection law that places restrictions on ‘regulated occupations,’ covering over 30 healthcare professions, and applies to any communication involving high-risk generative AI. High-risk AI interactions include collecting sensitive personal data (e.g. health data) and providing personalized advice that could significantly impact personal decisions (e.g., medical advice).
Actions for Executives
Be prepared to disclose AI usage and form a risk-management protocol around AI: Businesses must inform consumers when they are interacting with AI. In healthcare, clinicians should disclose AI use in high-risk interaction - particularly when sensitive information is collected or health advice is provided. The ONC’s transparency provides a concrete framework for transparency and risk management: the HTI-1 rule (finalized December 2023) updates certification criteria to cover “decision-support interventions,” while HTI-2 (finalized December 2024) does not delve deeply into AI but completes key Trusted Exchange Framework and Common Agreement (TEFCA) proposals, underscoring the need for robust disclosure and risk-management practices.
Establish a committee to centrally review AI contracts, with a focus on evaluating security measures and understanding how vendors and their AI tools will use your data: Even if it is not a formal committee, create a dedicated group to review AI procurement and contracts – serving as a centralized hub for storing agreements and incorporating checklists or input from business owners submitting the contracts.
→ For more information on vendor evaluation, view our UVM Vendor Analysis Questionnaire and insight article on Centralized vs. Decentralized Governance.
Develop and implement robust AI oversight, following our five action steps for structured, impactful governance: Key action steps include boosting organization-wide AI literacy; building on existing governance frameworks; integrating AI ethics into training and evaluations; planning ahead for worst-case AI scenarios; and staying agile to ensure governance keeps pace with rapid technological changes.
Create a platform for clinicians to report issues with AI tools, including data drift and hallucinations, and support resolution of these concerns: It is critical to provide a space where clinicians can flag errors or questionable outputs – enabling a pause for investigation and iteration before continued use.
4. Own the outcomes of AI tools – health systems are liable for AI-driven care and must reinforce human oversight.
AI systems themselves are not legally liable for their outcomes. Accountability remains complex and continues to evolve at the state level. Some states are beginning to define AI liability, placing responsibility on the deployer or director of the AI tool rather than the vendor or AI system itself. Regardless of how liability is distributed between vendors and healthcare organizations, providers ultimately own the outcomes. Even if financial liability can be contractually shifted to vendors in certain jurisdictions, the healthcare system's reputation and obligation to patients means they bear ultimate responsibility for AI-generated outcomes. Legal counsel can clarify the liability landscape within a specific state, while organizations must establish robust safeguards to monitor the technology and mitigate risks, recognizing that patient trust and safety take priority over innovation.
Actions for Executives
Ensure all AI-agent interactions are recorded and securely maintained for auditability and compliance: Recording these interactions supports effective review and technology iteration within your AI governance frameworks. If you work with a vendor that deletes recordings, you should establish alternative processes to monitor transcripts to default to human-in-the-loop for safety purposes.
Do not grant full autonomy to AI tools – ensure humans can review outcomes even when operating without real-time human-in-the-loop: Trends in state legislation highlight the growing importance of human and physician supervision of AI tools, which is essential for protecting patient safety and ensuring accountability for AI outputs. New York’s SB7543 requires all AI tools that assist in decision-making – including clinical decision support systems (CDSS) – are subject to ongoing human review. Similarly, Texas SB1188 mandates that healthcare practitioners using AI for diagnostic purposes must review all records generated by AI and disclose its use to patients.
In addition to disclosing AI use, provide patients the option to escalate interactions with a human being, in part to address liability concerns: To strengthen provider transparency, a few states introduced bills in 2025 – such as Illinois’ SB2259 and Nevada’s SB186 – requiring that patients be informed about how to contact a human healthcare provider rather than communicating solely with AI systems. Similarly, Massachusetts’ HB1210 mandates that providers give clear instructions on how to reach a healthcare professional directly in cases where AI-generated outputs have not been previously reviewed or approved by a provider.
5. Demand bias mitigation from vendors – prioritize equity and patient safety over speed.
Tech companies often favor disruptive innovation, while the healthcare sector remains far more cautious – a crucial consideration when developing AI tools or partnering with AI vendors. Additionally, historical trends in technology and AI policy reveal that the federal government tends to promote innovation through a generally deregulatory approach.
Consistent with the U.S.’s sector-specific model for AI regulation, the Office for Civil Rights (OCR)’s final rule prohibits “covered entities” from using AI in decision support tools in ways that discriminate based on race, age, sex, and other protected factors. Covered entities include organizations receiving financial assistance from the U.S. Department of Health and Human Services, such as hospitals or doctors’ offices that accept Medicare and Medicaid. Regulators are increasingly likely to require healthcare organizations to demonstrate that their AI systems do not perpetuate or worsen existing health disparities, as reflected in OCR’s current focus on preventing discriminatory use of AI in healthcare.
Actions for Executives
Set the expectations of vendors from the outset that it’s their job as much as the health system’s to identify model bias and ensure patient safety: Push vendors to disclose training data and any past instances of bias or model deterioration. Ask this of them going forward to disclose potential instances of bias at other sites or health systems. Share your governance strategy with them and ask them to provide insight on how they’re working with other health systems to address patient safety concerns. If they refuse, don’t go with them.
Choose AI vendors carefully – adopting too early can be as risky as adopting the wrong solution: Technology will always outpace policy, and deployment in healthcare carries unique complexities. Don’t be swayed by hype –prioritize a risk-averse, fast follow strategy that benefits from early adopters’ lessons learned instead of pioneering unproven results.
Cast a wide net for consensus, and closely observe which vendors are gaining traction in other health systems: Look for indicators of success, such as proven performance/success and endorsements that signal a vendor has earned trust from other health systems.
→ Stay up to date on AI success stories by subscribing to AI Catalyst’s Pulse newsletter. For more information, reach out to aicatalyst@hmacademy.com.
6. Engage lawmakers – proactively shape future policy by forging local relationships and translating technical concepts into real-world healthcare outcomes.
Legislative decisions are often made behind closed doors, well before public committee hearings take place. Whether or not your health system has a lobbyist on Capitol Hill, it’s critical for health administrators to build strong relationships with local lawmakers and elected officials to help ensure that regulations align with the best interests of patients.
Actions for Executives
Build relationships between hospital administrators and local elected officials and utilize lobbyists: Invest in personal relationships among hospital administrators and local elected officials, leverage skilled lobbyists, and prioritize long-term strategies over short-term wins. Building strong connections with local elected officials -- whether county commissioners, mayors, or others -- is especially crucial, as these relationships can often influence legislation even more effectively than lobbyists. Hospital administrators can foster these connections by attending community events and serving as visible community leaders. Engaging with the community and cultivating these relationships is an effective way to shape regulations that benefit patients, hospital systems, and employees alike.
Many lawmakers lack a foundational understanding of AI’s risks and benefits. Help close the gap by using layman’s terms and conveying how AI is being applied in healthcare: Refer to the AI 101 glossary for key AI terms and definitions. Examples of AI applications in healthcare include employing AI to produce personalized patient education materials, such as post-treatment recovery and rehabilitation plans; creating discharge summaries from ambient listening combined with EMR data; using AI-generated OR schedules and staff workflows that account for real-time clinical, workforce, and epidemiological factors; and drafting RFPs and contracts using foundational AI models, ambient listening, and historical tenderers.