Adopting AI in healthcare: Managing risk while keeping clinical judgment in focus

AI has already made its mark in the office. Now it's stepping into the clinic.

Doctors and other healthcare practitioners are increasingly using AI across a range of clinical tasks, from reducing routine workloads to supporting clinical decision-making.

But as AI tools embed more deeply into everyday practice, they can also create unforeseen sources of liability for clinicians, who must navigate AI use without compromising professional judgment, patient trust and accountability.

But as AI tools embed more deeply into everyday practice, they can also create unforeseen sources of liability for clinicians, who must navigate AI use without compromising professional judgment, patient trust and accountability.

Greater adoption of AI in the clinical setting will make understanding the technology’s risks just as important as maximising its outcomes.

Clinical AI: moving into the mainstream

Singapore is ahead of the curve where AI in healthcare is concerned. Home-grown GenAI platforms like AgentSea are presently automating routine administrative tasks, while machine learning platforms like Optim.AI are helping to identify promising treatments for diseases.

Local healthcare regulations like AIHGle 2.0 hold developers and health systems responsible for AI safety and effectiveness. Healthcare practitioners consequently bear both the benefits and costs of AI use in clinical practice.

“Understanding where responsibility sits, how AI outputs are validated and what protections are available should form part of the adoption process,” explains Karen Yeo, Howden’s Head of Financial Lines, Singapore. 

Clinical and operational risks surrounding AI use

Greater reliance on AI tools also carry significant consequences for clinical practice.

Automation bias occurs when clinicians give AI outputs greater weight over their own independent clinical judgment. For example, a radiologist deferring to a diagnostic algorithm without carrying out an appropriate independent review.

GenAI tools also remain susceptible to factual errors: misinterpreting patient consultations or even “hallucinating” incorrect information altogether. Healthcare practitioners must therefore manually review AI-generated content, creating additional busywork that cancels out AI’s promised efficiency gains.

These risks reinforce a fundamental point: AI cannot replace professional judgment. As cases like Sampson v. HeartWise Health Systems show, clinicians are legally obliged to independently scrutinise software outputs, not just act as a passive pass-through.

“AI should support clinical judgment rather than replace it,” Karen adds. “Doctors should continue to apply their own professional expertise. Having a clear process for validating AI outputs can help ensure that technology supports clinical decision-making, rather than replacing it.” 

Patient data and the AI “black box” 

Some of the risks associated with AI sit beneath the clinical application itself, in how the underlying systems operate.

One example is the opacity of some AI systems. Advanced models are functionally “black boxes”, preventing clinicians from understanding how they arrive at particular outputs. This challenges clinical norms surrounding transparency and informed consent, while adding another layer of liability risk.

Data privacy also needs careful consideration, especially if AI applications involve overseas storage or secondary use of Protected Health Information (PHI) without patients’ consent. These concerns are amplified by “shadow AI”, where staff independently use consumer AI tools for work without their organisation’s approval. The use of shadow AI can put PHI and other sensitive information at greater risk of exposure.

Practical safeguards around AI use

To minimise legal liability and adhere to standard-of-care expectations when using AI, clinical practitioners should adopt the following operational best practices.

1. Maintain active human supervision. If a clinician cannot explain how an AI output influenced a decision, that output should not be relied on without further review. The human-in-the-loop remains important in clinical AI; doctors need to retain oversight of how its outputs are interpreted and applied in patient care.

This means maintaining clear accountability for all decisions, whether informed by AI or not, regularly sense-checking AI outputs (verifying against trusted sources, or asking “Does this make clinical sense?”), and validating AI-generated information before acting on it.

2. Understand the system before using it. Clinicians should critically evaluate each AI tool’s intended clinical scope and supporting evidence. Check if its underlying datasets are appropriate for the patient population it’s intended for (like SIMFONI, trained specifically for Singaporean patients), mitigating diagnostic errors or algorithmic bias.

Healthcare organisations need to establish strict controls around shadow AI. This includes limiting AI use to platforms specifically authorised for healthcare environments; and establishing appropriate agreements with software vendors before they gain access to PHI.

3. Be transparent about how AI is used. Transparency helps patients and clinicians understand how AI fits into the care process.Where AI is used, it is important to be clear about its purpose, limitations and the level of professional oversight involved.

Disclose and document every part of your practice where AI contributes to patient care. Doing so creates a verifiable audit trail showing how professional judgment was exercised over AI recommendations; and will give patients a chance to opt out in favour of standard clinician care.

Patients must provide explicit written consent for their PHI to be processed by third parties or used for secondary AI model training. Otherwise, all personally identifiable information (PII) and protected health elements must be stripped from the data before use.

Building risk considerations into AI adoption from the start

Responsible AI adoption goes beyond listing the benefits of any given platform. A multitude of other factors must also be considered, including (but not limited to) AI’s impact on clinical workflows, data handling and the organisation's broader risk profile.

This is where risk and insurance considerations can enter into the adoption process. Insurance is one part of a broader risk-management approach to AI adoption.

AI-related incidents can give rise to different types of exposure, including professional liability and cyber liability. Understanding the potential exposures early can help doctors and healthcare organisations consider whether their existing protection remains appropriate as the way they work changes.

Howden can support healthcare professionals and organisations as they weigh the risks of AI against the protection available to them; the appropriate coverage will ultimately depend on the nature of the practice, how AI is used and the specific risks involved.

Proactively considering the clinic’s AI risks – and acting proactively to address them – can help your clinical practice embrace AI while still keeping clinical judgment, patient trust and accountability at the centre.

doctor

Take Control of Your Professional Protection

Every doctor’s risk profile is different. Your indemnity cover should reflect that. Contact us for personalised advice or a tailored medical professional indemnity quote.