How Clinical AI Earns Trust Through Narrow Use Cases and Expert Validation Starting at the DB Level

The AMA’s 2026 Physician Survey on Augmented Intelligence found that 72% of physicians already use AI in their practice, while 40% remain just as concerned as they are excited about what it could mean for clinical care. Issues such as patient privacy, safety, effectiveness, and accountability remain the biggest sources of concern.

Ahmed Mohamed is a Database Administrator at Al Moammar Information Systems Co., where he manages mission-critical SQL Server and PostgreSQL systems across cloud environments. He comes at clinical AI from the infrastructure side rather than the clinical one, with a family full of practicing physicians to pressure-test his work. He spent two months fine-tuning an open-weight model for medical reasoning, then turned to those clinicians to see whether its output held up in practice.

"They don't fully trust it because they don't want to involve something uncertain in decisions involving human life," Mohamed says. Drawing on that experience, he shared several practical lessons for healthcare teams exploring AI, from choosing the right model and data to validating its reasoning and running it within local infrastructure constraints.

Clinical AI rewards adaptation over reinvention and scale

In healthcare, the sensitivity of the domain can make a bespoke model sound necessary even when existing open-weight models already provide much of the underlying capability. "I don't think you need to start from zero and build a medical model completely from scratch, that would take a long time. You can take an existing model and fine-tune it for your specific domain. In many cases, that should be enough to achieve what you need.", Mohamed says.

Best results don’t come from a larger model, either. Extra capacity helps when a system needs to do many different things. But in medicine, once the task is narrow enough, what matters more is how well the model understands the domain it’s working in.

In his experiment, Mohamed used Qwen 2.5 14B, an older and comparatively small model, and tested it on a large publicly available dataset. "When I tested the base model with my colleagues at the beginning, it wasn't answering correctly," he notes. "But after I fine-tuned it, they were surprised. It was providing the correct medical reasoning, asking the right medical questions, and giving more appropriate answers." The improvement came from making the model more specific to the work in front of it, not moving to a larger foundation model.

Expert validation sets the boundary for medical AI

Standard AI benchmarks return a score, but don’t show if the model’s reasoning holds up against real clinical cases. Mohamed therefore loaded the model into Open WebUI and gave access to doctors in his family and professional network. "You need real medical professionals to test it," he says. He is also careful about the limits of that evaluation. Six or seven clinicians across different specialisms is not a controlled study, and he doesn’t present it as one.

The most useful thing to come out of the process was a boundary. “It doesn't necessarily give a final diagnosis,” Mohamed notes. “It helps indicate where the doctor should investigate based on the medical case.” It’s a claim clinicians were willing to accept because the final decision remains with the person accountable.

The hospital already holds the dataset it needs

"If a hospital wants to take AI seriously, it should take the cases it has encountered since it opened, create a dataset from those cases, and train a model on that data.", Mohamed notes. The cost of skipping this step won’t appear in accuracy points, but in real-world use, when a specialist stops opening the tool after the third irrelevant suggestion, and that kind of abandonment is considerably harder to reverse than a low score.

Another thing to consider is that medicine varies by geography, so general training data doesn’t suffice. "Each country has particular medical problems. For example, here in Saudi Arabia, because of the dusty environment, we see many cases related to breathing and respiratory problems." A model that performs well on a general medical exam set can still miss the presentation patterns a specific population produces, and the gap stays invisible until practitioners start running local cases through it.

One department is enough to prove the model. Mohamed has watched organizations commit to AI before identifying where it can create value, deciding they need it and planning to work out the use case along the way. He sees scope as the recurring failure point. "A hospital doesn't need to apply AI across every medical department immediately," he says. "Choose one department where you think AI could have an important impact and focus on that." 

AI governance is shaped by clinical data boundaries

Public data can be processed anywhere. A decade of a hospital's own clinical records can’t, and the reluctance to move it is close to universal. "One of the biggest issues in medicine is that organizations don't want their data going outside to ChatGPT or another provider," Mohamed notes. "They want it to stay local." 

In Saudi Arabia, this is not only a matter of trust but also of regulation. The PDPL places restrictions on cross-border transfers of personal data, NCA rules govern how regulated workloads can be hosted, and NDMO requirements add further obligations around data governance in the public sector. Retrofitting those requirements after a workload has already been designed around an API outside the Kingdom can turn what looks like a configuration change into a rebuild of the integration.

Infrastructure will determine how far local AI can go

The advantages of running AI locally come with their own constraints, starting with infrastructure. "These models still require GPUs, and there's currently limited availability of GPUs, RAM, and VRAM in the market," Mohamed says. "The first challenge is getting enough computing resources inside the organization."

Once the issue is resolved, he sees an opportunity to bring together medical records and cases from across hospitals and use that data to support AI systems aimed at problems specific to Saudi Arabia, especially since much of its healthcare system is already digitized. "In the future, I think AI will help researchers and medical professionals investigate problems that haven't yet been fully understood and ultimately help improve and extend human life," Mohamed concludes.