Case study · Healthcare AI
An AI tutor that refuses to make things up
Medical schools cannot use a tutor that invents a citation. A student who learns a fabricated guideline carries it onto a ward. That constraint, not the chat interface, was the whole engineering problem.
The problem
Mega Summit (U) Ltd, a Uganda-registered education company, was building an AI tutor for medical, nursing and allied-health students. Two features carried the risk. Students would interview an AI patient and be graded on their diagnosis. Students would upload medical PDFs and ask questions of them.
Both break in the same way. A language model asked a question it cannot answer from the evidence in front of it will produce a confident, plausible, wrong answer, complete with a citation to a page that does not say what it claims. In medical education that is not a bug to log. It is the reason the product cannot ship.
What we built
A virtual patient that stays in character
Students interview an AI patient that never reveals its own diagnosis, commit to an answer, and are graded by a separate agent against a versioned rubric with clinical guidelines cited. Because both the rubric and the system prompt are versioned, any grade can be reproduced months later and defended if a student challenges it.
A research assistant that cites or refuses
Uploaded PDFs are extracted with page mapping, chunked semantically, embedded at 1,536 dimensions and indexed with HNSW in pgvector. Answers are generated only from retrieved passages, and this is the part that matters: every citation is validated against the passages actually retrieved. A fabricated citation causes the answer to be rejected rather than shown. When retrieval is weak, the system returns "insufficient evidence in the available sources" instead of guessing.
Most retrieval systems will produce a confident answer from thin evidence. This one is built to decline. Refusing well is a feature you have to engineer deliberately; it does not come free with a vector database.
What it took
| Backend | Python 3.14, FastAPI, async SQLAlchemy 2.0, Alembic |
|---|---|
| Data | PostgreSQL 18.4 on Amazon RDS, pgvector with HNSW indexing |
| Web | React 19 and TypeScript across three builds: marketing site, faculty portal, admin console |
| Mobile | React Native and Expo, shipping to TestFlight and Google Play |
| Delivery | GitHub Actions to both servers, with a production health gate that blocks a broken release |
Multi-tenant from the first milestone, serving universities buying licence seats and individual students at the same time. Tenant isolation is enforced at the query layer and tested in continuous integration in both directions, proving both that permitted access works and that forbidden access fails.
Proving it would hold
A 300-seat college was pending approval and the client needed to know whether the existing server would carry it. We load tested the live system rather than a staging copy.
| Scenario | Result |
|---|---|
| 200 concurrent users | Zero failed actions, about 0.3 second typical response, 65% server effort |
| A realistic day: 25 faculty, 150 students, 10 live AI interviews | 38,500 actions, zero failures |
| Stress to 1,300 concurrent, six times the target | 99.45% success, about one second per action, no crash |
The answer was that the new college needed no upgrade at all. A delivery report should answer that question before the client has to ask it.
Delivery
First commits at the end of June 2026. Milestone one delivered ahead of the month-one schedule. Everything runs in the client's own cloud, model-provider and code accounts, so there is no lock-in to us. If they replaced us tomorrow, nothing would stop.
Have a model that needs to behave?
We build AI systems that are checkable: grounded answers, reproducible output, safety rules in code, and tests that gate every deploy. Tell us what is breaking and we will tell you whether we can fix it.
Start a conversationOr email info@codezbit.io