Product preview · KunXi Medical LLM · Validation & compliance
Design principles: abstain when unsure, approve when risky, keep raw data where it lives
Hallucination, overreach and data leaks are what hospitals, regulators and patients worry about most. We plan to build our defences into the architecture rather than patching them on later; five planned validation stages would release capabilities one by one, and until results are in we publish thresholds, not efficacy claims.
Product previewKunXi Medical LLM is at the design and concept stage
This section describes what we plan to build and how we intend to do it — not finished products or results. Product forms, model sizes, technical approaches, features, interfaces, metrics, scenarios and timelines are design concepts that have not been built or validated; they may be adjusted or changed as development, validation and regulatory requirements evolve. Nothing here is a commitment, an offer or a claim of clinical effectiveness.
All rights reserved. No reposting, adaptation or commercial use without written permission.Read the copyright notice
Three lines of defence
- Evidence tracingEvery conclusion
cites the source note, guideline or drug label; no source, no output
- Conformal abstention2% → 0.1%
A statistically guaranteed “I’m not sure” that hands over to a person; prostate-biopsy AI test set of 3,059 cases
- Harness governanceA3 needs approval
Least privilege, sandboxing, full audit; low confidence degrades to a prompt
Five planned stages, one at a time: thresholds before results, no efficacy claims
NoticeThis module is a design concept, not yet built, and may change.
We plan to validate technical ability, workflow improvement, clinical benefit, safety and economic value separately; medical exam scores are not evidence of clinical effectiveness.
- ①Internal testing
Retrospective data; compared with general models, rule engines and people under the same conditions
- ②External validation
Another hospital, another period; patients, institutions and time kept apart
- ③Prospective silent run
Three months in the background with no clinical effect; every case compared with the human result
- ④Human–AI evaluation
“Doctor + AI” versus “doctor”: time, edits, misses
- ⑤Economic & clinical studies
Change in denials and deductions; diagnostic aids follow device clinical-trial rules
| Task | Main metrics | Compared with | Release principle |
|---|---|---|---|
| Insurance compliance review | Recall, false-alarm rate, citation accuracy, critical-error rate | Rule engine, general LLM, human reviewers | Recall and false alarms meet thresholds agreed with partner hospitals; 100% of critical errors stopped by a human step |
| Record QC and coding | Coding agreement, DRG / DIP grouping agreement, defect detection | Medical coders | Coding agreement meets the threshold; defect detection no worse than people |
| Document drafting | Factual errors, omissions, citations, doctor edits and time | Doctors writing by hand | No serious factual errors; clearly less time per document |
| Chronic-disease risk prediction | Sensitivity, specificity, calibration, net clinical benefit | Existing risk scores | Well calibrated on external validation; net benefit no worse than existing scores |
| 100-step workflow tasks | Completion, human takeover, rework | Manual workflow | Completion ≥ 90% (2028 target) |
Note: release thresholds are KunXi target values to be agreed with partner hospitals and ethics committees; net benefit means decision-curve analysis (DCA); during the silent run, AI results never enter clinical workflow.
Error grades and post-launch monitoring (design)
Design rule: S2 and above need a human, S3 is zero tolerance
NoticeThis module is a design concept, not yet built, and may change.
- S0No effect
Logged and fed into iteration
- S1Needs editing
Released after a person edits it
- S2May affect care
Must be confirmed by a person
- S3May cause harm
Zero tolerance: the feature is paused and reviewed
- Conformal abstention; low confidence goes to a person
- Subgroup reports by age, sex, institution and disease
- Monthly drift monitoring; automatic rollback to the previous version above threshold
A0–A4: “self-driving levels” for agent autonomy
NoticeThis module is a design concept, not yet built, and may change.
By design, every action has a level: the closer it gets to a clinical decision, the higher the level and the stricter the approval; A4 high-risk actions are switched off for agents.
- A0Read-only · automaticSummarise metrics, write daily reports
- A1Draft · person confirmsDraft notes, follow-up plans
- A2Workflow · traceableSchedule follow-up, QC sampling
- A3Controlled · approvalOrder suggestions, compliance alerts
- A4High risk · offStopping or switching drugs: doctors only
By design, only four things leave a domain: model parameters · de-identified statistics · evidence packs · service APIs
NoticeThis module is a design concept, not yet built, and may change.
By design, raw data and datasets stay where they are; the model learns medical regularities, and personal records never go into its parameters.
Each authorised domain trains on its own and only parameters are pooled; each new domain widens the diseases and populations covered.
Training / validation / test split by patient, institution and time; de-duplicated; no future information; evaluation sets kept apart from public question banks.
The planned first product would run on-premises under the hospital’s data-processing agreement; data would never leave the hospital.
Design principle: agents suggest, doctors decide
NoticeThis module is a design concept, not yet built, and may change.
By design, every KunXi suggestion would carry its source evidence, its action level and its confidence. Low confidence means abstaining and handing over; A3 actions such as order suggestions and compliance alerts need approval; A4 actions such as stopping or switching drugs are carried out by doctors only.
- Suggestions come with evidence; doctors accept or reject in one click
- Reasons for rejection flow back as process labels
- Every action is logged and can be audited and traced at any time
Data, service, product, security and responsibility: each to be put in place and checkable
NoticeThis module is a design concept, not yet built, and may change.
- Data stays in its domain
Designed so training and inference both happen inside the authorised operating domain
- Service filing
Generative AI services to be filed as required (1,112 services filed nationwide so far)
- Product registration
Diagnostic aids would go through NMPA device registration; documentation and QC products would launch first
- Secure and auditable
Planned: MLPS level 3, data-security assessment, ethics review, full logs
- Doctors decide
Agents only suggest; doctors make clinical decisions; the consumer side does health management only
Diagnostic-aid products are planned to apply under the NMPA Special Review Procedure for Innovative Medical Devices for priority review.
SourcesOlsson et al. (Nat Commun 2022, conformal prediction); Cyberspace Administration notice 2026-09 (1,112 filings); the “Data 20 Measures”; NDRC and National Data Administration “1+3” (2025-01); NMPA Special Review Procedure for Innovative Medical Devices
Note: KunXi Medical LLM is a product preview at the design and concept stage. Unless a public study is cited, the products, capabilities, thresholds, metrics and timelines in this section are KunXi design targets, concepts or estimates — not achieved or measured results — and may change; product screens are concept illustrations with simulated data. By design, the agents only make suggestions; clinical decisions are made by doctors.
Explore putting a medical LLM to work on real clinical tasks
KunXi Medical LLM is at the design and concept stage. Hospitals, pharma companies, insurers, pharmacies and health-management providers who are interested are welcome to talk to us and assess data, workflow and the validation path together.