Imaging Reading & Reference-Standard Services
Reads your trial can stand on. Ground truth your model can be judged against.
Blinded reads for imaging CROs; double-read, adjudicated reference standards for medical AI. In LIDC-IDRI, four expert thoracic radiologists agreed on just 34.8% of lesions ≥3 mm — which is exactly why we fix the rules before anyone reads, then measure and document what the readers actually did.
Is reader capacity holding up your study?
Can you evidence how your ground truth was established?
For imaging CROs & core labs
Subcontracted specialist reading
We work under your charter, in your quality system, to your SOPs — the standard subcontracted-reader model. You keep the sponsor relationship, the platform and the record; we supply reading capacity and sub-specialty depth.
- Independent reads, second reads and adjudication
- Sub-specialty cover during peak demand
- Reader CVs and credentials submitted for your approval before any read
- Qualification pack: professional licences and CVs, current ICH-GCP training where applicable, study-specific training and delegation records, COI declarations, NDA/DPA
For imaging AI & medtech teams
Reference standards for validation sets
An independent layer over the data your performance claim is measured against: how truth was defined, who defined it, what they disagreed about, and how that disagreement was resolved — recorded so it can be re-examined later.
- Dual independent blinded reads, blinded to model output
- Pre-specified adjudication with a named adjudicator
- Inter-reader agreement reported with the disagreement record
- Provenance chain from image to locked label
- Modalities
- CTMRIX-rayUltrasound We do not offer histopathology, dermatology or ophthalmology reading.
- Reading tasks
- Independent blinded readsSecond readsAdjudicationLesion measurementAnnotation and segmentation
- Reading standard
- We read to the imaging charter your study specifies. Response criteria, grading scheme and case definition are yours; we train readers to them and evidence the training in the qualification record.
- Platform
- Your platform and your tenancy by default, so the record, the access log and the retention policy stay under your control.
- Engagement models
- Subcontracted reading under your charterReference standards for a validation setFixed-scope qualification pilot
What you receive
Deliverables, defined upfront
Scope, acceptance criteria and the adjudication rule are agreed and written down before the first case is read. Nothing below is decided after the fact.
A locked reference dataset
For reader-derived labels: two or more independent blinded reads resolved through a pre-specified rule, then version-locked, with reader identity, qualification and timestamp attached to every annotation. Where the claim calls for an external or composite reference standard instead, we specify and build that.
Agreement evidence
Agreement metrics chosen to match the endpoint — weighted kappa for ordinal ratings, ICC or Bland–Altman for continuous measurements, task-appropriate metrics for localisation and segmentation — reported with confidence intervals, the raw disagreement tables, the rationale for each adjudicated case, and the exclusion and indeterminate rates.
The process file
Reading charter, reader qualification records, grading scheme and version, blinding state, software and tool versions, correction history, and the dataset-lock record.
The TruthRule™ method
Seven steps. No improvisation.
A performance figure is a statement about a model and about the rule used to decide what was correct. TruthRule™ is how we build that second half: seven steps, each one fixed before the next begins, so that a year later anyone can see exactly how a label came to be.
-
STEP 1
Fix the claim type first
A measurement claim, a standalone detection claim and a reader-study claim need different comparators. Applying one recipe to all of them is the most common and most expensive technical error, and the claim type is fixed before anything else.
-
STEP 2
Approve and version-lock the protocol
Protocol, reader qualifications, case definition, grading scheme, blinding and the primary endpoint are approved, dated and version-locked before any reading begins. Fixing them in advance — rather than settling them once the numbers are visible — is itself part of what makes the result defensible.
-
STEP 3
Two or more independent blinded reads
Readers are blinded to each other and to any model output. Each read is captured with reader ID, credential, blinding state and timestamp.
-
STEP 4
Measure the agreement, then report it
Inter-reader agreement is computed and reported, not summarised away. Low agreement is a finding, not a failure: it quantifies how ambiguous the task genuinely is, which is something the buyer needs to know rather than discover later.
-
STEP 5
Adjudicate by a rule written in advance
Disagreements are resolved by a pre-specified rule — a named senior adjudicator or a consensus panel, agreed per engagement. Every disagreement and its resolution is logged. Hard cases are not quietly majority-voted away.
-
STEP 6
Keep the indeterminates
Cases readers cannot confidently call are labelled indeterminate rather than force-decided, under a rule stated up front. The statistical plan fixes one primary treatment of them in advance, reports their rate separately, and carries a sensitivity analysis — with a full accounting of uninterpretable results.
-
STEP 7
Lock, then hand over the provenance
The adjudicated reference set is version-locked and delivered with a provenance record that links each label back to its source identifiers, acquisition parameters, readers, adjudication trail, QC status, tool versions and lock timestamp.
The objection we lead with
High agreement is not proof of truth
Agreement statistics measure how often readers landed on the same answer, corrected for chance. They say nothing about whether the answer was right — three readers using one loose definition will agree beautifully and be wrong together. Kappa is also sensitive to prevalence and to how the readings are distributed, so a high value can reflect an easy case mix rather than a sound definition. Any agreement statistic quoted without the definition it was measured against is decoration.
The public data makes the point better than we can. Same scans, different truth rule, different dataset:
| Truth rule | What it does |
|---|---|
| LIDC-IDRI, 4-of-4 | Only 34.8% of marked lesions ≥3 mm survive requiring all four readers to agree |
| LUNA16, 3-of-4 | Accepts majority agreement and sets aside the rest — a different reference standard from the same images |
So a published sensitivity figure is a statement about a model and a statement about a truth rule — and only one of those is usually disclosed. Wherever the claim allows it, we anchor the reference standard outside the readers: histopathology, defined-interval follow-up, or a documented multidisciplinary team outcome. Where no external anchor exists, we say so in writing instead of substituting agreement for truth.
Two things make that checkable rather than rhetorical. A seeded-truth arrangement can be written into the scope of work — cases whose external truth you already know, run as a separate reader-qualification and ongoing-QC stream kept clear of the locked endpoint dataset, so that checking us never contaminates the analysis it is meant to protect. And the delivery includes the disagreement record itself — every split, how it was resolved, by whom, versioned. If our definition were loose, that file is where you would catch us.
Readers
Who reads your images
The division of labour
We supply the methodology — reference-standard design, adjudication rules, provenance, and the statistical framing. That part is domain-general and reusable.
The domain authority comes from a named, credentialed expert matched to the specific claim, the reading task and the intended-use population, who reviews and signs the protocol and adjudication for that engagement. The expert is identified and agreed before anything is signed.
If we cannot name and assign a matching expert, we do not take the engagement. That is a hard rule, not a preference.
Who reads
Licensed specialist radiologists holding senior hospital appointments — attending and chief-physician grade — working across CT, MRI, X-ray and ultrasound, and reading to the charter written for your study rather than to habit.
Every reader's identity, qualification, sub-specialty and years of experience are recorded and attached to each annotation. Full CVs, registration details and the location each reader will work from are disclosed for sponsor or CRO approval before any read, with your right of veto over any named individual. Adjudicator qualification is recorded separately. Our readers contribute to research, trial and validation records — not to clinical reports of care, and not to any diagnostic decision about a patient under treatment.
Our reading scope covers CT, MRI, X-ray and ultrasound. We do not offer histopathology, dermatology or ophthalmology reading; where a claim requires those specialties we contribute methodology only.
Evidence
Evidence you can inspect
We publish the reasoning behind our work. The documents below are available on request and set out exactly how we specify, adjudicate and lock a reference standard.
From the worked example — pulmonary nodule reference standard (public data)
Four commitments that go in the contract
These are the terms serious buyers ask for. We publish them rather than negotiate them one at a time.
- Your data is never used to build anything of ours
- Client images and labels are never used to train, tune or evaluate our own software, and labels are never reused across clients. Retention and destruction run to your schedule, written into the agreement.
- Named readers, disclosed and vetoable
- CVs, registration details and working location for every reader are provided for your approval before any read — with a right of veto over any named individual, and conflict-of-interest declarations in the qualification pack.
- Project-level access isolation
- Named personnel only, least privilege, no shared access between engagements. Multi-factor access from managed devices, no local download, and a logged audit trail of who opened what and when.
- Scope, acceptance criteria and turnaround fixed before reading starts
- Capacity and turnaround are agreed against a defined case count and written into the scope of work — not quoted as a headline number and revised once the study is underway.
Data handling
How we handle your data
For protocol design, audit and adjudication-rule work, no patient data needs to leave your environment at all. Where reading is performed, the strong preference is to work inside your platform and your tenancy, so that the record, the access log and the retention policy remain yours.
Server location alone does not settle the transfer question. Under UK GDPR what matters is who accesses the data and from where, and pseudonymised images remain personal data. Reader location is therefore a governed fact, agreed with you in advance and recorded in the engagement documentation.
Controller and processor roles, named readers and the locations they work from, any transfer mechanism and risk assessment those locations require, sub-processor authorisation, multi-factor access from managed devices, a no-local-download rule, DICOM header and burned-in identifier checks, and incident response — all agreed in writing before a study opens, and available for inspection at any point during it.
Corrections to any record are made as new attributed entries rather than overwrites, so the history of a label remains reconstructible.
Next step
Put your images in front of our readers
Tell us the modality, the claim and the volume, and we will come back with scope, a reading plan and timelines. Prefer to start with documents? Ask for the pulmonary-nodule reference-standard pack, or a full supplier qualification pack — reader CVs and licences, ICH-GCP training records where applicable, COI declarations, NDA and DPA drafts.
Request a reading planor write to enquiries@medfrontiersuppliers.com
Please do not send patient data, trial images or identifiable clinical information by email or through this site. Any exchange of clinical data follows a signed agreement and an agreed transfer route.