Harrison.Rad 1.5 Technical Report

Suneeta Mall, Vladimir Nekrasov, Ashnil Kumar, Aiden Nibali, Mateo Diaz Shine, Alix Bird, Sajith Karunasena, Jarrel Seah

Harrison.ai

This technical report is released for transparency and capability preview; selected sections are intended for submission to a peer-reviewed venue.

This report is released for research purposes only. Harrison.rad.1.5 has not been cleared for clinical use and is not a medical device. Use must be in accordance with the Terms and Conditions.

Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment alone. The most direct opportunity is reducing the time and effort radiologists spend producing reports, a task that requires interpreting images, integrating clinical history and prior studies, and drafting structured findings. We present Harrison.Rad 1.5 (HR1.5), a radiology-specific multimodal large language model that accepts interleaved text and visual inputs and generates structured and unstructured text across plain-film radiology, spanning computed radiography, chest, musculoskeletal, abdominal, spine, and pelvic x-rays, and mammography. HR1.5 is trained through a three-stage pipeline: domain adaptation of a base language model on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives on ~6 million image-report instances, and visual-question-answering fine-tuning on multi-turn conversations. We evaluate it with a Findings-Diagnosis scoring framework that extends RadGraph-XL entity extraction with ontology-based synonym matching and polarity-contradiction detection, benchmarked on RadBench, a simulated FRCR 2B Short Case examination scored against Angoff-method thresholds, ReXGradient, and internal multi-modality datasets. HR1.5 is the only system evaluated to meet the simulated FRCR passing standard and achieves the highest accuracy on closed-format clinical questions, across anatomical regions, on internal multi-body-part and mammography reporting, and on the primary clinically-aligned score for public chest reporting. We further examine explainability and model behaviour, including question-sensitive Grad-CAM heatmaps, attention analysis, and confidence estimation, to support responsible future evaluation toward clinical use, and a framework for clinically grounded assessment of report quality.

Radiology has become the diagnostic backbone of modern medicine, and that success is now its central operational problem: imaging demand is rising faster than the workforce that reads it can grow. In the United States, imaging utilisation is projected to increase by up to 25% over the coming decades, while radiologist supply is projected to remain constrained relative to demand (Neiman Health Policy Institute, 2025). In the United Kingdom, the consequences of this imbalance are already evident: the Royal College of Radiologists reports a 32% shortfall in consultant radiologists relative to need, a deficit of nearly 2,300 radiologists (Royal College of Radiologists, 2025). In 2024, imaging demand grew by 8%, while the radiologist workforce grew by only 4.7% (Royal College of Radiologists, 2025). If these trends continue, the workforce shortfall is projected to reach 40% by 2030 (Royal College of Radiologists, 2025).

Rising imaging volumes relative to workforce growth manifests as reporting backlogs, longer turnaround times, and reader fatigue that degrades diagnostic accuracy (Krupinski et al., 2010). The radiology industry is facing a scaling problem that training and recruitment cannot solve.

Modern radiology AI solutions have already demonstrated substantial clinical value through task specific models that detect important findings, prioritise urgent studies, and assist with time-consuming measurements. These systems have no doubt improved patient care: accurate prioritisation of truly urgent cases serves a valuable function in an environment of backlogged reporting lists, however, it does not solve the underlying problem of the radiologist workforce not having capacity to read all of the scans being performed.

The greatest opportunity to address radiology’s scaling challenge lies in reducing the effort and time required for radiologists to report studies, and this is a complex, multifaceted task that involves not only attention to detail in image interpretation, but also the synthesis of imaging findings with other clinical information, observation of the patient over time by comparing to prior examinations, and ultimately the integration of all of this context to consider appropriate differential diagnoses to answer a clinical question and guide the patient’s treatment.

Harrison.ai’s experience reflects this evolution; our current generation of AI models are trained to identify a predefined set of findings on images, and whilst they can do this with very high accuracy, they only assist the radiologist in one of the aforementioned cognitive processes required to produce a radiology report. Augmenting the radiologist on all aspects of their workflow requires an AI model that is able to integrate information the way a radiologist does by design.

Harrison.ai’s next phase involves building an AI model to help solve the global radiology capacity problem from the ground up, a  specialised radiology foundation model that can perform accurate detection, reason through clinical history and prior imaging, and draft a well-structured radiology report. To support the responsible development and eventual clinical translation of such systems, it is desirable that such a system also provides calibrated confidence estimates and interpretable evidence for its conclusions. We develop and evaluate each of these capabilities in the sections that follow.

These requirements have become attainable through a rapid evolution in multimodal deep learning. The Transformer (Vaswani et al., 2017) and its visual counterpart, the Vision Transformer (Dosovitskiy et al., 2021), gave images and text a common modelling substrate; large-scale contrastive image-text pre-training (CLIP, Radford et al., 2021; SigLIP 2, Tschannen et al., 2025) showed that aligning the two yields transferable, open-vocabulary recognition. A subsequent line of work connected pre-trained vision encoders to large language models such as Flamingo (Alayrac et al., 2022), BLIP-2 (Li et al., 2023a), and the visual instruction tuning of LLaVA (Liu et al., 2023), turning static classifiers into instruction-following systems that reason over interleaved images and text, a capability brought to general audiences by GPT-4 with vision (OpenAI, 2023). A vision encoder aligned to a generative language model, and instruction-tuned on multimodal dialogue is what makes a generalist, natively integrable radiology assistant feasible.

The same framework has been adapted to radiology. Contrastive alignment alone reached expert-level zero-shot detection from unannotated reports (CheXzero; Tiu et al., 2022), and biomedical-scale pre-training improved transfer (BiomedCLIP; Zhang et al., 2023). Instruction-tuned biomedical assistants followed (LLaVA-Med; Li et al., 2023b), alongside increasingly generalist radiology systems: RadFM extended generative pre-training to web-scale medical data (Wu et al., 2023); ELIXR grafted an X-ray encoder onto a fixed LLM for broad chest-X-ray tasks (Xu et al., 2023); CheXagent built an instruction-tuned foundation model and benchmark for chest-X-ray interpretation (Chen et al., 2024); and MAIRA-1/MAIRA-2 advanced grounded chest-X-ray report generation (Hyland et al., 2023; Bannur et al., 2024). General-purpose medical multimodal models such as Med-Gemini (Saab et al., 2024; Yang et al., 2024) and the openly released MedGemma (Sellergren et al., 2025) demonstrated strong cross-domain capability, and fully open foundation models for chest radiography have since appeared (Wang et al., 2025). Figure 1.1 situates these systems by year of release, modality, and access level. Yet most of these efforts remain confined to a single modality (overwhelmingly the chest radiograph), do not span the full range of plain-film studies encountered in practice, and are rarely evaluated against the standardised examinations used to certify human radiologists.

This evaluation gap exposes a deeper limitation: open and general-purpose models tend to fail on genuinely complex cases because they lack the dense domain knowledge radiological reasoning requires. The point is concrete: on a simulation of the Fellowship of the Royal College of Radiologists (FRCR) 2B Rapids examination, which only 40-59% of human candidates pass on first attempt, Harrison.Rad 1 scored 85.67% (51.4/60), while every other model evaluated scored below 50% (Harrison.ai, 2024). This advantage has since been borne out independently: in a peer-reviewed multi-reader study in the American Journal of Roentgenology involving radiologists at Stanford, Mass General Brigham, and Seoul National University Hospital, Harrison.Rad 1 was the preferred model for every reader, selected between 39.2% and 66.5% of the time, far ahead of the least-favoured model, Gemini 2.5 (5.7–9.4%). It also recorded the lowest hallucination rate of any model evaluated, at 5.7%, compared with 10.8% for Gemini 2.5, 19.3% for MAIRA-2, and 53.8% for MedRAX (Hong et al., 2026). In the 2025 Healthcare AI Challenge., an independent evaluation conducted by the Mass General Brigham AI Arena and the American College of Radiology comprising 117 cases, 113 participants, and 2,840 evaluations, radiology reports drafted by Harrison.Rad 1 were rated clinically acceptable in 65.4% of evaluations, compared with 79.6% for radiologist-authored reports, indicating that the model approaches but does not yet match expert reporting quality. Capability of this kind is earned from large-scale, clinically curated imaging and reporting, not general web data. At the same time, we believe the competitive landscape should remain open enough for the radiology foundation-model ecosystem to evolve collectively, through shared benchmarks, comparable protocols, and accessible models, within the bounds of patient privacy and intellectual property. It is in this spirit that Harrison.ai makes Harrison.Rad 1 and Harrison.Rad 1.5 available for research and capability evaluation, allowing the community to probe, compare, and build on them. Currently, however, neither model is cleared or approved for clinical or diagnostic use in any jurisdiction, and must not be used to inform or drive patient care.

We present Harrison.Rad 1.5 (HR1.5), an incremental update to our radiology-specific multimodal large language model Harrison.Rad 1 (HR1) that refines its training and evaluation while retaining the same overall design. Like its predecessor, HR1.5 accepts interleaved text and visual inputs and generates structured and unstructured text outputs across the full breadth of computed radiography, diagnostic x-rays for chest, musculoskeletal, abdominal, spine, and pelvic regions, and mammography. HR1.5 follows a three-stage training pipeline: domain adaptation of a base LLM on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives on approximately 6 million image-report instances, and visual-question-answering fine-tuning on crafted multi-turn clinical conversations. We evaluate it with a Findings-Diagnosis scoring framework that extends RadGraph-XL entity extraction (Jain et al., 2021; Delbrouck et al., 2024) with ontology-based synonym matching and polarity-contradiction detection, benchmarked on RadBench, a simulated FRCR 2B Short Case examination scored against Angoff-method thresholds, ReXGradient (Zhang et al., 2025), and internal multi-modality datasets. We further examine model behaviour and explainability to support responsible future evaluation toward clinical use, by adapting established techniques: question-sensitive Grad-CAM heatmaps, attention analysis, and confidence estimation.

This section describes how HR1.5 is built and deployed: the training methodology that produces the model (§1.1), the data engineering that supplies its learning signal (§1.2), and the serving stack that delivers it at inference (§1.3). The emphasis throughout is on the principles behind each choice rather than on implementation specifics.

1.1 Training Methodology

HR1.5 is trained through a multi-stage pipeline, an approach that has become standard for large vision–language foundation models. The now conventional strategy first establishes competence within each modality and then aligns them; a language model is adapted to the target domain, a vision encoder is aligned to text through large-scale contrastive learning in the manner of CLIP (Radford et al., 2021) and SigLIP (Tschannen et al., 2025), and the two are finally connected and tuned to follow instructions over interleaved image and text inputs. This approach, pioneered by Flamingo, introduced gated cross-attention to fuse a frozen vision encoder with a language model over interleaved sequences (Alayrac et al., 2022); was made more efficient by BLIP-2’s lightweight querying bridge between frozen image encoders and LLMs (Li et al., 2023a), and popularised as visual instruction tuning by LLaVA (Liu et al., 2023). This method has been scaled by recent open multimodal systems such as Qwen2-VL (Wang et al., 2024) and PaliGemma (Beyer et al., 2024). Staging the training in this way optimises one objective at a time before the components are trained together, e.g. the language model is first adapted to the target domain, the vision encoder is then aligned to text through contrastive learning, and only once these are in place are the two wired for harder joint objective and tuning to follow instructions over interleaved image and text inputs. Decoupling the stages in this way improves both training stability and sample efficiency.

This paradigm transfers naturally to radiology, and several radiology foundation models adopt a comparable multi-stage formulation; CheXagent (Chen et al., 2024) and CheXOne (Zhang et al., 2026), for example, both report a multi-stage training pipeline for chest-X-ray interpretation. HR1.5 follows the same broad philosophy; several stages of pre-training with and without contrastive objectives, precede one or more fine-tuning stages that confer instruction-style, image-grounded question-answering ability. Throughout, and in keeping with the overarching goal of a clinically capable radiology assistant, the learning objectives are chosen to be predominantly radiology-relevant, so that model capacity is spent on the distinctions that matter for image interpretation and reporting rather than on general-purpose breadth.

Figure 1.2 summarises this pipeline at a conceptual level: the progression from a domain-adapted language model, through contrastive vision–language alignment, to instruction fine-tuning for visual question answering, with the dominant input modality indicated at each stage.

One design choice worth highlighting relates to image captioning, a well-established auxiliary objective in vision–language model training that has demonstrated clear value across general-domain systems. In our setting, however, ablations found a dedicated captioning task to be ineffective for radiology foundation-model training, neither improving nor degrading downstream performance. HR1.5 therefore omits a separate captioning objective and proceeds directly from contrastive alignment to instruction fine-tuning.

1.1.1 Scaling Training to B200-Class Hardware

Whereas HR1 was trained on a large fleet of NVIDIA A100 accelerators, HR1.5 was trained at scale on NVIDIA B200-class hardware. The newer accelerators offer a large step-up in compute and memory bandwidth, which moved the bottleneck off compute and onto the surrounding data and memory machinery; realising their throughput meant re-architecting that machinery rather than simply rescaling it.

Two changes mattered most. We re-architected data loading so that each training rank reads only the shard it needs, rather than every worker materialising the full dataset, removing the redundant I/O that otherwise starved the system. And we restructured on-disk storage with lazy, per-rank caching so that host memory ceased to be the binding constraint. Together, these kept the B200s consistently fed.

1.3 Serving

Because HR1.5 and its predecessor HR1 use a custom architecture, we built a bespoke serving integration on top of vLLM (Kwon et al., 2023), the high-throughput, low-latency inference engine for large language models.

At inference, we apply methods that curtail hallucination and keep answers grounded in the clinically meaningful content of the input. Our approach is inspired by contrastive decoding (Li et al., 2023; Leng et al., 2024), which reduces bias and hallucination by contrasting output distributions, but is developed internally and adapted to our objective of clinically faithful generation. Together with the served model, this constitutes the HR1.5 Core API.

We extend the Core API through an agentic framework for higher-level tasks such as report generation and chat-completion/question-answer style interaction, using an architecture analogous to a system of experts. Within this workflow the constituent models are consulted according to the clinical task at hand, other harrison.ai models may contribute where appropriate, however HR1.5 is always consulted. This agentic configuration represents the HR1.5+ system reported in the evaluations that follow.

We evaluate HR1.5 across a comprehensive suite of benchmarks spanning open-ended and closed-format visual question answering, structured report generation, and clinical examination performance. Evaluations are reported across all models compared (HR1.5, HR1, CheXagent (Chen et al., 2024), CheXOne (Zhang et al., 2026), MedGemma (Sellergren et al., 2025), GPT-5.4 (OpenAI, 2026), Gemini-3-Flash-Preview (Google, 2025), Claude Opus 4.7 (Anthropic, 2026)) wherever applicable, with results broken down by body part and question type. All models were run with a single, fixed inference configuration held constant across datasets, body parts and question types. We additionally evaluated several other open-source models, but on manual review deemed them unsuitable for the clinical tasks assessed here; these models were therefore removed from the comprehensive review.

The open-weight baselines were served behind OpenAI-compatible endpoints (vLLM, except CheXagent which runs locally via HuggingFace Transformers) and queried greedily (temperature 0): HR1 and HR1.5 used a 300-token generation cap with top_k=1, a frequency penalty of 1.1, and JSON-schema-constrained (guided) decoding for closed questions; MedGemma (4B/27B) used a 500-token cap; CheXOne used a 300-token cap in instruct mode and 1024 tokens in reasoning mode, both with a repetition penalty of 1.1; and CheXagent generated up to 300 new tokens in bfloat16 using its shipped generation_config. For the proprietary API models, decoding temperature was set to 0 wherever the model accepts it (GPT-5.4 and Gemini-3-Flash-Preview) and left at the provider default otherwise (Claude Opus 4.7, which does not respect a temperature parameter). Generation was capped at 512 output tokens for GPT-5.4 and Gemini-3-Flash-Preview and at 100 tokens for Claude Opus 4.7; GPT-5.4 was queried through the OpenAI Responses API with reasoning effort “none”, and Gemini-3-Flash-Preview through its OpenAI-compatible endpoint with reasoning effort “medium”. All remaining sampling parameters were left at their provider or server defaults.

2.1 Evaluation

Our evaluation is designed to mirror both how radiologists are certified and how they work day-to-day. At its core, is a mock simulation of the Royal College of Radiologists (RCR) FRCR examination; the assessment used to certify practising radiologists. This simulation reconstructs the exam in both its legacy Rapid Reporting and current Short Case forms (Section 2.2.1). The legacy FRCR Rapids component (retired in 2025) presented 30 plain radiographs per exam sheet, roughly half of which were normal, to be reported within 35 minutes. For each film the candidate recorded only “normal” or the key diagnosis, rewarding fast, accurate triage and penalising over-calling. With a pass mark of 54 of 60, a score of 2 per question was rewarded if diagnosis was identified correctly, otherwise 0. The current FRCR 2B Short Cases format (the new standard since 2025) replaced Rapid Reporting, here we present 25 abnormal cases drawn mainly from chest (50-60%), musculoskeletal (40-50%) and abdominal (up to 4%) radiographs, each case scored out of up to five marks (total score 125). Around this we measure open- and closed-format question answering on plain-film X-ray interpretation tasks, so that both free-text reporting and discrete clinical decisions are captured.

The benchmarks span public and internal (held out set) sources across a range of body parts and question types: clinician-authored cases (RadBench), the FRCR examination simulations, the large-scale public chest-X-ray dataset RexGradient (Zhang et al., 2025), an internal held-out clinical question set built for broad body-part coverage over a mixed set of medium-to-hard clinical tasks (RadCoverage-VQA), and mammography (CBIS-DDSM). Together these probe HR1.5 from standardised examination performance through to everyday reporting across the breadth of plain-film radiology. The scoring methodology common to these evaluations is described next.

Evaluation Scoring: Findings-Diagnosis Methodology

Our primary evaluation scoring extends RadGraph-XL entity extraction (Jain et al., 2021; Delbrouck et al., 2024) into a Findings-Diagnosis framework used across RadCoverage-VQA, RexGradient, and mammography benchmarks. RadGraph-XL extracts clinical entities (anatomy, observations with polarity labels) and their relations from both predicted and ground-truth text. The Findings-Diagnosis score then applies:

• Normal/abnormal classification: Pattern-based text analysis with negation-aware phrase matching determines whether the prediction indicates a normal or abnormal study; when text matching is ambiguous, RadGraph entity analysis checks for the presence of pathological observations as a fallback
• Diagnosis overlap scoring: RadGraph-XL extracts clinical entities from both texts; observation tokens are filtered to remove generic descriptors (severity, laterality modifiers) and meta-terms (e.g., “finding”, “lesion”), then lemmatised and matched using fuzzy token matching with ontology-based synonym expansion from a radiology knowledge graph
• Polarity contradiction detection: Cases where the prediction asserts the opposite polarity to the ground truth (e.g., definitely denying a finding that is definitely present, or vice versa) receive zero credit; this is checked bidirectionally across both prediction and reference entity sets

This scoring is applied consistently across open-ended and closed-format evaluations. For closed-format (multiple-choice) questions, an LLM-as-a-judge approach (Gu et al., 2024) validates answer correctness when exact matching is insufficient.

Comparing every metric on the same studies, scored across models, makes the deficiencies of conventional metrics concrete. These metrics track how a report is worded far more than whether it is clinically correct. Lexical-overlap scores (BLEU, ROUGE-L) reward n-gram agreement with the reference phrasing, so they collapse toward zero for a factually excellent report worded differently and can rank a fluent, correct paraphrase below a clumsy near-copy of the reference. Embedding- and entity-based scores (BERTScore, CheXbert-cos, RadGraph) reward semantic or entity similarity but are largely insensitive to clinical polarity and salience, so they remain high even when a report hallucinates a clinically significant finding or misses one. Composite error metrics such as RadCliQ inherit the same blind spots from their components. In short, none of these metrics reliably separates a clinically correct report from a clinically wrong one, which is why we do not treat them as indicative of clinical quality.

The Findings-Diagnosis score is introduced to close this gap: by comparing extracted findings and penalising polarity contradictions, it separates clinically correct from clinically incorrect reports far more faithfully than any surface metric. It is not, however, infallible; its normal/abnormal decision and finding overlap are computed automatically, it can still credit a report that misses a subtle finding or accept a vague but incorrect descriptor as a match. We therefore adopt the following reporting policy for the open-ended evaluations that follow: the Findings-Diagnosis score is the primary measure, and where its automated finding-matching is insufficient to adjudicate clinical correctness, most notably on the free-text report generation of RexGradient, we additionally report a Judge Findings-Diagnosis score, an LLM-as-a-judge (Gu et al., 2024) adjudication of the same findings-versus-diagnosis comparison. Conventional report-generation metrics are still tabulated for consistency with the prior literature, but should be read only as context alongside these clinically aligned scores.

To make this concrete, the cases in Table 1 place the conventional metrics beside a clinician reference verdict (GT acceptable – whether a radiologist would accept the report) and the Findings-Diagnosis (F-D) verdict with its judge-based adjudication (Judge F-D), so the automated verdicts can be read directly against the clinical reference. The true (ground-truth) answer for each is given in the first column and the predicted answer verbatim in the second; ✓/✗ indicate whether the prediction is clinically acceptable.

2.2 Results

2.2.1 FRCR Evaluations – Mock examination

The FRCR examinations are not publicly available. We created a simulation of the FRCR examination, using external data that was not used for model training. In building the examination, we made our best effort to follow the published examination guidelines.

All results in this section are reported as sheets passed (for FRCR-style exams) or per-benchmark aggregate metrics. We report sheets passed to provide a clinically meaningful measure: a model either meets the standard for a given exam sheet or it does not. However, we also report median and interquartile range (IQR; 25th–75th percentiles) of the scores to provide a balance review across series of mock examination sheets.

2.2.1.1 Legacy FRCR Rapids (Retired since 2025)

The legacy FRCR 2B Rapid Reporting format consists of a mix of normal and abnormal plain-film X-ray cases. The RCR retired this format in 2025, having replaced it with the Short Case examination (Section 2.2.1.2); we retain it here as an additional point of comparison. We evaluate on 70 held-out exam sheets. Each case is assessed in two sequential parts: first, whether the study is normal or abnormal; and second, where the study is abnormal, the findings that are present. A case is scored correct only when both parts are answered correctly: the normal/abnormal call must be right and, if the case is abnormal, the findings must also be correctly identified. A fully correct case scores 2 and any other response scores 0, giving a maximum of 60 per sheet. Following the FRCR Rapids guideline, a sheet is counted as passed at a score of 54 or above. Results are reported as the number of exam sheets passed.

2.2.2 Closed Question Datasets

To complement publicly available benchmarks, we curated closed evaluation datasets targeting specific pathology categories and clinical scenarios. These datasets were generated using a structured pipeline: an LLM produces clinically plausible multiple-choice questions and distractors from ground-truth case data, and generated questions are subsequently reviewed and validated by expert radiologists.

Categories covered include abdominal pathologies, airspace opacification, chest lines and tubes, masses and bone lesions, C-spine levels, and body-part positioning, among others.

We report accuracy broken down by question type across all closed datasets (RadBench-Closed, ReXGradient-Closed, FRCR-Closed, Pathology-Closed), pooling items of the same type regardless of source dataset. This avoids aggregation artefacts from dataset-size imbalance. We restrict to question types with sufficient sample size (n > 500) and note the random baseline for each, which varies with the number of answer options.

Table 5 shows per-question-type accuracy. HR1.5 leads in Diagnosis (82.0%), Finding (81.2%), and Normal/Abnormal (82.8%). GPT-5.4 (OpenAI, 2026) achieves the highest Laterality accuracy (88.2%). All models substantially exceed random baselines, with the largest margins on Diagnosis where chance is 25%.

Table 6 presents overall closed-question accuracy grouped by body-part category. Body parts are mapped to four groups: CXR (chest/thorax), MSK (musculoskeletal, including spine, extremities, pelvis, head, and dental), Abdomen, and Other. ReXGradient items are assigned to CXR as the dataset contains only chest X-rays.

2.2.3 Open Question Datasets

The closed-format results above isolate discrete clinical decisions, but they constrain the model to a fixed set of options and so understate the harder problem a radiologist actually faces: producing free text that is both clinically correct and well formed. We therefore turn to open-ended evaluation, where each model generates unconstrained descriptions or full reports that are scored primarily with the Findings-Diagnosis methodology (Section 2.1), alongside conventional report-generation metrics stated in the literature. As discussed earlier, surface-level text-overlap scores can diverge sharply from clinical correctness, so the Findings-Diagnosis score is treated as the primary signal and the remaining metrics as context.

The following subsections report open-ended performance across four complementary settings: clinician-authored cases spanning multiple body parts and question types (RadBench, Section 2.2.3.1); large-scale public chest-X-ray report generation (RexGradient, Section 2.2.3.2); the internal, multi-body-part held-out set (RadCoverage-VQA, Section 2.2.3.3); and mammography (Section 2.2.3.4). Together they trace HR1.5’s open-ended behaviour from standardised, clinician-curated questions through to everyday reporting across the breadth of plain-film radiology.

2.2.3.1 RadBench Evaluation

RadBench is a clinician-authored evaluation framework developed at harrison.ai for benchmarking multimodal radiology foundation models. It consists of 105 unique X-ray cases with 497 questions (378 closed, 119 open) spanning question categories including finding presence, anatomical location, laterality, diagnosis, clinical reasoning, normal/abnormal classification, and modality identification.

Closed-format evaluation uses binary accuracy, precision, recall, F1, and Matthews correlation for yes/no questions, and accuracy with option-count-scaled weighting for multi-choice questions. Open-ended evaluation uses ROUGE (Lin, 2004), BLEU (Papineni et al., 2002), CheXBERT (Smit et al., 2020), RadGraph F1, and the Findings-Diagnosis score described above, and an LLM-as-a-judge (Gu et al., 2024) adjudication (Judge F-D).

A note on data availability. A portion of the RadBench cases is sourced from the NIH MedPix teaching atlas. Since RadBench was released (Sept, 2024), MedPix has been permanently retired by the U.S. National Library of Medicine (NLM); this is a permanent withdrawal rather than a temporary outage. As a consequence, many of the source images can no longer be retrieved, and the open-ended split has shrunk to only 69 usable cases. This remaining sample is too small to support an unbiased comparison across models. We therefore report the RadBench open-ended results below for completeness and posterity only, and caution against reading them as a reliable ranking; the larger-scale evaluations in the surrounding sections (ReXGradient, RadCoverage-VQA, and mammography), together with the FRCR simulations, provide the load-bearing evidence.

2.2.3.2 RexGradient

RexGradient (Zhang et al., 2025) is a large-scale chest X-ray evaluation benchmark comprising approximately 160,000 studies. We evaluate on the Zero-shot report generation task i.e. given patient demographics and clinical indication, generate a detailed findings statement

Similar to the RadBench evaluation, performance is measured using RadGraph F1, CheXBERT (Smit et al., 2020), RadCliQ (Yu et al., 2023) composite scores, and our Findings-Diagnosis methodology. Additionally, we evaluate over 11,000 closed-format questions derived from the dataset, spanning diagnosis (83%), finding presence, normal/abnormal, clinical diagnosis, report interpretation, and laterality categories.

Table 8 shows zero-shot report generation results. RadCliQ-v1 is a composite error metric (lower is better). CheXOne-Reasoning achieves the highest RadGraph F1 (0.238), RadCliQ-v1 (0.158), BLEU, and ROUGE-L, while CheXOne-Instruct edges ahead on CheXBERT (0.530). MedGemma-4B leads on the automated Findings-Diagnosis score (50.5%), while on the Judge Findings-Diagnosis score — the more reliable measure on this benchmark — HR1.5+ leads (73.0%). The gap between text-overlap metrics and F-D score highlights the distinction between surface-level textual similarity and clinical accuracy.

A note on this benchmark: a substantial portion of RexGradient consists of structured reports with additional sections and finer-grained nodes than a free-text findings statement. On such references the automated Findings-Diagnosis score performs suboptimally due to limitations of its underlying RadGraph entity extraction, whose finding-matching becomes brittle when the ground truth is heavily structured (for the reasons set out in Section 2.1), and the F-D column above reflects this. The Judge Findings-Diagnosis score is more representative of clinical correctness here, with HR1.5+ attaining the highest value (73.0%); even so, a residual gap remains under this stronger evaluator, underscoring that no current automatic evaluator fully captures report quality on structured data.

2.2.3.3 RadCoverage-VQA

RadCoverage-VQA is an internal, held-out evaluation set of clinical questions built for broad body-part coverage (chest X-ray, MSK, spine, abdomen, pelvis, and mammography) across a mixed set of medium-to-hard clinical tasks. Cases are drawn from validation-tagged production data, providing a diverse and clinically representative test set that extends well beyond chest-only evaluation.

Performance is measured using the Findings-Diagnosis methodology, with results broken down by body-part category. Body parts are grouped using the same mapping as the closed datasets (Section 2.2.2): CXR, MSK, Abdomen, and Breast, with remaining body parts (clavicle, coccyx, fibula, finger, sacrum) in Other.

Beyond aggregate benchmark scores, we also characterised qualitative aspects of HR1.5’s output through structured radiologist review. This section organises that assessment around six capabilities that together characterise how HR1.5 performs as compared to publicly available frontier models.

We compare (dated May, 2026) HR1.5+ (Harrison.Rad 1.5, agent-driven VQA, temperature 0) against three publicly available frontier multimodal models: GPT-5.4 (OpenAI), Gemini-3-Flash-Preview (Google) and Claude Opus 4.7 (bedrock-anthropic-claude-opus-4-7, AWS Bedrock). Decoding temperature was set to 0 wherever the model accepts it (HR1.5+, GPT-5.4 and Gemini-3-Flash-Preview) and left at the provider default otherwise: Claude Opus 4.7 (which does not respect a temperature parameter). All remaining sampling parameters were left at provider defaults. Every model received the same prompt and image(s) per case.

3.1 Accurate Findings Detection

Accurate image interpretation is core to the function of a radiology foundation model: correctly identifying the finding that is actually present, especially when it is subtle and easily mistaken for a more dramatic abnormality.

HR1.5+ names the scaphoid waist fracture directly. All three frontier models miss it, each with a different wrong read – GPT-5.4 a soft-tissue mass with “bones intact,” Gemini a giant-cell tumour of the tendon sheath, and Claude Opus 4.7 a distal radius (Colles’) fracture (figure 3.1).

3.2 Draft Report Generation

The primary requirement is that the draft be accurate and  faithfully to the findings, and correctness is never traded off against style. Given an accurate draft, the secondary aim is to present it in a form close to what the reporting radiologist would have written, so the content reads naturally and reduces cognitive load rather than expecting them to change practice. Crucially, matching the radiologist’s prose is intended to ease review, not to substitute for it. The draft is a starting point that the radiologist verifies and edits, and familiarity of style should make discrepancies easier to spot rather than encouraging uncritical acceptance.

Reference case 3.2, a portable chest radiograph demonstrating innumerable bilateral pulmonary metastases with an indwelling PICC line. The verbatim prompt and each model’s verbatim answer are shown below, first to the image alone (figure 3.2.1), then with the clinical history added (drafting a report is exactly where history should be used), with the key finding in each answer highlighted in bold (figure 3.2.2).

From the image alone, HR1.5+ names the metastases directly, while all three frontier models read the identical film as pneumonia / COVID-19.

With the history supplied, HR1.5+, Gemini-3-Flash-Preview and Claude Opus 4.7 all identify the bilateral pulmonary metastases, and HR1.5+ additionally reports the appropriately-sited PICC line. GPT-5.4 still reads the nodules as multifocal pneumonia and never mentions metastatic disease, a persistent miss even when handed the diagnosis-relevant history.

3.3 Longitudinal Reasoning with Priors

When prior imaging is available, HR1.5 automatically compares the current study against it and surfaces interval change, described in plain clinical language rather than as a flagged checklist. The comparison is performed at the moment the case is opened, so the report is contextualised against the patient’s history from the outset.

Reference case 3.3, current and prior chest radiographs (supplied labelled), with the clinical history “Day 2 of antibiotics, follow-up imaging.” The reference optimal report describes interval worsening of right-sided airspace opacification with a large right pleural effusion and new left mid-zone opacity (figure 3.3).

HR1.5+ reports the interval progression correctly, with increased right-sided airspace opacification, a new left mid-zone opacity and the enlarging right effusion, matching the reference, as does GPT-5.4. Gemini-3-Flash-Preview adds a spurious cardiomegaly, and Claude Opus 4.7 misses the large effusion and fabricates bilateral central venous catheters that are not present. Overt interval progression is thus within reach for HR1.5+; subtler change, for example a fracture occult on the prior and newly visible, is harder for a single pass, which the production comparison route addresses by reading each study independently and diffing the two.

3.4 Precise Anatomical Localisation

Findings are placed with greater anatomical specificity, including quantitative measurements such as the cardiothoracic ratio. The improvement is most pronounced on the spine, abdomen, and post-procedural imaging, among the areas of highest diagnostic complexity.

Reference case 3.4, a chest radiograph demonstrating a left apical mass in the left upper lobe, partially obscured by the clavicle (figure 3.4). HR1.5+ localises the mass correctly to the left upper lobe, as does Gemini-3-Flash-Preview; GPT-5.4 gets the side but mischaracterises it as a calcified mediastinal/chest-wall lesion. Claude Opus 4.7 lateralises it to the wrong (right) side, a laterality error with direct clinical consequences.

3.5 Complex Multi-Finding Reasoning

HR1.5 shows improved performance on high-cognitive-load studies that contain multiple findings, precisely the cases in which a finding is most likely to be missed. It also offers stronger coverage of rare-but-real findings that fall outside the classical reporting ontology.

Multiple coexisting findings (a chest study). Reference case 3.5a, PA and lateral chest radiographs demonstrating congestive cardiac failure, a study whose several coexisting findings (cardiomegaly, pulmonary venous congestion, interstitial oedema) must be tied to a single diagnosis (figure 3.5a).

HR1.5+ ties the cardiomegaly, pulmonary venous congestion and interstitial oedema together as congestive cardiac failure, as does GPT-5.4. Gemini-3-Flash-Preview instead diagnoses sarcoidosis, hallucinating bilateral hilar and paratracheal lymphadenopathy; Claude Opus 4.7 diagnoses combined pulmonary fibrosis and emphysema, calling the cardiomediastinal silhouette normal and so missing the cardiomegaly, two confident non-cardiac misreadings of a classic heart-failure film.

A complex single-region injury. Reference case 3.5b, a knee radiograph series demonstrating three coexisting findings: a lateral patellar dislocation, a medial patellar avulsion fracture, and a joint effusion (figure 3.5b).

HR1.5+ enumerates all three coexisting findings, as does Claude Opus 4.7 (describing the medial avulsion as an osteochondral fragment). GPT-5.4 mis-reads the displaced patella as a normal bipartite-patella variant, concluding “no acute fracture or dislocation,” while Gemini under-grades the dislocation as subluxation, the high-cognitive-load setting where a missed finding is most consequential.

3.6 Style and Template Adaptation

Slotting into an existing reporting workflow means following a prescribed report template exactly: populating the sections that apply and, just as importantly, suppressing those that do not, rather than padding the report with empty boilerplate. The test is therefore not whether a model can produce section headings (all of them can) but whether it treats the template’s conditional instructions as rules to obey.

Reference case 3.6, as shown in figure 3.6, a chest radiograph demonstrating interstitial thickening (a fibrosis pattern), with the clinical history “Shortness of breath” supplied. The study is reported against a fixed sectioned template (mirroring the structured variant the rad-agent production pipeline emits) in which Lines & Devices, Abdominal and Soft Tissue are explicitly marked optional, to be included only when relevant, while the remaining regions take a normal statement where no positive finding is present.

All four models detect the bilateral interstitial/fibrotic process, follow the capitalised heading convention, and produce the sectioned layout, so neither detection nor formatting separates them. The differentiator is the conditional instruction to include the optional sub-regions only when relevant. HR1.5+ and GPT-5.4 follow it exactly here, emitting only the populated regions and dropping all three optional sub-regions (Lines & Devices, Abdominal, Soft Tissue), none of which apply. Claude Opus 4.7 appends a redundant Soft Tissue: Unremarkable, and Gemini-3-Flash-Preview appends both Abdominal and Soft Tissue.

Where directions do align, the alignment tracks clinical relationships rather than arbitrary correlation. In CXR, the overlapping airspace and parenchymal processes group together: atelectasis and consolidation are the most aligned pair (0.77), with lung collapse (0.69) and ground glass opacity (0.67) nearby, while the support devices form their own cluster, the central venous catheter and endotracheal tube aligning at 0.75. In MSK, the degenerative-spine findings cluster tightly (osteoarthritis with joint space narrowing at 0.82, spondylosis with spondylolisthesis at 0.81, kyphosis with spondylosis at 0.75), and the fracture family clusters separately (the generic fracture aligning with displaced fracture at 0.74 and comminuted fracture at 0.67). The two clusters sit on opposite sides of the space: acute fracture directions are most anti-aligned with chronic degenerative ones (fracture versus spondylosis at −0.76), mirroring the clinical distinction between traumatic and degenerative disease. The representation therefore organises findings the way a radiologist would group them.

Beyond confirming that each finding is decodable, we test whether the model actually uses these directions rather than merely carrying them. For samples negative for a concept, the probe direction d is added to the residual stream during a forward pass as Δ = strength · s · d at every token position, with s the mean residual-stream norm so that strength is comparable across layers, swept over {−1.0, 0.0, +1.0, +2.0}. This follows additive activation steering (Turner et al., 2023; Rimsky et al., ACL 2024) along an inference-time intervention direction (Li et al., NeurIPS 2023), and the effect is read as the shift in the concept’s first-subword token logit relative to the unsteered baseline. We classify each concept vector by the control it affords, using suppression (Δlogit < 0 at strength −1.0) and induction (Δlogit > 0 at both +1.0 and +2.0) as the two tests:

• Bidirectional: subtracting suppresses the finding and adding induces it; the model both reads the direction and writes to it, so it is a usable handle for monitoring and for controlled intervention in either direction.
• Subtractive: subtracting reliably suppresses the finding, but adding does not sustain its induction; dependable for detection and suppression, not for induction.
• Additive: adding induces the finding, but subtracting does not suppress it; usable for eliciting or amplifying the finding, not for detection or suppression.
• None: neither direction produces the expected effect.

Tables 13 and 14 report the mean Δlogit at each strength for every concept, with its classification.

4.2 Concept Localisation: Attention as a Proxy

Localisation answers “where in the image did this answer come from?” by attributing each generated answer, and each individual answer word, back to spatial regions of the input. The engine offers several attribution methods that all produce the same artefact, a heatmap overlaid on the image, but derive it from different internal signals. Offering several is deliberate: gradient-based and attention-based attributions have different failure modes, and agreement between them is more trustworthy than any single map.

• GradCAM (answer-level): A gradient-based saliency method in the spirit of GradCAM (Selvaraju et al., ICCV 2017), adapted for HR1.5, attributes the model’s answer to the visual features that most influenced it and renders them as a heatmap over the image. The heatmap is question-sensitive: asking “Is there a pneumothorax?” vs “Are there any rib fractures?” on the same chest X-ray produces different maps focused on the relevant anatomical regions.
• Per-token gradient attribution: Because the answer is generated word by word, the same attribution can be computed for a single selected word rather than the whole answer, isolating the region responsible for “effusion” separately from the region responsible for “left”.
• Attention rollout (Abnar & Zuidema, ACL 2020): Composes the model’s attention across layers to approximate the cumulative flow of information that single-layer attention misses.
• Layer-wise Relevance Propagation (LRP) (Chefer et al., CVPR 2021): Propagates relevance backward from the answer to the input image regions, giving a more principled attribution than raw attention weights.
• Cross-modal attention: Reads out, directly from the model’s attention, which parts of the image each part of the answer drew on, requiring no gradients.

Mapping back to the image: Each method produces a value per image region, which is arranged into a spatial map and overlaid on the image. To keep the overlay faithful, regions that do not correspond to image content are excluded before the map is formed. The map can also be distilled into a set of discrete regions of interest, the contiguous areas of strongest attribution.

Attention as a proxy: It is well established that attention weights are not, by themselves, faithful explanations (Jain & Wallace, NAACL 2019), though counter-arguments show they are not meaningless either (Wiegreffe & Pinter, EMNLP 2019). These maps should not be read as a direct delineation of pathology: they indicate the regions the model attended to in forming its answer, not a segmentation of the finding itself. This distinction shapes how their quality should be measured. Overlap metrics such as IoU or DICE, which reward pixel-accurate agreement with a reference mask, are the wrong yardstick for an attention map: the model can attend to the correct anatomical region without its attribution tracing the lesion’s exact boundary. The appropriate measure is a hit rate: whether the attended region falls on the clinically relevant area the model should have considered when forming its view. We therefore adopt hit rate as a fidelity score for attended regions, and report it on two localisation datasets below.

We evaluate localisation on two public datasets that provide reference regions: FracAtlas (Abedeen et al., 2023), a musculoskeletal radiograph dataset with fracture bounding boxes, for MSK; and MS-CXR (Boecking et al., 2022), a chest X-ray phrase-grounding dataset with radiologist-drawn bounding boxes, for CXR. A case counts as a hit when the model’s attended region falls within the reference region for the finding. We report the localisation hit rate alongside answer accuracy on the same cases (Table 15).

FracAtlas localisation: ground-truth fracture regions (left of each pair) versus the model’s attended regions (right) across a sample of cases. MS-CXR examples cannot be shown here owing to dataset licensing restrictions, but the localisation behaviour is analogous and is deducible from the hit-rate results above.

These two gaps map onto the classical taxonomy of radiological interpretation error established through eye-tracking studies of visual search (Kundel, Nodine & Carmody, 1978), which distinguishes search errors (the finding is never fixated), recognition/perception errors (the region is fixated but the abnormality is not perceived, a category tied in human readers to insufficient dwell time), and cognition/decision errors (the abnormality is perceived but misinterpreted). The MSK gap, where the attended region more often falls outside the reference, is the analogue of a search error – the model’s attention does not reach the relevant area. The CXR gap, where the region is attended but the answer is still wrong, is the analogue of a cognition error – the model looks in the right place but draws the wrong conclusion. The intermediate, dwell-dependent recognition error has no natural counterpart here: inference completes in a single rapid forward pass, so dwell time is not a meaningful quantity for these systems.

4.3 Confidence and Certainty about the Response

Generative models, including large language models and vision language models, do not natively expose any signal of how confident they are in a given output. In high-stakes settings, where each output should be reviewed for correctness and clinical impact, this absence is a real barrier to adoption: a reader has no internal cue for which answers deserve closer scrutiny. Emerging approaches aim to derive a meaningful confidence estimate from the model’s own computation, but they remain early in their development and depend on substantial stabilisation and post-hoc steps such as tuning/calibration before a model’s reported confidence can be aligned with its actual reliability. With HR1.5 we explore this direction, with the aim of giving our users a more meaningful signal alongside each answer. The method described below is an early implementation: it already surfaces useful, discriminative signals, though it is not yet tuned/calibrated.

The confidence feature answers “how much should this answer be trusted?” by converting three internal signals (Table 16), all read at the final generated token, into a single uncertainty value, with confidence defined as its complement (Galil et al., ICLR 2023; in the spirit of Kadavath et al., 2022, showing a model’s activations carry information about its own reliability):

These are combined as a weighted sum U = w_e·H_attn + w_d·D_head + w_cv·CV, inverted to confidence = 1 − U (clipped to [0, 1]), and bucketed for display as HIGH / MEDIUM / LOW. This is a relative trust indicator, not a calibrated probability of correctness.

Why the raw score is biased by attention sinks: Transformer models exhibit attention sinks (Xiao et al., ICLR 2024; Sun et al., 2026), a small number of tokens (often positional or low-content) that attract a disproportionate share of attention mass regardless of semantics, tied to massive activation spikes in a few hidden channels. A sink concentrates attention mass on one or two tokens, making the distribution look sharp (low entropy), which the score above reads as high confidence, but the sharpness is a structural artefact of the architecture, not evidence the model is genuinely certain about this image and this question. Left uncorrected, sinks therefore deflate entropy and inflate confidence, and do so most where the model is least engaged with the content.

Sink detection and correction: The engine corrects for this in three steps:

1. Detect: for each candidate key position the received attention mass is averaged over batch, heads, and query positions; any position exceeding a fixed threshold is flagged as a sink (the same scan reports per-head sink ratios).
2. Mask and renormalise: the sink columns are removed from the last token’s attention row and the remainder renormalised to a proper distribution (p̃ = p[non-sink] / Σ p[non-sink]); entropy and head disagreement are recomputed on this reduced support. The representation-trajectory term is intrinsic to the hidden state and left unchanged.
3. Recompute and report inflation: the composite uncertainty is recalculated, and the confidence inflation inflation = max(0, (1 − U_raw) − (1 − U_corrected)) is reported explicitly. By construction it is non-negative: the correction can only reveal hidden uncertainty, never manufacture confidence.

Rather than silently replacing the number, the engine surfaces both values and the gap between them, for example, “Raw: HIGH → Corrected: MEDIUM (sinks inflate by 0.23)”, so a reader can see when the model’s apparent certainty was an artefact. This is a post-hoc, signal-level correction complementary to, not a substitute for, formal probability calibration such as temperature scaling on held-out data (Guo et al., ICML 2017).

Worked example, see figure 4.4: For the chest X-ray below, asked “Describe the findings in this chest x-ray,” HR1.5 responded with “The chest x-ray shows a 3.5cm mass in the left upper lobe, which is suspicious for malignancy. There are no other pulmonary nodules or masses identified. No pleural effusion is present. The cardiac size and mediastinal contours appear normal.”

4.4 Behaviour on Out-of-Distribution Inputs

A model’s response to inputs it could never have seen in training is as revealing as its performance on genuine studies. To probe this, we presented a synthetic chest X-ray generated with the GPT-5.5 image API from the prompt “generate a realistic looking x-ray of an apple in a human chest”, an image that looks radiographic but contains an object with no counterpart in any real study (figure 4.7).

The model accepts the synthetic image as a genuine radiograph and, when reporting findings, maps the unfamiliar object onto the nearest concept it has learnt (a left breast prosthesis) holding that interpretation with high confidence even when directly challenged. This exposes a real limitation: on concepts entirely outside its training distribution, the model is incorrect. Importantly, though, the error is a reasonable one: the synthetically generated apple has overlapping imaging characteristics with breast prosthesis or implant. The output is plausible given the model training data, rather than being outright nonsensical.

This points to a limitation that no confidence signal can fully remove: when an out-of-distribution input maps plausibly onto a learnt concept, as the apple does onto a prosthesis, the model can be confidently wrong, so a high value alone does not guarantee correctness. As the Röntgen case above shows, the signal can sometimes register an unfamiliar input as less certain, though not yet reliably. In our own system we already gate on this, rejecting inputs identified as out-of-distribution so that unreliable cases are held back rather than reported. Calibrating these confidence signals and measuring their reliability quantitatively remains future work; what we present here is an early, qualitative result.

Activation overlay for the synthetic case, concentrated on the apple-shaped opacity that the model labels a prosthesis, confirming that the confident misinterpretation is driven by the anomalous object itself, and that the model substitutes its closest learnt concept for one it was never trained to recognise.

Considered in aggregate, the evaluations support a consistent conclusion: HR1.5 attains its largest advantages on the tasks of greatest clinical consequence. On our internal, held-out mock simulations of the FRCR examinations, which reconstruct the assessments through which the Royal College of Radiologists certifies practising radiologists and serve as the closest available proxy in this study to authentic diagnostic reporting, HR1.5 and its agentic configuration HR1.5+ are the only systems to meet the passing standard, satisfying the Angoff-referenced threshold of the current 2B Short Case format. Every other general-purpose and medical-domain model evaluated falls substantially below it. Because this benchmark most directly approximates the clinical reporting task, the magnitude of this margin is of particular practical significance. We note that these examinations are internally constructed simulations rather than the official RCR sittings, which are not publicly available (Section 2.2.1)..

This advantage generalises beyond the examination setting. On closed-format clinical questions and on the multi-body-part internal evaluation, HR1.5 attains the highest accuracy across diagnosis, finding presence, and normal/abnormal classification, and across chest, musculoskeletal, and other anatomical regions, indicating that the improvement is not specific to a single modality or question type.

We are equally explicit about the settings in which HR1.5 does not lead. On a few open-ended chest-radiograph report-generation benchmarks, frontier general-purpose models and chest-specialised systems achieve higher values on individual metrics, and the agentic HR1.5+ configuration does not uniformly improve upon HR1.5, most notably under multi-turn dialogue evaluation. We attribute this in part to genuine headroom that remains to be addressed, and in part to the metric limitations characterised in Section 2.1, in which surface- and entity-level scores reward lexical form over clinical correctness. Neither consideration qualifies the principal result: on the task that most closely resembles, and is most likely to inform, routine radiological practice (examination-grade reporting) HR1.5 outperforms all evaluated systems by a substantial margin.

6.1 Towards Better Evaluation: The C6 Framework

Section 2.1 established why automated metrics diverge from clinical utility: conventional report-generation metrics track wording far more than clinical correctness, and even the Findings-Diagnosis score we adopt as the primary measure, with the Judge Findings-Diagnosis variant where automated matching is insufficient, remains a single aggregate. As noted there, it inherits the central limitation of all scalar scores: it can credit a report that misses a subtle finding, can accept a vague but incorrect descriptor as a match, and assigns comparable penalties to errors of very different clinical consequence. A report that identifies the correct diagnosis but omits a minor finding is scored similarly to one that hallucinates a dangerous pathology. A scalar score, however well-aligned, does not indicate why a report is adequate or inadequate: whether it identified the clinically important findings, whether it would be safe to act upon, and whether a radiologist would accept it.

Recent work has reached the same conclusion, that single-score metrics are insufficient for clinical text, and has begun to decompose evaluation into interpretable axes. DocLens (ACL 2024) evaluates medical text along completeness, conciseness, and attribution; GREEN (2024) introduces error-aware evaluation with interpretable explanations; RadCliQ (Yu et al., 2023) combines several signals into a clinically grounded composite; and CT-FineBench (2026) demonstrates that conventional metrics fail to capture granular diagnostic accuracy in CT report generation. These efforts point toward a consensus: clinically meaningful evaluation requires fine-grained, multi-dimensional assessment rather than a single opaque number.

To this end, we are developing the C6 evaluation framework, which scores a draft report along six interpretable clinical pillars (Table 17) and combines them into a single composite (Figure 6.1).

6.2 3D and Volumetric Imaging Support

HR1.5 is trained exclusively on 2D X-ray modalities. A natural extension is to support 3D volumetric imaging, particularly CT scans, which represent a substantial and growing share of radiology workload. Recent work has demonstrated the feasibility of foundation models for 3D radiology: CT2Rep (MICCAI 2024) introduced automated report generation for 3D medical images, Argus (ACL 2025) established the first comprehensive benchmark for 3D radiograph report generation, and several 2026 efforts have tackled challenges specific to volumetric data including entity hallucination mitigation, region-grounded report generation, and hierarchical vision-language modelling for efficient volume processing.

Extending HR1.5 to 3D introduces significant architectural and data challenges. Volumetric inputs are orders of magnitude larger than single X-ray images, requiring efficient spatial encoding strategies, whether through hierarchical 2.5D approaches, sparse attention over volume patches, or multi-view aggregation. The training data pipeline must be adapted for DICOM series with slice-level annotations and volumetric spatial relationships.

We plan to address these challenges in future model iterations, with the goal of delivering a unified radiology foundation model capable of interpreting both 2D and 3D imaging modalities within a single architecture.

We thank the radiologists and clinical collaborators whose expertise informed the design, scoring, and evaluation of HR1.5, and our colleagues across Harrison.ai for their engineering and research contributions. We gratefully acknowledge the open-source community, in particular PyTorch and Hugging Face and their contributors, whose frameworks supported the training, development, and evaluation of HR1.5, and the vLLM project and its contributors, whose tools underpin our serving stack, and Radiopaedia.org and its contributors for the teaching cases used under licence in the qualitative examples.