Design Archive
中EN

(2) A first working build

National Medical Insurance Imaging AI Contest, preliminary round, part 2: After moving RAD41 into the organizer environment and aligning it with the official 17 findings, we trained new task heads. ROI regression produced the first version that could be submitted, scoring 39.31 on platform validation; none of the local fixes that followed advanced.

L.02 / 14
CATEGORY
Study
YEAR
2026
STATUS
Note
TOOLS
pytorch / rad-dino

entryWrite-up

The previous part settled on RAD41: a backbone trained before the competition, with a 15-class classification head and a 14-class localization heatmap head. Inside the organizer environment we still had to cover the competition’s five tasks, T1 to T5. T1 and T2 together are called the safety items, and T4 quantifies mass diameter and effusion grade. The environment also differed from our pre-competition setup: the GPU was an RTX 5090, which needed Torch 2.8 with cu128 to support sm120, and the data changed to official DICOM images with 17 findings, unlike the pre-competition proxy task. T3 reused the RAD41 backbone with a new task head for the official 17 findings; at first the old 15-class and 14-class heads were not reused. On 16 August the one-step training check passed, and the training, calibration, and validation splits all covered the 17 classes. There was no accuracy baseline yet.

The identity field PatientId was missing for some records, and reading every image header took a long time, so training kept being held up. The grouping merged records that had a PatientId by patient, created a node from the Accession for records without one, and then merged duplicates found by duplicate detection into the same group. The current data snapshot came to 14,051 annotated cases, initially split into 11,250 for training, 1,397 for calibration, and 1,404 for validation, and the one-step training check passed. Because strict separation between patients was never confirmed, internal results from this grouping are used for development evaluation only, and the scale of the experiments was limited as well.

To get an official 17-class detection baseline that could actually be compared against, the first run kept the backbone fixed and trained only the heatmap task head for 512 steps, then checked candidate boxes and matches on a fixed internal validation set of 1,024 cases. TP (correctly matched boxes) was 0, FP (false alarms) 97,411, and FN (missed targets) 2,617: the heatmap head produced a large number of candidate boxes, and not one of them located a target correctly. The recipe was stopped there.

The new recipe partly unfroze the last encoder layer and switched to regressing ROI boxes directly, that is, learning box positions directly: 512 steps of 17-class classification, objectness, and cxcywh box regression, followed by calibrating the threshold and top1 per class to control how many boxes came out. With loose settings TP was 890 and FP 108,485; with a 0.5 threshold and top1 per class, TP was 307, FP fell to 3,827, and F1 was 0.090976. The next check was whether more training on the same recipe would still help: continuing from 512 to 1,536 and then 2,560 steps raised F1 to 0.114312 and then 0.118841. In the last round TP was 398, FP 3,685, and FN 2,217, so misses were still high, and the second extension had already gained clearly less. No more steps were added to this recipe. The 2,560-step version was kept, with service decoding fixed at objectness threshold 0.75, top1 per class, and NMS 0.5.

The competition required safety, detection, quantification, and explanation outputs together. The other four tasks were wired up as follows: T1 trained a binary classifier on frozen features; T2 ran MIL splice detection on frozen features, with duplicate detection handled separately by retrieval; T4 concatenated global768 and ROI768 into a 1,536-dimensional input followed by a small MLP; T5 reused the detection boxes and attributes. Once the functional chain ran on 16 sample cases, validation of the full service began. These results, however, came from small samples, ground-truth boxes, and functional checks respectively; there was still no unified held-out result across the five tasks.

Once F1 had improved, evaluation switched to mAP, the metric the competition’s T3 focuses on, to check the coverage, ranking, and localization quality of the candidates. Each of the 17 findings is evaluated separately and the mAP over the 17 classes is reported, written as mAP17. Evaluation used a fixed shared clinical evaluation cache of 1,024 cases and tried two changes: taking top2 per class, and enlarging every box by 1.25×. Baseline mAP17 was 0.094427; top2 lowered it to 0.077274, and box enlargement to 0.091068. Top1 per class and the original boxes were kept.

On 12–13 September, to reuse chest-X-ray-compatible knowledge in the old heads and to reduce box overlap and scale errors, the experiments transferred the old task head for nine classes and continued training, and separately tried GIoU, a log width/height loss, removing the width/height floor, and winner box decoding, plus area-based reweighting. The best AP in the GIoU parameter grid was 0.097243, and the best scale-loss candidate reached 0.089538. Broken down by area bucket, every predicted box still fell into the large bucket, so the box-shrinking problem for small and medium targets remained unsolved. These runs differed from the earlier experiments in training budget, evaluation samples, and decoding, so their results cannot be read as causal comparisons. Area reweighting also lowered paired AP; the run without GIoU finished training but was missing files at evaluation time and is recorded as not evaluated. None of these candidates were adopted.

The first platform validation ran from 14 to 16 September, and the first attempt timed out. Investigation found that the platform’s directory identities and request scope differed from the filtering rules used for internal training. Without changing the model weights or the scoring rules, we fixed the directory and image-header identity checks and removed a training-sample filter that had been wrongly applied in the T1 service. The second validation ran through 2,735 cases with a total score of 39.31: safety scored 19.80, and T3, T4, and T5 scored 1.92, 10.44, and 7.15. The per-task figures the platform reports are contributions to the total, not raw mAP, and cannot be converted to or from internal mAP17. The existing records are also not enough to establish the single cause of the first timeout. This version was kept as a fallback, and work then focused on improving clinical detection.

The objectness threshold of 0.75 had been chosen for F1, so on 16 September it was rechecked against the mAP objective. With the same 1,024 cases and model weights fixed and one shared forward pass, four thresholds were compared. At 0.25, mAP17 reached 0.106935, above the 0.094427 at 0.75, but predicted boxes rose from 2,253 to 4,864, past the cap of 4,506 set in advance. A later recheck on a different sample set that had been used repeatedly during development still showed an AP gain, but T5 did not meet its predefined requirement. The low threshold was not adopted: 0.75 stayed, and no other threshold was swapped in after the fact.

The 17 September experiments set out to answer three questions: whether the early advantage seen with old-head transfer would hold, whether neighborhood supervision helped, and whether re-ranking candidates helped. Three things were tried: under the same training budget, a randomly initialized head against a transferred one; added 3×3 normalized supervision; and an extra old ROI quality-ranking head to re-rank the candidates. The transfer run, after 1,024 cumulative training steps, gained only 0.001692 and regressed on individual classes; 3×3 supervision gained 0.000554, short of the bar; quality re-ranking lowered mAP17 from 0.094427 to 0.072679. All three were stopped.

On 18–19 September the experiments checked whether the box head, explicit coordinates, GIoU, or the decoding channel was the key limit. The runs updated only the box head, added explicit coordinate inputs, and compared a low threshold with winner box decoding on another sample set. Updating only the box head scored below the original model; results with and without coordinates were nearly identical; winner box decoding at a 0.25 threshold did not beat ordinary decoding at 0.25 on AP, and the downstream metrics still missed their requirements. Nodule had 362 annotated boxes, and the three relevant configurations matched only 1, 2, and 1 of them. The threshold and channel sweeps stopped there. Further re-ranking of existing candidates was unlikely to fix the missed detections.

By 19 September the platform still held the 39.31 version, with T3 contributing 1.92, and none of the local routes had advanced. The next step kept this version’s backbone and the in-competition update to the last encoder layer, and replaced the detection head for retraining. Internal scores are compared only on the same samples with the same decoding, because the training steps, evaluation samples, and decoding of these runs were not fully aligned. The grouping did not confirm patient separation, so internal numbers are for development evaluation only.

processBefore & after

Platform validation total

ROI · 09-1639.31
single · 09-2147.78
quality · 09-2747.85
E-QH · 09-2848.01
re-run · 09-3048.01

Local fixes (internal mAP17)

Baseline · top1
0.094427
top2
0.077274 · dropped
Boxes × 1.25
0.091068 · dropped
GIoU best
0.097243 · dropped
Scale loss best
0.089538 · dropped
Old-head transfer
+0.001692 · dropped
3×3 neighborhood
+0.000554 · dropped
Box head only
0.090700 · dropped
Old quality re-rank
0.072679 · dropped
Threshold 0.25
0.106935 · over box cap
TP · threshold 0.25 / 0.75
411 / 366

resultTakeaways

Finding
Training ROI regression out to 2,560 steps raised internal F1 from 0.090976 to 0.118841. With all five tasks wired up, platform validation gave a total of 39.31, of which T3 contributed 1.92.
Parameters
RAD41 backbone · last encoder layer partly unfrozen · 17-class classification + objectness + cxcywh regression · 2,560 steps · objectness 0.75 · top1 per class · NMS 0.5
Open
Internal data was grouped by whatever identity fields were available, and strict separation between patients was not confirmed, so internal numbers serve as development evaluation only. The single cause of the first platform validation timeout was not established.

logChangelog

2026-08-16
Moved to the organizer environment; one-step training check passes, all three splits cover 17 classes
2026-08-28
Data grouping done: 14,051 annotated cases
2026-09-01
Heatmap head alone, 512 steps: TP 0, FP 97,411
2026-09-02
ROI regression trained on to 2,560 steps, F1 0.118841
2026-09-04
T1–T5 wired up; 16-case functional run passes
2026-09-11
Switched evaluation to mAP17; top2 and box enlargement both below baseline
2026-09-16
Service adapter fixed; platform validation 39.31
2026-09-19
None of the local fixes advanced; decided to replace the detection head

Related experiments

note
L.01(STUDY)2026

(1) Choosing a base model

chest-xraypytorchrad-dino+1
note
L.03(STUDY)2026

(3) A new detection head

chest-xraypytorchrad-dino+1
note
L.04(STUDY)2026

(4) Recipes that missed the adoption bar

chest-xraypytorchretinanet
← LabL.02 / 14
1 / 1

100%