# OCR Comparison Summary
## Gemini Flash vs Control Test — Calgary Highlanders War Diary, Sep 44

---

## Overall Assessment

Gemini 2.5 Flash and 2.0 Flash produced **identical outputs on every page** — no meaningful difference between the two models for this document type. The `<think>` block leakage and inline CSS issues noted in the partial test earlier were consistent throughout.

---

## Issues Log

### Truncation

---

#### Issue #1 — Truncation: Dense multi-entry pages
| | |
|---|---|
| **Page** | 6 (doc page 7) |
| **Severity** | High |
| **Description** | Both models truncate after the 6 Sep entry, dropping the entire 7 Sep entry including the Loon Plage operational passage. Substantive content lost. |

---

#### Issue #3 — Truncation: Sparse pages with low-positioned content
| | |
|---|---|
| **Page** | 51 (signing page) |
| **Severity** | Medium |
| **Description** | Entire signature block missed (`R.L. Ellis, Major, A/O.C. The Calgary Highlanders CA(O)`). Model processes empty space and stops before reaching content at bottom of page. |

---

#### Issue #16 — Truncation: Long narrative pages
| | |
|---|---|
| **Page** | 29 (C.O.'s address) |
| **Severity** | Medium |
| **Description** | Excellent accuracy on degraded text but drops final 1.5 paragraphs. Third confirmed truncation instance. Pattern is not purely length-based — page 30 of similar density did not truncate. |

---

### Handwriting & Superscript Errors

---

#### Issue #2a — `pårs` → `pdrs`
| | |
|---|---|
| **Page** | 27 (30 Sep entry) |
| **Severity** | Low–Medium |
| **Description** | Standard military abbreviation for *pounders* misread. Corrupts a common artillery term. |

---

#### Issue #2b — `theme` → `them.`
| | |
|---|---|
| **Page** | 27 (30 Sep entry) |
| **Severity** | Low |
| **Description** | Superscript/correction mark misread as a letter. |

---

#### Issue #2c — `could not whether` → `could not say whether`
| | |
|---|---|
| **Page** | 27 (30 Sep entry) |
| **Severity** | Low–Medium |
| **Description** | Handwritten superscript insertion *say* missed entirely. Meaning subtly changed. |

---

#### Issue #8 — Cursive `I`/`J` confusion
| | |
|---|---|
| **Page** | 17 |
| **Severity** | Medium |
| **Description** | `I Sec` (Intelligence Section) misread as `J. See`. Domain-specific routing term lost. Pattern likely recurs across annotated documents throughout the corpus. |

---

#### Issue #18 — Fundamental failure on cursive handwritten pages
| | |
|---|---|
| **Page** | 32 (handwritten narrative appendix) |
| **Severity** | **Critical** |
| **Description** | Flash output largely unintelligible on cursive handwriting. `Leen Cire des. thae Laucklan`, `Euront pour Koon plage`, `th officers coat pere Loobeeghe`. olmOCR substantially superior on this document type. **Flash is not fit for purpose on handwritten content.** |

---

### Character Substitution on Degraded Scans

---

#### Issue #12 — Systematic `S`→`3` substitution
| | |
|---|---|
| **Pages** | 23, 24, others |
| **Severity** | Medium |
| **Examples** | `3ASK R` (SASK R), `30UTH` (SOUTH), `GUARD3` (GUARDS) |
| **Description** | Scan/font artefact where certain `S` characters render as `3`. Consistent enough to warrant a post-processing correction rule, particularly for known unit names. |

---

#### Issue #14 — `S`→`3` in ambiguous location codes
| | |
|---|---|
| **Page** | 25 |
| **Severity** | Medium |
| **Description** | `BL-U30T` → `BL-USOT`. Same substitution pattern as Issue #12 in an already-ambiguous map reference or location code. |

---

#### Issue #15 — `Z`→`S` substitution
| | |
|---|---|
| **Page** | 26 |
| **Severity** | Medium |
| **Description** | `PS FAUST` → `PZ FAUST` (Panzerfaust). Extends the character confusion cluster from Issue #12. Corrupts a specific weapons identification in a PW intelligence report. |

---

### Document Type Failures

---

#### Issue #4 — Complete failure: Complex dual-page appendix forms
| | |
|---|---|
| **Page** | 10 (Field Return of Officers) |
| **Severity** | **Critical** |
| **Description** | Both models abandon transcription after the header, producing no table data. Entire Part A strength returns, Part B officer movements, Part C, and Part D nominal roll of **44 named officers** completely lost. Most severe structured-data failure mode in the test. |

---

#### Issue #6 — Floating annotations misassigned to table rows
| | |
|---|---|
| **Page** | 13 (DF Task Table) |
| **Severity** | Medium–High |
| **Description** | `Cancelled. Amstaten. 28: 1800. hr.` is a table-wide annotation but gets assigned to DF 1 and DF 3 rows in flat output. Risk of silent data corruption in downstream structured extraction. |

---

#### Issue #7 — Diagram/trace pages: fragmented labels and partial edge reads
| | |
|---|---|
| **Page** | 14 (brigade disposition diagram) |
| **Severity** | Medium |
| **Description** | Unit names split across lines (`CALG` / `HIGHRS`, `FD` / `COY`). Partially visible edge text transcribed as garbled fragments (`CET ISSUED IN`, `ONJUNCTION`) with no uncertainty flag. Silent partial reads are more dangerous than explicit illegibility markers. |

---

### Message Form Issues

---

#### Issue #5 — Message form structure not preserved
| | |
|---|---|
| **Pages** | 12, 20, 21, 22, 23, 24, 25, 26 |
| **Severity** | Low |
| **Description** | Handwritten signatures replaced with `[handwritten signature/initials]` placeholder. Content present but unstructured. Minor relative to other issues. |

---

#### Issue #9 — Message form title not recognised from partial scan
| | |
|---|---|
| **Pages** | 21, 26 |
| **Severity** | Low–Medium |
| **Description** | `MESSAGE FORM` rendered as `SSAGE FORM` / `MASSARE FORM` on edge-truncated scans. Affects document type classification. |

---

#### Issue #10 — Redaction markers rendered as nonsense strings
| | |
|---|---|
| **Page** | 21 |
| **Severity** | Low–Medium |
| **Description** | `XXXXXXX` overtyping rendered as `Kaxxkuka` rather than `[redacted]` or `[illegible]`. Will confuse downstream parsers. |

---

#### Issue #11 — Systematic recipient line garbling on message forms
| | |
|---|---|
| **Pages** | All Army Form C2136 pages |
| **Severity** | Medium |
| **Description** | `TO R H C R DE MAIS CALG HIGHRS` consistently garbled. Worst instance: `CALO NIGHTS`. Root cause is multi-column header box layout being read linearly. Affects unit identification across multiple appendices. |

---

#### Issue #13 — Degrading accuracy on poor-quality scans
| | |
|---|---|
| **Page** | 26 |
| **Severity** | High |
| **Description** | Non-linear performance drop on faded scans — `MATSUHILLING` (MAJ SCHILLING), `MARZAM` (MERXEM), `CALO NIGHTS` (CALG HIGHRS). Model performance on message forms is scan-quality dependent in an unpredictable way. |

---

### Other

---

#### Issue #17 — Degraded newsprint misreads
| | |
|---|---|
| **Page** | 31 (The Glen news sheet) |
| **Severity** | Low |
| **Description** | `FRANGE` (FRANCE), `COLUMES` (COLUMNS), `AIR` (AFTER). Honest misreads on genuinely difficult source material rather than model failure. |

---

## Pattern Summary

Three distinct failure categories emerge:

| Category | Issues | Notes |
|---|---|---|
| **Truncation** | #1, #3, #16 | Consistent tail-end content loss across dense, sparse, and long pages. Not length-dependent. Needs prompt engineering fix or post-processing length check. |
| **Character substitution** | #12, #14, #15 | `S`→`3`, `Z`→`S`, `I`→`J` clusters. Scan quality dependent. Amenable to post-processing correction rules for known military vocabulary. |
| **Document type failures** | #4, #18 | Complex dual-page forms and handwritten pages beyond Flash capability entirely. Needs routing to different handlers. |

---

## Recommendation

Gemini Flash is **viable for typed appendix pages** — message forms, movement orders, typed narrative diary entries — where its performance is generally good to excellent.

It is **not viable** as a replacement for olmOCR on handwritten content, and should not be used as the primary OCR engine for complex structured forms.

The most productive use would be as a **supplementary model for document classification and typed-content extraction** in the pipeline you already have planned, not as an olmOCR replacement.