Gulum
615abfa510
CR-9 (major): gidByNameDob TryGetValue + ExternalRef-Fallback Verhindert KeyNotFoundException wenn Name/DOB zwischen zwei Laeufen driftet (z.B. correctDob-Remap oder UI-Umbenennung). Fallback: ExternalRef-Dict-Lookup; bei Miss: sauberes Ueberspringen + Note statt 500. +Test CR9_NameDOB_drift. CR-11 (major): Farbschlag aus Genotyp ableiten (fill-NULL-only) Deep-Band-Tiere (gen>=2, kein Farbschlag-Feld) landen nicht laenger mit null ColorVariety. GenotypePotentiallyMatches() vergleicht locus-pair-weise (??=wildcard, case-insensitive). Nur vollstaendige Genotypen (8 Loci, kein ??) loesen Ableitung aus. Plan-Loop: fuellt colorVarietyId bei null + vollstaendigem Genotyp. Post-Sweep: bestehende DB-Tiere mit null ColorVarietyId werden nachgefuellt. AnimalSummary.FarbschlagDerivedFromGenotype = Zaehler. +Test CR11_ColorVariety_from_geno. CR-10 (major, Python): malformed Override-Genotyp wird nicht angewendet apply_conflict_decisions validiert mapped8locus nach gt.parse(). Leeres Ergebnis = Genotyp unveraendert + decisionWarning statt stillem Blanken. Konflikt wird trotzdem aufgeloest (Entscheidung gilt, nur Genotyp-Override ausgelassen). +5 Python-Tests (CR-10-Block in test_extract.py). DB-1 (high): filtered unique index auf Gerbil.ExternalRef WHERE ExternalRef IS NOT NULL — verhindert doppelten Import bei Race-Conditions oder Lauf-Ueberschneidungen. Migration UniqueExternalRef. SQLite-Testhost: HasFilter() wird via EnsureCreated appliziert (SQLite unterstuetzt Partial-Indexes). PartialUpdateTests externalRef-Assertion auf NotNull geaendert (name-hash unique). GATE: 139/139 C# + Python ALL PASS, ef has-pending=No.
GerbilManager import tooling (FEAT-8b)
One-off migration tooling (Python, no third-party deps) that turns Julian's wife's hand-built spreadsheets into normalised JSON for review and, later, import. This is not product code — it lives outside the app and is run manually.
See the format analysis in FEAT-8a-format-spec.md (Pam's hive workspace).
What it does
extract.py runs stages 1–2 of the pipeline:
- Extract (stage 1)
- 10 Stammbaum pedigree charts → animals (name, DOB, death, Farbschlag, genotype, breeder, positionally-reconstructed parent links, photos).
- Wurfchronik litter chronicle → litters (date, dam, sire, Wurfstärke, sex breakdown, Zuchtnummer, notes). Columns are read by header row because the two sheets use different schemas.
- Embedded photos (
xl/media) →output/photos/<animal-slug>/, mapped to the animal by drawing anchor position.
- Dedup + review (stage 2)
- Merge animals on
normalise(call-name) + DOB, with the Zucht as discriminator (Julian's ruling: Wurfchronik[brackets]≡ Stammbaumof/von <line>suffix — both are the breeding line; same name+DOB but different Zucht stays two animals). - Match animals onto Wurfchronik litters (
litterRef) via DOB + (Vater, Mutter) — the Pam-validated build order (chronicle litters are canonical). - Emit a German-language
output/review-report.mdfor the breeder to verify (merges, conflicts, ambiguous/incomplete entries, unmapped genotype tokens, litter data-quality warnings). - Nothing is loaded into the database — stage 3 (API load) is separate and waits on DATA-2 + FEAT-1b phase 2.
- Merge animals on
Wurfchronik column semantics (Julian, authoritative)
A Wurfbezeichnung · B Geburtsdatum · C Mutter · D Vater ([…] = Zucht,
& = multiple sires) · E survivedToGoHome (Tabelle1 only, unlabeled —
detected positionally) · F Wurfstärke → totalBorn · G breakdown
Männchen,Weibchen,TG,s → males/females/stillborn/diedLater (s = died
after birth, before Abgabe) · last column → note. Validation: E should
equal F − TG − s; mismatches become German warnings in the review report
(data-quality signal, not an import blocker). A few Tabelle2 rows shift these
columns — they are read value-adaptively and flagged with a warning.
Genotypes are mapped to the frozen 8-locus contract (A C D E G P Sp Re) while
preserving everything: genotype.mapped8locus, genotype.rawGenotype (verbatim),
genotype.unmappedTokens (e.g. the Uw locus, markers WFNZ/WP/DP). A -
(unknown second allele) maps to ?.
Run
cd tools/import
python extract.py # uses the default source paths
python extract.py --stammbaeume "<dir>" --wurfchronik "<file.xlsx>"
Requires Python 3. Re-runnable / idempotent — re-run when more files arrive
(Wurfchronik Teil2+, or new charts).
Output (tools/import/output/, git-ignored except the report)
| File | Contents |
|---|---|
animals.json |
deduped animals with genotype, parentRefs, photos, sourceFiles |
litters.json |
litters from the Wurfchronik |
photos/<slug>/… |
extracted, anchor-mapped images |
review-report.md |
human review deliverable (committed) |
Files
xlsx_util.py— dependency-free.xlsxreader (zip + XML): shared strings, cells by reference, image/drawing anchors.genotype.py— genotype notation parser → 8-locus mapping + raw + unmapped.extract.py— the pipeline (stages 1–2).