conflict-decisions.json jetzt 16 Eintraege, alle D-Konflikte entschieden. Re-Import nicht mehr auf D6 gated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2018 war falsch (leakte als '/ +2018' in den Gencode; in IMPORT-POLISH gefixt). D6 noch offen: Hanami, Kazu (2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aa Cc[chm] Dd Ee gg P- spsp (c[hm]→c[chm] normalisiert). D6 noch offen: Hanami, Kazu, Skarlett (3).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aa Cc[chm] DD Ee Gg Pp Spsp. Noch offen in D6: Hanami, Vance Jr., Kazu, Skarlett (4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CR-9 (major): gidByNameDob TryGetValue + ExternalRef-Fallback
Verhindert KeyNotFoundException wenn Name/DOB zwischen zwei Laeufen driftet
(z.B. correctDob-Remap oder UI-Umbenennung). Fallback: ExternalRef-Dict-Lookup;
bei Miss: sauberes Ueberspringen + Note statt 500. +Test CR9_NameDOB_drift.
CR-11 (major): Farbschlag aus Genotyp ableiten (fill-NULL-only)
Deep-Band-Tiere (gen>=2, kein Farbschlag-Feld) landen nicht laenger mit null
ColorVariety. GenotypePotentiallyMatches() vergleicht locus-pair-weise (??=wildcard,
case-insensitive). Nur vollstaendige Genotypen (8 Loci, kein ??) loesen Ableitung aus.
Plan-Loop: fuellt colorVarietyId bei null + vollstaendigem Genotyp.
Post-Sweep: bestehende DB-Tiere mit null ColorVarietyId werden nachgefuellt.
AnimalSummary.FarbschlagDerivedFromGenotype = Zaehler. +Test CR11_ColorVariety_from_geno.
CR-10 (major, Python): malformed Override-Genotyp wird nicht angewendet
apply_conflict_decisions validiert mapped8locus nach gt.parse(). Leeres Ergebnis
= Genotyp unveraendert + decisionWarning statt stillem Blanken. Konflikt wird
trotzdem aufgeloest (Entscheidung gilt, nur Genotyp-Override ausgelassen).
+5 Python-Tests (CR-10-Block in test_extract.py).
DB-1 (high): filtered unique index auf Gerbil.ExternalRef
WHERE ExternalRef IS NOT NULL — verhindert doppelten Import bei Race-Conditions
oder Lauf-Ueberschneidungen. Migration UniqueExternalRef. SQLite-Testhost:
HasFilter() wird via EnsureCreated appliziert (SQLite unterstuetzt Partial-Indexes).
PartialUpdateTests externalRef-Assertion auf NotNull geaendert (name-hash unique).
GATE: 139/139 C# + Python ALL PASS, ef has-pending=No.
1) TOLERANTER KC-MATCHER (extract.py norm_zucht): trailing \b nach '.' greift nicht
zwischen two non-word-chars (z.B. nach '.' vor ' '). Fix: r\bv\.\s?d\. (kein \b
am Ende). Alle Schreibvarianten ('Zucht der Kleinen Chaoten', 'kleinen Chaoten',
'v.d. Kleinen Chaoten', '[ZdkC]') -> canon 'kleinechaote'. +is_clan_zucht() helper.
+11 Python-Tests (Regression + neue Varianten). FIX-1-Entscheidungs-Matching
unberuehrt (canon_pair loest v.d. bereits via split_name_zucht).
2) HERKUNFT-BACKFILL (ImportService.cs, fill-NULL-only): POST-Sweep ueber alle
DB-Tiere mit OriginBreeder==null && IsResident==true. Leitet OriginBreeder aus
dem Elterntier (FatherId/MotherId -> OriginBreeder) ab; Fallback 'Zucht der
Kleinen Chaoten'. NIEMALS ueberschreibt nicht-leeren OriginBreeder. Trocken-Lauf
zaehlt HerkunftBackfilled; Execute schreibt. Fixt Tier 'C' (OriginBreeder null).
3) FARBSCHLAG-WOULD-REBACKFILL (no-op Zaehler): zaehlt bereits-importierte Tiere,
bei denen der aktuelle Extraktor einen anderen Farbschlag liefert als in der DB.
Kein Overwrite (Sicherheitsmechanismus ausstehend auf god/Julian-Freigabe);
Optionen A/B/C an god gemeldet, Empfehlung: Option B (Timestamp-basiert).
4) ResidencySummary um HerkunftBackfilled + FarbschlagWouldRebackfill erweitert.
ImportReport-Notizen fuer beide neuen Zaehler.
GATE: 133/133 C#-Tests, 57 Python-Tests, ef has-pending=No.
FIX-1 ZUCHT: apply_conflict_decisions/apply_dob_remaps matchen jetzt
auf das VOLLE canon_pair-Tupel (nameCanon, zuchtCanon, dob) wenn die
Decision eine Zucht traegt; Fallback name-only wenn keine Zucht.
C3-Regel gewahrt: gleicher Name+DOB, andere Zucht -> kein Hit.
Neuer Regression-Test: Luna ZdkC-Decision trifft nur luna-kc,
nicht luna-bf (andere Zucht).
FIX-2 MERGE: dedup() waehlt das spezifischste Genotyp (fewest '?'
alleles) als sekundaeren Tiebreaker nach locus-count. CC schlaegt C-,
Gg schlaegt G- unabhaengig von der Reihenfolge. 3 neue Merge-Tests
(C- first/CC first/G-vsGg).
FIX-3 BACKFILL allDbNormToGid: ResolveParentForBackfill prueft jetzt
BEIDE Quellen: (a) createdAnimalByName (aktiver Lauf) und
(b) allDbNormToGid (alle DB-Tiere). Decktt den kritischen Fall:
Elterntier in fruehrem Lauf geladen, in diesem Lauf absent vom
Extract. Neuer 3-Lauf-SQLite-Test: Lauf 1 null-Vater, Lauf 2 laedt
Vater, Lauf 3 backfillt via allDbNormToGid.
Gate: 125/125 C#-Tests, Python ALL PASS, has-pending-model-changes=No.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
FIX-1 decision-matching: apply_conflict_decisions/apply_dob_remaps
nutzen jetzt canon_pair(name)[0] als Match-Key (Dedup-Identitaet:
call-name ohne Zucht, v.d.<->von den gefaltet). Workaround-Spelling
v.d. in Victoria Welbys Decision bleibt erhalten; beide Formen
matchen jetzt. Kommentar im decision-Eintrag aktualisiert.
FIX-2 specific-wins: _alleles_compatible aendert '? vs x = False'
-> '? vs x = True' (spezifischer Wert gewinnt). C- vs CC, G- vs Gg,
P? vs PP sind kein Konflikt mehr. Echte Wert-Widersprueche (DD vs Dd,
Ee vs ee, PP vs Pp) bleiben Konflikte. Loest Enya, Ella, Zac
automatisch (Konflikte 8->5 erwartet). 2 bestehende Tests angepasst,
7 neue Tests.
FIX-3 parent-FK backfill: nach dem Wurfchronik-Rueckverknuepfungs-
Block iteriert ImportService.RunAsync ueber bereits importierte
Wuerfe mit null Father/MotherId und setzt fehlende FKs wenn das
Elterntier jetzt ladbar ist. Trockenlauf zaehlt, Execute schreibt.
LitterSummary.ParentFksBackfilled + 2 neue C#-Tests (SQLite).
FIX-4 Skarlett-Artefakt: parse_detail() strippt trailing / +YEAR
aus dem Genotyp-Tail (re.sub). Sterbejahr bleibt als death-Date
erhalten -> Skarlett erscheint als reiner Sterbedatum-Konflikt.
2 neue Python-Tests.
Gate: 124/124 C#-Tests, Python test_extract/test_genotype ALL PASS,
has-pending-model-changes = No.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
god added a `correctDob` (DD.MM.YYYY) decisions field: the matched (name+dob)
record is a DUPLICATE with a wrong birthdate → remap its DOB to correctDob so
dedup MERGES it into the canonical same-named animal. apply_dob_remaps runs
BEFORE dedup (it changes the dedup identity); tolerates a missing file; logged
as "DOB-Remaps: N". First use: Chelsea *15.10.2021 → *02.04.2021 (merges into
the canonical record). test_extract covers the remap + that both records then
share one name+dob identity.
Extractor-only. python + dotnet 121/121 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Breeder merge rule: when two source variants of the SAME animal differ ONLY
by a token PRESENT in one and ABSENT in the other — a whole locus (e.g. spsp
charted in one source, omitted in another) or a modifier on the same base
allele (e^f vs e, the [f] marker) — keep the present token; that is NOT a
conflict. Genuine VALUE contradictions still quarantine: different base
alleles (Ee↔ee), unknown-vs-filled (D-↔DD), different modifiers (c[h]↔c[chm]),
C-↔Cc[h], P-↔Pp.
Replaces the old `len(distinct normalized geno keys) > 1` test with
_genotype_conflict() (per-locus, per-allele compatibility; '?'-vs-filled is a
contradiction, modifier-present-vs-absent and whole-locus-absence are not).
Markers/flags (WP/DP/WFNZ/hörend) are already tags/flags, never genotype, so
they never reach conflict detection; empty Farbschlag/death already don't
conflict (only non-empty values are compared).
Clears Daja (keep spsp), Ichika (keep ee[f]) and the D4 marker cases:
Konflikte 19 -> 15. test_extract covers spsp/[f] present-vs-absent =
no conflict and the four genuine-contradiction shapes. python + dotnet
121/121 green; extractor-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
god extended conflict-decisions.json with an optional dateOfDeath
(DD.MM.YYYY). apply_conflict_decisions now sets the animal's death date
(normalized) as authoritative when present — clearing D5 death-date
conflicts the same way genotype/farbschlag decisions are applied. No C#
change (death already flows to Gerbil.DateOfDeath). test_extract covers a
Flint dateOfDeath resolution. python green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
god maintains tools/import/conflict-decisions.json as Julian/his wife answer
the D-conflicts. extract.py now consumes it (apply_conflict_decisions): for an
animal matching normalize(name)+dob, it clears the conflict, marks
resolvedByDecision, and — when the decision carries a `genotype` (breeder
notation, parsed via genotype.py) and/or `farbschlag` — treats those as
AUTHORITATIVE. Tolerates a missing/empty/garbled file. Genuinely-unresolved
conflicts stay quarantined.
Loader (ImportService): SourceAnimal.ResolvedByDecision flows through; the
report surfaces Animals.ConflictsResolvedByDecision + a German note.
Result on real data: the 2 current decisions (Firefly D-/PP, WildFire PP)
un-quarantine → Konflikte 21 → 19. As god appends entries the count grows;
nothing else needed from me.
Tests: python test_extract (decision clears conflict + genotype authoritative
+ removes from conflicts list + tolerates missing file) and a C# loader test
(a resolved animal loads and is counted). Folded into the EXTRACT-BANDS branch
so the next re-extract applies band-aware Farbschlag + these decisions in one
pass. No schema change (JSON DTO fields). python + dotnet 121/121 green;
has-pending clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Julian-confirmed against the real xlsx (Kentucky, Akio Kids): pedigree
blocks have two shapes. EARLY bands (gen 0-1, cols B/E/H) are 5-cell
Name/DOB/Farbschlag/Genotype/Zucht — real Farbschlag present. DEEP bands
(gen>=2, cols K/N/Q...) are 3-cell Name/DOB/Genotype with NO Farbschlag —
the colour is derived from the genotype.
The old extractor grabbed the NEXT block's name or a stray health note
(e.g. "DD-Tumor", "Chronische Ohrenentzündung...") as the deep-band
Farbschlag — source of the D1 'name-in-Farbschlag' conflicts and a big
chunk of farbschlagUnmatched. Fix: gen_of(col)>=2 => never assign a
Farbschlag in that block (Name/DOB/Genotype only); the cell is left for the
next block's name search. Deep-band animals load with empty Farbschlag and
the UI computes 'Errechnet' from the stored genotype (lossless).
Dedup already prefers a real (non-empty) early-band Farbschlag, so an animal
appearing in both an early band (Chesnut -> 'Kohlfuchsschimmel') and deep
bands keeps the real value; Chesnut and Tennessee stay separate (distinct
genotype + name+DOB key).
Result: 248/306 loadable animals now have empty Farbschlag (genotype-
derived), 0 suspicious name/note values remain, 0 Farbschlag-based conflicts
(D1 dissolved). Regression test test_extract.py builds a two-band xlsx and
asserts the deep band yields no Farbschlag + the stray note doesn't bleed.
Extractor-only (no C# change). python test_genotype + test_extract green;
dotnet 118/118. Batches with Kevin's GEN-3f seed for one re-import.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Structural fix (god, Julian-reported via 'C'): the loader ignored
SourceAnimal.ParentRefs, so animals whose ancestry exists only as
Stammbaum chart-position refs loaded with LitterId=null ("unbekannt").
ImportService now synthesizes/reuses a derived litter from parentRefs:
resolves father+mother via name+DOB, groups siblings (same parents+dob)
into one litter, sets Father/Mother + offspring LitterId, dates it to the
offspring DOB, and tags Notes "aus Stammbaum-Diagramm abgeleitet
(Konfidenz: …)" so it's transparent/reversible. Existing animals that
become linkable are re-linked on re-run (sweep-idempotent). Dry-run counts
included. Projected: ~124 loadable animals gain a parent link.
Box-colour = sex (Julian): blue box = male, white box = female. All 11
pedigrees encode this as a solid theme-8 (accent5/blue) fill vs no fill.
xlsx_util.cell_fill_sex reads it; extract.py sets animal.gender from the
box; ImportService.InferGender prefers it over sire/dam name inference.
Result: 306/306 loadable animals now sexed (154♂/152♀).
Extractor noise fix (god): reject Farbschlag values that are actually a
parent NAME bled across cells (contain v.d./von/of/gen.) — cleared phantom
conflicts (e.g. Chayton). Combined with GEN-3 Uw→G: Konflikte 32→21.
Also skip Excel "~$" lock files in the glob.
GEN-3a contract (Kevin): ComposeGenotype appends "Slsl" for WP/Sls
carriers (wild-type sl/sl omitted) so 8-locus strings stay unchanged.
Importer-only. The live re-import into Julian's DB stays a separate
supervised gated step. 95 C# tests + python genotype tests green;
has-pending-model-changes clean (no schema change on this branch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
genotype.py:
- Uw/uw aliased to G/g (same locus) so the D2 conflict group + pure-Uw
cases stop being conflicts (Gg == Uwuw).
- Sls/WP recognized as a SECOND spotting locus (S(l)s(l)=WP het); carried
into mapped8locus alongside Sp (Sp+Sls = Superschecke).
- dea/Dea/taub/hörend -> hearing/deaf phenotype FLAG (not a locus).
- WFNZ/RV/GV/DP -> provenance/breeding tags (not genotype, not conflicts).
- test_genotype.py: zero-dep unit tests for all four.
extract.py: surface deaf+tags on animals; dedup conflict detection now
compares the NORMALIZED genotype key (mapped8locus) instead of the raw
string, so Uw=G no longer triggers a conflict. Result: Konflikte 32 -> 27,
Zucht-Splits stays 0. Dedup identity = name + DOB + Zucht.
Backend: Gerbil.IsDeaf (bool?) + additive migration AddGerbilDeafFlag
(has-pending-model-changes clean) + GerbilDto/GerbilInput round-trip.
ImportService sets IsDeaf from animal.deaf and preserves Sls + tags + deaf
in RawImportData (kept out of the 8-locus compact Genotype contract until
GEN-3a adopts them).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a machine-readable conflict:bool to each merged animal (true when files
disagree on genotype/farbschlag/death) so the import loader can quarantine the
32 conflicts without parsing the German review report.
- survivedToGoHome: unlabeled Tabelle1 col E detected positionally; value-
adaptive row parsing recovers it from schema-shifted Tabelle2 rows too
(118 recovered, 79 confirmed by the E=F-TG-s identity)
- breakdown G -> males/females/stillborn/diedLater ('s' = died before Abgabe)
- validation: E = F - TG - s; 113 mismatches as German review-report warnings
- (name, Zucht) canonicalisation: [brackets] == of/von suffix ([ZdkC] ==
von den Kleinen Chaoten); Zucht = dedup discriminator (0 splits in data)
- animal->litter matching via DOB+(Vater,Mutter): 95 high-confidence,
40 date-only, 9 ambiguous; litterRef in animals.json
- regenerated report: 889 raw -> 574 unique (279 dated), 32 conflicts
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
