Gulum/GerbilManager
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

IMPORT-POLISH v2: 3 Korrekturen nach god-Review

Browse Source 
FIX-1 ZUCHT: apply_conflict_decisions/apply_dob_remaps matchen jetzt
auf das VOLLE canon_pair-Tupel (nameCanon, zuchtCanon, dob) wenn die
Decision eine Zucht traegt; Fallback name-only wenn keine Zucht.
C3-Regel gewahrt: gleicher Name+DOB, andere Zucht -> kein Hit.
Neuer Regression-Test: Luna ZdkC-Decision trifft nur luna-kc,
nicht luna-bf (andere Zucht).

FIX-2 MERGE: dedup() waehlt das spezifischste Genotyp (fewest '?'
alleles) als sekundaeren Tiebreaker nach locus-count. CC schlaegt C-,
Gg schlaegt G- unabhaengig von der Reihenfolge. 3 neue Merge-Tests
(C- first/CC first/G-vsGg).

FIX-3 BACKFILL allDbNormToGid: ResolveParentForBackfill prueft jetzt
BEIDE Quellen: (a) createdAnimalByName (aktiver Lauf) und
(b) allDbNormToGid (alle DB-Tiere). Decktt den kritischen Fall:
Elterntier in fruehrem Lauf geladen, in diesem Lauf absent vom
Extract. Neuer 3-Lauf-SQLite-Test: Lauf 1 null-Vater, Lauf 2 laedt
Vater, Lauf 3 backfillt via allDbNormToGid.

Gate: 125/125 C#-Tests, Python ALL PASS, has-pending-model-changes=No.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gulum
2026-06-06 15:03:17 +02:00
parent 0c94cfcbf1
commit dfcd296119
4 changed files with 210 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
tools/import/extract.py
View File

@@ -627,9 +627,12 @@ def dedup(animals):
if a.get("deaf") is not None:
deaf_seen.add(a["deaf"])
tags_set.update(a.get("tags", []))
# pick the richest genotype (most mapped loci, then longest raw)
# pick the richest genotype: most mapped loci, then fewest unknowns ('?' alleles = specific
# wins, FIX-2), then longest raw string as final tiebreaker.
def _specificity(gd):
return sum(1 for pair in gd["mapped8locus"].values() for a in pair if a != "?")
best = max((a["genotype"] for a in grp),
key=lambda gd: (len(gd["mapped8locus"]), len(gd["rawGenotype"])))
key=lambda gd: (len(gd["mapped8locus"]), _specificity(gd), len(gd["rawGenotype"])))
out = {
"id": slug(base["name"], base["dob"]),
"name": base["name"],
@@ -892,22 +895,30 @@ def apply_dob_remaps(raw_animals, path):
"""PRE-dedup: a conflict-decision carrying `correctDob` marks a record as a DUPLICATE with a
wrong birthdate — remap that raw record's DOB to correctDob so dedup MERGES it into the
canonical same-named animal (e.g. Chelsea *15.10.2021 -> *02.04.2021). Match =
canon_pair(name)[0]+norm_dob(dob) (same identity as dedup — strips zucht suffix, folds
v.d.<->von den). Tolerates a missing/garbled file. Returns the remap count.
canon_pair(name)+(dob) with same Zucht-aware logic as apply_conflict_decisions (see there).
Tolerates a missing/garbled file. Returns the remap count.
Must run BEFORE dedup (it changes the dedup identity). (god/HUMANQUESTION D — Dubletten.)"""
remaps = {}
remaps_full = {} # (nameCanon, zuchtCanon, dob) -> correctDob — decision carries Zucht
remaps_name = {} # (nameCanon, dob) -> correctDob — no Zucht in decision
try:
with open(path, encoding="utf-8") as fh:
for r in (json.load(fh).get("resolutions") or []):
if r.get("correctDob"):
remaps[(canon_pair(r.get("name", ""))[0], norm_dob(r.get("dob", "")))] = r["correctDob"]
nc, zc = canon_pair(r.get("name", ""))
dob = norm_dob(r.get("dob", ""))
if zc:
remaps_full[(nc, zc, dob)] = r["correctDob"]
else:
remaps_name[(nc, dob)] = r["correctDob"]
except (OSError, ValueError):
return 0
if not remaps:
if not remaps_full and not remaps_name:
return 0
n = 0
for a in raw_animals:
new = remaps.get((canon_pair(a.get("name", ""))[0], norm_dob(a.get("dob", ""))))
nc, zc = canon_pair(a.get("name", ""))
dob = norm_dob(a.get("dob", ""))
new = remaps_full.get((nc, zc, dob)) or remaps_name.get((nc, dob))
if new and a.get("dob") != new:
a["dob"] = new
n += 1
@@ -917,26 +928,36 @@ def apply_dob_remaps(raw_animals, path):
def apply_conflict_decisions(merged, conflicts, path):
"""Consume human conflict resolutions (tools/import/conflict-decisions.json) so the wife's
answers UN-QUARANTINE animals. Schema: {"resolutions":[{name, dob, decision, genotype?,
farbschlag?, source}]}. Match = canon_pair(name)[0]+norm_dob(dob) — the same dedup identity
(call-name only, zucht stripped, v.d.<->von den folded). A matching animal: clear its
conflict, mark resolvedByDecision; an explicit `genotype` (breeder notation) is parsed and
becomes authoritative, `farbschlag` overrides too. Tolerates a missing/empty/garbled file.
farbschlag?, source}]}. Match = canon_pair(name)+(dob):
- When the decision name CARRIES a Zucht (zuchtCanon != ''), match on the FULL
(nameCanon, zuchtCanon, dob) triple — preserves the C3 rule that same name+DOB but
different Zucht = different animal.
- When the decision has NO Zucht, fall back to (nameCanon, dob) name-only match.
Both spellings v.d. / von den fold to the same canon. A matching animal: clear its conflict,
mark resolvedByDecision; an explicit `genotype` (breeder notation) is parsed and becomes
authoritative, `farbschlag` overrides too. Tolerates a missing/empty/garbled file.
Returns the number of conflicts resolved. (god/HUMANQUESTION D.)"""
decisions = {}
decisions_full = {} # (nameCanon, zuchtCanon, dob) -> r — when decision carries a Zucht
decisions_name = {} # (nameCanon, dob) -> r — fallback, decision has no Zucht
try:
with open(path, encoding="utf-8") as fh:
for r in (json.load(fh).get("resolutions") or []):
# FIX-1: use dedup identity (call-name only, zucht stripped) so that e.g.
# a decision written as "von den" matches a merged record with "v.d." spelling.
decisions[(canon_pair(r.get("name", ""))[0], norm_dob(r.get("dob", "")))] = r
nc, zc = canon_pair(r.get("name", ""))
dob = norm_dob(r.get("dob", ""))
if zc:
decisions_full[(nc, zc, dob)] = r
else:
decisions_name[(nc, dob)] = r
except (OSError, ValueError):
return 0
if not decisions:
if not decisions_full and not decisions_name:
return 0
resolved = 0
for a in merged:
d = decisions.get((canon_pair(a["name"])[0], norm_dob(a["dob"])))
nc, zc = canon_pair(a["name"])
dob = norm_dob(a["dob"])
d = decisions_full.get((nc, zc, dob)) or decisions_name.get((nc, dob))
if not d:
continue
a["resolvedByDecision"] = True