GEN-3b: import notation normalization (Uw=G, Sls, deaf flag, tags)

genotype.py:
- Uw/uw aliased to G/g (same locus) so the D2 conflict group + pure-Uw
  cases stop being conflicts (Gg == Uwuw).
- Sls/WP recognized as a SECOND spotting locus (S(l)s(l)=WP het); carried
  into mapped8locus alongside Sp (Sp+Sls = Superschecke).
- dea/Dea/taub/hörend -> hearing/deaf phenotype FLAG (not a locus).
- WFNZ/RV/GV/DP -> provenance/breeding tags (not genotype, not conflicts).
- test_genotype.py: zero-dep unit tests for all four.

extract.py: surface deaf+tags on animals; dedup conflict detection now
compares the NORMALIZED genotype key (mapped8locus) instead of the raw
string, so Uw=G no longer triggers a conflict. Result: Konflikte 32 -> 27,
Zucht-Splits stays 0. Dedup identity = name + DOB + Zucht.

Backend: Gerbil.IsDeaf (bool?) + additive migration AddGerbilDeafFlag
(has-pending-model-changes clean) + GerbilDto/GerbilInput round-trip.
ImportService sets IsDeaf from animal.deaf and preserves Sls + tags + deaf
in RawImportData (kept out of the 8-locus compact Genotype contract until
GEN-3a adopts them).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 10:26:01 +02:00
parent bd9cc83205
commit 2f089a902d
13 changed files with 1645 additions and 62 deletions

View File

@@ -227,6 +227,8 @@ def extract_stammbaum(path):
"gender": None,
"farbschlag": farbschlag,
"genotype": genodict,
"deaf": genodict.get("deaf"),
"tags": genodict.get("tags", []),
"breeder": breeder,
"zucht": zraw,
"parentRefs": [],
@@ -251,7 +253,8 @@ def extract_stammbaum(path):
animals.append({
"id": None, "name": part, "nameVariants": [],
"dob": "", "death": "", "gender": None, "farbschlag": "",
"genotype": gt.parse(""), "breeder": "", "zucht": zraw,
"genotype": gt.parse(""), "deaf": None, "tags": [],
"breeder": "", "zucht": zraw,
"parentRefs": [], "photos": [], "sourceFiles": [fname],
"_gen": gen_of(c), "_col": c, "_row": r, "_file": fname,
"_zucht": norm_zucht(zraw),
@@ -472,10 +475,17 @@ def _to_int(s):
# ------------------------------------------------------------- stage 2: dedup
def _geno_key(genodict):
"""Canonical, order-independent key of a genotype's mapped loci — used for conflict
detection so Uw==G (and allele ordering) no longer count as a conflict."""
m = genodict.get("mapped8locus", {})
return "|".join(f"{locus}:{','.join(sorted(m[locus]))}" for locus in sorted(m))
def dedup(animals):
"""Merge by normalise(call-name)+DOB, with the canonical Zucht as
DISCRIMINATOR (Julian: same name+DOB but different Zucht = different
animal). Returns (merged, conflicts, orphans, zucht_splits)."""
DISCRIMINATOR (Julian: same name+DOB+Zucht = same animal; different Zucht =
different animal). Returns (merged, conflicts, orphans, zucht_splits)."""
groups = {}
orphans = []
for a in animals:
@@ -521,19 +531,26 @@ def dedup(animals):
photos = list(base["photos"])
parent_refs = list(base["parentRefs"])
genos = set()
geno_keys = set() # GEN-3b: conflict on NORMALIZED genotype (Uw==G) not raw text
farb = set()
deaths = set()
deaf_seen = set()
tags_set = set()
for a in grp:
variants.add(a["name"])
files.update(a["sourceFiles"])
photos.extend(a["photos"])
parent_refs.extend(a["parentRefs"])
if a["genotype"]["rawGenotype"]:
if a["genotype"]["mapped8locus"]:
genos.add(a["genotype"]["rawGenotype"])
geno_keys.add(_geno_key(a["genotype"]))
if a["farbschlag"]:
farb.add(a["farbschlag"])
if a["death"]:
deaths.add(norm_dob(a["death"]))
if a.get("deaf") is not None:
deaf_seen.add(a["deaf"])
tags_set.update(a.get("tags", []))
# pick the richest genotype (most mapped loci, then longest raw)
best = max((a["genotype"] for a in grp),
key=lambda gd: (len(gd["mapped8locus"]), len(gd["rawGenotype"])))
@@ -554,13 +571,16 @@ def dedup(animals):
"photos": sorted(set(photos)),
"sourceFiles": sorted(files),
"mentions": len(grp),
# GEN-3b: hearing/deaf phenotype flag (deaf wins if any mention says so) + tags.
"deaf": (True if True in deaf_seen else (False if False in deaf_seen else None)),
"tags": sorted(tags_set),
# FEAT-8c: machine-readable quarantine marker so the API loader can skip
# conflicting records without parsing the German review report.
"conflict": False,
}
merged.append(out)
# conflict: same animal, disagreeing genotype or farbschlag or death
if len(genos) > 1 or len(farb) > 1 or len(deaths) > 1:
# conflict: same animal, disagreeing NORMALIZED genotype (Uw==G) or farbschlag or death
if len(geno_keys) > 1 or len(farb) > 1 or len(deaths) > 1:
out["conflict"] = True
conflicts.append({
"id": out["id"], "name": base["name"], "dob": out["dob"],