feat(migration): implement offline-resolved database-ready ingestion, fix circular FKs, duplicate contacts and parser scanning bugs

import_todo.md

@@ -0,0 +1,88 @@
+# TODO: Migration to Unified Database-Ready Format
+
+This is the checklist for migrating the GerbilManager Excel datasets (41 Stammbaum pedigree charts + 1 Wurfchronik litter chronicle) one-by-one into a clean, resolved, relational JSON format (`gerbilmanager_resolved.json`) that can be loaded directly into the database without on-the-fly reconciliation.
+
+---
+
+## 🟩 Phase 1: Preparation & Schema Definition
+- [x] Define the target unified JSON schema that maps 1-to-1 with the Entity Framework database entities:
+  - `Contacts` (breeders/buyers)
+  - `ColorVarieties` (genetics catalog)
+  - `Litters` (with resolved `MotherId` and `FatherId` Guid relations)
+  - `Gerbils` (with resolved `LitterId`, `ColorVarietyId`, `OriginContactId`, `ReceiverContactId` Guid relations)
+  - `GerbilPhotos` (mapped by Guid relation)
+- [x] Extract and verify existing `ColorVarieties` and `Contacts` to establish basic lookup dictionaries.
+
+---
+
+## 🟦 Phase 2: Spreadsheet Extraction & Conversion (One-by-One)
+Convert and extract each spreadsheet's records. For each chart, extract the animals, birth dates, death dates, genders (from cell background color), and genotypes.
+
+### Litter Chronicle
+- [x] `Wurfchronik der Kleine Chaoten Teil1.xlsx` (Litters data base)
+
+### Pedigree Charts (Stammbäume)
+- [x] `Stammbaum von Akio Kids.xlsx`
+- [x] `Stammbaum von Alberto Kids.xlsx`
+- [x] `Stammbaum von CP-Fuchs, CP-Sa Sp von Unity.xlsx`
+- [x] `Stammbaum von Chesnut.xlsx`
+- [x] `Stammbaum von Danako.xlsx`
+- [x] `Stammbaum von Ella.xlsx`
+- [x] `Stammbaum von Emi.xlsx`
+- [x] `Stammbaum von Fire Kids.xlsx`
+- [x] `Stammbaum von Goldfuchs Sp (Pikachu) Kids.xlsx`
+- [x] `Stammbaum von Hana.xlsx`
+- [x] `Stammbaum von Jeremy.xlsx`
+- [x] `Stammbaum von Jiminy of Black Forest.xlsx`
+- [x] `Stammbaum von Jin.xlsx`
+- [x] `Stammbaum von Kalea.xlsx`
+- [x] `Stammbaum von Kazuya.xlsx`
+- [x] `Stammbaum von Kentucky.xlsx`
+- [x] `Stammbaum von Kohlief, Goldfuchsef Sp von Chrissi.xlsx`
+- [x] `Stammbaum von Marlin of Black Forest.xlsx`
+- [x] `Stammbaum von Martin.xlsx`
+- [x] `Stammbaum von Oscar of Black Forest.xlsx`
+- [x] `Stammbaum von Pinto of Fiomi.xlsx`
+- [x] `Stammbaum von Quied Soldier of Black Forest.xlsx`
+- [x] `Stammbaum von Rainny.xlsx`
+- [x] `Stammbaum von Ren.xlsx`
+- [x] `Stammbaum von South Dakota.xlsx`
+- [x] `Stammbaum von Stella Kids.xlsx`
+- [x] `Stammbaum von Tennessee.xlsx`
+- [x] `Stammbaum von Unique of Wild Dreams.xlsx`
+- [x] `Stammbaum von Unit.xlsx`
+- [x] `Stammbaum von Uriana.xlsx`
+- [x] `Stammbaum von Valentino Firehearts Kids.xlsx`
+- [x] `Stammbaum von Vance.xlsx`
+- [x] `Stammbaum von Vestra von den Schlossmäusen.xlsx`
+- [x] `Stammbaum von Watarus Kids.xlsx`
+- [x] `Stammbaum von Wildfire und Vestras Kids.xlsx`
+- [x] `Stammbaum von Xelina.xlsx`
+- [x] `Stammbaum von Yuki.xlsx`
+- [x] `Stammbaum von Yurikas und Pintos Sohn.xlsx`
+- [x] `Stammbaum von Zac (Vance.Dorie).xlsx`
+- [x] `Stammbaum von Zenon von Elea.xlsx`
+- [x] `Stammbaum von little Heros Sohn of little runners.xlsx`
+
+---
+
+## 🟨 Phase 3: Resolution & Relational Mapping
+- [x] Resolve duplicates using the canonical key: `normalize(name) + normalize(dob)` and `Zucht` (breeding line).
+- [x] Inject resolutions from `conflict-decisions.json` to automatically resolve the remaining conflict animals (un-quarantining them and applying correct genotypes/dates).
+- [x] Parse and map all parent-child relationships positionally from the pedigree charts.
+- [x] Map and link all animals to their respective Wurfchronik litters using parentage and DOB.
+- [x] Resolve photo attachments to actual extracted media file paths.
+- [x] Assign permanent UUIDs (Guids) to all records and replace string names in relationships with the actual UUID foreign keys.
+
+---
+
+## 🟧 Phase 4: Data Export & Validation
+- [x] Generate the final `resolved_import.json` containing the relational array collections.
+- [x] Run automated validation tests on the JSON file (check for circular dependencies, broken foreign keys, invalid genotypes, and null dates).
+
+---
+
+## 🟥 Phase 5: Loader Implementation & Database Ingestion
+- [x] Build a simplified loader utility (C# endpoint or local python script) that takes the resolved JSON and directly runs `INSERT`/`UPDATE` transactions.
+- [x] Execute the import onto the target PostgreSQL database.
+- [x] Run verification tests (verify total counts: ~956 animals, and sample check pedigree trees).