GerbilManager/import_todo.md

TODO: Migration to Unified Database-Ready Format

This is the checklist for migrating the GerbilManager Excel datasets (41 Stammbaum pedigree charts + 1 Wurfchronik litter chronicle) one-by-one into a clean, resolved, relational JSON format (gerbilmanager_resolved.json) that can be loaded directly into the database without on-the-fly reconciliation.

---

🟩 Phase 1: Preparation & Schema Definition

• Define the target unified JSON schema that maps 1-to-1 with the Entity Framework database entities:
  • Contacts (breeders/buyers)
  • ColorVarieties (genetics catalog)
  • Litters (with resolved MotherId and FatherId Guid relations)
  • Gerbils (with resolved LitterId, ColorVarietyId, OriginContactId, ReceiverContactId Guid relations)
  • GerbilPhotos (mapped by Guid relation)
• Extract and verify existing ColorVarieties and Contacts to establish basic lookup dictionaries.

---

🟦 Phase 2: Spreadsheet Extraction & Conversion (One-by-One)

Convert and extract each spreadsheet's records. For each chart, extract the animals, birth dates, death dates, genders (from cell background color), and genotypes.

Litter Chronicle

• Wurfchronik der Kleine Chaoten Teil1.xlsx (Litters data base)

Pedigree Charts (Stammbäume)

• Stammbaum von Akio Kids.xlsx
• Stammbaum von Alberto Kids.xlsx
• Stammbaum von CP-Fuchs, CP-Sa Sp von Unity.xlsx
• Stammbaum von Chesnut.xlsx
• Stammbaum von Danako.xlsx
• Stammbaum von Ella.xlsx
• Stammbaum von Emi.xlsx
• Stammbaum von Fire Kids.xlsx
• Stammbaum von Goldfuchs Sp (Pikachu) Kids.xlsx
• Stammbaum von Hana.xlsx
• Stammbaum von Jeremy.xlsx
• Stammbaum von Jiminy of Black Forest.xlsx
• Stammbaum von Jin.xlsx
• Stammbaum von Kalea.xlsx
• Stammbaum von Kazuya.xlsx
• Stammbaum von Kentucky.xlsx
• Stammbaum von Kohlief, Goldfuchsef Sp von Chrissi.xlsx
• Stammbaum von Marlin of Black Forest.xlsx
• Stammbaum von Martin.xlsx
• Stammbaum von Oscar of Black Forest.xlsx
• Stammbaum von Pinto of Fiomi.xlsx
• Stammbaum von Quied Soldier of Black Forest.xlsx
• Stammbaum von Rainny.xlsx
• Stammbaum von Ren.xlsx
• Stammbaum von South Dakota.xlsx
• Stammbaum von Stella Kids.xlsx
• Stammbaum von Tennessee.xlsx
• Stammbaum von Unique of Wild Dreams.xlsx
• Stammbaum von Unit.xlsx
• Stammbaum von Uriana.xlsx
• Stammbaum von Valentino Firehearts Kids.xlsx
• Stammbaum von Vance.xlsx
• Stammbaum von Vestra von den Schlossmäusen.xlsx
• Stammbaum von Watarus Kids.xlsx
• Stammbaum von Wildfire und Vestras Kids.xlsx
• Stammbaum von Xelina.xlsx
• Stammbaum von Yuki.xlsx
• Stammbaum von Yurikas und Pintos Sohn.xlsx
• Stammbaum von Zac (Vance.Dorie).xlsx
• Stammbaum von Zenon von Elea.xlsx
• Stammbaum von little Heros Sohn of little runners.xlsx

---

🟨 Phase 3: Resolution & Relational Mapping

• Resolve duplicates using the canonical key: normalize(name) + normalize(dob) and Zucht (breeding line).
• Inject resolutions from conflict-decisions.json to automatically resolve the remaining conflict animals (un-quarantining them and applying correct genotypes/dates).
• Parse and map all parent-child relationships positionally from the pedigree charts.
• Map and link all animals to their respective Wurfchronik litters using parentage and DOB.
• Resolve photo attachments to actual extracted media file paths.
• Assign permanent UUIDs (Guids) to all records and replace string names in relationships with the actual UUID foreign keys.

---

🟧 Phase 4: Data Export & Validation

• Generate the final resolved_import.json containing the relational array collections.
• Run automated validation tests on the JSON file (check for circular dependencies, broken foreign keys, invalid genotypes, and null dates).

---

🟥 Phase 5: Loader Implementation & Database Ingestion

• Build a simplified loader utility (C# endpoint or local python script) that takes the resolved JSON and directly runs INSERT/UPDATE transactions.
• Execute the import onto the target PostgreSQL database.
• Run verification tests (verify total counts: ~956 animals, and sample check pedigree trees).