GerbilManager

A manager application for gerbil breeding data featuring a .NET Aspire backend and a React-based frontend.

• GerbilManagerWebAPI/ — ASP.NET Core backend (Swagger under /swagger)
• gerbil-manager-web/ — React Frontend (Vite + TypeScript), see gerbil-manager-web/README.md
• tools/import/ — Python import pipeline scripts and database schemas

Requirements

To manage database migrations, install the EF Core CLI tool globally:

dotnet tool install --global dotnet-ef

Running the Application

To run the full stack (PostgreSQL database, WebAPI, and Frontend Vite app) in one command:

dotnet run --project GerbilManager.AppHost

---

📥 Data Import & Normalization Pipeline

The application features a Python-based import pipeline that compiles gerbil breeding records, lineage, photos, and contact information from source files (markdown logs, Excel spreadsheets, Word documents) into a unified JSON format, and ingests them into the PostgreSQL database.

1. Structure of the Pipeline

• Source Data: Located at C:\Users\gulum\dev\Wurfchronik_Bilder (contains markdown files, .xlsx pedigree trees, and detail logs).
• Compilation: merge_and_resolve.py processes the source files, extracts breeding details, resolves lineage links, maps colors/genotypes, and normalizes contact names.
• Normalization Mapping: merge_and_resolve.py contains a norm_map dictionary of contact name normalizations. This maps various spelling variations and abbreviations to a single canonical name.

2. How to Run the Import

Automatic Way (All-in-One)

The PowerShell script in tools/import boots the Aspire stack, waits for the API to become ready, and automatically triggers the ingest:

powershell -ExecutionPolicy Bypass -File tools/import/run_app_and_import.ps1

Manual Way

1. Recompile the import data: Run the compiler script from the root directory to parse raw sources and output a fresh resolved_import.json:

   python tools/import/merge_and_resolve.py
   

2. Ingest into the running database: Ensure the application is running, then send an HTTP POST request to the API:

   Invoke-RestMethod -Uri "http://localhost:5179/import/ingest-resolved" -Method Post
   

---

👥 Contact Deduplication Workflow

When the database is wiped and re-imported, you can detect and resolve duplicate contacts (e.g. spelling mistakes, initials vs full names) using the deduplication checklist workflow.

1. Search for Duplicate Contacts

You can run duplicate checks using two different similarity parameter models:

• Round 1 (Clever matching on first 4 characters):

  python tools/import/find_duplicates_v5.py
  

  Creates: tools/import/output/potential_duplicates_decision.md
• Round 2 (Broader matching on initials, matching last names, phones, or emails):

  python tools/import/find_duplicates_round2.py
  

  Creates: tools/import/output/potential_duplicates_decision_round2.md

2. Fill out the Checklist

Open the generated markdown checklist file and edit the checkboxes:

• Combine/Merge: Put an x in the checkbox [x] of the name you want to choose as the canonical name. Leave the checkboxes empty [ ] for any names you want to merge into the canonical name.
• Standalone: Put an o in the checkbox [o] of any name that should remain a separate, standalone contact.
• New Name: Put an x next to Custom name: and type your new desired name inside the backticks.
• Keep all separate: Leave all checkboxes in a Choice empty [ ].

Example:

- [x] Max Mustermann   <-- canonical name
- [ ] Max M.           <-- merges into Max Mustermann
- [o] Moritz Mustermann <-- remains a separate standalone contact

3. Apply the Decisions

Once you have saved your decisions in the checklist file, run the parser:

python tools/import/parse_decisions.py

This automatically parses your selections, resolves the merges, and updates the norm_map inside merge_and_resolve.py so they are permanently saved in the codebase. Run the compilation pipeline again to produce the final deduplicated import JSON:

python tools/import/merge_and_resolve.py