This is the part most "AI + history" projects skip past. Here's exactly what happens between a scanned page and a claim on the site — including where the data is messy and what we do about it.
We work from full-resolution scans of the primary source — every page, not an excerpt someone else selected. Text is recovered with Tesseract OCR, run per volume, per page, so any downstream claim can be traced back to an actual scanned image, not a secondary transcription somebody typed up decades ago.
The raw OCR text is passed through structured extraction into five linked tables: entities (people, deities, places, motifs, rituals — whatever the source actually contains), relationships between them, the evidence backing each relationship (a direct quote and page range), competing scholarly interpretations, and open questions the text itself doesn't resolve.
Free-text extraction is messy by nature — across hundreds of passes, the same kind of figure gets typed a dozen slightly different ways ("Deity", "Deity/Hero", "Deity/Mythological being"...). We don't quietly clean that up and hide the mess. The graph buckets entity types into a controlled set for browsing and coloring, while keeping the original free-text label attached to every node — so the messiness of real extraction stays inspectable instead of getting smoothed into false precision.
Two independent checks run against every case. First, the graph is checked against existing published scholarship — for Veda Slovena, over 150 years of it, across Bulgarian, Russian, Serbian, Macedonian, Czech, German and English sources — with each interpretation tagged by who proposed it and how strong the evidence is. Second, where the case allows it, we run original quantitative analysis that plain reading can't do: for Veda Slovena, a Burrows' Delta stylometric comparison across the 80 most frequent word-forms in 25 text segments — a check almost never run on contested folklore texts, because it requires code, not just close reading.