A phylogenetic comparative experimental design assistant
phyloSophR turns a plain-language hypothesis into an auditable, preregistered analytical workflow, locked in before any data is touched. It's also an experiment in openness: the training data behind phyloSophR is being built with the phylogenetic comparative biology community, not behind closed doors.
From a hypothesis in plain language to an auditable, preregistered design.
phyloSophR takes a hypothesis stated in natural language, for example "The common ancestor of Australaves should have been a bird of medium size, rather than as large as an ostrich or as small as a New Zealand wren," and walks it through a fixed, versioned pipeline before any data is collected:
- 1
Parse the hypothesis
A fine-tuned language model extracts the entities and relationships in the hypothesis: trait, taxonomic group, process, predicted direction.
- 2
Assemble the workflow
Those entities are compiled into a directed acyclic graph (DAG) of analytical operations: an explicit, ordered, back-traceable structure, never a black box.
- 3
Validate, human-in-the-loop
A logic-checking layer flags structural gaps (like a missing tree-and-trait matching step) and suggests fixes; the researcher decides, never the model.
- 4
Simulate and check power
The proposed design is stress-tested via simulation across sampled parameter conditions before it's ever run on real data.
- 5
Export, pre-registered
The finished, resolved design exports as an annotated R script, a Quarto report, and a JSON-LD workflow, ready for an OSF pre-registration.
Because the design is locked in before results exist, it becomes structurally resistant to p-hacking and HARKing (Hypothesizing After Results are Known). That's the whole point of phyloSophR: making good scientific practice the path of least resistance, without adding to a researcher's workload.
Training data built through community annotation, in the open.
Scientists who work with mined or scraped data know that cleaning it can consume up to 80% of a project's time. Building a gold-standard, domain-specific text corpus faces the same problem, and it shouldn't be done by one person.
What "annotation" means here
To teach a language model to recognize PCM-specific concepts, someone has to read real methods sections and mark up what's in them: which model was fit, on what kind of tree, under which assumptions. Here's a real example: a single sentence, annotated.
"We fitted an OU-model in a Maximum Likelihood tree, a tree which was inferred under GTR+Gamma and 500 bootstrap replicates."
Phase one: a small expert team
The initial corpus, drawn from 500 peer-reviewed articles spanning the five main PCM analysis types, will be double-annotated by a team of three annotators with varying levels of PCM expertise, using Prodigy's active-learning tooling to prioritize the most informative texts and reduce annotation effort over time. Agreement between annotators will be measured with Cohen's Kappa and Krippendorff's Alpha before any annotated data is released.
Phase two: the community, annotating for the community
A team of three won't be enough to reach the scale a useful corpus needs. Beyond the initial dataset, phyloSophR plans to open annotation to the wider PCM community, where the researchers contributing by annotating their own publications, and student volunteers contributing more broadly.
This collaborative approach reinforces the importance of transparency and reproducibility in science, and lets the training corpus grow progressively over time. Done well, annotation becomes work done by the community, for the community.
phyloSophR takes inspiration from diffrent organizations and museums' model, in which volunteers contributing to documentation and museum-label transcription, and gain a community certificates or acknowledgment in dataset documentation for contributors.
Organizations phyloSophR hopes to work with
Several organizations could help this initiative reach the wider evolutionary-biology community. Butthis project could become a potential collaborator of worldwide project such as:
The community-annotation phase is planned, not active yet. The initial expert-annotated corpus doesn't exist yet, and there is no annotation tool or sign-up process to join today. This page exists so that when that phase opens, you'll already understand what you'd be signing up for.