From research papers
to tested ideas.

The question

Research papers often contain methods that could be useful together. Finding a plausible connection is one step; turning it into an experiment with a fair comparison is another.

I wanted to connect those steps in a workflow where an idea could be proposed, challenged, implemented and measured.

What I built

Congruity gathers research literature through OpenAlex, scores its relevance and asks a coding agent to propose combinations of existing methods. It reviews the proposals and tests selected ideas against executable benchmarks.

1. 01Gather papers relevant to a research question
2. 02Propose connections and examine the reasoning
3. 03Generate candidate code for a defined benchmark
4. 04Compare outcomes against a fixed baseline
5. 05Record the evidence and display the results

The Python pipeline includes structured agent outputs, retries, saved stage artifacts and resume support. A static dashboard makes the saved runs easier to inspect.

Keeping the evaluation useful

Candidate generation stays separate from the benchmark, baseline and grading rules. Bounded search uses a training split and reserves a separate holdout check for the end.

Model critiques help prioritize what to test. They do not automatically veto a candidate. The useful output is a measured result, including one that disproves the proposed idea.

Explore the project

The public repository includes the code, selected numerical fixtures, dated findings and an offline dashboard example. That example uses clearly labelled synthetic data and needs no AI account.

This is a research prototype. A win against a configured baseline does not establish a new state-of-the-art result, and the public snapshot does not include every historical run.