One person, a data factory, and the number that lied to me

TrueSET is not a team with a mission statement. It's me, a rented GPU, and a rule I learned the hard way: never trust a number you can't re-run.

How this started (by accident)

I didn't set out to start a company. Starting on July 11, 2026, in the space of about ten days, I went from writing plugins for Fusion 360 to accidentally running a solo data startup — without ever learning how you're supposed to do this.

The chain went like this: the plugins weren't good enough, so I tried building standalone graphics inside Fusion 360. Not good enough either. So I went all in and built a fully standalone CAD program — which actually worked for a while, until I ran into my own ugly UI/UX. My plan to fix it: build a dataset and fine-tune a model to be good at exactly that kind of repair work.

Somewhere in building that dataset, the dataset became the company. The CAD program is still sitting there with its ugly UI. What follows is the part I did on purpose.

The number that started it

When I started building training data for code repair, I did what everyone does: I generated examples and had a big model grade them. The scores were great. Then I fine-tuned a model and my evaluation said +42.5% improvement. For about a day, I thought I had something special.

I did. Just not the thing I thought. When I graded by actually running the code, the improvement collapsed: the model had learned to sound like my answers — same structure, same phrasing — without getting better at fixing anything. Style-mimicry, not skill. The loss curve applauded. The judge scores applauded. Execution didn't.

That day decided what this company is.

The rebuild

I tore down what I'd built on that number and started over with one non-negotiable rule: a claim only counts if the measurement can't be gamed. In practice, that means:

• Execution decides correctness. Every example must carry a proof you can run: the broken program fails a hidden checker, the fixed program passes the same checker. Both verified before the example exists. No AI judge anywhere in the correctness path.
• The eval is decided before the experiment. Held-out set frozen first, statistical test written down first. Whatever the result is afterwards, that's the result.
• Checkers get attacked before they're trusted. I try to fool every hard-task checker with wrong-but-plausible code. If a bad fix can sneak past, the task is cut — and once, that meant rebuilding an entire tier from scratch.
• Numbers are floors. I publish the conservative end with its confidence interval, and the misses print right next to the hits — the tiers that didn't move, the runs that came out worse, the models that beat mine. If you check my claims, you should find they were understated.

Where it stands today

The factory has produced 924 verified issue/fix pairs so far. Fine-tuning a 7B model on them doubled its fix-rate on hard held-out faults at the median — I trained the same recipe four times and published every run, spread and all. Five of those hard faults are ones GPT-4o and Claude Sonnet fail and my tuned 7B fixes — transcripts published, including the table where the frontier models win overall.

Everything ships with its receipt: the eval reruns on any GPU pod for under $0.50, the provenance is Apache-2.0-clean (authored by openai/gpt-oss-120b, no scraped code), and the one thing I keep private is the factory itself — the proofs are the product.

Why "TrueSET"

Because the whole point is a dataset where "true" is a property you can verify, not a promise you have to take. If that's the kind of work you want to exist, write me or fund the next experiment — every result lands on the blog either way.