In September, I asked Peter Riegler, a product manager at Soldd, for a testimonial. For six months, we worked on his team's AI setup together. I called it a graduation from the initial AI transformation.
I sent him questions, not a draft quote. I wanted to know what changed, in his words. His answers came back with four numbers he cleared for public use.
The AI setup became something you can version.
"Six months ago “our AI setup” was three Cursor licences and one quick entry in the retro," he wrote. "Now it’s a versioned pipeline with specs, plans and the agents’ own rules as artifacts in the repo. When an agent goes off the rails we quickly identify the problem and fix the configuration."
The last sentence is the shift. When the setup lives in the repo, an agent that does the wrong thing is a configuration problem with a fix. When the setup lives in people's heads, it is only a feeling.
Measurement has to stick before the agents scale.
I asked what would have happened if his team had run this transition on its own. "We’d have built twice as many agents, in the wrong order, and measured probably none of them."
AI amplifies your baseline. If you are slow, AI gives you the illusion that you are going faster, while you get slower in the bigger picture. You can have a really powerful car with a bad map, and go super fast into the middle of nowhere. The way to know which one you have is to put some data next to the feelings.
Here is where his team landed. Median PR cycle time went from 125 hours in May to 1.3 hours in September. High- and highest-severity escaped defects went from 11 in April to 2 in September. Deploy frequency rose 74% over the latest three months, against the three before. Velocity rose 51%, like-for-like.
Order matters more than speed.
His counterfactual is about order, not effort. Structured specs, test coverage and architecture encoding come first. Auto-merge comes later.
This is the pipeline now, in his words: "From a Jira ticket, we now produce a spec, a plan, tasks and an estimate not by hand but by 95% agentic support. Gates check the plan against the spec on the way in and the branch against its acceptance criteria on the way out. Low-risk changes merge without a human; a release agent ships to production."
Auto-merge arrived after the specs and the gates that make it safe to trust.
He named what he valued: "A course gives you a framework. We got someone running the same experiments the same week, who’d already hit the walls we were walking toward." And: "most valuable to me is her deep technical understanding, and the judgement to tell us which of our ideas not to build."
Review moves up to the product.
"Spec-to-implementation is now frequently a one-shot agent run," he wrote, "and review has become more a product review than a code review."
When the implementation runs in an autonomous loop, review on the spec becomes much more important. After the code exists, the human question is "did this have the intended outcome? Did the users appreciate it?" Engineers need to accept that we are becoming product people.
Most of it was about people.
His answer to "what surprised you?" started with the discipline: "Agendas framed as questions a day ahead, 80-minute sessions that actually covered what they set out to, written summaries with owners and dates the same day." Then: "Most surprising to me personally was actually how much of the work turned out to be about people rather than pipelines."
There is the technology. There is also the mindset of the people, the trust of the people, and their craft, which is changing. The pipeline is the part you can see.
My goal with clients is to raise their capabilities so they can do better without me. The numbers above belong to the Soldd team. So does the pipeline.
The return on the investment is the learning and the trust the team built.