Review: Our first 100% AI-driven project
We had to change our SLDC.
This is “Effective Delivery” — a bi-weekly newsletter from The Software House about improving software delivery through smarter IT team organization.
It was created by our senior technologists who’ve seen how strategic team management raises delivery performance by 20-40%.
TL;DR
We used our AI framework to develop the entire code in a project for the first time,
Development speed was great, but backlog and code review produced bottlenecks,
It turns out you can’t fully automate coding without speeding up the rest of the SDLC,
In an AI SDLC like this, all humans, not just engineers, focus on directing and verifying.
Contents
1. Our first 100% AI-driven project
2. Can you develop too quickly?
4. My view on AI-driven development now
Heyo! I’m back!
My company has long stopped thinking about AI as just a coding assistant.
We’ve understood that the real productivity gain comes when you let AI drive the work and have engineers focus on directing and verifying AI.
This is a philosophy behind our own AI-driven development framework, copilot-collections (link opens in a new tab).
We’ve already tested it in +50 commercial projects.
However, we only recently used it to cover 100% of the development in a single project.
This project, though ultimately successful, revealed delivery bottlenecks that informed the further development of our AI framework.
Our first 100% AI-driven project
The client chose a no-code approach and used Lovable to build a web app prototype for managing products and conducting research in its industry.
The goal was to turn it into maintainable software with a view to offering it as a SaaS.
Given the project’s somewhat limited budget, we proposed covering 100% of development with our framework for structured AI development.
It would lower the cost and provide an opportunity for an all-AI approach.
3 factors made this project a good fit:
It was a monorepo, giving the AI agent all the software context in one place,
It was mostly greenfield (after the Lovable cleanup), allowing AI to skip legacy burden,
The size was right, complex enough to be a test, not too large for a failure to sting,
We estimated it would take 3 developers several months to do it the traditional way.
We planned to deliver it with 1 developer at the same time, with no loss of quality.
The development would be supported by a BA and a designer, for a full team of 3 people, not counting a developer on the code review duty.
Can you develop too quickly?
The project used copilot-collections’ standard 4-phase workflow of Research, Plan, Implement, and Review.
The developer completed end-to-end features in 30 to 40 minutes for standard tasks and around 3 hours for complex ones.
Several pull requests landed every day from a single developer.
As a result, the team ran into 3 bottlenecks in quick succession:
1. A dry backlog
The developer completed tasks faster than the BA could generate new ones.
What’s more, AI-driven development produced code far closer to the original acceptance criteria, leading QA to find fewer bugs and empty the backlog ever faster.
2. Slow code review
While QA was fast, pull requests covering complete features required far more review time than those for partial ones.
3. Frequent merge conflicts
Before the client-side developer adopted copilot-collections, they used a different set of AI tools and processes, which led to branch conflicts during PR merges.
It was like a factory where the production line sped up, but the ability to supply the raw materials and move the finished product didn’t.
Soon, development was idle, waiting for new tasks, while the review work piled up.
Speeding up the whole SDLC
The idea of speeding up backlog generation and code review wasn’t new to us.
After all, copilot-collections always had a review agent and a separate frontend reviewer to verify UI implementations against Figma designs.
But when we chose to use AI for 100% of the coding in a somewhat large project, it turned out not to be enough.
We needed to speed up the extreme ends of the SDLC more.
Backlog generation
First, we added a new Business Analyst agent to copilot-collections to generate a full backlog of tasks from workshop notes and stakeholder materials.
This has resulted in 3 major changes to the backlog generation process:
The agent, rather than a human, now produces task descriptions with acceptance criteria and flags unclear or missing requirements,
The human BA’s role shifted from writing tasks to verifying AI output and facilitating stakeholder conversations,
Generating a backlog of 600 tasks with this agent took around 3 hours, compared to 2 weeks or more by hand,
Quality & review
Our framework’s Implement phase ends with a Code Reviewer agent check before the developer opens a pull request.
The IT team introduced new layers of automated quality control on top of the standard Code Reviewer agent that fires following the Implement phase:
SonarQube now runs alongside every check with a strict configuration designed to detect bugs, vulnerabilities, and code smells,
A dedicated AI job now performs a weekly Code Quality Review.
We’ve set it up to look for dead code or duplicate logic of the kind that SonarQube often misses,An agent automatically fixes everything the weekly job finds,
The weekly code quality job flagged multiple dead code instances that SonarQube had not caught.
An agent resolved all issues in a single session, keeping the codebase clean without adding manual work to developers’ schedules.
The merge phase is the least automated step in the current process.
When a developer opens a pull request, a second developer reviews the code, and a QA specialist tests that branch before it is merged.
Luckily, following the strict, mostly automated review process, there are usually few issues left for manual work.
After these changes to the SDLC, sprint velocity stabilized.
How my view on AI-driven development shifted
AI development is still a young field, and each experience alters my perspective on it.
This time, I’ve learned enough to call it a major shift in my understanding of AI-driven development.
True AI-driven development is coming
I’ve long wanted an AI to be the primary coder rather than a coding assistant, with a human acting as the director and ultimate verifier.
But it turned out that true AI-driven development was not sustainable when most of the SDLC was still manual.
BA’s role is evolving
A human BA should use their soft and analytical skills to communicate with stakeholders and produce quality primary materials that inform AI agents.
In this new role, they can be a creative force rather than an inputter of manual tasks.
Vibe-coding has its place
Apps built with tools like Lovable are justly criticized for subpar code, but they are great for quickly producing PoCs.
I can see full-on AI-driven development like ours becoming the natural next step in transforming those vibe-coded apps into production-ready software.
Next time
I built an AI version of myself called 🇵🇱 Polak AI, which my teammates consultwhen I don’t have time to reply (which happens often!).
Full breakdown in Effective Delivery #21
Thanks for reading today 🫡
We’re legally obligated to show you this
Published with a smirk by The Software House
Twarda 18, Warsaw, Poland
tsh.io →
 | A guest post by
|
Hi Adam!
Great post, thanks for sharing.
My question to this topic is: and how much money did it cost to run all the agents, environments, auto-configurations, etc.?
AI-driven development replaces the work of a programmer with the work of an agent. Nowadays agents are not free. Are you somehow able to compare (percentage maybe?), how much did you save (in costs)? The time aspect seems obvious, but I am curious if the costs of tokens are bearable.
Really interesting read, love the experimental nature, PoC type approach to the work. I wonder to what extent small batch sizes coupled with smaller stories would serve to reduce the review bottleneck. I wonder if speeding up the feedback loop on smaller pieces could create a predictable cycle (ie. stories small enough to code in 30 mins, code review allotted 30 mins). Thanks for sharing your experience.