AI skeptics won’t mimic your super user
Create shared rules for AI use.
This is “Effective Delivery” — a newsletter from The Software House about improving software delivery through smarter IT team organization.
It was created by our senior technologists who’ve seen how strategic team management raises delivery performance by 20-40%.
TL;DR
Only 5% of developers become AI power users,
AI skeptics are inherently unable to mimic the best AI users,
Champions can show the goal, but the rest must reach it on their own,
A shared AI framework is like a solid pair of boots for the journey.
Contents
1. Superusers will always be in the minority
2. Two ways of improving AI productivity
3. Tactics for raising the AI performance bar
Hey! Marek’s back.
One of our developers was going through so many tokens that we had to slow him down.
Another, a senior, refused to touch any AI tools for the longest time.
He was convinced he was fast enough without them.
Until a junior, armed with Copilot, matched his output, task for task.
If you are unsatisfied with your overall AI delivery gains and see things like this, you may be tempted to force everyone to work exactly like the top AI users.
It makes sense on paper, but doesn’t work in practice.
Superusers will always be in the minority
I’ve tried to copy one power user’s approach to working with AI for the rest of the team.
It didn’t work.
I’ve been implementing new tools in IT for 20 years, and I noticed that AI adoption follows the same normal (Gaussian) distribution as any other adoption process
There’s a minority of top performers and underachievers, and the rest fall in the middle.
At TSH, power users make up roughly 5% of the organization.
We’ve confirmed that, as we track how many tokens each developer uses and how much output they create with them.
The normal distribution is natural and will not flatten, no matter what you do.
Some will always use AI less earnestly and efficiently than others for various reasons:
Their personality causes them to naturally resist any major change,
Their satisfaction comes from the act of writing code itself, not from the result,
They fear losing their expert edge when AI closes the gap between seniors & juniors,
They lack the soft skills that AI-assisted development increasingly demands.
Two ways of improving AI productivity
When you pressure a developer to mimic the way a completely different person works, two things tend to happen.
They dig in further, or they leave.
Too many personal characteristics influence how developers interact with AI tools.
A better way is to focus on the middle of your Gaussian curve.
There are two broad team-level methods of doing that.
1. Removing AI delivery blockers
Your delivery processes may be limiting all of your developers’ AI performance:
Product owners cannot generate tasks fast enough for developers to code,
Delivery pipelines are not built for the pace AI can enable,
Client acceptance procedures create handoffs that eat up the time savings.
The review of our first 100% AI-driven project addresses these blockers.
2. Raising the AI performance bar
I’m going to focus on the second method, as it is a direct countermeasure to the fruitless attempt to copy the top AI user’s individual style.
It calls for:
encouraging everyone to get inspired by the top performers’ results voluntarily,
identifying certain standards of AI development that everyone should follow.
The second is about finding a minimum of standards that everyone can get behind without feeling forced to work in a way that is uncharacteristic for them.
Aside from following these rules, they are free to code, prompt, and work as they like.
That approach ensures lower turnover and higher satisfaction.
Tactics for raising the AI performance bar
Here’s how you can raise your team’s AI capability.
1. Involve your AI top performers
You do not have to pressure anyone to copy exactly how they work.
A better way is to have the top users show the AI skeptics all the possibilities.
Do a pair programming session:
The champion shows what is possible while the skeptic watches,
The skeptic tries while the champion observes and provides advice,
The skeptic develops their own method that incorporates some of the champion’s tips.
When the skeptic improves by their own initiative, they may become an AI advocate themselves.
One former skeptic gave a lightning talk about his conversion.
That talk resonated far more with the team than any power user showcase.
People could see themselves in his story.
2. Prove that AI works
Whether through power users or leadership, demonstrating AI’s effectiveness to skeptics is worth the effort.
I remember when we proved to some developers that the codebase in one project was not too complex for AI to handle.
On one project, the team was convinced AI could not understand their legacy system.
We asked them to skip the code and have AI generate documentation instead.
They read it and went quiet. The structure, the logic, the edge cases. All of it was there.
3. Start with the best models, then optimize
To drive your point about AI’s potential, give developers access to the most capable models from the start.
A poor first experience on a cheap model convinces a skeptic that the tools do not work.
On one client project, the team lost trust in AI due to inconsistent results.
When I asked which model they were using, it was the cheapest available.
We switched, and the complaints stopped.
4. Create feedback loops
Diesel engines improved by systematically addressing every reported problem.
AI tools work the same way, but only if the feedback actually reaches the people maintaining them.
Run monthly sessions where developers share what works and what needs fixing in the tools.
A complaint is not a feedback loop. The session must produce a specific issue to address.
For us, the sessions often produce feedback regarding our own AI framework, copilot-collections.
Those found valid by the framework team are implemented in the next version.
5. Give leaders real experience first
We have had cases where we asked a manager to promote AI adoption, even though they were skeptical.
When a team lead does not believe the tools work, the team quickly notices.
Leaders do not need to be power users, but they need to have used the tools enough to believe in them.
If they don’t, they should pair-program with top performers as described above.
6. Build shared frameworks that raise the floor
The highest-leverage change is tooling that makes good practices the default.
It should still be flexible enough to let developers feel they can put a personal spin on the outcome.
At TSH, we built copilot-collections with this in mind.
It is an end-to-end AI product development framework that provides every developer with a basic common workflow.
7. Involve skeptics in building the tools
An organization we learned from invited every developer to create their own agents and contribute to their shared toolkit.
It’s less likely to be skeptical about something you helped create.
At TSH, we run a more centralized model due to the sheer scale of our organization, but we still ask developers to suggest improvements and flag what does not work.
8. Accept the holdouts
After all the enablement, some developers will still not come around.
Don’t push them.
Give them tasks that require less AI-assisted delivery, or move them to areas where manual skill still dominates.
What works instead is time.
When everyone around them works faster and delivers more, the holdout tends to reconsider on their own terms.
That is what raising the bar means in practice.
Next time
In the next issue, Jakub Pleszewski will debut to explain how AI is changing the way software projects are estimated.
He will walk you through which aspects of estimation can be automated and which require human intervention.
Until next time 👋
More AI — more problems
Help us pay the bill for bitcoin mining
Published with a smirk by The Software House
Twarda 18, Warsaw, Poland
tsh.io →
 | A guest post by
|