AI agents in CI as a review gate, not as a code generator

Two different roles for the same tool

When people talk about agents in development today, they almost always mean writing code. The agent gets a task, writes a function, runs the tests and opens a pull request. It is impressive and on small, well-bounded tasks it works.

The problem is what happens when it gets something wrong. Generated code looks finished. It is formatted correctly, it has comments, it passed the linter. Whether it does what it should only shows once somebody reads it line by line, or at runtime.

The second role is duller and therefore safer. The agent writes nothing. It gets the diff that is about to land on the main branch and writes down what it considers risky about it. That reverses the roles: the human is no longer checking the machine, the machine is flagging things to the human.

Why this one can be checked

In our view this is the main reason, and it is the one talked about least.

When an agent writes "this commit changes the return type of a function called from five other places, and none of them changed", that is a claim about specific lines. Verifying it takes seconds: either it is so or it is not. When the agent is wrong, you see it immediately and it cost nothing.

When an agent writes a function, no such verification exists. You have to read the whole thing and work out what it does, which is exactly the work it was meant to save.

A check whose false alarms you spot in ten seconds is worth running even if it is wrong one time in three. A generator whose errors you cannot spot is not.

What such a gate watches well

The things that work best are hard to watch by eye and easy to describe in words. Of what we have gradually handed over, these have held up:

• Dependency drift. A new version of a library that reached the branch as a side effect of other work. A person skips it in the diff, because it is one line in a lock file.
• A change of public behaviour with no test. A function called from several places was edited and no test was added. It is not always wrong, but it is always worth a question.
• A schema migration with no way back. A dropped column, a renamed table, a changed type. Here the cost of missing it is highest and the cost of checking is lowest.
• Secrets in a diff. A token, key or password that made it into a commit by accident. Ordinary tools exist for this too, and it is sensible to have both.
• A broken layer boundary. An import from somewhere it should not come from. Static rules cover this until the first exception nobody wrote down.

What they share is that the answer is short and you can point a finger at it.

What we know about this and what we do not

Here it pays to be honest, because a great deal is written about agents in development in 2026 and very little is measured.

The advice to start with a review gate rather than with generation is consistent across sources. The argument is always the same: less resistance to adoption, less risk, easier to evaluate. It sounds reasonable and it matches our experience.

But there are no measured results for it. We did not find a study comparing a team with a review gate against a team without one on comparable projects, measuring how much review time dropped or how many defects were caught earlier. The figures in circulation mostly come from companies selling the relevant tool, and they do not publish the methodology.

So: it is a reasonable bet backed by practice, not a documented fact. If somebody sells it to you with percentages, ask them for the methodology.

How we have it ourselves

We did not build anything new for this. It turned out we had been using the same principle elsewhere in our systems for years.

In ClientDeck, our campaign control system, every intervention in a client account goes through an approval queue. Automation proposes, a person decides, and every change keeps a name, a reason and a review date. That is exactly the role this article is about, only in a domain other than code.

In Indepanel, our own hosting panel, a deployment has checks that must pass before a version reaches people, and an automatic rollback when they do not. No agent decides anything there; a rule does. An agent is useful one step earlier, over the diff, where the rule cannot be written.

And in the content hub we use to manage the group's sites, the check is an ordinary script: it verifies links, title lengths and whether every article has all its language versions. There is nothing clever about it, which is exactly why it can be trusted. An agent only makes sense where the rule cannot be written in advance.

That gives the order we would recommend: first write the rules that can be written, and only put an agent on what is left. The reverse order leads to an agent reporting things a linter should have caught.

Where it does not work

There is no point hiding where we ran into trouble.

• On large diffs. The bigger the change, the worse the agent is at telling what matters. A gate pushes towards smaller commits, which is good in itself, but it is a change of habit, not of configuration.
• When there are too many findings. A check that writes ten notes on every commit stops being read within a week. Better to start with a single thing it watches and add slowly.
• As a blocking step. An agent that cannot fail the build can be introduced in an afternoon. An agent that can needs trust it has to earn first.
• On unfamiliar code without context. Without domain knowledge an agent reports plenty of things that are deliberate in that project.

What to take from this

If you are starting with agents in development, start with what can be checked. One check over the diff, non-blocking, with one clear subject. In a month you will see whether it flags things you would otherwise have missed, or just noise. That is an evaluation you can do yourselves and it needs nobody else's percentages.

This does not write off code generation. It just makes more sense later, once you know where the agent helps you and where it makes things up, because nobody ships that knowledge with a tool.

We build and run our own systems, so none of this is theoretical for us. The rest of the group is at independency.cz.