What does the AI agent running this blog actually do unattended?

It picks the day's topics from a queue, verifies factual claims against source documents before writing them, drafts up to three posts with diagrams, checks that every link resolves, and commits the result. It does all of that without a human in the loop for the routine case.

Does a human review every post before it goes live?

Every commit lands on a branch that only reaches a staging site automatically. Publishing to production is a separate, manually triggered step a human runs after reading the staged version, so nothing this agent writes reaches a real reader without that click.

Can this agent publish something false without anyone catching it?

It has, which is the uncomfortable part of publishing this case study honestly. A validator meant to catch banned claims sat dead for months before anyone wired it into the build, and turning it on surfaced errors that had already shipped. The fix was making the gate actually run, not trusting that a draft looked clean.

Who is accountable when the agent gets a published claim wrong?

The operator who reviews and merges the pipeline's changes, not the agent and not the model vendor. Every correction in this post was made by a person who read the diff, decided it was wrong, and fixed it. The agent drafts; a human still owns what ships.

Can someone steer this agent into publishing something it shouldn't through a queued topic or a comment?

A malformed or misleading entry in the topic queue can absolutely misdirect a single day's draft, and it has: a status field left in the wrong state once caused twelve posts to sit unpublishable for weeks with nobody noticing. The mitigation is that the queue itself is reviewed by a person on a schedule, not left to run itself indefinitely.

What happens when the agent's own validation gate turns out to be broken?

The errors it should have caught ship anyway, silently, until someone checks by hand or a different gate catches the same problem downstream. That is exactly what happened here, and the response was to verify the gate itself against real output rather than assume a passing build meant a clean post.

Where does this content pipeline still fall short?

It cannot judge whether a topic is worth writing about in the first place, decide that a page is underperforming and should be retired, or push anything to production. Those three decisions are reserved for a person on a fixed schedule, on purpose, because none of them are the kind of mistake you want to discover after the fact.

An AI Agent Content Pipeline Case Study: Six Months In

This is an AI agent content pipeline case study, told plainly: for the past several weeks, an agent has picked the day's topics, verified the claims it planned to make, drafted up to three blog posts with diagrams and cover art, checked every link, and committed the result, without a person touching any of it until after the commit exists. This post is the honest account of what that setup actually does unattended, where it has been wrong, and what changed because it was wrong.

The short version: the agent is trustworthy for drafting and dangerous if the gates around it are trusted blindly, because a gate that isn't actually verified to run is functionally the same as no gate at all. That is the single failure mode behind nearly everything below.

TL;DR

This agent researches, verifies, drafts, diagrams, and link-checks up to three posts a day, then commits. It cannot push past a staging environment; a human clicks a separate, manual trigger before anything reaches production. Its biggest real incident wasn't a wild hallucination, it was a validation gate that had silently never run, which let stale and false claims accumulate until the gate was actually wired in and finally caught them. What changed as a result: the gate now runs for real, a measurement pass rewrote how its FAQ objections work, and three specific decisions, what to write about, what to retire, and what reaches production, stay with a person on a fixed schedule.

This AI Agent Content Pipeline Case Study: What Runs Unattended, Every Day

On a normal day the loop looks the same regardless of topic: pick the next eligible entries from a topic queue, check that no existing post already targets the same query, verify any factual claim against a real source rather than repeating what an earlier post said, write the post and its diagrams, confirm every internal and outbound link actually resolves, and commit. No person is in that loop for the routine case, and for most days that has been the right call: three posts is more daily output than a small team can hand-review start to finish, and the drafting, source-checking, and link-validation steps are exactly the pattern-rich, structured work an agent is good at.

The diagram below shows the full path from a queued topic to a published post, including where the human-only gates actually sit.

The Gates It Cannot Pass

Two gates sit outside the agent's authority entirely, on purpose. The commit it creates lands on a branch that reaches a staging environment automatically within minutes, but production is a second, manually triggered deploy that a person runs after reading the staged version. Nothing this pipeline writes reaches a real reader without that separate click, no matter how confident the commit message sounds.

A build step sits between those two: it checks frontmatter, verifies markdown structure, and, since a recent fix, scans every post for a list of specific claims the team has already gotten wrong once and does not want repeated. If that check fails, the run's drafts get thrown away rather than committed. That gate is the subject of the most useful incident this pipeline has produced.

The Day the Real Gate Finally Ran

The banned-claims check had existed in the codebase for a while before anyone discovered it had never actually executed in either the pipeline's build or the wider site's, dead code nobody had wired into a real build step. The day it was finally switched on, it flagged eleven matches across posts the team had believed were clean.

Seven were real. Six of the seven were the same claim in different words: a description of admin-configurable AI guardrails that do not exist in the product, repeated across posts that all cited the one page that had already been corrected, which is exactly how a false claim outruns a single fix. The other four were false positives, correct sentences that happened to match an intentionally broad detection pattern, most of them a sentence explicitly denying the false claim rather than making it. Those got a machine-readable marker explaining why, not a rewrite, because rewriting a sentence that was already correct would have made it worse.

The lesson that stuck: a validation gate is worth exactly nothing until you've confirmed it actually runs against the thing it's supposed to protect. A green build had been read as "no false claims" when it actually meant "the check that would catch false claims never executed." Measuring whether an agent is actually helping covers the general version of this mistake: trusting the metric that looks good instead of the one that's actually connected to the outcome you care about.

What Got Fixed Because the Agent Got It Wrong

The banned-claims incident wasn't the only correction this pipeline has needed. A single reader report that the free plan's project limit was wrong triggered a sweep that found the same handful of numbers, seat limits, storage caps, entry pricing, had been copied from post to post without anyone re-checking them against the current source of truth, wrong in roughly thirty places by the time anyone looked. Separately, a set of competitor claims about a Microsoft product's feature set had gone stale as the competitor shipped the exact capabilities the posts said it lacked, and those claims sat published for months before a fact-check caught and corrected them across seven files. And one drafted post described a product capability, a customer-configurable AI setting, that the product has never actually shipped; it was withdrawn before publication rather than corrected after.

None of these were the agent inventing something from nothing. Every one was a claim that had been true once, copied forward past the point where it stopped being checked. AI project management failure modes covers why this specific pattern, confident output built on stale rather than fabricated input, is one of the harder ones to catch, because it doesn't look like an error until someone checks the source again.

What We Still Will Not Let It Decide

Three decisions stay with a person, on a fixed schedule, regardless of how well the routine drafting has gone.

What reaches production. Every commit stops at staging. A human reads the staged result and triggers the separate production deploy by hand, every time, with no autopilot path around that click.

What gets retired. A page that stops earning traffic doesn't get deleted, merged, or redirected by the pipeline on its own initiative. It gets flagged in the pipeline's own output for a person to decide, because retiring the wrong page is a much more expensive mistake than leaving a weak one alone for another month.

What gets queued at all. The topic list the agent draws from is curated and re-ranked by a person against real search and citation data on a recurring cadence, not generated by the agent deciding what's worth writing about. A queue entry once sat in the wrong status field for weeks, silently blocking an entire quarter's worth of planned content, and the fix required a person reading the file by hand, not another automated check.

That last incident is the honest summary of the whole arrangement: the agent is reliable at the parts of this job that are pattern-rich and checkable, drafting, verifying a claim against a source, confirming a link resolves, and it is exactly as reliable as its gates are, which is why the gates, not the drafting, get the most scrutiny. Running a project autonomously with an AI agent covers the same verify-before-done discipline applied to project work instead of content; AI governance for PMOs covers the act, suggest, and stay-out boundary that decided which of this pipeline's three decisions stayed off-limits in the first place. More on how we think about running agents on real work is on the Onplana blog.