How We Build, Ship, and Support Software with AI Agents

How Brain Co. uses agents to build, review, and test software, and investigate production issues.

Ruhi Reddypalli

Member of Technical Staff

Nov 18, 2026

At Brain Co., we use AI agents throughout the software lifecycle. Engineers can hand off a ticket to an agent, which implements the change and follows up on CI failures and review feedback. If the change meets its repository’s policy, it can be approved automatically. Agents also test application workflows and investigate production issues.

The agents build and test in our dev sandbox, pull context from our engineering tools, and access models through an LLM gateway.

We track how much engineering time these workflows save, both across the team and per engineer. Our estimates cover work delegated to agents, automatic approvals, and CI repairs, counting each saving once based on the work it takes off an engineer’s plate. An hour of agent runtime doesn’t automatically mean an hour saved.

Here’s a closer look at those workflows and the setup behind them:

1. Turn a Slack message or Linear ticket into a PR

Engineers can hand off a well-scoped Linear ticket to a background agent. We route it to the right repository and workspace, where the agent implements the change, runs tests, opens a PR, and responds to feedback.

This workflow can also start in Slack. For example, one of our teams uses a designated emoji reaction to turn a support message into a ticket for the agent to work on. Engineers can also start tasks directly in Murmur.

A useful handoff includes what should change, the context behind it, and how to tell when it’s done. If something is unclear, the agent can ask for clarification. Our sessiongrep post covers how agents recover context from earlier coding sessions.

2. Give agents a dev sandbox, context, and model access

For agents to take on longer tasks, they need an environment where they can run code and check their work. Our dev sandbox provides that: a remote workspace with tools like Claude Code and Codex, where agents can check out repositories, build applications, and test changes. Persistent storage lets work carry over between sessions. Development permissions are kept separate from production access.

Agents also need context beyond the code. A ticket might describe the change, while the reasoning behind it lives in a Slack discussion or an internal document. Through MCP connections, agents can access services such as Linear, Notion, Slack, Granola, and Datadog. Engineers authorize those connections using their existing permissions and can reuse them across workspaces. Our post, Your Agent Isn’t Dumb. It’s Just Lost., explores this in more detail.

Supported tools access models through our LLM gateway, giving us one place to track usage and costs and set spending limits. Our platform internship retrospective covers the data pipelines behind that tracking.

3. Shepherd PRs through CI and review

After a PR opens, our PR Babysitter agent can fix CI failures, resolve straightforward merge conflicts, and help address review feedback. It retains the PR’s context between runs, so it can pick up where it left off when checks finish or new feedback arrives.

Engineers stay in control of the change’s direction. Changes requested in review require the author’s agreement, and every fix goes through CI again. Required reviews and checks still apply before merging.

4. Auto-approve changes within clear limits

Auto-approval lets routine changes move forward without waiting for a human reviewer. We use Macroscope to review PRs against each repository’s policy. When it recommends approval, a separate checker confirms whether the change qualifies for automatic approval.

The checker looks at the size of the change, the files it touches, and the permissions needed to approve it. It reads the CODEOWNERS file and the checking logic from the PR’s base commit. This prevents a PR from making itself eligible by changing the rules used to evaluate it. Each automatic approval is recorded against the commit that was reviewed.

Project owners choose where to enable auto-approval. Security changes, migrations, major deployment changes, and broader architectural work still need human review. Automatically approved PRs must also pass the required CI checks before merging, and production releases still follow their approval process.

Over the past month, one of our repositories recorded roughly 4,000 auto-approvals, with an approval rate of about 52%.

5. Test application workflows

We were still finding application issues during manual bug bashes, so we started using Validation Agent to automate some of that testing.

Teams describe the workflows they want tested and what should happen. The agent works through those scenarios, such as signing in, loading data, and checking saved results, then reports what happened with evidence an engineer can inspect.

We’ve used it to prepare for demos and test some of our products. Early runs caught issues such as intermittent loading failures and errors opening PDFs. Both were reproducible, which gave engineers something concrete to fix.

Each report makes clear what was tested, what passed or failed, and what still needs checking. If something fails and then works on a retry, both results stay in the report so intermittent problems don’t disappear behind a successful run.

6. Investigate production issues

Our use of agents continues after deployment. Smokey, our incident investigation agent, helps engineers figure out what’s going wrong when a service starts failing.

An engineer can start an investigation from Slack. Smokey looks through logs, recent deployments, and relevant PRs to work out when the problem began and which requests are affected. It compares failed and successful requests and examines code changes to narrow down possible causes.

Smokey reports its findings with supporting evidence, suggested next steps, and questions that still need answering. It has read-only access, so engineers decide what action to take. The person responding to an incident starts with that context instead of gathering it from scratch.

We’re building toward a workflow where engineers can hand off a task and have it will keep moving through implementation, review, testing, and follow-up. That depends on the environment, useful context, and clear limits on what agents can do on their own. As we expand these workflows, we’re measuring how much time engineering time they give back and keeping each one easy to inspect and steer.

If you want to work on systems like these, we’re hiring: https://brain.co/talent#open-positions

Join UsSee open roles

All posts