Develocity Failure Handler

The Develocity Failure Handler is an agent skill that diagnoses a single build failure, including a flaky test, using Develocity’s cross-build data. It tells you whose failure it is and whether a retry goes green. When you ask, it also fixes the failure in a pull request, opens an issue for the owning team, or records what it found.

The skill answers questions a working tree cannot: whether a failure is flaky, pre-existing, or caused by your own change. It collects the failed build’s data from Develocity, and your AI agent diagnoses the failure from a summary of that data. When you ask it to deal with the failure, it opens a pull request with the fix or an issue for the team that owns an operational cause, or records what it found. Your AI agent writes any fix, following your repository’s conventions.

When to Use It

Reach for the Failure Handler when you hold a failure and need its cause before you act on it. Common cases:

  • A failure on a build you did not run: A CI job fails on another branch and you have the log but not the workspace. The skill diagnoses the failure from Develocity data alone. Fixing it needs the checkout, so here you get the diagnosis to act on.

  • A failure you cannot yet attribute: You do not know whether your own change caused it. The skill compares the failing builds with the rest of the project, separating a pre-existing or systemic failure from one you introduced.

  • A flaky test: You need to know why a test flakes, not only whether it does. The skill reads the test’s flake rate across builds, what sets its failing builds apart from the rest of the project (such as an operating system or a CI job), and whether it still flakes after later commits.

  • A failure blocking a pull request, merge, or release: You need a citable diagnosis to decide whether the failure is yours to fix or belongs elsewhere.

The Failure Handler is built to be reached mid-task, the moment a failure surfaces. You need no Build Scan ID: finding it is where the run starts.

How It Works

The skill takes one failure at a time and progresses through three phases:

  1. Summarize: Collects the failed build’s data from the Develocity API, from seven days before the build to seven days after it. This covers the build and its failures, the failure groups those failures belong to, later builds of the same job, a sample of the project’s other builds, and the history of the failing tests. It writes the data to files and prints a summary.

  2. Triage: Takes four readings from the summary, each with its evidence:

    • Flaky: whether the failing test has flaky outcomes in the window.

    • Locality: whether the failure is local or systemic. A local failure is limited to one person, machine, or branch. A systemic one is specific to something wider, such as an operating system, a toolchain version, or a CI job.

    • Fix target: whether a change to the repository removes the cause (code-fixable) or the cause lies outside the repository (operational), such as an outage, an expired credential, or an artifact that was never published.

    • Recurrence: what happened in later builds of the same job. The reading is persistent, not resolved when the job passed again on the same commit, or fixed since when it passed on a later commit.

  3. Handle: When you ask the skill to deal with the failure, it takes one of three routes:

    • Fix: For a code-fixable cause with a checkout to commit from, it fixes the failure, verifies the fix, and opens a pull request. When it cannot verify the fix, it still opens the pull request and says why.

    • Route: For an operational cause, it opens an issue in the repository that owns the failure, following that repository’s issue templates, and posts a one-line notice where the owning team will see it.

    • Record: When it can do neither, such as when there is no checkout to fix from or a later commit has already fixed the failure, it writes what it found to a file.

When the fix target is undetermined, the skill looks further before it picks a route. It reads more of the failed build’s Develocity data, such as its test output and console log, and the source code where there is a checkout. When neither shows the cause, it records what it found.

Prerequisites

The Failure Handler reads Develocity data over the Develocity API and through the Develocity MCP Server, so confirm the following before you run it:

  • A running Develocity instance. If the instance requires authentication, you also need an access key whose user has the Access build data via the API and MCP permission. The skill reads the key from the DEVELOCITY_ACCESS_KEY environment variable or your build tool’s Develocity keys file. See Permissions and the Develocity API user manual.

  • The Develocity MCP Server enabled on your Develocity instance and connected to your AI agent. See the MCP Server installation manual.

  • The Develocity skills installed in your AI agent. See Getting Started.

  • Bash and Python 3.9 or later on the machine where your AI agent runs.

  • A published Build Scan for the failing build. The skill starts from that scan and stops when the build published none.

Triage also draws on two more sources of Develocity data. When one is missing, the skill keeps running, states what it could not collect, and reports the readings that depend on that source as undetermined:

  • Cross-build failure grouping supplies the failure groups that show where else the failure occurred. Without it, the skill cannot tell a local failure from a systemic one. See the Failure Classification Guide.

  • Test Analytics history supplies the failing test’s outcomes across builds, which the flaky and recurrence readings use.

The skill also relies on the Git branch and CI job custom values, recorded by the Common Custom User Data plugins. It uses them to find later builds of the same job, and to compare the failing builds' branch with the rest of the project. Without them, the skill treats any later build of the project as the same job, or, for a local build, any later build of yours. A failure limited to one branch can then look like one that is everywhere.

Run the Skill

You can invoke the Failure Handler by name, or let your AI agent start it when your request matches, such as asking why a test flakes or whether a failure is yours. Point it at the failure in plain language, with a Build Scan link, a Build Scan ID, the build’s console output, or a CI build link:

/develocity-failure-handler Why is this test failing on CI? https://ci.example.com/build/1234

The skill handles the failure only as far as you ask:

  • When the failure interrupts other work and you only need the answer, the skill diagnoses the failure and hands back. It changes nothing, and its reply says what handling the failure would produce.

  • When dealing with the failure is the task, the skill diagnoses it and then handles it: a fix in a pull request, an issue for an operational cause, or a written record.

A fix prompt pasted from the Develocity Failure Report is the second case: the report’s prompt asks for the failure to be fixed, so the run goes on to handle it.

To keep the skill at the diagnosis even when you asked it to fix the failure, tell it so:

/develocity-failure-handler Diagnose only: is this failure related to my change? https://develocity.example.com/s/abc123

The skill handles one failure per run. When a build carries several failures, it takes the one your request names. If your request names none, it picks one, usually the build’s first, and lists the others. Run the skill again for each remaining failure you want handled.

Run Behind a Proxy

On a network that reaches Develocity only through an HTTP or HTTPS proxy, the skill routes its Develocity API reads through it. Name the proxy in your prompt:

/develocity-failure-handler Diagnose this failure using the proxy http://proxy.example.com:8080. https://develocity.example.com/s/abc123

Or set the standard HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables before you start your AI agent. This suits unattended and CI runs, and it is the only way to give the skill a proxy that needs a username or password:

export HTTPS_PROXY=http://proxy.example.com:8080
export NO_PROXY=localhost,127.0.0.1

The skill takes a proxy only from your prompt or these environment variables, never from another tool’s configuration. The Develocity MCP Server reaches Develocity over its own connection, which you configure with the server rather than through the skill.

Results

Every run that diagnoses a failure ends on a short reply of three to five plain lines:

  • The failure the skill took, and any others it did not take.

  • Whose failure it is, with the facts that decided it. It is yours when the failure is code-fixable, not flaky, and local, so the fix belongs in the change you are making. When the data cannot settle the locality or the fix target, it is undetermined. Otherwise it is not yours.

  • Whether a retry goes green: yes for a flaky test, may pass when the job passed again on the same commit, no for a persistent failure, no need when a later commit already fixed it, or unknown.

  • The link to the pull request or issue, or the path of the record. When the run handled nothing, it gives a reproduction command instead.

  • What the skill did, or would do, such as opening a pull request against a named repository.

Several outcomes are correct results rather than failed runs. A reply that the failure is flaky and a retry should pass, or that a later commit already fixed it, means there is nothing to change.

The pull request, issue, or record carries the full findings in four sections:

  • Failure: the failing test or task, its message, when it was first seen, and a link to the failing Build Scan.

  • Diagnosis: the four readings (flaky, locality, fix target, and recurrence), each with its evidence.

  • Reproduction: the commit and the command to reproduce the failure, how faithfully it reproduces, the toolchain and operating system that matter, and how many runs it takes. A flaky failure takes more runs than a deterministic one.

  • Outcome: for a pull request, the reproduction before and after the change, or why it could not run; for an issue, what the owning team needs to restore; and for a record, why the failure was not fixed or routed.

Where your repository has its own pull request or issue templates, the findings go inside them. A record goes to fh-records/<failure>.md in the parent directory of your repository’s working tree.

Scope

The skill diagnoses one failure and, when you ask, coordinates handling it. It does not:

  • write the fix from its own expertise. Your AI agent authors the fix, following the repository’s conventions, and the skill supplies the diagnosis the fix has to address,

  • decide who reviews or merges a pull request. Those choices come from your repository, or

  • diagnose more than one failure per run.

When the cause is operational, such as an infrastructure incident or an expired credential, no repository change resolves it. Instead of proposing a fix, the skill opens an issue for the team that owns the cause. When the build does not show which repository owns the failure, or where to notify its team, the skill asks you.

Considerations

  • Diagnosis needs only Develocity data, so the skill runs against a failure on a build you did not run. Fixing a code-fixable failure is different: opening a pull request needs a checkout the skill can commit from, so run it where the affected project is checked out. Without one, the skill records the diagnosis instead.

  • Builds the skill runs to reproduce a failure or verify a fix carry the FAILURE-HANDLER tag and the failureHandler.purpose and failureHandler.failingBuild custom values. The skill leaves builds with that tag out of the data it reads, so its own builds never count as occurrences of a failure.

  • The skill stores the data it collects in a session folder under your system’s temporary directory. A session expires after 24 hours, and the skill deletes expired sessions and their data the next time it starts a run. Set the DEVELOCITY_AGENTS_DIR environment variable to an absolute path to keep the sessions elsewhere.

  • The skill uses your AI agent’s model, so each run consumes model tokens. Set budget and turn limits in your agent to bound a run.

  • Develocity is required throughout. The skill diagnoses from Develocity data and has no offline mode.