Develocity Failure Report

The Develocity Failure Report is an agent skill that surveys a Develocity instance and reports the recurring build failures and flaky tests worth fixing across your organization, for Gradle and Maven.

The skill reads live data across every project on the server, not your source code, and makes two sweeps, each with its own ranked report. The failures sweep ranks the recurring build and test failures by how many builds they affect. The flaky sweep ranks flaky tests by the CI time they waste, since a test retried until it passes rarely fails a build but still costs CI time. Each row carries a paste-ready fix prompt. It does no root-cause analysis and applies no fix. You read the report, decide which failures are worth the effort, and hand each one to the Develocity Failure Handler, which diagnoses it and, when asked, fixes it.

When to Use It

The Failure Report tells you where the recurring cost is across an instance, so you can focus fix effort on the failures that waste the most time. Common cases:

  • Prioritizing failure cleanup: You want the shortlist of failures worth fixing across the organization, ranked by how many builds each affects, rather than a per-project hunt.

  • A recurring review: Run the report on a schedule, such as per sprint or per release, to catch the failures and flaky tests that are accumulating cost.

  • Feeding the Failure Handler: You want a queue of concrete, ranked failures to diagnose and fix, each with a prompt you can paste into a project checkout.

How It Works

The skill surveys the whole instance. The failures sweep progresses through five phases:

  1. Checking Access: Confirms it can read the Develocity data each selected sweep needs.

  2. Finding Integration Branches: Resolves which branches are integration branches, so the ranking measures failures on the branches that matter.

  3. Ranking Failures: Reads and ranks the window’s failure groups by the number of builds each affects.

  4. Confirming Top Failures: Works down the ranking, verifying that each group’s members represent a single failure and clustering related failures, until it reaches the number of failures the run targets (10 by default).

  5. Writing the Report: Writes an HTML report for each sweep, with a paste-ready fix prompt per failure.

By default the skill also makes a flaky sweep, which ranks flaky tests at the test-class level by the CI time they waste and writes a separate report. A flaky-only run has its own four phases, replacing Ranking Failures and Confirming Top Failures with a single Ranking Flaky Tests. You can run either sweep alone.

Prerequisites

The Failure Report reads Develocity data through a connected Develocity MCP Server. Confirm the following before you run it:

  • A running Develocity instance with the Develocity MCP Server enabled, which can run the Develocity Analytics queries. See the MCP Server installation manual.

  • An access key whose user has the Access build data via the API and MCP permission, which the MCP Server requires on every request. The failures sweep additionally runs Develocity Analytics queries that require cross-project read access, so any run that includes it reads across every project. A flaky-only run needs no cross-project access. All are read permissions, so a read-only key is enough. See Permissions and the Develocity API user manual.

  • The Develocity skills installed in your AI agent. See Getting Started.

  • The Git branch custom value, recorded by the Common Custom User Data plugins. The skill resolves the branch environment before it ranks, and stops when no branch custom value is present.

  • The Develocity capability each selected sweep depends on:

    • The failures sweep needs cross-build failure grouping. A failures-only run stops when the instance does not expose it. A both-sweep run continues with the flaky sweep alone. See the Failure Classification Guide.

    • The flaky sweep needs Test Analytics.

The access key above is what the MCP Server authenticates with. The skill can also read an access key from the DEVELOCITY_ACCESS_KEY environment variable or your build tool’s Develocity keys file, and use it to read the failure groups directly over the Develocity API. This is recommended but optional: the direct read reaches far more failure groups than the MCP Server alone, and without it the survey falls back to the shallower MCP read and ranks far fewer failures, which the report records.

Run the Skill

The Failure Report reads live data across every project on the server, so it runs only when you start it by name. Start it with no arguments to survey both sweeps over the default window, the 7 full days ending yesterday (UTC):

/develocity-failure-report

The skill states what it will do and waits for you to confirm before it surveys anything. You can name any of three things: which sweeps to run, another time window, or extra branches to treat as integration branches.

Survey only flaky tests over a shorter window:

/develocity-failure-report flaky tests only, over the last 3 days

Survey only recurring failures, and treat a release line as an integration branch:

/develocity-failure-report failures only, and treat release/* as an integration branch

Run Behind a Proxy

On a network that reaches Develocity only through an HTTP or HTTPS proxy, the skill routes its own Develocity API reads through it. Name the proxy in your prompt, or set the standard HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables before you start your AI agent:

export HTTPS_PROXY=http://proxy.example.com:8080
export NO_PROXY=localhost,127.0.0.1

The skill takes a proxy only from your prompt or these environment variables, and masks any credentials in its output. The Develocity Analytics queries run through the Develocity MCP Server, which reaches Develocity over its own connection that you configure with the server.

Results

The skill writes an HTML report for each sweep it ran, in your working directory, and ends the run on it, after printing a ranked summary of each.

The failures report ranks the recurring failures worth fixing. Each row carries the failure and its classification (code-fixable or operational), how many builds it affects in the window, when it was last seen, a Build Scan link as evidence, and a paste-ready fix prompt. You open your AI agent in the affected project’s checkout and paste the prompt, which starts the Failure Handler on that failure.

The flaky report ranks the flaky tests worth fixing by the CI time they waste. Each row carries the test class and its flaky cases, how much CI time they lost and on which integration branches, and a paste-ready fix prompt.

A run over a healthy window reports few or no failures. That is a result, not an error.

Scope

The Failure Report surveys and ranks. It does not:

  • diagnose a root cause or apply a fix. It hands each failure to the Failure Handler, which you run yourself in the affected project,

  • survey anything but Gradle and Maven builds, or

  • set up recurring monitoring. Each run is a one-time survey of the window you give it.

The survey covers every project on the server, except a flaky-only run, which is limited to the projects your access key can read.

Considerations

  • A run reads every project on the server and typically takes 10 to 20 minutes.

  • The skill uses your AI agent’s model, so each run consumes model tokens. Set budget and turn limits in your agent to bound a run.

  • Without an access key for its direct API reads, the survey falls back to a much shallower MCP read and ranks far fewer failures. The report states when it did.