Develocity Failure Classification Guide

Develocity’s failures dashboard classifies every build failure as either non-verification or verification. The split lets you measure disruptive failures, such as build configuration or infrastructure failures, separately from failures in the software being built. Compilation and test failures are the second kind.

How Failure Classification Works

Develocity classifies each failure with a model that reads the exception message, the stack trace, and identifying build information. The model was trained on thousands of build failures, so some of its classifications will not match what you expect.

What to Do to Improve Invalid Classifications

The classification model recognizes a set of definitive error message prefixes. A prefix overrides every other heuristic and forces the classification. Develocity analyzes the exception chain from the bottom up: the root exception first, then each encapsulating exception in turn.

Where you control the exception messages your build logic throws, use one of the prefixes below to force a classification.

Verification Failures

This classification covers failures that are expected within a standard application development lifecycle. They typically represent a problem with the developer’s inputs to the build such as the source code.

To force classification as a verification failure, prefix the exception message with any of the following phrases (case-insensitive).

  • Analysis failed

  • Check failed

  • Compilation failed

  • Code generation failed

  • Generation failed

  • Lint failed

  • Processing failed

  • Test failed

  • Tests failed

  • Testing failed

  • Verification failed

For example, the following exception stack would normally be classified as non-verification:

[org.gradle.api.tasks.TaskExecutionException] Execution failed for task ':example-project:build'.
└── [com.example.custom.tasks.ClassDocGenerationException] Could not load the class documentation for class 'com.example.MyClass'.
    └── [java.lang.RuntimeException] Expected 2 td elements in tr, found none

The RuntimeException is thrown by the custom Gradle task logic. Change the exception message to Generation failed: Expected 2 td elements in tr, found none, and Develocity classifies the failure as verification.

Gradle

Gradle users can also extend org.gradle.api.tasks.VerificationException.

Non-Verification Failures

This classification covers failures that are typically not expected within a standard application development lifecycle. Build configuration failures, dependency resolution failures, and infrastructure failures are the usual examples.

To force classification as a non-verification failure, prefix the exception message with any of the following phrases (case-insensitive).

  • Unexpected error

  • Unexpected failure

For example, the following exception stack would normally be classified as verification:

[org.gradle.api.tasks.TaskExecutionException] Execution failed for task ':example-project:setupMockServer'.
└── [org.gradle.api.GradleException] Mock server failed to bootstrap correctly

The GradleException is thrown by a custom Gradle task. Change the exception message to Unexpected error - Mock server failed to bootstrap correctly, and Develocity classifies the failure as non-verification.

Third Party Failures

Develocity does not support customizing classification for third-party failures. If you cannot change the failure messages your build logic emits, the built-in model decides the classification. If the classifier gets it wrong, please contact Develocity support so we can improve it in a future release.

If you have any questions or need any assistance contact the Develocity support team or your customer success representative.