Build Cache

Build Caches store outputs from Gradle tasks and Maven goal executions and reuse them for later builds, rather than building the outputs again.

Every Develocity installation includes a Build Cache service. You can also connect one or more remote nodes, to use as separate caches or as replicas that reduce network latency.

For information on how to use the Develocity Build Cache, consult the Getting Started guides for Gradle or Maven.

Configuration

The Develocity provided Build Cache service is configured with the following defaults:

All cache access is disabled
Target storage size of 50 GiB (how much data to retain in the cache)
Maximum artifact size of 100 MiB (the largest cache artifact to accept)

To change this configuration, click Build Cache in the Administration dashboard, accessible from the upper right of the application.

Build Cache Administration Dashboard

Build Cache Storage

Develocity stores Build Cache artifacts in Object Storage: either the embedded Object Storage that Develocity provides, or external Object Storage that you manage.

See the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for Object Storage configuration details.

Embedded Object Storage

By default, Develocity stores Build Cache artifacts in embedded Object Storage. The Build Cache service in the Develocity server scales horizontally, but the embedded Object Storage it uses by default does not. Gradle strongly recommends connecting Develocity to a highly scalable Object Storage service, such as Amazon S3, Google Cloud Storage, or Microsoft Azure Blob Storage. Cloud-based Object Storage services are fault-tolerant and reduce Develocity operational overhead.

External Object Storage

Host your Develocity installation as close as possible to the external Object Storage service. For example, use AWS for Amazon S3, GCP for Google Cloud Storage, or Azure for Azure Blob Storage. Within that cloud provider, also use the same geographic region.

By default, Build Cache data shares the bucket used for your Build Scan data and other application data. You can instead store Build Cache data in a dedicated bucket, separate from your Build Scan data. A dedicated bucket lets you apply different policies to Build Cache data than to your other data, for operational cost, security, or other policy reasons. For example, the dedicated bucket can have its own retention policy, storage class, access controls, or backup and replication strategy. See the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for details.

Space Management

The amount of space used by the Build Cache service is controlled by the target storage size setting. The right target storage size depends on the number and the size of the cache entries stored. If you increase the Build Cache maximum artifact size, expect to need more space to keep the cache effective.

As new projects are created or existing projects grow, increasing the target storage size could become necessary to maintain cache hit rates.

Embedded Object Storage has a fixed capacity. If you use it, make sure that the Build Cache target storage size is less than the configured embedded Object Storage capacity. See the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for embedded Object Storage configuration details.

Build Cache Storage and Usage Observability

The Usage tab of the Build Cache administration page charts cache usage patterns. It also shows whether the cache is undersized, which shortens the retention period and hurts build performance.

These metrics apply to the logical Build Cache that the Develocity server and Develocity Edges serve. They do not provide insight into Develocity Build Cache nodes.

Least Recently Used Cache Entry Age

For least recently used cache entries, visualizes how long since the entry was last used.

Specifically, two metrics are charted:

The time since last access of the least recently used entry (among all entries) stored in the cache ("Oldest").
The median time since last access of entries stored in the cache ("Median").

Ideally, the "Oldest" value matches the typical lifetime of a feature branch in your organization’s source control.

A Graph Showing the Oldest and Median Time Since Last Access of Cache Entries

Cache Misses for Recently Evicted Entries

The rate of cache misses for recently evicted entries.

If builds often request entries soon after eviction, the trend stays above zero more often than not. That is a sign that the cache is churning.

A Graph Showing the Rate of Cache Misses for Recently Evicted Entries

Storage Size

The current size of the cache and the storage size required to achieve a seven-day retention of cache entries.

Develocity is conservative in recommending increases to the storage size. It recommends an increase when the need persists over a longer period, rather than at a single point in time.

Develocity cannot recommend a decrease in storage size when Build Cache storage is over-provisioned.

A Graph Showing Cache Size and Storage Needed for a Seven-Day Retention Period

Incoming Data Storage Rate

The rate at which data is incoming to the cache.

A Graph Showing the Rate at Which Data Is Incoming to the Cache

Access Control

By default, access to the Build Cache is disabled. There are two access control mechanisms that can be used. Which one you should use depends on the version of the Develocity Gradle plugin or Develocity Maven extension you will be using.

Version 3.11+ of the Develocity Gradle plugin and version 1.15+ of the Develocity Maven extension can authenticate to the Build Cache with Develocity access keys. Permissions for these users are managed via the same access control mechanism used for other functions. Users with the “Build Cache read” permission can read data from any connected Build Cache. Users with the “Build Cache write” permission can read and write data in any connected Build Cache.

To configure your builds to supply credentials, see the Develocity Gradle Plugin User Manual for Gradle or the Develocity Maven Extension User Manual for Maven.

Anonymous access control can also be configured using the same anonymous access control mechanism used for other functions.

Remote Cache Nodes

Remote cache nodes can be installed separately from Develocity. Use them to separate cache artifacts, to distribute load, or to give builds a better network link to the cache.

Connect remote cache nodes to a Develocity instance to configure them centrally and have them replicate entries from other nodes in the network.

The Develocity Edge User Manual covers how to deploy Develocity services like Build Cache closer to where builds are run.

Connecting With Develocity

To connect a remote node to your Develocity installation, first register the node with Develocity in one of these ways:

Manual Registration
Unattended Registration

Manual Registration

Visit /cache-admin of your Develocity installation, and select Nodes from the left menu. In the Remote nodes Create new node section, enter the name for your node and click Create new node.

Remote Build Cache Node Administration Dashboard

The node will now be listed in the Existing nodes section. Each node is assigned a key and a secret. To view it, click the View button in the Secret column for the newly created node.

Example of Viewing a Remote Node Secret

You can view the secret for the first 5 minutes after you create the node. If the node secret is lost, use the regenerate function to issue a new secret, which will then be viewable for 5 minutes.

As a last step you need to update the node configuration with the assigned key and secret.

Unattended Registration

If you manage nodes with provisioning software, such as Terraform, register them unattended through the Develocity API. Consult the access control section of the API manual on how to connect to the Build Cache section of the Develocity API.

Once you have access to the Develocity API, you can register a node via the Develocity API.

The following example uses develocity.example.com as the installation host, 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja as the access key of the user which has the Cache admin role, and cURL as the HTTP client.

First, register a new node in Develocity with the Create/Update endpoint. Give it a unique, descriptive name, in this example node-1:

curl -X PUT https://develocity.example.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

If applicable, a replication configuration (in this example a preemptive replication from parent-node-1 is configured) can be specified at creation time:

curl -X PUT https://develocity.example.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true,"replication":{"source":"parent-node-1","preemptive":true}}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

Second, a request to the secret endpoint is made using the name of the node used in the previous step. This generates a key-secret pair that the node requires for connecting to Develocity:

curl -X POST https://develocity.example.com/api/build-cache/nodes/node-1/secret \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja"

Output

{
  "key":"fqysvmfd3u5dcgis5reunrxjce",
  "secret":"cpgovxdo6f6phkdyxdrfzyy43q5hflatc2umzgguxv6adap2qyyi"
}

As a last step you need to update the node configuration with the returned key and secret. Consult the reference documentation of the Develocity API for further functionality with regard to node management.

Update Node Configuration

The node must also be configured with details of the server to connect to, including the assigned key and secret. Consult the Develocity Registration section of the Build Cache Node User Manual for information on how to do this.