Verifying dependencies
- Enabling dependency verification
- Verifying dependency checksums
- Verifying dependency signatures
- Understanding signature verification
- Forcing use of local keyrings only
- Disabling verification or making it lenient
- Disabling dependency verification for some configurations only
- Trusting some particular artifacts
- Trusting multiple checksums for an artifact
- Skipping Javadocs and sources
- Adding keys manually to the keyring
- Dealing with a verification failure
- Cleaning up the verification file
- Refreshing missing keys
Working with external dependencies and plugins published on third-party repositories puts your build at risk. In particular, you need to be aware of what binaries are brought in transitively and if they are legit. To mitigate the security risks and avoid integrating compromised dependencies in your project, Gradle supports dependency verification.
Dependency verification is, by nature, an inconvenient feature to use. It means that whenever you’re going to update a dependency, builds are likely to fail. It means that merging branches are going to be harder because each branch can have different dependencies. It means that you will be tempted to switch it off.
So why should you bother?
Dependency verification is about trust in what you get and what you ship.
Without dependency verification it’s easy for an attacker to compromise your supply chain. There are many real world examples of tools compromised by adding a malicious dependency. Dependency verification is meant to protect yourself from those attacks, by forcing you to ensure that the artifacts you include in your build are the ones that you expect. It is not meant, however, to prevent you from including vulnerable dependencies.
Finding the right balance between security and convenience is hard but Gradle will try to let you choose the "right level" for you.
Dependency verification consists of two different and complementary operations:
-
checksum verification, which allows asserting the integrity of a dependency
-
signature verification, which allows asserting the provenance of a dependency
Gradle supports both checksum and signature verification out of the box but performs no dependency verification by default. This section will guide you into configuring dependency verification properly for your needs.
This feature can be used for:
-
detecting compromised dependencies
-
detecting compromised plugins
-
detecting tampered dependencies in the local dependency caches
Enabling dependency verification
The verification metadata file
Currently the only source of dependency verification metadata is this XML configuration file. Future versions of Gradle may include other sources (for example via external services). |
Dependency verification is automatically enabled once the configuration file for dependency verification is discovered.
This configuration file is located at $PROJECT_ROOT/gradle/verification-metadata.xml
.
This file minimally consists of the following:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>false</verify-signatures>
</configuration>
</verification-metadata>
Doing so, Gradle will verify all artifacts using checksums, but will not verify signatures. Gradle will verify any artifact downloaded using its dependency management engine, which includes, but is not limited to:
-
artifact files (e.g jar files, zips, …) used during a build
-
metadata artifacts (POM files, Ivy descriptors, Gradle Module Metadata)
-
plugins (both project and settings plugins)
-
artifacts resolved using the advanced dependency resolution APIs
Gradle will not verify changing dependencies (in particular SNAPSHOT
dependencies) nor locally produced artifacts (typically jars produced during the build itself) as by nature their checksums and signatures would always change.
With such a minimal configuration file, a project using any external dependency or plugin would immediately start failing because it doesn’t contain any checksum to verify.
Scope of the dependency verification
A dependency verification configuration is global: a single file is used to configure verification of the whole build.
In particular, the same file is used for both the (sub)projects and buildSrc
.
If an included build is used:
-
the configuration file of the current build is used for verification
-
so if the included build itself uses verification, its configuration is ignored in favor of the current one
-
which means that including a build works similarly to upgrading a dependency: it may require you to update your current verification metadata
An easy way to get started is therefore to generate the minimal configuration for an existing build.
Configuring the console output
By default, if dependency verification fails, Gradle will generate a small summary about the verification failure as well as an HTML report containing the full information about the failures.
If your environment prevents you from reading this HTML report file (for example if you run a build on CI and that it’s not easy to fetch the remote artifacts), Gradle provides a way to opt-in a verbose console report.
For this, you need to add this Gradle property to your gradle.properties
file:
org.gradle.dependency.verification.console=verbose
Bootstrapping dependency verification
It’s worth mentioning that while Gradle can generate a dependency verification file for you, you should always check whatever Gradle generated for you because your build may already contain compromised dependencies without you knowing about it. Please refer to the appropriate checksum verification or signature verification section for more information.
If you plan on using signature verification, please also read the corresponding section of the docs.
Bootstrapping can either be used to create a file from the beginning, or also to update an existing file with new information. Therefore, it’s recommended to always use the same parameters once you started bootstrapping.
The dependency verification file can be generated with the following CLI instructions:
gradle --write-verification-metadata sha256 help
The write-verification-metadata
flag requires the list of checksums that you want to generate or pgp
for signatures.
Executing this command line will cause Gradle to:
-
resolve all resolvable configurations, which includes:
-
configurations from the root project
-
configurations from all subprojects
-
configurations from
buildSrc
-
included builds configurations
-
configurations used by plugins
-
-
download all artifacts discovered during resolution
-
compute the requested checksums and possibly verify signatures depending on what you asked
-
At the end of the build, generate the configuration file which will contain the inferred verification metadata
As a consequence, the verification-metadata.xml
file will be used in subsequent builds to verify dependencies.
There are dependencies that Gradle cannot discover this way.
In particular, you will notice that the CLI above uses the help
task.
If you don’t specify any task, Gradle will automatically run the default task and generate a configuration file at the end of the build too.
The difference is that Gradle may discover more dependencies and artifacts depending on the tasks you execute. As a matter of fact, Gradle cannot automatically discover detached configurations, which are basically dependency graphs resolved as an internal implementation detail of the execution of a task: they are not, in particular, declared as an input of the task because they effectively depend on the configuration of the task at execution time.
A good way to start is just to use the simplest task, help
, which will discover as much as possible, and if subsequent builds fail with a verification error, you can re-execute generation with the appropriate tasks to "discover" more dependencies.
Gradle won’t verify either checksums or signatures of plugins which use their own HTTP clients. Only plugins which use the infrastructure provided by Gradle for performing requests will see their requests verified.
Using generation for incremental updates
The verification file generated by Gradle has a strict ordering for all its content. It also uses the information from the existing state to limit changes to the strict minimum.
This means that generation is actually a convenient tool for updating a verification file:
-
Checksum entries generated by Gradle will have a clear
origin
that starts with "Generated by Gradle", which is a good indicator that an entry needs to be reviewed, -
Entries added by hand will immediately be accounted for, and appear at the right location after writing the file,
-
The header comments of the file will be preserved, i.e. comments before the root XML node. This allows you to have a license header or instructions on which tasks and which parameters to use for generating that file.
With the above benefits, it is really easy to account for new dependencies or dependency versions by simply generating the file again and reviewing the changes.
Using dry mode
By default, bootstrapping is incremental, which means that if you run it multiple times, information is added to the file and in particular you can rely on your VCS to check the diffs. There are situations where you would just want to see what the generated verification metadata file would look like without actually changing the existing one or overwriting it.
For this purpose, you can just add --dry-run
:
gradle --write-verification-metadata sha256 help --dry-run
Then instead of generating the verification-metadata.xml
file, a new file will be generated, called verification-metadata.dryrun.xml
.
Because --dry-run doesn’t execute tasks, this would be much faster, but it will miss any resolution happening at task execution time.
|
Disabling metadata verification
By default, Gradle will not only verify artifacts (jars, …) but also the metadata associated with those artifacts (typically POM files).
Verifying this ensures the maximum level of security: metadata files typically tell what transitive dependencies will be included, so a compromised metadata file may cause the introduction of undesired dependencies in the graph.
However, because all artifacts are verified, such artifacts would in general easily be discovered by you, because they would cause a checksum verification failure (checksums would be missing from verification metadata).
Because metadata verification can significantly increase the size of your configuration file, you may therefore want to disable verification of metadata.
If you understand the risks of doing so, set the <verify-metadata>
flag to false
in the configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>false</verify-metadata>
<verify-signatures>false</verify-signatures>
</configuration>
<!-- the rest of this file doesn't need to declare anything about metadata files -->
</verification-metadata>
Verifying dependency checksums
Checksum verification allows you to ensure the integrity of an artifact. This is the simplest thing that Gradle can do for you to make sure that the artifacts you use are un-tampered.
Gradle supports MD5, SHA1, SHA-256 and SHA-512 checksums. However, only SHA-256 and SHA-512 checksums are considered secure nowadays.
Adding the checksum for an artifact
External components are identified by GAV coordinates, then each of the artifacts by their file names. To declare the checksums of an artifact, you need to add the corresponding section in the verification metadata file. For example, to declare the checksum for Apache PDFBox. The GAV coordinates are:
-
group
org.apache.pdfbox
-
name
pdfbox
-
version
2.0.17
Using this dependency will trigger the download of 2 different files:
-
pdfbox-2.0.17.jar
which is the main artifact -
pdfbox-2.0.17.pom
which is the metadata file associated with this artifact
As a consequence, you need to declare the checksums for both of them (unless you disabled metadata verification):
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>false</verify-signatures>
</configuration>
<components>
<component group="org.apache.pdfbox" name="pdfbox" version="2.0.17">
<artifact name="pdfbox-2.0.17.jar">
<sha512 value="7e11e54a21c395d461e59552e88b0de0ebaf1bf9d9bcacadf17b240d9bbc29bf6beb8e36896c186fe405d287f5d517b02c89381aa0fcc5e0aa5814e44f0ab331" origin="PDFBox Official site (https://pdfbox.apache.org/download.cgi)"/>
</artifact>
<artifact name="pdfbox-2.0.17.pom">
<sha512 value="82de436b38faf6121d8d2e71dda06e79296fc0f7bc7aba0766728c8d306fd1b0684b5379c18808ca724bf91707277eba81eb4fe19518e99e8f2a56459b79742f" origin="Generated by Gradle"/>
</artifact>
</component>
</components>
</verification-metadata>
Where to get checksums from?
In general, checksums are published alongside artifacts on public repositories. However, if a dependency is compromised in a repository, it’s likely its checksum will be too, so it’s a good practice to get the checksum from a different place, usually the website of the library itself.
In fact, it’s a good security practice to publish the checksums of artifacts on a different server than the server where the artifacts themselves are hosted: it’s harder to compromise a library both on the repository and the official website.
In the example above, the checksum was published on the website for the JAR, but not the POM file. This is why it’s usually easier to let Gradle generate the checksums and verify by reviewing the generated file carefully.
In this example, not only could we check that the checksum was correct, but we could also find it on the official website, which is why we changed the value of the of origin
attribute on the sha512
element from Generated by Gradle
to PDFBox Official site
.
Changing the origin
gives users a sense of how trustworthy your build it.
Interestingly, using pdfbox
will require much more than those 2 artifacts, because it will also bring in transitive dependencies.
If the dependency verification file only included the checksums for the main artifacts you used, the build would fail with an error like this one:
Execution failed for task ':compileJava'. > Dependency verification failed for configuration ':compileClasspath': - On artifact commons-logging-1.2.jar (commons-logging:commons-logging:1.2) in repository 'MavenRepo': checksum is missing from verification metadata. - On artifact commons-logging-1.2.pom (commons-logging:commons-logging:1.2) in repository 'MavenRepo': checksum is missing from verification metadata.
What this indicates is that your build requires commons-logging
when executing compileJava
, however the verification file doesn’t contain enough information for Gradle to verify the integrity of the dependencies, meaning you need to add the required information to the verification metadata file.
See troubleshooting dependency verification for more insights on what to do in this situation.
What checksums are verified?
If a dependency verification metadata file declares more than one checksum for a dependency, Gradle will verify all of them and fail if any of them fails.
For example, the following configuration would check both the md5
and sha1
checksums:
<component group="org.apache.pdfbox" name="pdfbox" version="2.0.17">
<artifact name="pdfbox-2.0.17.jar">
<md5 value="c713a8e252d0add65e9282b151adf6b4" origin="official site"/>
<sha1 value="b5c8dff799bd967c70ccae75e6972327ae640d35" origin="official site" reason="Additional check for this artifact"/>
</artifact>
</component>
There are multiple reasons why you’d like to do so:
-
an official site doesn’t publish secure checksums (SHA-256, SHA-512) but publishes multiple insecure ones (MD5, SHA1). While it’s easy to fake a MD5 checksum and hard but possible to fake a SHA1 checksum, it’s harder to fake both of them for the same artifact.
-
you might want to add generated checksums to the list above
-
when updating dependency verification file with more secure checksums, you don’t want to accidentally erase checksums
Verifying dependency signatures
In addition to checksums, Gradle supports verification of signatures. Signatures are used to assess the provenance of a dependency (it tells who signed the artifacts, which usually corresponds to who produced it).
As enabling signature verification usually means a higher level of security, you might want to replace checksum verification with signature verification.
Signatures can also be used to assess the integrity of a dependency similarly to checksums. Signatures are signatures of the hash of artifacts, not artifacts themselves. This means that if the signature is done on an unsafe hash (even SHA1), then you’re not correctly assessing the integrity of a file. For this reason, if you care about both, you need to add both signatures and checksums to your verification metadata. |
However:
-
Gradle only supports verification of signatures published on remote repositories as ASCII-armored PGP files
-
Not all artifacts are published with signatures
-
A good signature doesn’t mean that the signatory was legit
As a consequence, signature verification will often be used alongside checksum verification.
It’s very common to find artifacts which are signed with an expired key. This is not a problem for verification: key expiry is mostly used to avoid signing with a stolen key. If an artifact was signed before expiry, it’s still valid.
Enabling signature verification
Because verifying signatures is more expensive (both I/O and CPU wise) and harder to check manually, it’s not enabled by default.
Enabling it requires you to change the configuration option in the verification-metadata.xml
file:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-signatures>true</verify-signatures>
</configuration>
</verification-metadata>
Understanding signature verification
Once signature verification is enabled, for each artifact, Gradle will:
-
try to download the corresponding
.asc
file -
if it’s present
-
automatically download the keys required to perform verification of the signature
-
verify the artifact using the downloaded public keys
-
if signature verification passes, perform additional requested checksum verification
-
-
if it’s absent, fallback to checksum verification
That is to say that Gradle’s verification mechanism is much stronger if signature verification is enabled than just with checksum verification. In particular:
-
if an artifact is signed with multiple keys, all of them must pass validation or the build will fail
-
if an artifact passes verification, any additional checksum configured for the artifact will also be checked
However, it’s not because an artifact passes signature verification that you can trust it: you need to trust the keys.
In practice, it means you need to list the keys that you trust for each artifact, which is done by adding a pgp
entry instead of a sha1
for example:
<component group="com.github.javaparser" name="javaparser-core" version="3.6.11">
<artifact name="javaparser-core-3.6.11.jar">
<pgp value="8756c4f765c9ac3cb6b85d62379ce192d401ab61"/>
</artifact>
</component>
For the At the time, V4 key fingerprints are of 160-bit (40 characters) length. We accept longer keys to be future-proof in case a longer key fingerprint is introduced. In |
This effectively means that you trust com.github.javaparser:javaparser-core:3.6.11
if it’s signed with the key 8756c4f765c9ac3cb6b85d62379ce192d401ab61
.
Without this, the build would fail with this error:
> Dependency verification failed for configuration ':compileClasspath': - On artifact javaparser-core-3.6.11.jar (com.github.javaparser:javaparser-core:3.6.11) in repository 'MavenRepo': Artifact was signed with key '8756c4f765c9ac3cb6b85d62379ce192d401ab61' (Bintray (by JFrog) <****>) and passed verification but the key isn't in your trusted keys list.
The key IDs that Gradle shows in error messages are the key IDs found in the signature file it tries to verify. It doesn’t mean that it’s necessarily the keys that you should trust. In particular, if the signature is correct but done by a malicious entity, Gradle wouldn’t tell you. |
Trusting keys globally
Signature verification has the advantage that it can make the configuration of dependency verification easier by not having to explicitly list all artifacts like for checksum verification only. In fact, it’s common that the same key can be used to sign several artifacts. If this is the case, you can move the trusted key from the artifact level to the global configuration block:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>true</verify-signatures>
<trusted-keys>
<trusted-key id="8756c4f765c9ac3cb6b85d62379ce192d401ab61" group="com.github.javaparser"/>
</trusted-keys>
</configuration>
<components/>
</verification-metadata>
The configuration above means that for any artifact belonging to the group com.github.javaparser
, we trust it if it’s signed with the 8756c4f765c9ac3cb6b85d62379ce192d401ab61
fingerprint.
The trusted-key
element works similarly to the trusted-artifact element:
-
group
, the group of the artifact to trust -
name
, the name of the artifact to trust -
version
, the version of the artifact to trust -
file
, the name of the artifact file to trust -
regex
, a boolean saying if thegroup
,name
,version
andfile
attributes need to be interpreted as regular expressions (defaults tofalse
)
You should be careful when trusting a key globally.
Try to limit it to the appropriate groups or artifacts:
-
a valid key may have been used to sign artifact
A
which you trust -
later on, the key is stolen and used to sign artifact
B
It means you can trust the key A
for the first artifact, probably only up to the released version before the key was stolen, but not for B
.
Remember that anybody can put an arbitrary name when generating a PGP key, so never trust the key solely based on the key name. Verify if the key is listed at the official site. For example, Apache projects typically provide a KEYS.txt file that you can trust.
Specifying key servers and ignoring keys
Gradle will automatically download the public keys required to verify a signature. For this it uses a list of well known and trusted key servers (the list may change between Gradle versions, please refer to the implementation to figure out what servers are used by default).
You can explicitly set the list of key servers that you want to use by adding them to the configuration:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>true</verify-signatures>
<key-servers>
<key-server uri="hkp://my-key-server.org"/>
<key-server uri="https://my-other-key-server.org"/>
</key-servers>
</configuration>
</verification-metadata>
Despite this, it’s possible that a key is not available:
-
because it wasn’t published to a public key server
-
because it was lost
In this case, you can ignore a key in the configuration block:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>true</verify-signatures>
<ignored-keys>
<ignored-key id="abcdef1234567890" reason="Key is not available in any key server"/>
</ignored-keys>
</configuration>
</verification-metadata>
As soon as a key is ignored, it will not be used for verification, even if the signature file mentions it. However, if the signature cannot be verified with at least one other key, Gradle will mandate that you provide a checksum.
If Gradle cannot download a key while bootstrapping, it will mark it as ignored. If you can find the key but Gradle does not, you can manually add it to the keyring file. |
Exporting keys for faster verification
Gradle automatically downloads the required keys but this operation can be quite slow and requires everyone to download the keys. To avoid this, Gradle offers the ability to use a local keyring file containing the required public keys. Note that only public key packets and a single userId per key are stored and used. All other information (user attributes, signatures, etc.) is stripped from downloaded or exported keys.
Gradle supports 2 different file formats for keyrings: a binary format (.gpg
file) and a plain text format (.keys
), also known as ASCII-armored format.
There are pros and cons for each of the formats: the binary format is more compact and can be updated directly via GPG commands, but is completely opaque (binary). On the opposite, the ASCII-armored format is human-readable, can be easily updated by hand and makes it easier to do code reviews thanks to readable diffs.
You can configure which file type would be used by adding the keyring-format
configuration option:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<verify-metadata>true</verify-metadata>
<verify-signatures>true</verify-signatures>
<keyring-format>armored</keyring-format>
</configuration>
</verification-metadata>
Available options for keyring format are armored
and binary
.
Without keyring-format
, if the gradle/verification-keyring.gpg
or gradle/verification-keyring.keys
file is present, Gradle will search for keys there in priority.
The plain text file will be ignored if there’s already a .gpg
file (the binary version takes precedence).
You can ask Gradle to export all keys it used for verification of this build to the keyring during bootstrapping:
./gradlew --write-verification-metadata pgp,sha256 --export-keys
Unless keyring-format
is specified, this command will generate both the binary version and the ASCII-armored file.
Use this option to choose the preferred format.
You should only pick one for your project.
It’s a good idea to commit this file to VCS (as long as you trust your VCS).
If you use git and use the binary version, make sure to make it treat this file as binary, by adding this to your .gitattributes
file:
*.gpg binary
You can also ask Gradle to export all trusted keys without updating the verification metadata file:
./gradlew --export-keys
This command will not report verification errors, only export keys. |
Bootstrapping and signature verification
Signature verification bootstrapping takes an optimistic point of view that signature verification is enough. Therefore, if you also care about integrity, you must first bootstrap using checksum verification, then with signature verification. |
Similarly to bootstrapping for checksums, Gradle provides a convenience for bootstrapping a configuration file with signature verification enabled.
For this, just add the pgp
option to the list of verifications to generate.
However, because there might be verification failures, missing keys or missing signature files, you must provide a fallback checksum verification algorithm:
./gradlew --write-verification-metadata pgp,sha256
this means that Gradle will verify the signatures and fallback to SHA-256 checksums when there’s a problem.
When bootstrapping, Gradle performs optimistic verification and therefore assumes a sane build environment. It will therefore:
-
automatically add the trusted keys as soon as verification passes
-
automatically add ignored keys for keys which couldn’t be downloaded from public key servers. See here how to manually add keys if needed
-
automatically generate checksums for artifacts without signatures or ignored keys
If, for some reason, verification fails during the generation, Gradle will automatically generate an ignored key entry but warn you that you must absolutely check what happens.
This situation is common as explained for this section: a typical case is when the POM file for a dependency differs from one repository to the other (often in a non-meaningful way).
In addition, Gradle will try to group keys automatically and generate the trusted-keys
block which reduced the configuration file size as much as possible.
Forcing use of local keyrings only
The local keyring files (.gpg
or .keys
) can be used to avoid reaching out to key servers whenever a key is required to verify an artifact.
However, it may be that the local keyring doesn’t contain a key, in which case Gradle would use the key servers to fetch the missing key.
If the local keyring file isn’t regularly updated, using key export, then it may be that your CI builds, for example, would reach out to key servers too often (especially if you use disposable containers for builds).
To avoid this, Gradle offers the ability to disallow use of key servers altogether: only the local keyring file would be used, and if a key is missing from this file, the build will fail.
To enable this mode, you need to disable key servers in the configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<key-servers enabled="false"/>
...
</configuration>
...
</verification-metadata>
If you are asking Gradle to generate a verification metadata file and that an existing verification metadata file sets enabled to false , then this flag will be ignored, so that potentially missing keys are downloaded.
|
Disabling verification or making it lenient
Dependency verification can be expensive, or sometimes verification could get in the way of day to day development (because of frequent dependency upgrades, for example).
Alternatively, you might want to enable verification on CI servers but not on local machines.
Gradle actually provides 3 different verification modes:
-
strict
, which is the default. Verification fails as early as possible, in order to avoid the use of compromised dependencies during the build. -
lenient
, which will run the build even if there are verification failures. The verification errors will be displayed during the build without causing a build failure. -
off
when verification is totally ignored.
All those modes can be activated on the CLI using the --dependency-verification
flag, for example:
./gradlew --dependency-verification lenient build
Alternatively, you can set the org.gradle.dependency.verification
system property, either on the CLI:
./gradlew -Dorg.gradle.dependency.verification=lenient build
or in a gradle.properties
file:
org.gradle.dependency.verification=lenient
Disabling dependency verification for some configurations only
In order to provide the strongest security level possible, dependency verification is enabled globally. This will ensure, for example, that you trust all the plugins you use. However, the plugins themselves may need to resolve additional dependencies that it doesn’t make sense to ask the user to accept. For this purpose, Gradle provides an API which allows disabling dependency verification on some specific configurations.
Disabling dependency verification, if you care about security, is not a good idea. This API mostly exist for cases where it doesn’t make sense to check dependencies. However, in order to be on the safe side, Gradle will systematically print a warning whenever verification has been disabled for a specific configuration. |
As an example, a plugin may want to check if there are newer versions of a library available and list those versions. It doesn’t make sense, in this context, to ask the user to put the checksums of the POM files of the newer releases because by definition, they don’t know about them. So the plugin might need to run its code independently of the dependency verification configuration.
To do this, you need to call the ResolutionStrategy#disableDependencyVerification
method:
configurations {
"myPluginClasspath" {
resolutionStrategy {
disableDependencyVerification()
}
}
}
configurations {
myPluginClasspath {
resolutionStrategy {
disableDependencyVerification()
}
}
}
It’s also possible to disable verification on detached configurations like in the following example:
tasks.register("checkDetachedDependencies") {
val detachedConf: FileCollection = configurations.detachedConfiguration(dependencies.create("org.apache.commons:commons-lang3:3.3.1")).apply {
resolutionStrategy.disableDependencyVerification()
}
doLast {
println(detachedConf.files)
}
}
tasks.register("checkDetachedDependencies") {
def detachedConf = configurations.detachedConfiguration(dependencies.create("org.apache.commons:commons-lang3:3.3.1"))
detachedConf.resolutionStrategy.disableDependencyVerification()
doLast {
println(detachedConf.files)
}
}
Trusting some particular artifacts
You might want to trust some artifacts more than others. For example, it’s legitimate to think that artifacts produced in your company and found in your internal repository only are safe, but you want to check every external component.
This is a typical company policy. In practice, nothing prevents your internal repository from being compromised, so it’s a good idea to check your internal artifacts too! |
For this purpose, Gradle offers a way to automatically trust some artifacts. You can trust all artifacts in a group by adding this to your configuration:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<trusted-artifacts>
<trust group="com.mycompany" reason="We trust mycompany artifacts"/>
</trusted-artifacts>
</configuration>
</verification-metadata>
This means that all components which group is com.mycompany
will automatically be trusted.
Trusted means that Gradle will not perform any verification whatsoever.
The trust
element accepts those attributes:
-
group
, the group of the artifact to trust -
name
, the name of the artifact to trust -
version
, the version of the artifact to trust -
file
, the name of the artifact file to trust -
regex
, a boolean saying if thegroup
,name
,version
andfile
attributes need to be interpreted as regular expressions (defaults tofalse
) -
reason
, an optional reason, why matched artifacts are trusted
In the example above it means that the trusted artifacts would be artifacts in com.mycompany
but not com.mycompany.other
.
To trust all artifacts in com.mycompany
and all subgroups, you can use:
<?xml version="1.0" encoding="UTF-8"?>
<verification-metadata xmlns="https://schema.gradle.org/dependency-verification"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://schema.gradle.org/dependency-verification https://schema.gradle.org/dependency-verification/dependency-verification-1.3.xsd">
<configuration>
<trusted-artifacts>
<trust group="^com[.]mycompany($|([.].*))" regex="true" reason="We trust all mycompany artifacts"/>
</trusted-artifacts>
</configuration>
</verification-metadata>
Trusting multiple checksums for an artifact
It’s quite common to have different checksums for the same artifact in the wild. How is that possible? Despite progress, it’s often the case that developers publish, for example, to Maven Central and another repository separately, using different builds. In general, this is not a problem but sometimes it means that the metadata files would be different (different timestamps, additional whitespaces, …). Add to this that your build may use several repositories or repository mirrors and it makes it quite likely that a single build can "see" different metadata files for the same component! In general, it’s not malicious (but you must verify that the artifact is actually correct), so Gradle lets you declare the additional artifact checksums. For example:
<component group="org.apache" name="apache" version="13">
<artifact name="apache-13.pom">
<sha256 value="2fafa38abefe1b40283016f506ba9e844bfcf18713497284264166a5dbf4b95e">
<also-trust value="ff513db0361fd41237bef4784968bc15aae478d4ec0a9496f811072ccaf3841d"/>
</sha256>
</artifact>
</component>
You can have as many also-trust
entries as needed, but in general you shouldn’t have more than 2.
Skipping Javadocs and sources
By default Gradle will verify all downloaded artifacts, which includes Javadocs and sources. In general this is not a problem but you might face an issue with IDEs which automatically try to download them during import: if you didn’t set the checksums for those too, importing would fail.
To avoid this, you can configure Gradle to trust automatically all javadocs/sources:
<trusted-artifacts>
<trust file=".*-javadoc[.]jar" regex="true"/>
<trust file=".*-sources[.]jar" regex="true"/>
</trusted-artifacts>
Adding keys manually to the keyring
Adding keys to the ASCII-armored keyring
The added key must be ASCII-armored formatted and can be simply added at the end of the file. If you already downloaded the key in the right format, you can simply append it to the file.
Or you can amend an existing KEYS file by issuing the following commands:
$ gpg --no-default-keyring --keyring /tmp/keyring.gpg --recv-keys 8756c4f765c9ac3cb6b85d62379ce192d401ab61
gpg: keybox '/tmp/keyring.gpg' created
gpg: key 379CE192D401AB61: public key "Bintray (by JFrog) <****>" imported
gpg: Total number processed: 1
gpg: imported: 1
# Write its ASCII-armored version
$ gpg --keyring /tmp/keyring.gpg --export --armor 8756c4f765c9ac3cb6b85d62379ce192d401ab61 > gradle/verification-keyring.keys
Once done, make sure to run the generation command again so that the key is processed by Gradle. This will do the following:
-
Add a standard header to the key
-
Rewrite the key using Gradle’s own format, which trims the key to the bare minimum
-
Move the key to its sorted location, keeping the file reproducible
Adding keys to the binary keyring
You can add keys to the binary version using GPG, for example issuing the following commands (syntax may depend on the tool you use):
$ gpg --no-default-keyring --keyring gradle/verification-keyring.gpg --recv-keys 8756c4f765c9ac3cb6b85d62379ce192d401ab61
gpg: keybox 'gradle/verification-keyring.gpg' created
gpg: key 379CE192D401AB61: public key "Bintray (by JFrog) <****>" imported
gpg: Total number processed: 1
gpg: imported: 1
$ gpg --no-default-keyring --keyring gradle/verification-keyring.gpg --recv-keys 6f538074ccebf35f28af9b066a0975f8b1127b83
gpg: key 0729A0AFF8999A87: public key "Kotlin Release <****>" imported
gpg: Total number processed: 1
gpg: imported: 1
Dealing with a verification failure
Dependency verification can fail in different ways, this section explains how you should deal with the various cases.
Missing verification metadata
The simplest failure you can have is when verification metadata is missing from the dependency verification file. This is the case for example if you use checksum verification, then you update a dependency and new versions of the dependency (and potentially its transitive dependencies) are brought in.
Gradle will tell you what metadata is missing:
Execution failed for task ':compileJava'. > Dependency verification failed for configuration ':compileClasspath': - On artifact commons-logging-1.2.jar (commons-logging:commons-logging:1.2) in repository 'MavenRepo': checksum is missing from verification metadata.
-
the missing module group is
commons-logging
, it’s artifact name iscommons-logging
and its version is1.2
. The corresponding artifact iscommons-logging-1.2.jar
so you need to add the following entry to the verification file:
<component group="commons-logging" name="commons-logging" version="1.2">
<artifact name="commons-logging-1.2.jar">
<sha256 value="daddea1ea0be0f56978ab3006b8ac92834afeefbd9b7e4e6316fca57df0fa636" origin="official distribution"/>
</artifact>
</component>
Alternatively, you can ask Gradle to generate the missing information by using the bootstrapping mechanism: existing information in the metadata file will be preserved, Gradle will only add the missing verification metadata.
Incorrect checksums
A more problematic issue is when the actual checksum verification fails:
Execution failed for task ':compileJava'. > Dependency verification failed for configuration ':compileClasspath': - On artifact commons-logging-1.2.jar (commons-logging:commons-logging:1.2) in repository 'MavenRepo': expected a 'sha256' checksum of '91f7a33096ea69bac2cbaf6d01feb934cac002c48d8c8cfa9c240b40f1ec21df' but was 'daddea1ea0be0f56978ab3006b8ac92834afeefbd9b7e4e6316fca57df0fa636'
This time, Gradle tells you what dependency is at fault, what was the expected checksum (the one you declared in the verification metadata file) and the one which was actually computed during verification.
Such a failure indicates that a dependency may have been compromised. At this stage, you must perform manual verification and check what happens. Several things can happen:
-
a dependency was tampered in the local dependency cache of Gradle. This is usually harmless: erase the file from the cache and Gradle would redownload the dependency.
-
a dependency is available in multiple sources with slightly different binaries (additional whitespace, …)
-
please inform the maintainers of the library that they have such an issue
-
you can use
also-trust
to accept the additional checksums
-
-
the dependency was compromised
-
immediately inform the maintainers of the library
-
notify the repository maintainers of the compromised library
-
Note that a variation of a compromised library is often name squatting, when a hacker would use GAV coordinates which look legit but are actually different by one character, or repository shadowing, when a dependency with the official GAV coordinates is published in a malicious repository which comes first in your build.
Untrusted signatures
If you have signature verification enabled, Gradle will perform verification of the signatures but will not trust them automatically:
> Dependency verification failed for configuration ':compileClasspath': - On artifact javaparser-core-3.6.11.jar (com.github.javaparser:javaparser-core:3.6.11) in repository 'MavenRepo': Artifact was signed with key '379ce192d401ab61' (Bintray (by JFrog) <****>) and passed verification but the key isn't in your trusted keys list.
In this case it means you need to check yourself if the key that was used for verification (and therefore the signature) can be trusted, in which case refer to this section of the documentation to figure out how to declare trusted keys.
Failed signature verification
If Gradle fails to verify a signature, you will need to take action and verify artifacts manually because this may indicate a compromised dependency.
If such a thing happens, Gradle will fail with:
> Dependency verification failed for configuration ':compileClasspath': - On artifact javaparser-core-3.6.11.jar (com.github.javaparser:javaparser-core:3.6.11) in repository 'MavenRepo': Artifact was signed with key '379ce192d401ab61' (Bintray (by JFrog) <****>) but signature didn't match
There are several options:
-
signature was wrong in the first place, which happens frequently with dependencies published on different repositories.
-
the signature is correct but the artifact has been compromised (either in the local dependency cache or remotely)
The right approach here is to go to the official site of the dependency and see if they publish signatures for their artifacts. If they do, verify that the signature that Gradle downloaded matches the one published.
If you have checked that the dependency is not compromised and that it’s "only" the signature which is wrong, you should declare an artifact level key exclusion:
<components>
<component group="com.github.javaparser" name="javaparser-core" version="3.6.11">
<artifact name="javaparser-core-3.6.11.pom">
<ignored-keys>
<ignored-key id="379ce192d401ab61" reason="internal repo has corrupted POM"/>
</ignored-keys>
</artifact>
</component>
</components>
However, if you only do so, Gradle will still fail because all keys for this artifact will be ignored and you didn’t provide a checksum:
<components>
<component group="com.github.javaparser" name="javaparser-core" version="3.6.11">
<artifact name="javaparser-core-3.6.11.pom">
<ignored-keys>
<ignored-key id="379ce192d401ab61" reason="internal repo has corrupted POM"/>
</ignored-keys>
<sha256 value="a2023504cfd611332177f96358b6f6db26e43d96e8ef4cff59b0f5a2bee3c1e1"/>
</artifact>
</component>
</components>
Manual verification of a dependency
You will likely face a dependency verification failure (either checksum verification or signature verification) and will need to figure out if the dependency has been compromised or not.
In this section we give an example how you can manually check if a dependency was compromised.
For this we will take this example failure:
> Dependency verification failed for configuration ':compileClasspath': - On artifact j2objc-annotations-1.1.jar (com.google.j2objc:j2objc-annotations:1.1) in repository 'MyCompany Mirror': Artifact was signed with key '29579f18fa8fd93b' but signature didn't match
This error message gives us the GAV coordinates of the problematic dependency, as well as an indication of where the dependency was fetched from.
Here, the dependency comes from MyCompany Mirror
, which is a repository declared in our build.
The first thing to do is therefore to download the artifact and its signature manually from the mirror:
$ curl https://my-company-mirror.com/repo/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar --output j2objc-annotations-1.1.jar $ curl https://my-company-mirror.com/repo/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar.asc --output j2objc-annotations-1.1.jar.asc
Then we can use the key information provided in the error message to import the key locally:
$ gpg --recv-keys B801E2F8EF035068EC1139CC29579F18FA8FD93B
And perform verification:
$ gpg --verify j2objc-annotations-1.1.jar.asc gpg: assuming signed data in 'j2objc-annotations-1.1.jar' gpg: Signature made Thu 19 Jan 2017 12:06:51 AM CET gpg: using RSA key 29579F18FA8FD93B gpg: BAD signature from "Tom Ball <****>" [unknown]
What this tells us is that the problem is not on the local machine: the repository already contains a bad signature.
The next step is to do the same by downloading what is actually on Maven Central:
$ curl https://my-company-mirror.com/repo/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar --output central-j2objc-annotations-1.1.jar $ curl https://my-company-mirror.com/repo/com/google/j2objc/j2objc-annotations/1/1/j2objc-annotations-1.1.jar.asc --output central-j2objc-annotations-1.1.jar.asc
And we can now check the signature again:
$ gpg --verify central-j2objc-annotations-1.1.jar.asc gpg: assuming signed data in 'central-j2objc-annotations-1.1.jar' gpg: Signature made Thu 19 Jan 2017 12:06:51 AM CET gpg: using RSA key 29579F18FA8FD93B gpg: Good signature from "Tom Ball <****>" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: B801 E2F8 EF03 5068 EC11 39CC 2957 9F18 FA8F D93B
This indicates that the dependency is valid on Maven Central. At this stage, we already know that the problem lives in the mirror, it may have been compromised, but we need to verify.
A good idea is to compare the 2 artifacts, which you can do with a tool like diffoscope.
We then figure out that the intent wasn’t malicious but that somehow a build has been overwritten with a newer version (the version in Central is newer than the one in our repository).
In this case, you can decide to:
-
ignore the signature for this artifact and trust the different possible checksums (both for the old artifact and the new version)
-
or cleanup your mirror so that it contains the same version as in Maven Central
It’s worth noting that if you choose to delete the version from your repository, you will also need to remove it from the local Gradle cache.
This is facilitated by the fact the error message tells you were the file is located:
> Dependency verification failed for configuration ':compileClasspath': - On artifact j2objc-annotations-1.1.jar (com.google.j2objc:j2objc-annotations:1.1) in repository 'MyCompany Mirror': Artifact was signed with key '29579f18fa8fd93b' but signature didn't match This can indicate that a dependency has been compromised. Please carefully verify the signatures and checksums. For your information here are the path to the files which failed verification: - $<<directory_layout.adoc#dir:gradle_user_home,GRADLE_USER_HOME>>/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/976d8d30bebc251db406f2bdb3eb01962b5685b3/j2objc-annotations-1.1.jar (signature: GRADLE_USER_HOME/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1/82e922e14f57d522de465fd144ec26eb7da44501/j2objc-annotations-1.1.jar.asc) GRADLE_USER_HOME = /home/jiraya/.gradle
You can safely delete the artifact file as Gradle would automatically re-download it:
rm -rf ~/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.1
Cleaning up the verification file
If you do nothing, the dependency verification metadata will grow over time as you add new dependencies or change versions: Gradle will not automatically remove unused entries from this file. The reason is that there’s no way for Gradle to know upfront if a dependency will effectively be used during the build or not.
As a consequence, adding dependencies or changing dependency version can easily lead to more entries in the file, while leaving unnecessary entries out there.
One option to cleanup the file is to move the existing verification-metadata.xml
file to a different location and call Gradle with the --dry-run
mode: while not perfect (it will not notice dependencies only resolved at configuration time), it generates a new file that you can compare with the existing one.
We need to move the existing file because both the bootstrapping mode and the dry-run mode are incremental: they copy information from the existing metadata verification file (in particular, trusted keys).
Refreshing missing keys
Gradle caches missing keys for 24 hours, meaning it will not attempt to re-download the missing keys for 24 hours after failing.
If you want to retry immediately, you can run with the --refresh-keys
CLI flag:
./gradlew build --refresh-keys
See here how to manually add keys if Gradle keeps failing to download them.