Thread by @jeoj

Given recent events #log4j #log4jshell, here are some thoughts about dealing with dependencies and their versions in the Java-ecosystem when using #gradle. Some thoughts/explanations and an idea I had when thinking about the current state of things.

Dependency management is hell. Always. If you rely on external open-source components, which again rely on other open-source components, you are already in trouble. Luckily, that's what everyone is doing. So we are all in trouble together.

Of course it is great to reuse. But using code you have not written yourself, and you don't fully understand yourself, requires a lot of trust. As we have just seen, this can lead to incidents where you need to be ready to respond quickly.

That's how it is and it's not necessarily bad. It's a tradeoff so that you do not always need to develop things from scratch. (Sometimes that's a valid option too though.)

Dependency management/resolution systems, like the one Gradle provides, cannot solve this automagically. But they provide you with the tools to help you find the right balance between (blind) trust and verification effort.

It depends on many many factors, where the sweet spot is for you and your product. There is no 'one-fits-all' solution.

So what does Gradle offer? For me, the most central concept here is 'Dependency Constraints'. A dependency constraint tells the resolution engine which versions of a certain dependency - let's say 'org.apache.logging.log4j:log4j-core' - it is allowed to pick.

We can make a clear distinction between a dependency to a component:

dependencies {
api("org.apache.logging.log4j:log4j-core")
}

And a constraint on that component:

dependencies.constraints {
api("org.apache.logging.log4j:log4j-core:2.16.0")
}

Constraints can express many things. And they can just "lay around unused". They only jump into action once there is a component they are concerned with. So it won't matter if you use Log4J or not. You may just add a constraint to be sure a vulnerable version is never picked:

dependencies.constraints {
api("org.apache.logging.log4j:log4j-core") {
version { reject("(,2.15.0)") }
}
}

A powerful feature ist that constraints can be published as a 'platform' (like a BOM made of Gradle metadata) and so you can share the information that certain sets of versions of certain components are 'forbidden' (rejected).

If you develop a (open-source) library yourself, you can also publish such information as part of the metadata. So you could express things like "If you use my library, I forbid to use another library in a certain version" and Gradle will respect that.

Now for the dependencies you declare (direct dependencies) you can make the distinction between 'dependency' and 'dependency constraint' by not adding a version to any of your dependency declarations. Gradle will then fail if there is no constraint for a certain component.

Which is a good thing, because then you can decide which version(s) is/are allowed - which version(s) you trust. And while the dependencies are different in each (sub)project, the constraints can be the same everywhere and you can declare them in a central place.

But what about the dependencies you get transitively from other components? Components could publish constraints for their dependencies (in their metadata). But in reality, they often only do it in a very limited way.

It's hard: The (open-source) components do not know about all the (crazy) contexts they will be (re)used in. So the metadata is usually rather lenient about what is needed. Usually, for the transitive dependency, you get the version that was used when building the component.

Gradle treats this as a so called "required version", which translates into a constraint as follows: "this or a higher version is allowed; lower versions are forbidden". If you have more constraints 'laying around' demanding a higher version, a higher version will be chosen.

The reasoning behind this is 'assumed backward compatibility', which, of course, is not true in all cases. But since metadata right now does not express anything about the versioning schema of a component, assumptions need to be made (there is a lot of room for improvement).

The assumption right now also does not care much about the stability/trustworthiness of a certain version. It only looks at numbers* and compares them.

*well... also letters (which of two versions is "higher" is sometimes surprising)

So there IS a lot of room for improvement on which information components could publish in their metadata. But even if it is technically possible and folks would publish this information... the trust issue remains (what is published could be total bogus).

Let's look at the other extreme: If you don't trust that the components you use provide you with good versions of their transitive dependencies, you could ignore the transitive dependencies altogether. Declare everything you need yourself. I have seen folks follow this philosophy

When you do that, you almost don't need the dependency management of Gradle anymore. It then becomes a dumb downloader that downloads and caches Jar files.

With the separation between 'dependencies' and 'dependency constraints' though, there is a middle way. Which may still seem radical, but could be an interesting approach for certain projects.

Here is the idea: We do not throw away all the dependencies, but only the, usually rather weak, dependency constraints (the required versions) that come with the transitive dependencies.

If we do this, Gradle will complain about each dependency for which it has no constraint. Then we are forced to declare a constraint for each dependency. No matter if we declared it directly or got it indirectly.

Which can be a good thing, because then you can decide which version(s) is/are allowed - which version(s) you trust. And since you want the constraints to be the same everywhere, you can declare them in a central place and find all the versions there.

You can do this with Component Metadata Rules. They allow you to modify metadata after Gradle downloaded it. You can add a rule that removes all versions from dependencies:

allVariants{withDependencies{forEach{it.version{require("")}}}}

Full example: github.com/jjohannes/grad…

If that's too strong, you can have exceptions. E.g. do not remove anything for components you absolutely trust.

I haven't actually used this approach in practice yet, but every larger project I see struggles with how to do this best. And folks usually look for a tradeoff between "Gradle, just do everything automatically" and "I want to have full control and declare all versions myself"

And then there are a lot of variations of this I could imagine. useful or not, it's an example of Gradle's flexibility so that you can decide what's best for your product with the current "not so perfect" situation in the ecosystem. If you are willing to invest some time into it.

Anyway, there is helpful tooling that checks the sanity of the result of the dependency resolution you can use in any build. E.g.:
- OWASP dependency-check-gradle plugin jeremylong.github.io/DependencyChec…
- dependency-analysis gradle plugin github.com/autonomousapps… (by @AutonomousApps )

My videos about dependencies in Gradle - if you want to dive deeper:
- Declaring Dependencies
- Centralizing Dependency Versions
- Dependency Version Conflicts
- Capability Conflicts