Dependencies between Maven dependencies and Linux package dependencies


I'm working on ways to make Java software installable under Linux via packages, i.e. generate .deb-files or .rpm-files of software systems or libraries, that, if possible, respect the File Hierarchy Standard [1]. One aspect that makes generating these packages difficult, are dependencies. Using Maven [2] as the build system, every so-called 'artifact' usually depends on other artifacts. A software product depends on libraries (.jar-files) and libraries depend on other libraries. These dependencies between Maven artifacts should be reflected in dependencies between packages we create and packages where our packages depend on.

It is generally not a good idea to provide external libraries with out products if these external libraries can be stored on a central place on our systems. This saves us multiple occurences of exactly the same artifact on our system, and/or the same artifact in different versions. If a bug is solved in a specific library and a new version is released, we do not want to upgrade every product that depends on this library, only because it provides that library with the full product.

More and more Java libraries become available as Debian or RPM packages. We should use these packages if possible.

Dependencies on multiple levels


A Java class can use instances of external classes or static methods from such classes. A dependency can be required or optional. The latter depends on the way the class is used or the environment under which the class is used. The dependency can be limited to specific versions of these other classes, because a necessary method was introduced in a specific version, a bug has been fixed since a certain version, or a method that is used is present until a certain version. A class can also depend on components like configuration files, XSL files et cetera.

Libraries and end products (jar, war and brothers)

A library or end product is a collection of classes and resources like configuration files, XSL scripts, scripts to start and stop a service et cetera. Usually these products are released with specific versions. Dependencies of every component in such product to components within another product are reflected in dependency rules like 'requires product X version Y.Z or better'.

Maven artifacts

A Maven artifact is a library or end product as described above. A Maven artifact contains a lot of metadata on the 'thing' and also a list of dependencies on other Maven artifacts. A lot of Maven artifacts are available in publicly available repositories. The biggest repository is called 'Central'. That's the de facto repository. Artifacts can also be stored in a locally maintained repository or in one or more organizational repositories. During the build of a product, all direct and indirect (transient) dependencies can be resolved to build a product with all dependencies included.


A package (RPM, DEB or Slackware package) with Java libraries, can contain one or more jars. If it contains more jars, these usually are interrelated, like a core jar, a jar with samples, a jar with a web application. All jars in such a package have the same version and normally are all included in a zip-file one can download from the products' website.

Relationships between Maven artifacts and software packages and dependencies

Figure 1 shows an UML object relationship diagram explaining the relationships.

Figure 1: relationship diagram

On the left hand side you see the Maven artifacts and their dependencies. A Maven artifact has a groupId, an artifactId and a version. For example, the current version of XOM, an XML API for Java, has groupId 'xom', artifactId 'xom' and version '1.1'. It provides 'xom-1.1.jar'. Another example is JSAP, a library for parsing command line parameters. The current version is groupId='com.martiansoftware', artifactId='jsap' and version='4.2'. It provides 'jsap-4.2.jar'. An artifact has 0 or more dependencies. Dependencies do not mention exact versions, but version constraints. More on version constraints can be found at [3]. Every dependency should match at least one artifact.

On the right hand side you see the software packages and their dependencies. A software package has a name and version and it has 0 or more dependencies. Just like in the Maven artifacts situation, a dependency should match at least one software package.

A Java library software package provides one or more Maven artifacts. Debian package 'libxom-java' currently provides 'xom-1.1.jar'. Installing it leads to this file installed in directory '/usr/share/java/', with a symbolic link 'xom.jar' pointing to 'xom-1.1.jar'.


[1] http://www.pathname.com/fhs/

[2] http://maven.apache.org/

[3] http://docs.codehaus.org/display/MAVEN/Dependency+Mediation+and+Conflict+Resolution


Generating Linux packages from Maven artifacts


Java applications usually provide all dependent libraries (jars) with the application. This has advantages, like having less problems getting the application to work and no problems with possibly slightly different versions of libraries. But providing all libraries with an application has disadvantages too, like numerous copies (and versions) of the same library installed on a single system and the necessity to upgrade all software package using a specific library if this library should contain a serious bug.

More and more Java applications become available as Debian packages. Debian policies state that statically linked libraries must not be used, for the reasons mentioned above. Providing external jars with a program is like using statically linked libraries in C or C++. Debian Java packages typically use external jars that are contained in separate packages. After installing such a package (via dependencies), a jar is installed in directory /usr/share/java. If we look at XOM, the best XML API for Java, package 'libxom-java', at the time of writing, installs xom-1.1.jar and a symlink xom.jar in directory /usr/share/java:

$ ls -l /usr/share/java/*xom*
-rw-r--r-- 1 root root 265858 2008-01-15 06:59 /usr/share/java/xom-1.1.jar
lrwxrwxrwx 1 root root 11 2008-07-08 12:14 /usr/share/java/xom.jar -> xom-1.1.jar
-rw-r--r-- 1 root root 160069 2008-01-15 07:00 /usr/share/java/xom-samples-1.1.jar
lrwxrwxrwx 1 root root 19 2008-07-08 12:14 /usr/share/java/xom-samples.jar -> xom-samples-1.1.jar

There's more in this package, but for now we concentrate on the XOM jar. After installing the 'libxom-java' package, we can add /usr/share/java/xom.jar to our classpath, and if XOM version 1.1.1 is released because of some bug, or version 1.2 is released with new features, we get an automatic update and our program will still work.

A lot of libraries are now available as Debian packages. If a library is available via a Debian package, we should use these instead of libraries we provide with our software, if we distribute to Debian or Ubuntu systems. But a lot of libraries are not (yet) available as packages. How to cope with such a mixed environment? Below I'll propose a way to at least make things work for your local environment.

Sample application 'Telex' and sample library PircBot

Telex is a one day weekend software project of mine. It's an application that can run several IRC robots on different IRC servers and channels. It sends the titles of new items of as many RSS feeds as you like, to these robots. It's very flexible. You can send different feeds to different robots for instance.

Telex depends on two libraries, PircBot [1] and Rome [2]. PircBot is used for the IRC bots and Rome is used to check RSS feeds. Both libraries are not available as Debian packages for Ubuntu 8.04, at least not in the default repositories. I'm going to show you how to you can make a Debian package of PircBot. We'll do that in small steps.

Get PircBot

The pom.xml file of Telex, contains the following dependency:


If I run command % mvn package or another Maven command that results in fetching dependencies and transitive dependencies, this will result in fetching the PircBot jar from some external Maven repository and placing it in my local repository if it is not already there.

Version constraints in Maven

The version constraint in the dependency you see above is a so-called soft constraint. It doesn't mean that this exact version will be fetched, though most Maven dependencies mention an exact version of an existing jar as a soft constraint. The latter is a pity. They should mention all versions of the library that meet the requirements. More information on version constraints in Maven can be found at [3]. There you will find that a version constraint can contain a series of ranges, like '[2.1,2.4.3),(2.5,)', meaning every version greater than or equal to 2.1 and smaller than and not including 2.4.3 or greater than (not including) 2.5. Versions 2.1, 2.2, 2.4.2 and 6.7 meet this sample constraint and versions 2.0, 2.4.3 and 2.5 do not.

Install file with make

It is easy to install the PircBot jar using GNU Make [4] if this jar is in your local Maven repository. Here's the simplest Makefile you can think of to do this. As you see, my home directory is /home/vosf-dev. Store the following contents in a file called Makefile and change the path to the PircBot jar according to your situation. Use a tab at the beginning of the second line, not spaces, otherwise it won't work.

install --owner=root --group=root /home/vosf-dev/.m2/repository/pircbot/pircbot/1.4.2/pircbot-1.4.2.jar /usr/share/java/

Make sure you have installed GNU Make and then run make install as root. Sample:

$ sudo make install
install --owner=root --group=root /home/vosf-dev/.m2/repository/pircbot/pircbot/1.4.2/pircbot-1.4.2.jar /usr/share/java/

Throw the installed jar away immediately, because now we are going to install it via a package:

$ sudo rm /usr/share/java/pircbot-1.4.2.jar

Make a package

Using Checkinstall [5] it is not difficult to create a Debian package. If you haven't yet installed Checkinstall, do that first via $ sudo apt-get install checkinstall.

In the directory where Makefile is located, create a file called description-pak containing a description of the package. Here's a sample:

Library for building IRC robots
PircBot is a Java framework for writing IRC bots quickly and
easily. Its features include an event-driven architecture to handle
common IRC events, flood protection, DCC resuming support, ident
support, and more.

We want a symbolic link called pircbot.jar to the pircbot-1.4.2.jar file (and have it removed when the package is removed). One way to achieve that is via scripts that are executed before and after an install and before and after removal. We'll use two of these scripts. Create file postinstall-pak with the following content:

cd /usr/share/java
ln -s pircbot-1.4.2.jar pircbot.jar
exit $?

Create a file called preremove-pak:

cd /usr/share/java
rm pircbot.jar
exit $?

Both scripts return 0 if creation or removal of the link is sucessful. You do not need to make the scripts executable with chmod.

Now issue the following command:
$ sudo checkinstall --maintainer="Fred Vos \<fred.vos\@mokolo.org\>" --pkgname="libpircbot-java" --pkgversion=1.4.2 --pkggroup=libs --nodoc --install=no

You'll use other settings of course. This produces a list with settings, most as you provided on the command line. You need this settings list, because the architecture setting on the command line doesn't seem to work properly. Press <7><Enter>all<Enter> to change 'Architecture' from 'i386' to 'all' and <Enter> to accept. If the architecture setting works (--pkgarch=all), you can add --default to the command line options.

Basically what Checkinstall does here is run the make install command and intercept all actions GNU Make was planning to do and generate a package out of that; a Debian package by default on a Debian (or Ubuntu) Linux system and an RPM on SuSE and Redhat systems and a Slackware package on a Slackware system. But you can make RPMs too on a Debian system, with an extra parameter. Check the Checkinstall man page for that.

Files postinstall-pak and preremove-pak are included and will work in both .deb and .rpm packages.

Test it

$ ls -l /usr/share/java/pircbot*
ls: cannot access /usr/share/java/pircbot*: No such file or directory
$ sudo dpkg -i libpircbot-java_1.4.2-1_all.deb
Selecting previously deselected package libpircbot-java.
(Reading database ... 124836 files and directories currently installed.)
Unpacking libpircbot-java (from libpircbot-java_1.4.2-1_all.deb) ...
Setting up libpircbot-java (1.4.2-1) ...
$ ls -l /usr/share/java/pircbot*
-rwxr-xr-x 1 root root 74259 2008-08-10 20:08 /usr/share/java/pircbot-1.4.2.jar
lrwxrwxrwx 1 root root 17 2008-08-10 20:19 /usr/share/java/pircbot.jar -> pircbot-1.4.2.jar
$ sudo apt-get remove libpircbot-java
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 102kB disk space will be freed.
Do you want to continue [Y/n]?
(Reading database ... 124836 files and directories currently installed.)
Removing libpircbot-java ...
$ ll /usr/share/java/pircbot*
ls: cannot access /usr/share/java/pircbot*: No such file or directory
$ sudo rm libpircbot-java_1.4.2-1_all.deb

Make available for apt

Next step is to make the package available for apt for internal use, so you can do something like:
$ sudo apt-get install libpircbot-java
This is not the scope of this page. There are plenty of pages on the Internet on that subject.

Final thoughts

Though you can install and remove the packages you create this way, these are only Debian packages in a technical sense. These 'packages' make it possible to install some files at the right locations. Making a true Debian package requires a lot more files to be distributed. Still, installing external libraries using this technique is acceptable in cases where you need one or two external libraries to be available as packages.

I didn't handle the case where this external library has dependencies of its own. I'll handle dependencies between Maven dependencies and package dependencies in another article. In that article I'll show you how this can be handled.

Further steps

In following articles, I'll try to join Maven dependencies with package dependencies and show you how to make Debian packages from our own Java software.


[1] http://www.jibble.org/pircbot.php

[2] https://rome.dev.java.net/

[3] http://docs.codehaus.org/display/MAVEN/Dependency+Mediation+and+Conflict+Resolution

[4] http://www.gnu.org/software/make/

[5] http://www.asic-linux.com.mx/~izto/checkinstall/