Managing 673 maven projects with POM Explorer

What a tool can do to help managing maven projects

Arnaud Tournier, le 26-06-2015

When a team works with a lot of maven projects it becomes quickly painful to do some basic tasks like:

  • manage versionning and connections between the different projects.
  • releasing and opening versions, especially when the maven-release plugin needs to be run on many projects and when versionning is not standard.
  • managing external dependencies also can become complex, and ensuring that a single version of a dependency is used accross different projects is sometimes not a trivial question to ask.
  • in a one word, applying transformations on a dependency graph is difficult.
  • mind-mapping the dependency graph is difficult when the number of projects grows. That can increase the amount of time needed by new people to understand a project graph, and that also makes maintaining and changing things difficult.
  • checking consistency and optimizing the dependency graph is not an easy task neither.
  • having an always up-to-date build of snapshots and release is not easy when projects are distributed everywhere.

Pom Explorer

So there is the Pom Explorer tool which tries to address those problems by providing those functionalities :

  • release a graph : release a pom or all poms and its/theirs dependencies and updates all dependent poms accross multiple repositories and projects,
  • change a gav : updates a project’s gav and make all the project which depends on it follow this change.
  • manages properties, dependency management, and so on. Pom-explorer knows what pom.xml to update and where to update. If a dependency specifies <version>${foobar.version}</version>, pom-explorer will go to update the foobar.version property.
  • query the dependency graph to retrieve pertinent information about your projects,
  • statistics and check functions are also available,
  • display 3d interactive graph,
  • export graphml files,
  • find not used dependencies and other similar problems,
  • list java classes provided by artifacts,
  • list java classes referenced by artifacts,
  • runs a light and efficient web server so local and shared usage is possible.

The tool will also support automatically building projects in order to always have such or such project always up to date.

Use cases

In this article, I will show some common use cases possible with this tool.

Installation

First one needs to install and run the software. Put yourself in a temporary directory and type those commands :

git clone https://github.com/ltearno/pom-explorer.git
cd pom-explorer
java -jar target/pom-explorer.jar

The program should welcome you and ask you to go to this address :

http://localhost:90

This is the console to the application. You can type commands in the prompt, they will be sent to the server and it will answer. You can use up and down arrows to recall past commands.

Let’s start by typing ? to get the available commands :

help command

Analyze of repositories

OK. First we will analyse a directory where there are many maven projects, then we will work a bit to optimize those projects. You will have to adapt the exercise to your computer.

Let’s analyse my git repositories directory :

analyze directory c:\documents\repos

This will analyze my projects and construct an in-memory graph of the dependency graph :

analyze command

Now, the program knows about everything on my projects, let’s start asking questions !

List of GAVs…

Let’s get the list of all existing GAVs (groupId, artifactId, version) in the graph. There will be my projects and all the GAVs on which they depend. Type this command :

gav li

Note that you can type only the first letters of a command, as long as there is no ambiguity. Here li stands for list.

Find dependencies on an obsolete artifact

As I look through the list of GAVs, I remark that there are still an old snapshot version of the hexa.binding artifact hanging around. The latest released version is 1.3 and the working version is 1.4-SNAPSHOT so the version 1.3-SNAPSHOT should not be used anymore.

obsolete versions

Which is the project still depending on this very deprecated this version ? Let’s ask the question :

depends on fr.lteconsulting:hexa.binding:1.3-SNAPSHOT

depends on

Here it is ! the project rigpa.org:regsys-clients:1.0.0-SNAPSHOT is still using an old snapshot. Let’s arrange that. Pom Explorer is able to change the pom properties and dependencies by itself.

Updating this wrong dependency

What we want is to change fr.lteconsulting:hexa.binding:1.3-SNAPSHOT to fr.lteconsulting:hexa.binding:1.3 so that the project uses the latest release available. We could desire to change for fr.lteconsulting:hexa.binding:1.4-SNAPSHOT which would be possible with the same command as we’ll see.

For that we will use the change gav command :

cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3

Here is what Pom Explorer answers :

change gav

So first Pom Explorer finds what needs to be changed in the graph. This might be the project itself and all projects which depend on it. After that the program begins a loop in which all changes are checked and appropriately transformed when needed. For instance changing a dependency version can become changing a property value. Changes are first resolved as described before and they are then transformed in a change list to apply to be applied to pom.xml files.

In the ouput, there is first a little warning saying the fr.lteconsulting:hexa.binding:1.3-SNAPSHOT project was not found. That’s normal because the project in now in version 1.4-SNAPSHOT. So there is no need to modify it.

Then in the change list section, the changes that are to be applied to pom.xml files are listed. The first one says ‘project not found’ and that’s ok as seen before. The second one says to modify the C:\documents\repos\regsys-clients\pom.xml and change the dependency ([DEPENDENCY]) fr.lteconsulting:hexa.binding:1.3-SNAPSHOT to fr.lteconsulting:hexa.binding:1.3. The “causes” message is useful when a change is caused by other changes (as said before a dependency change can become one or several property changes).

If we had properties involved, Pom Explorer would have found them and included them in the change set.

Now that we reviewed the proposed changes and agreed with them, let’s apply them by using the same command with the -apply flag :

cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3 -apply

change gav -apply

We see that at the end of the same process, the program updated the dependency in the right pom.xml file.

Let’s have a look at the file it self :

<dependency>
    <groupId>fr.lteconsulting</groupId>
    <artifactId>hexa.binding</artifactId>
    <version>1.3</version>
    <scope>compile</scope>
</dependency>

OK, the file is correct now… Oh well no ! I just find other dependencies in SNAPSHOT versions !

Finding more duplicate and obsolete dependencies

Let’s accept it, our projects are not up to date. Well let’s see how many of those artifacts there are with multiple versions used. For that i type the check command :

check

Ok there is some work to do !

Opening a version

Now let’s look at another use case.

Say that the hexa.binding project is in version 1.3 and i want to open the version 1.4-SNAPSHOT. I also want all the projects which depend on version 1.3 to move to 1.4-SNAPSHOT. On the way, I want all modified projects still in a release version to be SNAPSHOT-ized too. And i want this to happen recursively as new projects are opened.

With Pom Explorer, that’s only one command :

change gav fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT

As you can see, warnings are generated when projects are reopened :

opening

Those are normal warnings, they are just here so that you know what happens. Then, there is a big list of changes to be made, because the hexa.binding artifact is used in many central projects that were in a release state. Glad that we didn’t do that by hand ! Even with the maven-update-version plugin, there would have been a lot of repositories to go to open and update.

Let’s apply the changes with the -apply flag :

cha ga fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT -apply

All the changes have been made, about 30 of them. In one go !

Refresh the page so that a new session is created from the changed files.

We can see that many of the projects have been reopened :

snapshots

You now have to commit all the repositories with this update. Pom explorer does not do that yet, but maybe in the future !

Releasing many poms

Imagine the sprint is almost finished now and it’s now time to release the projects. Type the gav li fr.lteconsulting again to get the GAVs list (fr.lteconsulting is my projects package name, so I filter GAVs with that), choose one and let’s release it :

fr.lteconsulting:hexa.binding.samples:1.4-SNAPSHOT

The thing in the release is to have all direct and transitive dependencies released too. That’s what Pom Explorer checks. It then generates a change list to materialize your requirements.

Other use cases

Listing provided and referenced classes

You can ask which Java classes are provided and referenced by GAVs. That’s sometimes a useful information to have. Try those commands :

classes providedBy fr.lteconsulting:hexa.css:1.3

classes referencedBy fr.lteconsulting:hexa.css:1.3

Optimizing your project’s dependencies

Sometimes, you ask yourself “do I still need this and that dependency ?” but you are not very sure, and since you lack time to investigate, eventually the dependency stays in your project for a long time, causing of course maintenance issues sometime.

Let’s have Pom Explorer help us in the quest for the obsolete dependency.

garbage dep fr.lteconsulting:carousel:1.0-SNAPSHOT

This will give you something like that :

garbage

You can refer to the project documentation to find how to use those informations. But sure that it can help you give away those useless dependencies !

Other goodies : graphs !

Pom Explorer can do two other things to help you visualize your dependency graph :

  • export GraphML files so you can use them in another graph software (like yEd for instance).
  • display an interactive 3d graph

Exporting GraphML files

GraphML is an open format to describe graphs. With the graph export command, you can get graphml files of your working session. The program will create two files and display the links to them.

Those two files are corresponding to two graphs :

  • the dependency graph as usual,
  • and the dependency graph between the git repositories containing your projects. Sometime one git repository can contain multiple projects and a view of the dependencies at the repository level is useful in those cases.

This is the kind of picture you can get easily from editors like yEd :

graphml in yEd

Interactive 3D graph

Thanks to the WebGL standard which allows direct access to the 3D hardware on the running machine and thanks to libraries like three.js and ngraph.pixel, it is possible to display an interacive 3d graph. More over it is possible to customize the appearance of the graph to give account of different perspectives.

Type the graph command and click on the link. This will open another tab containing the living 3d graph of your projects. When focus is given to the 3d viewport, the W, A, S, F and arrow keys allow to move in the 3d space. On the right, there is a text area where you can edit some javascript callback to customize the graph appearance. You can also stop the moving of the particle with the checkbox at the bottom right of the screen.

3d graph page

It is not necessarilly useful, but sometimes it is relaxing to admire your work in the form of a living and moving graph !

Conclusion

There are many other functions in Pom Explorer, but they are for you to discover now. This tool finds easily its place in the daily workflow because of the functions it provides. The fact that one can run it locally or on a shared server allows to use it as you wish.

It is still in early development phase so many more functionalities could come up. On this subject, don’t hesitate submitting a little pull request on the GitHub repository

Pom Explorer is made with love by LTE Consulting


Arnaud Tournier, le 26-06-2015