Apache Maven is one of the most widely used software build systems today. If you have a codebase size of multiple hundreds of megabytes or even few gigabytes, then Maven will be the best option in terms of easiness of modularity and management of components.
However when you have a codebase of multiple hundreds of megabytes or few gigabytes, it leads to several issues even you build your system with Maven. One such pressing matter is build time for your codebase. If I explain it more clearly, when you change a single component, you need to build all the systems depending on that component. But currently maven does not support this dependency based component building. Therefore you need to build the whole codebase after each commit to your codebase. This means you build a set of components you are really never changed by your commit or which are never affected by the change you made, which is clearly a waste of time and resources. One way of reducing this wastage is, introducing Build Profiles. But still, this will build whole lot of components which never changed or affected by your commit. So you will understand build profile is not the ideal solution for this. Then what is the ideal solution for this issue? It is INCREMENTAL BUILDING support for the codebase.
What is Incremental Building?
Incremental Build means rebuild only the minimal set components of the source code after making a change to the source code. This means avoid rebuilding the complete codebase when a change made in a component and build only the required set of components which are,
1. Changed Component
2. The dependent components of the changed component.
Why we cannot simply use "mvn install"?
Let's say we have a component-parent with 2 child components, component-api and component-impl where component-api defines the API and component-impl contains the implementation of the API defined in the component-api.
component-parent
|- component-api
|- component-impl
As you can see in the above simple diagram, component-api is a dependency for the component-impl since component-impl implements the APi defined in the component-api.
Let's think of a scenario where we modify the component-api but not component-impl accordingly which should lead to a compilation error . Then we build the component-parent with "mvn install". When we do this what happens is, Maven will rebuild the component-api and it will skip building component-impl displaying "Nothing to compile - all classes are up to date" message. Hence the build is successful despite the compilation errors at the component-impl. This means, though we have modified the dependency of the component-impl, Maven only looks at the current component. Therefore using mvn install is not the solution for Incremental Build with Maven.
Is Maven useless for Incremental Building?
The answer is NO. Maven does support Incremental Building but the support which comes by default is useless and inaccurate since the inability of Maven to look at dependencies and identify the changes. Therefore when this issue is fixed, Maven supports incremental building.
Is there a solution which is already Implemented?
Yes. Lucky for us, there is a solution implemented called Maven-Incremental build plugin. This is a maven plugin for Apache Maven and we can use it to enable fully functional true incremental build for Maven.
How does it work?
Maven-Incremental build plugin executed in the first phase (Validate) of the default maven Lifecycle. What it does is, it verifies the current component source (src/main/java) /resources (src/main/resources) or it's dependencies are not being updated after last build. If there is a change at least in one of them, maven-incremental-build plugin automatically cleans the target so that it enforces to recompile the project.
If you think of the scenario, you will understand this is the perfect solution. If I explain it here, when we change the component-api, while building the component-impl, maven-incremental build plugin identifies that the dependency component-api is changed and so that the component-impl target is cleaned. Therefore the component-impl is recompiled and in the process, it will detect the compilation error.
How to enable the Maven-Incremental build for your project?
All you have to do is declare the maven-incremental build plugin in your root pom file.
<build>
<plugins>
<plugin>
<groupId>org.jvnet.maven.incrementalbuild</groupId>
<artifactId>incremental-build-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<goals>
<goal>incremental-build</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The above xml element declares the maven-incremental build plugin for your project. But you need to add the following section to your pom to enable automatic download of the maven-incremental build plugin from it's plugin repository.
<pluginRepositories>
<pluginRepository>
<id>repository.dev.java.net-maven2</id>
<name>Java.net Repository for Maven</name>
<url>http://download.java.net/maven/2/</url>
<layout>default</layout>
</pluginRepository>
</pluginRepositories>
After adding these 2 elements to your pom file. you are ready to use maven-incremental-plugin.
Is there anyone use this plugin? If Yes, How?
Yes. There are lot of people who uses this plugin to improve their productivity. WSO2 is one of them.
When WSO2 Carbon 3.2.0 release work started, introducing incremental build support to Carbon was one of the major task scheduled for that release. Hence I took the ownership of this task [1] and integrated the Maven Incremental Build plugin to WSO2 Carbon 3.2.0 build. Hence WSO2 Carbon now supports build using Maven Incremental builder plugin.
As I have explained previously in this post, First you need to add Maven Incremental plugin and java.net plugin repository to your root pom file. You can find how I have done it for WSO2 Carbon here [2].
Then you need to check whether all your maven modules are inherit from the root pom. This has to be done since there can be dependencies which are not directly extend your root pom file. This is the case for our WSO2 Carbon dependencies. These dependencies do extend their respective Apache product poms such as Axis2 parent pom is org.apache:apache:8. In that case you need to add those plugin repository entry and plugin entry to Axis2 pom as well. You can see how we have done it for Axis2 from here [3]. This is true for other dependencies such as ODE as well. You can see all our dependencies from here [4].
Everything other component in our WSO2 Carbon do extend our Carbon root pom. For example Carbon Core [5] inherits from Carbon root pom. Therefore we don't need to add Incremental plugin entries to it since Carbon Core inherits those plugins from root pom.
Now you are ready to use maven Incremental build plugin.
How to execute the Maven Incremental build?
Maven Incremental build plugin is mapped to the Validate phase of the default lifecycle [6]. Hence it is executed by default when you execute any phase which comes after Validate. Hence you can execute it with a maven command such as "mvn compile" or "mvn package" or "mvn install" or "mvn deploy" since all these phases (compile, package, install, deploy) comes after validate phase.
Since Incremental build handles the cleaning of targets, you need to avoid mentioning "clean" phase in your maven command.
Hence you can use "mvn compile" or "mvn package" or "mvn install" or "mvn deploy" commands to execute Incremental build.
Is Maven useless for Incremental Building?
The answer is NO. Maven does support Incremental Building but the support which comes by default is useless and inaccurate since the inability of Maven to look at dependencies and identify the changes. Therefore when this issue is fixed, Maven supports incremental building.
Is there a solution which is already Implemented?
Yes. Lucky for us, there is a solution implemented called Maven-Incremental build plugin. This is a maven plugin for Apache Maven and we can use it to enable fully functional true incremental build for Maven.
How does it work?
Maven-Incremental build plugin executed in the first phase (Validate) of the default maven Lifecycle. What it does is, it verifies the current component source (src/main/java) /resources (src/main/resources) or it's dependencies are not being updated after last build. If there is a change at least in one of them, maven-incremental-build plugin automatically cleans the target so that it enforces to recompile the project.
If you think of the scenario, you will understand this is the perfect solution. If I explain it here, when we change the component-api, while building the component-impl, maven-incremental build plugin identifies that the dependency component-api is changed and so that the component-impl target is cleaned. Therefore the component-impl is recompiled and in the process, it will detect the compilation error.
How to enable the Maven-Incremental build for your project?
All you have to do is declare the maven-incremental build plugin in your root pom file.
<build>
<plugins>
<plugin>
<groupId>org.jvnet.maven.incrementalbuild</groupId>
<artifactId>incremental-build-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<goals>
<goal>incremental-build</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The above xml element declares the maven-incremental build plugin for your project. But you need to add the following section to your pom to enable automatic download of the maven-incremental build plugin from it's plugin repository.
<pluginRepositories>
<pluginRepository>
<id>repository.dev.java.net-maven2</id>
<name>Java.net Repository for Maven</name>
<url>http://download.java.net/maven/2/</url>
<layout>default</layout>
</pluginRepository>
</pluginRepositories>
After adding these 2 elements to your pom file. you are ready to use maven-incremental-plugin.
Is there anyone use this plugin? If Yes, How?
Yes. There are lot of people who uses this plugin to improve their productivity. WSO2 is one of them.
When WSO2 Carbon 3.2.0 release work started, introducing incremental build support to Carbon was one of the major task scheduled for that release. Hence I took the ownership of this task [1] and integrated the Maven Incremental Build plugin to WSO2 Carbon 3.2.0 build. Hence WSO2 Carbon now supports build using Maven Incremental builder plugin.
As I have explained previously in this post, First you need to add Maven Incremental plugin and java.net plugin repository to your root pom file. You can find how I have done it for WSO2 Carbon here [2].
Then you need to check whether all your maven modules are inherit from the root pom. This has to be done since there can be dependencies which are not directly extend your root pom file. This is the case for our WSO2 Carbon dependencies. These dependencies do extend their respective Apache product poms such as Axis2 parent pom is org.apache:apache:8. In that case you need to add those plugin repository entry and plugin entry to Axis2 pom as well. You can see how we have done it for Axis2 from here [3]. This is true for other dependencies such as ODE as well. You can see all our dependencies from here [4].
Everything other component in our WSO2 Carbon do extend our Carbon root pom. For example Carbon Core [5] inherits from Carbon root pom. Therefore we don't need to add Incremental plugin entries to it since Carbon Core inherits those plugins from root pom.
Now you are ready to use maven Incremental build plugin.
How to execute the Maven Incremental build?
Maven Incremental build plugin is mapped to the Validate phase of the default lifecycle [6]. Hence it is executed by default when you execute any phase which comes after Validate. Hence you can execute it with a maven command such as "mvn compile" or "mvn package" or "mvn install" or "mvn deploy" since all these phases (compile, package, install, deploy) comes after validate phase.
Since Incremental build handles the cleaning of targets, you need to avoid mentioning "clean" phase in your maven command.
Hence you can use "mvn compile" or "mvn package" or "mvn install" or "mvn deploy" commands to execute Incremental build.
2 comments:
Which version of maven are you using? IIRC, Maven 3.0 addressed incremental building...
Hi,
The maven version I'm using is 2.2.x (2.2.1 to be exact).
I think they might have improved the native incremental build support in maven3... But still bit difficult to find a reference which explains how much they have improved it...
Post a Comment