export LANG=en_US.iso88591
The journey of one man's pursuit of love, happiness, and a working build.
The following describes a high level overview of the steps taken to build a community release. The sections following describe in detail the steps that go into each task and some developer notes along the way.
Setup the build environment (JDK, Maven, Git, Database)
Checkout and branch the RHQ source
Make any release-specific modifications to the branch
Run a test build and unit tests
Package the release, including setting the release version as well as the snapshot version for the future
Cherry pick the pom files with the new versions back to master
Final testing and upload
Throughout the following sections, relative time estimations have been added to provider the builder with a rough idea of how to fit the build in with other tasks.
Version numbers for releases take the following format: "X.Y.Z.BWW" The X.Y.Z portion refers to the actual version of RHQ being developed. The BWW indicates this is a community build, where the B is a literal (it stands for "build" in case you were curious). For example: 3.0.0.B02 refers to the 3.0.0 version, build #2, or the second community release of the 3.0.0 version.
Time Investment: Short
The longest time sink is likely downloading the required packages necessary to build; their configuration is quick.
Before getting started, make sure the machine and user performing the release are configured with commit access to the RHQ git repository. This basically means to make sure the user's SSH keys are configured with write permissions on the RHQ repo. The release build process will automatically tag and commit versioned files and will fail if the pushes cannot be done.
Also the git user must be configured in settings.xml <username/> tag.
I personally like to do the builds from a clean machine or guest. Part of these instructions involve removing your existing maven repository, but I prefer the extra level of ensuring the build is clean and repeatable. Besides, the builds can take a while, so offloading it to a guest kept the build process as noninvasive to my other tasks as possible.
The following software is required in order to build a community release. These are the same as found in the Building RHQ page, so be sure to take a quick look there for any updates to versions required.
Java Development Kit - There is currently no requirement on using either Sun's JDK or OpenJDK. JDK 6 is required to build the CLI, so for a complete build, version 6 is required.
Maven - Version 2.1 or greater.
Git
PostgreSQL
Once the above packages are installed, the following environment variables must be set to point to their respective installations:
JAVA_HOME
M2_HOME
Set the LANG environment variable to "en_US.iso88591" to ensure correct handling of umlauts and potentially other characters:
export LANG=en_US.iso88591
To avoid having to do this step every time you cut a release, add the above line to either /etc/mavenrc or ~/.mavenrc.
If the build is being created on a machine that has previously build RHQ, it's best to clean out the maven repository. This will ensure that the build isn't accidentally picking up an old or uncommitted artifacts.
rm -rf ~/.m2/repository mkdir -p ~/.m2/repository
A database is required by the build process. The easiest solution is to use the build default database configuration, which is summarized below. Again, for more explanation, see the PostgreSQL quick start installation section of this wiki.
Database: rhq
User: rhqadmin
Password: rhqadmin
Make sure postgres is configured to allow password authentication from localhost (which is not the default after installation).
Time Investment: Short
The actual checkout itself is a pretty quick operation.
The only noteworthy point of checking out the source is to make sure to use the committer URL when checking out the source (as compared to the anonymous access). The URL is as follows, but more information can be found at the Source Control wiki page.
ssh://git.fedorahosted.org/git/rhq/rhq.git
For reference, the clone command therefore looks like:
git clone ssh://git.fedorahosted.org/git/rhq/rhq.git
Again, the user doing the build must have its private SSH key registered with fedorahosted and be set up with commit access to RHQ.
Time Investment: Trivial
Creating a branch for the release is a cheap (as in time and resources, not as in poor) way of freeing up master (and more importantly, not having to coordinate a long running code freeze) to continue development without interfering with the release build.
Once master contains all of the code to be included in the build, a remote branch is created and switched to with the following commands:
git push origin master:release-X.Y.Z.BWW git checkout -b release-X.Y.Z.BWW origin/release-X.Y.Z.BWW
See the note in the Overview section above for more information on the version number format.
Time Investment: Short
Now that the release has its own branch, any changes that need to be made specifically for the release can be made directly in the codebase without disrupting other work. These changes will likely focus around removing maven modules from inclusion, such as those deemed not ready for public consumption.
When disabling plugins, the following describes the pom files that must be edited depending on the type of plugin.
Server
Comment out the module reference in enterprise/server/plugins/pom.xml
Agent
Comment out the module reference in modules/plugins/pom.xml
Comment out the dependency in server/ear/pom.xml (note: this dependency may not exist in all cases)
Once these changes are made, they must be committed to the branch before proceeding. The maven release plugin will make sure the local repo has no outstanding changes prior to running and will fail if it finds any.
git push origin release-X.Y.Z.BWW
Time Investment: Trivial
You'll need to copy the project's maven settings file (etc/m2/settings.xml) to the user's <home>/.m2 directory in order to pick up some required settings.
Ensure the line that enables the dev profile is disabled.
<!-- <activeProfile>dev</activeProfile> -->
Time Investment: Long
The first time this is run, all of the dependencies need to be downloaded. Regardless of whether the build is successful or fails, this can take between 20 and 30 minutes to run. Subsequent builds, such as if the first one fails or during the maven release plugin, are considerably quicker, on the order of around 5 minutes.
Now that all of the initial configuration is in place, it's time to build the source and run the unit tests.
$M2_HOME/bin/mvn install -Penterprise,dist,prod -Dmaven.test.skip=false -Ddbsetup
This step tends to be an iterative process as any unit test failures are shaken out. Keep at it until a build has run with all tests passing.
Time Investment: Long
While this step performs a full build, it already has the dependencies downloaded from the previous step. However, there are a number of extra steps that take place in addition to the build. Each run of this should last around 10 minutes.
This step requires an active internet connection as the maven plugin will tag and commit files to git.
This is the main step in what differentiates a normal build from a release. In this step, we delegate to maven to handle building with the correct version of the release. In the process, maven will do some (albiet unnecessary) tagging in git for the release as well.
When the release task is run, early in the process it stops to request input from the user. The following are requested:
Version of the release (i.e. X.Y.Z.BWW from above)
Version of the next release (for a community release, this is likely just X.Y.Z-SNAPSHOT, since we often won't be incrementing the Z value between community releases)
Tag name to tag the release with (assuming it's not passed in with -Dtag=)
The following is a rough outline of what is done by the maven release plugin. It may do more; this is just what I could discern based on various failed attempts.
Make sure the current branch is up to date. If there are changes that have yet to be committed, the task will fail.
Increment the version tag in all of the modules in the reactor. At this point, you'll notice local changes to (almost) every pom.xml file. The maven plugin will later commit these.
Run a new build. At this point, we indicate maven.test.skip=true since we've run the tests in a previous step.
Commit the changes to the pom.xml files.
Push the latest commits.
Tag the current state of the repo.
Change all pom.xml files in the reactor to the version for the next release.
Commit the changes to the pom.xml files.
Push the latest commits.
Any failures in the interactions with git cause the entire build to fail. The sorts of errors I hit include:
A tag with the given name already exists.
The push fails because one or more of the local branches failed a push (i.e. changes have taken place in master that are not reflected in the local master branch)
The version at the start of the build does not end with -SNAPSHOT. For some reason, maven refuses to update all of the versions to the release version if the current version isn't a snapshot. This caused problems when a previous attempt failed after updating and committing all of the pom.xml files to the release version (I had to repeatedly revert the commit that made these changes).
Start with a dry run of the process to shake out any immediately apparent issues that may arise.
$M2_HOME/bin/mvn release:prepare -Penterprise -Dpackage-connectors -Dexclude-webdav -Dresume=false -X -e \ "-DpreparationGoals=clean install -Penterprise,dist,prod -Dpackage-connectors -Dexclude-webdav \ -Dmaven.test.skip=true" -Dtag=RHQ_X_Y_Z_BWW -DdryRun=true
From the line, there are two things of note:
Be sure to change the -Dtag= argument to reflect the release being built.
The -DdryRun means no changes will be committed to git.
If you were to do a git status after any call to release:prepare, you'll notice a lot of changes. Depending on how far the task got before failing, you may see some changes to pom.xml files with the version of the release. You'll probably also see a number of temporary files that have been created. Between each call to release:prepare, it's important to run a clean to reset the environment.
$M2_HOME/bin/mvn release:clean -Penterprise
Assuming the dry run ran successfully, repeat the above command, letting the changes be committed to the live repo.
$M2_HOME/bin/mvn release:prepare -Penterprise -Dpackage-connectors -Dexclude-webdav -Dresume=false -X -e \ "-DpreparationGoals=clean install -Penterprise,dist,prod -Dpackage-connectors -Dexclude-webdav \ -Dmaven.test.skip=true" -Dtag=RHQ_X_Y_Z_BWW
As with the dry run, be sure to update the -Dtag argument with the appropriate tag name. Keep in mind that this tag cannot already exist in the git repository. The entire release:clean task will fail if this tag cannot be created. That said, if the task fails later, the tag will still be created. So if an attempt at release:prepare fails, it may be necessary to delete the tag from the remote git repo before attempting this again. See git documentation for more details on deleting tags, it's kind of wonky to delete it from the remote repo in addition to your local repo.
Deleting a tag isn't as straight-forward as I would have thought. More specifically, local changes to tags aren't picked up in a normal git push. I used the following sequence to delete tags on a remote repo:
git push origin :refs/tags/<tag-name>
Time Investment: Short
The maven release plugin takes care of setting the version for the next release (e.g. 3.0.0-SNAPSHOT). The problem is, since we've done the build in a branch, these changes aren't reflected in master.
Don't try to merge the release branch into master. That defeats the purpose of branching and changing the build for release purposes. Rather, cherry pick the change to the pom.xml files that sets the version to the next snapshot.
git checkout master git pull --rebase git cherry-pick <commit-sha>
The maven plugin will only update the version for modules currently in its reactor. There are a number of modules that are disabled by default that will thus be left at the previous version. The simplest way is to run the following command to pick up any remaining pom.xml files that haven't been updated.
find -name 'pom.xml' | xargs sed -i 's/OLD/NEW/g'
In the above command, OLD refers to the previous snapshot before the release and NEW refers to the version maven set for the next release (indicated by the user at the start of the release plugin task). For example:
find -name pom.xml | xargs sed -i 's/1.4.0-SNAPSHOT/3.0.0-SNAPSHOT/g' git commit -a
After the maven version changes have been cherry picked and the remaining modules manually updated, don't forget to push the commits to master.
git push origin master
If you had requested a code freeze at the start of the maven release plugin run, it's now safe to allow master to be open again.
Time Investment: Variable
The community release QA process is still a work in progress. Work with someone from the RHQ team to determine what tests should be run against the build before officially declaring it a release.
At very least, install the server against a new database and start the agent, connecting it to the server. The most basic smoke test is to ensure the agent can connect and introduce resources into the server.
Time Investment: Long
RHQ hosts its files on its sourceforge project (https://sourceforge.net/projects/rhq/). RHQ releases are each in their own folder under All Files > rhq > rhq-X.Y.Z.BWW
Run the following command to tell the release plugin to delete all its temporary files:
$M2_HOME/bin/mvn release:clean -Penterprise
Time Investment: Very Long
Putting out a release, even something as frequent as a community release, is an awesome deal. E-mail the RHQ mailing lists, blog, tweet, and appreciate the fact that you've introduced good karma into the universe.