Tutorial: Simple Continuous Integration
Continuous Integration
The goal with continuous integration is to have the individual developers committing new code to source control frequently and to integrate their changes with those of other developers many times a day. The role of AnthillPro3 in this environment is to try to identify when the integration has caused the quality of the project to degrade, so that corrective action can be taken quickly.
In this example, we will configure a simple project from scratch, configure it first with a rapid fire schedule and then discuss next steps to improve the process.
Configuring our basic build
We will use the basic Anthill-Example project in this tutorial. It is a tiny Java project but the principals are the same as for any project.
1) Configure the Source Repository
The first step is to tell AnthillPro3 where the source code is located. For this example, we will use the open source SCM tool Subversion. Click into the Administration Tab then choose Repositories and Create New Repository. Select the repository type appropriate for your project and configure it. For most repository types, you will not enter detailed information about the project or module here.
2) Create a schedule
We want to see if this project has been changed on a very regular basis so that we can check the status of the integration as constantly as possible. To do this, we should create a new schedule. From the Administration tab, choose the Schedules menu item.
We just want to create a basic schedule, so choose Create Schedule and select the Interval Schedule option. After naming the schedule, set the build interval to something quite short, perhaps three minutes. Also set a start time. For a rapid fire schedule like this, the start time does not matter. It is more useful for schedules that run less frequently, perhaps ones that run twice a day. That start time on a schedule with a twelve hour interval, would dictate when the first daily execution was, and the second one would happen twelve hours later. For a schedule that runs every three minutes, this level of control does not matter.
3) Create the new Project
To create a new project, go to the Administration Tab and select Projects. Then click the Create Project button. Select the default status, stamp style, artifact set and environment groups. Give the project a name.
4) Create a Source Configuration
Once the project is named identify the details of where the project is stored in source control. To do so, scroll down the page slightly and create a new Source Configuration. This Source Configuration will represent the mainline source for our project. Give it a name, a short description. A typical name would be something like Trunk or Branch 1.2. The work directory script, and change log rules are advanced features that can be left as defaults for this example.
Save that information and click the Create New Module button at the bottom of the page.
The modules are where you tell AnthillPro3 exactly where the source code can be found
in the repository. Many projects will only use one module, but some more complex projects
will use multiple modules. Enter the information requested for your repository driver. In
the example, we complete the Subversion URL to the project and identify a location where
tagged builds can be copied. The Tag and Revision fields are advanced fields used in
Source Configurations that expect to only be used to build specific baselines. We can
safely ignore them in this example. The directory offset will checkout this module into a
subdirectory relative to the working directory. This is useful if your project uses several
modules.
6) Stamping Strategy
Create a new stamping strategy. A stamping strategy dictates how a project is 'stamped'. This is a generic term for tracking build numbers, and similar identifiers.
Choose the default stamping template. This will take a version number and on every build, calculate the next version number fairly intelligently. That calculated number could be made available to build scripts, emails, and other participants in the system. If a workflow fails, this number will be rolled back. If it is successful, it will become the next version number.
If this numbering scheme doesn't work for you, you can create a new stamping template that has a custom algorithm for determining the build number.
Assign a name to this strategy as well as an appropriate version number for the main line of development for this project.
7) Create a Build Job
Create a new Job Configuration. A Job Configuration is essentially a list of steps that should be executed in order to accomplish something. Because we are trying to accomplish a standard build, select the Build Job option.
On the main tab for the build job, there are a number of controls, mostly dealing with how the build job should interact with source control. Choose the DEV stamping style, for now. In the workflow configuration, we will tell the system to use the stamping strategy we created previously for that style.
Since this example users Subversion and tags are cheap, we will set tagging to tag on every successful build. This will facilitate accurate rebuilds, a being able to retag the exact same source in promotions.
Setting cleanup to try will produce 'clean' builds. But will likely slow down the build process. Setting it to 'Yes' will produce clean builds that are certain to only contain artifacts from the checked out source, setting it to 'No' will provide better performance.
The change log status should be set to Successful. That will behave much as AnthillPro 2 did by default. The main use for making this selectable is to allow you to get the changelog since the last time a custom status was applied. Perhaps one indicating the build life was released to another group.
The change log status selection script is an advanced concept used mostly in complex environments. It calculates what status to use dynamically based on a beanshell script. Leave it blank for now.
Save the settings on the main tab, and go to the Builders tab. Here we will configure the scripts used to actually build the project. For the example, a simple Ant script will be used.
Create a builder of the appropriate type for your project. If you don't find the appropriate type listed, use the Shell Script builder to enter an arbitrary command line. This project uses Ant, so we will select and configure an Ant Builder.
After naming the builder, enter the location of the build script. This will be relative to the directory Anthill issued the SCM checkout command from. For some build systems, there will be a subdirectory named after the project or module, to include. Unless the directory offset option was used, this is not an issue for Subversion. Then enter the target name.
The Ant and JVM Configurations are interesting. What is requested is the location where Ant and Java are installed. If every agent you might run a build on, has these tools configured in the same location, a simple path can be entered here. More commonly, the location will vary between agents. So, AnthillPro 3 supports using a simple expression language to look up environment variables on the agent. The Ant environment variable is actually provided by AnthillPro 3 to target the version of Ant that ships with the product. To use your own copy of Ant, create an environment variable on each agent (either at those agents, or in the Agent sections under the System menu) with the path. The default options will work in this example.
The Show Additional Options item will display the Continuation Configuration, Timeout and Ignore Failures options. The Continuation Condition asks about which conditions we should run this step under. If any previous steps failed, running the build is probably pointless, so we will accept the default.
The Timeout option, will automatically fail the build and try to kill processes that take too long. This build should only take a minute or two, so enter 10 minutes for the timeout. If the build hangs for some reason, it will be killed off and failed automatically. Ignore Failures can be checked for processes that often fail but whose failure can be safely ignored for purposes of continueing with the process.
When we save the builder, we will be able to configure extra properties and environment variables that should be in place when it runs. Neither of these is needed for the example. They can be either hard coded or use simple script expressions like the Ant or JVM expressions do.
With the builder configured, go to the publishers tab. To keep the example simple, add only a Changelog Publisher. This will put a nicely formatted version of the change log in the build's artifacts (reports) tab. Configure the publisher and return to the project configuration page.
8) Configure the Agent Filter
Jobs are responsible for determing what machines that are allowed to run on. The Agent Filter tab for jobs plays this role. For a simple example, any machine in the build farm will work fine and we can leave the default 'Any Agent' filter in place.
9) Create a Workflow
We have now configured source repository types, stamping rules, jobs, activities and other items that do not know a lot about each other. In a workflow, we assemble these various pieces into something that can be run in a useful matter. Create a new workflow from the Example Project's configuration screen.
Since our workflow will be creating a new Build Life, it will be an Originating Workflow. Once the workflow has a name (Mainline Build Workflow), assign it a server group to build on. The Build-Farm group is a natural fit for a simple build. Also choose a notification scheme. If you have entered your email address and configured your mail server already, select the Default Notification Scheme, otherwise you can leave this blank for now and come back to it later.
This build does not use any resources outside of AnthillPro that need to be identified and reserved. Do not select a lockable resource. Each workflow is tied to a specific source configuration. That allows you to identify builds of the mainline as distinct from builds of some branch, while keeping your jobs as consistent as possible Select the Trunk source configuration created earlier. We also get a chance to assign the stamping style selected in the build job a stamping strategy. Assign the Dev stamping style to the Version Stamping strategy.
Save the basic workflow settings and edit the Workflow Definition below. Once in edit mode, add job after the Start item. This will be our build job. Then save the workflow definition.
The Properties tab makes it possible to list properties that must be set every time this workflow is executed. We do not need this. The Artifacts and Dependencies tabs are both tied to interproject dependencies and can be ignored in this simple example. Save the workflow and return to the project configuration page.
10) Add a trigger
Earlier we created a schedule. Now that we have a workflow that can be scheduled, we can go to the trigger tab on the workflow and add a schedule trigger using the schedule we set up earlier.
Run a build
From the Dashboard, we can navigate to the Example Project's Dashboard screen, by clicking
the name of the project. We can then manually trigger the build, or wait a couple of
minutes and let the schedule trigger the first build.
Next Steps
We have a nice basic build going and can see if there are problems with it. But we still don't have an ideal continuous integration system in place. Let's consider some options for improving this project.
Configure Notification
We need Anthill to notify the development team when the build fails so they can take correct action quickly. Configuring notifications was beyond the scope of this tutorial but is an important early step.
Switch to a Repository Trigger
The Scheduled Trigger spends a lot of time checking out the source and checking for revisions. We don't really need this to happen. We could convert the build to use a repository trigger and configure our SCM system to contact Anthill Pro3 whenever there are files committed. We could then build every time there is a commit without polling the SCM system.
Incorporate Testing
A full test suite that takes many minutes or hours to run, might make continuous integration less continuous if included in these quick builds, but a small testing suite would help identify a decrease in quality that the team should be made aware of.