AnthillPro Blog

Webcast: The Role of Binary Repositories in Software Configuration Management

Thu, 11 Mar 2010 16:33:00 GMT

Broadcast Date: Mon., Mar. 15, 2010 at 1:00 p.m. EDT
Speakers: Jeffrey Fredrick, Urbancode Technical Evangelist; Eric Minick, Urbancode Lead Consultant
Register Now

How confident are you with the content of what you are about to release? Do you know every change since the last release? Can you reproduce the release at will? Many of the teams we meet struggle with these fundamental issues of Software Configuration Management.

The most successful teams solve these problems by leveraging a binary repository or -- in ITIL terminology -- a Definitive Software Library (DSL). By providing an authoritative source for release artifacts, a well-managed binary repository can help address issues of compliance, traceability, and audit. Combined with dependency management, a binary repository provides a valuable starting point for impact and risk analysis.

But the benefit of using a binary repository is also felt earlier in the software-development lifecycle: By enabling effective component-based development, a binary repository can enable faster build times, facilitate more efficient collaboration between teams, and reduce the learning curve for new developers.

For software-configuration-management professionals who are concerned with audits or improving the efficiency of the software-development lifecycle, Eric Minick and Jeffrey Fredrick invite you to join them for this presentation where they will address:

traditional approaches to controlling binary artifacts, including their shortcomings;
essential elements of well-managed binary repository and inter-project dependency management; as well as
patterns for adopting a binary repository and implementing dependency management.

Maven's Strengths and Weaknesses as a Dependency Management System

Fri, 26 Feb 2010 16:07:13 GMT

For many Java teams, switching to Maven will introduce them to formal dependency management. Maven actually does a pretty decent job and is a fantastic system out of the box for informal projects. However, for teams looking to implement rigorous component reuse policies, Maven falls short.

Last month, in our article Chaperoning Promiscuous Software Reuse, we reviewed four elements of a successful strategy for managing components. Briefly, these elements are:

Testing and Validation of 3rd party components Only libraries approved for reuse should be used by developers. If developers want to use additional libraries, they can submit them for screening.
A Definitive Software Library: A team should have an official storage place for reusable, versioned components (both internal and 3rd party).
Using Protection: A team should have an "immune system" that rejects libraries other than those from the DSL.
Rapid impact analysis audit When something goes wrong, it should be easy to determine what versions of which components were used by the project.

Basic Maven

Basic Maven points to a common Maven repository hosted centrally. Various open source projects publish their releases to this repository and a developer can configure her build file to depend on these releases. At build time, the artifacts are automatically pulled down into the workspace and the build can produce a report of what was used.

This is actually a pretty good start. If the build script depends on exact versions, the team has some traceability. The Maven repository is an official repository of artifacts (a DSL) used by the team although the team does not control it. Unfortunately, there's little protection against a developer using a rogue component, but it's easier to do the right thing than the wrong thing, and that goes a long way.

The major failing here is that the team does not control the repository. Rather, it is an arbitrary resource in the cloud. This eliminates the team's ability to test artifacts prior to including them in the DSL, and there's no guarantee that an artifact used by a production release today will still be available from the repository tomorrow unaltered and intact.

Bring the Repository in House

What's needed is to bring the repository in house. While this can be done without special tooling, there are a handful of decent Maven repositories that provide the team more control. Nexus and Artifactory are two of the major choices here. They provide security over who can publish artifacts to the repository as well as more sophisticated storage and indexing than a simple file system.

With one of these repositories, the team has it's own DSL they control and can ensure that only approved versions of libraries are made available to the team. Again, as long as the team does not use snapshot dependencies a basic audit trail of what is in a build is available. Impact analysis reports indicating which projects have used a given component version are a little more difficult. Protecting the code-base from a developer checking an arbitrary library into source control is still a manual process, but Maven does discourage the practice by making reuse of a library from the repository more straight-forward.

With an in-house repository, a team is able to extend Maven nicely. Dependency management is controlled by developers (for better or worse) but dependencies must come from an approved collection of versioned components controlled by the team. These libraries can be tested prior to being made available. Protection and audit capabilities are less than ideal but not wholly absent. For many teams, this kind of system may be adequate. The most disciplined are likely to find it wanting.

Other Concerns with Maven Dependency Management

No ability to depend on the newest version with traceability

Maven draws a strict line between depending on a released version of a component and depending on the latest built version. Depending on the latest build version is a common practice in multi-component continuous integration. Maven's strict line allows for either approach to be used, but only provides traceability when dependencies are based on releases. In multi-component CI, teams often do not want to release particular components for reuse upstream, but rather consider every successful CI build (with passing unit tests) a mini-release.

Functional tests performed at a system level determine the quality of the entire dependency graph (the application). In these cases, it's only after testers validate a system build, that a release will be pushed out (ideally the binary that was tested) and full traceability to every component is valuable. That Maven discourages mutli-component CI through the arbitrary separation of Releases and Snapshots is unfortunate.

Inability to depend based on status

Sometimes, teams don't want to depend on either the latest build of a component nor a specific version. Rather, they want to depend on the latest version that has passed some sort of quality gate. Perhaps a validation from a testing team, or approval from a release manager. Maven has no real concept of statuses on components (version numbers and component names are pretty much it) so status based dependencies are out of the question.

2 Talks At ESDC

Sat, 20 Feb 2010 23:27:00 GMT

I'm traveling to Atlanta this week but the first week of March I will be back in the San Francisco Bay Area for a new conference, ESDC, the Enterprise Software Development Conference. While this is is a new conference the timing is familiar; ESDC has taken over the traditional calendar-slot of the now defunct SD West. If you’re interested in attending you can use my last name as a discount code when you register to get $100 of the full conference pass. You can also register to attend the expo for free as long as you register by February 28th.

I'll be there in two different roles. First is as Urbancode "booth babe" showing off AnthillPro on the Expo floor Tuesday and Wednesday. I’ll also be giving two talks, both on Wednesday. I start the day bright and early with #605 Going Lean: Slash Waste with Build & Deployment Automation at 8:15 am, and finish the day at 4 pm with #906 Creating Habitable Code: Lessons in Longevity from CruiseControl. Lean software development has been a hot topic for me lately, as members of BayXP and the Silicon Valley Agile Meetup know, while practices for creating long-term sustainable code has been a long term sustained interest. I’ve delivered the Habitable Code talk before but this’ll be the first time doing it solo, without Paul Julius by my side.

AnthillPro Security for Auditors

Wed, 17 Feb 2010 08:26:00 GMT

Eric Starr over at Anthillizer provides a how-to for creating an Auditor Role in your AnthillPro. He suggests default "read" or "view" permissions across the board but no execute permissions. Seems like a reasonable strategy for most teams.

Anthillizer is on a bit of a security kick recently and provided some cheat sheets for security settings.

Configure a Multiplatform Build in AnthillPro

Mon, 15 Feb 2010 15:16:12 GMT

A common build scenario is to need to compile on several different platforms. The basic build is all the same, but a couple parameters change platform to platform. Here we look at the basics of configuring this kind of scenario in AnthillPro.

You need:

A build job that is parameterized
A workflow that iterates over that build job for each platform
An agent filter that directs each iteration to the appropriate platform
Properties on the iteration that are the build job parameters and inform the filter

Below, there's a 6 minute video providing an overview of this process.