Multiple SCMs Plugin

Skip to end of metadata
Go to start of metadata

Allows a job to check out sources from multiple SCM providers.

Plugin Information

Plugin ID multiple-scms Changes In Latest Release
Since Latest Release
Latest Release
Latest Release Date
Required Core
Dependencies
0.3
Jan 02, 2014
1.509
Source Code
Issue Tracking
Maintainer(s)
GitHub
Open Issues
Kevin Bell (id: kbell)
Usage Installations 2013-Apr 1584
2013-May 1783
2013-Jun 1890
2013-Jul 2104
2013-Aug 2202
2013-Sep 2354
2013-Oct 2615
2013-Nov 2732
2013-Dec 2785
2014-Jan 3078
2014-Feb 3350
2014-Mar 3668

Background

This plugin is more of a proof-of-concept than a robust and fully functional component. It does work in my particular build environment, and is meant to serve as a demonstration of what might be possible with more work. It was inspired by issue #7155 requesting multiple repository checkouts for Mercurial similar to what is possible with the Subversion plugin. It's currently implemented as a plugin, but if enough people find it useful, I think the idea would work better in the Jenkins core.

We are a small team and have been using Subversion which has been adequate for our needs. However, I wanted to experiment with distributed VCS systems, so I tried using Mercurial for a medium-sided update to one of our projects and enjoyed it very much. Everything went great until I got to the Jenkins build part. Our projects are small to mid-sized and are usually structured like:

/
  /project-code
  /inhouse-common-library
  /3rd-party-deps
     /lib1
     /lib2

The in-house library is built by an upstream job, but the 3rd-party dependencies are checked out as part of the build. With subversion, we could check each one out individually, and initially I thought this is what I wanted for Mercurial as well. However, quite often we have the entire source for the library checked in, but the build really only needs a single dll or jar. Since Mercurial doesn't support partial clones, it now seems better to leave these in subversion since they're rarely changed and we can check out just the parts needed during a build.

I googled for solutions, but the ones I found all seemed to have limitations compared to the working and adequate subversion solution:

  1. Script the dependency checkouts as part of the build. This would work and would be easy, but changes to third party libraries would not automatically trigger a build or be included in the change log.
  2. Have an upstream job to check out the dependencies. This could work, but it sounded tedious to set up and maintain, and would clutter the job list with lots of superfluous jobs.
  3. Mercurial forests/subrepos. The general feel I got was that the forest extension is not actively supported, and subrepos are not yet supported by the Mercurial plugin. Furthermore, this would require changes to how we structure our code. If we were a bigger shop, this would probably be worthwhile, but for me it seemed like more maintenance.
  4. Just leave everything in subversion and live with it. The easiest of all, but loses all the benefits of DVCS (however minimal for a team our size).

Usage

In the SCM section of the Jenkins job configuration screen, choose 'Multiple SCMs'. You'll then see a drop-down list of the available SCM plugins which can be configured similar to the way build steps are configured for a freestyle job. When chosen, each SCM plugin will present its normal set of configuration options.

Make sure each SCM checkout is done in a separate sub-directory of the workspace to avoid conflicts between the SCM providers. If using Mercurial, this option is hidden in the 'Advanced' section, so don't forget to configure it.

If changing the SCM provider for an existing job, I recommend wiping out the workspace.

Limitations

  • Currently tested only with Mercurial and Subversion plugins, as that is what I use locally.
  • Post-commit type triggers don't currently work (at least for subversion), so it is necessary to configure 'cron' type polling.
  • Repository browser configuration is also not supported in the current version.
  • I haven't tested any pre/post build tagging type operations, although they will probably work.
  • I also haven't tested master/slave configurations. This might work if the underlying SCM plugins support this mode of operation.

Implementation Notes

The implementation was easier than I originally expected, and learned a lot along the way. It basically serves as a proxy between Jenkins and existing SCM plugins. The job configuration panel uses a hetero-list similar to the build steps section. That way, all the configuration options are handled by the real SCM plugins. For the actual SCM functions, it pretty much just iterates over each configured SCM plugin and forwards the request. There is some messiness in dealing with changelogs, but it's not too bad.

Changelog

Version 0.2 (Jan 19, 2012)

  • Fix changelog parsing for subversion (and possibly others). Extracted log component had an extra newline at the top of the file which made parsing fail if the document contained an <?xml...> declaration.
  • issue #7155, issue #12298 Allow polling to work with multiple instances of a single SCM type (Thanks to Jesse Glick).
  • Implement getKind() to possibly allow other clients (such as NetBeans IDE) to parse the change logs (Thanks to Jesse Glick).
  • Add override of getModuleRoots() to return the union of all contained SCMs module roots.
  • For SCM implementations that add SCMRevisionState actions to the build, these are now correctly recorded in the build, so subsequent polling works correctly.
  • Forward requests for build environment variables to the contained SCMs. Fixes missing MERCURIAL_REVISION and possibly others depending on SCM plugins used.

Version 0.1 (Mar 08, 2011)

  • Initial release

Labels

Edit
plugin-scm plugin-scm Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Add Comment