Jenkins : External Resource Dispatcher

Plugin Information

No information for the plugin 'externalresource-dispatcher' is available. It may have been removed from distribution.

This plugin adds support for external resources in Jenkins. An external resource is something external
attached to a Jenkins slave and can be locked by a build, which thus gets exclusive access to it, then released after the
build is done. Examples of external resources are phones, printers and USB christmas trees.
Note that even though the 'b' fell off in the latest release, this plugin is still considered to be in beta stage.

Quick usage guide

  • Connect an external resource of some kind to a Jenkins node.
  • Create Metadata on the node for the ExternalResource.
  • Create a project in Jenkins. Click the Resource Selection Criteria check box and add a name-value pair that matches your ExternalResource.
  • Start a build of the project. Note on the node page that the external resource is locked to your build.
  • Start another build using the same selection criteria. Note that this build will wait for the first one to finish, due to no free resource.

Adding ExternalResources to a node

Configure the node and click the Metadata check box. Add an ExternalResourceTreeNode with an ExternalResource as a child.
An ExternalResourceTreeNode is a collection of ExternalResource objects and the only Metadata type that can contain ExternalResource objects.
Fill in Name, Id and Description and add any amount of other Metadata for the resource.


The External Resource metadata for a node.

Selection criteria

In any project, selection criteria can be added using the check box named "Resource Selection Criteria".
When the project is built, the plugin tries to find a slave among the slaves to run on (selected as usual via a label)
that matches the selection criteria. It does this by matching metadata values on the external resource defined
on the slave. If the plugin finds a match, it locks the resource to the build and starts the build on the matched slave.
Once the build finishes, the resource is unlocked. Multiple selection criteria can be added, the current "Simple Selection"
adds an implicit "and" between the criteria, so that all the criteria must be matched. We plan to add a more advanced
resource selection in the future.

Currently, only one selector type exists, the StringResourceSelection. This is extendable, so plugin developers
can add their own. The String selection criteria tries to match a String value to a metadata value exactly.


Selection criteria.

Locked resource

For a build with a locked resource, we expose all the properties of the resource as environment variables, with the names:
MD_EXTERNAL_RESOURCES_LOCKED_NAME_OF_THE_PROPERTY_IN_CAPS. These can be used to refer to the locked resource from build steps.

Administrator settings

To add ExternalResources to a node, write permission to the node is needed. The same goes for adding resource selection
on a project, i.e. write permission to projects is needed.

On the configure system page in Jenkins, under External Resource Dispatcher Configuration, a couple of new settings are available:

  • Release Key - not used yet, will be used for a "Release All" feature where all resources on a node can be released by clicking a button.
  • Reservation Time - the number of seconds to reserve an ExternalResource. Since we use a QueueTaskDispatcher, we can't say for sure that
    a build will run on a particular machine, so we reserve the resource and then lock it once it starts to run.
  • External Resource Manager - described in detail further down, this sets the default manager for the Jenkins master.
  • Admin Notifier File - the path to a file in which to log problems with the resource handling.

Under the hood

Here we describe the different parts of the plugin in greater detail, with more focus on implementation details.

Dependency to Metadata plugin

This plugin has a dependency to the Metadata plugin and won't run without it. We use new Metadata types (ExternalResource and ExternalResourceTreeNode)
to describe the external resources on the nodes.

External Resource Metadata

We have added two new Metadata types, ExternalResource and ExternalResourceTreeNode. Only an ExternalResourceTreeNode can contain ExternalResources.

ExternalResource

An external resource is based on a TreeNodeMetadataValue, so it can contain other MetadataValues as children.
Other than those, it should always contain the following properties:

  • Name - the name of the resource.
  • Id - a unique identifier for the resource.
  • Description - text describing the resource.
  • Enabled - true if the resource is up and running, ready to be used, false if not.

ExternalResourceTreeNode

This is also based on TreeNodeMetadataValue, but can only contain ExternalResources as children.
It is also the only type of node that can contain ExternalResources. The reason for this is the resource handling.
In addition to its children, an ExternalResourceTreeNode contains an ExternalResourceManager. All the children of the ExternalResourceTreeNode will use the same manager.

ExternalResourceManagers

An ExternalResourceManager is what takes care of the resource handling. It is responsible for locking and releasing the
resources. Currently, two manager types exist, NoopExternalResourceManager and ResourceMonitorExternalResourceManager.
Plugin developers can add their own managers.

NoopExternalResourceManager

The easiest way of getting up and running with the External Resource Dispatcher. The NoopExternalResourceManager takes
care of the resource handling all from within Jenkins. If other systems outside of Jenkins are using the same External
Resources, this manager should not be chosen since the locking/releasing of resources will not look outside Jenkins.

ResourceMonitorExternalResourceManager

The difference between the NoopExternalResourceManager and this one is that the ResourceMonitor expects something external to Jenkins
to take care of the resources, and communicates with the plugin through CLI or http requests. At Sony, we have a service running on the slave machines
that can communicate with the ResourceMonitorExternalResourceManager which handles the locking and releasing of our Android phones.
When a phone is connected via usb to a slave computer, it also automatically adds it to Jenkins via http.
The service will hopefully soon be released as open source as well.

Command Line Interface (CLI) and HTTP

There are some CLI and HTTP commands intended primarily to be used by an ExternalResourceManager but can be used by a human or other system as well.
Most of the commands give a response in JSON format for example

{type: "OK", errorCode: 0, message: "OK"}

for an ok response.
All HTTP commands are placed under http://yourjenkins/external-resource-httpcli
Below is a brief description of each command, you can look in the Jenkins CLI help for information about their parameters.

disable-external-resource

HTTP name: /disable
Disables a resource on the slave. The resource can no longer be booked by jobs until it is enabled again.
Requires that the user has the EnableDisable permission set for the node that the resource is on.

enable-external-resource

HTTP name: /enable
Enables a previously disabled resource.
Requires that the user has the EnableDisable permission set for the node that the resource is on.

expire-external-resource

HTTP name: /expireReservation
Expires the lease of a reserved resource making it available again to be used by a build. The build that reserved it in the first place will probably fail when it starts and tries to lock the resource.

lock-external-resource

HTTP name: /lockResource
Adds the lock information to the resource so that no build can use it untill it is released. This is to inform Jenkins that another external process has locked it instead of a Jenkins build.
Requires that the user has the LockRelease permission set for the node that the resource is on.

reserve-external-resource

HTTP name: /reserveResource
Similar to the lock but for the reserve info instead.
Requires that the user has the LockRelease permission set for the node that the resource is on.

release-external-resource

HTTP name: /releaseResource
Removes any lock and reservation from the resource making it available again for builds to grab.
Requires that the user has the LockRelease permission set for the node that the resource is on.

Integration with the Build Flow Plugin

There is an extension point for Build Flow Plugin that can be used for easy access to the resource manager from within the build flow script like:

def manager = extension.'externalresource-dispatcher'
manager.lock(...)

Change Log

Version 1.0b (released Nov 22, 2012)

Initial beta release

Attachments: