Jenkins : Rationalising parallel

Problem Statement 

Sophisticated users of Pipeline want a more usable way to visualize and interact with their Pipelines. Blue Ocean cannot visualise pipelines in a meaningful way to users where there are a large number of stages or parallels.

Goals

• Reduce unnecessarily visualised parallel branches by reserving parallel syntaxes for visualisation and non-visualisation purposes.
• Increase developer satisfaction by making the rules of Pipeline visualisation more consistent and the visualisation usable at any scale.

Types of parallel 

Parallel Stages

In this usage each parallel branch has meaning in the stage graph and should be visualised. Usually there are very few branches and no nesting is allowed.

We introduced this syntax in Declarative Pipeline 1.2 but it is not available in Scripted.

stage('foo') {
    parallel {
        stage('Firefox') {
            steps {
                echo "First branch"
            }
        }
        stage('Chrome') {
            steps {
                echo "Second branch"
            }
        }
    }
}

Parallel Execution

Parallel execution is used to speed up the execution of the pipeline by having different parts run in parallel. They have little or no bearing on the stage flow graph and in generally not useful to visualise. These executions can be nested within each other and tend to have many branches.

Jenkinsfile example

stage('foo') {
    parallel(
		'copy-1': {
			... copy some files ...
		},
		'copy-2': {
			... copy some files ...
		},
		'some other process': {
			 parallel(
				'branch-1': {
					... run process 1 ...
				},
				'branch-2': {
					... run process 2 ...
				}
			)
		}
	)
}

Proposed

Parallel Stage syntax from Declarative is made available for scripted Pipelines. The visualisation is updated so that only parallels using the Parallel Stages syntax are visualised. Parallels using the Parallel Execution syntax are collapsed into a single stage and their steps are shown in order of execution.

Rollout phases

Phase 1

Warnings are given on the Blue Ocean UI for Pipelines using the Parallel Execution syntax. Users are told to use the Parallel Stage syntax if they want to continue visualising these parallels.

Users can switch off warnings and move to the new parallel visualisation rules by passing a parameter to Jenkins on startup.

Phase 2

Warnings are removed from Blue Ocean and the new parallel visualisation is enabled by default.

Related issues

• JENKINS-44820 - Parallel branches with no stage should not be displayed in Blue Ocean Resolved  - discussion where user wants some parallels branches but not others visualisable in the same parallel statement
• JENKINS-47799 - Blue Ocean shouldn't visualize the parallel keyword of a scripted pipeline directly Open  - discussion where user has come to this conclusion theirselves.