Jenkins : Extending the CA APM Jenkins Plugin

Created by Unknown User (pramodanarase), last modified on Feb 14, 2019


Extending the plugin lets you add new sub commands to the existing plug-in to perform new tasks. To write your own custom comparison-strategy and output-handler or both, add the ca-apm-api-1.0.jar and ca-apm-core-1.0.jar files, in the class path of your project. These jars are available in repo directory in the Jenkins plug-in package. You can access the comparison-strategy contract and output-handler contract, and the helper utilities provided with this plug-in from these JAR files.

Helper Utilities

The utility helps you to write minimal code and get your custom comparison-strategy and output-handler.

MetricDataHelper

The MetricDataHelper utility lets you fetch build performance metric from CA APM. The MetricDataHelper uses the doCompare static helper method. This method uses the following three arguments:

agentSpecifier
Values is available in the StrategyConfiguration object

metricSpecifier
Values is available in the StrategyConfiguration object

build entity
Is provided to you in the doCompare comparison-strategy method

Syntax: MetricDataHelper.getMetricData(agentSpecifier, metricSpecifier, currentBuildInfo)

Example Implementation

BuildPerformanceData currentBuildPerformanceData = MetricDataHelper.getMetricData(agentSpecifier,metricSpecifier, currentBuildInfo);


FormulaHelper

Use the following methods in your custom comparison-strategy to process the build performance data:

  • getAverageValues(BuildPerformanceData buildPerformanceData)
  • getMaxValues(BuildPerformanceData buildPerformanceData)
  • getMinValues(BuildPerformanceData buildPerformanceData)
  • double getPercentageChange(double expectedValue, double actualValue)
  • public static AgentComparisonResult thresholdPercentageBasedCrossBuildMetricPathComparison(
  • double thresholdPercentage, Map<String, Double> benchMarkValues, Map<String, Double> currentValues)
  • public static AgentComparisonResult thresholdValueBasedCrossBuildMetricPathComparison(double thresholdValue, Map<String, Double> benchMarkAverageValues, Map<String, Double> currentAverageValues)

Syntax: FormulaHelper.getAverageValues(currentBuildPerformanceData);

Example Implementation

Map<String, Double> currentAverageValues = FormulaHelper.getAverageValues(currentBuildPerformanceData);


EmailHelper

The EmailHelper utility helps you to send email. Provide email configuration details in the Strategy.properties file. The EmailInfo entity lets you fill the email contents and attachments (if any). The basic fields are populated by default from the configuration. You can include more fields like message subject, body, and content-type.

Syntax: EmailHelper.sendEmail();

Example Implementation

EmailInfo emailInfo = EmailHelper.getEMailInfo();
emailInfo.setMessageContentType("text/html");
emailInfo.setMessageSubject("Build Performance Report for " + outputConfiguration.getCommonPropertyValue("jenkins.currentbuild"));
emailInfo.setMessageBody(htmlOutput);
boolean emailSendStatus = false;
try {
    emailSendStatus = EmailHelper.sendEmail();
 } catch (BuildComparatorException e) {
  JenkinsPlugInLogger.severe("Error occured while sending email ", e);
}


DataFormatHelper

The DataFormatHelper utility helps you convert your data to JSON or XML format. You can pass your custom StrategyResult object to get JSON or XML equivalent output of your result. You can export the output to your output-handlers. You can use the other helpers like EmailHelper to send email, or FileHelper to populate the data to a file.

Syntax: generateJSONOutputForStrategy(StrategyResult<?> strategyResult)

Example Implementation

String json = DataFormatHelper.generateJSONOutputForStrategy((StrategyResult<?>) strategyResult);

FileHelper

The FileHelper utility lets you check the availability of a file in a directory, create a directory, and export the output file.

Syntax: exportOutputToFile(String folder, String fileName, String content)

Example Implementation

FileHelper.exportOutputToFile(outputPath, "output.json", outputObject.toString())


 Custom Comparison Strategy

  • The ComparisonStrategy<T> is the contract to implement custom comparison-strategy. The return type of each comparison-strategy has to be of type T. You can define your own T and can implement your own output. You can refer DefaultStrategyResult for creating your own strategy-result of your defined type T. To convert your custom output to JSON or XML, use DataFormatHelper class.
  • The ComparisonStrategy<T> method is triggered automatically when the plug-in runs. The method provides you a comparison strategy attribute entity, which has complete information about the particular comparison strategy in question. You can create a local variable in your implemented class. To capture the properties you must execute the comparison strategy.
  • The customizing class name should be postfixed with ComparisonStrategy e.g., GCHeapComparisonStrategy and it should be defined in the package  com.ca.apm.jenkins.performancecomparatorplugin.comparisonstrategy
  • This method compares the performance of current build with the benchmark build.

param benchMarkBuild
Contains build number, start time, and end time of benchmark selected build number

param currentBuild
Contains build number, start time, and end time of the current build

return ComparisonStrategyResult
This object is returned, which contain detailed transaction to transaction metric comparison values

throws BuildComparatorException
ComparisonStrategyResult object is returned, which contain detailed transaction to transaction metric comparison values

public interface ComparisonStrategy<T> {
    public void setConfiguration(StrategyConfiguration strategyConfiguration);
    public StrategyResult<T> doCompare(BuildInfo benchMarkBuild, BuildInfo currentBuild)
            throws BuildComparatorException;
}


The strategy uses the average value of the transactions and compares one on one using threshold percentage. If the current build’s performance is greater than the expected result (benchmark build’s corresponding transaction) by threshold percentage, it is taken as slow transaction else a good performing transaction.


Sample Comparison Strategy 

  • Create a class implementing the ComparisonStrategy<DefaultStrategyResult> interface.    
  • Implement setConfiguration method, doCompare method.
  • Invoke getMetricData method of MetricDataHelper class to fetch the metric data of your application from em.
  • Apply customized implementation on BuildPerformanceData object of current, benchmark build.
  • Return result of type StrategyResult.


public class GCHeapComparisonStrategy implements ComparisonStrategy<DefaultStrategyResult> {
public void setConfiguration(StrategyConfiguration strategyConfiguration) {
this.strategyConfiguration = strategyConfiguration;
this.comparisonStrategyName = strategyConfiguration.getPropertyValue("name");
}
public StrategyResult<DefaultStrategyResult> doCompare(BuildInfo benchMarkBuild, BuildInfo currentBuild)
throws BuildComparatorException {
StrategyResult<DefaultStrategyResult> comparisonOutput = new StrategyResult<DefaultStrategyResult>();
try {
BuildPerformanceData benchMarkPerformanceData = MetricDataHelper.getMetricData(agentSpecifier,
metricSpecifier, benchMarkBuild);
BuildPerformanceData currentBuildPerformanceData = MetricDataHelper.getMetricData(agentSpecifier,
metricSpecifier, currentBuild);
Map<String, Double> benchMarkAverageValues = FormulaHelper.getAverageValues(benchMarkPerformanceData);
Map<String, Double> currentAverageValues = FormulaHelper.getAverageValues(currentBuildPerformanceData);
agentComparisonResult = FormulaHelper.thresholdPercentageBasedCrossBuildMetricPathComparison(
thresholdValue, benchMarkAverageValues, currentAverageValues);
Map<String, List<TimeSliceValue>> benchMarkSliceValues = FormulaHelper
.getTimeSliceGroupByMetricPath(benchMarkPerformanceData);
Map<String, List<TimeSliceValue>> currentSliceValues = FormulaHelper
.getTimeSliceGroupByMetricPath(currentBuildPerformanceData);
agentComparisonResult.attachEveryPointResult(benchMarkSliceValues, currentSliceValues);
} catch (BuildComparatorException e) {
JenkinsPlugInLogger.severe("An error has occured while collecting performance metrics for "
+ comparisonStrategyName + "from APM-> for agentSpecifier=" + agentSpecifier
+ ",metricSpecifier =" + metricSpecifier + e.getMessage(), e);
continue;
}
strategyResult.addOneResult(agentSpecifier, agentComparisonResult);
}
JenkinsPlugInLogger.fine("GC Heap Strategy comparison has been completed successfully");
return comparisonOutput;
}

Add the following configuration for this custom strategy in the performance-comparator.properties file, and strategy name to metric.list. For more details, see List of Available Comparison Strategies section in the CA APM Jenkins Performance Comparator Plug-in page.

metric.list=gcheapstrategy
gcheapstrategy.threshold=2
gcheapstrategy.agentspecifier=.*
gcheapstrategy.metricspecifier=.*GC Heap:Bytes In Use
gcheapstrategy.comparator=GCHeap
gcheapstrategy.outputhandlers=plaintextemail, jsonfilestore,chartoutputhtml

Custom Output Handler 

  • The outputHandler class implements OutputHandler<T> interface which is the contract to implement custom outputhandler.  
  • Create the customized output handler class in the package com.ca.apm.jenkins.performancecomparatorplugin.outputhandler.
  • The output handler class name should be postfixed with “OutputHandler” and implements OutputHandler<T> interface


Sample Output Handler


public class JSONFileStoreOutputHandler implements OutputHandler<StrategyResult> {
private OutputConfiguration outputConfiguration;
public void setOutputConfiguration(OutputConfiguration outputConfiguration) {
this.outputConfiguration = outputConfiguration;
}
public void publishOutput(List<StrategyResult> comparisonStrategyResults) throws BuildComparatorException {
String outputPath = outputConfiguration.getCommonPropertyValue("output.directory");
JSONObject outputObject = new JSONObject();
JSONArray resultsArray = new JSONArray();
outputObject.put("strategy_results", resultsArray);
for (StrategyResult<?> strategyResult : comparisonStrategyResults) {
String json = DataFormatHelper.generateJSONOutputForStrategy((StrategyResult<?>) strategyResult);
JSONObject strategyJson = new JSONObject(json);
resultsArray.put(strategyJson);
}
FileHelper.exportOutputToFile(outputPath, "output.json", outputObject.toString());
}
}



Add the following configuration for this custom strategy in the performance-comparator.properties file, strategy name to outputhandlers.list. For more details, see List of Available Output Handlers section in the CA APM Jenkins Performance Comparator Plugin page.

outputhandlers.list=json
json.outputhandler=JSONFileStore


Logging for Custom Implementation

To implement custom comparison strategy or output handler or custom load runner data reader, add logging to your code. This logging is redirected to a file comparison-runner.log file which is present in <current_build>/workspace directory. The logging utility class is JenkinsPluginLogger. The available methods are:
severe, warn, info, fine, finer, finest
severe has overloaded method with argument to exception to print the detailed stack trace for better understanding of the problems.

For Example: JenkinsPluginLogger.severe("An error occurred while fetching performance metric data from APM", ex);JenkinsPluginLogger.info("Email is sent successfully");

The logging configuration is available at performance-comparator.properties file.

Deploy the Custom Strategy and Custom Handler

After you create custom strategies and output handlers or both, package the artifact as JAR. Place the packaged JAR in the extensions directory that is configured as extensions.directory in the configuration file.

Ensure that you add all the required custom strategies or ourput handlers or both in the configuration file.

For Example: extensions.directory=C:\\APM\\AutomicJenkins\\Jenkins\\Jenkins Server\\extensions\\