Created by Unknown User (olivergondza), last modified on Sep 02, 2015

 Test results

  • OK - test passed without any problem
  • WARN - there was some problem, functionality doesn't do what is expected, but it doesn't cause failure of whole Jenkins or any other significant problems
  • FAIL - cause failure of whole Jenkins or similar critical issue which affects all jobs (e.g. when plugin is installed, it's not possible to start Jenkins or save configuration of any job)

Acceptance tests

Most of the test has bean automates as acceptance tests. Instead of retesting manually, feel free to run the suite and report the results here.

Environment

LTS RC / result

Tested by

 

 

 

Installation, upgrade

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Install from RPM

Install Jenkins from RPM package

Jenkins is installed without error and runs

FC 19, RHEL 6.4, openJDK7


 

Install from .deb

Install Jenkins from Debian package

Jenkins is installed without error and runs

Ubuntu 14.04, OpenJDK 7
Ubuntu 14.04, Oracle JDK 8
Debian Testing x64, Oracle JDK 8
Debian 7 x64, Oracle JDK 7





 

Install from war on Tomcat

Install Jenkins from war by deploying on Tomcat

Jenkins is deployed and runs

Arch, openJDK7, Tomcat 7
Arch, openJDK7, Tomcat 6
AIX, IBM J9 VM 7 SR 7, Tomcat 7



JBoss AS

Install Jenkins from war by deploying on JBoss AS

Jenkins is deployed and runs

Arch, openJDK7, JBoss AS 7

 

 

Core functionality 

Freestyle jobs 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Batch build step

Add a Windows batch file build step

Batch build steps are executed successfully

 

 

 
Plug-ins 
Green balls

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Success results

Success results

Success results are marked as green

 


 
Sonar

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Sonar analysis

Setup sonar analysis

Analysis is executed and published on sonar server

 

 

 
Analysis collector

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Collects and show results

Setup collection of checkstyle, findbugs, pdm and warnings results collection

Results are collected and shown

pmd not tested


 
Jira

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Update JIRA issue

Update JIRA issue which is related to changes contained in the build

JIRA is updated and on build page is a link to appropriate JIRA

 automated - run manually

1.609.1 RC / OK
1.609.2 RC / OK
1.609.2 RC / OK (Jira plugin dropped support for the version ATH integrate against - unable to verify)  

ogondza
ogondza ogondza

Deploy

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Deploy on Tomcat 7

Build war app and deploy it on Tomcat 7

App is successfully deployed and running

automated - run manually

1.609.1 RC / OK
1.609.2 RC / OK
1.609.3 RC / OK

ogondza
ogondza
ogondza

Xvnc

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Build with xvnc

Tests should correctly use xvnc

Tests are passing

automated - run manually

 

 
Dry

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish DRY analysis results

Setup CPD analysis and publish analysis results

Results are published

 

 

 
Backup

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Backup Jenkins configuration

Setup backup dir and run Jenkins backup

Once no builds run, backup is performed

 

 

 
Scp publisher

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish artifact over scp

Setup remote destination and artifact to be archived over scp

Artifact is archived on remote machine in speficied dir

automated - run manually

1.609.2 RC / OK
1.609.3 RC / OK

ogondza
ogondza

IRC

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Send IRC notification

Setup IRC notification and schedule a build

Notification is send to appropriate IRC channel

 


 

Run jenkins bot command 

Run bot command on IRC channel

Command is executed

 

 

 
LDAP

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

LDAP test

LdapPluginTest in test-harness

All tests succeed

automated - run manually

1.609.1 RC / OK
1.609.2 RC / OK
1.609.3 RC / OK

ogondza
ogondza
ogondza

LDAP NoDocker Test

LdapPluginNoDockerTest in test-harness

All tests succeed

automated

 

 
Publish Over FTP

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Ftp test

Ftp testin test-harness

All tests succeed

automated - run manually

1.609.1 RC / OK
1.609.2 RC / OK

ogondza
ogondza

Subversion

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Subversion test

SubversionPluginTest && Subversion_Version154_PluginTest in test-harness

All tests succeed

automated - run manually


 
Active Directory

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

ActiveDirectory test

ActiveDirectoryTest in test-harness

All tests succeed

automated


 
Gerrit Trigger

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

GerritTrigger test

GerritTriggerTest in test-harness

All tests succeed

automated


 
Exploratory Tests

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Stress/Load Test

Run Xenu link checker while other interactive tests are running, and while jobs are executing

Performance slower, but still functional and reliable, memory growth reasonable

 

 

 

Git plugin

Explore multiple configurations of the Git plugin, including JGit and CliGit implementations, using ssh, https, and git protocol repositories.
Run tests with multiple platforms (Windows, Linux) and multiple git versions (1.7.1, 1.8,1.9, etc.), multiple git configurations (CLI, JGit), multiple providers (GitHub, BitBucket, local git repository), and multiple permissions (private repositories, public repositories).

Reasonable behavior, no detected Jenkins related regressions

 

 

 

Task Scanner plugin

Explore different configurations of the plugin, including file name patterns, exclude patterns, different task tags, status thresholds, different source file sets inside the workspace. Job has run on master and slave.

Reasonable behavior, graphical and textual results represent actual occurrences of tags, build status according to threshold settings, no detected Jenkins related regressions