Manual Tests (LTS 1.532.x)

Test results

  • OK - test passed without any problem
  • WARN - there was some problem, functionality doesn't do what is expected, but it doesn't cause failure of whole Jenkins or any other significant problems
  • FAIL - cause failure of whole Jenkins or similar critical issue which affects all jobs (e.g. when plugin is installed, it's not possible to start Jenkins or save configuration of any job)

Selenium tests

Most of the test has bean automates as selenium tests. Instead of retesting manually, feel free to run the suite and report the results.

Environment

LTS RC / result

Tested by

Arch / openJDK7

1.532.1 / OK
1.532.2 / OK
1.532.3 / OK*

ogondza
ogondza
ogondza

* Following tests are broken

features/script.feature - ScriptTest in acceptance-test-harness passes
features/scriptler_plugin.feature:62 - ScriptlerPluginTest#create_and_run_parameterized_script in acceptance-test-harness passes
features/scriptler_plugin.feature:74 - ScriptlerPluginTest#override_default_parameters in acceptance-test-harness passes

Installation, upgrade

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Install from RPM

Install Jenkins from RPM package

Jenkins is installed without error and runs

FC 19, RHEL 6.4, openJDK7


 

Install from .deb

Install Jenkins from Debian package

Jenkins is installed without error and runs

Debian 6 x86, Oracle JDK 7
Debian 7 x64, Oracle JDK 7
Debian 6 x64, Oracle JDK 7

1.532.2 / OK
1.532.2 / OK 1.532.3 / OK
1.532.2 / OK

mwaite
mwaite
mwaite

Install from war on Tomcat

Install Jenkins from war by deploying on Tomcat

Jenkins is deployed and runs

Arch, openJDK7, Tomcat 7

Arch, openJDK7, Tomcat 6

1.532.1 / OK
1.532.2 / OK
1.532.3 / OK
1.532.1 / OK
1.532.2 / OK
1.532.3 / OK 

ogondza
ogondza
ogondza
ogondza
ogondza
ogondza

JBoss AS

Install Jenkins from war by deploying on JBoss AS

Jenkins is deployed and runs

Arch, openJDK7, JBoss AS 7

1.532.1 / OK
1.532.2 / OK 
1.532.3 / OK 

ogondza
ogondza
ogondza

disk

Make some changes directly on disk and launch "Reload Configuration from Disk"

Made changes are reflected

Arch / openJDK7

1.532.1 / OK : added to jenkins-test-harness
1.532.2 / OK
1.532.3 / OK 

ogondza
ogondza
ogondza

Core functionality 

Freestyle jobs 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Batch build step

Add a Windows batch file build step

Batch build steps are executed successfully

 

1.532.1 / OK
1.532.3 / OK 

kohsuke
ogondza

Plug-ins 
Green balls (1.13) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Success results

Success results

Success results are marked as green

Arch / openJDK7

1.532.1 / OK
1.532.2 / OK
1.532.3 / OK

ogondza
ogondza
ogondza

Sonar (2.0.1, 2.1) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Sonar analysis

Setup sonar analysis

Analysis is executed and published on sonar server

 

 

 

Analysis collector (1.35, 1.36) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Collects and show results

Setup collection of checkstyle, findbugs, pdm and warnings results collection

Results are collected and shown

 

 

 

Batch tasks (1.17) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Execute batch task

Define batch task in project1, setup executions of this task in project 2 and schedule a build

Batch task of project1 is executed

Arch / openJDK7

1.532.1 / OK
1.532.2 / OK : added to selenium-tests
1.532.3 / OK 

ogondza
ogondza
ogondza

Jira (1.35, 1.37) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Update JIRA issue

Update JIRA issue which is related to changes contained in the build

JIRA is updated and on build page is a link to appropriate JIRA

 

1.532.1 / OK : added to selenium-tests
1.532.2 / OK
1.532.3 / OK

kohsuke
ogondza
ogondza

Deploy (1.9) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Deploy on Tomcat 7

Build war app and deploy it on Tomcat 7

App is successfully deployed and running

 

1.532.1 / OK : added to selenium-tests
1.532.2 / OK
1.532.3 / OK 

kohsuke
ogondza
ogondza

Dry (2.34) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish DRY analysis results

Setup CPD analysis and publish analysis results

Results are published

 

 

 

Backup (1.6.1) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Backup Jenkins configuration

Setup backup dir and run Jenkins backup

Once no builds run, backup is performed

 

 

 

XUnit () 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish testing tools

Create Test Result Trend graph

 

 

1.532.1 / OK
1.532.2 / OK : added to selenium-tests
1.532.3 / OK 

ogondza
ogondza
ogondza

Scp publisher (1.8) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish artifact over scp

Setup remote destination and artifact to be archived over scp

Artifact is archived on remote machine in speficied dir

 

1.532.1 / OK : added to selenium-tests
1.532.2 / OK
1.532.3 / OK 

kohsuke
ogondza
ogondza

IRC (2.23) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Send IRC notification

Setup IRC notification and schedule a build

Notification is send to appropriate IRC channel

Arch / openJDK7

1.532.1 / OK
1.532.3 / OK 

ogondza
ogondza

Run jenkins bot command 

Run bot command on IRC channel

Command is executed

 

 

 

Exploratory Tests

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Stress/Load Test

Run Xenu link checker while other interactive tests are running, and while jobs are executing

Performance slower, but still functional and reliable, memory growth reasonable

 

1.532.3 / OK

mwaite

Git plugin

Explore multiple configurations of the Git plugin, including JGit and CliGit implementations, using ssh, https, and git protocol repositories

Reasonable behavior, no detected Jenkins related regressions

 

1.532.2 / OK 
1.532.3 / OK

mwaite