Manual Tests (LTS 1.509.x)
Test results:
- 1.509.1 / OK - test passed without any problem
- 1.509.1 / WARN - there was some problem, functionality doesn't do what is expected, but it doesn't cause failure of whole Jenkins or any other significant problems
- 1.509.1 / FAIL - cause failure of whole Jenkins or similar critical issue which affects all jobs (e.g. when plugin is installed, it's not possible to start Jenkins or save configuration of any job)
Installation, upgrade
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Install from RPM |
Install Jenkins from RPM package |
Jenkins is installed without error and runs |
FC 19, RHEL 6.4, openJDK7 |
1.509.1 / OK |
vjuranek |
Install from .deb |
Install Jenkins from Debian package |
Jenkins is installed without error and runs |
Debian 6 x86, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Install from war on Tomcat |
Install Jenkins from war by deploying on Tomcat |
Jenkins is deployed and runs |
RHEL6, openJDK7, Tomcat 7 |
1.509.1 / OK |
vjuranek |
JBoss AS |
Install Jenkins from war by deploying on JBoss AS |
Jenkins is deployed and runs |
RHEL6, openJDK7, AS 7 |
1.509.1 / OK |
vjuranek |
Winstone |
Copy war file to file system, start with "java -jar jenkins.war" |
Jenkins is deployed and runs |
Windows 8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Install plugins from update center |
Install some plugins from update center |
Selected plugins are installed |
Windows 8, Oracle JDK 7 |
1.509.1 / WARN |
mwaite |
disk |
Make some changes directly on disk and launch "Reload Configuration from Disk" |
Made changes are reflected |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
upgrade |
Launch "Restart when no jobs are running" after plugin upgrade/install |
Jenkins restarts when no jobs are running |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Install plugins from update center reports a failure on Windows 8 for the Subversion and Credentials plugin. The message is "Failed to dynamically deploy this plugin". The command window where I started the Jenkins job includes a stack trace which reports that the plugins\credentials\WEB-INF\lib\findbugs-annotations-1.3.9-1.jar file is being used by another process. Stopping and restarting Jenkins enables the plugins and no further stack trace messages appear related to that condition.
Confirmed JENKINS-17681 (symlinks incorrect on Windows) exists in long term support release candidate 1 on Windows 7 and Windows 8 with Oracle JDK 7.
Confirmed JENKINS-17677 (digest mismatch from update center) was detected in LTS RC 1 but was an infrastructure problem unrelated to the LTS RC. Issue is believed to be infrastructure related, not related to any issue in core code. When I replace the default update center with a direct reference to one of the mirrors (as described in the bug report), the update center worked as expected. Changed issue from FAIL to WARN.
Core functionality
Global configuration
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
JDK 1.6 auto install |
Add JDK 1.6, select "Install automatically" |
JDK 1.6 is installed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
JDK 1.7 auto install |
Add JDK 1.7, select "Install automatically" |
JDK 1.7 is installed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Add ant installation |
Add existing ant installation |
Ant installation is added |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Ant auto install |
Add Ant, select "Install automatically" |
Ant version is installed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Add maven installation |
Add existing maven installation |
Maven installation is added |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Maven auto install |
Add Maven, select "Install automatically" |
Maven is installed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Setup email notification |
Setup SMTP server, admin email and send test email |
SMTP is setup, test email is sent and delivered |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Copy job config |
Create job by copying existing on |
Job is created with config same as already existing job |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Freestyle jobs
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Create freestyle job |
Create freestyle job |
Freestyle job is created |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Build rotation |
Setup "Discard Old Builds" and specify number of build to be kept |
Old builds are deleted |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Parametrized build |
Setup parametrized build and check the parameter value |
The appropriate variable is setup and contains right value |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Disable build |
Configure project as diabled |
Project is diabled |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Concurrent builds |
Allow "Execute concurrent builds" in project config and schedule two or more builds |
Builds are executed concurently |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Tie project to a label |
Tie project to a label and schedule a build |
Build is executed on slave with given label |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
SVN checkout |
Configure SVN repository and schedule build |
Project is checkout from SVN |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Ant build step |
Add ant build step and configure ant targets |
Ant targets are executed and finish with success |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Maven build step |
Add maven build step and configure maven goals |
Maven goals are executed and finish with success |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Shell builds step |
Add shell build step and setup simple shell script |
Shell script is executed and finish with success |
Debian7, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Batch build step |
Add a Windows batch file build step |
Batch build steps are executed successfully |
Win8, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Archive artifacts |
Setup "Archive artifacts" post-build action |
Artifacts are archived |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Publish JavaDoc |
Setup JavaDoc publishing |
JavaDocs are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Publish JUnit results |
Setup publishing of JUnit results |
JUnit results are published |
Debian6, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Email notification |
Setup email notification and schedule build |
Notification email is sent and delivered |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Periodical build triggers |
Setup periodical triggers |
Build is launched in given time |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
|
|
|
|
|
|
Maven jobs
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Create maven job |
Create maven job |
Maven job is created |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Build maven project |
Schedule maven job |
Maven build succedded |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Matrix jobs
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Create matrix job |
Create matrix job |
Matrix job is created |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Combination filter |
Setup combination filter |
Only jobs which pass the filter are build |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Plug-ins
SSH slaves (0.25)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Connect ssh slave |
Connect ssh slave by providing login/passwd |
Slave is connected |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
SVN (1.45)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Simple checkout |
Setup SVN SCM and do checkout |
Sources are checked out |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Maven (1.509.1-SNAPSHOT)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Maven2 build |
Build maven2 project |
Build is executed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Maven3 build |
Build maven3 project |
Build is executed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Green balls (1.12)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Success results |
Success results |
Success results are marked as green |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Git (1.3.0, 1.4.0) + Git-Client (1.0.5, 1.1.2)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Simple checkout |
Add Git SCM and checkout project |
Project is checked out |
Debian7, Oracle JDK 7 |
1.509.1 / OK |
mwaite |
Confirmed that JENKINS-16941 (Git publisher cannot push from slave, can push from master) is still visible in those versions of the plugin.
Checkstyle (3.35, 3.36)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish check-style result |
Configure project to publish check-style result |
Results are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Cobertura (1.8, 1.9.2)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish cobertura results |
Configure project to publish cobertura results |
Results are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Findbugs (4.48, 4.49)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish find-bugs results |
Configure project to publish find-bugs results |
Results are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
PMD (3.34, 3.35)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish PMD results |
Configure project to publish PMD results |
Results are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Warnings (4.25, 4.32)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish warnings |
Configure project to publish warnings |
Warnings are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Sonar (2.0.1, 2.1)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Sonar analysis |
Setup sonar analysis |
Analysis is executed and published on sonar server |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Disk usage (0.18, 0.20)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Show values and graphs |
Enable "Show disk usage trend graph on the project page" on global config |
Disk usage number and graphs are shown on project page |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Reflect changes on disk correctly |
Made some changes on disk (e.g. delete workspace) and launch disk usage |
Changes on disk will be reflected in disk usage number |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Violations (0.7.11)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish violations report |
Setup findbug, pmd etc reports and violations reports |
Violations reports are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Analysis collector (1.35, 1.36)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Collects and show results |
Setup collection of checkstyle, findbugs, pdm and warnings results collection |
Results are collected and shown |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Dashboard view (2.5, 2.9.1)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Create dashboard view |
Create dashboard view and some portlets |
View as well as portlets are created and shown |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
HTML publisher (1.2)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Record HTML report |
Setup directory containing HTML report |
HTML report is published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Email-ext (2.28, 2.32)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Email notification |
Setup email notification and schedule build |
Notification email is sent and delivered |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Batch tasks (1.17)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Execute batch task |
Define batch task in project1, setup executions of this tast in project 2 and schedule a build |
Batch task of project1 is executed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Jira (1.35, 1.37)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Update JIRA issue |
Update JIRA issue which is related to changes contained in the build |
JIRA is updated and on build page is a link to appropriate JIRA |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Deploy (1.9)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Deploy on Tomcat 7 |
Build war app and deploy it on Tomcat 7 |
App is successfully deployed and running |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Dry (2.34)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish DRY analysis results |
Setup CPD analysis and publish analysis results |
Results are published |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Backup (1.6.1)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Backup Jenkins configuration |
Setup backup dir and run Jenkins baskup |
Once no builds run, backup is performed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
XUnit ()
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish testing tools |
Create Test Result Trend graph |
|
|
|
|
Build timeout (1.11)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Build time out |
Configure build with shorter time out that the duration of the build |
Build is aborted |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Scp publisher (1.8)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish artifact over scp |
Setup remote destination and artifact to be archived over scp |
Artifact is archived on remote machine in speficied dir |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
|
|
|
|
|
|
Gradle (1.22, 1.23)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Gradle build step |
Setup gradle build step |
Gradle build step is executed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Gradle auto install |
Add Gradle, select "Install automatically" |
Gradle is installed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Groovy (1.13, 1.14)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Groovy build step |
Setup Groovy build step |
Groovy build step is executed |
RHEL6, openJDK7, groovy 1.8.* |
1.509.1 / OK |
vjuranek |
Systyem Groovy build step |
Setup system Groovy build step |
System Groovy build step is executed |
RHEL6, openJDK7, groovy 1.8.* |
1.509.1 / OK |
vjuranek |
Groovy auto install |
Add Groovy, select "Install automatically" |
Groovy is installed |
RHEL6, openJDK7, groovy 1.8.* |
1.509.1 / OK |
vjuranek |
Audit trail (1.7, 1.8)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Log operations |
Setup audit trail plugin |
Operations like changes job config are logged in audit log |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Job config history (2.3, 2.4)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Record changes |
Record changes in job config |
Changes are recored and shown |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Nested view (1.10)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Create nested view |
Create nested view |
Nested view is created |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
IRC (2.22, 2.23)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Send IRC notification |
Setup IRC notification and schedule a build |
Notification is send to appropriate IRC channel |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Run jenkins bot command |
Run bot command on IRC channel |
Command is executed |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Workspace clean up (0.12, 0.16)
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Delete workspace before build |
Setup delete workspace before build |
Workspace is deleted before build |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Delete workspace after build |
Setup delete workspace after build |
Workspace is deleted after build |
RHEL6, openJDK7 |
1.509.1 / OK |
vjuranek |
Exploratory Tests
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Stress/Load Test |
Run Xenu link checker while other interactive tests are running, and while jobs are executing |
Performance slower, but still functional and reliable, memory growth reasonable |
Debian Squeeze x64, Oracle JDK7 |
|
|
Old JDK test |
Run Jenkins with Debian Squeeze default JDK (JDK 1.6.0_20) |
Performs as expected |
JDK 1.6.0_20 is not well behaved with Jenkins. A newer JDK 6 is needed. Since the JDK 6 provided with Debian Squeeze is not being updated, it is probably best to switch to the Oracle JDK 6 or Oracle JDK 7 on Debian Squeeze. A PPA installer for Oracle JDK 7 is available from launchpad and works quite well with Debian Squeeze. |
|
|
Test name