Jenkins : 2.0 Test Fest

When

April 14, 2016.

Resources

We're hanging out on the #jenkins IRC channel on the Freenode network. You can also join the Google Hangout

The current release is rc1 which can be downloaded directly

Filing issues

When filing issues in the Issue Tracker please use the testfest label so we can see all the filed issues together in this filter

Scenarios

The following are a list of scenarios/cases that should be tested. If you wish to volunteer to tackle some scenarios, please put your name next to the items you intend to test.

This is not a comprehensive list, so please add missing items!

  • Fresh install verify that the packaging works and the service starts up on localhost:8080 and a Getting Started wizard is started.
    • Verify administrator password is in platform's specific logs ((tick) on Windows - Unknown User (rtyler),)
    • Verify administrator password is written to file mentioned in Getting Started Wizard ((tick) on Windows - Unknown User (rtyler), (tick)  WAR on Mac - Sam Van Oort)
    • Platforms?:
      • (tick) Windows (Unknown User (rtyler) - Windows installer worked well on Windows Server 2012)
      • (tick)  Ubuntu/Debian (.dpkg) (Sam Van Oort, testing on CentOS 6 14.04 docker image)
        • Nothing deep, just install + suggested plugins + create and run a job, restart
      • (tick)  Red Hat (.rpm)  (Sam Van Oort, testing on Ubuntu 14.04 docker image)
        • Nothing deep, just install + suggested plugins + create and run a job, restart
      • Mac OS X
      • Docker (Unknown User (batmat))
      • .war startup (Kristin Whetstone on Windows and Centos)
  • Fresh install behind proxy verify that when Jenkins is started behind a proxy the Getting Started guide behaves appropriately (Unknown User (recena))
  • Upgrade installation with an existing 1.x installation
    • (tick) Verify the Getting Started wizard is suppressed in favor of the Upgrade Wizard. Does the behavior make sense… (verified going from 1.656 to 2.0 RC1 on Windows Unknown User (rtyler))
      • if you only changed some security options in that dialog?
      • if you only started Jenkins 1.x and never did anything? (Kristin Whetstone)
      • if you only created a single job and never built it? (Kristin Whetstone)
      • (tick) if you only created a slave? (Keith Zantow)
    • If you upgrade from a version older than when a plugin was detached, does it get installed? If you upgrade from a version that already bundled the plugin in question, does it (not) get downgraded to the bundled version?
  • Getting Started (Antonio Muñiz)
    • "Install suggested plugins" works properly - (tick)  from svanoort, but noting issue with network failures/timeouts
    • (tick)  "Select plugins to install" allows selecting and installing plugins
    • Setting up admin user works properly - (tick)  from svanoort
    • (tick)  Skipping admin user setup creates an admin user with the admin password from the logs properly
  • New Item creation view after "Install suggested plugins" verify the correct Folder and Pipeline projects show up in view (Unknown User (recena))
  • GitHub Organization Folder verify a small GitHub org has its pipeline projects auto-created (Unknown User (recena))
  • Pipeline (with "suggested plugins") (Antonio Muñiz is taking care of this whole section):
    • (tick)  Verify the built-in simple Hello world example works
    • (tick)  Verify Pipeline from SCM works from a Git repository with a Jenkinsfile in it.
    • (tick)  Verify Multibranch project works and creates multiple jobs
    • (tick)  Verify a multi-stage Pipeline is represented in the Pipeline Stage View plugin correctly
    • (tick)  Verify running pipelines resume upon restart (Sam Van Oort)
  • Plugin Compatibility
  • JENKINS-33926 testing: Is the queue.xml persisted when shutting down via Ctrl-C? -  BROKEN (error)(warning)  - WAR on Mac - Sam Van Oort
    • *A* queue.xml file is created but without job data to restart them.  It is as if the queued jobs never existed (build #s are reused, they don't restart).  The entire queue is dumped.
  • CLI