Jenkins : Essentials client/server setup Quality Bar (Draft To be discussed)

The purpose of this is to get a set of quality-related tasks agreed with the development team that need to be completed before going to production

AreaTasksOwnerPriorityStatus
Functional

 Definition of P1 cases for Milestone 1

This task is about having a list of P1 cases for the milestone 1 of essentials, so we can be sure all of them have been properly tested

 MUST

JENKINS-51448 - Get a list of P1 cases for Essentials M1 Resolved

Functional

Testing of all P1 cases

This task is about getting sure all P1 cases have been tested before releasing Milestone 1 of Essentials, how they are tested is not as important at this point but just ensure that they are tested, this task will be updated to reflect the testing of each P1 case here

MUST

JENKINS-51449 - Testing of P1 cases for evergreen client/server setup Closed

FunctionalBug bash session to check P1 cases

MUST

JENKINS-51451 - BugBash session for Evergreen client/server setup Closed

Functional

DogFooding

This task is about getting a running instance of the essentials client/server system and use it on a daily basis before launch to try to catch problems not detected in the previous phases

TBD

GOOD TO HAVE

JENKINS-51452 - Set up a live instance for dogfooding Evergreen client/server setup Open

Performance Definition of minimum performance requirements for the milestone 1 of essentials client/server setup

TBD

 GOOD TO HAVE

  JENKINS-51453 - Define performance requirements of Evergreen client/server system Open

Performance

Testing of the system to be sure that it meets the acceptance criteria provided by JENKINS-51453 - Define performance requirements of Evergreen client/server system Open for each P1 case

TBD

GOOD TO HAVE

JENKINS-51455 - Performance testing of evergreen client/server setup Open

Automation

Code coverage report for P1 cases:

For each P1 case defined in JENKINS-51448 - Get a list of P1 cases for Essentials M1 Resolved we want to know:

  • If there is an automated test to cover it
  • The health status of those tests
  • An automation proposal if there are no automated tests
TBD

GOOD TO HAVE

JENKINS-51456 - Code coverage report for essentials client/server setup Resolved

Rollout plan

We need a definition of a rollout plan for the system that has to include at least:

  • Detailed instructions on how to deploy the system
  • Detailed instructions on how to smoke test the system just after rollout
 TBD

 GOOD TO HAVE

JENKINS-51458 - Provide documentation for the essentials client/server setup Open