Testing Releases
The information on this page, as of Feb 2016, is outdated and links are incomplete. This page reflects work that may not have been maintained.
Some day we will have a full team to run various tests (integration, load testing, application compatibility, etc.). Until that day, the simple recipes below will guide a user to make sure that different parts of the application work properly before a release.
Testing Using Real-Life(R) Data
If you are looking for documentation on how to test a release candidate against your own database, see the Release Testing Helper Module page.
Testing the API Layer
It is assumed that unit tests have been written against any new API code since the last release. All unit tests are run after every commit by our Continuous Integration server. You can see how well the tests are covering the api by looking at the unit test coverage output.
Testing the Web Application UI Layer
The "release-test" mavenmodule in trunk contains selenium-over-jbehave unit tests.
There is a continuous integration project that runs these tests weekly. TODO: create project and/or add link
See the currently defined stories in code.
Read about Selenium and JBehave. TODO: add appropriate links
How to add a story
If you would like to add a story to our suite, open a new ticket with the text of the story. It should look something like this:
create_encounter_type.story
GivenStories: org/openmrs/stories/go_to_admin_page.story
Given I am on Admin page
When I choose to manage encounter types
And I choose to add a new encounter type
And I mention name YOUTHINITIAL and description Outpatient Youth initial visit
When I save the encounter type
Then the new encounter type should be saved
This story goes into the release-test/src/main/resources/org/openmrs/stories folder.
Creating a runnable story
JBehave looks for a java class with the same name as the .story class. So from our example above we would need a class named CreateEncounterType.java in the org.openmrs.stories package.
public class CreateEncounterType extends Story {
@Override
public List<Steps> includeSteps() {
return asList(new LoginSteps(driver), new AdminSteps(), new CreateEncounterTypeSteps(driver));
}
}
The LoginSteps and AdminSteps are required for the parts of the story covered in our required go_to_admin_page.story.
Our CreateEncounterTypeSteps code and annotations should match our story:
Annotations in our steps class
@Given("I am on Admin page")
@When("I choose to manage encounter types")
@When("I choose to add a new encounter type")
@When("I mention name $name and description $description")
@When("I save the encounter type")
@Then("the new encounter type should be saved")
The text must match up exactly or the story will not be able to run.
See CreateEncounterTypeSteps.java for full code.
How to run the stories
Tests are run from the main openmrs root directory. This is because maven must launch a jetty server using the webapp project code.
Since openmrs is running in memory, you need to increase the amount of memory given to maven:
Using the release-test.sh script
Execute this script at a command line with:
This runs the test in the background (-b) so no browser pops up (requires xvfb to be installed), gives verbose output (-v) uses "MyOpenmrsPassword" and "admin" as the credentials in openmrs, and only runs the GoToAdminPage story/steps.
This requires an openmrs database to be installed into a running mysql that can be found using the openmrs runtime properites.
If you want ALL tests run against an empty in-memory (requires that the initialization step be run, which takes 10 mins), use the -i flag.
Running directly using maven
or
You must skip tests because jbehave runs these as classes, not as Junit tests.
"integration-test" means run ALL tests against an empty in-memory db (which means the intialization step must be run, which takes 10 mins).
"smoke" means run against a pre-defined mysql database that already contains openmrs code. A runtime properties file must point at that.
Possible stories that still need to be coded up
TODO: this list needs to be compared with the currently stories. Some of these might be completed already.
void the whole encounter - make sure the obs get voided too
unvoid that encounter - make sure that the obs get unvoided too
Forms (InfoPath, htmlform & xforms) (module dependent)
Duplicate a form
Add a new field to a form schema
Concepts
Update All Concept Words
Update a range of concept words
Add/change/delete concept
Add/change/delete concept map
Add/change/delete concept class
Scheduler tasks (make sure they execute)
Message service (smtp)
Merge patient records
Create/modify program/workflow/state
Patient dashboard
Review header for name, age, ids, bmi, regimens, last encounter, etc.
Overview tab
Add/delete patient identifiers
Enroll patients in program/workflow/state
Add/delete relationships
Add/delete allergies
Add/delete problem list
Regimen tab
Add/stop regimen and drugs
Encounter tab
View an encounter on the dashboard
Demographics tab
Edit patient identifier
Edit address
Edit name
Edit patient attributes
Edit with short and long form
Edit deceased along with date of death
Graph tab
Add CD4 count along with obs
Run cohort builder
Create/edit data export
Manage users
Create new user roles
Modify privileges for existing user roles
Manage relationships
Add a new relationship
Change display order
Patient attribute
Add new patient attribute with various formats (including org.openmrs.concept)
Encounters
Create encounter location
Create encounter type
Retire/purge encounter type
Locations
Change location name
Add/retire location
Manage drugs
Add/change/retire drug
Check adding concept drug
Global properties
Add/change global property
Change theme and locale
Modules
Install module from repository
Install module from file
Update module
Start/stop module
Check for upgrades
Test that the key features of the bundled modules are not broken, see below for details for the features to test for some of the bundled modules
Features of Bundled Modules
Its possible in
*Logic
To be updated...
*Form EntryCreate a form
Fill out a form and submit it. Verify the new encounter is added to the patient
Test the search feature in the task pane (done when selecting the provider while filling out the form)
Upload an XSN file
*HTML Form EntryCreate a form
Fill out a form and submit it
View entered form
Edit previously entered form
Check for text boxes, dates, check boxes, radio buttons, drop down menus, missing required value
If a required value is missing (ie. encounterProvider), does it reset the other selected values?
Has the new release slowed the load time of an htmlform?
Do the unit tests all run correctly?
Reporting
Generate a report (need more details here)
Reporting Compatibility (cohort builder)
Create a cohort of all male patients with weight > 50
Save that cohort as a dynamic cohort
Save that cohort as a static cohort
Load the dynamic cohort
Load the static cohort
Create a data export and run it
Serialization.xstream
Nothing to do here, tested while doing reporting tests...
Confirm that it starts
Xforms
Create form
Fill out a form and submit it
Features of other modules
These modules are used widely and could benefit from having stories written for them.
HTML Widgets
Nothing to do here, tested while doing reporting tests...
IDGEN
Do all of the configuration pages open correctly?
Configure a sequential number generator and use the idgen 'view' page to generate a batch of IDs.
NOTE: a conversation with Mike Seaton this morning makes me think that this will be broken in 1.8
Metadata sharing
Import file from another server
Create metadata sharing zip
Download and import to other server
MRNGen
Is the 'create patient' identifier portlet correctly overwritten?
Use mrngen to successfully to autogenerate an identifier while creating a patient.
Patient summary
Does module start?
Look at patient dashboard (overview tab) for "Patient summary" link
Look at the summary.
BIRT reporting
Start module
Make certain all the global properties are correct
etc.
SimpleLabEntry
This module extends dojo for some of its functionality so this will definitely be broken in 1.8
The test is to 1) configure the module, 2) create an order, 3) register a result for that order
And, do the 'create patient' mechanisms in the module successfully create a patient?