Background

Most of the data collected by OpenMRS is stored as patient-specific observations (e.g., weight, pulse, serum potassium level, their answer to "How many children live in your home?", etc.).  OpenMRS has a finite list of data types for observables:

• Numeric

• Coded

• Text

• N/A

• Date

• Time

• Datetime

• Boolean

• Structured Numeric

• Complex

We have long wanted a way to include coded observables for users, patients, locations, and other OpenMRS domain objects; however, we do not want to add attributes to our obs table for every domain object.  For example, adding attributes for value_location, value_person, etc. would allow us to store references to these other domain objects within observations, but would grow our obs table wider & wider.  Also, making changes to the obs table are expensive – e.g., the AMPATH system in Kenya contains nearly 100 million observations, so adding a lot of columns to the obs table is not desirable.

Purpose

Domain Expert(s) / User(s)

TBD

Required Skills

• Strong Java skills

• Familiarity with the OpenMRS API

Objectives

• Create a mechanism that allows for observations to be stored with values that reference a person or location in a way that can be expanded to other domain objects in the future.

Design Ideas

1. We could accommodate coded answers that include users, patients, location, etc. by simply adding columns to the obs table.

   • PRO: straightforward, database constraints can be used

   • CON: obs table grows wider and where does it end?

2. An alternate approach would be realize that each of these represents just another code space – i.e., alternate "dictionaries" to our concept dictionary. In this approach, we could add one column to the obs table: value_code_system, making the source of value_coded vary depending on the code system (concept dictionary vs. locations vs. something else).

   • PRO: smaller change to data model

   • CON: cannot use database constraints, big API changes

3. We could use complex observations for these alternate domain objects – e.g., ship OpenMRS with pre-defined complex obs handlers for Person, Location, Program, etc.

   • PRO: no change to data model or API

   • CON: values would be stored as a string in value_complex, might end up coding against complex obs handlers within core

Extra Credit

• Support a way for consumers of the API to find out the possible values for a given domain object (e.g., possible locations) when applicable.

Resources

• ?Complex Obs Support

Proposed Design Solution

After Initial discussions regarding design requirements we have decided to drop design idea number two in favor of three. (See above)

The reason behind this decision is the recent dialogue regarding the need to,

1. Allow users to create and use custom data types instead of only pre defined ones

2. The need to be able to expand on the newly implemented method so that other data types may be added in the future

3. The discussion regarding similar data types used in OpenMRS  (see Person Attributes and visit Attributes)
   
   

For an idea on Complex Obs handlers, see here

The aim of moving towards the handler approach is as follows,

1. Needs no change to the existing database

2. Creates a clear separation of code – Code required to store and retrieve handlers are managed by the Handler class itself.

3. A proper custom data type solution will allow a user to add a custom data type by simply including the necessary handler class.

4. The existing handlers were developed with the purpose of managing complex data, especially large text and image files that require uploading.

Therefore we need to develop the existing UI to support data types such as Person, location etc.

This will require some reworking of the existing UI and controllers.

Test and finalize a good interface hierarchy for handlers, one which will support the existing system while providing above mentioned functionality.

TO DO // Needs further clarification on-

Research to identify the best solution for managing custom data types, given the already existing means used in OpenMRS

enhancements to the interface -- e.g., to answer questions like "what are the valid choices" or "what are the valid choices given this search string" or "how should this data type be rendered in the user interface"

Automating registration of handlers as opposed to the manual entry of a Spring Key.

Technical Design

I will follow this technical design with the main objective of changing as little of the existing code as possible.

Step one :

 Creating new Handler classes for Domain Objects.

Existing handler classes store / retrieve complex data from an external file system. (See Text and ImageHandlers) these handlers perform file I/O operations.

On the other hand the implementation of handlers for domain objects will be much more different.

These will record data in the Obs table itself.

Example: the saveObs(obs) method of PatientHandler will look as follows,

@Override

      public Obs saveObs(Obs obs) throws APIException {

            Object data = obs.getComplexData().getData();

            //No File I/O operations necessary for this handler

            // Set the required value to be stored in the database column

            obs.setValueComplex(data.toString());

            // Remove the ComlexData from the Obs

            obs.setComplexData(null);

            return obs;

      }

A value similar to PatientHandler|PatientId:1 will initially be stored in the database.

The getObs(obs, view) method will retrieve this value and use it to return a patient object where patient Id equals 1.

Creating a Concept of Type ‘Patient’.

There will be no changes to the ‘Creating new Concept’ page initially.

Therefore creating a concept of type Patient is similar to the existing process to create other complex concepts.

Go to the UI screen, Select concept datatype = Complex and then select the ’PatientHandler’ displayed in the drop down list displayed below.

Creating a Patient Concept, an image of the UI screen

Design modifications to the Observation page

Initially, all complex concepts were based on uploading complex files into the file system (and their subsequent retrieval). However this changed drastically with the new requirements to store / retrieve domain objects.

In the design of the existing ObsForm.jsp page, javascript identified the data type of the concept selected by the user, and then displays one of eight input options (each contained within a <tr> row) for him to input data values.

For example, a Boolean concept will prompt the page to display ‘valueBooleanRow’ etc. etc.

However in the case of a complex concept, the page needs to be modified to display one of two different <tr>s based on whether the concept handler is a text or image hander(thus needing to upload external files) or an PatientHandler(needing to select existing domain objects)

To this end,

1. In the obsForm.jsp, create two table rows, one for the existing data upload facility, the second for object selection using an auto complete box.

2. Display the appropriate <tr> accordingly.

Design Issue:

A minor complication here is that the custom <openmrs_tag:conceptField> used to retrieve concepts does not downcast concepts to ConceptComplex. This means a concept is only aware of its data type, such as (in the case of a complex concept)

tmpConcept.hl7Abbreviation = ‘ED’

Thus the concept is not aware of what its handler value is.

This led to problems, since I do not wish to alter the existing and widely used <openmrs_tag:conceptField> tag class to meet my requirements. Also given that casting would not achieve my desired result, I decided on the following method,

In the JavaScript of the ObsForm.jsp page,

else if (datatype == 'ED') {

                        DWRConceptService.getConceptComplex(tmpConcept.conceptId,checkHandler);

{color:#3366ff}}

As you can see, if datatype == 'ED', I call the DWRConceptService to retrieve a ConceptComplex object, access its handler string and use that to decide on which option to select.

To provide an auto complete box to search for Patient objects, I have decided to use the existing <openmrs:fieldGen> tag class.

If I am unable to expand this tag to support multiple domain objects (in the event of further expansion to admit other domain data types) then I will consider the option of creating a similar tag with my own custom implementation.

Midterm Evaluation

Project Milestones for midterm evaluation

Implement ability to store complex Obs of a single domain object type (etc. patients)

1. Creating concepts for the given domain object type

   1. Register basic handler class for selected domain object

   2. Support creation of concept using this handler
      
      

2. Creating basic observations using the above created concept

   1. Implement basic structure to store observations (UI, data retrieval)

   2. Implement structure to retrieve and display Observations  created using the above concept

   3. Optimize handler class to render save / restore facilities more efficient                                                                                                           

3. Improve on created functionality to support other handler types as well

   1. Possibility of altering UI changes to jsp view to make them as minimal as possible

                              i.      Consider possibility of a generic auto complete box which supports all domain object types

                              ii.      Consider possibility to create a custom tag class which supports all domain objects, in order to carry out the above step more efficiently.

Timeline

Noteworthy alterations

1. Made alterations to PatientHandler class so that the displayable version of 'valueComplex' for Patients follows the format 'firstName middleName familyName (id) | patient id'   -- e.g., "Bob Smith (123-0)|4"

This change will provide a more detailed description when displaying multiple observations.

Example : See the Obs search option found on the 'Find /Edit Observation(s)' widget in the Observation management page.