Advanced Concept Management Module Project

Owned by Former user (Deleted)
Last updated: Oct 15, 2010 by Ellen Ball
8 min read

Project summary

This is a Google Summer of Code 2010 project.

There are multiple names for this project/module: advanced concept search, enhanced concept management, and concept search, but please use Advanced Concept Management. The downloadable module is called "Concept Search Module".

Overview

The OpenMRS open-source medical records system is implemented with an EAV data model to store patient-level observations. This provides implementations with a great deal of flexibility with regard to the data that they are able to capture, without the need for extensive changes to the application or database. In OpenMRS terminology, this EAV namespace is known as the Concept Dictionary.

Currently, the only supplied interface for searching the Concept Dictionary this is a simple ajax search by Concept name. This can be seen here (login with admin/test). However, there are many use cases that would greatly benefit from enhanced searching. Looking at the available attributes on Concept by which you can search (http://svn.openmrs.org/openmrs/trunk/src/api/org/openmrs/Concept.java), there are a number of other fields that we could easily make available for searching - name, description, datatype, class, set membership, associated forms, etc.

Goals

One of the goals of this project is to create a new OpenMRS module which can be installed on an OpenMRS 1.6 installation, and which provides the ability to search the concept dictionary as laid out in the provided mockup.

Ideas

from Justin

This section covers the features that could be implemented during this project. We are waiting for feedback from the major stakeholders (Ada, Ellen, Mike).

Improve Concept Search

  • Allow user to perform basic concept search

  • Allow user to perform more advanced concept searches

  • Allow user to interact with concept dictionary dashboard with obs / form breakdown.

  • Add more search filters/options (filter by class, datatype, usage, etc).

  • Add real autocomplete widget for concept search page.

  • Show more context in concept search results.

  • Implement widgets LAST 10 concepts created feature.

  • Add TOP 10 concepts used in observations.

Improve Concept Editor

  • Improve the concept editor user interface

  • Improve usability around editing a concept.

  • Integrate the stats and editor page.

  • Implement a more user friendly wizard for creating a new concept.

  • Allow user to cascade edit of a datatype change (i.e. change datatype of any rows in the obs table).

  • Allow user to view forms using concept (currently only shows InfoPath form use, but should also show htmlform usage).

Improve Concept Localization

  • Improve the localization user interface.

  • Allow user to export/import message bundles for their concepts for bulk localization updates.

Improve Concept Permissions

  • Implement a new permission scheme that allows privileges to be based on concept classes, sets of concepts, data type, etc.

  • Allow admin to assign specific privileges to an individual user or group.

  • Allow user to manage only concepts that they have permission to view/edit based on more fine-grained permissions.

  • Allow user to a specific privilege to create/edit concepts of certain class

from Ellen

  • Should have an easy way to add concept mappings. Maybe a nice ui for adding concept mappings to groups of concepts? Especially for concepts added with a module.

  • Is it possible to set a global property to make mandatory concept mappings?

  • What about adding bunches of the same datatype concepts? So I want to add 10 new drugs – I don't want to keep having to select drug and N/A. It would be useful to have a memory for the last one, instead of defaulting to Test/Numeric.

  • Add a method to search for duplicate concepts. If a duplicate is found, select one to be retired AND change observations that used the retired one to use the remaining one ** Ada's comment added - I agree with Ellen on this one and it's similar to the ticket #1954.

  • It would be great to have a simple way to provide translations. So if I want to translate all concepts used by a form, there would a entry page with columns for multiple languages (ie. English and French) and concept names could be added in either column.

from Mike

I like everything that has been described above, so don't want to take away from that. I'll just add a bit more information off the top of my head that I think would be useful. The main use cases I am thinking of at a high level are:

  • Allow us to clean up the Concept dictionary and easily monitor it's health, by:

    • Finding Concepts which are not used

    • Finding Concepts which are duplicates

    • Finding Concepts which are incorrectly modelled

  • Allow for it to be much easier to create Forms

    • Find all Concepts in use on a form or forms, or in a particular section of a form

    • Find Concepts which match particular question text on a form or forms

    • List all Concepts which have the characteristics of one I am looking for to both see if it already exists, or to help me to model the new one properly

    • Have tools in place to find and clean up duplicate or incorrect Concepts such that:

      • It's less common to create duplicate or incorrect Concepts in the first place

      • It's less problematic if you do, since they are easy to deal with

Below are some more detailed thoughts on some of the features. Happy to help sift through this as needed...

1. Provide the ability to search for Concepts based on all Concept attributes and common use criteria. Display results in paging List format. Possible search criteria to include:

Property matching:

  • Has a specific ID or an ID in a specific range

  • Has a name that contains particular text

  • If possible, check if it is on a Form with particular Form Field text

  • Is of a particular Data Type or Set of Data Types

  • Is of a particular Class or Set of Classes

  • Is a Set

  • Is Complex

  • Is an Answer to a Question

  • Has a mapping in a particular Concept Source

  • Has a mapping in any Concept Source

  • Has a name in a specific language

  • Is missing a name in a specific language

  • Was created and/or modified by a particular User

  • Was created and/or modified within a particular Date Range

Metadata Usage:

  • Is on a particular Form or Forms, or a particular section of a Form

  • Is referenced by a Drug

  • Is referenced by a Program/Workflow/State

  • Is referenced by a PersonAttributeType

  • Is referenced by a GlobalProperty

Data Usage:

  • Is the question for at least 1 non-voided observation

  • Is the coded answer for at least 1-non voided observation

  • Is the obs group for at least 1 non-voided observation

  • Is contained in an obs group for at least 1 non-voided observation

  • Is the value of at least 1 non-voided person attribute

2. Provide useful statistics / summary views of all concepts in the database in tabular format.
Example: (picture to be added)

3. Allow searches from (1) and viewing rows from (2) to display useful Concept Lists that match. It should be possible to toggle the inclusion of additional sortable columns on these lists (i.e. the number of observations). The goal being that you could:

  • a) From step 1, search for all Concepts that are the coded answer to at least 1 obs

  • b) In the resulting list of Concepts, view the number of times each was the Coded Answer to a Question, and sort the list by this value descending

4. Provide a richer "View Concept" page. From a List of Concepts in (3), you should be able to click into a particular Concept and get an abundance of useful information, including basically all of the underlying information that would be searchable in step (1). This should be divided into sections. For example:

Section 1: Concept Definition (similar to what currently exists, but nicer)

  • Concept ID

  • All Concept Names and Synonyms

  • Description

  • DataType, Class, Is Set, Is Complex

  • If it is Coded, the Answers; If it is a Set, the Members; If Complex, the Handler, etc

  • All Mappings

  • Date created and who by; Date modified and who by

Section 2: Metadata Usage

  • Each form it is contained on, along with the form text if possible

  • Any drugs that reference it

  • Any Programs / Workflows / States that reference it

  • Any PersonAttributeTypes that reference it

  • Any GlobalProperties that reference it

Section 3: Aggregate Data Usage

  • All of the below broken down between voided and non-voided totals

    • # of Obs for which it is the Question

    • # of Obs for which it is the Coded Answer

    • # of Obs for which it is the grouping Concept

    • # of PersonAttributes for which it is a Value

  • Also useful (non-voided obs only)

    • % of Patients who have Obs for which it is a Question

    • % of Patients who have Obs for which it is the Coded Answer

    • % of Patients who have Obs for which it is the grouping Concept

    • % of Persons who have PersonAttributes for which it is a Value

5. Provide a more intuitive "Create / Edit Concept" page. Perhaps something that allows the use of templates to follow. For example, ask a user "What type of Concept do you need?", and give them options like:

  • Coded Question...

  • Numeric Lab Result...

  • Program Workflow...

  • Drug...

  • Advanced...

  • etc...

from Ada

So far, we have several thousands of concepts in AMPATH. In order for us to answer simple questions such as "HIV Positive", it includes more than 10 different criteria/concepts for the derived concept of "HIV Positive". Similarly, we also use those derived concepts/concepts to further define indicators. If there is some kind of utility to link the concept and derived concept as well as concept, derived concept, and indicator, that's be extremely helpful.

  • In the concept dictionary, indicate if such concept has been used for defining derived concepts and/or indicators.

  • In the concept dictionary, indicate if such concept has been used in any cohort, indicator, report. I understand that it will make the concept dictionary very crowded. So, maybe just a link to further drill down to the details. This way, it keeps the concept dictionary page a little cleaner.

  • When doing a concept search (on the name or synonyms), should provide an option of Advanced Search of the description.

  • When there is no map based on name, synonym, or description, provide option of suggested links of better descriptions for such new concept.

  • Now, the concept search requires at least three characters. We have a concept dictionary of "No" which has been used as coded answers in many concepts, especially we no longer use boolean datatype. We need to have option how to display concepts which have less than three characters.

  • Use the examples of "Yes" and "No" concepts ("Yes" is a concept and "No" is another concept). By phasing out Boolean, "Yes" and "No" concepts are now used as answers for more and more concepts. The current page has a lot list of "Yes" concept has been used as answers for tons of other concept questions. It has become very long. Need to have better way to handle cases like this.

  • On the current concept page, it lists on all the forms the concept has been used. Maybe we can have the option of displaying the published forms only and all published/unpublished forms.

  • When someone made changes in between creation (v1) and last revised (v12), it's hard to know what happened in between those 12 versions. Maybe to include the option to display different versions so that it's easier to review in case it needs to roll back to previous version.

Backlog

Concept Search

  • Allow user to perform basic concept search (by name)

  • Allow user to perform basic concept search (by data type)

  • Allow user to perform basic concept search (by concept class)

  • Add auto-complete widget to the fields (concept name, data type, concept class)

  • Allow user to sort the results (by name, data type, concept class)

  • Add filters to concept search results

    • Concepts which are not used (does not appear in obs table)

    • Concepts which are duplicates (how do we find duplicates)

    • Concepts which are incorrectly modeled (how do we find these)

Concept Editor

  • Make creating new concepts easier

  • Remember last datatype added

  • Save changes when a concept is changed

Milestones

These are the milestones till midterm (12-July)

Milestone 0

Milestone 1

31-May to 13-June

  • Allow user to search by concept datatype

  • Allow user to search by concept class

  • Copy comments from here to OpenMRS-wiki page

  • integrate new class ConceptSearch in basic search

  • integrate ConceptSearch in advanced Search

Milestone 2

14-June to 27-June

  • Allow user to sort the results

  • Add filters to concept search results

  • Display summary (1/2)

Milestone 3

28-June to 11-July

  • Add auto-complete widget to the field 'concept name'

  • Display summary (2/2)

Mockups