Strategy for Having Local and CIEL Concept Dictionaries

Owned by Joaquin Blaya
Last updated: May 21, 2020 by Andrew Kanter
4 min read

If you need multiple OpenMRS instances that share concepts, as well as those that could have both local concepts and use those from an external dictionary, such as the CIEL dictionary or from the Open Concept Lab dictionary (which uses the CIEL dictionary among others), currently the most commonly used method is an OpenMRS instance that acts as a central repository for all of your concepts and using the Metadata Sharing Module to transfer concepts to the different servers. See also Getting and Using the CIEL Concept Dictionary. 

The working strategy is as follows:

  1. Maintain all concepts (both locally-created and externally-managed) in a single OpenMRS instance (could be cloud hosted). Use this central concept dictionary server to maintain all of the concepts, make changes and additions, and then push them out to other servers using the MDS module.

  2. Import new concepts from the CIEL concept dictionary (the manner of import will depend on where you get the source from). You can use the MCL server with the MDS Module to import a subset of concepts. NOTE: Need to see what happens with MDS in regards to updates to existing concepts such as map changes, etc. The MCL server is updated fairly frequently with the concepts from the CIEL/MVP dictionary. Search for needed concepts and when you have all of them, create an MDS export (zip package) which is imported into your server. The MCL is running OpenMRS Version 1.6.3 with MDS Version 1.0.6.  NOTE: Check on OpenMRS versions compatible with the MCL server. If you get the concept tables directly from CIEL via the dropbox, the SQL files will overwrite all the concept tables in your local implementation.


Suggestions for Organizing Local Concepts

  1. Include your organization as the concept source for your concepts (ie. AMPATH, PIH, MVP). This will ensure that you know the source of the concepts. Check on this: "and you could export them by selecting the source". Due to synchronization and MDS (and perhaps other factors), the concept IDs are not maintained between servers, but the mappings and the uuids are preserved.

  2. You might want to place some identifier (for example the name of your organization) to make them more easily identifiable. For example, in HTML forms, it's easier to read  <obs conceptId="PIH: Malaria" /> or <obs conceptId="ICD-10-WHO: B54" /> than <obs conceptId="123" /> or worse yet, with the uuids <obs conceptId="be3f86fa-1691-11df-97a5-7038c432aabf" />. If you use the concept name in the conceptId (ie. PIH: Malaria), MDS does not convert that mapping when importing the htmlform. However, if it used the concept id (ie. conceptId="123"), MDS converts the concept id to uuid. It makes the htmlform difficult to read and debug.

Adding New Concepts

To add a concept, it's best to search the CIEL/MVP dictionary first to ensure that the concept is not already there by another name, etc. Do this by going to the MCL server http://maternalconceptlab.com. Concepts can be search by the concept name or by relevant maps (such as the SNOMED CT code or ICD-10 code, if you know it). If there is ANY REASON why the concept could be used by others, the it should be included in the CIEL/MVP dictionary. Do not assume that a concept would only be used by you. Email Andy (at akanter@ei.columbia.edu) if you have any questions. If you cannot find the concept that you want in the dictionary, it could be because it is in the dictionary and you can't find it, or because it is a new concept and is not there. If you are SURE that the concept would only be used locally then add it to your local server.

Another way of adding concepts from an external source is to get the sql dump of all of the concepts (either from the MVP OpenMRS dropbox or from the OpenMRS Standalone concepts tables which comes with all MVP concepts), adding those concepts with mysql to a intermediate server and finally using MDS from that server to your concept server.

This command will dump all the concept tables into a single file:

mysqldump -u openmrs -p -q -e --single-transaction -r"concept-dump.sql" openmrs
concept concept_answer concept_class concept_complex concept_datatype concept_derived
concept_description concept_map concept_name concept_name_tag concept_name_tag_map
concept_numeric concept_set concept_set_derived concept_source concept_state_conversion concept_word

Which concepts to select

One way to do this is to only get the concepts from the external dictionary that you need.  The benefit of that is you maintain a usable set of concepts, because it get difficult to manage if there are too many concepts. Of note, managing a concept dictionary is extremely complicated and requires a significant time commitment to do properly. It is frequently seen that locally-managed concept dictionaries become corrupted, lead to duplication of data, or otherwise make it difficult to analyse data previously collected. Currently, the CIEL/MVP dictionary has over 60,000 concepts. The disadvantage of that is it requires someone to pull concepts every time you want to add additional forms or programs.

It might be helpful to use MDS to select all concepts with a particular map and export those to another server.

Automatically updating Dictionary and MDS Module

When thinking about getting the automatic updates from either the MVP/MCL dictionary or MDS updates, always make sure to test them out in the concept server before going into production and always backup the concept tables on the server before doing any updates. You probably want to update the concept dictionary when there are new concepts that you want to use. If you rely on maps to reference terminologies such as SNOMED or ICD-10, you probably should make sure that you update your concept tables at least once or twice a year at a minimum.

Updating/Synchronizing many servers

If you have multiple servers who are going to be using the same concepts, there seem to be 2 ways to keep them updated with the central concept server

  1. Use the sync module, but limit to synchronizing concepts.

  2. The MDS module should have this functionality, but I haven't been able to find it yet.