Data Migration & ETL Module (Design Page)

Owned by Ben Wolfe
Last updated: Jan 21, 2011
2 min read

Abstract

The Data Migration & ETL project will focus on creating a module that will allow users to migrate data from an existing system into OpenMRS or from OpenMRS to another system.

Goal

Support data migration using a standard format used by both systems

This will require us to define/adopt a standard to represent a complete patient record. While there are standard data exchange formats for patient records (e.g. HL7 CDA, ), we may decide to start with an OpenMRS-proprietary data exchange format in order to avoid the complexity and extensibility issues around current standards.

  • Allow user to export data from a legacy system in a specified format from an existing system.

  • Allow user to import data into an OpenMRS instance by uploading exported data from another system using a standard format.

Support data migration through an ETL-capable module

We will also explore an approach to data migration that will allow users to design ETL data transformation "scripts" (for lack of a better word) that can be used to extract data from OpenMRS to another system (and vice versa). This module will provide very basic features, essentially providing a wrapper around the Pentaho Data Integration API:

  • Allow user to upload a transformation file from a system like Pentaho Data Integration to be run within OpenMRS.

  • Allow user to manage all transformations (edit, delete, preview, execute).

  • Allow user to execute a transformation from within OpenMRS to extract/transform/load data from OpenMRS to a dimensional data model.

  • Allow user to execute a transformation from within OpenMRS to extract/transform/load data from a legacy system to OpenMRS (requires a minimal dataset to exist within OpenMRS).

  • Allow user to schedule transformations to run at certain times in order to perform incremental ETL from one system to another.

Mentor and other interested parties

?@Justin Miranda (mentor)

?@Mike Seaton

Resources

Please see the following resources for more information: