Data Migration & ETL Module (Design Page)
Abstract
The Data Migration & ETL project will focus on creating a module that will allow users to migrate data from an existing system into OpenMRS or from OpenMRS to another system.
Goal
Support data migration using a standard format used by both systems
This will require us to define/adopt a standard to represent a complete patient record. While there are standard data exchange formats for patient records (e.g. HL7 CDA, ), we may decide to start with an OpenMRS-proprietary data exchange format in order to avoid the complexity and extensibility issues around current standards.
- Allow user to export data from a legacy system in a specified format from an existing system.
- Allow user to import data into an OpenMRS instance by uploading exported data from another system using a standard format.
Support data migration through an ETL-capable module
We will also explore an approach to data migration that will allow users to design ETL data transformation "scripts" (for lack of a better word) that can be used to extract data from OpenMRS to another system (and vice versa). This module will provide very basic features, essentially providing a wrapper around the Pentaho Data Integration API:
- Allow user to upload a transformation file from a system like Pentaho Data Integration to be run within OpenMRS.
- Allow user to manage all transformations (edit, delete, preview, execute).
- Allow user to execute a transformation from within OpenMRS to extract/transform/load data from OpenMRS to a dimensional data model.
- Allow user to execute a transformation from within OpenMRS to extract/transform/load data from a legacy system to OpenMRS (requires a minimal dataset to exist within OpenMRS).
- Allow user to schedule transformations to run at certain times in order to perform incremental ETL from one system to another.
Mentor and other interested parties
?Justin Miranda (mentor)
Resources
Please see the following resources for more information: