See the main Data Synchronization Project for more information about the background and breakdown of the OpenMRS Synchronization Project

GSoC 2008 Intern: Nzeyimana Antoine
Mentor(s): Maros Cunderlik and Christian Allen
Other Developers(s): Darius Djazayeri,Chase Yarbrough

General Ideas and Description

Data synchronization is a new OpenMRS feature allowing synchronization of data amongst a set of loosely networked servers. Such ability to exchange data is essential for operation of EMR system in rural areas where connectivity amongst sites maybe unreliable yet the need for timely centralized collection and analysis of data from remote sites exists.
Because of the distributed and disconnected nature of the sites where servers may be deployed, having a set of administration views that provide key statistics about the status of the synchronization process is critical to the local administrators.
We need a comprehensive Sync Status Page; that page should help local administrators to:
1) Visualize every synced record/item real-time sync status; the tracked-out change set record may be:

• Waiting for Synchronization
• Pending for send . . .
• Being transmitted . . .
• Committed
• Failed and waiting for Retry . . .
• Retried . . .
• Failed and Stopped

2) View statistical sync reports between two selected date/times:

• How many records/items were tracked-out for synchronization
• How many of them have been uploaded successfully to the server (+percentage)
• How many failed (+percentage)
• How many of them (percentage) have been sent after retry( # of retry)
• Which sync method was used for both above cases:-manually synced via web, -manually synced via file, -automatic synced via web(scheduled sync)

• View and Edit details of record items being synchronized
• Search Logs
• . . .
  4) Perform maintenance tasks such as:
• Archive sync journal
• Change various timeouts
• . . .

Project Complete Goals

1.User login to the Admin screen and find Synchronization section links
2.Click on one of the following links to enter related Sync Pages:

• Sync Settings
• Manage Synchronization
• Sync Statistics
• View Sync Changes

3.The SETTINGS screen is divided in three main parts:


a)_General Settings_ where you can:

-Enable or Disable Synchronization process by check/uncheck a check box

-Set up automated (scheduled) synchronization (check box and input field)

-Input or Edit local server name and administrator Email

-View local server ID

-Save your settings by clicking a save button(settings change or take effect after this
button is pressed)

b)_Servers Settings_ :

-Here you can Add,View Details,Edit or Remove currentls configured synchronization servers

-Click on Add a Child Server link to configure a new Child server that will synchronize
data with this local server

-On the "Child Server Configuration Page" you input the Child Server Nickname,username and
password and optionally its admin email then save

-Click on Set Up a Parent Server link to go to the Parent Server Configuration Page

-On the Parent Server Configuration Page input the PARENT Nickname,Address(example: http://your.server.name/openmrs or http://localhost:8080/openmrs_parent),login username and
password,test connection by pressing a Test button and optionally set up scheduled synchronization then save.

-At each server configuration page, you can configure synchronizable items/classes.

(Here remember that Synchronization is done in a tree network topology where one server can
have one PARENT and many *CHILD*s)

c)_Advanced Configuration_:

-Select synchronizable items/classes at local server then save settings

4.Go on the MANAGE SYNCHRONIZATION Screen:



-find currently Sync items (local server model changes) that are not yet synchronized with
the parent server.

-there are radio buttons to select whether to Sync via web or via file export
by one button click

-the control page is refreshed 5 seconds after each manual synchronization

-there is option to import data from child server or upload response(s) from parent server

-Monitor Sync Record transmission flow

-Edit failed Sync record items in a way that they can be reset to "NEW" state and
"retry_count" reset to 0;

-View details of sync items/records

4.Go on the SYNC STATISTICS Screen:

-view current synchronization settings

-generate sync statistics reports between two dates

-View Sync Changes grouped by their state(SYNCHRONIZED,WAITING TO SYNCHRONIZE,FAILED,...)

5.Go to the SYNC CHANGES/HISTORY OF CHANGES screen:

-You can browse sync records , they are linked to their View/Edit page so that the admin is able to re-edit the content of a given sync record before it synchronize
-The page is now pagified so that the browser doesn't need to load a lot of records, hence increasing server performance.

6.Go to the SYNC MAINTENANCE screen :

There you have optins such as:

• search for Sync records and be able to view or edit their content
• Archive the sync journal and imports into compressed zip files
  -manage some sync timeouts such as synchronization maximum retry count , the maximum number of change records that will be send to the server at one time,maximum records displayed on one page.

7.A HELP PAGE should be implemented in order to guide local admins to configure and manage Synchronization.

Project codes repository branch

Project PlanDRAFT

The project should be divided into different tasks implemented
from the most needed or the most important part in the synchronization process.
We should consider these steps as creating pages/forms for specific purpose but all of them are handled in the Spring MVC and at the DWR-DOJO-JSP-JSTL

Task 1: CONFIGURATION/SETTINGS PAGE

_Duration:_ 1 WEEK

• CHILD Server configuration

  -Able to: Add New, Edit and Remove CHILD Server account
  -Choose Synchronizable Items @ CHILD

• PARENT Server Configuration and Testing

  -Add ,Edit,Test and Remove a PARENT server
  -Able to choose Synchronizable Items @ PARENT

• AUTOMATIC SYNC Setting(Scheduler)

Task 2: SYNCHRONIZATION CONTROL PAGE

_Duration:_ 2 WEEKS

• One-click Sync Initialization(manually) via Web or Disk(may use a RadioButton)
• Able to select Items to Synchronize (Default to Select All)
• View , Edit Sync Items Details

  _ISSUES_
  Editing any kind of OpenMRS Class that's going to be Synchronized
  needs importing related forms at in the Sync Views and this should need more
  time for a whole integration.
  About Edit Sync Items: shouldn't it result in a recursive cycle by having two
  (almost identical) items synchronized in the same record ,
  how to handle @ PARENT
  OR just take it as a modification occurred at Management(Admin) level before
  the last change being reported to the parent .

Combining the two solutions makes a good way for manipulating the Sync records.

• Monitor Sync Record transmission flow
• Handle failed Sync record items in a way that they can be reset to "NEW" state and "retry_count" reset to 0;

Task 3: SYNC STATISTICS PAGE

_Duration:_ 1 WEEK

• Monitor Sync process Statistics
• Be able to browse Sync records from the statistics reference page

Task 4: HISTORY PAGE ENHANCEMENT

_Duration:_ 1 WEEK

• View Sync Items and Historical Changes in detail

  -Criteria: Item Class,State,Origin,date changed ...

• Able to arrange Items by criteria
• Have content spread over multiple pages with only so many sync records per page.

Task 5: SYNC MAINTENANCE PAGE

_Duration:_ 1 WEEK

• View and search logs
• Archive Sync journal
• Manage different timeouts
• Manage some sync properties

Task 6: SYNC HELP PAGE

_Duration:_ 4 DAYS

• Abstract about Synchronization
• Synchronization How To:

  -Configure
  -Control
  -View Statistics & perform maintenace

Other Ideas & Comments

We welcome more Ideas and Comments from the community

Please feel free to leave your feedback

---Antoine