Share Metadata Design Document by Mary-Anne Wolf

This project was migrated

to ?Metadata Sharing Project

What The Share Metadata Module Does

Because every MRSObject now has a UUID (universally unique identifier), it is now possible to serialize these objects, to turn them into XML and back. The primary purpose of this module will be to allow a developer to select a subset of the MRSObjects to serialize, and publish the XML file on a URL. Then an administrator can subscribe to the XML, download it, and de-serialize the objects and update the database that way.

The downloading will be done by daemons. Actually updating the database will be done by an administrator interactively. Subscribing will primarily be done by installing a module.

(Not) Handling Data Conflicts

The publish/subscribe relationship is treated one-directional. The subscriber downloads the data, but does not otherwise modify it. Thus, if the data downloaded conflicts with the data already in the local database, then the local database is overwritten.

Subscribing

The receiver (the system which downloads the XML and deserializes it) subscribes to URLs by putting a row for each into the Subscribe Table.

The subscribe mechanism is generic, in the sense that what gets downloaded need not be an XML file containing serialized metadata. It can be any file.

XML files containing serialized metadata are distinguished from other files by the setting of the Handler. The Handler is a String which names a class which implements an Interface which includes a method in which the File is passed as a parameter. The Handler must use a constructor without parameters, and the class must be available to run when the downloaded file is processed.

There is already code which is intelligent about only downloading a File if the File has been touched since a given date. This is how the subscribe code decides whether a File needs downloading again. Having been touched does not necessarily mean that the content of the file has been changed, although if the content has changed this always changes the File Date. We distinguish whether an XML file containing serialized data has been changed by looking at a Version Date embedded within the XML. This can only be determined by the Handler, which parses the XML.

A File can have one of three statuses.
UpToDate – means that the File has been downloaded successfully at least once,
that the download completed without error last time it was attempted,
and that as far as we know, the content is up to date.
Stale – means that the file was extracted from a module installation package, and is assumed not to be up to date until it has been checked against the copy at the URL.
Incomplete- means that an error happened last time we tried to download the file. As other software does, we will keep the old version of a File around until the new version has been downloaded successfully.

In order to restrict the allowable statuses to the prescribed list, we have a separate SubscribeStatus table which lists the allowed values.

Column
Type
Description

String

Serialization, Replacement, and Creating Objects

We have existing serialization code that turns an MRSObject or instance of one of its derived classes into XML. Unfortunately, one MRSObject may reference other MRSObjects, and we may or may not want to replace the referenced Objects with ones that make more sense on the local system. The current plan is to use the existing serialization code to create the XML, and then fix up the XML as a separate step. (An obvious alternative would be to enhance the serialization code to handle this.)

The assumption in the initial version of this module is that the substitution it does will consist only of replacing an MRSObject of one class with another MRSObject of the same class, but a different UUID. Having created the XML, a developer can also modify the XML in whatever convoluted way they need, in order to allow for more complex substitutions and other changes, but we won't automate that in the first release.

The combination of Handle and OwningURL is required to be unique. There will be code that references an Object by Handle, which throws an Exception if the Handle is not unique. There will be code that references an Object by Handle which returns a Collection of MRSObjects, so that the caller can decide to do something else if the Handle is not unique. There will also be a method that references an Object by both Handle and OwningURL, if one chooses to use a Handle that is likely not to be unique. One might, for example, have a set of modules that use similar handles but in slightly different ways.

Defining the Subset to Serialize

In defining which subset of the possible Objects to put into the XML file, it was suggested that the user pasted text containing the classname and a comma-separated set of UUIDs into a text window. Typing that data in manually sounds rather more prone to error than is preferred. A fine line must be walked between solutions that take too long to code, and those which are simple to code but too painful to use.

The short term solution is to create a utility class which offers a Java method that takes, as parameters, a File, and a Collection of MRSObjects or instances of classes derived from MRSObjects. It then writes into the File a set of Properties of the form, uuid=classname, where each Property corresponds to an MRSObject to be serialized. The reason for uuid being the key is that uuid should be unique, whereas classname may not be unique. There will be corresponding methods to turn the Properties File (or a Properties object) back into an ArrayList of MRSObjects. In another method, we can also take a String as a parameter, and parse the String like the content of any Properties file. This would allow for pasting the String, but creating that output programmatically would be much easier than if we used an idiosyncratic syntax.

The reason for choosing the Properties format is that it is easy for a developer to create code to output that format if choosing objects to serialize by some more complex means than having a Collection.

In a later version, we could have a GUI front-end, which creates the Properties or Collection or MRSObjects, and then calls the same code to use what it creates.

More Details About Replacement

Although it is acceptable to force someone publishing Objects to write code, we assume that the user of the receiving system is an Administrator User, who does not write code.

For the first release, all replacements will be manual. This means that the developer will list in the XML the MRSObjects that need to be replaced and the administrator will be shown a GUI that allows them to select each replacement from MRSObjects of the same class. UUIDs are not a very meaningful way for humans to identify objects, so it would help to display something else in the GUI, such as the object's name. To accomplish this, the developer will need to specify a Method to be called on each Class of object to be replaced. This Method should take no parameters. This method will be called, and toString will be invoked on the return-value of the call, and this String, with the UUID will be displayed in the GUI for the administrator. A typical method might be “getNameâ€Â. If there is a Class for which the developer has not defined such a Method, or if there is a mis-spelling of the Method name so that no Method can be found, then toString() will be invoked directly on the MRSObject subclass instead.

Thus, an administrator who is installing a Form that contains an EncounterType might need to replace the user associated with the EncounterType. The Hint from that row might say “Select a user to own the EncounterTypeâ€Â, and the method might cause the username Admin1 to be displayed as part of a drop-down list.

Deleting

While it may seem that the ability to delete objects as part of installing a module is something we can live without, once one thinks in terms of having multiple versions of a module installed at a large number of different locations, keeping the set of MRSObjects that are installed as part of that module become important. As with creating, the objects will be specified using Class and UUID. The rule enforced by convention should be that a module should only delete MRSObjects that an earlier version of it created. Mapped MRSObjects created in another way should not be deleted by installing a new version. We will not enforce this rule in code.

Integration with Installing a Module

The typical way in which the share module will be used is to create MRSObjects as a side-effect of installing a module. The module will also map a handle to an MRSObject that replaces another. More details TBD.

A Tale of Two Daemons

When the Share Metadata Module is installed, it also installs 2 daemons, the“Yes Daemon†and the “Maybe Daemonâ€Â. The Yes Daemon should be run multiple times per day, and should perform updates where we can be pretty sure there is action that needs to be taken. This would involve downloading Files that are marked as Stale or Incomplete. The Yes Daemon will iterate down its list some number of times, and sleep for some amount of time between these iterations, each time it runs. The Maybe Daemon should run once per day or possibly less often. It is supposed to check whether URLs marked as UpToDate have been changed since the last time that they were downloaded. It checks each URL only once each time it runs.