NOTES:

• Sun, Sep 13 EDITORIAL Check out Pascal's notes on Security for a template of good note-taking.

• Sun, Sep 13: Please be patient while Justin enters his notes ...

• Mon, Sep 14: Update: It took a while to get these notes compiled and uploaded ... apologies. The "raw" notes are currently uploaded. Need to clean up the notes a bit.
  --Jmiranda 03:18, 14 September 2009 (EDT)

Data Quality Reports

(consensus is that these should not be called reports since they usually require some interactivity)

Revised Notes

Questions to answer

• For what purpose are we/you creating data quality reports?

  • To improve clinical care

  • To facilitate research studies

  • To improve data quality accuracy

  • For aggregate reporting

  • To improve availability/timeliness of data entered into system (is data ready for next visit?)

more to come soon ...

Raw Notes

• For what purpose?

  • clinical care

  • research

  • data entry errors

  • timeliness of data (consult date vs reporting date) -> useful for find things like missed clinical appt

• standards for data quality

  • double entry module

• how can we check data input

• tools used for data quality

  • data exports on cohort (very manual process)

  • BIRT reports using SQL to implement data quality rules

    • e.g. RETURN VISIT DATE, CURRENT DRUG REGIMEN IF ON ARV, IDENTIFIER FORMAT MATCHES PROGRAM, MISSING PATIENT VISIT

• catch errors when they occur (form entry)

• data quality rules will be implemented in next version of reporting tool (v1.5)

• Implementation specific examples

  • AMPATH (ada)

    • uses SQL export + SAS to run data quality checks

    • more validation added to form schema

    • there are always exceptions in form validation

    • infopath - no way to handle complex validation like "weight gain"

  • MVP (andy) Who gets/uses the report? What's the workflow around data quality reports?

    • Providers? Data Managers?

  • Rwanda (cheryl)

    • currently using hard-coded module to perform data quality checks

      • found 2000 historical errors, only 50 left after about a year

    • adding an indicator requires programming (e.g. it's difficult)

    • need to configure these rather than program them

    • needs: Data manager should be able to click a button to view report

    • automated report sent to each site every week

    • need to be able to distinguish between errors and abnormal conditions

  • Malawi (Evan)

    • point-of-care data entry (e.g. touchscreen) helps improve quality

    • in order to enter an "invalid" weight the user must authorize the change

  • AMPATH (Ada)

    • Warnings/alerts are ignored if they continuously popup.

  • Lesotho (Jeremy)

    • Re-evaluation of paper forms to find errors

  • Mike

    • brought up Google Summer of Code project data integrity module as a potential solution for some

  • Ada

    • module needs to be tested, workflow needs to be

    • data quality checks MUST happen on form entry!!!

    • OR at worst, same day validation

    • e.g. of an alert trigger "patient grows 30 centimeters in 3 weeks"

    • clinical reminders triggered on inaccurate data entry

  • Christina

    • clinical (difficult) vs technical (easy) validation

  • Evan

    • another challenge: once we find the errors, how do we resolve them

    • suggestion: batch entry form

    • cross patient off the validation fail list

  • Andy

    • what about metadata quality reports

      • # of days to get forms completed

      • provider is no longer seen as completing reports ... what happened?

  • Cheryl

    • uses data entry stats and data quality modules

    • has queries that support # fo forms completed by day and location

    • how long does it take for forms to be completed

    • what was behind a correction

    • important to know who made the error

    • is someone making an error a lot

    • trends are more important than finger pointing

      • fix training, workflow

  • Darius

    • can we get a concrete set of requirements for data quality rules engine

      • patient list / cohort builder query that shows all patients that have failed a particular check. could lead to a batch entry form to fix errors.

      • dashboard to show trends over time

  • Evan

    • define error -> cohort

    • alert on patient dashboard

    • track that error happened (don't imply)

    • error should not go away, we should comment that "error was fixed by ____"

  • Andy

    • A better term for data quality errors is "exception"

  • Evan

    • Add alert/exception to patient

  • Types of exceptions

    • universally impossible (absolutes) rules that are applied to all patients (birthdate cannot be in future)

    • errors that occur when accidentally "merging" patients (entering form for the wrong patient)

      • e.g. two encounters on the same date

      • always use identifier to search for patient (never allow search by name when entering a form)

    • suggestions ("did you really mean to do this ...?")

    • data quality vs clinical problem

    • list of patients who should have come in

    • tracking encounters over steps (registration > intake > ?)

    • rapid SMS -> alert form needs to be filled in

    • form design implications

      • validation as post processing (rather than form validation) will allow you to find issue with the form design/workflow

    • system allows user to "force" enter through authorization (like in Malawi)

    • not a report, but an interactive reporting/analysis tool (should not be a printed PDF)

    • what about when it looks like an error but isn't

      • e.g. height cannot increase a certain amount of time

      • "stop showing" rule (alerts create data): ignore forever | ignore until next visit

    • want to know that exceptions existed even if someone ignores the alert

    • e.g. "# of pills given" does not match calculation of "# of pills needed until next visit date"

      • associate alerts with single data points

    • are clinical issues different from data quality checks

      • workflow issue (who uses system), escalate quality error to clinicians (clinical issue)

      • always start as a data quality issue, move to a clinical issue

    • let's keep the collaboration alive: what's the best means for that?