Exploring Validation in an End-to-end XML Architecture

Chimezie Ogbuji

John L. Clark

Cleveland Clinic Foundation

Our niche

• Clinical outcomes research

• Distributed, web-based data entry

• Patient records in XML

  • Repurposing

  • Ease of inspection and "hacking"

• Conversion to RDF for merging, inference, and querying

• Example query: "Patients who underwent an aortic or mitral valve repair or replacement at CCF for endocarditis between 1990 and 2006."

• SemanticDB

The flow

• Edit patient event or patient metadata (XForms in XHTML)

• "REST", "POX", and XML Processing

• Commit information to the patient record (HTTP → XSLT filter → XUpdate → save to 4Suite XML CMS)

• Generate and store RDF

• Query RDF (SPARQL)

• Content management "patterns"

  • Patient-focused ⇒ “process” the XML

  • Cohort-focused ⇒ “investigate” a collection of patient RDF graphs

The flow (diagram)

Why validate?

• Early error discovery and correction

  • Price for error discovery grows with the "distance travelled"

  • Error discovery may happen after a critical event

• High-value (high integrity) data

• Instant user feedback

Where to validate?

• As close to the user as possible

  • Web based system

  • XForms - user editing XML directly

  • Validate in the form, in the browser

Where to validate? (ETL gate)

• ETL: Extract, transform, load (Data warehousing)

• Between data sources

  • Conversion from deprecated RDBMs store to custom XML serialization

  • Conversion from custom XML to RDF

Validation and XForms

• Model Item Properties (MIPs)

  • readonly, relevant, calculate, p3ptype

  • type

  • required

  • constraint ☆

• The xf:bind element

• The :valid, :invalid, :required, and :optional CSS pseudo-classes

• The validate attribute on the xf:submission element

View source - XForms

From the example:

• The constraint binding:

  <xf:bind nodeset="date"
    constraint="translate(substring(., 1, 10),
                          'T:-', '') &lt;=
                translate(substring(now(), 1, 10),
                          'T:-', '')"/>

• The form control:

  <xf:input incremental="true" ref="date"/>

• CSS:

  :invalid:after { content: "\a0\2716 " }

Observations of XForms validation

• Boolean validation

  • Is a value valid or not?

  • No information about why a value might be invalid

• Directly architected on XPath context + XPath boolean expressions

• Enter Schematron

Intro to basic Schematron

• Example rule:

  <sch:rule context="date">
    <sch:assert
          test="translate(substring(., 1, 10),
                          'T:-', '') &lt;=
                translate(substring(now(), 1, 10),
                          'T:-', '')"
    >The date must not be
    in the future.</sch:assert>
  </sch:rule>

• Wait, where have we seen this before?

• Now with constraint descriptions!

• now() is problematic

XForms + Schematron?

• Not directly

• Associate a Schematron schema with an XForm

• Transform XForm to include assertions from schema

• Submit to validation service that uses same schema

• Get results in XML!

• Can XForms process submission results in XML?

What we do - the setup

• One "interesting entity" per XML file

• Set of high-level chunks of the XML file

  • Need for some sort of primary key notion

  • Used to allow editing of chunks instead of the entire file (using XUpdate for merging)

• Set of (possibly overlapping) views into potential chunks

• Each view corresponds to an XForm

• Describe set of XForms using a slightly abstracted vocabulary (called UIP)

• Each form description can optionally contain a Schematron schema

The setup: Diagram

What we do - the process

• Use XSLT to "compile" an XForm per view

• Compilation transforms inline Schematron into "equivalent" xf:binds

• Compilation "extracts" inline Schematron

• Salt XForms instance data with "state" information

• Serve and enjoy

• In the future: abstract aspects of Schematron for better XForms integration

The rest: Schematron critique

• Flat rule space

• Issues based on the (XSLT) implementation:

  • Contexts must be mutually exclusive

• Lacking useful metadata (hint: title on rules, other documentation)

• XPath function portability

The rest: XForms critique

• MIPs should (optionally) propagate up the DOM

• Multiple MIPs per node?

• Need for variables

• XPath function portability