Even more metadata enables better data use
By Ed Tittel
In my last tip, I walked you through the original DTD that we built for our testing engine, which lets us build question-and-answer sequences for all kinds of information that we can present to users, score automatically, and use to provide feedback on correct and incorrect answers. While this functionality proved pretty potent when we first created it about 30 months ago, we've made some changes to the DTD since then--no surprise, those changes relate almost entirely to metadata. Here I'll explain why we found the changes necessary.
To begin with, let me explain exactly what our XML test documents can do today:
- They can drive a Java applet for repeated standalone use on a CD or via download;
- They can drive a Java servlet for controlled access, where we can also monitor responses and gather question statistics;
- They can go through a transform and produce "electronic flashcards" for review purposes on a Palm Pilot, or other Palm OS PDA;
- They can go through a transform into pretty-printed access for print and review purposes;
- They can go through a transform into perfectly normal HTML documents for Web access;
- They support either straight line, fixed sequence tests, or branching adaptive exams;
- They can be subjected to random selection of a superset of n questions from a set of k questions, where k > n; selection criteria can also follow a menu based on categories and topics if necessary.
As we learned to use the stored documents that make up tests, and had to mix and match (not to mention update) their contents, we learned to make much more effective use of the information that described questions in the XML documents that contained them.
To be more specific, we added numerous attributes to our question elements so that we could group questions generally by category for scoring purposes (the category attribute), and by topic particularly to identify specific areas of coverage for adaptive testing (the topic attribute). Likewise, we added a difficulty attribute to permit questions to be assigned a numerical degree of difficulty from 0 to 999, and an association attribute to permit a question to be linked to a separate scenario description document (with its own independent DTD). We even added another legal value for the question element's type attribute (QHS) for hierarchical sort questions that require graphical ordering of a list of elements in one or more columns of values.
As we made these changes, we learned that this kind of metadata made it easy for us to expand our potential universe of coverage (more question types, more ways to organize and categorize questions). Adding this information also paid the unexpected dividend of allowing us to combine multiple documents on the same general topic more easily, particularly with the addition of a quick and dirty tool to make sure all id attributes for the question element stayed unique as documents were merged. We even figured out how to map individual questions into a SQL database, so that we could store our data in the form of organized records, yet continue to treat them entirely as XML documents. Clever use of a report generator made it easy for us to create custom XML documents on demand, in fact.
All of this leads me to a single brilliant point of insight: the more you know about the data, how you plan to use it, in what ways you wish to organize and categorize it, the more useful the metadata that abstractly represents this kind of information becomes. At present, several publishers and numerous training and teaching organizations have found our model easy enough to learn and use to make it part of their test-delivery strategies, and we keep finding new ways to use this technology in our own company. Did I mention it also does questionnaires? Ah, the joys of legitimate extensibility...which puts the well-deserved X into XML!
Contact Ed by sending an email to tips@searchxmlresources if you want a copy of the extended DTD, or if you have comments or questions. You can see a sample of the test engine's capabilities online at http://www.lanw.com/training/software.htm.
Ed Tittel is a principal at LANWrights, Inc., a wholly owned subsidiary of LeapIt.com. LANWrights offers training, writing, and consulting services on Internet, networking, and Web topics (including XML and XHTML), plus various IT certifications (Microsoft, Sun/Java, and Prosoft/CIW).