Jakoblog — Das Weblog von Jakob Voß

Metadaten – Versuch einer Kurzdefiniton

Obgleich ich dem Lexikon der Bibliotheks- und Informationswissenschaft (LBI) von Anfang an mit gemischten Gefühlen gegenüber stand – das Vorhaben eines gedruckten Lexikons ist anachronistisch und verspielt eine Chancen, die deutschsprachige Bibliotheks- und Informationswissenschaft als auf der Höhe der Zeit darzustellen – habe ich als Enzyklopädist inzwischen einige Artikel übernommen. Ich muss zugeben, dass die Beschränkungen des LBI auch einen gewissen Reiz haben. Vor allem ist die Länge der Artikel vorgegeben, so dass es darauf ankommt, einen Begriff in seiner Gänze auf das Wesentliche zu reduzieren. Der Begriff „Metadaten“, für den nächste Woche Abgabefrist ist, fällt mit bis zu 4.000 Zeichen in die umfangreichste Kategorie. Ich habe mit der Geschichte des Begriffs begonnen und versucht, das Wesentliche in diesem Umfang zusammenzufassen. Da sich die Bedeutung eines Begriffs erst aus seinen Relationen zu anderen Begriffen ergibt, habe ich auf möglichst viele andere, verwandte Einträgen des LBI verwiesen. Im Laufe der Diskussion vorgenommene Änderungen sind orange markiert.

Bei Google Books Ngram kann man schön den Anstieg der Verwendung des Begriffs nachvollziehen: Der deutlich zu erkennende Knick 1995 ist auf die Dublin Core Initiative zurückzuführen. Nun aber die Definition in ihrer aktuellen Form:

Über Korrekturen, Ergänzungen, Kritik und vorschläge für ein bis drei Literaturangaben würde ich mich freuen.

P.S.: Bei Mendeley habe ich eine Bibliographie mit Encyclopaedias of Library and Information Science erstellt. Im Terminosaurus Rex gibt es leider keinen Eintrag „Metadaten“.

XML Schema vs. Library APIs (OAI-PMH/SRU/unAPI…)

Much of our work at GBV library network has to do with record formats and APIs. We harvest or get metadata records in a wide range of formats (with many different interpretations and misconstructions of these formats), convert records to a wide range of formats (with many special request how to interpret this formats), and provide records through various APIs. Some of these APIs allow you to select different record formats, for instance OAI-PMH (first published 2001), SRU (2003), and unAPI (2006). These APIs are based on HTTP for transport and XML for encoding of the records. There are also older APIs and encoding formats like Z39.50 and newer APIs like pure Linked Data and SPARQL for RDF. unAPI also supports non-XML formats, but in this article I will concentrate on XML-based formats.

The basic question (that I deal with since years) is „what exactely is a format and how do you refer to it?“. All three APIs provide a method for listing of all formats that are supported by a particular server. unAPI provides a „list of object formats“. Each format has a „name“, a „type“ (which must be an official Internet media type), and an optional documentation URL („docs“), which may refer to some human-readable documentation, or to an XML Schema (XSD) file. Here are three examples:

<format name="oai_dc" type="application/xml"
  docs="http://www.openarchives.org/OAI/2.0/oai_dc.xsd" 
/>
<format name="pubmed" type="application/xml" 
  docs="http://www.nlm.nih.gov/bsd/licensee/elements_descriptions.html"
/>
<format name="mods" type="application/xml"
  docs="http://www.loc.gov/standards/mods/" 
/>
<format name="marcxml" type="application/xml" 
  docs="http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"
/>

To avoid the uncertainty whether „docs“ references a formal schema or a plain document, there should have been a „schema“ attribute (first problem). To refer to a format in an unAPI request, you use the format’s „name“. In OAI-PMH you refer to a format by its „metadataPrefix“. You can get a list of supported formats with the ListMetadataFormats request. In addition to the „metadataPrefix“ each format has the location of an XML Schema („schema“) and an XML Namespace URI („metadataNamespace“). In theory the latter is dispensable, because each XSD document declares a namespace URI in its „targetNamespace“ attribute: Given a format with a schema that defines namespace „http://example.org/“ like this

<xs:schema targetNamespace="http://example.org/">

I would expect records in this format to use this namespace, at least for the XML root element:

<record xmlns="http://example.org/">

The OAI-PMH specification does not explicitly say that the „metadataNamespace“ must match the namespace in the schema file „schema“. What does it mean if they differ? (second problem).

In SRU a format is known as „schema“. A list of supported formats is contained in an explain request. Each schema has an optional „title“, a „name“ (used to refer to schemas in the „recordSchema“ HTTP parameter when doing a search query), an „identifier“, and an optional „location“. The „identifier“ contains an additional URI, and the „location“ contains a link to an XML Schema file or to some human-readable documentation (like the „docs“ attribute in unAPI). There is a list of known schemas at the SRU page, for instance:

Note that one name (for instance „mods“) can refer to several schemas, but one particular SRU server can only provide one particular format under this name. The additional identifier neither refers to a particular XML Schema (Third problem). The identifier may only give a hint which particular version or interpretation of a format is provided.

Does anyone really need this diverse methods to refer to formats? I found in practice you cannot rely on the claimed format anyway, unless you can automatically validate it. That’s what XML Schema can be used for. I don’t say that XML Schema is the best or only method to formally describe an XML-based format (personally I much bettter like RELAX NG), but if there is an XML Schema – shouldn’t this schema be enough to identify the format?. Is there really a need of four independent identifiers to refer to an XML-based format? In the worst case we have:

• Schema Name (e.g. mods)
• Schema Location (e.g. http://www.loc.gov/standards/mods/v3/mods-3-3.xsd)
• Schema Identifier (e.g. info:srw/schema/1/mods-v3.3)
• Schema Namespace (e.g. http://www.loc.gov/mods/v3)

This is bad design, because you cannot say which of the four is the right one and how they relate to each other. A clean solution would only have two identifiers for XML-based formats:

• The local name, which is only unique for a particular API and a particular server
• The global schema Location, which is a cool URI that resolves to an XML Schema file.

The Schema Namespace is included as „targetNamespace“ in the XML Schema, and the Schema Identifier is delusion anyway. Either you can identify a format by a formal schema (that can also be used to validate records) or you just cannot guarantee which format your records will be in. Sure you can give some hints by linking to documentations, examples, and guidelines. But adding more identifiers is a fakery of control. You are still allowed to provide more specific formats, variants, application profiles, and interpretations under different names. But these formats don’t get more clear or usable if you give them a „Schema Identifier“. Does anyone uses SRU’s Schema Identifiers anyway? I think for XML we can better live with XML Schemas that the XML namespaces can be extracted from. An application can identify a format by its schema location, by the XML namespace, and/or by other information contained in the schema. Additional pointers to human-readable documentation are great. But don’t confuse description with identification if you need to refer to a data format.

P.S. At Code4lib mailing list Rob Sanderson pointed to our discussion we had about the same topic in 2009, and one of my earlier postings on XML4Lib also deals with SRU and namespaces.

Named Entity Recognition with DBPedia

Yesterday the DBPedia team released DBPedia Spotlight, a named entity recognition service based on structured data extracted from Wikipedia. You can access the service via Web APIs or download the software as Open Source. I could not resist to feed Spotlight its own description:

DBpedia Spotlight is a tool for annotating mentions of DBpedia resources in text, providing a solution for linking unstructured information sources to the Linked Open Data cloud through DBpedia. Text annotation has the potential of enhancing a wide range of applications including search, faceted browsing and navigation. By connecting text documents with DBpedia, our system enables a range of interesting use cases. For instance, the ontology can be used as background knowledge to display complementary information on web pages or to enhance information retrieval tasks. Moreover, faceted browsing over documents and customization of web feeds based on semantics become feasible. Finally, by following links from DBpedia into other data sources, the Linked Open Data cloud is pulled closer to the Web of Documents.

Pretty cool, isn’t it? Natural Language Processing (NLP) for information extraction seems to be the next hype after Web 2.0 and Semantic Web. I don’t neglect the innovative capabilities of DBPedia Spotlight and similar tools, but you should never forget that these are just tools, which won’t automatically solve information problems, or replace all other tools. Given the example above, there is little chance that an automatic system will extract you an exact topic of the text (for instance „named entity recognition based on data extracted from Wikipedia“) because this requires much background knowledge combining domain-specific expertise with common sense. By the way: as long as both Wikipedia and NLP-software is mainly written by white males, the result of will always mirror a limited world-view.

You can compare the results of Spotlight with similar open services:

• OpenCalais
• Extractiv
• Headup Entity Extraction
• Zeemanta
• AlchemyAPI

I found little overlap between the different services. Spotlight seems to provide more results (depending on the Text) on an error rate between 10% and 30%. You could use such tools for automatic subject indexing based on abstracts and use the result at least for ranking. Unfortunately in library metadata we often have no full text or abstract to annotate. Furthermore many library entities have no DBPedia entry but catalogers create new authority records if needed. What do you think, named entity recognition and other NLP techniques can be used for in metadata land? Can we give up controlled subject indexing in libraries in favour of automatic NLP-based indexing on the one side and social tagging on the other? Or is room for all of these approaches, and how can you successfully combine them?