So called „best“ ECDL paper ignores tagging

15. September 2008 um 13:39 4 Kommentare

The first normal talk at the ECDL2008 was about „Improving Placeholders in Digital Documents“ by George Buchanan and Jennifer Pearson. Placeholders (also known as bookmarks) in physical form are common and easy to use: you just put a peace of paper (or similar flat thing) inside a book, and optionally add notes on it. Bookmarks in the digital world are known but have low usability. A bookmark in a browser is just a plain link without additional information (color, notes, etc.), it only points to a webpage instead of specific parts of the content and it does not show up if you open the object that had been bookmarked. Moreover there is other content you may want to bookmark, for instance PDF files.

Buchanan and Pearson identified current uses of placeholders by user studies (interviews with a dozen of researchers) and created a demo interface for bookmarking PDFs. The talk was nice but I really wonder why it was selected as best paper. Especially the lack of referencing Social Tagging (Brewster Kahle brought it up in a question after the talk) would have been a clear reason for rejection if I was the reviewer! Social Tagging and Social Bookmarking has limitations – but it is much more than normal digital bookmarks and there are several tools to support tagging the files on your own hard disk. You should better try enhance tagging tools so tagging becomes more like annotating documents with placeholders instead of asking people to use artificial bookmarking tools. I hope the next talks will be better in scientific and practical relevance. Maybe the fact that Buchanan also got the best paper award last year (while beeing in the ECDL programm commitee) has influenced the decision? One more argument for open review.

Anyway: as long as there are no good interfaces to easily read and annotate documents, people will prefer paper notes for good reason. Give people a working e-paper-device with high resolution, long-lasting battery, touchscreen and the possibility to share documents without technical barriers like DRM and we can create a collaborative workspace for bookmarking and annotating documents; or as I already called it 2005 in the article „Mehr als Marginalien – das E-Book als gemeinsamer Zettelkasten“ in the German library science e-journal „LIBREAS“: a common file-card box.

4 Comments »

RSS feed for comments on this post. TrackBack URI

  1. Hi Jakob,

    I see you have some strong opinions about my paper.

    I think you’ve completely misinterpreted the scope of the paper, which was on individual users. We were studying the review of a paper by a sole user; nothing in our method was intended to address multiple users. Social tagging is a field eminently worthy of study, but no one paper can make an adequate go at all the issues surrounding even one topic.

    In my opinion, individual use provides a baseline of tools for social use, and is a necessary (though not sufficient) precondition for shared referencing. It also is not clear how the higher level tasks we extracted from placeholder use map onto social uses, which I would suspect would form an excellent ECDL paper in itself.

    Just because a paper does not match one’s own bias (and we all have them), does not make it a bad paper. I hope that you would treat papers by others within the terms and scope that they define – which is proper science – rather than on arbitrary terms imposed by oneself. Ultimately, if we take that approach as a community, we will end up rejecting much good work and letting through other material on the basis of „trendiness“ – which does happen in other conferences. PhDs would also prove impossible!

    For what it is worth, I believe that the choice of best paper in any conference has a lot of „noise“ in it, and though I have received many best paper awards, it is wise not to let it go to one’s head. I personally saw papers at the conference that I was impressed by, and would have been pleased to see recognised with that esteem. I was extremely shocked to be awarded the best paper twice. I can attribute my success not to my own merits, but rather those of my students, who I believe have been outstanding in every way. I would be appalled if some underhand process was at play. I strongly doubt that the Program Chairs who steer this critical decision would have engaged in any such behaviour.

    Turning to Open Review, whilst I strongly support a rethinking of the review process to make it transparent, those of us who have served as programme chairs of large conferences and international journals would be wary of its unbridled use. Science is sadly littered with feuds between „schools of thought“ and also anger over perceived insults given in review. The JCDL review process this year (for the second year at least) allowed reviewers to see each other’s comments without anonymity. I believe that this adds to the transparency of the review process, and I wish that conferences did more to explicitly state the degree to which accountability for one’s comments as a reviewer is upheld.

    Comment by George Buchanan — 18. September 2008 #

  2. Hi George – thanks for your comment. The more detailed I (re)read your paper, the more minor shortcomings I find (in addition to the lack of a reference to social bookmarking). But I also find more and more interesting points and additional research questions about bookmarks. As you conclude: „more detailed research is now needed“. Too sad that your paper is neither freely available online nor are there established ways to attach digital placeholders to specific parts of it. At least I would tag your paper and the related ECDL2008 paper „Superimposed Information Architecture for Digital Libraries“ (about referencing and annotating parts of digital documents) in the same way. Can you please at least upload your presentation slides to slideshare or some other open repository? This way the reader can get an own impression of the scope – and of course bookmark it!

    Comment by jakob — 19. September 2008 #

  3. There certainly are plenty of minor gaps throughout the paper – as is always the case when one looks closely! I would hope that others will follow up the many pathways from the paper to future work – I certainly can’t cover them all, and no doubt some interesting ideas will come from others that I would not have dreamed of!

    I was planning to put the slides for the talk on my website when I get back from a holiday in about 10 days time. It isn’t quite slideshare – but perhaps the conference will organise something?! If not, I will organise myself to put it in a useful public place…

    Comment by George Buchanan — 20. September 2008 #

  4. It’s funny that on a conference on digital libraries there is no visible and quick strategy to collect conference slides in a digital library 😉 More thinking about bookmarks I think that usually one differences between physical bookmarks and Internet bookmarks in an Web browser. But Internet bookmarks evolve, there are more and more different types, variants, and applications, especially with the evolution of social bookmarking. You paper deals about bookmarking (the general principle of using bookmarks) which is most time not explicitely dealed with. Further study on bookmarking should analyze social tagging but also respect traditional methods (you could ask Frank Divendal). hm, I’d like a student willing to do a broader thesis on bookmarking!

    Comment by jakob — 22. September 2008 #

Sorry, the comment form is closed at this time.