Everybody's Libraries

January 1, 2012

Public Domain Day 2012: Five things we can do in the US

Filed under: copyright,libraries,online books,open access — John Mark Ockerbloom @ 10:24 am

It’s New Year’s Day again, and in much of the world, this means another year’s worth of works enter the public domain.  That’s a cause for celebration, as Europe and many other countries that have “life+70 years” copyright terms welcome works by James Joyce, Virginia Woolf, Jelly Roll Morton, and Elizabeth von Arnim into the public domain.  The Communia Project’s Public Domain Day website focuses on works by these and many other authors that are entering (in many cases, re-entering) the public domain in “life+70 years” countries.  Meanwhie, folks in Canada, New Zealand, and other countries that have held the line at the “life+50 years” terms of the Berne Convention can now freely enjoy the works of people like James Thurber, Ernest Hemingway, and H.D.

There’s not so much excitement about Public Domain Day in the US, where no published works are scheduled to enter the public domain for another 7 years, due to a 20-year copyright extension enacted in 1998.  But Americans don’t have to simply sigh and contemplate what might have been if our copyright terms hadn’t been extended.  The new year still provides a number of important opportunities for Americans to improve access to the public domain.

1. Find and free newly public domain unpublished works

Some works are going into the public domain in the US today: works never published prior to 2003 (or copyrighted under US law prior to 1978) by authors who died in 1941– the same authors whose published works go into the public domain in Europe today.

But who would care about such obscure works? one might ask.  Well, if you’re at all interested in understanding the dense, allusion-laden fiction of Joyce, or the psychology of Woolf, or the jurisprudential thinking of Louis Brandeis, or the inner lives of any of the rest of the “class of 1941″, having the right to freely access, publish, and build on their unpublished works can be crucial.

Up until now, for instance, scholars studying James Joyce have often been frustrated by sharp restrictions and legal threats made by the administrator of Joyce’s literary estate.  In 2008, Rebecca Ganz characterized the administrator thus: “[His] primary purpose is to quell any scholarship that he finds distasteful or an invasion of his family’s privacy. He has a history of harassing authors and artists until they buckle under the strain of trying to obtain legal rights to quote from the late author’s writings.”  Scholars wishing to invoke Joyce’s unpublished works in their work have either had to undertake multi-year legal battles, or cut back on the lines of inquiry they might otherwise pursue.

American libraries and archives have many illuminating papers by authors who died in 1941– even non-US authors like Joyce and Woolf.  US digitizers, librarians, and archivists can open up and publicize these works.   In some cases, we’re uniquely positioned to do so, since their unpublished works may still be under copyright in some other countries.

2. Increase worldwide availability of public domain works

Many of the millions of digitized books on the Internet are hosted in the US, in large-scale repositories like Google Books, HathiTrust, and the Internet Archive.  Many of these services give limited access to non-US readers or materials.  Google and HathiTrust, for instance, limit non-US access by default to books published as long as 140 years ago, to avoid falling afoul of “life+70 years” copyright terms abroad.  JSTOR likewise limits access to non-US journal volumes published in 1870 or later.

With another year’s worth of copyrights expiring in “life+70 years” countries, it should be safe for these US-based services to also open up worldwide access to another year’s worth of works, further freeing up the public domain.  HathiTrust is also willing to manually review copyrights on specific books to open up access.  If you come across any books in HathiTrust solely by authors who died in 1941 (or before) that are currently labeled only as “public domain in the United States”, you can request that they review it for opening up access worldwide.  Just use the “Feedback” button at the bottom of the book’s HathiTrust page, or the suggestion form on my Online Books Page; and make sure you ask specifically for non-US access.

3. Restore access to obscure copyrighted works from 1936 (and earlier)

After libraries and archives expressed concerns about the fate of obscure works under longer copyright terms, Congress included a special exemption in their 1998 copyright tem extension.  The exemption, codified as section 108(h) of the copyright law, states that “during the last 20 years of any term of copyright of a published work, a library or archives, including a nonprofit educational institution that functions as such, may reproduce, distribute, display, or perform in facsimile or digital form a copy or phonorecord of such work, or portions thereof, for purposes of preservation, scholarship, or research”, under certain conditions.  In particular, if the institution finds, after a reasonable investigation, that such a work is not “subject to normal commercial exploitation” (such as by being in print) and cannot “be obtained at a reasonable price”, and no rightsholder has filed a claim otherwise, the work qualifies for this special exemption.  As of this year’s Public Domain Day, qualifying publications from 1936 join what is now 14 years of works in this category.

So far, I have found very little digitized content online where this exemption is explicitly invoked.  (There are advantages to explicitly doing so, both because it helps clarify the right to use the material, and helps prevent inadvertent unauthorized propagation of the works, such as the commercial reprints of digitized books that are now common on many large bookselling sites.)  Yet many of the works in HathiTrust’s (currently suspended) orphan works initiative, and in the Internet Archive’s lending library, and more besides, could well qualify for this treatment– and unlike orphan works, where legislation has yet to be passed, the exemption for these materials is already explicitly authorized by statute.

Providing online access for these works is not without controversy.  A 2002 article by lawyer Mary Minow details some of the potential possibilities and risks.   While she concludes that libraries can put such works on the Web, the recent Author’s Guild complaint in its lawsuit against HathiTrust includes some push-back against this idea. But as the public domain in the US recedes further into history, and digital library projects increasingly look for ways to make our cultural heritage available online, American libraries would do well to proactively establish and exercise these rights for older works now languishing in obscurity.

4. Strengthen and sustain coalitions for reasonable copyright limits

The curtailment of the public domain is just one aspect of the overreach of copyright law in the US and elsewhere.  Right now, Congress is considering two bills, the Stop Online Piracy Act (SOPA) and the PROTECT IP Act (PIPA), whose enforcement provisions threaten to disrupt the core structures of the Internet and enable far-reaching censorship, in the name of stopping piracy.  Supporters of these bills hoped to have them passed by Christmas, but opposition from both “left” and “right” sides of the political spectrum has slowed the process down, caused some companies to withdraw support, and led to the proposal of less harmful alternatives for fighting piracy.

It’s still quite possible that SOPA and PIPA will pass, though.   Public Domain Day provides an opportunity for Americans to reflect on some of the good reasons for limiting the power and scope of copyright enforcement, and to redouble efforts to keep those limits reasonable.  Moreover, a coalition that can stop SOPA and PIPA can also work to prevent further extensions of copyright terms.  This can ensure that Americans will have more to celebrate in Public Domain Days to come– especially starting in 2019, when the remaining 1923 copyrights should finally expire in the US.

5. Give copyrights of your own to the public domain

Of course, those wishing to maximize public access and use of their works don’t have to wait for their copyrights to expire on their own.  They can dedicate them to the public domain any time they want.  Public Domain Day is a particularly auspicious time to make such gifts, no matter what country you’re in.  And with tools like the CC0 declaration, it’s easier than ever to do so.

A few years ago, I started an annual personal tradition of reviewing copyrights to works I’d created more than 14 years ago (the original initial term of copyright enacted by the founders of the US, and also approximately the ideal copyright term given in a recent economic analysis) and dedicating works to the public domain that I didn’t feel needed further copyright.  Accordingly, today I dedicate all the work of my creation that I published in 1997, and for which I still control rights, to the public domain.  For me, this consists primarily of websites like The Online Books Page as of that year, and other online writings.  But others have dedicated more high-profile material to the public domain after the same term.   And I’d be very happy to hear from others who are making similar dedications today (whether or not it’s after 14 years).

So, happy Public Domain Day to everyone in the US and elsewhere!  We all have things to celebrate, and things we can do, in the name of the public domain.

September 27, 2011

Libraries: Be careful what your web sites “Like”

Filed under: crimes and misdemeanors,data,libraries,people,privacy — John Mark Ockerbloom @ 6:15 pm

Imagine you’re working in a library, and someone with a suit and a buzz cut comes up to you, gestures towards a patron who’s leaving the building, and says “That guy you were just helping out; can you tell me what books he was looking at?”

Many librarians would react to this request with alarm.  The code of ethics adopted by the American Library Association states “We protect each library user’s right to privacy and confidentiality with respect to information sought or received and resources consulted, borrowed, acquired or transmitted.”  Librarians will typically refuse to give such information without a carefully-verified search warrant, and many are also campaigning against the particularly intrusive search demands authorized by the PATRIOT Act.

Yet it’s possible that the library in this scenario is routinely giving out that kind of information, without the knowledge or consent of librarians or patrons, via its web site.  These days, many sites, including those of libraries, invoke a variety of third-party services to construct their web pages.  For instance, some library sites use Google services to analyze site usage trends or to display book covers.  Those third party services often know what web page has been visited when they’re invoked, either through an identifier in the HTML or Javascript code used to invoke the service, or simply through the Referer information passed from the user’s web browser.

Patron privacy is particularly at risk when the third party also knows the identity of users visiting sensitive pages (like pages disclosing books they’re interested in).  The social networking sites that many library patrons use, for instance, can often track where their users go on the Web, even after they’ve left the social sites themselves.

For instance, if you go to the website of the Farmington Public Library (a library I used a lot when growing up in Connecticut), and search through their catalog, you may see Facebook “Like” buttons on the results.  On this page, for example, you may see that four people (possibly more by the time you read this) have told Facebook they Liked the book Indistinguishable from Magic.  Now, you can probably easily guess that if you click the Like button, and have a Facebook account, then Facebook will know that you liked the book too.  No big surprise there.

But what you can’t easily tell is that  Facebook is informed you’ve looked at this book page, even if you don’t click on anything.  If you’re a Facebook user and haven’t logged out– and for a while recently, even if you have logged out– Facebook knows your identity.  And if Facebook knows who you are and what you’re looking at, it has the power to pass along this information. It might do it through a “frictionless sharing” app you decided to try.  Or it might quietly provide it to organizations that it can sell your data to as permitted in its frequently changing data use policies.  (Which for a while even included tracking non-members.)

For some users, it might not be a big deal if it’s generally known what books they’re looking at online. But for others it definitely is a big deal, at least some of the time.  The problem with third-party inclusions like the Facebook “Like” button in catalogs is that library patrons may be denied the opportunity to give informed consent to sharing their browsing with others.  Libraries committed to protecting their patron’s privacy as part of their freedom to read need to carefully consider what third party services they invite to “tag along” when patrons browse their sites.

This isn’t just a Facebook issue.  Similar issues come up with other third-party services that also track individuals, as for instance Google does.  Libraries also have good reasons to partner with third party sites for various purposes.  For some of these purposes, like ebook provision, privacy concerns are fairly well understood and carefully considered by most libraries.  But librarians might not keep as close track of the development of their own web sites, where privacy leaks can spring up unnoticed.

So if any of your web sites (especially your online catalogs or other discovery and delivery services) use third party web services, consider carefully where and how they’re being invoked.  For each third party, you should ask what information they can get from users browsing your web site, what other information they have from other sources (like the “real names” and exact birthdates that sites like Facebook and Google+ demand), and what real guarantees, if any, they make about the privacy of the information.  If you can’t easily get satisfactory answers to these questions, then reconsider your use of these services.

June 15, 2011

A digital public library we still need, and could build now

Filed under: citizen librarians,copyright,libraries,people,sharing — John Mark Ockerbloom @ 12:39 pm

It’s been more than half a year since the Digital Public Library of America project was formally launched, and I’m still trying to figure out what the project organizers really want it to be.  The idea of “a digital library in service of the American public” is a good one, and many existing digital libraries already play that role in a variety of ways.  As I said when I christened this blog, I’m all for creating a multitude of libraries to serve a diversity of audiences and information needs.

At a certain point after an enthusiastic band of performers says “Let’s put on a show!”, though, someone has to decide what their show’s going to be about, and start focusing effort there.  So far, the DPLA seems to be taking an opportunistic approach.  Instead of promulgating a particular blueprint for what they’ll do, they’re asking the community for suggestions, in a “beta sprint” that ends today.   Whether this results in a clear distinctive direction for the project, or a mishmash of ideas from other digitization, aggregation, preservation, and public service initiatives, remains to be seen.

Just about every digital project I’ve seen is opportunistic to some extent.   In particular, most of the big ones are opportunistic when it comes to collection development.  We go after the books, documents, and other knowledge resources that are close to hand in our physical collections, or that we find people putting on the open web, or that our users suggest, or volunteer to provide on their own.

There are a number of good reasons for this sort of opportunism.  It lets us reuse work that we don’t have to redo ourselves.  It can inform us of audience interests and needs (at least as far as the interests of the producers we find align with the interests of the consumers we serve).  And it’s cheap, and that’s nothing to sneer at when budgets are tight.

But the public libraries that my family prefers to use don’t, on the whole, have opportunistically built collections.  Rather, they have collections shaped primarily by the needs of their patrons, and not primarily by the types of materials they can easily acquire.   The “opportunistic” community and school library collections I’ve seen tend to be the underfunded ones, where books in which we have yet to land on the Moon, the Soviet Union is still around, or Alaska is not yet a state may be more visible than books that reflect current knowledge or world events.  The better libraries may still have older titles in their research stacks, but they lead with books that have current relevance to their community, and they go out of their way to acquire reliable, readable resources for whatever information needs their users have.  In other words, their collections and services are driven by  demand, not supply.

In the digital realm, we have yet to see a library that freely provides such a digital collection at large scale for American public library users.   Which is not to say we don’t have large digital book collections– the one I maintain, for instance, has over a million freely readable titles, and Google Books and lots of other smaller digital projects have millions more.  But they function more as research or special-purpose collections than as collections for general public reference, education, or enjoyment.

The big reason for this, of course, is copyright.  In the US, anyone can freely digitize books and other resources published before 1923, but providing anything published after that requires copyright research and, usually, licensing, that tends to be both complex and expensive.  So the tendency of a lot of digital library projects is to focus on the older, obviously free material, and have little current material.  But a generally useful digital public library needs to be different.

And it can be, with the right motivation, strategy, and support.  The key insight is that while a strong digital public library needs to have high-quality, current knowledge resources, it doesn’t need to have all such resources, or even the most popular or commercially successful ones.  It just needs to acquire and maintain a few high-quality resources for each of the significant needs and aptitudes of its audience. Mind you, that’s still a lot of ground to cover, especially when you consider all the ages, education levels, languages, physical and mental abilities, vocational needs, interests, and demographic backgrounds that even a midsized town’s public library serves.  But it’s still a substantially smaller problem, and involves a smaller cost, than the enticing but elusive idea of providing instant free online access to everything for everyone.

There are various ways public digital libraries could acquire suitable materials proactively.  The America.gov books collection provides one interesting example.  The US State Department wanted to create a library of easy-to-read books on civics and American culture and history for an international audience.  Some of these books were created in-house by government staff.  Others were commissioned to outside authors.  Still others were adapted from previously published works, for which the State Department acquired rights.

A public digital library could similarly create, commission, solicit, or acquire rights to books that meet unfilled information needs of its patrons.  Ideally it would aim to acquire rights not just to distribute a work as-is, but also to adapt and remix into new works, as many Creative Commons licenses allow.  This can potentially greatly increase the impact of any given work.  For instance, a compellingly written,  beautifully illustrated book on dinosaurs might be originally written for 9-12 year old English speakers, and be noticeably obsolete due to new discoveries after 5 or 10 years.  But if a library’s community has reuse and adaptation rights, library members can translate, adapt, and update the book, so it becomes useful to a larger audience over a longer period of time.

This sort of collection building can potentially be expensive; indeed, it’s sobering that America.gov has now ceased being updated, due to budget cuts.  But there’s a lot that can be produced relatively inexpensively.  Khan Academy, for example, contains thousands of short, simple educational videos, exercises, and assessments created largely by one person, with the eventual goal of systematically covering the entire standard K-12 curriculum.  While I think a good educational library will require the involvement of many more people, the Khan example shows how much one person can get accomplished with a small budget, and projects like Wikipedia show that there’s plenty of cognitive surplus to go around, that a public library effort might usefully tap into.

Moreover, the markets for rights to previously authored content can potentially be made much more efficient than they are now.  Most books, for instance, go out of print relatively quickly, with little or no commercial exploitation thereafter.  And as others have noted, just trying to get permission to use  a work digitally, even apart from any royalties, can be very expensive and time-consuming.  But new initiatives like Gluejar aim to make it easier to match up people who would be happy to share their book rights with people who want to reuse them. Authors can collect a small fee (which could easily be higher than the residual royalties on an out-of-print book); readers get to share and adapt books that are useful to them.   And that can potentially be much cheaper than acquiring the rights to a new work, or creating one from scratch.

As I’ve described above, then, a digital public library could proactively build an accessible collection of high-quality, up to date online books and other knowledge resources, by finding, soliciting, acquiring, creating, and adapting works in response to the information needs of its users.  It would build up its collection proactively and systematically, while still being opportunistic enough to spot and pursue fruitful new collection possibilities.  Such a digital library could be a very useful supplement to local public libraries, would be open any time anywhere online, and could provide more resources and accessibility options than a local public library could provide on its own.  It would require a lot of people working together to make it work, including bibliographers, public service liaisons, authors, technical developers, and volunteers, both inside and outside existing libraries.  And it would require ongoing support, like other public libraries do, though a library that successfully serves a wide audience could also potentially tap into a wide base of funds and in-kind contributions.

Whether or not the DPLA plans to do it, I think a large-scale digital free public library with a proactively-built, high-quality, broad-audience general collection is something that a civilized society can and should build.  I’d be interested in hearing if others feel the same, or have suggestions, critiques, or alternatives to offer.

April 9, 2011

Opt in for open access

Filed under: copyright,libraries,online books,open access — John Mark Ockerbloom @ 8:40 am

There’s been much discussion online about Judge Chin’s long-awaited decision to reject the settlement proposed by Google and authors and publishers’ organizations over the Google Books service. Settlement discussions continue (and the court has ordered a status conference for April 25).  But it’s clear that it will be a while before this case is fully settled or decided.

Don’t count on a settlement to produce a comprehensive library

When the suit is finally resolved, it will not enable the comprehensive retrospective digital library I had been hoping for.  That, Chin clearly indicated, was an over-reach.  The  proposed settlement would have allowed Google to sell access to most pre-2009 books published in the English-speaking world whose rightsholders had not opted out.   But, as Chin wrote, “the case was about the use of an indexing and searching tool, not the sale of complete copyrighted works.”  The changes in the American copyright regime that the proposed settlement entailed, he wrote, were too sweeping for a court to approve.

Unless Congress makes changes in copyright law, then, a rightsholder has to opt in for a copyrighted book to be made readable on Google (or on another book site).  Chin’s opinion ends with a strong recommendation for the parties to craft a settlement that would largely be based on “opt-in”.  Of course, an “opt in” requirement necessarily excludes orphan works, where one cannot find a rightsholder to opt in.  And as John Wilkin recently pointed out, it’s likely that a lot of the books held by research libraries are orphan works.

Don’t count on authors to step up spontaneously

Chin expects that many authors will naturally want to opt in to make their works widely available, perhaps even without payment.  “Academic authors, almost by definition, are committed to maximizing access to knowledge,” he writes.  Indeed, one of the reasons he gives for rejecting the settlement is the argument, advanced by Pamela Samuelson and some other objectors, that the interests of academic and other non-commercially motivated authors are different from those of the commercial organizations that largely drove the settlement negotiations.

I think that Chin is right that many authors, particularly academics, care more about having their work appreciated by readers than about making money off of it.  And even those who want to maximize their earnings on new releases may prefer freely sharing their out of print books to keeping them locked away, or making a pittance on paywall-mediated access.  But that doesn’t necessarily mean that we’ll see all, or even most, of these works “opted in” to a universally accessible library.  We’ve had plenty of experience with institutional repositories showing us that even when authors are fine in principle with making their work freely available, most will not go out of their way to put their work in open-access repositories, unless there are strong forces mandating or proactively encouraging it.

Don’t count on Congress to solve the problem

The closest analogue to a “mandate” for making older books generally available would be orphan works legislation.    If well crafted, such a law could make a lot of books available to the public that now have no claimants, revenue, or current audience, and I hope that a coalition can come together to get a good law passed. But an orphan works law could take years to adopt (indeed, it’s already been debated for years). There’s no guarantee on how useful or fair the law that eventually gets passed would be, after all the committees and interest groups are done with it.  And even the best law would not cover many books that could go into a universal digital library.

Libraries have what it takes, if they’re proactive

On the other hand, we have an unprecedented opportunity right now to proactively encourage authors (academic or otherwise) to make their works freely available online.  As Google and various other projects continue to scan books from library collections, we now have millions of these authors’ books deposited in “dark” digital archives.  All an interested author has to do is say the word, and the dark  copy can be lit up for open access.  And libraries are uniquely positioned to find and encourage the authors in their communities to do this.

It’s now pretty easy to do, in many cases.  Hathi Trust, a coalition of a growing number of research institutions, currently has over 8 million volumes digitized from member libraries.  Most of the books are currently inaccessible due to copyright.  But they’ve published a permission agreement form that an author or other rightsholder can fill out and send in if they want to make their book freely readable online.  The form could be made a bit clearer and more visible, but it’s workable as it is.  As editor of The Online Books Page, I not infrequently hear from people who want to share their out of print books, or those of their ancestors, with the world.  Previously, I had to worry about how the books would get online.  Now I usually can just verify it’s in Hathi’s collection, and then refer them to the form.

Google Books also lets authors grant access rights through their partner program.  Joining the program is more complicated than sending in the Hathi form, and it’s more oriented towards selling books than sharing them.  But Google Books partners can declare their books freely readable in full if they wish, and can give them Creative Commons licenses (as they can with Hathi).  Google has even more digitized books in its archives than Hathi does.

So, all those who would love to see a wide-ranging (if not entirely comprehensive), globally accessible digital library now have a real opportunity to make it happen.  We don’t have to wait for Congress to act, or  some new utopian digital library to arise.  Thanks to mass digitization, library coalitions like Hathi’s, and the development of simplified, streamlined rights and permissions processes, it’s easier than ever for interested authors (and heirs, and publishers) to make their work freely available online.  If those us involved in libraries, scholarship, and the open access movement work to open up our own books, and those of our colleagues, we can light up access to the large, universal digital library that’s now waiting for us online.

October 29, 2010

Two talks on making the most of metadata

Filed under: libraries,metadata,online books,subjects — John Mark Ockerbloom @ 9:11 pm

Earlier this week, I participated in the Books Online workshop in Toronto. The workshop featured an update from James Crawford on Google Books, and papers on social reading and ebook interfaces, and the development of ebook services for various communities with special needs, including children, isolated First Nations communities, and the visually impaired. There were also reports from various other research projects, mostly outside the US.  It was an especially good opportunity to get in touch with projects and people I don’t ordinarily encounter, since I don’t do much international travel at the moment.

My own contribution was a keynote titled “The Metadata Challenge“, discussing some ways in which metadata can be used effectively to support discovery, access, and usability in large-scale digital libraries.  The talk covers a wide range of topics, including several metadata-related projects I’ve written about here, such as subject maps, copyright data, and work-oriented catalog views.  I’ve posted slides and notes from the talk on my Metadata Challenge page, which you’re welcome to download and read.

This coming Tuesday, I’ll be giving a more tightly focused metadata talk at the Digital Library Federation forum in Palo Alto.  I’ll be going into detail about how we use freely distributed linked authority data on subjects from the Library of Congress to improve discovery in catalogs being maintained and developed at Penn.  In the same session, Kevin Ford of the Library of Congress will talk about recent changes and new improvements in the service we’re using.

I expect my presentation to include a live demo and other interactive elements, and we both hope to leave plenty of time for questions and discussion.  If you’re interested in how you can use LC’s authority data, or data like it, to improve subject-oriented catalogs, I encourage you to attend the session if you can make it.

September 8, 2010

“What do you read, my lord?” “Works, works, works”

Filed under: discovery,libraries,metadata,online books — John Mark Ockerbloom @ 10:08 pm

There’s been a lot of talk lately in the library world about the coming age of FRBR-ized library catalogs (prompted in part by development of RDA, a cataloging standard that uses FRBR).  Exactly what such catalogs will look like, and whether they will actually help readers use the library more effectively, are matters of ongoing debate.  One of the key differences between FRBR and older catalog models is that books and other resources that share common properties can be grouped together at various levels of abstraction.

FRBR, highly abridged

Such grouping can be helpful, for instance, for people looking for a suitable copy of Shakespeare’s play Hamlet.  In a traditional library catalog, a title search for Hamlet might yield a long list of hundreds of hits, in which it is difficult to select a particular copy, or to find other appropriate search results (like William Faulkner’s book The Hamlet) among all the hits for various versions of Shakespeare’s creation.  In a FRBR-ized catalog, though, the various editions (or “Manifestations“, in FRBR-speak) of Hamlet can be grouped into various “Expressions“, denoting particular texts of Hamlet, and those Expressions can be grouped into a single “Work” denoting Shakespeare’s dramatic creation.

Ideally, a general search in a FRBR-ized catalog for Hamlet would yield one hit for Shakespeare’s Hamlet as a Work, instead of hundreds of editions (unless the search terms implied a more specific edition or version).  Users could then examine the Work hit to choose an appropriate copy; or they could more easily find other “Hamlet” hits if they really wanted something else, like the similarly-titled book by Faulkner.  (The specific copy that the user finally obtains is known as an “Item”, and represents a level of abstraction below Manifestation.  The full resource abstraction stack is sometimes referred to as “WEMI”, short for “Work, Expression, Manifestation, Item”.)

In practice, there have been plenty of arguments over the details of the FRBR information model, such as where exactly the distinction is made between a Work and an Expression, or where one should model multiple Works instead of one.  Ultimately, though, FRBR is intended to make it easier for users to find, identify, select, and obtain library resources.  Each of these four verbs is explicitly defined as a FRBR “user task”.  The information model is useful only to the extent that it supports these user tasks.

FRBR-like catalogs on the Web today

Even while libraries are debating whether or not they will develop FRBR-based catalogs, several FRBR-like catalogs have appeared on the Web, and are being used by growing audiences of online users.  They have some interesting things in common.  They all support some degree of representation of works at a higher level than the traditional library records.  And they all honor the FRBR information model more in the breach than the observance  None of them use the FRBR WEMI model as-is, at least in the user interface, though many of their designers cite FRBR as an influence.

Typically, the  “FRBR-like” catalogs now in common use show “works” and “editions”. “Works” here refer to something like FRBR Works or Expressions; the distinction between the two is not usually made in the user-facing production catalogs I’ve seen.  “Editions” refer to particular FRBR Manifestations. These catalogs consolidate search results that all fall under a particular “work” (thus streamlining the FRBR “Find” task), and also offer “work”-oriented displays that include details about the “editions” encompassed by the work (thus supporting the “Identify” task).  They do not give much explicit assistance towards Selecting (though most traditional library catalogs do not either).  And in many cases the metadata for editions is not consistent enough to support a consistent Work grouping throughout the catalog (let alone one that is consistent with other catalogs).

Example 1: OpenLibrary

As an example of how this works in practice, consider the user searching for “Hamlet” in the Open library.  Once we get past an oddly malformed first hit, the second hit is for the play “Hamlet” as a work, consolidating 665 editions beneath it.  (We see also Faulkner’s The Hamlet right below that, instead of several screenfuls later.)

If we click on the hit for the play, we then go to a work page that shows information about the play in general, followed by the list of 665 associated editions.  There’s not a whole lot here to help users choose a good edition, though the display does show covers or title pages when available.  Also, the online editions, which are the easiest for Web users to obtain, are shown first. (The online book metadata comes from the Internet Archive, the sponsor of Open Library.)

Example 2: WorldCat.org

OCLC uses similar technologies in its Worldcat.org and WorldCat Local products.  Searching for Hamlet in WorldCat.org, for instance, we get a hit for the play followed by hits for various movies and books. If you click on the title “The tragedy of Hamlet, prince of Denmark” in the first search result, you’ll just see information on the featured edition (from Yale University Press).   But if you click instead of “View all editions and formats” note at the bottom of the first search result, you’ll see that over 4000 editions are lumped under this work.  You can cut down this list somewhat using facets that specify format (book, video, audio), language, and/or year of release.

As far as I know, OCLC does not maintain an explicit work record for Hamlet; a lot of the grouping here appears to be automatically inferred from existing ‘edition’ records.  The list contains a wide variety of Hamlets, including translations, audio readings and performances, and videos of stage productions and film adaptations.  But it’s an incomplete list; the DVD of Kenneth Branagh’s film of Hamlet, for instance, shows up in the initial Hamlet search results, and doesn’t seem to included as one of the DVD editions under the first “Hamlet” hit.

Example 3: LibraryThing

In LibraryThing, the work is the thing, and editions are secondary.  Their work page for Hamlet combines information on the play with information on various editions, without saying very much about editions.

Since most LibraryThing users are more concerned with a work in general than with a particular edition, this sort of catchall display works reasonably well in context. Amazon has done similar consolidation in its catalog for years, which has well-known pros and cons.  (You’ll see Amazon reviews of multiple editions of a work together, for instance, which is useful if you want a wide range of views but possibly misleading when a reviewer complains about the features of a specific edition.)  Amazon still gives support to identifying and selecting particular editions, though, since they need to distinguish them for customers.  Although LibraryThing, not driven by sales in the way that Amazon is, gives less edition-level support, they do point to 9 distinct Amazon records for Hamlet.

In a few cases, specially notable editions of Hamlet, such as this Norton Critical edition, get their own pages on LibraryThing.  This is useful for folks who want to record information about an edition that’s particularly significant to them.  But at present, there’s no easy way within LibraryThing to link the generic work pages with the edition-specific pages, so information gets somewhat fragmented.

Example 4: Google Books

The widely used Google Books also tries to aggregate similar books together in somewhat FRBR-like ways.  Different “editions” are consolidated in search results, and related books are linked together.  Unfortunately, Google’s metadata is dirty enough that the “FRBR-ization” is only partial, and confusing.  Different “Editions” of a book may simply be different volumes from the same multi-volume edition.  And editions of the same work may sometimes be combined together, and sometimes left separate, seemingly at random.

If you do a search for Hamlet in Google Books, each of the hits aggregates lots of volumes, but this is not immediately apparent.  If you click on the “Book overview” for a hit, you’ll see some of these volumes listed under “Related books” and “Other editions”.  The metadata is not consistent enough to be properly “FRBR-like”, but that seems to be what they’re aiming for.

Users not familiar with the quirks of Google Books may miss some volumes that are actually in the system.  For books less popular than Hamlet, users may see a much shorter list of hits that does *not* include the particular volume or edition that a searcher is looking for, and not realize they have to look through “Other editions” under the hits shown to find the volume they want.  For instance, this search for Oldcourt (as I write this) makes it appear as if Google has only two of the three volumes of this novel available in full view.  In fact, they have all three volumes, but the results can mislead even experienced literary researchers.

Reflections

All of the catalogs above, then, are somewhat “FRBR-like”, but they don’t fully implement the FRBR functional or data model.  I’m not sure, though, how closely they need to conform to those models. I can see room for improvement in each catalog, but they all seem to work well enough to have gained notable user communities.

How should a “FRBR”-like catalog treat Hamlet?  Personally, I like the rich work-level data (both formal and infomal) that I find on pages like LibraryThing.  I also like the easy access to online copies provided by Open Library, the faceting and wide-angle “work” aggregation of WorldCat.org, and the scale and full-text searchability of Google Books.   On the other hand, I’d like to see more consistent grouping and descriptions in each of the catalogs, and more assistance for users in selecting an appropriate edition than I currently see in any of them.

Out of hundreds of editions of Hamlet, some are particularly useful for various audiences, such as students trying to understand Elizabethan speech, actors and directors preparing to perform it, literary researchers examining and comparing source texts, and cultural historians considering famous stagings and adaptations.  Why not include more data highlighting editions that are particularly useful for these and other purposes?  I don’t recall such metadata being a significant part of the FRBR data model, but it seems to be in the spirit of the functional model of helping people select the right book for their needs.

On the other hand, I don’t see a great need to make a lot of differentiation between works of Hamlet and expressions of Hamlet.  Insisting on lots of sharp distinctions between various high-level records in the user interface could well confuse users more than it helps them, unless there are effective ways of presenting them as a unit when appropriate.

What’s next?

As my library and others roll out the next generation of library management systems and discovery tools, determining the best use of FRBR-ish catalog data and applications may make a big difference in the effectiveness of our services to patrons. So I’m very interested in seeing how well catalogs and records designed along FRBR lines work in practice.  I’m also piloting some prototypes of FRBR-like features on The Online Books Page, and I hope to have more to say about them shortly.  (Update: Now I do; see my next post.)

In the meantime, I’d love to hear comments from readers on any of the catalogs I’ve mentioned, or examples of other “FRBR-like” catalogs in use that are worth looking at.

June 11, 2010

Journal liberation: A primer

Filed under: copyright,libraries,open access,publishing,sharing — John Mark Ockerbloom @ 10:07 am

As Dorothea Salo recently noted, the problem of limited access to high-priced scholarly journals may be reaching a crisis point.  Researchers that are not at a university, or are at a not-so-wealthy one, have long been frustrated by journals that are too expensive for them to read (except via slow and cumbersome inter-library loan, or distant library visits).  Now, major universities are feeling the pain as well, as bad economic news has forced budget cuts in many research libraries, even as further price increases are expected for scholarly journals.  This has forced many libraries to consider dropping even the most prestigious journals, when their prices have risen too high to afford.

Recently, for instance, the University of California, which has been subject to significant budget cuts and furloughs,  sent out a letter in protest of Nature Publishing Group’s proposal to raise their subscription fees by 400%.  The letter raised the possibility of cancelling all university subscriptions to NPG, and having scholars boycott the publisher.

Given that Nature is one of the most prestigious academic journals now publishing, one that has both groundbreaking current articles and a rich history of older articles, these are strong words.  But dropping subscriptions to journals like Nature might not be as as much of a hardship for readers as it once might have been.  Increasingly, it’s possible to liberate the research content of academic journals, both new and old, for the world.  And, as I’ll explain below, now may be an especially opportune time to do that.

Liberating new content

While some of the content of journals like Nature is produced by the journal’s editorial staff or other writers for hire, the research papers are typically written by outside researchers, employed by universities and other research institutions.  These researchers hold the original copyright to their articles, and even if they sign an agreement with a journal to hand over rights to them (as they commonly do), they retain whatever rights they don’t sign over.  For many journals, including the ones published by Nature Publishing Group, researchers retain the right to post the accepted version of their paper (known as a “preprint”) in local repositories.  (According to the Romeo database, they can also eventually post the “postprint”– the final draft resulting after peer review, but before actual publication in the journal– under certain conditions.)  These drafts aren’t necessarily identical to the version of record published in the journal itself, but they usually contain the same essential information.

So if you, as a reader, find a reference to a Nature paper that you can’t access, you can search to see if the authors have placed a free copy in an open access repository. If they haven’t, you can contact one of them to encourage them do do so.  To find out more about providing open access to research papers, see this guide.

If a journal’s normal policies don’t allow authors to share their work freely in an open access repository, authors  may still be able to retain their rights with a contract addendum or negotiation.  When that hasn’t worked, some academics have decided to publish in, or review for, other journals, as the California letter suggests.  (When pushed too far, some professors have even resigned en masse from editorial boards to start new journals that are friendlier to authors and readers.

If nothing else, scholarly and copyright conventions generally respect the right of authors to send individual copies of their papers to colleagues that request them.  Some repository software includes features that make such copies extremely easy to request and send out.  So even if you can’t find a free copy of a paper online already, you can often get one if you ask an author for it.

Liberating historic content

Many journals, including Nature, are important not only for their current papers, but for the historic record of past research contained in their back issues.  Those issues may be difficult to get a hold of, especially as many libraries drop print subscriptions, deaccession old journal volumes, or place them in remote storage.  And electronic access to old content, when it’s available at all, can be surprisingly expensive.  For instance, if I want to read this 3-paragraph letter to the editor from 1872 on Nature‘s web site, and I’m not signed in at a subscribing institution, the publisher asks me to pay them $32 to read it in full.

Fortunately, sufficiently old journals are in the public domain, and digitization projects are increasingly making them available for free.  At this point, nearly all volumes of Nature published before 1922 can now be read freely online, thanks to scans made available to the public by the University of Wisconsin, Google, and Hathi Trust.  I can therefore read the letters from that 1872 issue, on this page, without having to pay $32.

Mass digitization projects typically stop providing public access to content published after 1922, because copyright renewals after that year might still be in force.  However, most scholarly journals– including, as it turns out, Nature — did not file copyright renewals.  Because of this, Nature issues are actually in the public domain in the US all the way through 1963 (after which copyright renewal became automatic).  By researching copyrights for journals, we can potentially liberate lots of scholarly content that would otherwise be inaccessible to many. You can read more about journal non-renewal in this presentation, and research copyright renewals via this site.

Those knowledgeable about copyright renewal requirements may worry that the renewal requirement doesn’t apply to Nature, since it originates in the UK, and renewal requirements currently only apply to material that was published in the US before, or around the same time as, it was published abroad.  However, offering to distribute copies in the US counts as US publication for the purposes of copyright law.  Nature did just that when they offered foreign subscriptions to journal issues and sent them to the US; and as one can see from the stamp of receipt on this page, American universities were receiving copies within 30 days of the issue date, which is soon enough to retain the US renewal requirement.  Using similar evidence, one can establish US renewal requirements for many other journals originating in other countries.

Minding the gap

This still leaves a potential gap between the end of the public domain period and the present.  That gap is only going to grow wider over time, as copyright extensions continue to freeze the growth of the public domain in the US.

But the gap is not yet insurmountable, particularly for journals that are public domain into the 1960s.  If a paper published in 1964 included an author who was a graduate student or a young researcher, that author may well be still alive (and maybe even be still working) today, 46 years later.  It’s not too late to try to track authors down (or their immediate heirs), and encourage and help them to liberate their old work.

Moreover, even if those authors signed away all their rights to journal publishers long ago, or don’t remember if they still have any rights over their own work, they (or their heirs) may have an opportunity to reclaim their rights.  For some journal contributions between 1964 and 1977, copyright may have reverted to authors (or their heirs) at the time of copyright renewal, 28 years after initial publication.  In other cases, authors or heirs can reclaim rights assigned to others, using a termination of transfer.  Once authors regain their rights over their articles, they are free to do whatever they like with them, including making them freely available.

The rules for reversion of author’s rights are rather arcane, and I won’t attempt to explain them all here.  Terminations of transfer, though, involve various time windows when authors have the chance to give notice of termination, and reclaim their rights.  Some of the relevant windows are open right now.   In particular, if I’ve done the math correctly, 2010 marks the first year one can give notice to terminate the transfer of a paper copyrighted in 1964, the earliest year in which most journal papers are still under US copyright.  (The actual termination of a 1964 copyright’s transfer won’t take effect for another 10 years, though.)  There’s another window open now for copyright transfers from 1978 to 1985; some of those terminations can take effect as early as 2013.  In the future, additional years will become available for author recovery of copyrights assigned to someone else.  To find out more about taking back rights you, or researchers you know, may have signed away decades ago, see this tool from Creative Commons.

Recognizing opportunity

To sum up, we have opportunities now to liberate scholarly research over the full course of scholarly history, if we act quickly and decisively.  New research can be made freely available through open access repositories and journals.  Older research can be made freely available by establishing its public domain status, and making digitizations freely available.  And much of the research in the not-so-distant past, still subject to copyright, can be made freely available by looking back through publication lists, tracking down researchers and rights information, and where appropriate reclaiming rights previously assigned to journals.

Journal publishing plays an important role in the certification, dissemination, and preservation of scholarly information.  The research content of journals, however, is ultimately the product of scholars themselves, for the benefit of scholars and other knowledge seekers everywhere.   However the current dispute is ultimately resolved between Nature Publishing Group and the University of California, we would do well to remember the opportunities we have to liberate journal content for all.

March 23, 2010

Lots of conversation keeps stuff sustainable

Filed under: libraries,people,preservation,sharing — John Mark Ockerbloom @ 10:12 pm

Among the hats I wear at my place of work is that of LOCKSS cache administrator. LOCKSS is a useful distributed preservation system built around the principle “Lots of copies keep stuff safe” (whose initials give the system its name).  The idea is that, with the cooperation of publishers, a bunch of libraries each harvest copies of selected online content, and keep backups on our own LOCKSS caches, which are hooked up to local library proxy services.  Then, if the material ever becomes inaccessible from the publisher, our users will automatically be routed to our local copies.  Each LOCKSS cache also periodically checks with other LOCKSS caches to ensure that our copies are still in good shape, and to repair or replace copies that have been lost or damaged.  (Various security features protect against leaks of restricted content, or unauthorized revisions of content.)

LOCKSS is open source software that runs on commodity hardware.  It was originally envisioned to run virtually automatically.  As Chris Dobson described the ideal in a 2003 Searcher article, “Take a computer a generation past its prime…. Hook it up to the Internet and put it in a closet. Stick in the LOCKSS CD-ROM and boot it up. Close the closet door.”  And then presumably walk away and forget about it.

Of course, it’s not that simple in practice, particularly if your library is proactive about its preservation strategy.  The thing about preservation at scale is there’s always something that needs attention.  It might be something technical, or content-related, or planning-related, but preserving a growing collection requires ongoing thought.  And if you want to think as clearly and sensibly as you can, you’ll want to collaborate.

Right now, for instance, I’m trying to get my cache to harvest the full run of a journal that’s just been made available for LOCKSS harvesting, where we hope to provide post-cancellation access through LOCKSS.  Someone at Stanford just gave me a useful tip on how to give this journal priority over the other volumes I’ve got queued up for harvest.  Unfortunately, I can’t try it out until I get my cache back up after it failed to reboot cleanly after a power failure. While I wait to hear back instructions about how best to remedy this, I wonder whether switching to a new Linux-based version of LOCKSS might make such operating system-level problems easier to deal with.  But it would be useful to hear from folks who are running that version to see what their experience has been.

Meanwhile, we’re wondering how best to approach new publishers who have content that our bibliographers would like to preserve via LOCKSS. Our special collections folks wonder whether we should preserve some of our own home-grown content via a private LOCKSS network.  I’m also doing some ongoing monitoring and testing of our LOCKSS cache’s behavior (some of which I’ve reported on earlier), and would be interested in knowing if others are seeing some of the same kinds of things that I see on the cache I administer.

In short, there are a lot of things to think about, when LOCKSS plays a significant role in a preservation plan.  And a lot of the issues I’ve mentioned above are ones that others may be thinking about as well.  So let’s talk about them.  As the LOCKSS group has said, “”A vibrant, active, and engaged user community is key to the success of Open-Source efforts like LOCKSS.”

One thing you need for such an engaged community is a forum for them to talk to each other.  As it turns out, the LOCKSS group at Stanford tell me they created a LOCKSS Forum mailing list a while back, but I haven’t yet seen it publicized.   Its information page is at https://mailman.stanford.edu/mailman/listinfo/lockss-forum .  (Currently, archived email messages are not visible on the open web, though this may change in the future.)  If you’re interested in talking with others about how you use or might use LOCKSS to preserve access to digital content, I invite you to sign up and help get the conversation going.

December 10, 2009

Some concepts and their catalogs

Filed under: architecture,discovery,libraries,metadata — John Mark Ockerbloom @ 6:28 pm

This is the second of a series of think-aloud posts on what I’m calling “concept-oriented catalogs”; catalogs that, as Lorcan Dempsey aptly describes them, go “beyond the bibliographic record”.  This post will present examples of concept-oriented catalogs, describe the concepts they use, and describe some of the features that make them work.

Reflections on the story so far

As I stated in my first post, a catalog helps a user get from some set of concepts (ideas, citations, words, people, places, etc.) of the information they’re seeking, to some useful knowledge resources (books, articles, web sites, songs, etc.)  A concept-oriented catalog uses a variety of concepts, not just those of knowledge resources themselves, as first-class entities to help locate useful resources.

Note that this is essentially a functional view of a catalog, from a user’s perspective, as a means of knowledge discovery.   Library catalogs also serve a number of other important functions.  In particular, they also manage the inventory of resources the library has acquired and makes available. That function is necessarily resource-oriented.  Some concept-oriented catalogs don’t need to be resource-oriented; Wikipedia, for example, maintains no inventory of the sites it links to from its concept-based articles.  A concept-oriented library catalog, however, will probably also need to be resource-oriented to fulfill all its functions.  That’s fine– the two qualities are not mutually exclusive.

Concepts can come from a variety of places; from libraries, from experts, and from ordinary readers.   I’ll show examples of all three.

Fun with FRBR

I noted in the last post that current library catalogs are largely based around the entities that FRBR describes as manifestations and items.  FRBR defines a number of other entities as well. These entities can also provide useful focuses for concept-oriented catalogs.

Fiction Finder, mentioned in my last post, is not the only example of a work-oriented catalog.  A recent status report from the OpenLibrary Project indicates that they are moving to make their catalog work-oriented as well.  The Amazon book catalog, and its aggregation of various editions of a book (and their reviews) on one catalog page, is oriented around expressions in the FRBR sense.

In practice, the line between works and expressions tends to be blurry in catalogs.  But we’re definitely seeing more catalogs present search results at these higher bibliographic levels. (See, for instance, the Worldcat.org search results for War and Peace, which include various “view all editions and formats” links.) Catalogs that can’t easily and sensibly aggregate their manifestations of works or expressions will be at a competitive disadvantage over catalogs that can.

Numerous catalogs also provide special information for authors, whether persons or corporate bodies.  I mentioned WorldCat Identities in my previous post; a commenter to that post noted the New Zealand Electronic Text Centre’s catalog, which also aggregates useful information on its authors, as well as other concepts.

Google Maps and other geospatial platforms provide the basis for many catalogs oriented around places, mediating access to information resources of all types, from local histories to violent incident reports, to hospital ratings. Catalog interfaces can also be built around events, as demonstrated in MIT’s Simile Timelines widget and in Google’s Living Stories news tracker.  Objects, concepts (which in FRBR-ese denote abstract subjects, rather than the broader use I’m making of “concept” in this post), and other kinds of subjects are the focus of the subject map-based catalogs I mentioned in my last post.

Catalogs need not limit their focus to one kind of concept. They often get more interesting when users can move between different kinds of concepts.  For instance, if you look up the author Fanny Jackson Coppin on The Online Books Page, you’ll see that there are not only resources by her, but resources about her.  If you look for the latter, you’ll also discover that Coppin was an African-American teacher in Philadelphia, and then be able to follow further links to more books about African American teachers, or Philadelphia, or other related subjects as you see fit.  Or, if you started a search with African American teachers in mind, you’ll find that concept linked to Coppin.  So users can move back and forth between particular subjects, and people that are relevant to them, as they browse the catalog.

What makes this sort of navigation work?  Part of it involves analysis of existing bibliographic records: in this case, catalogs like The Online Books Page (as well as others like WorldCat Identities) analyze patterns of subject headings to relate particular people to particular subjects.  Having common identifiers helps as well: the authority-controlled string “Coppin, Fanny Jackson” is used both for author and subject metadata, making it possible to link author- and subject-based concepts.  Subject maps also require additional data beyond just bibliographic records. Mine use a variety of data sources, including authority records and a small but essential set of records I created for certain geographical entities.

Consult the experts

Librarians have lots of understanding and experience working with FRBR-like concepts. But users also find value in many other concepts.  A recent research report from Project Information Literacy on information seeking patterns of college students included some interesting findings on favored starting points.  Wikipedia, as I expected, was a very popular starting point for everyday research, but for course-related research, the most popular starting point, edging out not only Wikipedia but even Google itself, was course readings.

Course readings are highly focused guideposts for academic research.  The list of readings is a set of knowledge resources chosen carefully by the instructor to educate students about the subject of the course.  The readings themselves typically have bibliographies, or at least lists of references, that support their own assertions and suggest further avenues of research.  Essentially, the syllabus and the readings represent a careful curation by subject experts of important knowledge resources for a particular topic.  The value of that curation often extends well beyond a particular class.  At Penn, many of the reading lists originally developed for a class get adapted into research guides on our library’s Web site, helping others researching in similar areas.  And interesting things also start to happen as you aggregate multiple scholars’ reading and citation lists, as we’ve seen with services like PennTags (originally designed, and still frequently used, for class projects), CiteULike, and Google Scholar.

There are also many collections assembled and curated by non-faculty experts.  The curation of many of these collections often involves a rich set of concepts appropriate to the focus of each collection.  Consider, for instance, the Freedman Jewish Sound Archive, curated by a lawyer and his wife and now housed at Penn.   The catalog for the archive is oriented around several important concepts: among them songs that are musical compositions, tracks that record performances of the songs,  albums that contain the tracks, sheet music in which songs are published, and artists that have various kinds of relationships with each of those other concepts.  Each of these concepts has an expressive metadata schema that includes relations to the other concepts, and to resources that can be read or listened to onsite or online.  The catalog is not only a guide to knowledge resources, but a valuable knowledge resource in its own right.

Because of the complexity and interrelationship between concepts, we had to write a specialized interface to bring out the full expressiveness of the catalog.  Simply providing a flat search index of tracks was not enough.   But many of the special concepts used to provide depth for this catalog can also be mapped to some extent into more common bibliographic data structures and interfaces.  And new technologies I’ll discuss later on can make it easier to build, and share data from, these sorts of specialized catalogs.

There are a lot more scholars, and expert curators, of all types, than there are professional librarians.  And they often know more about knowledge resources in their areas of interest than we do.  We can potentially build much more broad, conceptually rich, and carefully curated catalogs, if we develop effective ways to work with them.

Power to the people

Ordinary readers or scholars might not have the information science training or metadata expertise that professional librarians have.  But they can still help us greatly in finding useful knowledge resources, just through their ongoing reading and commentary, if we have some way of tracking and aggregating what they do.

Social software gives us a way to do that, and its benefits have been widely discussed in recent years.  Social software introduces the user as a first-class concept that can be associated with particular descriptions and knowledge resources.  Implicitly or explicitly, users build up collections of resources that they have noted, possibly using tags to describe them, or posts to comment on them.  The tags and posts are typically informal expressions, rather than controlled terms or highly structured records.  Tags can often describe resources in more accessible language, or to greater specificity, than established controlled descriptive terms.  And users often find useful resource recommendations from other users who share their interests.

Many social communities have built up around different user groups, types of resource, or styles of communication. PennTags (oriented around Penn scholars), Flickr (oriented around photos and other images), and Twitter (oriented around short real-time posts) are just a few examples.  I’m far from the only person who relies heavily on the users I follow on Twitter for reading tips and current awareness.   Longer-form posts that review and invite comments on books and other works can also be very helpful in finding useful knowledge resources.

Social software has been making its way into libraries for a few years now.   Many library catalogs, ours included, now let users tag and annotate bibliographic records.  The catalogs provide a smattering of new functions based on this tagging, but tagging is essentially an add-on, not fully integrated into the catalog.

Catalogs designed from the ground up to be socially powered can look quite different.  We’re starting to see some of them now.  One notable “social catalog” is LibraryThing, which has been developed by Tim Spalding, a staff of about a dozen professionals, and a membership of nearly a million readers.  Tim gave a talk to librarians in October called “What is Social Cataloging?“.   In the talk, he describes and demonstrates the features and the concepts of his catalog (including many of the concepts I’ve mentioned in this post), and the many benefits that emerge when hundreds of thousands of readers collaboratively catalog their personal libraries.  The video of the full talk runs just under an hour, and is well worth watching.

LibraryThing does not provide all of the functionality you’d find in a good research library catalog.  But it provides many useful forms of guidance that most traditional catalogs lack, and it also suggests some ways in which libraries and their users can join forces in building comprehensive, user-focused, concept-oriented catalogs.   (Much of the bibliographic metadata used in LibraryThing comes from traditional library catalog records; and LibraryThing in turn now sells a service that embeds some of the other information it aggregates back into traditional library catalog displays.)

Recap and coming attractions

In this post, I’ve shown how catalogs can use a wide variety of concepts to help users find resources.  The concepts come from, and are maintained by, various groups of people, including librarians, scholars, collectors and other domain experts, and lots and lots of everyday readers.  The catalog concepts may be derived in part from existing MARC bibliographic metadata (sometimes through automated analysis), but often draw from additional data sources.  The catalogs may require new user interface designs, going beyond the standard search-box and list-of-hits paradigm, for users to take full advantage of their concepts.

The examples I’ve shown should demonstrate the versatility of concept-oriented catalogs, and also suggest some of the challenges of implementing them.  In future posts, I hope to discuss the technologies, data models, system architectures, and social structures that can help make useful concept-oriented catalogs practical for libraries to build and maintain.

December 4, 2009

Understanding concept-oriented catalogs

Filed under: architecture,discovery,libraries,metadata — John Mark Ockerbloom @ 6:03 pm

I’ve recently been thinking a fair bit about the future of library catalogs, particularly after reading Diane Hillmann’s review of the recent Library of Congress-commissioned Study of the North American MARC Records Marketplace.  Hillmann laments that the study tries to propose economic tweaks to the current system of MARC bibliographic record distribution, but doesn’t consider the more basic questions of whether this sort of distribution is what cataloging should be focusing on in the first place, or how the nature of cataloging (broadly conceived) in the Internet age is already  changing.  She concludes “The change we need is not really about records, or catalogers; it’s a new way to think about information and added value.”

I agree with a lot of what she has to say in the review, but I also worry that, by itself, what she recommends might be too easily dismissed by library planners.   They might imagine this sort of thinking as utopian, airy speculation with little relevance to the work they have to do now, getting books and other resources into their catalogs more efficiently. But in fact, new ways of cataloging and adding value to knowledge are already being practiced in many places online, and can be practically built on much of what libraries are already doing.  So I’d like to spend a few blog posts thinking out loud about the potential I see developing for catalogs, and practical ways for realizing that potential.

What does a catalog do?

Let’s start with the basics: What are people doing when they’re using a library catalog, or engaging in other kinds of search?  Typically, they have in mind some concept [1], or combination of concepts, that they want to find information about.  Their concepts might not be fully formed or understood at the start of a search, and people may not be able to ideally express the concepts in ways their interface expects, but the concepts are there.

As they search, users often will refine their concepts, change the way  they express them, explore related concepts, or altogether shift the concepts they’re interested in.  They do this as they see what happens when they search, what resources come up, and what other relevant concepts (such as particular people, works, or focuses of study) become apparent.  But the user’s ultimate goal is to obtain useful knowledge.  A catalog, then, is a way of helping people get from concepts to useful knowledge resources.

There are many kinds of concepts people that searchers might be thinking of.  Familiar kinds of library concepts include books, people, places, and topics of study, but there are many additional important kinds that we’ll discuss later on.  Sometimes, users already have a concept of a specific knowledge resource.  They might have a citation they’re trying to follow, or a particular book they want to read.  Library catalogs tend to be particularly good at this kind of scenario, known in the trade as a “known item search“. [2]

This is not surprising, because today’s library catalogs tend to be strongly oriented around resources, and much less around other concepts.  They primarily use MARC bibliographic records (which describe particular manifestations, to use FRBR terminology), and holdings records (which describe particular items that can be accessed or borrowed).   Library science has meticulously defined and curated a number of other concepts: subjects with complex taxonomies; authors with wide-ranging authority control; uniform titles and series that link many bibliographic items together, and other bibliographic features that have carefully defined controlled vocabularies.

These concepts are represented in our MARC records, but as distinctly second-class entities.   They’re typically attributes of the records that are the focus of the catalog, rather than focused records in their own right.  The closest things to first-class non-resource concepts in most catalogs are authority records.  But those records often contain minimal information about the concepts they describe, they’re typically invisible to most catalog users, and most online catalog interfaces don’t effectively exploit the information those records do contain. [3]

Concepts as first-class entities in catalogs

One of the long-recognized strengths of research libraries is the knowledge and organization they have of their resources, and of the concepts that these resources represent.  Since a catalog maps concepts to resources, it should be possible for catalogs to represent important concepts more explicitly and expressively in the catalog, and use them as guideposts to help people find useful resources.  We’re starting to see some prototype catalogs of this sort develop in the library world: consider, for instance, OCLC’s Worldcat Identities, oriented around authors; or their Fiction Finder, oriented around works with multiple manifestations; or the Online Books Page’s subject map views (such as this one on alphabets).  In these catalogs, the concepts aren’t simply metadata attributes, or headings in a list of choices, but information-rich reference points for finding knowledge resources.  These systems are all what I would call concept-oriented catalogs: catalogs that use various concepts (and not just concepts of the resources themselves) as first-class locuses of information to help readers find useful knowledge resources.

The promise of concept-oriented catalogs is still largely unrealized in the library world.  Not only do current library catalogs often do little with conceptual knowledge, but proposals for future catalog architectures also often stick to keeping a tight focus on resources and make other concepts secondary.   For instance, Coyle and Hillmann note in their 2007 review of the proposed RDA cataloging standard that “the focus of RDA is called ‘the resource’ and the resource is a FRBR manifestation/item described using the same concept of a pre-coordinated ‘record’ as we find in AACR2″ (the older cataloging standard first published in 1978).  Similarly, the final project report for OLE, which aims to build a next-generation library architecture, treats resources as first-class entities, but not the descriptions of those resources.  The metadata describing knowledge resources are simply attributes of the resources, and not entities that can managed and shared in their own right. [4]

Concept-oriented catalogs at Internet scale

As more and more knowledge resources become available to users, via the expansion of the Internet, the streamlining of interlibrary loan services, and the mass digitization of print library materials, well-defined, well-documented, and well-connected concepts will become increasingly important for readers that want to find what is most useful to them in a sea of information.  While we will never have well-defined concepts for everything readers might be interested in, the concepts that have been defined by someone, somewhere, can serve as valuable guideposts for subsequent information seekers, if we’re smart about managing and using them.

This view might strike some longtime users of the Internet as hopelessly naive and ignorant of history.  After all, the Web started out in the 1990s with various conceptually organized catalogues like the WWW Virtual Library and the Yahoo Directory.  But most people soon forsook these in favor of search engines like Google that are much more comprehensive, and that work with arbitrary keywords instead of predefined concepts.   Why should we think that concept-oriented catalogs will work at the Internet scale if they’ve already been tried and rejected on the Web?

In fact, though, concept-oriented cataloging is still very widely used in online information seeking today.  It just isn’t the same kind of concept-oriented cataloging.  The most popular concept-oriented catalog online today doesn’t force readers to go through a particular concept hierarchy before they can get at the resources they want.  Instead, it typically shows up prominently whenever you do a Google search for one of the concepts it includes.  Each of its millions of concepts gets reviewed for naming, redundancy, and relevance, and can be easily linked to from elsewhere on the Web. Most of its concepts have links to various external online knowledge resources, as well as to related concepts.  The catalog also has ever-increasing amounts of harvestable, structured, semantic metadata about its concepts.  And it’s very often characterized, even by users who acknowledge its many flaws, as a useful starting point for finding information resources online.  Oh, and anyone can edit it.  You’ve probably figured out by now that I’m talking about Wikipedia.  While Wikipedia is not usually billed as a catalog, my description should make it clear that it does in fact serve that function, and in a concept-oriented way.

Coming attractions

I hope what I’ve written so far gives you a good idea of what concept-oriented catalogs are, and why they’re worth thinking about as we plan for the future of libraries.  In posts to come, I hope to discuss various examples of concept-oriented catalogs inside and outside the library world, and talk about how they work, what concepts (and related information) they focus on, and how they are built up and maintained. And I hope to show how we  construct useful, practical concept-oriented catalogs for the future, building both on the knowledge and expertise we have in libraries, and on the contributions of others.

(Update: The second post in the series is now available.)

Notes

[1] I’m using “concept” here in the broad sense of any kind of thing that might be the object of someone’s search, rather than the narrower sense used in FRBR and elsewhere of a particular kind of abstract subject.  It’s surprisingly difficult to find a straightforward word for this general idea that hasn’t already been claimed for some other purpose.

[2] Though even here, with the explosion of resources available online, we’ve found it useful to augment the traditional catalog with link resolvers to help people find resources like journal articles that we don’t specifically include in our own catalogs.

[3] I’ve discussed this at length in the past with respect to LCSH subjects; see my work on subject maps for some attempts to address this problem.  I’ll talk more about subject maps as one type of concept-oriented catalog in later posts.

[4] This may well change as OLE develops, and the data architecture is fleshed out.  I’m not directly involved in OLE at present, but I do work for one of the lead development partners.

Next Page »

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.