Everybody's Libraries

December 4, 2009

Understanding concept-oriented catalogs

Filed under: architecture,discovery,libraries,metadata — John Mark Ockerbloom @ 6:03 pm

I’ve recently been thinking a fair bit about the future of library catalogs, particularly after reading Diane Hillmann’s review of the recent Library of Congress-commissioned Study of the North American MARC Records Marketplace.  Hillmann laments that the study tries to propose economic tweaks to the current system of MARC bibliographic record distribution, but doesn’t consider the more basic questions of whether this sort of distribution is what cataloging should be focusing on in the first place, or how the nature of cataloging (broadly conceived) in the Internet age is already  changing.  She concludes “The change we need is not really about records, or catalogers; it’s a new way to think about information and added value.”

I agree with a lot of what she has to say in the review, but I also worry that, by itself, what she recommends might be too easily dismissed by library planners.   They might imagine this sort of thinking as utopian, airy speculation with little relevance to the work they have to do now, getting books and other resources into their catalogs more efficiently. But in fact, new ways of cataloging and adding value to knowledge are already being practiced in many places online, and can be practically built on much of what libraries are already doing.  So I’d like to spend a few blog posts thinking out loud about the potential I see developing for catalogs, and practical ways for realizing that potential.

What does a catalog do?

Let’s start with the basics: What are people doing when they’re using a library catalog, or engaging in other kinds of search?  Typically, they have in mind some concept [1], or combination of concepts, that they want to find information about.  Their concepts might not be fully formed or understood at the start of a search, and people may not be able to ideally express the concepts in ways their interface expects, but the concepts are there.

As they search, users often will refine their concepts, change the way  they express them, explore related concepts, or altogether shift the concepts they’re interested in.  They do this as they see what happens when they search, what resources come up, and what other relevant concepts (such as particular people, works, or focuses of study) become apparent.  But the user’s ultimate goal is to obtain useful knowledge.  A catalog, then, is a way of helping people get from concepts to useful knowledge resources.

There are many kinds of concepts people that searchers might be thinking of.  Familiar kinds of library concepts include books, people, places, and topics of study, but there are many additional important kinds that we’ll discuss later on.  Sometimes, users already have a concept of a specific knowledge resource.  They might have a citation they’re trying to follow, or a particular book they want to read.  Library catalogs tend to be particularly good at this kind of scenario, known in the trade as a “known item search“. [2]

This is not surprising, because today’s library catalogs tend to be strongly oriented around resources, and much less around other concepts.  They primarily use MARC bibliographic records (which describe particular manifestations, to use FRBR terminology), and holdings records (which describe particular items that can be accessed or borrowed).   Library science has meticulously defined and curated a number of other concepts: subjects with complex taxonomies; authors with wide-ranging authority control; uniform titles and series that link many bibliographic items together, and other bibliographic features that have carefully defined controlled vocabularies.

These concepts are represented in our MARC records, but as distinctly second-class entities.   They’re typically attributes of the records that are the focus of the catalog, rather than focused records in their own right.  The closest things to first-class non-resource concepts in most catalogs are authority records.  But those records often contain minimal information about the concepts they describe, they’re typically invisible to most catalog users, and most online catalog interfaces don’t effectively exploit the information those records do contain. [3]

Concepts as first-class entities in catalogs

One of the long-recognized strengths of research libraries is the knowledge and organization they have of their resources, and of the concepts that these resources represent.  Since a catalog maps concepts to resources, it should be possible for catalogs to represent important concepts more explicitly and expressively in the catalog, and use them as guideposts to help people find useful resources.  We’re starting to see some prototype catalogs of this sort develop in the library world: consider, for instance, OCLC’s Worldcat Identities, oriented around authors; or their Fiction Finder, oriented around works with multiple manifestations; or the Online Books Page’s subject map views (such as this one on alphabets).  In these catalogs, the concepts aren’t simply metadata attributes, or headings in a list of choices, but information-rich reference points for finding knowledge resources.  These systems are all what I would call concept-oriented catalogs: catalogs that use various concepts (and not just concepts of the resources themselves) as first-class locuses of information to help readers find useful knowledge resources.

The promise of concept-oriented catalogs is still largely unrealized in the library world.  Not only do current library catalogs often do little with conceptual knowledge, but proposals for future catalog architectures also often stick to keeping a tight focus on resources and make other concepts secondary.   For instance, Coyle and Hillmann note in their 2007 review of the proposed RDA cataloging standard that “the focus of RDA is called ‘the resource’ and the resource is a FRBR manifestation/item described using the same concept of a pre-coordinated ‘record’ as we find in AACR2″ (the older cataloging standard first published in 1978).  Similarly, the final project report for OLE, which aims to build a next-generation library architecture, treats resources as first-class entities, but not the descriptions of those resources.  The metadata describing knowledge resources are simply attributes of the resources, and not entities that can managed and shared in their own right. [4]

Concept-oriented catalogs at Internet scale

As more and more knowledge resources become available to users, via the expansion of the Internet, the streamlining of interlibrary loan services, and the mass digitization of print library materials, well-defined, well-documented, and well-connected concepts will become increasingly important for readers that want to find what is most useful to them in a sea of information.  While we will never have well-defined concepts for everything readers might be interested in, the concepts that have been defined by someone, somewhere, can serve as valuable guideposts for subsequent information seekers, if we’re smart about managing and using them.

This view might strike some longtime users of the Internet as hopelessly naive and ignorant of history.  After all, the Web started out in the 1990s with various conceptually organized catalogues like the WWW Virtual Library and the Yahoo Directory.  But most people soon forsook these in favor of search engines like Google that are much more comprehensive, and that work with arbitrary keywords instead of predefined concepts.   Why should we think that concept-oriented catalogs will work at the Internet scale if they’ve already been tried and rejected on the Web?

In fact, though, concept-oriented cataloging is still very widely used in online information seeking today.  It just isn’t the same kind of concept-oriented cataloging.  The most popular concept-oriented catalog online today doesn’t force readers to go through a particular concept hierarchy before they can get at the resources they want.  Instead, it typically shows up prominently whenever you do a Google search for one of the concepts it includes.  Each of its millions of concepts gets reviewed for naming, redundancy, and relevance, and can be easily linked to from elsewhere on the Web. Most of its concepts have links to various external online knowledge resources, as well as to related concepts.  The catalog also has ever-increasing amounts of harvestable, structured, semantic metadata about its concepts.  And it’s very often characterized, even by users who acknowledge its many flaws, as a useful starting point for finding information resources online.  Oh, and anyone can edit it.  You’ve probably figured out by now that I’m talking about Wikipedia.  While Wikipedia is not usually billed as a catalog, my description should make it clear that it does in fact serve that function, and in a concept-oriented way.

Coming attractions

I hope what I’ve written so far gives you a good idea of what concept-oriented catalogs are, and why they’re worth thinking about as we plan for the future of libraries.  In posts to come, I hope to discuss various examples of concept-oriented catalogs inside and outside the library world, and talk about how they work, what concepts (and related information) they focus on, and how they are built up and maintained. And I hope to show how we  construct useful, practical concept-oriented catalogs for the future, building both on the knowledge and expertise we have in libraries, and on the contributions of others.

(Update: The second post in the series is now available.)

Notes

[1] I’m using “concept” here in the broad sense of any kind of thing that might be the object of someone’s search, rather than the narrower sense used in FRBR and elsewhere of a particular kind of abstract subject.  It’s surprisingly difficult to find a straightforward word for this general idea that hasn’t already been claimed for some other purpose.

[2] Though even here, with the explosion of resources available online, we’ve found it useful to augment the traditional catalog with link resolvers to help people find resources like journal articles that we don’t specifically include in our own catalogs.

[3] I’ve discussed this at length in the past with respect to LCSH subjects; see my work on subject maps for some attempts to address this problem.  I’ll talk more about subject maps as one type of concept-oriented catalog in later posts.

[4] This may well change as OLE develops, and the data architecture is fleshed out.  I’m not directly involved in OLE at present, but I do work for one of the lead development partners.

May 16, 2009

Free Decimal Correspondence: Now with more detail and CC0 goodness

Filed under: libraries,metadata,open access,repositories,sharing — John Mark Ockerbloom @ 7:38 pm

I’ve just released a new version of the Free Decimal Correspondence (FDC).  FDC is a set of associations between numbers and subject terms that can be used as the basis for linear shelving or subject browsing of a simple library collection or repository. Like the earlier releases, it’s intended to be reasonably compatible with existing library standards; in particular, with OCLC’s Dewey Decimal Classification (DDC) and the Library of Congress Subject Headings (LCSH).  (Keep in mind, though, that both of those standards go into considerably more detail than FDC does; and neither OCLC nor the Library of Congress sponsors or endorses the FDC in any way.)   This version of the FDC, like the previous releases, is dedicated to the public domain.  More information about the FDC can be found here.

What’s new?

There are two noteworthy features of this release.  First of all, I’ve now defined corresponding subjects for every whole number (from 000 to 999) that’s also defined (and non-optional) in the DDC.  In addition, I’ve gone past the decimal point in some cases to include additional subjects that may be useful in some collections and repositories.  This means that the FDC should now cover at least as many subjects as OCLC’s Dewey Decimal Classification summaries, the most detailed official free-to-read summary of the present-day DDC I know about.

In addition, I’ve now using Creative Commons‘ new CC0 dedication to dedicate the FDC to the public domain.  I’m hoping this will provide more authoritative legalese for clarifying the public domain status of FDC, and also raise awareness of CC0.

OCLC claims copyright and trademark on the DDC, and while the summary of DDC can be freely read on the Web, its use appears to be governed by this license, which states, among other things, that users may “not copy, reproduce, alter, modify, create derivative works, or publicly display any content (except for your own non-commercial use)”.  It further clarifies that “‘non-commercial use’ … is not to be construed as permitting use of the DDC that (although internal) supports or otherwise facilitates your commercial activities.”  (Note also that the linked license applies only to the DDC summary, not to the complete DDC.  If you want a license for the full DDC, you’ll need to contact OCLC.)

With FDC, as with all CC0 and other public domain works, none of the restrictions above apply.  Do with it what you like.

What’s next?

This new release is version 0.05.  As the number suggests, this is still essentially a working draft.  My first draft was released a few months ago on Public Domain Day.  I intend my work on FDC to be a lightweight, spare time project, and therefore hope to advance to a “1.0″ release soon, and leave further development, if desired, to others.

Between now and then, I’m not planning to make FDC much bigger than it is now.  But there are still various other subjects not defined at the unit level that are common in repositories and small library collections, and that might be useful to add.  (I originally started this after someone said they wanted to have a Dewey-like classification system for their repository, without worrying about licensing issues.)  I’m happy to take suggestions of additional subjects to include in the next release.  (But use your own subjects of interest, and not terms taken from an OCLC publication.)  I’d also be very interested in hearing about (and fixing) mistakes, inconsistencies, redundancies, and correspondences that could be made more compatible with current library practice than what I have now.

You can send me email about any of the above. I’ll also be at Open Repositories 2009 next week, and would be happy to talk about FDC there with anyone interested.

What else?

There are numerous classification schemes you can use for collections.  My page about FDC includes some discussion of the better-known alternatives you might want to consider.  And if you’d like to work on a new scheme without worrying about compatibility with other systems, you might be interested in the Open Shelves Classification, an ongoing project sponsored by Tim Spalding of LibraryThing.

If FDC sounds like something you’d like to use, adapt, or experiment with, go right ahead.  There’s no need to wait for any further releases, or to ask me about what you want to do, unless you’d like to.  Have at it, and have fun!

May 15, 2009

Will OCLC move to a service-oriented business model for bibliographic data?

Filed under: architecture,libraries,metadata,open access,sharing — John Mark Ockerbloom @ 2:21 pm

As you may recall, there’s been a fair bit of criticism (including some of my own) against OCLC’s proposed records use policy, which asserts OCLC’s rights over libraries’ cataloging in ways that would make it very difficult for libraries and other organizations to make innovative uses of shared bibliographic data, if they use OCLC for their shared cataloging.

In response to criticism, OCLC convened a review board to recommend principles for a revised policy.  Now Peter Murray tells us that the board will present their recommendations at Monday’s OCLC Member’s Council meeting.  I’m told presentation materials should eventually be posted publicly.

I’m looking forward to seeing what they recommend.   By now, large library organizations like the Association of Research Libraries and the International Coalition of Library Consortia have publicly criticized the policy as proposed, and the process through which it was developed.  I was gratified to see ICOLC agree that “the policy hinders rather than encourages innovation”.  And I hope that OCLC will adopt a policy that encourages sharing and innovative applications of bibliographic data, while maintaining a healthy, sustainable shared cataloging base.

As I mentioned at my talk at ALA in January, one of the ways they might do this is to move to a business model that’s based on charging for services  rather than for bibliographic data.  OCLC already offers a number of paid services, such as WorldCat Local.  Recently, they’ve announced an ambitious suite of “Web-scale management services” , effectively offering a shared ILS service, using the shared bibliographic records that have been contributed to WorldCat.  The announced services could be attractive to many libraries.   They could introduce economies of scale, by having a single ILS installation serving many institutions.  And the shared records in the service could promote new efficiencies in areas like acquisitions, collection development, and inter-library loan, as well as in information discovery.  OCLC’s new paid services, then, could advance the interest both of libraries and OCLC’s shared catalog.  They could raise the bar for competition in ILS-related services.

Or, at least, they could if everyone has the same opportunity to compete.  For lots of reasons, different libraries may choose to use different ILS vendors and products, or even develop something on their own.  (For instance, the library where I work uses Ex Libris’ Voyager ILS, and is also involved in the open source OLE Project.)  Many of these libraries have also been adding and enhancing bibliographic records in OCLC’s WorldCat.  If OCLC keeps these records locked up, so that its ILS can use them but competitors can’t, these contributing libraries will be giving a special advantage to one particular ILS by sharing their records through WorldCat.  And that skews the competition in ways that, like monopolies in general, can be bad for libraries and for innovation.

So I think OCLC needs to make a choice here:   Will they try to retain a proprietary interest in libraries’ shared cataloging, and stay neutral towards other entities (such as ILS vendors) that provide services on this cataloging?  Or will they move into competing with others to provide services on our collective data, while relinquishing their proprietary claims on it?  Though I’ve argued that we’re best off if they move to the latter, I’ve heard some thoughtful, conservative arguments for sticking with the former while they try to figure out how to sustain their shared catalog.

But they shouldn’t try to have it both ways. As we saw recently with a company that does both academic publishing and commercial PR, combining two valid but disparate interests can easily create worrisome conflicts.  As I’ve heard it, OCLC justifies whatever proprietary rights they claim on shared cataloging data on behalf of the cataloguing “cooperative”: the libraries that contribute the data.  The work of this cooperative should not be held captive in the interests of any single commercial venture  (whether OCLC’s or anyone else’s) that wants to exploit it.

With open bibliographic data, OCLC can still enjoy important competitive advantages in their ILS and other services, simply by being the entity that coordinates WorldCat, and that has the deepest knowledge about how to work with the WorldCat knowledge base. There are a number of possible ways for OCLC to support itself, and encourage data contributions, in its cataloging coordination and catalog-based services.   I hope that, in the upcoming Member’s Council meeting and in ensuing discussions, we’ll have some thoughtful conversations and proposals on how to do this with open data.

May 5, 2009

Do we need a “reader’s rights registry”?

Filed under: censorship,copyright,metadata,online books,open access,reading,sharing — John Mark Ockerbloom @ 4:18 pm

This spring’s Digital Library Federation forum, the last under DLF as an independent organization,  included a session for “lightning talks”, where speakers take 5 minutes or less to present an idea or make a demonstration.  Here are the remarks I prepared for the one I gave, building on some recent discussion of the Google Books Settlement.  I’ve added some links to provide a bit more context to these remarks.

Like many other folks, I’m excited about the Google Books settlement, and the possibility that millions of out of print books may become available for our patrons to search and read online.  I think the services that Google proposes could greatly expand the scope and the usability of our book collections.

But I also appreciate the concern expressed over some of the details of the settlement:  in particular, the concerns over the monopolies that it creates.  Monopolies have a tendency to impose increasingly onerous terms on consumers.   They may raise prices to levels that make it difficult for libraries to fulfill their independent missions.  They may also pay insufficient attention to concerns like privacy or intellectual freedom that are important for many of our readers.

A lot of people have commented on Google’s monopoly here, but the settlement actually creates another monopoly, one that may be more problematic, and that’s the Book Rights Registry.  The name suggests a passive store of information, but in fact it’s an active agent that has the power to negotiate and set terms of access on behalf of publishers and authors for millions of out of print books.  This is the body—and the only body—that will set the default wholesale prices for copyrighted books, whether they’re provided by Google or by anyone else that might get similar rights in the future.

And the information the registry maintains might not be what’s most useful to libraries and the public.   As agents for copyright holders, they’re likely to keep very good records about who claims rights to various books, and who you should pay if you want to use them.   But they won’t have much incentive to determine, for instance, that a particular work is actually in the public domain.  As we saw in yesterday’s first panel, many works published after 1922 actually are public domain, but it’s often very difficult for someone to find this out. And I’ve already heard some folks advise publishers to claim anything they might have rights to, if there’s any uncertainty about its status.  So there needs to be some counterbalance to the claims made by commercial publishers and rightsholder organizations.

That’s why I wonder whether it might be a good idea to imagine a sort of “Reader’s Rights Registry”, that would serve as a counterbalance to the Book Rights Registry.  The reader’s registry would track information and advocate on behalf of readers and libraries.  For instance, it could compile facts related to public domain status or open content licenses, and share these facts widely with the community.  It could press Google and other vendors to keep prices affordable, and to make it easy for readers to protect their privacy.   It could track books withheld or withdrawn from full text access, for whatever reason, and work to provide access through alternative channels where appropriate.  It could press Congress or the Justice Department for reasonable and nondiscriminatory blanket licensing of orphan works and out of print works.

I’m not saying we necessarily have to create a brand new organization to do these things.   They could be done by an existing organization, or set of organizations, or a newly merged organization.  What’s; important is that we find a way to articulate what our readers need from mass-digitized book services, gather appropriate information to support these needs, and organize the resources to make that a reality.  We shouldn’t wait for someone else, whether Google or the courts, to do it for us.

Now, there’s already work underway along these lines.  Many of you may be aware, for instance, of OCLC’s Copyright Evidence Registry.   It collects facts relevant to determining whether a book is in copyright or in the public domain.  At the moment, it’s just a pilot, and the pilot runs until the end of June, with an uncertain future afterwards.  And unfortunately, its user interface is limited, and its complex and restrictive terms of use looks to me even more convoluted that what they’re proposing for WorldCat.  And I worry that those constraints will effectively choke off the project before it can get sufficient participation.  In general, I think the library community is severely handicapping itself by keeping its metadata locked up, but that’s a whole other talk.

There’s also more work to be done in digitizing copyright registrations and renewals. I’m working on some of that now, and would be glad to talk about it later on.  Google also is doing its own digitization of the complete Catalog of Copyright Entries, and plans to release it publicly. I’m looking forward to that.

Hathi Trust is also doing important work.   As we heard yesterday, they’re doing their own copyright research on their books, and I hope they’ll eventually open up their research registry for the library community to use and contribute to.  The scope of activities one might imagine for a Reader’s Rights Registry is a bit broader than what Hathi Trust is currently doing, and it would require a broader base of support that what Hathi Trust has now.  But they’ve made a very good start on many of the kinds of independent activities I think we’ll need.

I know that all our libraries are going through difficult times in this economy, and any sort of external investment or organizational overhead can be hard to justify these days.  But these mass-digitized collections could seriously reshape the nature of research and reading in the years to come.  So I think it’s worth our putting some careful thought into how we want to influence that reshaping.  Whatever happens in the courtroom in the next few months, we should marshal the information, the will, and the organization to make sure our readers will be well-served.

March 18, 2009

The Google Books settlement: A symposium, and a call for library action

Filed under: libraries,metadata,online books,open access,reading,sharing — John Mark Ockerbloom @ 2:14 pm

Last Friday I went to a fascinating symposium at the Columbia Law School: “The Google Books Settlement: What Will it Mean for the Long Term?” The symposium included presentations by US copyright register Marybeth Peters, and antitrust expert Randal Picker [slides], followed by panels featuring speakers from the legal, publishing, and library world, as well as a few folks representing Google, authors, international publishing groups, and photographers.  The audience filled most of a large room, and included listeners from all these communities.  I prepared for the day by going through Walt Crawford’s lengthy summary of the settlement and its commentators, which I recommend to anyone needing to get up to speed on the issues.

Cornell’s Peter Hirtle has posted a useful summary of the proceedings, and I’ve also seen some interesting commentary on aspects of the meeting from Peter Brantley and Adam Hodgkin.  (The session was also live-Twittered.)

What I heard from audience members, both during and between sessions, was interesting as well.  Like Peter Hirtle, I didn’t hear many people saying directly that the settlement should not go forward (though I did hear that from a few people who were there but not on the stage).  I did hear repeated concerns over how the deal came about, and how it made substantial changes to the contours of copyright through class action litigation rather than legislation; and I also heard substantial concerns over the monopolies given to Google and the Book Rights Registry, and how those monopolies might stifle innovation and competition, drive up prices for online access to books, and infringe on rights of readers.  (Despite its name, the Book Rights Registry is not simply a passive store of information; it also actively negotiates terms and prices of online book uses on behalf of rightsholders, and manages the distribution of revenues from these uses.)

There was also some concern about whether those not involved in the negotiations, such as small publishers, unknowing owners of out-of-print works, foreign authors and publishers, and illustrators and photographers, were being properly respected.  Though with these latter concerns, there seemed to be more of a desire for a voice and piece of the action than a desire to stop the settlement from happening.

My basic take on the settlement didn’t change much. I’m guardedly hopeful about it, and would much rather see the settlement go forward than fail, because of the new and enhanced access it will give to millions of books for readers throughout the US.   That’s not to say it doesn’t pose some problems and dangers, many of the relevant ones having been  well-expressed by James Grimmelmann back in November. (Grimmelmann made additional remarks in one of the panels.)  The symposium did give me some new appreciation of some of the pitfalls to watch for, and some more ideas on how libraries might respond to them.  It also gave me a stronger impression that trying to modify the settlement itself is not likely to be a very fruitful response; my lay understanding is that the judge will likely just rule up or down on it with little or no modifications, with any major changes in terms requiring starting over negotiations with all the parties involved.

Still, there were some important constituencies that weren’t really addressed in the formal sessions.  One of them, noted late in the day by Harvard’s Robert Darnton, was individual readers.  (If you haven’t read Darnton’s pessimistic take on the settlement, or the response from Michigan’s Paul Courant, who was also there, you should.)  But another, I’d argue, was libraries as active agents in shaping the future of reading in the electronic age.  For libraries, I kept hearing questions like: Should they endorse or protest the settlement?  Should they buy or forgo the institutional subscriptions that will be offered?  Should they accept or pass on any of four digitization partnership deals that the settlement allows?  These are important questions, but they’re basically all reactive: choosing to go with or against the flow, without considering how we might shape or rechannel it.

How might libraries reshape this flow? Going into full detail would require a number of additional blog posts at least, but among other things, we can work individually and collectively to

  • Safeguard readers’ privacy (as we do now to some extent via proxy servers).
  • Ensure the preservation of digital and print books (through efforts like Hathi Trust and shared print collection management).
  • Collectively press for reasonable pricing and terms of use, respecting traditional user rights under copyright law, for institutional subscriptions.
  • Encourage Congress to pass legislation that would give other digitizers the rights to offer similar services to what Google will; enable the more widespread use of orphan works; and apply appropriate oversight to the monopolistic aspects of the Book Rights Registry.  In her talk, Marybeth Peters said that no one from Congress had asked the Copyright Office for comment on the proposed settlement.  While her office doesn’t appear interested in petitioning Congress, libraries can and should do so.
  • Integrate book digitization (and not just Google’s) with our library’s operations and services, so our users can make the most of what’s available digitally.   This can include integrating digital copy information and links in our catalogs, and pressing for links from those copies back to our library services (not just indirectly via OCLC’s “find in a library” link, but also direct links through technologies like OpenURL).
  • Promote and support the widespread, open sharing of information about digital books and their rights.  This information will make it easier to expand and improve what’s available online, and bring it to the widest possible audience.  We can’t rely on Google and the Registry to do all the work for us. Google’s own digitizations and metadata will not be perfect or incomplete.  The Book Rights Registry will not be freely sharing all of its information, and in any case will be more interested in paying rights claimants than in, say, enhancing access to the public domain.  If we want to solve this problem, we’ll need to continue to address some more general issues with sharing metadata more broadly.

We don’t have to wait for the settlement to go forward, or break down, to start thinking and experimenting in these areas.  I hope that a lot of libraries will be considering the issues raised in the Columbia symposium, and deciding how best to act (and not just react) to best serve our readers in a world where millions of books are accessible online.

January 14, 2009

Chances to stop and think about the future of library catalogs

Filed under: libraries,metadata,open access,sharing — John Mark Ockerbloom @ 3:41 pm

OCLC yesterday announced that it is putting its new catalog record usage policy on hold while its Members Council convenes a group to review the proposed policy and suggest changes.  I’m very happy to hear this, as the policy was due to take effect next month, and has been subject to extensive criticism online (both from me and from many others).

Whatever one thinks about the policy, it’s clear that changing the basic terms of sharing our knowledge about library resources– either to tighten the restrictions on use or open them up– is a fundamental amendment to the cooperative that has built up the information base of WorldCat.  It’s akin to a constitutional amendment in the political sphere.  And like the political unions of people in states, and of states in the United States, the members of the union should have the opportunity not only to suggest, but also to approve or reject, fundamental changes to the basic agreement.

The policy review group will initially meet in late January, according to OCLC’s press release, with the goal of producing a report that can be discussed in the member’s council meeting in May.  I’d imagine they would welcome thoughts and suggestions from anyone who’s involved with or thinking about the future of catalogs.

The upcoming ALA Midwinter meeting in Denver will provide a number of opportunities to think and talk about the future of catalogs, and how we manage knowledge about our knowledge.  I’ll be out there, albeit briefly, for a couple of events.

One is an ALCTS forum “Creating and Sustaining Communities Around Shared Library Data”, which will include a discussion of the data sharing policies of OCLC and other organizations.  I’ll be speaking at this forum (which will be on Monday at 8am in the Denver Convention Center), along with Karen Calhoun of OCLC, Brian Schottlaender of UC San Diego, and Peter Murray of OhioLINK.  I’m hoping to give an accessible presentation of the concerns of library practitioners, application developers, and metadata hackers, and would be happy to hear suggestions on issues I should take into account.

I’ll also be talking at the LITA Next Generation Catalogs interest group about the ILS-Discovery interface recommendations I worked on along with various other library professionals for DLF.  (This will be on Sunday at 4pm, also in the Denver Convention Center, and will also include a presentation from the BiblioCommons folks.)

If you’re around for the Midwinter meeting, you might also want to check out biblios.net, a freely shared database of catalog records (and an open source editor for working with them) that LibLime will be demonstrating during the meeting.  I haven’t played around with it yet myself, but it suggests some interesting new ways of working with and sharing library metadata.   Midwinter will also include sessions on RDA, an updated system for describing library resources that’s now looking for feedback and testers to see if it’s ready to become the next standard for cataloging.

From WorldCat to Google, the way we use catalogs and other metadata services is changing rapidly.  I hope we’ll have a chance during ALA, and during OCLC’s policy review period, to think carefully and creatively about how we should change these services to meet the needs of today’s and tomorrow’s information seekers.  And then I hope we’ll make those changes happen.

November 24, 2008

Drawing a line in the sand, Part 3: How to respond?

Filed under: copyright,discovery,libraries,metadata,open access,sharing — John Mark Ockerbloom @ 7:38 pm

In my previous two posts, I discussed how open library metadata is becoming increasingly important for the future of library content, and how OCLC’s new catalog policy works against it. By asserting proprietary rights over the records in WorldCat, OCLC risks relegating libraries’ painstaking descriptions of their resources to the sidelines of Web-based innovation and aggregation. In the long term, this threatens to marginalize libraries, and their missions.

In this last post of the series, I suggest responses that may improve the situation. OCLC’s new policy won’t go into effect until February, and it’s evolved since it first hit the Net a few weeks ago. Commenting on the policy, OCLC vice-president Karen Calhoun has said

We believe that libraries, the wider library-archives-museum community, and those they serve will benefit from the updated policy without placing our shared investment in WorldCat at peril.

I’ll take her word that OCLC believes that they’re acting in the best interests of the members of the WorldCat cooperative. I just don’t think they actually are acting in libraries’ best interests in their current proposal. But ultimately the libraries themselves are the most appropriate ones to determine that, not I, or OCLC’s officials. So my first suggestion is:

Get informed, and speak up. There’s been a lot of discussion about OCLC’s new policy, and you can find pointers to much of the public online debate here. So far, I’ve mostly just seen OCLC officials defending the policy, and various individuals inside and outside libraries criticizing it. The libraries in the WorldCat cooperative need to make their voices heard as well. Whether they’re for or against the policy, they should have a say on the fate of the metadata corpus they’ve worked to support and populate.

OCLC’s new policy doesn’t just affect traditional library catalog applications. The MARC records OCLC manages can hold lots of important additional information about library materials, including digitizations, copyright, and preservation. I’ve advocated cooperative information sharing in these areas (and participated in planning discussions for some of them), and I’ve been pleased to see OCLC’s leadership in these areas. I don’t want to see these initiatives held back by restrictive sharing policies attached to their records.

Know (and assert) your rights. The main legal justifications for restricting the reuse and sharing of catalog records are copyright law and contract law. Contract law may sometimes override rights that one would normally have under copyright (such as fair use and public domain), but generally requires some sort of specific agreement between parties. If you’re working with OCLC, you might want to make sure you don’t sign away rights you might otherwise have to your own cataloging, or to that of other libraries. I’m not a lawyer, and I can’t give you advice on how best to do this, but adding explicit statements of your rights, and modifying or striking out questionable concessions in OCLC contracts, might be in order.

It’s worth noting that OCLC’s policy, in its current draft, specifically says that it does not restrict simply associating an OCLC control number with an otherwise independent record, or a library reusing and redistributing WorldCat records marked as its own original cataloging. Libraries may want to explicitly affirm these prerogatives. (And third parties can freely attach OCLC numbers to their own records, and could make arrangements with libraries that have done original cataloging to free their records.)

The extent to which copyright can apply to catalog records is not a settled matter, to my knowledge, but the US Supreme Court’s Feist decision makes it clear that, under US law, simply compiling facts without “requisite originality” is not sufficient for copyright. It follows straightforwardly, in my opinion, that basic factual citation information in a catalog record (such as title, author, and edition) and other objective facts (such as page counts and physical dimensions) are no more copyrightable than the names, addresses and telephone numbers in the Feist telephone directory case were. In addition, works originally created by federal government agencies (like the Library of Congress) are not subject to US copyright. In other countries, though, copyright or other restrictions might apply to objective facts; and US copyright might apply to more subjective parts of a catalog record (like prose descriptions or subject analysis). If you want to remove doubt about the reusability of your metadata, and prevent it from being made proprietary, I suggest:

Attach a Creative Commons Attribution-ShareAlike license to your records. (Update: But see the discussion in the comments below about alternatives, and pros and cons of different approaches, before deciding to go this route. – JMO)  As I’ve noted previously, the Attribution-ShareAlike license allows any sort of reuse of a record, but requires that its source be credited, and that any copies or derivatives of the record remain shareable. Creative Commons licenses specifically reaffirm all the normal rights that users have under copyright law, so it should be safe to add them even to records that might be public domain, without compromising their status. That’s the license I use for my Online Books Page records.

The license should cover the record itself, and records that derive from it. It should not apply to the entire contents of any database that contains the record (as some have suggested). A database-wide viral license has both philosophical and practical problems. (There are lots of reasons why I might want to have an ILS or union catalog that contains both proprietary and Share-Alike records. For instance, I might have various electronic collections for which I’ve previously bought proprietary records, completely independent from my open records.)

OCLC can use Attribution-ShareAlike records in WorldCat and in any of the services they sell, just as any other commercial or noncommercial organization can. (They could also include them alongside proprietary records.) But they can’t add their own restrictions to the records — which I’ve noted previously are incompatible with ShareAlike — without violating the Creative Commons terms, or implicitly admitting that the record is public domain. Thus, it’s possible to pre-emptively “inoculate” records contributed to WorldCat, or to any other aggregation that includes the records, against their being made proprietary.

OCLC could decide to refuse records with Attribution-ShareAlike licenses. But they’d have to go out of their way to do this, and they’d be acting against the stated wishes of the contributor. I think it’s worth seeing if they’d push back like this. If they did, it’d be worth bringing to the public’s attention, and may also be worth distributing the records outside of OCLC. Which brings me to my last suggestion:

Consider alternative methods for sharing library metadata. Just as OCLC is not the creator of most of the library metadata in WorldCat, it is not the only possible coordinator of library metadata. A number of other organizations are also aggregating descriptions of library resources, some for specific applications, like Google’s Book Search, some for social networking businesses, like LibraryThing, some for preservation, like Hathi Trust, some for free-for-all sharing, like Open Library. Other data hubs may arise as well. (Indeed, an open WorldCat could remain a vital hub that enriches those other aggregations, and is in turn enriched from the information aggregated at the other hubs.) Libraries might contribute records to multiple aggregators.

It’s important to remember that any broad-based data cooperative is not likely to completely satisfy all its members. An alternative to OCLC will not necessarily be more open, unless its members hold it to that standard. Reliability, quality control, and seamless interaction are not easy to provide, and participants in alternative networks will have to put time and effort into getting them right.

New cataloging cooperatives could also provide places to experiment with better representations and workflows for library metadata. Implementors need to be careful about getting sidetracked here; past experiences with RDA and other proposed changes to cataloging infrastructure show that new initiatives can be argued over for years without much progress. But if new union catalogs are compatible with existing catalog systems (such as by providing and accepting standard MARC records), and support efficient workflows, they can potentially represent metadata internally in new structures that might be more informative and easier to maintain. This could improve library cooperation and sharing across the board (and maybe improve WorldCat itself in the process).

This ends my discussion, at least for now, of OCLC’s new policy and its conflict with open library metadata. I hope it’s helped inform and advise readers about the debate, and the issues at stake. And I hope it will help readers determine where they stand, and how they should respond.

November 21, 2008

Drawing a line in the sand, Part 2: Problems with OCLC’s catalog policy

Filed under: copyright,discovery,libraries,metadata,open access,sharing — John Mark Ockerbloom @ 7:53 pm

As I mentioned in my last post, OCLC’s new policy for the use of catalog records from WorldCat (a policy now in at least its third revision) has generated substantial controversy, particularly among library bloggers.

The controversy focuses on statements that prohibit reuse (without express OCLC approval) of records contributed to WorldCat, unless the use is both “noncommercial” and “reasonable”. What does “reasonable” mean here? OCLC’s policy statement (as of November 20) specifically excludes any use that

a. discourages the contribution of bibliographic and holdings data to WorldCat, thus
damaging OCLC Members’ investment in WorldCat, and/or

b. substantially replicates the function, purpose, and/or size of WorldCat

But OCLC makes it clear that this is not an exhaustive list, and that “OCLC has the sole discretion to determine whether any Use and/or Transfer of WorldCat Records complies with this policy”. OCLC has stated in its FAQ that starting in February, it will add notices to every record downloaded from WorldCat stating that use of the record is subject to OCLC’s new usage policy.

In other words, OCLC claims the right to decide whether any application using WorldCat-deposited catalog data should be allowed to exist.

OCLC admits that their policy is intended to protect their revenue, and claims that they are acting to protect its subscriber libraries. But as plenty of other commenters have pointed out, the policy puts a big OCLC-sized bottleneck in the way of innovative use and reuse of library metadata. This further promotes the marginalization of library resources, which could easily hurt those same subscriber libraries. Ultimately, I think this policy is self-defeating even for OCLC itself, because the policy itself “discourages the contribution of… data to WorldCat”. At least it does for anyone who sees open library metadata as important, and doesn’t want confusion or licensing fights to erupt when OCLC’s terms of use are inserted into the records they contribute.

OCLC Vice President Karen Calhoun likened the new OCLC policy to the Creative Commons Attribution-NonCommercial-ShareAlike licenses, but it’s incompatible in both letter and spirit with that license, and with any other ShareAlike license. The Creative Commons license allows enhancement, adaptation and reuse for any “function, purpose, or size”. (At least in the noncommercial domain. If, as I recommend, the NonCommercial aspect is not used, then both commercial and noncommercial applications– including any that OCLC sells– can draw on and enhance the content.) The terms are straightforward, and not subject to later reinterpretation or revocation if the grantor decides that their business is threatened. Users do not have to apply for permission to innovate.

The ShareAlike aspect, furthermore, prohibits any downstream organization from trying to lock up or impose additional restrictions, such as OCLC’s, on what was given to them freely. Open data is kept open.

Opening WorldCat data might make OCLC a leaner organization that it is now; Microsoft and Elsevier show that quasi-monopolies can sustain bigger margins than most other businesses. But OCLC also sells lots of services on metadata that bring in money, and it employs smart people who have deep knowledge of library practices and needs, and who continue to bring out innovative and useful services. (To take just one example from OCLC’s many services, WorldCat Local could attract lots of subscription fees from libraries that no longer want to run their own ILS, if the service is well enough designed and run.) In the information technology world, companies like Red Hat and IBM do quite well selling services on free software, and I’m confident that folks at OCLC also have the expertise to sustain themselves with services on open data.

As Stefano Mazzocchi points out, OCLC’s new policy might buy them some time, but the Web isn’t going away, and the people (and companies) who might find library metadata useful aren’t going away either. OCLC is hardly the only organization that can create library metadata. (Indeed, they largely get it from their own subscribers.) But OCLC’s ShareAlike-incompatible license effectively draws a line in the sand: a catalog record can support the agenda of open library data, or support OCLC’s agenda, but not both.

If forced to take sides, the choice is clear to me: open library data is ultimately the best way to support libraries and their missions. If possible, though, I’d rather work with OCLC than work around them. In Part 3 of this series (coming soon now posted), I’ll suggest some possibly useful ways to respond between now and February, when OCLC’s new policy is scheduled to take effect.

November 19, 2008

Drawing a line in the sand, Part 1: The importance of open library metadata

Filed under: copyright,discovery,libraries,metadata,open access,sharing — John Mark Ockerbloom @ 11:11 pm

(First of a series of 3 posts.  See also Part 2: Problems with OCLC’s catalog policy, and Part 3: How to respond?)

A new bibliographic record distribution policy from OCLC threatens to split the library community from the increasingly large and valuable data sharing resources and applications on the Web, if it doesn’t simply fracture the library community internally. In this post, I’ll try to explain the basic issues. I hope to follow this with specific critiques of OCLC’s policy, and suggestions of useful responses. (If you’d just like a concise overview of the controversy, this Inside Higher Ed article is a good place to start.)

Librarians often lament that people all too often bypass the well-written, informative resources collected by libraries in favor of material on the open Web. One reason for this practice is that Web content is widely harvested, indexed, and annotated by a variety of entities. Some are well-known, large-scale services like Google, Technorati, or Delicious. Some are smaller services that specialize in a particular niche. Each of these services reuses openly accessible data to make Web content easy to find, search, and annotate. Information about what’s in libraries (“metadata”, in library parlance) has also been compiled with great detail and effort, but it tends to be locked up behind individual libraries’ online public access catalogs (OPACs) that tend to be less usable, and less visible, than services like Google.

It doesn’t have to be this way. If you follow this blog, for instance, you’re aware of the DLF ILS Discovery Interfaces recommendation, which is meant to free library metadata from the constraints of the OPAC and make it available to a wide variety of discovery applications. And a growing number of Web services like Open Library, LibraryThing, WorldCat.org, and Google Book Search aggregate metadata from a variety of libraries and other knowledge sources. They open up interesting new ways for people to find, use, and annotate books and other knowledge sources. Bringing library resources into the light through services like these helps readers find the best information, and helps libraries fulfill their missions.

These services, as well as libraries themselves, rely on aggregating the metadata they need from a wide variety of sources. Creating catalog records for books is a laborious and painstaking process, one that would be too labor intensive for most libraries acting on their own. So librarians long ago agreed to partition the work, exchange their records, and enhance them jointly, though the use of shared cataloging and union catalogs that combined the different libraries’ records. Union catalogs were first devised well before the Web, when libraries mainly traded information just among themselves. Union catalog participants typically contribute their own catalog records, and pay a subscription fee for the right to retrieve and reuse records from the union catalog. One current union catalog, OCLC’s WorldCat, has absorbed other union catalogs over time and is currently much larger than any other of its kind. The WorldCat.org website gives free public access to some, but not all, of the information in the subscription-based WorldCat.

In effect, libraries are paying their staff to create catalog records, giving them to OCLC, and then paying to get them back. This isn’t necessarily a bad thing if you’re paying a reasonable price for a useful service, such as OCLC provides with its WorldCat service. But such arrangements can turn exploitative or obstructionist over time. We in libraries are all too aware of this when we see the invoices for the journals we subscribe to, where the articles are written and refereed for free by our faculty. The subscription fee in some cases can cost as much as a new car just for one year of a single journal. And the scholars who wrote the articles typically sign away their rights to get them published, and then can be surprised to discover that they are restricted from redistributing or reusing what they themselves wrote.

An alternative advocated by many library professionals (myself included) is open access, where intellectual content can be freely shared and reused. We have lots of arguments about how open access can lower costs, increase visibility, and promote the global spread of knowledge. The arguments are not just about economics and philanthropy, but about improving scholarship. For instance, when data is freely shared, it can be fruitfully be reused, repurposed, remixed, and reanalyzed in new scholarship and teaching. Yet, even while libraries have promoted open access, open access has not been the principal ways in which we have shared and distributed our own cataloging.

At Palinet’s future of cataloging forum I attended earlier this year, I heard folks in various parts of libraries start to speak up about opening up access to our own cataloging data. My own talk at that forum argued for opening access to catalog data, and recommended doing so via Creative Commons Attribution-ShareAlike licenses. There were also OCLC staff at the same forum, and while they did not promise anything specific, I got the sense that they were planning on opening up access to their WorldCat records.

The new policy does clarify how individual researchers and libraries can reuse and repurpose WorldCat records in some useful ways. Unfortunately, it also explicitly asserts OCLC control over these records, in a way that threatens to dampen much of the sharing and independent collective action that can make our library metadata much more visible and useful. In a followup post, I’ll summarize the problems I see in the policy, and then suggest some things that we might do to help free our library metadata for the benefit of our users.

« Previous Page

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 49 other followers