Everybody's Libraries

May 6, 2010

Making discovery smarter with open data

Filed under: architecture,discovery,online books,open access,sharing,subjects — John Mark Ockerbloom @ 9:06 am

I’ve just made a significant data enhancement to subject browsing on The Online Books Page.  It improves the concept-oriented browsing of my catalog of online books via subject maps, where users explore a subject along multiple dimensions from a starting point of interest.

Say you’d like to read some books about logic, for instance.  You’d rather not have to go find and troll all the appropriate shelf sections within math, philosophy, psychology, computing, and wherever else logic books might be found in a physical library.  And you’d rather not have to think of all the different keywords used to identify different logic-related topics in a typical online catalog. In my subject map for logic, you can see lots of suggestions of books filed both under “Logic” itself, and under related concepts.  You can go straight to a book that looks interesting, select a related subject and explore that further, or select the “i” icon next to a particular book to find more books like it.

As I’ve noted previously, the relationships and explanations that enable this sort of exploration depend on a lot of data, which has to come from somewhere.  In previous versions of my catalog, most of it came from a somewhat incomplete and not-fully-up-to-date set of authority records in our local catalog at Penn.  But the Library of Congress (LC) has recently made authoritative subject cataloging data freely available on a new website.  There, you can query it through standard interfaces, or simply download it all for analysis.

I recently downloaded their full data set (38 MB of zipped RDF), processed it, and used it to build new subject maps for The Online Books Page.   The resulting maps are substantially richer than what I had before.  My collection is fairly small by the standards of mass digitization– just shy of 40,000 items– but still, the new data, after processing, yielded over 20,000 new subject relationships, and over 600 new notes and explanations, for the subjects represented in the collection.

That’s particularly impressive when you consider that, in some ways, the RDF data is cruder than what I used before.  The RDF schemas that LC uses omit many of the details and structural cues that are in the MARC subject authority records at the Library of Congress (and at Penn).  And LC’s RDF file is also missing many subjects that I use in my catalog; in particular, at present it omits many records for geographic, personal, and organizational names.

Even so, I lost few relationships that were in my prior maps, and I gained many more.  There were two reasons for this:  First of all, LC’s file includes a lot of data records (many times more than my previous data source), and they’re more recent as well.  Second, a variety of automated inference rules– lexical, structural, geographic, and bibliographic– let me create additional links between concepts with little or no explicit authority data.  So even though LC’s RDF file includes no record for Ontario, for instance, its subject map in my collection still covers a lot of ground.

A few important things make these subject maps possible, and will help them get better in the future:

  • A large, shared, open knowledge base: The Library of Congress Subject Headings have been built up by dedicated librarians at many institutions over more than a century.  As a shared, evolving resource, the data set supports unified searching and browsing over numerous collections, including mine.  The work of keeping it up to date, and in sync with the terms that patrons use to search, can potentially be spread out among many participants.  As an open resource, the data set can be put to a variety of uses that both increase the value of our libraries and encourage the further development of the knowledge base.
  • Making the most of automation: LC’s website and standards make it easy for me to download and process their data automatically. Once I’ve loaded their data, and my own records, I then invoke a set of automated rules to infer additional subject relationships.  None of the rules is especially complex; but put together, they do a lot to enhance the subject maps. Since the underlying data is open, anyone else is also free to develop new rules or analyses (or adapt mine, once I release them).  If a community of analyzers develops, we can learn from each other as we go.  And perhaps some of the relationships we infer through automation can be incorporated directly into later revisions of LC’s own subject data.
  • Judicious use of special-purpose data: It is sometimes useful to add to or change data obtained from external sources.  For example, I maintain a small supplementary data file on major geographic areas.  A single data record saying that Ontario is a region within Canada, and is abbreviated “Ont.”, generates much of my subject map for Ontario.  Soon, I should also be able to re-incorporate local subject records, as well as arbitrary additional overlays, to fill in conceptual gaps in LC’s file.  Since local customizations can take  a lot of effort to maintain, however, it’s best to try to incorporate local data into shared knowledge bases when feasible.  That way, others can benefit from, and add on to, your own work.

Recently, there’s been a fair bit of debate about whether to treat cataloging data as an open public good, or to keep it more restricted.  The Library of Congress’ catalog data has been publicly accessible online for years, though until recently only you could only get a little a time via manual searches, or pay a large sum to get a one-time data dump.  By creating APIs, using standard semantic XML formats, and providing free, unrestricted data downloads for their subject authority data, LC has made their data much easier for others to use in a variety of ways. It’s improved my online book catalog significantly, and can also improve many other catalogs and discovery applications.  Those of us who use this data, in turn, have incentives to work to improve and sustain it.

Making the LC Subject Headings ontology open data makes it both more useful and more viable as libraries evolve.  I thank the folks at the Library of Congress for their openness with their data, and I hope to do my part in improving and contributing to their work as well.

January 1, 2010

Public domain day 2010: Drawing up the lines

Filed under: copyright,online books,open access — John Mark Ockerbloom @ 12:01 am

As we celebrate the beginning of the New Year, we also mark Public Domain Day (a holiday I’ve been regularly celebrating on this blog.)  This is the day when a year’s worth of copyrights expire in many countries around the world, and the works they cover become free for anyone to use and adapt for any purpose.

In many counties, this is a bittersweet time for fans of the public domain.  For instance, this site notes the many authors whose works enter the public domain today in Europe, now that they’ve been dead for at least 70 years.  But for many European countries, this just represents reclaimed ground that had been previously lost.   Europe retroactively extended and revived copyrights from life+50 to life+70 years in 1993, so it’s still three more years before Europe’s public domain is back to what it was then.  Many other countries, including the United States, Australia, Russia, and Mexico, are in the midst of public domain freezes.  For instance, due to a 1998 copyright extension, no copyrights of published works will expire here in the US due to age for another 9 years, at least.

In the past, many people have had only a vague idea of what’s in the public domain and what isn’t.  But thanks to mass book digitization projects, the dividing line is becoming clearer.  Millions of books published before 1923 (the year of the oldest US copyrights) are now digitized, and can be found with a simple Google search and read in full online.  At the same time, millions more digitized books from 1923 and later can also be found with searches, but are not freely readable online.

Many of those works not freely readable online have languished in obscurity for a long time.   Some of them can be shown to be in the public domain after research, and groups like Hathi Trust are starting to clear and rescue many such works.  Some of them are still under copyright, but long out of print, and may have unknown or unreachable rightsholders.  The current debate over Google Books has raised the profile of these  works, so much so that the New York Times cited “orphan books”, a term used to describe such unclearable works, as one of the buzzwords of 2009.

The dividing line between the public domain and the world of copyright could well have been different.   In 1953, for instance, US copyrights ran for a maximum of 56 years, and the last of that year’s copyrights would have expired today, were it not for extensions.  Duke’s Center for the Study of the Public Domain has a page showing what could have been entering the public domain today– everything up to the close of the Korean War.  In contrast, if the current 95-year US terms had been in effect all of last century, the copyrights of 1914 would have only expired today.  Only now would we be able to start freely digitizing the first set of books from the start of World War I.

With the dividing line better known nowadays, do we have hope of protecting the public domain against more expansions of copyright?  Many countries still stick to the life+50 years term of the Berne Convention, including Canada and New Zealand.  In those countries, works from authors who died in 1959 enter the public domain for the first time.  There’s pressure on some of these countries to increase their terms, so far resisted.  Efforts to extend copyrights on sound recordings continues in Europe, and recently succeeded in Argentina.  And secret ACTA treaty negotiations are also aimed at increasing the power of copyright holders over Internet and computer users.

But resistance to these expansions of copyright is on the rise, and public awareness of copyright extensions and their deleterious effects is quite a bit higher now than when Europe and the US extended their copyrights in the 1990s.  And with concerns expressed by a number of parties over a possible Google monopoly on orphan books, one can envision building up a critical mass of interest in freeing more of these books for all to use.

So today I celebrate the incremental expansion of the public domain, and hope to help increase it further. To that end, I have a few gifts of my own.  As in previous years, I’m freeing all the copyrights I control for publications (including public online postings) that are more than 14 years old today, so any such works published in 1995 and before are now dedicated to the public domain.  Unfortunately, I don’t control the copyright of the 1995 paper that is my most widely cited work, but at least there’s an early version openly accessible online.

I can also announce the completion of a full set of digitized active copyright renewal records for drama and works prepared for oral delivery, available from this page.  This should make it easier for people to verify the public domain status of plays, sermons, lectures, radio programs, and similar works from the mid-20th century that to date have not been clearable using online resources.  We’ve also put online many copyright renewal records for images, and hope to have a complete set of active records not too far into 2010.  Among other things, this will help enable the full digitization of book illustrations, newspaper photographs, and other important parts of the historical record that might be otherwise omitted or skipped by some mass digitization projects.

Happy Public Domain Day!  May we have much to enjoy this day, and on many more Public Domain Days to come.

(Edited later in the day January 1 to fix an inaccurately worded sentence.)

October 26, 2009

Promoting access to the best literature of the past

Filed under: online books,open access — John Mark Ockerbloom @ 3:24 pm

Last week saw widespread observance of Open Access Week 2009 .  The week primarily focused on opening access to current research and scholarship (though there’s also been a growing community working on opening access to teaching and learning content).  You can find lots of open access resources at the Open Access Directory.

Current scholarship is not spontaneously generated from the brain or lab of the writer.  Useful scholarship must understand and interpret past work, to be effective in the present.  In many fields, and not just the classical humanities, the relevant past work may stretch back hundreds or even thousands of years.  Current scholarship and study will be more effective if its source material is also made openly accessible, and if proper attention is drawn to the most useful sources.  And now is an especially opportune time for scholars of all sorts, professional and amateur, to get involved in the process.

This may seem a strange thing to say at a time when the digitization of old books and other historic materials is increasingly dominated by large-scale projects like Google and the Internet Archive.  With mass digitizers putting millions of public domain book and journal volumes online, and with a near-term possibility of millions more copyrighted volumes going online as well, how much of a role is left for individual scholars and readers?

A very important role, as it turns out.  Mass digitization projects can quickly produce large scale aggregations of pass content, but as many have pointed out, aggregation is not the same as curation, and as aggregations grow larger, being able to find the right items in a growing collection becomes increasingly important.  That’s what curation helps us do, and the large-scale digitizers are not doing a very effective job of it themselves.  Google’s PageRank algorithm may take advantage of implicit curation of web pages (through the choices of authors’ page links), but Google and other aggregators have had a much harder time drawing attention to the most useful books, scholarly articles, or other works created without built-in hyperlinks.

Sometimes this is because they haven’t digitized them, even as they’ve digitized inferior substitutes.  Over three years after Paul Duguid lamented the republication of a bowdlerized translation of Knut Hamsun’s Pan by Project Gutenberg, that version remains the only freely available one of this book available there, or at Google Books, or anywhere else online that I’ve found.   Even though an unexpurgated version of this translation was published before the bowdlerized version, no digitizer that I know of has gotten around to finding and digitizing it; and countless readers may have used the existing online copies without even knowing that they’ve been censored.  Extra bibliographic and copyright research may be necessary to determine whether a better resource is available for digitization, as it in this case.

Sometimes the content is digitized, but can’t be found easily.  Geoff Nunberg’s post on Google Books’ “metadata train wreck” shows plenty of examples of how difficult it can be to find and properly identify a particular edition in Google Books, much less figure out which edition is the best one to use.  I’ve commented in the past about the challenges of finding multi-volume works in that corpus.  And Peter Jacso has pointed out Google’s problems indexing current scholarship.  If you can’t find the paper or book you need for your research, your work will be no better than it would be if the source had never existed.

This is where scholars can potentially play a useful role.  We don’t individually digitize books by the thousands, but we do individually find, cite, and recommend useful sources, down to the particular edition, as we find them and use them in our own writings and teaching.  These citations and recommendations now often go online, in various locations.  It would be very useful to have these recommendations made more visible, and tied to freely available online copies of the sources cited, whenever legally possible. Sometimes, we also create or digitize our own editions of past works, with useful annotations, for our classes or our own work.  It would be very useful to have these made visible and persistent as well, whenever appropriate.

I hope that large resource aggregations will make it easier for scholars and others to curate the collections to make them more useful to their readers.  In the meantime, we can start with resources we have.  For example, on The Online Books Page, my catalog entry for Hamsun’s Pan notes its limitations.  My public requests page includes information on a better edition that could be digitized, by someone who has access to the edition and has some time to spare.  And my suggestion form is ready to accept links to better editions of this book, or to other online books that merit special attention.  Indeed, most of the books that I now add to my catalog derive from submissions made by various readers on this form, and I invite scholars to suggest the freely accessible books and serials that they find most useful for my catalog.

As the Little Professor notes in a recent post, the sort of bibliographic work I’ve described can be time-consuming but vitally important for making effective use of old sources, and that work has often not been done by anyone for many books outside the usual classical canons.  Yet it’s the sort of thing that scholars do, bit by bit, as part of their everyday work.  The aggregate effect of their curation and digitization, appropriately harnessed in open-access form, could greatly improve our ability to build upon the work of the past.

September 17, 2009

Google Book settlement: Alternatives and alterations

Filed under: copyright,online books,open access — John Mark Ockerbloom @ 1:35 pm

In my previous post, I worried that the Google Books settlement might fall apart in the face of opposition from influential parties like the Copyright Office, and that such a collapse might deprive the public of meaningful access to millions of out of print books.

Not everyone sees it that way.  I’ve seen various suggestions of alternatives to the settlement for making these books available.  In this post, I’ll describe some of the suggested alternatives, explain why they don’t seem to me as likely to succeed on their own, and discuss how some of them could still go forward under a settlement.

Compulsory licenses

Both the Open Book Alliance’s court filings and the Copyright Office’s testimony mention the possibility of compulsory licensing, which essentially lets people use a copyrighted work without getting permission, provided that they meet standard conditions determined by the government.  Compulsory licenses already exist in certain areas, such as musical performances and broadcasts.  If I want to cover a Beatles song on my new record, I can, as long as meet some basic conditions, including paying a standard royalty.  The (remaining) Beatles can’t hold out for a higher rate, or say that no one else is allowed to cover the recordings they’ve released.

The Google Books settlement has some similarities to a compulsory license, but with some important differences, including:

  1. Book rightsholders can choose to deny public uses of their work, or hold out for higher compensation, which they generally can’t do under a compulsory license regime. (They have to explicitly request this, though.  So it’s really what one might call a “default” license.)
  2. The license has been negotiated through a court settlement rather than Congressional action. (This was one of the main complaints of the Copyright Office.)
  3. The license given in the settlement is granted only to Google, not to other digitizers. (This has justifiably raised monopoly concerns.)

I do have a problem with the last difference as it stands.  I’d like to see the license widened so that anyone, not just Google, could digitize and make available out of print books under the same terms as Google. But there are various ways we can get to that point from the settlement.  The Book Rights Registry created by the settlement could extend Google-like rights to anyone else under the same terms, as the settlement permits them to do.  The Justice Department could require them to do so as part of an antitrust supervision.  Or Congress could decide to codify the license to apply generally.  (They’ve done this sort of thing before with fair use and the first sale doctrine, both of which originated in the courts.)

If the settlement falls apart, though, negotiation over an appropriate license has to start over from scratch, and has to persuade Congress to loosen copyrights for benefits they might not clearly see. As I suggested in my previous post, Congress’ recent tendencies have heavily favored tightening, rather than loosening, copyright control.   And I haven’t yet seen a strong coalition pushing for laws granting compulsory (or default) licenses that are as broad as would be needed.

For instance, the Open Books Alliance’s amicus brief suggests the possibility of a compulsory license, but only as “but one approach”, and that suggestion seems as much aimed at getting hold of Google’s scans as licensing the book copyrights themselves.  Their front page at present shows no explicit advocacy of compulsory copyright licenses.  Perhaps they will unite behind a workable Google Books-style compulsory license proposal in the future, but I’m not counting on that.  (Update: Just after I posted this, I saw this statement of principles go up on the OBA site.  We’ll see what develops from that.)

The Copyright Office’s congressional brief also mentions but tries to damp down the idea.  It repeatedly characterizes compulsory licensing as something that Congress only does “reluctantly” and “in the face of marketplace failure”. But despite its strong words on other subjects, it does not appear concerned over whether we in fact have a marketplace failure around broad access to out-of-print books.

Orphan works legislation

The Copyright Office filing also suggests passing orphan works legislation (as have various other parties, including Google).  An orphan works limitation on copyrights would be nice, but it’s not going to enable the sort of large, comprehensive historical corpus that the Google Books settlement would allow.

As Denise Troll Covey has pointed out, the orphan works certification requirements recommended in last year’s bill, like many other case-by-case copyright clearance procedures, are labor-intensive and slow, and may be legally risky.  (In particular, the overhead for copyright clearance, not including license payment, can be several times the cost of digitization.)  Hence, these methods are not likely to scale well.  And they would not cover the many out-of-print books that aren’t, strictly speaking, orphans.  I don’t consider it likely that a near-comprehensive library  of millions of out-of-print 20th century books will come about by this route alone any time soon.

Even so, despite its limited reach, last year’s orphan works legislation was stopped in Congress after some creator organizations objected to it.  Some of the objectors, including the  National Writers Union and the American Society of Journalists and Authors, are now members of  the Open Book Alliance, which makes me wonder how effectively that group would act as a united coalition for copyright reform.

Private negotiation

Some critics suggest that Google and other digitizers simply negotiate with each rightsholder, or a mediator designated by each  rightsholder.   It’s possible that this actually might work for many future books, if authors and publishers set up comprehensive clearinghouses (like ASCAP and Harry Fox mediate music licensing).  If new books get registered with agents like these going forward, with simple, streamlined digital rights clearing, private arrangement could work well for future books both in-print and out-of-print.  Indeed, Google’s default settlement license privileges don’t apply to new books from 2009 onward.

But it’s much less likely that this will be a practical solution to build a comprehensive collection of past out of print books from the 20th and early 21st century, because of the sheer difficulty and cost of determining and locating all the current rightsholders of books long out of print.   The friction involved in such negotiation (involving high average cost for low average compensation) is too great.  Without the settlement and/or legal reform, we risk having what James Boyle called a “20th century black hole” for books.

Copyright law reform

As James Boyle points out, it would solve a lot of the problems that keep old books  in obscurity if books didn’t get exceedingly long copyrights purely by default.  It would also help if fair use and public domain determination weren’t as risky as they are now. I’d love to see all that come to pass, but no one I know that’s knowledgeable on copyright issues is holding their breath waiting for it to happen any time soon.

Moving forward

As I’ve previously mentioned, the settlement is imperfect.  It may well need antitrust supervision, and future elaboration and extension.  (And I’ve suggested some ways that libraries and others can work to improve on it.)  It’s still the most promising starting point I’ve seen for making comprehensive, widely usable, historic digital book collections possible.  I hope that we get the chance to build on it, instead of throwing away the opportunity.  In any case, I’d be happy to hear people’s thoughts and comments about the best way to move forward.

September 15, 2009

Google Books, and missing the opportunities you don’t see

Filed under: copyright,online books,open access — John Mark Ockerbloom @ 9:12 pm

The Google Books settlement fairness hearing is still a few weeks away, but in the last few weeks the deal has been talked and shouted about with ever-higher volume.  Still, it wasn’t until the other day, in a House Judiciary Committee hearing where US Copyright Register Marybeth Peters came loaded for bear, that I started thinking there was a significant likelihood that the settlement might fall apart.

There are a number of people in different communities, including libraries, who hope this  happens.   I’m not one of them.  I’m not a lawyer, so I can’t comment with authority on whether the settlement is sound law.  But I’m quite confident that it advances good policy.  In particular, it’s one of the best feasible opportunities to bring a near-comprehensive view of the knowledge and culture of the 20th and early 21st centuries into widespread use.  And I worry that, should the settlement break down, we will not have another opportunity like it any time soon.  The settlement has flaws, like the Google Books Project itself has, but at the same time, like Google Books itself, the deal the settlement offers is incredibly useful to readers, while also giving writers new opportunities to revive, and be paid for, their out of print work.

The potential

Under the status quo, millions of books are greatly under-utilized.  It isn’t just that people don’t have easy access to them; it’s that people don’t know that particular books useful to them exist in the first place.  I work in a library that has collected millions of volumes, many of which are hardly ever checked out. Not only would Google’s expanded books offerings give our users access to millions more books, but it would also make millions of books that we already own easier for our users to find and use effectively.

Want to know what books make mention of a particular event, ancestor, or idea?  With existing libraries, and good search skills, you might be able to find books, if any, that are written primarily about those things. But you’ll probably miss much other information on those same topics, information in works that are primarily about something else.  With expanded search, and the ability to preview old book content, it could be much easier to get a more comprehensive view on a topic, and find out which books are worth obtaining for learning more.

And if that’s a big advance for people in big universities like ours, it’s an even bigger step forward for people who have not had easy access to big research libraries.  Once a search turns up a book of interest, Google Books would offer a searcher various ways of getting that book: buying online access; reading it at their library’s computer (either via a paid subscription, or via a free public access terminal); buying a print copy; or getting an inter-library loan.  These options all involve various trade-offs of cost and convenience, as is the case with libraries today.  While one could wish for better tradeoff terms, the ones proposed still represent big advances from what one can easily do today.

And as with other large online collections like Wikipedia or WorldCat, or the Web as a whole, the advantages to large book corpuses like Google’s aren’t just in the individual items, but in what can be done with the aggregation.  I don’t know exactly what new kinds of things people will find to do with a near-comprehensive collection of  20th century books, but having seen all that people have done with other information aggregated on the Internet, I’m confident that there would be many great uses found, large and small.

The peril

If the Google settlement does fall apart, are we likely to see any collection like the one it envisions any time soon?  I’m not at all confident we will.  The basic problem is that, without some sort of blanket license, it’s impractical (and in the case of true orphan works, currently impossible) to clear all the copyrights that would be required to build such a collection.  This represents a failure in copyright law.  Instead of “promot[ing] the progress of science and useful arts”, as the Constitution requires, current US copyright law effectively keeps millions of out-of-print books in obscurity, not producing significant benefits either to their creators or to their potential users.

The current proposed Google Books settlement is, among other things, an attempt to get around this failure.  If the settlement fails, would the parties make a new agreement that would allow a readable collection of millions of post-1922 online books?  The divergence in the complaints I’ve seen (for instance, on one hand that the collection would cost readers too much, and on another hand that it would pay writers too little) suggest the difficulty of coming to a new consensus that satisfies all the parties, if negotiations have to start again from scratch.  And, if the arguments of the Copyright Office and some of the other parties carry the day, even if such an agreement were reached, the agreement could not be ratified by a court anyway.  Instead, it would require acts of Congress, and maybe even re-negotiations of international treaties.

Based on past history, there are two things that would make the government likely to reform copyright law to permit mass reuse of out-of-print books.  Ether there needs to be a clear example of the benefits of such a reform, or there needs to be a strong coalition pushing for such a reform.  Clear examples have usually come from businesses that are actually in operation; for example, the player piano roll industry that successfully persuaded Congress to streamline music copyright clearance in the previous century (or the Betamax that persuaded a slender majority of the Supreme Court to declare the VCR legal).

If the proposed Google Books library service goes online, even under a flawed initial settlement, it too could provide a compelling example to encourage general copyright reform.  But without such an example, it can be hard to move Congress to act.   It’s easy to undervalue the opportunities you don’t clearly see.

What about a strong coalition pushing for a reform in the law that would let anyone create the comprehensive online collections of out of print books I’d described?  I’d like to see one, but I haven’t yet.  (Yes, there’s the Open Book Alliance, but its members don’t seem to be distinctly allied in anything particular other than objecting to the settlement.)  In my next post, I’ll discuss reforms that might do the job, and the reasons I believe they would be difficult to enact without the settlement.

August 31, 2009

Why should reuse be hard?

Filed under: architecture,open access,publishing — John Mark Ockerbloom @ 11:04 pm

By far the most widely cited paper with my name on it is a 1995 paper on architectural mismatch.  The journal version of the paper was subtitled “Why reuse is so hard”.  It was a paper about failure, rather than success, which most researchers prefer to write about when they’re talking about their own work.  We discussed the problems we’d encountered trying to build a new software system from existing parts, and analyzed some of the reasons for the failures, and how systems could be improved in the future to make reuse easier.

The paper was unexpectedly well received, and was recently named as one of the most influential papers to appear in IEEE Software.  (I can’t claim too much credit for this myself; my adviser David Garlan and my fellow grad student Robert Allen rightly appear ahead of me in the author credits.)  ISI Web of Knowledge, which tracks the journal version of the paper, reports it’s been cited over 100 times in other journal articles; Google Scholar, which tracks both the journal version and the conference version that was published earlier the same year, reports hundreds more citations.

Google Scholar also reports an unexpected statistic: even though the journal version of a computer science paper is generally considered more authoritative than the earlier conference version (and rightly so, in our case), the conference paper has been cited even more often than the journal version.  Why is this?  I can’t say for sure, but there’s one important difference between the two versions:  the conference paper has been freely accessible on the web for years, and the journal paper hasn’t.  It’s in a highly visible journal, mind you– pretty much anywhere with a CS department subscribes to IEEE Software, and many individual computer practitioners subscribe as well.  So I suspect that most of the authors who cited our paper could have cited the journal paper (especially since it came out only a few months after the conference paper did).  But the conference paper was that much more easily accessible, and it was the one that got the wider reuse.

We’ve recently published a followup to our paper, appearing in the July/August issue of IEEE Software.  As we note in the followup, the problem of architectural mismatch has not gone away, but several developments have made it easier to avoid.   One of them is the great proliferation of open source software that has occurred since the mid-1990s, which provide a wide selection of software components to choose from in many areas, and “a body of experience and examples that clarify which architectural assumptions and application domains go with a particular  collection of software” (to quote from our paper).

Just as the growth of open source has made software easier to reuse, the growth of open access to research can make ideas and research results easier to reuse.  We saw that with our initial paper, I think, and I hope we’ll see it again with the followup. I’ve made it available as open access, with IEEE’s blessing.  Interested folks can check it out here.

May 16, 2009

Free Decimal Correspondence: Now with more detail and CC0 goodness

Filed under: libraries,metadata,open access,repositories,sharing — John Mark Ockerbloom @ 7:38 pm

I’ve just released a new version of the Free Decimal Correspondence (FDC).  FDC is a set of associations between numbers and subject terms that can be used as the basis for linear shelving or subject browsing of a simple library collection or repository. Like the earlier releases, it’s intended to be reasonably compatible with existing library standards; in particular, with OCLC’s Dewey Decimal Classification (DDC) and the Library of Congress Subject Headings (LCSH).  (Keep in mind, though, that both of those standards go into considerably more detail than FDC does; and neither OCLC nor the Library of Congress sponsors or endorses the FDC in any way.)   This version of the FDC, like the previous releases, is dedicated to the public domain.  More information about the FDC can be found here.

What’s new?

There are two noteworthy features of this release.  First of all, I’ve now defined corresponding subjects for every whole number (from 000 to 999) that’s also defined (and non-optional) in the DDC.  In addition, I’ve gone past the decimal point in some cases to include additional subjects that may be useful in some collections and repositories.  This means that the FDC should now cover at least as many subjects as OCLC’s Dewey Decimal Classification summaries, the most detailed official free-to-read summary of the present-day DDC I know about.

In addition, I’ve now using Creative Commons‘ new CC0 dedication to dedicate the FDC to the public domain.  I’m hoping this will provide more authoritative legalese for clarifying the public domain status of FDC, and also raise awareness of CC0.

OCLC claims copyright and trademark on the DDC, and while the summary of DDC can be freely read on the Web, its use appears to be governed by this license, which states, among other things, that users may “not copy, reproduce, alter, modify, create derivative works, or publicly display any content (except for your own non-commercial use)”.  It further clarifies that “‘non-commercial use’ … is not to be construed as permitting use of the DDC that (although internal) supports or otherwise facilitates your commercial activities.”  (Note also that the linked license applies only to the DDC summary, not to the complete DDC.  If you want a license for the full DDC, you’ll need to contact OCLC.)

With FDC, as with all CC0 and other public domain works, none of the restrictions above apply.  Do with it what you like.

What’s next?

This new release is version 0.05.  As the number suggests, this is still essentially a working draft.  My first draft was released a few months ago on Public Domain Day.  I intend my work on FDC to be a lightweight, spare time project, and therefore hope to advance to a “1.0″ release soon, and leave further development, if desired, to others.

Between now and then, I’m not planning to make FDC much bigger than it is now.  But there are still various other subjects not defined at the unit level that are common in repositories and small library collections, and that might be useful to add.  (I originally started this after someone said they wanted to have a Dewey-like classification system for their repository, without worrying about licensing issues.)  I’m happy to take suggestions of additional subjects to include in the next release.  (But use your own subjects of interest, and not terms taken from an OCLC publication.)  I’d also be very interested in hearing about (and fixing) mistakes, inconsistencies, redundancies, and correspondences that could be made more compatible with current library practice than what I have now.

You can send me email about any of the above. I’ll also be at Open Repositories 2009 next week, and would be happy to talk about FDC there with anyone interested.

What else?

There are numerous classification schemes you can use for collections.  My page about FDC includes some discussion of the better-known alternatives you might want to consider.  And if you’d like to work on a new scheme without worrying about compatibility with other systems, you might be interested in the Open Shelves Classification, an ongoing project sponsored by Tim Spalding of LibraryThing.

If FDC sounds like something you’d like to use, adapt, or experiment with, go right ahead.  There’s no need to wait for any further releases, or to ask me about what you want to do, unless you’d like to.  Have at it, and have fun!

May 15, 2009

Will OCLC move to a service-oriented business model for bibliographic data?

Filed under: architecture,libraries,metadata,open access,sharing — John Mark Ockerbloom @ 2:21 pm

As you may recall, there’s been a fair bit of criticism (including some of my own) against OCLC’s proposed records use policy, which asserts OCLC’s rights over libraries’ cataloging in ways that would make it very difficult for libraries and other organizations to make innovative uses of shared bibliographic data, if they use OCLC for their shared cataloging.

In response to criticism, OCLC convened a review board to recommend principles for a revised policy.  Now Peter Murray tells us that the board will present their recommendations at Monday’s OCLC Member’s Council meeting.  I’m told presentation materials should eventually be posted publicly.

I’m looking forward to seeing what they recommend.   By now, large library organizations like the Association of Research Libraries and the International Coalition of Library Consortia have publicly criticized the policy as proposed, and the process through which it was developed.  I was gratified to see ICOLC agree that “the policy hinders rather than encourages innovation”.  And I hope that OCLC will adopt a policy that encourages sharing and innovative applications of bibliographic data, while maintaining a healthy, sustainable shared cataloging base.

As I mentioned at my talk at ALA in January, one of the ways they might do this is to move to a business model that’s based on charging for services  rather than for bibliographic data.  OCLC already offers a number of paid services, such as WorldCat Local.  Recently, they’ve announced an ambitious suite of “Web-scale management services” , effectively offering a shared ILS service, using the shared bibliographic records that have been contributed to WorldCat.  The announced services could be attractive to many libraries.   They could introduce economies of scale, by having a single ILS installation serving many institutions.  And the shared records in the service could promote new efficiencies in areas like acquisitions, collection development, and inter-library loan, as well as in information discovery.  OCLC’s new paid services, then, could advance the interest both of libraries and OCLC’s shared catalog.  They could raise the bar for competition in ILS-related services.

Or, at least, they could if everyone has the same opportunity to compete.  For lots of reasons, different libraries may choose to use different ILS vendors and products, or even develop something on their own.  (For instance, the library where I work uses Ex Libris’ Voyager ILS, and is also involved in the open source OLE Project.)  Many of these libraries have also been adding and enhancing bibliographic records in OCLC’s WorldCat.  If OCLC keeps these records locked up, so that its ILS can use them but competitors can’t, these contributing libraries will be giving a special advantage to one particular ILS by sharing their records through WorldCat.  And that skews the competition in ways that, like monopolies in general, can be bad for libraries and for innovation.

So I think OCLC needs to make a choice here:   Will they try to retain a proprietary interest in libraries’ shared cataloging, and stay neutral towards other entities (such as ILS vendors) that provide services on this cataloging?  Or will they move into competing with others to provide services on our collective data, while relinquishing their proprietary claims on it?  Though I’ve argued that we’re best off if they move to the latter, I’ve heard some thoughtful, conservative arguments for sticking with the former while they try to figure out how to sustain their shared catalog.

But they shouldn’t try to have it both ways. As we saw recently with a company that does both academic publishing and commercial PR, combining two valid but disparate interests can easily create worrisome conflicts.  As I’ve heard it, OCLC justifies whatever proprietary rights they claim on shared cataloging data on behalf of the cataloguing “cooperative”: the libraries that contribute the data.  The work of this cooperative should not be held captive in the interests of any single commercial venture  (whether OCLC’s or anyone else’s) that wants to exploit it.

With open bibliographic data, OCLC can still enjoy important competitive advantages in their ILS and other services, simply by being the entity that coordinates WorldCat, and that has the deepest knowledge about how to work with the WorldCat knowledge base. There are a number of possible ways for OCLC to support itself, and encourage data contributions, in its cataloging coordination and catalog-based services.   I hope that, in the upcoming Member’s Council meeting and in ensuing discussions, we’ll have some thoughtful conversations and proposals on how to do this with open data.

May 8, 2009

What you’re asked to give away

Filed under: copyright,crimes and misdemeanors,open access,publishing,serials — John Mark Ockerbloom @ 9:54 pm

If you’ve published an article in an Elsevier journal, you might have missed an interesting aspect of the contract you signed with them to get published.  It goes something like this:

I grant Elsevier the exclusive right to select and reproduce any portions they choose from my research article to market drugs, medical devices, or any other commercial product, regardless of whether I approve of the product or the marketing.

What, you don’t remember agreeing to that?  Actually, the words above are mine.  But while it isn’t explicitly stated in author agreements, Elsevier authors usually grant that right implicitly. Elsevier’s typical author agreement requires you to sign over your entire copyright to them. Why ask for the whole copyright, instead of just, say, first serial rights,  and whatever else suffices for them to include the article in their journal and article databases?  Elsevier explains:

Elsevier wants to ensure that it has the exclusive distribution rights for all media. Copyright transfer eliminates any ambiguity or uncertainty about Elsevier’s ability to distribute, sub-license and protect the article from unauthorized copying or alteration.

That “unauthorized” would be “unauthorized by them”.   Not “unauthorized by you”.  Once you sign, you’ve given up the right to authorize copying or alteration, or any other rights in the copyright, except for rights they offer back to you.  For instance, you can’t “sub-license” your article for anything Elsevier deems “commercial purposes”.  But they can, and do.

And sometimes those commercial purposes have had questionable ethics.  The Scientist reported about a week ago that “Merck published [a] fake journal” with Elsevier.  (Free registration may be required to read the article.)  As they report:

Merck paid an undisclosed sum to Elsevier to produce several volumes of a publication that had the look of a peer-reviewed medical journal, but contained only reprinted or summarized articles–most of which presented data favorable to Merck products–that appeared to act solely as marketing tools with no disclosure of company sponsorship.

The publication, Australasian Journal of Bone and Joint Medicine, was published by an Elsevier subsidiary called Excerpta Medica.  As that subsidiary explains on their web site, “We partner with our clients in the pharmaceutical and biotech communities to educate the global health care community and enable them to make well-informed decisions regarding treatment options.”  In other words, they’re a PR agency for drug companies and other companies selling medical products.  Part of what they do is publish various periodicals designed to promote their clients.

Now, a number of companies publish sponsored magazines, and usually such publications clearly disclose their sponsorship, or are otherwise easily recognizable as “throwaway” commercial journals.  But this publication was designed to look more like a peer-reviewed scientific journal.   The Scientist reports this court testimony from a medical journal editor:

An “average reader” (presumably a doctor) could easily mistake the publication for a “genuine” peer reviewed medical journal, [George Jelinek] said in his testimony. “Only close inspection of the journals, along with knowledge of medical journals and publishing conventions, enabled me to determine that the Journal was not, in fact, a peer reviewed medical journal, but instead a marketing publication for MSD[A].”

Indeed, one of the publication’s “honorary editors” admitted to the Scientist that it included marketing material, but that “[i]t also had papers that were excerpted from other peer-reviewed journals. I don’t think it’s fair to say it was totally a marketing journal.”  But that was what Merck paid Elsevier for, and the excerpts from real Elsevier-acquired research articles helped the publication as a whole look like disinterested scholarship instead of advertising.

Elsevier did show some embarrassment from these revelations, particularly after widespread online outrage.  A statement posted yesterday by an Elsevier spokesman admitted the journal did not have “the appropriate disclosures”, and added

I have affirmed our business practices as they relate to what defines a journal and the proper use of disclosure language with our employees to ensure this does not happen again.

That’s certainly a step up from a previous statement quoted in the Scientist article, which, after also admitting the disclosure problems in the “journal”, simply said “Elsevier’s current disclosure policies meet the rigor and requirements of the current publishing environment,” and made no promises about what they would do in the future.

But the new statement still  leaves unanswered the question of why there are still  4 “peer reviewed journals” published under the imprint of a PR agency whose stated mission is to “support our client’s marketing objectives with strategic communications solutions in [areas that include] Medical Publishing.”  And legally, Excerpta Medica still has the right to cherry-pick from any article signed over to Elsevier in any of their marketing publications.  Or, as they announce to potential clients, “we can leverage the resources of the world’s largest medical and scientific publisher.”  Even with what Elsevier considers “proper use of disclosure language”, some authors might not want their writing used in this way.

Am I being unfair to Elsevier here?  They’re not the only academic publisher that asks its authors to sign over their copyrights.  And some of the more liberal open publication licenses, which I’ve been known to recommend, are broad enough that they too give marketers rights to reuse one’s work in their promotions.

On the first of those points, I recommend in general that authors avoid signing over their rights entirely (as I’ve managed previously), no matter who the publisher is.  But last I checked, most other academic publishers don’t also own a PR firm for commercial product marketing.  (And if any do,  they should disclose this possible use in their interactions with authors. I find no explicit disclosure of this in either Elsevier’s model agreement or on the current version of Elsevier’s author rights page.)

On the second point, if you grant an open publication license, you generally know what you’re getting into.  And you can still defend against misuse of your work in ways that you can’t do if you just sign over your copyright to a publisher.   Some open access licenses, for instance, include an attribution condition that requires any reuse of the article to credit and point to the original source, and derivation conditions that either prohibit changes or require changes to be disclosed.  (And some licenses simply prohibit commercial use altogether except by permission.)  Whatever license you choose, if a company does quote your work out of context in its marketing, and you’ve kept your own rights to reprint the article, you can publish a rebuttal as widely as you like, showing the omitted context that counters a company’s claims.  These conditions and rights can provide potent deterrents against misuse of your articles.

Often the debates over scholarly author rights and open access focus on who gets to read and use scholarly articles, and what gets paid to whom.  This episode highlights another important part of the debate: who gets the right to guard the integrity of one’s scholarship.  In the light of recent revelations, authors might want to think carefully about whether to sign that right away, and to whom.

[Updates, 9 May 2009: Some spelling corrected, and a note added that disclosure is not the only potential concern of authors whose works are used for marketing purposes.]

May 5, 2009

Do we need a “reader’s rights registry”?

Filed under: censorship,copyright,metadata,online books,open access,reading,sharing — John Mark Ockerbloom @ 4:18 pm

This spring’s Digital Library Federation forum, the last under DLF as an independent organization,  included a session for “lightning talks”, where speakers take 5 minutes or less to present an idea or make a demonstration.  Here are the remarks I prepared for the one I gave, building on some recent discussion of the Google Books Settlement.  I’ve added some links to provide a bit more context to these remarks.

Like many other folks, I’m excited about the Google Books settlement, and the possibility that millions of out of print books may become available for our patrons to search and read online.  I think the services that Google proposes could greatly expand the scope and the usability of our book collections.

But I also appreciate the concern expressed over some of the details of the settlement:  in particular, the concerns over the monopolies that it creates.  Monopolies have a tendency to impose increasingly onerous terms on consumers.   They may raise prices to levels that make it difficult for libraries to fulfill their independent missions.  They may also pay insufficient attention to concerns like privacy or intellectual freedom that are important for many of our readers.

A lot of people have commented on Google’s monopoly here, but the settlement actually creates another monopoly, one that may be more problematic, and that’s the Book Rights Registry.  The name suggests a passive store of information, but in fact it’s an active agent that has the power to negotiate and set terms of access on behalf of publishers and authors for millions of out of print books.  This is the body—and the only body—that will set the default wholesale prices for copyrighted books, whether they’re provided by Google or by anyone else that might get similar rights in the future.

And the information the registry maintains might not be what’s most useful to libraries and the public.   As agents for copyright holders, they’re likely to keep very good records about who claims rights to various books, and who you should pay if you want to use them.   But they won’t have much incentive to determine, for instance, that a particular work is actually in the public domain.  As we saw in yesterday’s first panel, many works published after 1922 actually are public domain, but it’s often very difficult for someone to find this out. And I’ve already heard some folks advise publishers to claim anything they might have rights to, if there’s any uncertainty about its status.  So there needs to be some counterbalance to the claims made by commercial publishers and rightsholder organizations.

That’s why I wonder whether it might be a good idea to imagine a sort of “Reader’s Rights Registry”, that would serve as a counterbalance to the Book Rights Registry.  The reader’s registry would track information and advocate on behalf of readers and libraries.  For instance, it could compile facts related to public domain status or open content licenses, and share these facts widely with the community.  It could press Google and other vendors to keep prices affordable, and to make it easy for readers to protect their privacy.   It could track books withheld or withdrawn from full text access, for whatever reason, and work to provide access through alternative channels where appropriate.  It could press Congress or the Justice Department for reasonable and nondiscriminatory blanket licensing of orphan works and out of print works.

I’m not saying we necessarily have to create a brand new organization to do these things.   They could be done by an existing organization, or set of organizations, or a newly merged organization.  What’s; important is that we find a way to articulate what our readers need from mass-digitized book services, gather appropriate information to support these needs, and organize the resources to make that a reality.  We shouldn’t wait for someone else, whether Google or the courts, to do it for us.

Now, there’s already work underway along these lines.  Many of you may be aware, for instance, of OCLC’s Copyright Evidence Registry.   It collects facts relevant to determining whether a book is in copyright or in the public domain.  At the moment, it’s just a pilot, and the pilot runs until the end of June, with an uncertain future afterwards.  And unfortunately, its user interface is limited, and its complex and restrictive terms of use looks to me even more convoluted that what they’re proposing for WorldCat.  And I worry that those constraints will effectively choke off the project before it can get sufficient participation.  In general, I think the library community is severely handicapping itself by keeping its metadata locked up, but that’s a whole other talk.

There’s also more work to be done in digitizing copyright registrations and renewals. I’m working on some of that now, and would be glad to talk about it later on.  Google also is doing its own digitization of the complete Catalog of Copyright Entries, and plans to release it publicly. I’m looking forward to that.

Hathi Trust is also doing important work.   As we heard yesterday, they’re doing their own copyright research on their books, and I hope they’ll eventually open up their research registry for the library community to use and contribute to.  The scope of activities one might imagine for a Reader’s Rights Registry is a bit broader than what Hathi Trust is currently doing, and it would require a broader base of support that what Hathi Trust has now.  But they’ve made a very good start on many of the kinds of independent activities I think we’ll need.

I know that all our libraries are going through difficult times in this economy, and any sort of external investment or organizational overhead can be hard to justify these days.  But these mass-digitized collections could seriously reshape the nature of research and reading in the years to come.  So I think it’s worth our putting some careful thought into how we want to influence that reshaping.  Whatever happens in the courtroom in the next few months, we should marshal the information, the will, and the organization to make sure our readers will be well-served.

« Previous Page — Next Page »

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.