Git Product home page Git Product logo

caltechlibrary / dibs Goto Github PK

View Code? Open in Web Editor NEW
27.0 7.0 6.0 37.23 MB

DIBS is an implementation of a basic controlled digital lending (CDL) system using IIIF to make scanned books available for time-limited viewing.

Home Page: https://caltechlibrary.github.io/dibs

License: Other

Python 3.60% Smarty 1.20% HTML 0.08% CSS 42.56% Shell 0.01% JavaScript 52.41% Makefile 0.14%
caltech-library controlled-digital-lending library-management python iiif library-systems folio folio-lsp tind-lsp tind

dibs's Introduction

Caltech DIBS

Caltech DIBS ("Digital Borrowing System") is the Caltech Library's implementation of a basic, standalone, controlled digital lending system.

License Python Latest release

Table of contents

Introduction

DIBS ("Digital Borrowing System") is a web-based system that enables users to borrow, in a time-limited fashion, scanned materials that are not otherwise available in electronic format. The concept of controlled digital lending (CDL) is to provide the digital equivalent of traditional library lending. Libraries digitize a physical item from their collection, then lend out a secured digital version to one user at a time while the original, printed copy is simultaneously marked as unavailable. The number of digital copies of an item allowed to be loaned at any given time is strictly controlled to match the number of physical print copies taken off the shelves, to ensure an exact "owned-to-loaned" ratio.

DIBS was developed in the year 2021 to help Caltech students and faculty continue their studies and work during the global COVID-19 pandemic. The open-source system provides three main components for CDL: a loan tracking system, an integrated digital content viewing interface, and basic administrative interface. DIBS embeds the Universal Viewer to display materials that comply with the International Image Interoperability Framework (IIIF). To help library staff convert scans of materials into IIIF format, DIBS also provides a way to connect to an external workflow, and Caltech's workflow automation for this purpose is available as an example in a separate repository.

DIBS can currently interface to either FOLIO or TIND to get metadata about books, but its interface layer should be straightforward to extend for other LSP (library services platform) implementations.


Requirements

The core DIBS server is written in Python 3 and makes use of some additional Python software libraries that are installed automatically during the installation step steps.

Requirements to run a demo

Although DIBS relies on the existence of a IIIF image server (and content to serve), for initial exploration and demonstration purposes, you don't need to set up a IIIF server; you can reference content located in any of a number of publicly-accessible IIIF servers around the world, and DIBS includes a sample IIIF manifest to illustrate that. You shouldn't need anything else to run the DIBS demo on your local computer.

Requirements to use DIBS for real use at another institution

In order to run a multiuser DIBS server at another institution, certain additional things are needed.

  1. A IIIF server. At Caltech, we use a serverless component running on an Amazon cloud instance, but many other IIIF server options exist. If you're looking at DIBS, presumably it means you want to serve content that is not freely available in a public IIIF server, which means you will need to set up a server of your own.
  2. A web server to host DIBS. The current version of DIBS has only been tested with Apache2 on Linux (specifically, Ubuntu 20) and macOS (specifically 10.14, Mojave). DIBS comes with a WSGI adapter file and sample config file for Apache, but it should be possible to run DIBS in other WSGI-compliant servers.
  3. An authentication layer. DIBS assumes that the web server takes care of user authentication in such a way that DIBS is behind the authentication layer and all users who can reach DIBS pages are allowed to view content. DIBS itself only implements checks to distinguish between regular users versus staff who are allowed to access restricted pages. For the authentication layer, at Caltech we use the Shibboleth single sign-on system, but it is possible to use other schemes, including Apache Basic Authentication. The installation and configuration of a single sign-on system depends on the specifics of a given institution, and are not described here.
  4. The use of FOLIO LSP or TIND ILS for retrieving metadata based on barcodes or unique identifiers, or a willingness to extend the existing metadata retrieval layer in DIBS. The interface for adding items to DIBS requires looking up metadata based on an item's barcode, and lacking a universal scheme to do that, we had to write our own interface layer. The use of another LSP will require extending this interface layer. (Thankfully, the code is short and the amount of metadata required is small.)
  5. Modification to the HTML templates to change the branding. The template files in dibs/templates are specific to Caltech, and will need to be edited to suit another installation. (We are open to making the branding customization easier and would welcome a pull request for suitable changes!)

Installation and operation

Please refer to the detailed installation instructions in the DIBS manual.

General information

Documentation for DIBS is available online at https://caltechlibrary.github.io/dibs/.

Known issues and limitations

DIBS is in active development. The current version was produced rapidly, and has a limited scope and streamlined design. We continue to improve DIBS in various ways.

It is worth mentioning that DIBS does not (currently) implement a queue or wait list for loan requests. This is a conscious design decision. Queuing systems tend to lead to complexity quickly, and we want to delay implementing a queue until it becomes clear that it's really essential. (After all, in a physical library, there are no queues for borrowing books: you go to see if it's available, and if it's not, you can't borrow it.) Perhaps we can implement interfaces and behaviors in DIBS that avoid the need for a queue at all!

The DIBS server acts as an intermediary between the IIIF server and patrons viewing content in IIIF viewers – all content goes through DIBS. This is how DIBS can implement loan policies and secure content management: it's a choke point. However, it means the DIBS server is a potential performance bottleneck. At our institution, we have not found the speed impact to be objectionable for CDL in an academic setting, even using single server hardware. But other sites may have different experiences. If you experience performance issues, first try to increase the number of parallel threads that your Apache server will use for DIBS, and also increase IIIF_CACHE_SIZE in settings.ini. If that is not enough, let the developers know, and we can start thinking about architectural changes.

Getting help and support

If you find an issue, please submit it in the GitHub issue tracker for this repository.

Contributing

We would be happy to receive your help and participation with enhancing DIBS! Please visit the guidelines for contributing for some tips on getting started.

License

Software produced by the Caltech Library is Copyright (C) 2021–2022, Caltech. This software is freely distributed under a BSD-type license. Please see the LICENSE file for more information.

This software repository includes a copy of the Universal Viewer version 3.1.1, obtained from the GitHub repository as it existed on 2021-04-27. The Universal Viewer is released under the MIT license. Please see the Universal Viewer website and documentation for more information about any applicable copyrights or licensing terms.

Authors and history

DIBS was designed and implemented by Michael Hucka, Robert Doiel, Tommy Keswick and Stephen Davison of the Caltech Library's Digital Library Development team.

Acknowledgments

This work was funded by the California Institute of Technology Library.

The vector artwork of a book with a clock on it, used as the icon for this project, was created by Royyan Wijaya from the Noun Project. It is licensed under the Creative Commons CC-BY 3.0 license. DIBS also uses other icons created by Scott Desmond and SlideGenius.

We gratefully acknowledge bug reports and fixes submitted by the following users, who helped improve DIBS:


@dlasusa

@stanonik

@hadro

@phette23

DIBS makes use of numerous open-source packages, without which it would have been effectively impossible to develop DIBS with the resources we had. We want to acknowledge this debt. In alphabetical order, the packages are:

  • Arrow – a library for creating & manipulating dates
  • Bootstrap – CSS framework for developing responsive websites
  • Bootstrap Table – extended table framework for Bootstrap
  • Boltons – package of miscellaneous Python utilities
  • Bottle – a lightweight WSGI micro web framework for Python
  • Bottle-fdsend – Bottle plugin to enable sending content from in-memory files
  • Clipboard.js – JavaScript library for copying text to the clipboard
  • Coif – a Python package for finding book cover images
  • CommonPy – a collection of commonly-useful Python functions
  • expiringdict – an ordered dictionary class with auto-expiring values
  • Font Awesome – scalable vector icons for web design
  • humanize – make numbers more easily readable by humans
  • ipdb – the IPython debugger
  • jQuery – JavaScript library of common functions
  • LRU Dict – an implementation of Least Recently Used caching
  • mod_wsgi – an Apache module for hosting Python WSGI web applications
  • Peewee – a simple ORM for Python
  • Pillow – a fork of the Python Imaging Library
  • Plac – a command line argument parser
  • Pokapi – a simple Python OKAPI interface
  • Popper.js – web page tooltip engine
  • Python Decouple – a high-level configuration file interface
  • Rich – library for writing styled text to the terminal
  • Sidetrack – simple debug logging/tracing package
  • str2bool – convert a string to a Boolean value
  • Trinomial – a simple name anonymization package
  • Topi – a simple package for getting data from a TIND.io ILS instance
  • Universal Viewer – a browser-based viewer for content in IIIF format
  • Yurl – URL manipulation library
  • Werkzeug – WSGI application library

dibs's People

Contributors

dependabot[bot] avatar hadro avatar mhucka avatar phette23 avatar rsdoiel avatar t4k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

dibs's Issues

Path to uv.js is not prefixed

The bottom of uv.tpl has

<script src="uv/uv.js"></script>

whereas that should be

<script src="{{base_url}}/viewer/uv/uv.js"></script>

Change the "remove" button

Initial user feedback suggests that we change the "remove" button in the /list view to be something else. Perhaps the functionality should be named "suppress" or something like that.

Viewer page fails to work for some users

As reported internally at Caltech, some people experience a situation with the viewer page where the page keeps reloading rapidly and never showing the content.

Null users are not checked for

This may only happen in debug mode, but if REMOTE_USER is not set, server.py doesn't check for None and some things fail.

Refactor Person and GuestPerson classes

The structure of Person and GuestPerson behaves like GuestPerson is a subclass of Person. The Person class should be the parent and not sited to a specific database presentation or storage. A subclass would then be used for stored Person objects via the ORM. This will allow us to store a minimal class of users while allowing general access through the fact of having a Shibboleth/Basic Auth managed account.

Person class when instantiated holds a default user description with the uname assigned on instantiation (e.g. the value that Shibboleth passes as REMOTE_USER). It should contain the following attributes - uname, display_name, role. The person class would define a standard set of methods like has_role() these would return results based on the attributes held in the object instance.

A generalize function like person_from_environment() could then return a Person compatible object based on either the default Person class or the subclass where the attributes are populated via the ORM.

Ideally the server code should think all Person objects are the same but behind the scenes we only store rows in the SQL person table for specific people who require more than the minimum access.

The eventual goal is for people.py and people-manager to become a common package use across applications that we build for apps.library.caltech.edu.

stats: malfunction in average loan duration sort

clicking the average loan duration column does not result in a meaningful sort by that column at all; when the page auto-reloads the order changes immediately to something else that is meaningless

Viewer needs to check actively for expired loans

If you have 2 windows open on a loan item, and end the loan early in one window, the other window will still show the viewer but with none of the content accessible (you get spinning squares in the thumbnails). This happens because the viewer page only has a (self-contained) mechanism for time-based expiration and does not actively check if something else ended the loan. This needs to be fixed.

Mobile viewer configuration

The mobile UV viewer is missing some features that are available on desktop:

  • No reservation timer
  • No end loan button
  • No contents panel
  • Non-functional bookmark button - see screenshot (not present in desktop)

IMG_3313

Switch to multithreaded server backend

Currently we're using the single-threaded server option in Bottle. We should switch servers. This may have implications for software development, if we discover issues due to multithreaded operations on the database.

people-manager setting password

In current version testing on dibs on apps.caltechlibrary.dev the people-manager shows the WARNING if you try to use secrets. The WARNING message should be "secret" not "secrets" or more specific about Basic Auth and htpasswd not being enabled. Second, had to use htpasswd to reset my password to finish testing. The database scheme needs to be updated so a warning about that would be have been helpful too.

Item page is still getting cached

When you visit an /item page, you can still see old content. Not sure what the circumstances are, but the current browser-cache-prevention isn't working well enough.

Poor screen size handling of item page

The text and elements on the item info page (nor any other page) does not resize well for smaller screens and mobile devices. This is independent of the UV widget and issue #60.

Cookie warning about SameSite being None

The bottle session plugin uses Cookies. In addition to setting the cookie name and duration we also should set the cookie_secure status. False should set hte SameSite status to Lax and True should set it to Secure based on the Python 3.8 http.cookie support.

This can be rolled out in the next refresh for dibs.caltechlibrary.dev after we get some feed back.

Make wait time configurable

The time patrons must wait before being able to re-loan the same title needs to be adjusted lower, per initial user feedback. At this point, I think it makes sense to make the value configurable in the settings file, on the theory that different sites that use DIBS might want to use different values.

run-server should permit defining a user in other modes

Currently, run-server only supports setting a REMOTE_USER in debug mode. However, during development, you really do want to be able to set the user to test adapter.wsgi too, but the latter is only used when debug mode is not in effect (i.e., when -m is either verbose or normal).

Now, it does turn out to be possible to set REMOTE_USER using --setenv when invoking mod_wsgi-express. However, this would weaken the security intentions; we originally wanted to have 2 things preventing someone from overriding security (use debug mode and use the -u flag), and so disconnecting -u from the mode would allow someone to use -u in any run mode. OTOH, we tell people not to use run-server in production, so maybe this doesn't matter?

Bug: next time "unknown"

In some situation it seems the item page will display a message about "the item will be available at at unknown".

Mobile viewer features

On mobile the viewer is extends past the window, which makes it hard to scroll through pages. The viewer size should reflect the phone screen size.

Add a live statistics page

We should move the "copies in use" column off the /list page and have a separate page for stats where the info is updated automatically.

list sticky footer not behaving well

  • footer doesn't stay stuck to the bottom when many items exist and page is scrolled
  • when the height of the window and number of items is just right the add a new item button is blocked

Add announcements mechanism

If we need to do something on the server, we should announce it ahead of time. There should be a page or other way to display that.

As @rsdoiel wrote in a Slack conversation recently:

1. It'd be good to include an option to do a maintenance page while longer maintenance is happening
2. It'd be nice to be able to insert an "announcement" at the top of pages to let people know if something was coming up like, we'll be down for five minutes at 2:00pm for system updates.

Also from Robert:

On many of our systems you create an annoucement document and if it exists it's included in the template. For lone downtimes I think we set a boolean in the settings.ini so we display the page for all URLs request until we're back up. If maintence == true then redirect to maintenance page, continue as normal or something.

Make the template files more configurable for other institutions

There are a lot of references to Caltech all over the HTML template files. At some point (possibly with input from another interested potential user group) we should make the templates configurable. At the very least, we could replace "Caltech" with a variable reference.

Allow alternative ways of setting configuration variables

Currently, we rely on a settings.ini file to set the values of certain configuration variables. The decouple package supports putting values into a .env file and reading values from the run-time Unix process environment (as described here, but we may want a more direct way to change values, perhaps via command-line arguments. In any case, this topic should be revisited at some point.

Split out TIND code into its own Python package

I'm thinking it would make sense to pull the tind code out into a separate Python package. Wouldn't take too much time and it would be conceptually cleaner, since there's no good reason to have tind-specific code in dibs per se.

people-manager problems with paths and command output

The code in people.py can pick up the wrong copies of htpasswd and the password file. It should probably have full paths to those things. Also, the way it checks for output from htpasswd can mislead it into think there is an error when there is not.

Improve: list page currently only updates on refresh

The current simple way of handling the /list page leaves the values unchanged even if loans are taken out while a user is looking at the list. We should have a way to update the "Copies in use" column dynamically.

Solve potential bibliographic data update issues in database.py

Currently, the implementation of Item in database.py includes fields for metadata like title, year, etc. These metadata fields are populated from the LSP/ILS at the time the Item object is created. The fields are used to fill out the /item page. Unfortunately, keeping this data cached introduces the problem that the metadata may be updated in the LSP but never be updated in the Item object kept in the database.

Getting the data dynamically every time /item pages are generated is not an option due to obvious performance reasons, but we should find a way to (e.g.) refresh the data, or otherwise avoid potential data inconsistencies.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.