Git Product home page Git Product logo

thoth's Introduction

Thoth

Open bibliographic metadata management and dissemination system

GitHub Workflow Thoth Releases License Info

About

Thoth (/toʊt, θoʊθ/, Greek Θώθ < Coptic Ⲑⲱⲟⲩⲧ < Egyptian ḏḥwtj) is an Open Dissemination System for Open Access books. Written purely in rust, it consists of:

  • A GraphQL API, implementing a data model specifically designed for OA books
  • A REST API to export metadata in formats like ONIX, MARC, etc.
  • A WebAssembly GUI to manage metadata records.

For more information about Thoth, its data and metadata formats, and more, see the repo's wiki. You can also use GraphiQL to explore the GraphQL API (click on "Docs" at the top right), or RapiDoc to inspect the REST API.

Getting Started

Requirements

  • Rustup
  • Stable Toolchain: rustup default stable
  • wasm-pack
  • rollup
  • A PostgreSQL database (included in docker-compose.yml if ran using docker)
  • libssl-dev

Running with docker (development)

Config

git clone https://github.com/thoth-pub/thoth.git
cd thoth
cp .env.example .env  # Edit the credentials in .env

Run

make docker-dev

or

docker-compose -f docker-compose.dev.yml build
docker-compose -f docker-compose.dev.yml up

Running with docker (production)

git clone https://github.com/thoth-pub/thoth.git
cd thoth
cp .env.example .env  # Edit the credentials in .env
docker-compose up

Running with rust (cargo)

Config

git clone https://github.com/thoth-pub/thoth.git
cd thoth
cp .env.example .env  # Edit the credentials in .env

Creating Postgres DB and User

psql
psql -U postgres
CREATE ROLE thoth SUPERUSER LOGIN PASSWORD 'thoth';
CREATE DATABASE thoth WITH OWNER thoth;

Exit the psql command line with:

\q

An example of a .env file:

THOTH_GRAPHQL_API=http://localhost:8000
# THOTH_EXPORT_API is used at compile time, must be a public facing URL
THOTH_EXPORT_API=http://localhost:8181
# Authentication cookie domain
THOTH_DOMAIN=localhost
# Full postgres URL (With the role and db we created in the orevious step, it will look like this)
DATABASE_URL=postgres://thoth:thoth@localhost/thoth
# Authentication cookie secret key (can be any string really)
SECRET_KEY=we_like_s%_books_255
# Logging level
RUST_LOG=info

GraphQL API

cargo run init

Export API

cargo run start export-api

GUI

cargo run start app

Building with docker

The wasm APP needs to know the endpoint the API will be running at compile time, we must provide THOTH_API as a build argument to the docker daemon upon build:

docker build \
    --build-arg THOTH_GRAPHQL_API=https://api.thoth.pub \
    --build-arg THOTH_EXPORT_API=https://export.thoth.pub \
    . -t openbookpublishers/thoth

Acknowledgements

Thoth is being developed as part of the COPIM project, an international effort to build community-owned, open systems and infrastructures to enable Open Access book publishing to flourish. COPIM is funded by the Research England Development (RED) Fund, and Arcadia, a charitable fund of Lisbet Rausing and Peter Baldwin.

thoth's People

Contributors

brendan-oconnell avatar eshellman avatar ja573 avatar kkoci avatar rhigman avatar vincentwj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

thoth's Issues

Develop set of minimum metadata requirements

Recommendation 1. COPIM must develop two metadata requirements for OA monographs, a minimum set of metadata requirements and an enriched set. Any technical report used to build the ODS needs to include both.

Output to Library of Congress

Recommendation 6. COPIM to make contact with the Library of Congress committee to initiate a discussion about OA books

Implement chapter structure

Book chapters are to be stored as work of the work_type book-chapter, and linked to the parent work through the new (M-M) work_relation table.

Changes to the data model

work_relation table

This table allows a M-M relationship of work with itself. Its attributes are: relator_work_id (FK, UUID), related_work_id (FK, UUID) and relation_type (relation_type ENUM).

relation_type
replaces
hasTranslation
hasPart
hasChild
isReplacedBy
isTranslationOf
isPartOf
isChildOf

For each relation we create we will need to insert two rows in this table, one for the active and one for the passive relation.

active relation_type passive relation_type
replaces isReplacedBy
hasTranslation isTranslationOf
hasPart isPartOf
hasChild isChildOf

e.g., Book --- hasChild ---> Chapter and Chapter --- isChildOf ---> Book

work table

The following existing attributes must be constrained to work_type != book_chapter: edition, width, height, toc, lccn and oclc.

A new attribute page_interval needs to be added. This attribute will be constrained to work_type == book_chapter. Similar to full_title, it should be the combination of first_page and last_page (accepting both arabic and roman numerals).

Changes to the GraphQL API

  • New Work attribute relations(relationType: RelationType), which returns the relation type and the related work (querying all work_relation in which the work_id matches the relator_work_id)
  • Note which Work attributes are restricted to what workType
  • New root query books() which returns [Work] of workType: 'monograph', 'edited-book', 'textbook', and 'journal-issue'.
  • New root query chapters() which returns [Work] of workType: 'book-chapter'
  • New root query bookCount()
  • New root query chapterCount()
  • New root query bookByDoi()
  • New root query chapterByDoi()

Changes to the Export API

  • Review which specifications support chapter-level data
  • Restrict publisher-level outputs, except CSV, to books. discussion needed

Changes to the APP

  • New relation form component to add work relations by searching an existing work and selecting the relation type in a modal
  • Conditional display of constrained fields based on work_type
  • Two new widgets to dashboard to display the book and chapter counts
  • Two submenu items under Works: Books (/admin/books) and Chapters (/admin/chapters)

Connect with Metadata 2020

Recommendation 3. COPIM keeps a watching brief on this project and makes its findings available to Metadata 2020.

Output to NBK/Library Hub

Recommendation 2. COPIM WP5 needs to develop a set of formal links with OPERAS and NBK/Library Hub in order for a two-way exchange of information and metadata. This needs to include key deliverables.

Connect with library systems vendors

Recommendation 10. COPIM to develop channels to address distributors and library systems vendors

Recommendation 11. COPIM to discuss outputs of the OAPEN workshops in order to develop this area

Output to JSTOR

Recommendation 13. Some OA publishers see JSTOR as an essential part of their dissemination. Therefore, COPIM needs to engage with JSTOR

Develop set of enriched metadata requirements

Recommendation 1. COPIM must develop two metadata requirements for OA monographs, a minimum set of metadata requirements and an enriched set. Any technical report used to build the ODS needs to include both.

Connect with Lyrasis

Recommendation 12. Liaise further with LYRASIS to better understand the approach of Library Simplified and SimplyE

Connect with British Library

Recommendation 5. COPIM to liaise with the British Library regarding its metadata services and the Cataloguing-in-Publication (CIP) Programme

Connect with library discovery vendors

Recommendation 15. To conduct a number of interviews with key library discovery vendors to better understand their use of metadata in relation to that agreed by COPIM

Connect to BL

Recommendation 36. As the main supplier of data to the British Library CIP programme and to UK academic libraries, COPIM should liaise with BDS in order to understand workflows and business models

Output to OPERAS

Recommendation 2. COPIM WP5 needs to develop a set of formal links with OPERAS and NBK/Library Hub in order for a two-way exchange of information and metadata. This needs to include key deliverables.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.