Git Product home page Git Product logo

oregondigital's Introduction

OregonDigital

Circle CI Coverage Status

Overview

Current production Oregon Digital running Fedora 3 and Hydra 6.

New development is focused on Oregon Digital 2: http://github.com/OregonDigital/OD2

Connected projects:

Local Development Setup

Using Docker

Grab everything you need

# Grab the repository
git clone [email protected]:OregonDigital/oregondigital.git

# Start the whole stack - on your first run, this can take a while to download
# all the necessary images
docker-compose up -d

Making zoomable images work

Create config/app.yml with information about how to reach your local system's RAIS container; e.g.:

iip_server:
  location: http://localhost:12415/images/dzi

You may need to choose something other than localhost if you're doing a demo or running docker inside a virtual machine. The rest of the settings (port and path) should remain untouched.

Generate filler data

First, run the rake task:

docker-compose exec workers bundle exec rake filler_data

Next, log into the main application by browsing to http://localhost:3000/users/sign_in. Docker sets up the admin user as "[email protected]" with the password "admin123". Once you're signed in, visit http://localhost:3000/resque/overview and watch the queue slowly get processed. This can take a while, but once it's done you should have 3 collections of dummy data.

Testing

Easy as ./docker/test.sh! Except....

  • Want to focus tests? Just pass in a path: ./docker/test.sh spec/models.
  • If you need to ensure your environment is pristine, use the --destroy flag, which will destroy and rebuild all containers.

Updating Gems

The safest way to update gems is to destroy and rebuild the dev image, otherwise you're only updating gems within a container, and containers do not necessarily persist very long.

docker-compose stop
docker-compose rm
docker rmi oregondigital/od1-dev
docker-compose build workers

You can also run "bundle install" inside a particular container, but this should only be done if you know what you're doing. It can get really confusing when you accidentally use docker-compose run when you meant docker-compose exec. Or if you choose the wrong container name.

Various other docker stuff

This list of commands should get you moving forward with most typical development tasks:

# When you change code, if you start getting odd errors, you may have to
# restart the OD containers:
docker-compose restart web workers resquehead

# If things are still "weird", rebuild the OD containers:
docker-compose stop web workers resquehead
docker-compose rm web workers resquehead
docker-compose build web workers resquehead
docker-compose up -d

# If you need a "hard reset", nuke it!  WARNING: THIS WILL REMOVE ALL YOUR
# DEVELOPMENT-INGESTED ASSETS!
./docker/nuke.sh

# When nukes just don't destroy enough, try out a SUPERNOVA!  This will remove
# every Docker entity related to OD, which will mean re-downloading the images:
./docker/supernova.sh

# Watch the logs for everything
docker-compose logs -f

# Just watch the Rails web app logs
docker-compose logs -f web

Manual

Requires Ruby 2.0

(Mongo needs to be running, 2.4 is known to work: https://docs.mongodb.com/v2.4/tutorial/install-mongodb-on-ubuntu/ )

git clone [email protected]:OregonDigital/oregondigital.git
cd oregondigital
bundle install
rake db:migrate
git submodule init
git submodule update
rake hydra:jetty:config

On a newer install, may have to pin

Symlink media directories:

ln -s /path/to/rails/media public/media
ln -s /path/to/rails/media/thumbnails public/thumbnails

Install memcached if needed, or make sure it's running (needed for login sessions):

Start the servers:

rake jetty:start
rails server

Vagrant Setup

Requires Git, VirtualBox, and Vagrant. Also requires 3 gigs of RAM to be available for the VM which vagrant creates.

Option 1: manual submodule setup:

git clone [email protected]:OregonDigital/oregondigital.git
cd oregondigital
git submodule init
git submodule update

Option 2: automatic submodule setup:

git clone [email protected]:OregonDigital/oregondigital.git --recursive
cd oregondigital

Either way, you then tell vagrant to download and start the virtual machine:

vagrant up
vagrant ssh

After vagrant ssh you'll be logged into the VM. From there, you'll want to start the Rails server:

cd /vagrant
rails server

You can browse the app via http://localhost:3000, and check on the jetty container (which houses solr and fedora) at http://localhost:8983.

oregondigital's People

Contributors

jechols avatar wickr avatar lsat12357 avatar kestlund avatar mickeroo avatar revgum avatar decimalator avatar dependabot[bot] avatar straleyb avatar cgillen avatar luisgreg99 avatar mccallum avatar sseymore avatar

Stargazers

 avatar  avatar  avatar Derek C. Zoladz avatar Jay avatar  avatar Anna Malantonio avatar Hardy Pottinger avatar pratyush kumar avatar  avatar Rebecca Appleyard avatar Giuseppe De Santis avatar padraic avatar  avatar Brendan Quinn avatar Chris Syversen avatar Shawn Averkamp avatar  avatar Shion avatar  avatar Adam Wead avatar Rob Hilliker avatar  avatar Michael J. Giarlo avatar

Watchers

James Cloos avatar tamsin woo avatar  avatar Brian E. Davis avatar  avatar  avatar Julia Simic avatar  avatar  avatar  avatar  avatar Trevor Sandgathe avatar azle avatar Michael Boock avatar  avatar  avatar  avatar  avatar  avatar

Forkers

hz3 lpmagnuson osulp

oregondigital's Issues

Exhibits

Think about them. Maybe we want to have some.

EAD

Have an EAD conversation with Anne.

Questioning Authority

The QA implementation needs some work. We need a working default RDF search and it's not necessarily clear to me how the full record requests are intended to be used. Hopefully our needs for QA become more clear as we work on the form.

Implement JPEG == small image rule

Instead of small image viewer being used when title matches "iip", use datastream's mimetype. If jpg, it's a small image and uses the small image logic.

Indexing Controlled Vocabs/Deep Graph Nodes

We need a solid understanding of how we index vocabs and deep nodes. Need displayable strings and possibly URIs for vocab terms. Need to consider indexing up to parent for deep nodes in general and how this should be configured.

Look into persistence API

Persistence code in lib/oregon_digital/rdf/rdf_resource.rb appears to be different than the typical ActiveRecord-based persistence. We set it to not be persisted on change, whereas AR says an object is persisted if it's tied to a back-end structure at all - i.e., not new and not destroyed: http://api.rubyonrails.org/classes/ActiveRecord/Persistence.html#method-i-persisted-3F

Are we doing it our way for a specific reason? If not, I'd like to push for consistency with ActiveRecord's persistence as I believe it'll be what other devs expect.

OAI-PMH

Build OAI endpoint with OAIDC

Document Deployment Process

Document the process for deploying OregonDigital to production.

This is probably substantially a copy-paste job from existing OSULP rails apps, but I'm sure there are points requiring some discussion.

See if pyramidal tiff derivative system can easily be backed with files

With pyramidal tiffs in Fedora, we have a potentially major rendering problem.

IIP expects files on the filesystem, so we'd have to either duplicate the images or mount them somehow (FUSE or something). This would probably not work well (or at all) given how slow Fedora 3 is.

We could use a front-end that just pulls individual tiles by URL, but that doesn't really solve the problem - we still need to have those files somewhere they can be accessed fairly quickly.

If possible, we should see if our semi-generic pyramidal processor can just take a switch to store the files on the filesystem instead of Fedora.

We need to have an answer to this soon, even if we defer implementation.

Clean up Catalog Controller

Catalog controller's full of default logic and practically impossible to look at. Need to split this out into modules and add the proper solr fields.

Show View

Attributes to be shown need to be identified by a decorator, then those fields need to be iterated through with the values shown as object.resource.property.rdf_label. The field labels need to come from somewhere.

This API is up for discussion.

Look into vcr

https://github.com/vcr/vcr

This could make local testing blazing fast if vcr can work with both fedora and solr. It's only for HTTP, and I'm not sure if the protocols for Solr and Fedora use HTTP exclusively. Not going to look into this right now, but it's something I may pick up when I get annoyed at slow tests.

If we go this route, we'll want to start with the heaviest tests first just to see if it helps. I'd also recommend we not put the recordings in the repo, but I could be persuaded otherwise. My thoughts on this are that we still want travis to do full, real tests - vcr would really just be for speeding up local development in my mind, not replacing integration testing.

Do Something About Simplecov

Right now the test which spawns threads for testing NOIDs is sending out coverage information, which breaks our coverage reports.

Build UI for typeahead

Issues with UI

  • No styling so far, so suggestions look terrible - they have no background and for keyboard users there's no obvious way to see what's got focus
  • No template engine - typeahead suggests Hogan.js for simple mustache templating (I think this is sane to at least look at - it's very simple)
  • Fields with autocomplete turn "weird" - the field background changes to gray for half the field. This has to do with how the typeahead replaces styles and dom elements. Not sure how to fix just yet, but it's probably not hard.
  • Not sure about accessibility in terms of marking the autocomplete aria role and such

Look into rdf-redis gem

Looks like this has a bug. When using RDF::Repository:

[1] pry(main)> OregonDigital::ControlledVocabularies::Subject.new "http://id.loc.gov/authorities/subjects/sh85050282"
=> #<OregonDigital::ControlledVocabularies::Subject:0x3ffd2ac(default)>

When using RDF::Redis:

[1] pry(main)> OregonDigital::ControlledVocabularies::Subject.new "http://id.loc.gov/authorities/subjects/sh85050282"

NoMethodError: undefined method `keys' for "":String
from /home/jeremy/.rvm/gems/ruby-1.9.3-p194@oregondigital/bundler/gems/rdf-redis-b9149ebded9c/lib/rdf/redis.rb:88:in `statement_keys'

URL Transition Work

Do that anxiety inducing thing with reverse proxy to get us concurrent Hydra and CDM systems on OregonDigital.org

List Terms for Controlled Vocabularies

There's not currently a way to list the terms in a particular vocabulary. This would be nice to have for generating drop down menus. This might involve a minor patch to RDF::Vocabulary and/or RDF::StrictVocabulary. I've delayed on it while I think about whether that's the correct approach.

Namespace-ify datastreams in app/models

Rails mostly seems okay with app/models/datastreams without any namespacing, but it's kind of confusing given the typical layout of these sorts of things, and may be contributing to Travis weirdness.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.