Git Product home page Git Product logo

lidia-zotero's Introduction

LIDIA Annotations

Zotero plugin for capturing PDF reader annotations in a structured format for the LIDIA project.

The project is set up as a bootstrapped plugin for Zotero 6 and 7. Tested only under Zotero 6.

About

This extension is used to create a database of linguistic diagnostic arguments (hence LiDiA). Users can add linguistic academic articles or books to a shared Zotero folder, create annotations using the PDF reader and use the extension to describe the diagnostic arguments they find in the text. The data is saved as Zotero annotation comments in YAML format. The data will then be read out in a separate web application [1] in order to create a database for the sake of the general community of linguists. The project, together with this Zotero extension, is being documented at [2].

This project is still under active development and is not ready for production use. The plugin was specifically created for use with the LIDIA project at Utrecht University, but the general principle of this Zotero extension may be applicable to other academic projects as well.

Feedback is welcome! You may create an issue or contact one of the developers.

[1] https://github.com/UUDigitalHumanitieslab/lidia [2] https://lidia.readthedocs.io/

Development

Make Zotero ready for the extension (you only need to do this once):

  • Create a new Zotero profile with zotero --ProfileManager (here named Develop).
  • Configure a custom data directory for the new profile (in this example we use ~/.zotero/zotero/Develop).
  • Start the new profile: zotero -P Develop and exit.
  • git clone [email protected]:UUDigitalHumanitieslab/lidia-zotero.git /path/to/Projects/LIDIA/lidia-zotero
  • mkdir $HOME/.zotero/zotero/Develop/extensions
  • echo "<absolute-path-to-source>/build" > $HOME/.zotero/zotero/Develop/extensions/[email protected]
  • sed -i '/.*extensions\.lastAppBuildId.*/d' $HOME/.zotero/zotero/Develop/prefs.js
  • sed -i '/.*extensions\.lastAppVersion.*/d' $HOME/.zotero/zotero/Develop/prefs.js

This will enable the plugin from the source directory.

Install npm packages:

  • npm install

Build and run:

  • Make changes in your source
  • npm run build
  • zotero -P Develop -purgecaches -ZoteroDebugText

Use of React

This extension uses React for rendering part of the UI, as was done earlier in Diego de la Hera's plugin zotero-cita. To make that possible we include a patch to react-dom that is included in zotero-cita as well. This patch forces the use of the XHTML namespace when creating elements, because Zotero 6 by default creates XUL elements.

React also needs to have access to global objects like window and document. This is achieved by passing the window object of the Zotero main window to the bundled JavaScript file when it is loaded in bootstrap.js using the Services.scriptloader.loadSubScript method.

Release

Releases and local builds are made using zotero-plugin.

To create a local build, run npm run build.

To create a release:

  • Switch to main branch
  • Bump the version in package.json (e.g. 0.2.1)
  • Create a signed/annotated tag with the same version (e.g. git tag --sign -m "Release v0.2.1" v0.2.1)
  • git push --follow-tags

Reading

lidia-zotero's People

Contributors

ar-jan avatar tijmenbaarda avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

lidia-zotero's Issues

Add LIDIA after startup in Zotero 7

If the LIDIA panel is built directly after startup, this fails in Zotero 7 because the window is not yet ready.

The line where it fails is:
const tabbox = tabContainer.querySelector("tabbox");

Fix this, perhaps by waiting for a certain amount of time if it fails and try again?

Implement automatic saving or a warning when switching to another annotation

Currently annotations are not automatically saved. When switching to another annotation without explicitly clicking the save button, the changes are discarded. This is not userfriendly and counter-intuitive.

Saving each time the user presses a key is probably too expensive or it may cause synchronization issues, but possible solutions are:

  • Saving each time a user selects a different field
  • Saving every couple of seconds
  • Saving just before activating a different annotation
  • Warning the user before activating a different annotation if there are unsaved changes

In case of automatic saving, it is probably necessary to give the user feedback about the saving status, to avoid data loss if the user exits the application.

Give all annotations a LIDIA-specific annotation ID

Currently we use the Zotero key for cross-references to annotations, but this is problematic because Zotero assigns a different ID when moving items from the local library to a group library. We can solve this by giving all LIDIA annotations a LIDIA-specific unique annotation ID. We may use the uuid npm package for unique IDs.

Since users have already been adding cross-references, we need a transition mechanism.

Use annotation id for relationships

Relations to other arguments currently only store the annotation's zoteroKey which identifies the publiation or pdf, not the annotation.

Figure out if there's a global id we can use (shared also when syncing via groups), or if it should be zoteroKey + argument name/id.

UI tweaks

Small changes needed in the UI that can quickly be implemented:

  • Lack of space after Page start / Page end (as noted by @bbonf)
  • Inconsistent aligning of terms and lack of colons

Make a separate tab and panel for every PDF document

Needed for Zotero 7.

Currently the LIDIA tab (= the tab button itself) and the LIDIA panel are reused for all documents, but this is not how it is supposed to be: every document should have their own. This would solve the following issues:

  • Right now we force-activate the LIDIA tab every time the user switches documents, but this does not work on Zotero 7 for some reason. When this activation does not happen, the tab button remains selected (because it is the same XUL element, which is still active), but the panel does not show.
  • Making separate panels for each document allows the user to keep working in an annotation when looking something up in a different document

Make argument text selectable and copyable

It should be possible to select and copy (part of) the argument text (i.e. the annotation's selected text), either in the annotations sidebar or the annotation form on the right.

Relations to other diagnostic arguments

A field prepopulated with annotation-ids (labeled with argument name) from the Zotero group library, and an option to specify the type of relation (confirms, contradicts, etc.; Jan will provide a list).

Option to configure default values per article

Argument language and linguistic level (subfield) should receive default values per article.

We'll probably need to store this in the Zotero item. Some plugins use key: value in Zotero's extra field, maybe we can use that and provide it to the plugin?

Correctly handle deletion of annotations in case they are linked to

At the moment the behaviour is not well managed in case an annotation is deleted that:

  • Is linked to from a different annotation
  • Is continued in a different annotation

Especially the second case may lead to confusion, because automatically the annotation before the deleted one will be marked as the continued annotation.

Check if it is possible to act on deletion of an annotation, and if not, handle it correctly when a form is opened.

Datamodel cleanup

Outfactor everything that belongs to the data model to a separate class

Update item pane for Zotero 7

A recent Zotero 7 beta release removed item pane tabs, so our annotation form no longer shows up. We'll need to wait for the new API to add the form to the new vertical sections instead:

item-pane tabs have been replaced with collapsible vertical sections. We'll be introducing a new API soon that lets plugins create sections there. You'll need an icon that will appear for your section in the navigation bar on the side.
https://groups.google.com/g/zotero-dev/c/A-_e78q_W5o

Documentation

Notes

  • Installing: Zotero: Tools > Addons > Install Add-on From File.
  • If you had PDFs open in Zotero, reopen them to allow loading the LIDIA extension.
  • Toggle both the left and right sidebars (left: annotations, right: contains a tab LIDIA with the form).
    (Add screenshot showing the buttons and example annotation list + form?)
  • To activate the LIDIA form for an annotation, click the annotation in the left sidebar.
  • Converting PDF annotations to Zotero annotations: ...

Group use (later):

  • If you add a publication from your personal library to a group library, the attachment item (pdf) will also be the same (same key). If you want to have the publication separately in your own library (e.g. with your own annotations separate from LIDIA's), it may be easiest to create a duplicate of the publication and attachment.
  • If you select an annotation made by someone else in the group, you will see the form, but cannot edit it. #15

Display vocabulary term URL

Display a link to the entry for vocabulary terms (Lexicon of Linguistics term, Clarin concept registry item from a subset of included concepts). Low priority.

Using React component, JSX, NPM packages with esbuild

Using React component in Zotero Plugin?

Minimal example.

Using JSX

text: The esbuild loader for this file is currently set to "js" but it must be set to "jsx" to be able to parse JSX syntax. You can use "loader: { '.js': 'jsx' }" to do that.

In build.mjs, add: loader: { '.js': 'jsx' },

Using NPM packages

Some Zotero plugin authors are using require and note that in that case the module should be declared as external in order to be left alone by esbuild.

I'm using esbuild these days, and I have found you can declare modules as external, and it will just leave those require's as-is. src

Test versioned releases and update manifest

  • Create .xpi addon upon releasing a new version
  • Update the update manifest
    • Place in public location
    • Test

Creating the xpi and update manifest can probably be done via Github Actions workflow. Not sure if there's a way to upload it to dhstatic though, maybe we should just make the repo public and use Github for all of it.

Argument continuation

A boolean field which indicates that a text selection belongs to the annotation started on a preceding page.

If checked, the rest of the form should be disabled.

Import Acrobat annotations in Zotero

External annotations are read-only by default — indicated by a lock icon — but you can transfer them into Zotero by selecting File → “Import Annotations…” from within the PDF reader, after which they'll be fully editable. The annotations are removed from the PDF to avoid conflicts and duplicates.
https://www.zotero.org/support/kb/annotations_in_database

Make sure we use copies of the annotated PDFs in case someting happens to the Zotero annotations.

Try to also import the annotation text into the short description field.

Revise annotation text format

We currently use key = value as the text format. We picked this since it's a bit easier to read/edit by hand, but we could consider using JSON or YAML. Especially now that we probably need to be able to store multiple subfield+term combinations per argument.

Annotation panel not enabled

For a user the plugin appears to be installed correctly, but the Lidia panel remains disabled when activating an annotation. Possibly related to the PDF being synchronized via Zotero.

[JavaScript Error: "Error: getAnnotations() can only be called on file attachments" {file: "chrome://zotero/content/xpcom/data/item.js" line: 3945}] [JavaScript Error: "Error: getAnnotations() can only be called on file attachments" {file: "chrome://zotero/content/xpcom/data/item.js" line: 3945}]

Annotation fields

  • Add article term: subfield + lexicon term
  • Add custom article term: subfield + custom term
  • Ability to add multiple combinations, for both article term and lexicon terms.
  • Argument name should become the human readable label for the annotation ID. Also display the annotation ID in the form?

Run example plugin from source directory

Following Setting Up a Plugin Development Environment:

  • Create a new Zotero profile via zotero --ProfileManager(here: "Develop").
  • Configure a custom data directory for the new profile.
  • Start the new profile: zotero -P Develop; quit.
  • git clone [email protected]:UUDigitalHumanitieslab/lidia-zotero.git /path/to/Projects/LIDIA/lidia-zotero
  • git checkout feature/example-make-it-red # src-1.2 = Bootstrapped plugin for Zotero 6 and 7
  • mkdir $HOME/.zotero/zotero/Develop/extensions
  • echo "/absolute/path/to/Projects/LIDIA/lidia-zotero" > $HOME/.zotero/zotero/Develop/extensions/[email protected]
  • sed -i '/.*extensions\.lastAppBuildId.*/d' $HOME/.zotero/zotero/Develop/prefs.js
    sed -i '/.*extensions\.lastAppVersion.*/d' $HOME/.zotero/zotero/Develop/prefs.js
  • zotero -P Develop # The plugin is now enabled from the source directory
  • Make changes and restart Zotero...
  • zotero -P Develop -purgecaches -ZoteroDebugText -jsconsole

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.