Git Product home page Git Product logo

openminds_generator's Introduction

WARNING: DEPRECATED REPOSITORY

openMINDS moved now to a new GitHub organization: https://github.com/openMetadataInitiative

This movement led to a couple of changes. SUMMARY OF CHANGES:

openMINDS generator logo

openMINDS_generator!

The openMINDS_generator repository is part of the open Metadata Initiative for Neuroscience Data Structures (openMINDS). It contains the openMINDS integration pipeline which builts the main openMINDS GitHub repository, the central access point to all openMINDS metadata models for all versions.

As such, it interprets and exends the openMINDS schema syntax to formal, well-known formats, such as JSON-Schema or HTML, as well as extracts the vocabulary used across all metadata models. All schemas in all supported formats and the vocabulary are stored for central maintenance on the main openMINDS repository. Note that the pipeline is configured in such a way, that each commit on one of the openMINDS submodules will trigger a new build of the main openMINDS repository ensuring that its content is always up-to-date.

The openMINDS_generator repository also hosts a small openMINDS Python library which allows you the dynamic usage of openMINDS in your Python application. Please find below a small documenation on how to install and use openMINDS Python.

For more technical details please go to the central openMINDS repository or the openMINDS Collab.

openMINDS Python logo

openMINDS Python

openMINDS Python is a small library that allows you the dynamic usage of openMINDS metadata models and schemas in your Python application for generating your own collection of openMINDS conform metadata representations (instances) as JSON-LDs.

Please note that openMINDS Python only helps you to generate correctly formatted JSON-LD metadata instances - the preparation on how you want to describe your research product with openMINDS is still up to you. If you need support in designing your own openMINDS metadata collection, check out the openMINDS Collab Tutorials which might give you hints on how to tackle your individual case or, of course, get in touch with us directly via our support-email ([email protected]).

Installation

The official versions are available at the Python Package Index and can be installed using pip install in your console:

pip install openMINDS

The latest unstable version is available on this GitHub.

Usage

As stated above, the openMINDS Python allows you the dynamic usage of openMINDS metadata models and schemas in your Python application for generating your own collection of openMINDS conform metadata representations (instances) as JSON-LDs. Here a small example:

# Import openMINDS and version manager
import openMINDS
import openMINDS.version_manager

# Initialise a local copy of openMINDS
openMINDS.version_manager.init()

# Select which version of openMINDS to use
openMINDS.version_manager.version_selection('v2.0.0')

# initiate the helper class for the dynamic usage of a specific openMINDS version
helper = openMINDS.Helper()

# initiate the collection into which you will store all metadata instances
mycollection = helper.create_collection()

# create a metadata instance for (e.g.) the openMINDS Person schema
person_open = mycollection.add_core_person(givenName="open")

# add more metadata to a created instance
mycollection.get(person_open).familyName = "MINDS"

# add connections to other metadata instances
email_openminds = mycollection.add_core_contactInformation(email="[email protected]")
mycollection.get(person_open).contactInformation = email_openminds

# save your collection
mycollection.save("./myFirstOpenMINDSMetadataCollection/")

This example generates two linked JSON-LDs, one conform with the openMINDS (v3) Person schema and the other conform with the openMINDS (v3) ContactInformation schema.

To learn in general about the available openMINDS metadata models, schemas and their required or optional metadata properties, check out the openMINDS HTML views which are deployed as GitHub pages on the main openMINDS repository. You can also have a look at the full openMINDS documentation on the EBRAINS Collaboratory.

Within the openMINDS Python you can also get an overview of the requirements of a schema and all its properties by using the 'help_X' function. Here an example:

# Getting help for expected schema properties
mycollection.help_core_actors_person()

License

This work is licensed under the MIT License.

openminds_generator's People

Contributors

skoehnen avatar olinux avatar ulrikes91 avatar apdavison avatar

Stargazers

Leon Martin avatar  avatar  avatar  avatar

Watchers

Gael Varoquaux avatar  avatar  avatar  avatar James Cloos avatar  avatar Tom Gillespie avatar Benjamin Weyers avatar  avatar  avatar

openminds_generator's Issues

vocab extraction seems to have hickups

Going through the vocab (properties.json) I realized some weird things in the vocab extraction:

  • one time a property was shown as "deprecated": true but it should have been false
  • one time a property had "deprecated": "description of the property" + "description": null

I will have a look at the script and see if I can find where such errors may occur from.
If I find something I'll provide more details.

Feedback for HTML

First @olinux thanks for setting up the HTMLs for the documentation so quickly 🙂
I had now the time to go over the layout and wrote down some feedback:

  • we should not capitalize the header (this is my personal view, feel free to object)
  • the Name can be deleted from the table (I would focus here on what the people handle / see on the GitHubs and the Python Cli)
  • we should change Property to property name
  • we should change Description to property description
  • we should change Instruction to value instruction or instruction for value
  • we should change Content type(s) to expected value type (because the ContentType is a schema, and that will confuse people)
  • if possible, we should add to the expected value type string the format (e.g., string (format: date))
  • we should add a column for expected value number or expected value count with:
    • 0 - 1 for optional values with a single entry allowed
    • 1 for required values with a single entry allowed
    • 0 - n for optional values with multiple entries allowed
    • 1 - n for required values with multiple entries allowed
    • 0 - X for required values with X entries allowed
    • 1 - X for required values with X entries allowed
  • if possible we should maybe additionally highlight required properties (e.g., making the property name bold)
  • we should try to highlight the value types that are links to other schemas (e.g., making them bold or using a different color)
  • we should change the line below the title to:
  • we should maybe change the column order to:
    • property name
    • property description
    • expected value type
    • expected value count
    • instruction for value
  • we should modify a bit the footnote (if possible) to:
    • *This is the simplified property name - within a metadata instance of this schema (JSON-LD) the properties are extended to map to the openMINDS vocabulary namespace ("https://openminds.ebrains.eu/vocab/PROPERTY"). To learn more please go to: LINK to collab page with the JSON-LD docu

What do you think? Are these good ideas? Are they feasible?

openMINDS.version_manager.init() hangs

The code:

import openMINDS
import openMINDS.version_manager

openMINDS.version_manager.init()

The error:


Target directory will created
Cloning openMINDS to target directory
Traceback (most recent call last):
  File "test.py", line 38, in <module>
    main()
  File "test.py", line 9, in main
    openMINDS.version_manager.init()
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/openMINDS/version_manager.py", line 116, in init
    update_openMINDS(target_dir)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/openMINDS/version_manager.py", line 48, in update_openMINDS
    repo = git.Repo.clone_from(REPO_URL, target_directory, branch='documentation')
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 1148, in clone_from
    return cls._clone(git, url, to_path, GitCmdObjectDB, progress, multi_options, **kwargs)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 1086, in _clone
    finalize_process(proc, stderr=stderr)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/util.py", line 386, in finalize_process
    proc.wait(**kwargs)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/cmd.py", line 501, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --branch=documentation https://github.com/HumanBrainProject/openMINDS/ /home/maciek/.openMINDS_python
  stderr: 'Cloning into '/home/maciek/.openMINDS_python'...
POST git-upload-pack (477 bytes)
error: RPC failed; curl 16 Error in the HTTP2 framing layer
fatal: the remote end hung up unexpectedly

mycollection = helper.create_collection() only works for v3 and not v2.0.0

Only syntax error, just wanted to give you a headsup. Example on the github pages lists v2.0.0.

Error report:
File "C:\Users\eier\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3326, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 2, in
mycollection = helper.create_collection()

File "C:\Users\eier\Anaconda3\lib\site-packages\openMINDS_init_.py", line 41, in create_collection
signature, func = openMINDS.MetaSchemaCollection.build_generator(self.core.schemas[schema])

File "C:\Users\eier\Anaconda3\lib\site-packages\openMINDS\MetaSchemaCollection.py", line 73, in build_generator
d = {}

File "", line 1
def core_v2.0.0_affiliation(self, organization, ):
^
SyntaxError: invalid syntax

Unable to create a controlledTerms schema

As it is easily done with core schemas through the command mycollection.add_core_protocol for example, I am missing the functionality to do the same with controlledTerms schemas, something such as mycollection.add_controlledTerms_species. There are some core schemas that require as argument a controlledTerms schema.

extension of concept schemas not working in all cases

It seems that the pipeline is not extending the schemas properly in the cases where the concept schema is located in a different openMINDS repo.

Example:
https://github.com/HumanBrainProject/openMINDS_core/blob/v4/schemas/products/device.schema.tpl.json
-> this is the concept schema for all devices, but the Electrode (device) schema:
https://github.com/HumanBrainProject/openMINDS/blob/documentation/v3/ephys/v1/device/electrode.schema.json
-> does not actually extend the core/device concept schema as it should according to the syntax definitions:
https://github.com/HumanBrainProject/openMINDS_ephys/blob/v1/schemas/device/electrode.schema.tpl.json

@olinux @skoehnen @apdavison Did we miss anything? is this a version reference issue?

provision of openMINDS spreadsheet templates (as csv/tsv or even xls)

For openMINDS users that have no or only very little programming experience it would be beneficial to provide spreadsheet templates for the openMINDS schemas as it was done before for MINDS.

Before we implement something like this I would like to discuss, though, if would like to stick to the same format as we did before (all instances in one huge spreadsheet table) or if we would like to handle openMINDS spreadsheets the same way as we handle the openMINDS JSON-LDs (each instance is getting it's own spreadsheet/table).

What do you think?

openMINDS - JSON schema issue: deprecated property?

I've noted that there is a _deprecated property appearing in the JSON-Schema.
This seems to be build incorrectly, since it seems to be set to True for currently in use schemas...

@olinux we should have a closer look again at the target JSON-Schemas

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.