humanbrainproject / openminds_generator Goto Github PK

• the @type requirement for embedded schemas is missing in the json-schemas

I've noted that there is a _deprecated property appearing in the JSON-Schema.
This seems to be build incorrectly, since it seems to be set to True for currently in use schemas...

@olinux we should have a closer look again at the target JSON-Schemas

First @olinux thanks for setting up the HTMLs for the documentation so quickly 🙂
I had now the time to go over the layout and wrote down some feedback:

• we should not capitalize the header (this is my personal view, feel free to object)
• the Name can be deleted from the table (I would focus here on what the people handle / see on the GitHubs and the Python Cli)
• we should change Property to property name
• we should change Description to property description
• we should change Instruction to value instruction or instruction for value
• we should change Content type(s) to expected value type (because the ContentType is a schema, and that will confuse people)
• if possible, we should add to the expected value type string the format (e.g., string (format: date))
• we should add a column for expected value number or expected value count with:
  • 0 - 1 for optional values with a single entry allowed
  • 1 for required values with a single entry allowed
  • 0 - n for optional values with multiple entries allowed
  • 1 - n for required values with multiple entries allowed
  • 0 - X for required values with X entries allowed
  • 1 - X for required values with X entries allowed
• if possible we should maybe additionally highlight required properties (e.g., making the property name bold)
• we should try to highlight the value types that are links to other schemas (e.g., making them bold or using a different color)
• we should change the line below the title to:
  • type: https://openminds.ebrains.eu/core/SCHEMANAME - view full JSON-Schema
• we should maybe change the column order to:
  • property name
  • property description
  • expected value type
  • expected value count
  • instruction for value
• we should modify a bit the footnote (if possible) to:
  • *This is the simplified property name - within a metadata instance of this schema (JSON-LD) the properties are extended to map to the openMINDS vocabulary namespace ("https://openminds.ebrains.eu/vocab/PROPERTY"). To learn more please go to: LINK to collab page with the JSON-LD docu

What do you think? Are these good ideas? Are they feasible?

Only syntax error, just wanted to give you a headsup. Example on the github pages lists v2.0.0.

Error report:
File "C:\Users\eier\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3326, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 2, in
mycollection = helper.create_collection()

File "C:\Users\eier\Anaconda3\lib\site-packages\openMINDS_init_.py", line 41, in create_collection
signature, func = openMINDS.MetaSchemaCollection.build_generator(self.core.schemas[schema])

File "C:\Users\eier\Anaconda3\lib\site-packages\openMINDS\MetaSchemaCollection.py", line 73, in build_generator
d = {}

File "", line 1
def core_v2.0.0_affiliation(self, organization, ):
^
SyntaxError: invalid syntax

It seems that the pipeline is not extending the schemas properly in the cases where the concept schema is located in a different openMINDS repo.

Example:
https://github.com/HumanBrainProject/openMINDS_core/blob/v4/schemas/products/device.schema.tpl.json
-> this is the concept schema for all devices, but the Electrode (device) schema:
https://github.com/HumanBrainProject/openMINDS/blob/documentation/v3/ephys/v1/device/electrode.schema.json
-> does not actually extend the core/device concept schema as it should according to the syntax definitions:
https://github.com/HumanBrainProject/openMINDS_ephys/blob/v1/schemas/device/electrode.schema.tpl.json

@olinux @skoehnen @apdavison Did we miss anything? is this a version reference issue?

For openMINDS users that have no or only very little programming experience it would be beneficial to provide spreadsheet templates for the openMINDS schemas as it was done before for MINDS.

Before we implement something like this I would like to discuss, though, if would like to stick to the same format as we did before (all instances in one huge spreadsheet table) or if we would like to handle openMINDS spreadsheets the same way as we handle the openMINDS JSON-LDs (each instance is getting it's own spreadsheet/table).

What do you think?

Besides "_linkedCategories", we need to add "_embeddedCategories" into the pipeline for complementing schemas. @olinux / @apdavison could you have a look at this?

On save set:

• type https://openminds.ebrains.eu/<schema-name>
• id https://localhost/<schema-name>/<random-id>

As it is easily done with core schemas through the command mycollection.add_core_protocol for example, I am missing the functionality to do the same with controlledTerms schemas, something such as mycollection.add_controlledTerms_species. There are some core schemas that require as argument a controlledTerms schema.

Going through the vocab (properties.json) I realized some weird things in the vocab extraction:

• one time a property was shown as "deprecated": true but it should have been false
• one time a property had "deprecated": "description of the property" + "description": null

I will have a look at the script and see if I can find where such errors may occur from.
If I find something I'll provide more details.

The code:

import openMINDS
import openMINDS.version_manager

openMINDS.version_manager.init()

The error:

Target directory will created
Cloning openMINDS to target directory
Traceback (most recent call last):
  File "test.py", line 38, in <module>
    main()
  File "test.py", line 9, in main
    openMINDS.version_manager.init()
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/openMINDS/version_manager.py", line 116, in init
    update_openMINDS(target_dir)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/openMINDS/version_manager.py", line 48, in update_openMINDS
    repo = git.Repo.clone_from(REPO_URL, target_directory, branch='documentation')
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 1148, in clone_from
    return cls._clone(git, url, to_path, GitCmdObjectDB, progress, multi_options, **kwargs)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/repo/base.py", line 1086, in _clone
    finalize_process(proc, stderr=stderr)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/util.py", line 386, in finalize_process
    proc.wait(**kwargs)
  File "/home/maciek/anaconda3/lib/python3.7/site-packages/git/cmd.py", line 501, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --branch=documentation https://github.com/HumanBrainProject/openMINDS/ /home/maciek/.openMINDS_python
  stderr: 'Cloning into '/home/maciek/.openMINDS_python'...
POST git-upload-pack (477 bytes)
error: RPC failed; curl 16 Error in the HTTP2 framing layer
fatal: the remote end hung up unexpectedly