skohub-io / skohub-vocabs Goto Github PK

Discovered project via signing up for SWIB20, so forgive me if am jumping ahead, but generating the static site is exactly what we need for a project. I've sort of got it working (tree looks amazing) but I'm obviously failing to get it in quite the right format/sematics. As a sample I have:

<https://data.vam.ac.uk/CIT/CIT284490> a skos:ConceptScheme ;
    dc:description "This is the Project" ;
    dc:title "Project Title" ;
    skos:hasTopConcept <https://data.vam.ac.uk/CIT/CIT286958>,
        <https://data.vam.ac.uk/CIT/CIT286959>,
        <https://data.vam.ac.uk/CIT/CIT286960>,
        <https://data.vam.ac.uk/CIT/CIT286961>,
        <https://data.vam.ac.uk/CIT/CIT286963>,
        <https://data.vam.ac.uk/CIT/CIT287802>,
        <https://data.vam.ac.uk/CIT/CIT288669>,
        <https://data.vam.ac.uk/CIT/CIT289138> 

<https://data.vam.ac.uk/CIT/CIT286960> a skos:Concept ;
    dc:source "A Source" ;
    skos:altLabel "An Alt Label"@en ;
    skos:broader <https://data.vam.ac.uk/CIT/CIT284490> ;
    skos:narrower <https://data.vam.ac.uk/CIT/CIT284537>,
        <https://data.vam.ac.uk/CIT/CIT284541>,
        <https://data.vam.ac.uk/CIT/CIT287833>,
        <https://data.vam.ac.uk/CIT/CIT287835>,
        <https://data.vam.ac.uk/CIT/CIT288261>,
        <https://data.vam.ac.uk/CIT/CIT288365>,
        <https://data.vam.ac.uk/CIT/CIT288428>,
        <https://data.vam.ac.uk/CIT/CIT288441>,
        <https://data.vam.ac.uk/CIT/CIT288546> ;
    skos:prefLabel "Term Label"@en ;
    skos:scopeNote "Scope Notes"@en ;
    skos:topConceptOf <https://data.vam.ac.uk/CIT/CIT284490> .

But for some reason the Title isn't picked up, and pages aren't generated (just the navigation tree).

I would also like the ability to navigate the tree in more than one language ideally, but that's probably a feature request!

When using webhooks on test.skohub.io it's not working:

gatsbyError: error glob error [Error: EMFILE: too many open files

even if the user's ulimit -n gives 100000.

From #1 (comment):

Going forward with the vocab, we might add some documentation properties (scopeNote etc.) with content from the description pages under https://www.hochschulkompass.de/studienbereiche-kennenlernen.html, e.g. https://www.hochschulkompass.de/kunst-musik-und-design.html for F146.
EDIT: We should of course also add a name and description of the vocab as a whole.

Reported by Raphaëlle via Mastodon.

See e.g. https://skohub.io/dini-ag-kim/hochschulfaechersystematik/heads/master/w3id.org/kim/hochschulfaechersystematik/n1.html

"Inbox" links to a URL at skohub.lobid.org, should be skohub.io.

At least those properties used in https://github.com/dini-ag-kim/hochschulfaechersystematik/blob/f6b6400c9c13e6cf85f2b640cdb10fdea5510fad/hochschulfaechersystematik.ttl#L11-L15 plus dct:alternative are missing in https://github.com/hbz/skohub-vocabs/blob/master/src/context.js

See #87 (comment)

For reviewing I triggered new builds for test an production at https://github.com/dini-ag-kim/hochschulfaechersystematik/settings/hooks (I made you admin so you can take a look at the webhooks). The test system had a direct response 400 with "Payload was invalid, build not triggered" and production had problems building the page based on the payload, see https://skohub.io/build/?id=3cf02780-c406-429f-b12f-f5b303edcaaa. This is strange (at least for production) as it was build before on both systems (see production and test) and I did not change the file.

@sroertgen asked for this via Slack:

weisst Du zufällig, ob es einfach möglich ist, die Sprache in skohub-vocabs auf “de” zu setzen, wenn sie da ist

We should think about how to adress this. There are different options:

• Let publishers configure the default language to be shown.
• Choose language based on language header sent by browser

@sroertgen, what do you have in mind?

See e.g. https://skohub.io/hbz/vocabs-edu/heads/master/w3id.org/class/esc/n0322.json:

{
  "id":"https://w3id.org/class/esc/n0322",
  "type":"Concept",
  "followers":"https://skohub.io/followers?subject=hbz%2Fvocabs-edu%2Fheads%2Fmaster%2Fw3id.org%2Fclass%2Fesc%2Fn0322",
  "inbox":"https://skohub.io/inbox?actor=hbz%2Fvocabs-edu%2Fheads%2Fmaster%2Fw3id.org%2Fclass%2Fesc%2Fn0322",
  "prefLabel":{
    "en":"Library, information and archival studies"
  },
  "notation":[
    "0322"
  ],
  "broader":{
    "id":"https://w3id.org/class/esc/n032",
    "prefLabel":{
      "en":"Journalism and information"
    }
  },
  "inScheme":{
    "id":"https://w3id.org/class/esc/scheme",
    "title":{
      "en":"Educational Subjects Classification"
    }
  },
  "@context":{
    "id":"@id",
    "type":"@type",
    "@vocab":"http://www.w3.org/2004/02/skos/core#",
    "xsd":"http://www.w3.org/2001/XMLSchema#",
    "dct":"http://purl.org/dc/terms/",
    "schema":"http://schema.org/",
    "vann":"http://purl.org/vocab/vann/",
    "title":{
      "@id":"dct:title",
      "@container":"@language"
    },
    "description":{
      "@id":"dct:description",
      "@container":"@language"
    },
    "issued":{
      "@id":"dct:issued",
      "@type":"xsd:date"
    },
    "created":{
      "@id":"dct:created",
      "@type":"xsd:date"
    },
    "modified":{
      "@id":"dct:modified",
      "@type":"xsd:date"
    },
    "creator":"dct:creator",
    "publisher":"dct:publisher",
    "preferredNamespacePrefix":"vann:preferredNamespacePrefix",
    "preferredNamespaceUri":"vann:preferredNamespaceUri",
    "isBasedOn":"schema:isBasedOn",
    "source":"dct:source",
    "prefLabel":{
      "@container":"@language"
    },
    "definition":{
      "@container":"@language"
    },
    "scopeNote":{
      "@container":"@language"
    },
    "note":{
      "@container":"@language"
    },
    "notation":{
      "@container":"@set"
    },
    "narrower":{
      "@container":"@set"
    },
    "narrowerTransitive":{
      "@container":"@set"
    },
    "broaderTransitive":{
      "@container":"@set"
    }
  }
}

For example instead of:

{
   "id":"http://w3id.org/class/hochschulkompass/S349#",
   "type":"Concept",
   "broader":{
      "id":"http://w3id.org/class/hochschulkompass/B348#"
   },
   "inScheme":{
      "id":"http://w3id.org/class/hochschulkompass/scheme#"
   }
}

it should be:

{
   "id":"http://w3id.org/class/hochschulkompass/S349#",
   "type":"Concept",
   "broader":{
      "id":"http://w3id.org/class/hochschulkompass/B348#",
      "prefLabel":{
         "de":"Philosophie"
      }
   },
   "inScheme":{
      "id":"http://w3id.org/class/hochschulkompass/scheme#",
      "title":{
         "de":"Fächersystematik Hochschulbildung in Deutschland"
      }
   }
}

Would it make sense to provide a mechanism for vocabulary versioning like described here?: NatLibFi/Skosmos#677

That's just an idea. I'm not sure about it at all (it seems to overlap with what Ilik/Koster are after), and also I am not sure what it would require in terms of github history parsing/mapping... But it seems like a nice thing to have, especially insofar as this is already linked with github.

...at best per level of hierarchy. Current example is https://test.skohub.io/dini-ag-kim/hochschulfaechersystematik/w3id.org/class/hochschulfaechersystematik/scheme.html

Currently, all levels are obviously ordered alphabetically by prefLabel. The original classification is ordered by notation on the first and second level and alphabetically on the third and lowest hierarchy level, see https://www.destatis.de/DE/Methoden/Klassifikationen/Bildung/studenten-pruefungsstatistik.pdf?__blob=publicationFile

To reach the same ordering in SkoHub, it would be great to somehow configure the ordering per level.

From #85. Currently, narrower statements are required in the turtle file for hierarchical relationships being shown in the UI, broader are not. It would be good if you could choose only one of both and the vocab would build as expected.

In the long run, wee should support all skos properties. For now, we will cover those we need for the vocabularies we publish. https://github.com/VIVO-DE/destatis_faecherklassifikation – which is published at https://test.skohub.io/acka47/skos/onto.tib.eu/destf/cs/index.html – has the following SKOS properties that are not covered yet: notation, narrowerTransitive, broaderTransitive and note.

See https://www.hochschulkompass.de/studium/studiengangsuche/erweiterte-studiengangsuche.html

There are three classifications:

1. by subject group ("Fächergruppe")
2. by study area ("Studienbereiche")
3. by field of study ("Studienfelder")

We will have to find out which of those best to skosify.

See e.g. https://skohub.io/build/?id=29dcd2e2-745c-4873-b8e9-4c8d642b0b43, where a warning causes the build to be considered erroneous:

Due to this, the link to the build is not displayed on top of the log after completion.

This might be used to add test to a GitHub repo that let you know whether your SKOS vocab is "SkoHub-ready".

For example https://w3id.org/kim/hcrt/slide.

$ curl -LH "accept: application/json" https://w3id.org/kim/hcrt/slide | jq .idgives"https://w3id.org/kim/hcrt/slide"`

~$ curl -LH "accept: application/activity+json" https://w3id.org/kim/hcrt/slide | jq .id gives "https://skohub.io/dini-ag-kim/hcrt/heads/master/w3id.org/kim/hcrt/slide"

@literarymachine, is this intended like this? My impression is that both ids should be the same (https://w3id.org/kim/hcrt/slide).

For small controlled vocabularies like the "Audience Role" vocab it might not really make sense to have an inbox for each element. Thus, it would be nice to be able to turn off inboxes and SkoHub pubsub functions.

@literarymachine, we have talked about a Yaml config file in the repo along with a vocabulary. This would be a good place for configuring this. (I think each ttl file could have a yml file with the same name.)

This only refers to the version on lobid servers. Example link: https://skohub.lobid.org/build/?id=d7649625-b911-410b-a227-170288a335e2

At skohub.io I can view the build information.

All commit pointers (branches or tags) from a repository should be published via SkoHub.

Today, I published the Educational Subject Classification (ESC) via SkoHub.
However, although this is also a slash URI vocab, the HTML rendering did not work out of the box. The first problem was the single quotes for scopeNote. When I replaced them with triple quotes (acka47/skos@781c542) it got a bit better, but no labels were shown. (I fixed this with only using @de language tags, see #24.)

Depends on #1.

There should be a start script and an init.d script and a 'monit.rc'.

This issue is analog to skohub-io/skohub-pubsub#34.

Example screenshot

:

I published the NWBib spatial classification with ~4,500 concepts via SkoHub: https://skohub.io/acka47/nwbib-spatial/heads/master/nwbib.de/spatial.html

As you can see, only the top level concepts are shown on the left-hand side with no possibility to expand the narrower concepts. The concepts themselves are there, though, see e.g. https://skohub.io/acka47/nwbib-spatial/heads/master/nwbib.de/spatial.html#Q61726649 or https://skohub.io/acka47/nwbib-spatial/heads/master/nwbib.de/spatial.html#33

Support a wbhook-based approach fro publishing SKOS files in Turtle notation from GitLab.

Currently, it still reads SkoHub-SSG on the web page. This should be replaced with "SkoHub Vocabs".

Until now, we've been happy with only one deployment environment running on https://test.skohub.io. It is time to switch to both a test and production environment with produvtion running on https://skohub.io.

See https://test.skohub.io/build/?id=0f71e7ef-98b4-4fc6-9e84-3332fe2bf62e#68 for https://github.com/hbz/lobid-vocabs. There are some SKOS files contained in folders which might also be a problem.

We created a record for the classification in BARTOC: https://bartoc.org/en/node/18914 Once the classification is available in SKOS, the URL should also be added to BARTOC.

When working with larger files (e.g. https://raw.githubusercontent.com/sroertgen/oeh-framework-bb/master/data/curriculum_bb_competences_skos.ttl) skohub-vocabs failed with the error: EMFILE: too many open files.

This gatsby issue seems to be related: gatsbyjs/gatsby#12011

I did a quick test installing graceful-fs and implementing the following like mentioned above in gatsby-node.js and webHookServer.js:

const gracefulFs = require('graceful-fs');
gracefulFs.gracefulify(fs);

Before making a pull request I would like to ask if you get the same error or it might be something special to my machine.

You do $ npm run build and get:

gatsby UNHANDLED REJECTION Cannot read property 'forEach' of undefined

This is because you have no data in the proper place. RTM, do:

$ cp test/data/systematik.ttl data/

This was addressed in today's meeting by the OER metadata group of DINI AG KIM. It would be nice to have an overview of all vocabs published with SkoHub by a particular GitHub/GitLab organization or user. For example, at https://skohub.io/hbz/ should be a list of all vocabs from https://skohub.io/hbz/vocabs-edu/ and https://skohub.io/hbz/lobid-vocabs/ (where currently nothing is listed so that this is not a good example but I don't know a better one).

For the start, we should at least support JSON-LD. Currently, it does not work:

$ curl -LIH "Accept: application/ld+json" http://w3id.org/class/hochschulfaecher/scheme
HTTP/1.1 301 Moved Permanently
Date: Mon, 19 Aug 2019 11:38:08 GMT
Server: Apache/2.4.7 (Ubuntu)
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubDomains
Location: https://w3id.org/class/hochschulfaecher/scheme
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 302 Found
Date: Mon, 19 Aug 2019 11:38:08 GMT
Server: Apache/2.4.7 (Ubuntu)
Access-Control-Allow-Origin: *
Location: https://test.skohub.io/literarymachine/skos/w3id.org/class/hochschulfaecher/scheme
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 406 Not Acceptable
Date: Mon, 19 Aug 2019 11:38:08 GMT
Server: Apache/2.4.18 (Ubuntu)
Alternates: {"scheme.html" 1 {type text/html} {length 299785}}, {"scheme.index" 1 {type text/index} {length 650627}}, {"scheme.json" 1 {type application/json} {length 60354}}
Vary: negotiate,accept
TCN: list
Content-Type: text/html; charset=iso-8859-1

In the long run, we might also support other RDF serializations.

When a vocab file is updated on GitHub (in the master branch), the correcponding HTML representation should automatically be updated.

Don't know if this is actually of importance - however, every time i do $ npm i it's warning

found 16160 vulnerabilities (16156 high, 4 critical)
run npm audit fix to fix them, or npm audit for details

My npm version is:

$ npm --version
6.9.0

When I do the audit fix, some packages are downgraded like e.g.:

"convert-source-map": "1.6.0" => "convert-source-map": "^1.1.0",

Even so, things seems to be builded and running just fine.

Also, my npm prefixes the version numbers of the packages with an ^ , I think to be even more on the sure side that the version number will be the correct one.

You may reject this PR, just in case, to let you know, for what it's worth.

It would be great to just publish a SKOS vocab on GitHub and then be able to have all the functionality in SkoHub without setting up a redirect.

Background

I wanted to publish https://github.com/tawahle/destatis_faecherklassifikation/blob/master/faecherklassifikation.rdf with SkoHub for interested parties to inspect the vocabulary. Everything worked quite well, the classification was shown as expected. However, clicking on a concept made a problem of course, because the concept URIs (namespace https://onto.tib.eu/destf/cs/) do not redirect to SkoHub (in fact, they do not go anywhere).

As I am not in control of the domain and did not want to use another HTTP namespace where I'd have to setup a redirect myself, I tried to use a local namespaces. This did not work, and in fact sometimes even broke the other Skohub-published vocab in the repo:

• @prefix : <#> . did not work and broke https://test.skohub.io/acka47/skos/w3id.org/class/hochschulfaecher/scheme.html (acka47/skos@592f076)
• @prefix : <faecherklassifikation#> . & @prefix : </faecherklassifikation#> . did not break the other vocab but did not work. (acka47/skos@4abe0c6)
• @base </faecherklassifikation/> . same as the previous. (acka47/skos@27a5dfe)
• @base <faecherklassifikation/> . also broke https://test.skohub.io/acka47/skos/w3id.org/class/hochschulfaecher/scheme.html (acka47/skos@f75a835)

As a user I want to be able to collapse and expand the whole schema as well as parts of it in order to easily get an overview over a classification.

E.g. with GitHub or GitLab icon.

https://github.com/nextapps-de/flexsearch

Today, I published the Educational Subject Classification (ESC) via SkoHub. Everything went fine regarding the process (pushing int eh GitHub master and updating SkoHub via webhook).

For labels to be shown, I had to replace @en tags by de`, see acka47/skos@b791afb

From #1 (comment):

We agreed to adjust the namespace (not use "Hochschulkompass") and to configure the redirect with w3id.

README tells a fork can be made public at https://test.skohub.io/YOUR_GITHUB_USER/skos/w3id.org/class/hochschulfaecher/scheme. It would help to also link to an existing instance. https://test.skohub.io/hbz/skos/w3id.org/class/hochschulfaecher/scheme does not exist.

With the current, you can not see whether something is a link, see e.g. https://skohub.io/dini-ag-kim/hochschulfaechersystematik/w3id.org/class/hochschulfaechersystematik/scheme.html or https://skohub.io. I suggest making them blue as people are used to.

Hello,

I tried to follow your blog entry (http://blog.lobid.org/2019/09/27/presenting-skohub-vocabs.html#implementation) and documentation along, but when running npm run build or npm run develop I get the following error:

> [email protected] develop /skohub/skohub-vocabs
> gatsby develop

success open and validate gatsby-configs - 0.065s
success load plugins - 1.245s
success onPreInit - 0.011s
success initialize cache - 0.097s
success copy gatsby files - 0.149s
success onPreBootstrap - 0.024s

 ERROR

Encountered an error parsing the provided GraphQL type definitions:
Syntax Error: Expected Name, found }

  21 |     inbox: String
  22 |   }
  23 |
  24 |   type LanguageMap {
  25 |
> 26 |   }
     |   ^
  27 |


not finished createSchemaCustomization - 0.065s
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] develop: `gatsby develop`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] develop script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /.npm/_logs/2020-02-07T18_22_43_407Z-debug.log

log file:

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'develop' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'predevelop', 'develop', 'postdevelop' ]
5 info lifecycle [email protected]~predevelop: [email protected]
6 info lifecycle [email protected]~develop: [email protected]
7 verbose lifecycle [email protected]~develop: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~develop: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/steffenroertgen/gwdg_owncloud/coding/skohub/skohub-vocabs/node_modules/.bin:/usr/local/opt/python/libexec/bin:/usr/local/sbin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Users/steffenroertgen/.npm-global/bin
9 verbose lifecycle [email protected]~develop: CWD: /Users/steffenroertgen/gwdg_owncloud/coding/skohub/skohub-vocabs
10 silly lifecycle [email protected]~develop: Args: [ '-c', 'gatsby develop' ]
11 silly lifecycle [email protected]~develop: Returned: code: 1  signal: null
12 info lifecycle [email protected]~develop: Failed to exec develop script
13 verbose stack Error: [email protected] develop: `gatsby develop`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:321:20)
13 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:321:20)
13 verbose stack     at maybeClose (internal/child_process.js:1026:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
14 verbose pkgid [email protected]
15 verbose cwd /Users/steffenroertgen/gwdg_owncloud/coding/skohub/skohub-vocabs
16 verbose Darwin 18.7.0
17 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "develop"
18 verbose node v13.7.0
19 verbose npm  v6.13.6
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] develop: `gatsby develop`
22 error Exit status 1
23 error Failed at the [email protected] develop script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]