kuhumcst / dannet Goto Github PK
View Code? Open in Web Editor NEWThe Danish WordNet as an RDF graph.
Home Page: https://wordnet.dk
License: MIT License
The Danish WordNet as an RDF graph.
Home Page: https://wordnet.dk
License: MIT License
Apparently, the version of Marl I have cached, version 0.2, is significantly older than the current version, version 1.2. This is because the version I have cached comes from the PURL which was last updated in 2011. It seems that—although the PURL version is probably more widespread—it makes sense to use the current version from 2021 instead.
Since this newer version also has a more thorough documentation, it makes sense to see if the current representation in DanNet can be improved.
The above were suggested by Eric Scott (ont-app).
I would like to annotate every property, class, resources, so that it is clear to which entity it belongs. This can be used both for export purposes—although named graphs might be a better fit—as well as for browsing data and metadata.
No matter what is decided in #10, inferencing will be a key component of the new DanNet in some way. Obviously, if the end goal is a single shared database with inferencing, performance will be extremely important, but it also is in many other cases.
Currently, the lowest level of OWL inferencing - OWL Micro - is used (specifically: OntModelSpec/OWL_MEM_MICRO_RULE_INF
). This still incurs a heavy performance penalty which results in 10+ minute long queries in some cases... with subsequent queries being only a few seconds, though.
Another issue is that the inferred information is not really all that valuable, e.g. Owl:thing
or some of the transitive SKOS relations, e.g.skos:narrowerTransitive
which returns hundreds of results, and also many useless anonymous resources included in e.g. :rdf/type
, presumably also through some transitive relation.
So it is in the interest of the inference model itself to make it more performant as this will also lead to much less cluttered results.
See e.g. rdfs:isDefinedBy
in http://localhost:3456/dannet/external/svs/term_status
The linked resource ends in #
and this doesn't resolve to anything. However, removing the #
and retrying the fetch does. One could argue that a URI with an empty #
(= empty fragment string) is in fact the same as one without a #
at all, so upgrading to the resource without the #
at the query level should be entirely legal. However, for the odd case where the #
-appended resource identifier actually resolves to something we want to return that entity instead.
It should be implemented as a basic dual search, but only in cases where the resource identifier matches this pattern and nothing is found on the first attempt.
Rather than being represented as a single triples containing the current text literal (e.g. "BoundedEvent+Agentive+Mental+Purpose"), the ontological type(s) should be represented as a set of triples - one for each of the types contained in the text.
Searches for a prototypical collection of ontological types can still happen, but with this change the types can also be queried individually.
They seem to be derived from https://archive.illc.uva.nl/EuroWordNet/corebcs/ewnTopOntology.html although no OWL schema exists, so the relation and the concepts themselves will still have to be part of a DanNet OWL schema rather than an external one.
See: https://csvw.org/
Apparently a product of Rick Moynihan and others. Could be an interesting way to marry RDF/CSV?
See: https://rdfa.info/
Aristotle only wraps in-memory databases, but we will surely need to kind of persistence layer. Apache Jena's persistent triplestore is called TDB. There is also SDB which uses SQL databases as storage, but this has been deprecrated.
In order to use, TDB I will need to learn a bit more about the object-oriented architecture of Jena, specifically the relation between Graph vs Model vs Dataset classes.
The default content negotiation interceptor of Pedestal doesn't do language negotiation. However, according to one user on Clojurians Slack, it is possible to copy and reuse most of the existing interceptor to also do language negotiation:
Fredrik 19:17
The argument :content-param-paths specifies where to look for the "Accept" header. If you set it to [:request :headers "accept-language"] , and change this line to store the negotiation result in the correct place, you should be good. https://github.com/pedestal/pedestal/blob/d20065013abf5d3793ae5301e18a2398707fa2a9/service/src/io/pedestal/http/content_negotiation.clj#L186
content_negotiation.clj
(assoc-in ctx [:request :accept] content-match)
So ideally, this interceptor is inserted ahead of all routes and the content-generating interceptors make use of this language information.
See: https://github.com/lambdaisland/hiccup
This is basically Reagent Hiccup, but on the server side. I intend this to be a first step towards turning the DanNet website into a progressive web app. Once loaded in the frontend, reagent takes over rendering.
The new DanNet schemas should come with labels/comments in both Danish and English.
In addition, the labels of the external schemas can also be translated into Danish (and published as a companion dataset). This would, as a side-effect, create a mostly Danish HTML experience.
The DanNet-specific resources/entities should be resolvable via their URI.
Navigating to the URI should result in some form of content negotiation and the return of either a data structure (JSON, EDEN, TTL) or perhaps a basic HTML web page. The data comes directly from an instance of a TDB graph. This is in line with recommendations for linked data.
The new DanNet OWL schema should also resolve. This is a more specific case (effectively just serving static data).
(this issue extends #13 and is essentially a nice-to-have)
In the current iteration of the new DanNet web presence there is an unsolved issue with certain labels being identical in result sets, making resources hard to differentiate. One example might be centrum which evokes multiple synsets and has multiple senses all with the same label, making it impossible to distinguish the list of resources.
One solution could be to have a way of obtaining and displaying additional information depending on the relation in question (or alternatively via the type, although this requires class information to be fetched in the original query). This could be a graph path, set of triples, a function, etc. which defines how to get the additional data point, e.g. for the ontolex:evokes
relation the graph path would be
[:skos/definition]
while for ontolex:lexicalizedSense
the graph path would be
[:lexinfo/senseExample :rdf/value]
Thus, for words the definitions and example sentences are available for decorating the listed resources. The decoration might be represented as a sublist underneath the item as this would also allow for the support of multiple returned items for the given path, e.g. multiple example sentences.
Some inspiration here: https://www.reddit.com/r/semanticweb/comments/qke4gu/til_how_to_roundtrip_blank_nodes_in_jenafuseki/
Some RDF nodes are blank—i.e. they don't have a stable ID—and dealing with them can be troublesome. It may be possible to find a generic way to "expand" blank node content and include it with RDF resource lookups.
In the old RDF export, several thousand references to both DanNet and Princeton Wordnet IDs use a format that is not recognised by Apache Jena. In more concrete terms, attempting to import DanNet as-is results in many thousands of lines of this style of warning:
...
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9618, col: 114] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-beard%1:08:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9619, col: 82] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-salon%1:06:02::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9620, col: 113] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-salon%1:06:02::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9621, col: 94] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-supermarket%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9622, col: 120] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-supermarket%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9623, col: 80] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-hall%1:06:04::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9624, col: 113] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-hall%1:06:04::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9625, col: 80] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-home%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9626, col: 112] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-home%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9627, col: 90] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-residence%1:15:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9628, col: 117] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-residence%1:15:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9629, col: 80] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-flat%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9630, col: 113] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-flat%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9631, col: 80] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-tent%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9632, col: 113] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-tent%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
[nREPL-session-10db3318-811e-47e9-beec-95f4f004cc0b] WARN org.apache.jena.riot - [line: 9633, col: 84] {W107} Bad URI: <http://www.wordnet.dk/owl/instance/2009/03/instances/synset-stable%1:06:00::> Code: 30/ILLEGAL_PERCENT_ENCODING in PATH: The host component a percent occurred without two following hexadecimal digits.
...
The issue has to do with the suffixes such as %1:06:02::
or %1:15:00::
not being valid according to the XML processor used by Jena.
According to Nicolai Hartvig Sørensen (the old maintainer of DanNet) the IDs have apparently undergone several changes since then and will have to be changed anyway. In addition, they might have been mangled in the current export, e.g. %5:00:00:rich:03
is also an example suffix.
Nicolai says the current IDs are based on this paper: https://www.aclweb.org/anthology/W11-0129.pdf
We need to come up with some type of scheme to reliably convert these old IDs to the newer IDs used by Princeton - or the Open English Wordnet in case we use that.
Some further complications:
Some of the w3c schema do not always properly resolve. They should probably be downloaded and cached locally.
Typical error:
[qtp454953625-30] INFO io.pedestal.http - {:msg "GET /dannet/data", :line 80}
[qtp454953625-30] INFO io.pedestal.http.cors - {:msg "cors request processing", :origin nil, :allowed true, :line 84}
[qtp454953625-30] ERROR io.pedestal.http.impl.servlet-interceptor - {:msg "error-ring-response triggered", :context {:response nil, :io.pedestal.interceptor.chain/stack (#Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:protocol "HTTP/1.1", :async-supported? true, :remote-addr "[0:0:0:0:0:0:0:1]", :servlet-response #object[org.eclipse.jetty.server.Response 0x6222621b "HTTP/1.1 200 \nDate: Mon, 28 Feb 2022 16:44:05 GMT\r\n\r\n"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x250e158e "io.pedestal.http.servlet.FnServlet@250e158e"], :headers {"sec-fetch-site" "none", "sec-ch-ua-mobile" "?0", "host" "localhost:8080", "user-agent" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.109 Safari/537.36", "sec-fetch-user" "?1", "sec-ch-ua" "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google Chrome\";v=\"98\"", "sec-ch-ua-platform" "\"macOS\"", "connection" "keep-alive", "upgrade-insecure-requests" "1", "accept" "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", "accept-language" "da,en-GB;q=0.9,en-US;q=0.8,en;q=0.7,zh-CN;q=0.6,zh;q=0.5", "sec-fetch-dest" "document", "accept-encoding" "gzip, deflate, br", "sec-fetch-mode" "navigate", "cache-control" "max-age=0"}, :server-port 8080, :servlet-request #object[org.eclipse.jetty.server.Request 0x6180c05a "Request(GET //localhost:8080/dannet/data)@6180c05a"], :path-info "/dannet/data", :url-for #object[clojure.lang.Delay 0x115d2083 {:status :pending, :val nil}], :uri "/dannet/data", :server-name "localhost", :query-string nil, :path-params {}, :body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x728a946a "HttpInputOverHTTP@728a946a[c=0,q=0,[0]=null,s=STREAM]"], :accept-language {:field "da", :type "da", :subtype nil}, :scheme :http, :request-method :get, :context-path "", :accept {:field "text/html", :type "text", :subtype "html"}}, :bindings {#'io.pedestal.http.route/*url-for* #object[clojure.lang.Delay 0x115d2083 {:status :pending, :val nil}]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x349029a "io.pedestal.http.impl.servlet_interceptor$start_servlet_async@349029a"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__15172 0x642c90b9 "io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__15172@642c90b9"]), :servlet-response #object[org.eclipse.jetty.server.Response 0x6222621b "HTTP/1.1 200 \nDate: Mon, 28 Feb 2022 16:44:05 GMT\r\n\r\n"], :route {:path "/dannet/data", :method :get, :path-re #"/\Qdannet\E/\Qdata\E", :path-parts ["dannet" "data"], :interceptors [#Interceptor{:name :io.pedestal.http.content-negotiation/negotiate-content} #Interceptor{:name :dk.cst.dannet.web.resources/negotiate-language} #Interceptor{:name :dk.cst.dannet.web.resources/entity}], :route-name :clojure.core/dn-dataset-entity, :path-params {}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[clojure.core$constantly$fn__5740 0x1c0ded87 "clojure.core$constantly$fn__5740@1c0ded87"]}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x250e158e "io.pedestal.http.servlet.FnServlet@250e158e"], :servlet-request #object[org.eclipse.jetty.server.Request 0x6180c05a "Request(GET //localhost:8080/dannet/data)@6180c05a"], :url-for #object[clojure.lang.Delay 0x115d2083 {:status :pending, :val nil}], :io.pedestal.interceptor.chain/execution-id 1, :servlet-config #object[org.eclipse.jetty.servlet.ServletHolder$Config 0x3a8213a8 "org.eclipse.jetty.servlet.ServletHolder$Config@3a8213a8"], :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x71949bb0 "io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@71949bb0"]}, :line 253}
clojure.lang.ExceptionInfo: java.util.concurrent.ExecutionException in Interceptor :dk.cst.dannet.web.resources/entity - org.apache.jena.atlas.web.HttpException: org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out {:execution-id 1, :stage :leave, :interceptor :dk.cst.dannet.web.resources/entity, :exception-type :java.util.concurrent.ExecutionException, :exception #error {
:cause "Operation timed out"
:via
[{:type java.util.concurrent.ExecutionException
:message "org.apache.jena.atlas.web.HttpException: org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out"
:at [java.util.concurrent.FutureTask report "FutureTask.java" 122]}
{:type org.apache.jena.atlas.web.HttpException
:message "org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out"
:at [org.apache.jena.riot.web.HttpOp exec "HttpOp.java" 1095]}
{:type org.apache.http.conn.HttpHostConnectException
:message "Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out"
:at [org.apache.http.impl.conn.DefaultHttpClientConnectionOperator connect "DefaultHttpClientConnectionOperator.java" 156]}
{:type java.net.ConnectException
:message "Operation timed out"
:at [sun.nio.ch.Net connect0 "Net.java" -2]}]
:trace
[[sun.nio.ch.Net connect0 "Net.java" -2]
[sun.nio.ch.Net connect "Net.java" 579]
[sun.nio.ch.Net connect "Net.java" 568]
[sun.nio.ch.NioSocketImpl connect "NioSocketImpl.java" 588]
[java.net.SocksSocketImpl connect "SocksSocketImpl.java" 327]
[java.net.Socket connect "Socket.java" 633]
[org.apache.http.conn.socket.PlainConnectionSocketFactory connectSocket "PlainConnectionSocketFactory.java" 75]
[org.apache.http.impl.conn.DefaultHttpClientConnectionOperator connect "DefaultHttpClientConnectionOperator.java" 142]
[org.apache.http.impl.conn.PoolingHttpClientConnectionManager connect "PoolingHttpClientConnectionManager.java" 376]
[org.apache.http.impl.execchain.MainClientExec establishRoute "MainClientExec.java" 393]
[org.apache.http.impl.execchain.MainClientExec execute "MainClientExec.java" 236]
[org.apache.http.impl.execchain.ProtocolExec execute "ProtocolExec.java" 186]
[org.apache.http.impl.execchain.RetryExec execute "RetryExec.java" 89]
[org.apache.http.impl.execchain.RedirectExec execute "RedirectExec.java" 110]
[org.apache.http.impl.client.InternalHttpClient doExecute "InternalHttpClient.java" 185]
[org.apache.http.impl.client.CloseableHttpClient execute "CloseableHttpClient.java" 83]
[org.apache.http.impl.client.CloseableHttpClient execute "CloseableHttpClient.java" 56]
[org.apache.jena.riot.web.HttpOp exec "HttpOp.java" 1082]
[org.apache.jena.riot.web.HttpOp execHttpGet "HttpOp.java" 322]
[org.apache.jena.riot.web.HttpOp execHttpGet "HttpOp.java" 381]
[org.apache.jena.riot.RDFParser openTypedInputStream "RDFParser.java" 389]
[org.apache.jena.riot.RDFParser parseURI "RDFParser.java" 302]
[org.apache.jena.riot.RDFParser parse "RDFParser.java" 296]
[org.apache.jena.riot.RDFParserBuilder parse "RDFParserBuilder.java" 540]
[org.apache.jena.riot.RDFDataMgr parseFromURI "RDFDataMgr.java" 921]
[org.apache.jena.riot.RDFDataMgr read "RDFDataMgr.java" 252]
[org.apache.jena.riot.RDFDataMgr read "RDFDataMgr.java" 221]
[org.apache.jena.riot.RDFDataMgr read "RDFDataMgr.java" 151]
[org.apache.jena.riot.RDFDataMgr read "RDFDataMgr.java" 142]
[org.apache.jena.riot.adapters.RDFReaderRIOT read "RDFReaderRIOT.java" 76]
[org.apache.jena.rdf.model.impl.ModelCom read "ModelCom.java" 259]
[jdk.internal.reflect.NativeMethodAccessorImpl invoke0 "NativeMethodAccessorImpl.java" -2]
[jdk.internal.reflect.NativeMethodAccessorImpl invoke "NativeMethodAccessorImpl.java" 77]
[jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43]
[java.lang.reflect.Method invoke "Method.java" 568]
[clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167]
[clojure.lang.Reflector invokeInstanceMethod "Reflector.java" 102]
[dk.cst.dannet.db$__GT_schema_model$fn__27300 invoke "db.clj" 46]
[clojure.core.protocols$fn__8249 invokeStatic "protocols.clj" 168]
[clojure.core.protocols$fn__8249 invoke "protocols.clj" 124]
[clojure.core.protocols$fn__8204$G__8199__8213 invoke "protocols.clj" 19]
[clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
[clojure.core.protocols$fn__8236 invokeStatic "protocols.clj" 75]
[clojure.core.protocols$fn__8236 invoke "protocols.clj" 75]
[clojure.core.protocols$fn__8178$G__8173__8191 invoke "protocols.clj" 13]
[clojure.core$reduce invokeStatic "core.clj" 6886]
[clojure.core$reduce invoke "core.clj" 6868]
[dk.cst.dannet.db$__GT_schema_model invokeStatic "db.clj" 43]
[dk.cst.dannet.db$__GT_schema_model invoke "db.clj" 40]
[dk.cst.dannet.db$__GT_dannet invokeStatic "db.clj" 142]
[dk.cst.dannet.db$__GT_dannet doInvoke "db.clj" 115]
[clojure.lang.RestFn invoke "RestFn.java" 457]
[dk.cst.dannet.web.resources$eval27472$fn__27473 invoke "resources.clj" 32]
[clojure.core$binding_conveyor_fn$fn__5823 invoke "core.clj" 2047]
[clojure.lang.AFn call "AFn.java" 18]
[java.util.concurrent.FutureTask run "FutureTask.java" 264]
[java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1136]
[java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 635]
[java.lang.Thread run "Thread.java" 833]]}}
at io.pedestal.interceptor.chain$throwable__GT_ex_info.invokeStatic(chain.clj:35)
at io.pedestal.interceptor.chain$throwable__GT_ex_info.invoke(chain.clj:32)
at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:57)
at io.pedestal.interceptor.chain$try_f.invoke(chain.clj:44)
at io.pedestal.interceptor.chain$leave_all_with_binding.invokeStatic(chain.clj:254)
at io.pedestal.interceptor.chain$leave_all_with_binding.invoke(chain.clj:237)
at io.pedestal.interceptor.chain$leave_all$fn__11219.invoke(chain.clj:268)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at io.pedestal.interceptor.chain$leave_all.invokeStatic(chain.clj:266)
at io.pedestal.interceptor.chain$leave_all.invoke(chain.clj:260)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:379)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:389)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)
at io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15197.invoke(servlet_interceptor.clj:351)
at io.pedestal.http.servlet.FnServlet.service(servlet.clj:28)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.concurrent.ExecutionException: org.apache.jena.atlas.web.HttpException: org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at clojure.core$deref_future.invokeStatic(core.clj:2317)
at clojure.core$future_call$reify__8544.deref(core.clj:7041)
at clojure.core$deref.invokeStatic(core.clj:2337)
at clojure.core$deref.invoke(core.clj:2323)
at dk.cst.dannet.web.resources$__GT_entity_ic$fn__27528.invoke(resources.clj:151)
at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:54)
... 39 more
Caused by: org.apache.jena.atlas.web.HttpException: org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1095)
at org.apache.jena.riot.web.HttpOp.execHttpGet(HttpOp.java:322)
at org.apache.jena.riot.web.HttpOp.execHttpGet(HttpOp.java:381)
at org.apache.jena.riot.RDFParser.openTypedInputStream(RDFParser.java:389)
at org.apache.jena.riot.RDFParser.parseURI(RDFParser.java:302)
at org.apache.jena.riot.RDFParser.parse(RDFParser.java:296)
at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:540)
at org.apache.jena.riot.RDFDataMgr.parseFromURI(RDFDataMgr.java:921)
at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:252)
at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:221)
at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:151)
at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:142)
at org.apache.jena.riot.adapters.RDFReaderRIOT.read(RDFReaderRIOT.java:76)
at org.apache.jena.rdf.model.impl.ModelCom.read(ModelCom.java:259)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:102)
at dk.cst.dannet.db$__GT_schema_model$fn__27300.invoke(db.clj:46)
at clojure.core.protocols$fn__8249.invokeStatic(protocols.clj:168)
at clojure.core.protocols$fn__8249.invoke(protocols.clj:124)
at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8236.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6886)
at clojure.core$reduce.invoke(core.clj:6868)
at dk.cst.dannet.db$__GT_schema_model.invokeStatic(db.clj:43)
at dk.cst.dannet.db$__GT_schema_model.invoke(db.clj:40)
at dk.cst.dannet.db$__GT_dannet.invokeStatic(db.clj:142)
at dk.cst.dannet.db$__GT_dannet.doInvoke(db.clj:115)
at clojure.lang.RestFn.invoke(RestFn.java:457)
at dk.cst.dannet.web.resources$eval27472$fn__27473.invoke(resources.clj:32)
at clojure.core$binding_conveyor_fn$fn__5823.invoke(core.clj:2047)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to www.w3.org:80 [www.w3.org/128.30.52.100] failed: Operation timed out
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1082)
... 41 more
Caused by: java.net.ConnectException: Operation timed out
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:579)
at java.base/sun.nio.ch.Net.connect(Net.java:568)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
at java.base/java.net.Socket.connect(Socket.java:633)
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
... 51 more
[qtp454953625-30] INFO io.pedestal.http.impl.servlet-interceptor - {:msg "sending error", :message "Internal server error: exception", :line 215}
The mapping between the old DanNet and Princeton WordNet are quite of messy. Rather than mapping directly to the instances of one WordNet dataset, they map to two different kinds of sense IDs using the dns:eq_has_synonym
relation:
wn:synset-sheep-noun-1
in WordNet 2.0 as it has the synsetID 102324230.It may be beneficial to completely ignore these relations for now and revisit them once Sussi is done mapping relations, at which point they can be properly converted and included in the dataset.
Perhaps the easy fix is to separate frontend code into a shadow-cljs.edn file, although I am unsure whether this will be possible since shadow-cljs does depend on some backend libs too.
Other resources:
The DanNet dataset has at ~401 words with multiple instances:
(->> (q/run (:graph @db) op/word-clones)
(filter (fn [{:syms [?w1 ?w2]}]
(and (= "dn" (namespace ?w1))
(= "dn" (namespace ?w2)))))
(group-by (juxt '?writtenRep '?pos))
(count))
These words exist in the old dataset files with multiple IDs and thus result in multiple resources when creating the new DanNet graph. In the new DanNet dataset they should all be merged as part of the import procedure. Triples that reference the obsolete IDs must be modified too.
For the individual labels, it makes sense that they are converted into select elements in cases where multiple languages exist and that the selected language becomes the default as opposed to all of the language info dissappearing from the UI entirely. For example, in some cases it might be nice to be able to see what the English label was as opposed to the Danish. This would be a relatively intuitive, hyperlocal way to inspect this data.
Implementing it entails creating a new component and passing down additional information in many places, e.g. languages and complete result sets.
Too many kryb:
Nathalie found this issue. Not sure whether it is something I will fix or whether it will come in as part of the overall changeds the lexicographers are deciding on.
The usages have been extracted and should probably be represented as
#{[wordsense :ontolex/usage '_usage]
['_usage :rdf/value usage-example]}
but they will need to be added to the Jena db using a suitable query that can find the right wordsense to attach them to, based on a Synset and a writtenRep.
Unfortunately, usage is not a well-specified part of the Ontolex standard.
Once relation which doesn't seem to exist in the CSV files, but which does exist in the old DanNet RDF is the dns/connotation
relation which can be e.g. "negative". I guess this is the sentiment data that Sussi is still working on. Rather than attempting to parse the old XML, it might be better to simply ignore these for now and include them at a later point, getting the data directly from Sussi.
... plus many more examples found under the hyponyms of http://localhost:3456/dannet/data/synset-559
These synsets contain the same single sense, so the fact that they exist must either mean that the sense is wrong or that one of them should be deleted.
This information should be sniffed out of the RDFS comments and the appropriate rows marked as inherited. Perhaps the comments could be entirely removed in the UI? Food for thought.
Since these IDs are not supposed to be a part of the real DanNet dataset, I should probably define them in a separate schema as well as exporting them separately.
See: https://elexis-eu.github.io/elexis-rest/
Bolette suggested implementing this API, after hearing Taryn talk about this, that we do the same.
From what I can tell, much of the work is essentially about reworking the search endpoint and extending/proxying the existing content-negotiation solution so that Turtle and JSON-LD (at least) will be supported.
See: https://www.w3.org/2016/05/ontolex/#lexical-sense-reference (e.g. lexinfo:dating, dct:subject)
The register
is included as part of the values in the wordsenses.csv
data. This metadata can also be translated and translated into triples.
Depends on #21. Attempt to mostly reuse the same component in frontend and backend code.
Currently, the Jena implementation can be persistent through TDB, but not when coupled with a reasoner doing inference. I am unsure whether if means that inference cannot realistically be coupled with TDB. Here are some options for database architecture:
The functions rendering the value for that row need to have the key passed down to the value rendering function, so that it can be dispatched on to modify the appearance of the value. An example might be :vann/preferredNamespacePrefix
which could turn the value into a (prefix-elem ...)
call.
Work on version 1.2 of the GWA schema is ongoing. See: globalwordnet/schemas#58 (comment)
It should be possible to infer the Ontolex triple from the GWA relation, but it will require rewriting - or perhaps uncommenting the correct section of - the Apache Jena rule DSL. Once that is done, the bootstrap code should only include the wn:partOfSpeech
triple, letting Jena's inference take care of the rest.
Presumably, this will be primarily transit endpoints that pull RDF resources based on an id, but perhaps with some special queries available too.
It should have some kind of content negotiation, such that it will return transit, JSON, or maybe even HTML when applicable. The IDs should match the RDF resource IDs and the main public site should be the namespace of the new DanNet.
The API will be used by a CLJS app available at the same domain that can be used to browse and visualise the WordNet.
Transito can be used to create the communication channel. Pedestal is probably the obvious backend framework.
Some kind of React graph element, presumably.
Should probably be a special element that can be opened or a separate page that is navigated to.
Requires moving the frontend dependencies from deps.edn
into shadow-cljs.edn
since I will need to use the NPM library integration of shadow-cljs.
The new DanNet schemas should reference the older schemas for good measure.
Currently, prefixes are hidden by default in the value part of the attribute-value table. The original intention was to let the user (eventually) hide/show the prefixes through some toggle.
However, now I think it might make the most sense to dynamically display prefixes in that column through a simple rule:
If the value column prefix matches the prefix of the current RDF resource -OR- if it matches the prefix of the attribute column, then it is hidden. This would essentially hide all relations that are part of the same dataset. The colour would be left to make their implicit relation more obvious.
In all other cases, the prefixes will be shown. This will make it clear that we are referencing e.g. cor:kickstarter
in the owl:sameAs
relation emanating from dn:kickstarter
, while removing a tonne of superfluous ontolex:
and dn:
prefixes from view.
See:
Some senses are "inserted by DanNet" and thus have missing words and are missing proper labels. This should be remedied.
e.g. http://localhost:3456/dannet/external?subject=%3Chttp%3A%2F%2Fpurl.org%2Fdc%2Felements%2F1.1%2F%3E
We want the < http://purl.org/dc/elements/1.1/ >
to be a clickable hyperlink so that it is easier for the user to visit this resource in their browser.
Extracted from issue #7:
Furthermore, Ontolex OWL (and other) files will also need to be referenced inside Apache Jena instance. I will also likely have so set up some kind of Inference for opposite relations.
First of all, dataset IRIs need to be discovered. Various methods can be used, e.g. is the object of a :rdfs/isDefinedBy
rel or is the subject of a :vann/preferredNamespaceUri
or :vann/preferredNamespacePrefix
rel.
When a resource can reliably be said to be a dataset, a special query can be used to attach all of the attributes defined in that namespace and list them, perhaps on a separate page.
Currently, the system is bootstrapped from various existing data sources, but there is no official export.
In the exported version, the different input datasets will need to be separated entirely, necessitating a separation into separate graphs or some other means of data separation at the point of export.
Jena (SPARQL in general) has the concept of named graphs, however the way this is implemented in Jena seems to be basically as a union of separate graph objects. One major complication is the fact that the web frontend currently relies on inference to generate "missing" triples and this is done for a single "data" graph + a "schema" graph, so having multiple graphs will likely not work.
Basically, there should be a :default
set of sections for most entities, but for certain ones there is an alternate set of sections dispatched on the entity's :rdf/type
. Multimethods are probably the way to go? Perhaps in another namespace.
Basically, the overall layout, colours, etc. should all be configurable using a Clojure map. Should probably be on a by-resource-class basis, allowing for more flexible pages.
In some synsets, there are too many words to make display of them practical
—e.g. { bold1§3 • byld§3 • bær§3 • bøtte§2 • hoved§1 • kasse1§6 • knold§5 • knop§2c • nød2§3 • roe§2 • skal§6
}—when displaying in certain situations, for instance when displaying a graph visualisation or as a way to adjust what words are display directly in the UI, barring user interaction explicitly expanding the list.
By using the ordnet.dk indices as a heuristic (1§3, §2, 1§6, §2c, ...) it should be possible to limit to the words the most canonical representations. Basically, the lower the number, the better, so in the above case the canonical word(s) would be hoved as it doesn't share this low index with any other words in the synset.
https://en.wikipedia.org/wiki/Comparison_of_triplestores
The few DanNet relations that could not be represented using the GWA relations will need to be represented inside a new OWL file with a fitting namespaces. This file can probably be generated using one of these Clojure OWL libraries:
If that is not ideal, then writing them by hand is also an option.
Furthermore, Ontolex OWL (and other) files will also need to be referenced inside Apache Jena instance. I will also likely have so set up some kind of Inference for opposite relations.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.