Comments (27)
This library finally has 100% compliance with the W3c tests in rdf-tests
for parsing NTriples, Turtle and RDF/XML 👍
All 1099 tests passed (1.47s)
https://travis-ci.org/robstewart57/rdf4h
Thanks to @wismill for the Turtle parser and XML parser bug fixes!
This compliance is reflected in release 4.0.0 of this library.
from rdf4h.
@robstewart57 I've just committed an update to Makefile
to get and "deploy" all W3C test files for the tests (cordawyn@75d67ca). You can finally try things out at your place. I've also updated README.md
to mention that special make task. Feel free to adjust Makefile
, as it's just hastily concocted to get things running asap.
from rdf4h.
Update on RDF/XML tests (as of b84de4a):
Test Cases Total
Passed 1 1
Failed 161 161
Total 162 162
Uh-oh :)
from rdf4h.
But we're doing slightly better on Turtle tests, with my latest fixes:
Test Cases Total
Passed 186 186
Failed 105 105
Total 291 291
from rdf4h.
So a 0.6% pass rate for the XML tests leaves room for improvement :-) A 63% pass rate for the Turtle tests is a lot more respectable. It's a great job that we've picked up the W3C tests. I'm travelling for the next two weeks, but after that I would like to tackle some of these bugs, probably starting with the failing Turtle tests given its simpler syntax.
from rdf4h.
Btw, I got a reply from @gkellogg saying they're going to fix those bugs in W3C test files. That's very nice. I'll start compiling a list of the bugs, as we find more.
from rdf4h.
NTriples test
(added in 3b1e916) fails for now - it stumbles upon some test file where it starts consuming all of RAM, then dies. I'm going to find out where exactly it fails, but it's definitely not the first file ;-)
UPDATE. The list of culprits:
<#nt-syntax-bad-esc-01>
<#nt-syntax-bad-esc-02>
<#nt-syntax-bad-esc-03>
<#nt-syntax-bad-string-01>
<#nt-syntax-bad-string-06>
<#literal>
<#literal_ascii_boundaries>
<#literal_all_controls>
<#literal_all_punctuation>
<#literal_with_BACKSPACE>
<#literal_with_FORM_FEED>
<#literal_with_UTF8_boundaries>
With the above tests disabled, I'm getting:
Test Cases Total
Passed 39 39
Failed 17 17
Total 56 56
from rdf4h.
@robstewart57 Whenever you're going to start fixing TurtleParser, could you start with #15 ? This would enable me to load "manifest.ttl" of "RDF Entailment" test suite and possibly get some test results from it. Since we don't handle TriG and N-Quads in RDF4H, this should be the last W3C test suite. (Although I don't think we're handling RDF entailment at all. But let's have the tests for it, for completeness sake).
from rdf4h.
For the sake of documentation, #15 is fixed.
from rdf4h.
At the moment when I try testing, I get an unknown pattern PositiveEntailmentTest
.
$ cabal test
Building rdf4h-1.2.7...
Preprocessing library rdf4h-1.2.7...
In-place registering rdf4h-1.2.7...
Preprocessing executable 'rdf4h' for rdf4h-1.2.7...
Preprocessing test suite 'test-rdf4h' for rdf4h-1.2.7...
Running 1 test suites...
Test suite test-rdf4h: RUNNING...
test-rdf4h: unknown TestEntry pattern in mfEntryToTest: PositiveEntailmentTest {name = "xmlsch-02-whitespace-facet-4", comment = "\n Ill-formed datatyped literals now are inconsistent.\n Used to be negative entailment.\n ", approval = UNode("http://www.w3.org/ns/rdftest#Approved"), action = UNode("data/w3c/turtle/xmlsch-02/test002.ttl"), result = LNode(TypedL(false,"http://www.w3.org/2001/XMLSchema#boolean")), entailmentRegime = "RDFS", recognizedDatatypes = [UNode("http://www.w3.org/2001/XMLSchema#int")], unrecognizedDatatypes = []}
Is this what I should be seeing at this point?
from rdf4h.
No, please do not touch RDF Entailment tests yet. You can ignore their results. Since we do not handle any kind of inference in RDF4H, I just left those tests half-way through and switched to #12 . I'll get back, of course, but I think we should finish with that issue first.
You can either comment those tests out or add some temporal stub, if that's bothering you.
The rest of W3C tests should be working, though.
from rdf4h.
How do I disable the entailment tests in order to test the rest of the W3C tests?
from rdf4h.
I've just tried my "w3test" branch, no local changes - tests run just fine. The HEAD is "a07888b Merge branch 'master' into w3tests". (It appears I did not commit RDF entailment tests themselves, so the issue that you've mentioned should not happen ;) ). Prehaps you need to update your local branch?
(Btw, you may need to uncomment "W3CNTripleTest.tests" in Test.hs
- they are going to enter an infinite loop, as I described above).
from rdf4h.
I've just tried this again, having merged my w3tests with the current head of my master ( dd936aa ). I've just re-ran make test/w3c/fetch
and then tried running the tests. The error I still get is:
Test suite test-rdf4h: RUNNING...
test-rdf4h: unknown TestEntry pattern in mfEntryToTest: NegativeEntailmentTest {name = "datatypes-intensional-xsd-integer-decimal-compatible", comment = "\n The claim that xsd:integer is a subClassOF xsd:decimal is not\n incompatible with using the intensional semantics for\n datatypes.\n ", approval = UNode("http://www.w3.org/ns/rdftest#Approved"), action = UNode("data/w3c/turtle/datatypes-intensional/test001.nt"), result = LNode(TypedL(false,"http://www.w3.org/2001/XMLSchema#boolean")), entailmentRegime = "RDFS", recognizedDatatypes = [UNode("http://www.w3.org/2001/XMLSchema#decimal"),UNode("http://www.w3.org/2001/XMLSchema#integer")], unrecognizedDatatypes = []}
Test suite test-rdf4h: FAIL
@cordawyn I don't understand why you do not also see this error, apart from if your data/w3c/turtle/manifest.ttl
has been manually edited to comment out any entries for http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#NegativeEntailmentTest
?
from rdf4h.
@robstewart57 I've just pulled the latest changes, re-downloaded W3C test files afresh and run the tests. No, I'm not getting that error :-/
I get warnings about incomplete pattern matches during compilation, but that's OK for now -- please ignore that, I'll take care of it later (when we start passing W3C tests). Hopefully, it's not the cause of your error. Could you investigate that issue on your side, since I cannot reproduce it?
(Speaking of W3C tests - they seem to have updated the files (hopefully, thanks to our input), but the turtle tests are now packed in a directory, so they get unpacked to "data/w3c/turtle/TurtleTests/". You'll have to update the path in TurtleTest.hs
. And it appears they've deleted NQuads (N4) tests too).
from rdf4h.
It doesn't happen with W3CRdfXmlTest.tests
. If I comment out the other W3C tests in Test.hs
, then I get a report:
Test Cases Total
Passed 68 68
Failed 94 94
Total 162 162
If I comment back in the W3CTurtleTest.tests
line, then I see the unknown TestEntry pattern in mfEntryToTest ...
error. My guess:
TurtleTest.hs
loads the manifest.ttl tile fromdata/w3c/turtle/
.- In
triplesToTestEntry
, the the UNodehttp://www.w3.org/2001/sw/DataAccess/tests/test-manifest#NegativeEntailmentTest
is pattern matched to callmkNegativeEntailmentTest ts rdf
. - In
manifest.ttl
, there are numerous NegativeEntailmentTest entries. An example is below. - Thus, numerous
TestEntry
values with theNegativeEntailmentTest
constructor are passed tomfEntryToTest
in TurtleTest.hs . - The
NegativeEntailmentTest
pattern is not matched inmfEntryToTest
.
The example:
<#datatypes-intensional-xsd-integer-decimal-compatible> a mf:NegativeEntailmentTest;
mf:name "datatypes-intensional-xsd-integer-decimal-compatible";
rdfs:comment """
The claim that xsd:integer is a subClassOF xsd:decimal is not
incompatible with using the intensional semantics for
datatypes.
""";
rdfs:approval rdft:Approved;
mf:entailmentRegime "RDFS" ;
mf:recognizedDatatypes ( xsd:decimal xsd:integer ) ;
mf:unrecognizedDatatypes ( ) ;
mf:action <datatypes-intensional/test001.nt>;
mf:result false .
from rdf4h.
Only the tests in the manifest of "rdf-mt", that we do not use, have NegativeEntailmentTest
entries, I double-checked that. So that UNode...
shouldn't even match anywhere. Perhaps your manifest.ttl
files are messed up or something? Could you remove "data/w3c" and download the latest W3C tests?
from rdf4h.
Aha, thanks! Yes, somehow the manifest.ttl
file had incorrect content. I might even admit to wget
ing it manually a few weeks ago from the wrong URL :-) My bad. It works nicely now, which gives me something to work on...
Test Cases Total
Passed 185 185
Failed 106 106
Total 291 291
from rdf4h.
As of deb4632 :
Test Cases Total
Passed 190 190
Failed 101 101
Total 291 291
About half the parsec combinators follow the W3C grammar rules for Turtle. The other combinators are combinations of numerous grammar rules, so are less easy to correspond (and therefore to debug). This perhaps reflects the accepted proposoals in the change log added after the first TurtleParser
implementation. http://www.w3.org/TR/turtle/#sec-changelog
from rdf4h.
As of f891ac5 :
Test Cases Total
Passed 203 203
Failed 88 88
Total 291 291
(I'm now out of action for a couple of weeks)
from rdf4h.
RDF newbie here - how concerned should I be about these failures?
from rdf4h.
Hi @ddssff
These failing cases are corner cases for the N3, TTL and RDF/XML parsers. This unit test suite is the one specified by the W3C RDF specification. It's on my TODO list to squish these parser bugs in the next month or so.
As a side point, I notice that you clones from amccausl/RDF4H . The upstream version of RDF4H is this repository at https://github.com/robstewart57/rdf4h .
Do please create Github issues to track any bugs you encounter with RDF4H.
from rdf4h.
When I can carve out some time, I'd like to get these added to our copies of the various language test suites.
from rdf4h.
Thanks, I found the newer repo a few minutes later. I will be in touch.
from rdf4h.
@robstewart57 Rob, I'm trying the latest stuff from master branch, but the tests (stack test
) seem to fail, because the turtle parser fails to parse the manifest files. I tried this in ghci:
*Data.RDF> let f = parseFile (TurtleParser (Just $ BaseUrl "") Nothing) "data/w3c/turtle/TurtleTests/manifest.ttl" :: IO (Either ParseFailure TriplesGraph)
*Data.RDF> f
Left (ParseFailure "(line 18, column 2):\nunexpected Invalid URI in Turtle parser: \"\"\nexpecting subject resource")
So it seems to stumble while reading the manifest file:
...
<> rdf:type mf:Manifest ;
...
If, however, I provide some base URI for the parser, even more terrible things happen:
*Data.RDF> let f = parseFile (TurtleParser (Just $ BaseUrl "http://example.org/") Nothing) "data/w3c/turtle/TurtleTests/manifest.ttl" :: IO (Either ParseFailure TriplesGraph)
*Data.RDF> f
*** Exception: data/w3c/turtle/TurtleTests/manifest.ttl: hGetContents: invalid argument (invalid byte sequence)
*Data.RDF> *** Error in `/home/vagrant/.stack/programs/x86_64-linux/ghc-7.10.2/lib/ghc-7.10.2/bin/ghc': double free or corruption (out): 0x00007f8970008a50 ***
Aborted
from rdf4h.
An update.. For e62e714 across all tests, i.e. API property and test cases and the W3C test cases:
Properties Test Cases Total
Passed 61 482 543
Failed 2 122 124
Total 63 604 667
Which is down from about 180 failing test cases a few months ago, prior to the current round of test driven development, so progress is good if a little steady.
from rdf4h.
The current W3C test results are:
-
NTriples
stack test --test-arguments="--pattern /parser-w3c-tests-ntriples/"
All 136 tests passed (0.14s)
-
Turtle
stack test --test-arguments="--pattern /parser-w3c-tests-turtle/"
All 596 tests passed (1.17s)
-
RDF/XML
stack test --test-arguments="--pattern /parser-w3c-tests-xml/"
47 out of 162 tests failed (1.03s)
I will close this issue ticket once all RDF/XML parser bugs have been fixed.
from rdf4h.
Related Issues (20)
- Cannot use parseFile with RDF that does not have a base URI and URI without "http:"
- Invalid RDF format when writing to file
- Invalid blank node creation when using BNode or bnode HOT 1
- Have common Vocabularies compiled into the library HOT 12
- Use rdfs:comment string to generate documentation in genVocabulary
- Parser not handling correct turtle file HOT 5
- Implement RDF Canonicalization algorithm HOT 1
- Base URI not detected with TurtleParser HOT 1
- Corner case property failure for query_match_spo
- Another corner case property failure for query_match_spo
- Can't override default prefixes in TurtleSerializer HOT 1
- TurtleSerializer baseUrl and prefix mappings effect HOT 8
- mkRdf using default prefixes HOT 5
- Build failure due to GHC option "-threaded" HOT 1
- Questions about Blank Nodes HOT 4
- Add a Namespace for schema.org HOT 2
- Exclude GPL hgal dependency HOT 10
- Aliased URI Not Properly Serialized
- Non-URI Nodes (TurtleSerializer)? HOT 2
- Bug: TurtleSerializer Does Not Properly Group Common Subjects HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rdf4h.