drkane / datasette-reconcile Goto Github PK
View Code? Open in Web Editor NEWAdds a reconciliation API endpoint to Datasette, based on the Reconciliation Service API specification.
License: MIT License
Adds a reconciliation API endpoint to Datasette, based on the Reconciliation Service API specification.
License: MIT License
The fts_table
setting in the metadata.json is ignored, the plugin uses db.fts_table(table)
which does introspection to find fts tables.
example:
{
"databases": {
"common": {
"tables": {
"species": {
"fts_table": "species_fts",
"plugins": {
"datasette-reconcile": {
"name_field": "ValidScientificName"
}
}
}
}
}
}
}
Currently uses /<database>/<table>/reconcile
- but this could conflict in the case when a table has a row with id "reconcile".
Hello,
Thank you for your plugin. I successfully installed the plugin with Datasette 0.55 and modified the metadata.json file to add a reconcilation service on one table.
The /-/reconcile URL returns:
{
"versions": [
"0.1",
"0.2"
],
"name": "PRELIB Personnes reconciliation",
"identifierSpace": "http://rdf.freebase.com/ns/type.object.id",
"schemaSpace": "http://rdf.freebase.com/ns/type.object.id",
"defaultTypes": "personne",
"view": "/prelib/prelib_personne/{{id}}"
}
And /-/reconcile?queries={"q1":{"query":"Christophe"},"q2":{"query": "Hers"}} successfully returns candidates.
However, when I try to add the service in OpenRefine (Reconcile > Start reconciling > Add Standard Service and add the /-/reconcile URL), OpenRefine displays an error message: Error contacting recon service: parsererror : Error: jQuery111105708109859784188_1614767957684 was not called -
Do you have any suggestions to solve the problem ?
Thank you,
Hello,
Following #13 (thank you for solving the issue), I have a new problem: when I try to add the service in OpenRefine (Reconcile > Start reconciling > Add Standard Service and add the /-/reconcile URL), OpenRefine displays a spinner and nothing occurs. The OpenRefine console is stuck to the instruction: POST /command/core/guess-types-of-column.
I may send you the URL of the reconciliation service in a mail, if you wish (I would like to keep this URL private).
Thank you,
Hi! Thanks for sharing such a useful plugin :)
I found out that IDs containing special characters are not escaped correctly in the URL when the endpoint is datasette itself (which is the default). For example, datasette will fail to load http://localhost/db/table/FA3/N/819
, but it would work with http://localhost/db/table/FA3~2FN~2F819
I think an alternative variable should be made available, which is escaped according to Datasette, which should also be the default.
def tilde_encode(s: str) -> str:
"Returns tilde-encoded string - for example ``/foo/bar`` -> ``~2Ffoo~2Fbar``"
return "".join(_tilde_encoder(char) for char in s.encode("utf-8"))
Currently types are included as strings, and are not used in filtering results.
According to the spec types should be implemented as a JSON object with id
and value
fields.
Hello,
As for the Wikidata reconcile API, the view template for entities could not have the same domain as the reconciliation API (OpenRefine displays Wikidata candidates with an URL as https://www.wikidata.org/wiki/{{id}}). Would it be possible to add a new plugin configuration item to eventually replace the datasette instance URL (as specified below) for the "view" by a combination of a domain given as parameter and the id_field.
To be more concise, would it be possible to modify the domain of the URLs of the candidates:
Thanks,
https://reconciliation-api.github.io/specs/latest/#dfn-matching-feature
A matching feature is a numerical or boolean value which can be used to determine how likely it is for the candidate to be the correct entity. It contains the following fields:
id
A string which identifies the feature, such as "name_tfidf" or "pagerank". This id must be unique among all the matching features returned for a given candidate;
value
The value of the feature for the candidate, which can be any boolean or numerical value.Multiple matching features are often used in combination to provide the final matching score (available in the score field). By exposing individual features in their responses, services make it possible for clients to compute matching scores which fit their use cases better.
Particularly for requests, use the available JSON schemas to validate the queries object:
For the manifest and results, validate them against JSON schema in tests
Either need to allow user to ignore full text search index on table, or check that name_field
is part of the full text search index.
Hello,
Thank you for v0.2.0 release. It works fine with OpenRefine. However, the suggested alignments have no active links (the tags have no href):
This is probably due to the fact that the url of the view in the manifest should be absolute:
"view": {"url": "http://www.example.com/database/table/{{id}}"}
the current version looks like:
"view": {"url": "/database/table/{{id}}"}
Might be solved using https://docs.datasette.io/en/stable/internals.html?highlight=absolute#absolute-url-request-path in get_view_url ?
In reconciliation query response: https://reconciliation-api.github.io/specs/latest/#reconciliation-query-responses
Could allow configuration to include various options for testing and scoring matches.
https://reconciliation-api.github.io/specs/latest/#structure-of-a-reconciliation-query
properties
Optionally, a map from property identifiers to a list of property values (or list of property values). These are used to further filter the set of candidates (similar to a WHERE clause in SQL), by allowing clients to specify other attributes of entities that should match, beyond their name in the query field. How reconciliation services handle this further restriction ("must match all properties" or "should match some") and how it affects the score, is up to the service;
See https://reconciliation-api.github.io/specs/latest/#versions
Would potentially create breaking changes, and not sure if the latest version is compatible with clients, e.g. OpenRefine.
Could allow user to select the version(s) of the API that are supported, but this would add complexity.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.