drkane / datasette-reconcile Goto Github PK

The fts_table setting in the metadata.json is ignored, the plugin uses db.fts_table(table) which does introspection to find fts tables.

example:

{
    "databases": {
        "common": {
            "tables": {
                "species": {
                    "fts_table": "species_fts",
                    "plugins": {
                        "datasette-reconcile": {
                            "name_field": "ValidScientificName"
                        }
                    }
                }
            }
        }
    }
}

https://reconciliation-api.github.io/specs/latest/#preview-service

Currently uses /<database>/<table>/reconcile - but this could conflict in the case when a table has a row with id "reconcile".

Hello,
Thank you for your plugin. I successfully installed the plugin with Datasette 0.55 and modified the metadata.json file to add a reconcilation service on one table.

The /-/reconcile URL returns:

{
  "versions": [
    "0.1",
    "0.2"
  ],
  "name": "PRELIB Personnes reconciliation",
  "identifierSpace": "http://rdf.freebase.com/ns/type.object.id",
  "schemaSpace": "http://rdf.freebase.com/ns/type.object.id",
  "defaultTypes": "personne",
  "view": "/prelib/prelib_personne/{{id}}"
}

And /-/reconcile?queries={"q1":{"query":"Christophe"},"q2":{"query": "Hers"}} successfully returns candidates.

However, when I try to add the service in OpenRefine (Reconcile > Start reconciling > Add Standard Service and add the /-/reconcile URL), OpenRefine displays an error message: Error contacting recon service: parsererror : Error: jQuery111105708109859784188_1614767957684 was not called -

Do you have any suggestions to solve the problem ?

Thank you,

https://reconciliation-api.github.io/specs/latest/#data-extension-service

Hello,
Following #13 (thank you for solving the issue), I have a new problem: when I try to add the service in OpenRefine (Reconcile > Start reconciling > Add Standard Service and add the /-/reconcile URL), OpenRefine displays a spinner and nothing occurs. The OpenRefine console is stuck to the instruction: POST /command/core/guess-types-of-column.

I may send you the URL of the reconciliation service in a mail, if you wish (I would like to keep this URL private).

Thank you,

Hi! Thanks for sharing such a useful plugin :)

I found out that IDs containing special characters are not escaped correctly in the URL when the endpoint is datasette itself (which is the default). For example, datasette will fail to load http://localhost/db/table/FA3/N/819, but it would work with http://localhost/db/table/FA3~2FN~2F819

I think an alternative variable should be made available, which is escaped according to Datasette, which should also be the default.

def tilde_encode(s: str) -> str:
    "Returns tilde-encoded string - for example ``/foo/bar`` -> ``~2Ffoo~2Fbar``"
    return "".join(_tilde_encoder(char) for char in s.encode("utf-8"))

-- https://github.com/simonw/datasette/blob/f0fadc28ddb9f82e5cc1ecaa51e8a342eb6dc528/datasette/utils/__init__.py#L1166-L1168

Currently types are included as strings, and are not used in filtering results.

According to the spec types should be implemented as a JSON object with id and value fields.

https://reconciliation-api.github.io/specs/latest/#cross-origin-access

Hello,
As for the Wikidata reconcile API, the view template for entities could not have the same domain as the reconciliation API (OpenRefine displays Wikidata candidates with an URL as https://www.wikidata.org/wiki/{{id}}). Would it be possible to add a new plugin configuration item to eventually replace the datasette instance URL (as specified below) for the "view" by a combination of a domain given as parameter and the id_field.

To be more concise, would it be possible to modify the domain of the URLs of the candidates:

Thanks,

https://reconciliation-api.github.io/specs/latest/#dfn-matching-feature

A matching feature is a numerical or boolean value which can be used to determine how likely it is for the candidate to be the correct entity. It contains the following fields:

id
A string which identifies the feature, such as "name_tfidf" or "pagerank". This id must be unique among all the matching features returned for a given candidate;
value
The value of the feature for the candidate, which can be any boolean or numerical value.

Multiple matching features are often used in combination to provide the final matching score (available in the score field). By exposing individual features in their responses, services make it possible for clients to compute matching scores which fit their use cases better.

Particularly for requests, use the available JSON schemas to validate the queries object:

• Reconciliation query batch schema

For the manifest and results, validate them against JSON schema in tests

• Manifest schema
• Reconciliation result batch schema

https://python-jsonschema.readthedocs.io/en/stable/

https://reconciliation-api.github.io/specs/latest/#suggest-services

Either need to allow user to ignore full text search index on table, or check that name_field is part of the full text search index.

Hello,
Thank you for v0.2.0 release. It works fine with OpenRefine. However, the suggested alignments have no active links (the tags have no href):

This is probably due to the fact that the url of the view in the manifest should be absolute:

"view": {"url": "http://www.example.com/database/table/{{id}}"}

the current version looks like:

"view": {"url": "/database/table/{{id}}"}

Might be solved using https://docs.datasette.io/en/stable/internals.html?highlight=absolute#absolute-url-request-path in get_view_url ?

In reconciliation query response: https://reconciliation-api.github.io/specs/latest/#reconciliation-query-responses

simonw/datasette-jellyfish#1

Could allow configuration to include various options for testing and scoring matches.

https://reconciliation-api.github.io/specs/latest/#structure-of-a-reconciliation-query

properties
Optionally, a map from property identifiers to a list of property values (or list of property values). These are used to further filter the set of candidates (similar to a WHERE clause in SQL), by allowing clients to specify other attributes of entities that should match, beyond their name in the query field. How reconciliation services handle this further restriction ("must match all properties" or "should match some") and how it affects the score, is up to the service;

See https://reconciliation-api.github.io/specs/latest/#versions

Would potentially create breaking changes, and not sure if the latest version is compatible with clients, e.g. OpenRefine.

Could allow user to select the version(s) of the API that are supported, but this would add complexity.