sshikov / ldpath Goto Github PK

Field definitions follow the pattern

    FIELDNAME = PATH :: FIELDTYPE FIELDCONF

However FILDNAME currently does not allow for URIs as it is already possible 
for the PATH and the FIELDTYPE

### Intended usage:

The simplest possible usage is for schema translation:

    @prefix schema : <http://schema.org/>;
    schema:name = rdfs:label :: xsd:string;

The field name of the result MUST be the URI

Current Situation:
---------------

Currently the apply function of NodeFunctions provides access to the RDFBackend 
as well as var-arg of type Collection<Node>

This variable arguments are created by evaluating the parameters defined in the 
LDPath program based on the current context


e.g. A graph

    urn:Me --foaf:knows--> urn:Alex
    urn:Alex --foaf:givenName--> "Alex"
    urn:Alex --foaf:familyName--> "Mustermann"

and the ld-path program

    friendName = foaf:knows/fn:concat(foaf:givenName," ",foaf:familyName) :: xsd:string

would result in the function concat to be called with the following values

    ConcatenateFunction#apply({backend},"Alex"," ","Mustermann")

thats because the evaluation of the path "foaf:knows" provides "urn:Alex" as 
result and this result is than used as context to evaluate the parameters 
defined for the "fn:concat" function in the ld-path program.


Proposed change:
--------------- 

The context should be parsed as additional parameter to the 
NodeFunction#apply(..)

Suggested Method signature:

    Collection<Node> apply(RDFBackend<Node> rdfBackend, Node context, Collection<Node>... args)

Advantages

1. currently functions that need to access the context require users to specify 
the '.' parameter. 

The best example is the "fn:content" function that is often used as 
"fn:content(.)" to ensure the the current context is available within the 
function.


2.reduced number of required parameters and better integration of methods in 
the path language

currently to get the content of the "foaf:homepage" can be written as both

    homepageContent = foaf:homepage/fn:content(.)
    homepageContent = fn:content(foaf:homepage)

or here an example using the XPath function

     homepageTitle = fn:xpath("//head/title/text()", fn:content(foaf:content)) :: xsd:string ;
     homepageTitle = fn:content(foaf:homepage)/fn:xpath("//head/title/text()",.) :: xsd:string ;

With the suggested change the parameter defining the path could be optional 
allowing calls such as

    homepageContent = foaf:homepage/fn:content()
    homepageTitle = foaf:homepage/fn:content()/fn:xpath("//head/title/text()");


Optionally one could even allow to omit '()' for functions that do not require 
parameters

    homepageContent = foaf:homepage/fn:content
    homepageTitle = foaf:homepage/fn:content/fn:xpath("//head/title/text()");


Affects on current Implementations
------------------------------

There are about 15 implementations of NodeFunction that would be affected by 
this change in the Interface. However as this change only suggest an additional 
parameter that can be safely ignored there is no immediate need to adapt such 
implementation.

However as shows by the example for some of the method it would make sense to 
use the "Node context" as default if no explicit parameter is parsed.

In case omitting of '()' for functions without parameters should be supported 
there is also a need to change the ld-path parser accordingly

This would ease use of LDPath with OSGI.

The following proposal suggests the addition of "dynamic fields" (I use this 
name because the suggested feature is somewhat similar to the same feature in 
Apache Solr).

The addition of this feature is required to use LDPath instead of the current 
[Mapping language](http://wiki.iks-project.eu/index.php/RepresentationMapping) 
within the [Apache Stanbol 
Entityhub](http://incubator.apache.org/stanbol/docs/trunk/entityhub.html).

### Dynamic Field and LDPath (proposal)

Currently ldpath only field definitions like

    FIELDNAME = PATH :: FIELDTYPE FIELDCONF

Dynamic fields would allow

    FIELD_PREFIX* = PATH_PREFIX* [:: FIELDTYPE FIELDCONF]


The typical usecase would be to select all properties of a given namespace

   dc_* = dc:* :: xsd:string

Adding this would greatly improve the useability of the already existing 
Wildcard support.

NOTE: While I am using here '*' in the sense of Wildcards one could also 
imaging to allow other kind of pattern definitions.

### Dealing with conflicts

A conflict is of the FIELDNAME of a normal field is the same as 
FIELD_PREFIX{suffix}.

In such cases the statically configured Pattern for FIELDNAME MUST HAVE 
precedence.

e.g. the program

    dc_* = dc:* :: xsd:string
    dc_title = dc:title[@en] :: xsd:string

MUST only select English titles.


### Implementation:

I would suggest to add a 

    listProperties(Node context)

to RDFBackend. This would allow the NodeSelector to filter properties of an 
node against the defined pattern.

Currently the default namespaces are hard coded in a static member variable of 
Program. Because of this it is not possible to change it.

The proposal is to allow to register namespace mappings via the LDPath class 
similar as it is already possible for NodeTransformers and SelectorFunctions

Currently a Field Definition MUST define 

* FIELDNAME : the name of the field
* PATH: the path and
* FIELDTYPE: the type of the field

This suggest to make both the FIELDNAME and the FIELDTYPE optional by defining 
reasonable defaults.

### FIELDNAME

If the FIELDNAME is not defined the first property of the specified PATH shall 
be used as default. In case this is ambiguous (e.g. because the PATH uses an 
OR, AND ...) a "LDPathParseException" MUST BE thrown.

Example 1:

    dc:title[@en] :: xsd:string;

would select all English titles in the field

    http://purl.org/dc/elements/1.1/title

Example 2: 

    foaf:knows/foaf:name | foaf:knows/rdfs:label :: xsd:string;

would select all names of fiends (regardless if defined as foaf:name or 
rdfs:label) in the field

    http://xmlns.com/foaf/0.1/knows

Example 3:

This tries to select the hierarchy by using both rdfs:subClassOf and 
skos:broader

    skos:broader | rdfs:subClassOf :: xsd:anyUri

This would result in a "LDPathParseException" because the first element of the 
path is ambiguous.


### FIELDTYPE

Currently the FIELDTYPE is a required parameter for Field definitions. All 
values selected by the PATH for a given context are converted to the specified 
field type.

Currently it is not possible to specify that <Node>'s as returned by the 
RDFBackend should be added to the result set.

Here the suggestion is to do exactly this if no FIELDTYPE is specified.
Components that need to process results could still use the already available 
is**(Node node) and "**Value(Node node)"  methods of the RDFBackend dynamically 
inspect and (if necessary) convert the selected values.

In cases one might not want to deal with Node instances in the result set one 
could add an ENVIRONMENT_PARAMETER that tells LDPath to convert values for 
fields with missing FIELDTYPE to xsd:string.


However LDPath does already specify means to d

Currently Test such as [@en] do only work on individual paths. It is not 
possible to create an union or intersection and than apply a the test that.

Adding support for that would allow for more simple LD Path statements and 
improve performance in some cases (Applying filter to individual elements might 
reduce the number of items to build the union/intersection; Applying the filter 
to the union/intersection reduces the number of times the filter must be 
applied).

e.g. 

    people = (* | dct:subject/^dct:subject | dct:subject/^skos:broader/^dct:subject)[rdf:type is dbp-ont:Person])

must be currently written as

   people = (*[rdf:type is dbp-ont:Person]) | (dct:subject/^dct:subject[rdf:type is dbp-ont:Person]) | (dct:subject/^skos:broader/^dct:subject[rdf:type is dbp-ont:Person]) :: xsd:anyURI;

Namespaces used in this Example:

    @prefix dct : <http://purl.org/dc/terms/> ;
    @prefix dbp-ont : <http://dbpedia.org/ontology/> ;
    @prefix skos : <http://www.w3.org/2004/02/skos/core#> ;

Currently Functions do have to use the the

  fn: http://www.newmedialab.at/lmf/functions/1.0/

namespace.

This is fine for the built-in functions and still ok for general purpose 
function. However for domain specific functions it would be more intuitive to 
not restrict method names to a single namespace.

The proposal is to allow the use of any URI token as function name.

This would require to update

* the LDPath parser to support uris
* the LDPath configuration to accept function that use URIs with a namespace 
different to 'fn'

To add additional functions for specific domains it would be more in

Hi all!

A small question on LDPath. Here the scenario:

@prefix mo: <http://purl.org/ontology/mo/>;
made = foaf:made :: xsd:anyURI ;
genre = foaf:made/mo:genre :: xsd:string;

I get two lists, one for "made" and one for "genre". But how can I made a 
LDPath, so that I get the results nested?


Before ----------------------- 

PROGRAM
@prefix mo: <http://purl.org/ontology/mo/>;
made = foaf:made :: xsd:anyURI ;
genre = foaf:made/mo:genre :: xsd:string ;

RESULT
{made: {<url1>, <url2>, ...}}
{genre: {pop, rock, techno}}



After ----------------------- 

PROGRAM (example: look at the variable "made")
@prefix mo: <http://purl.org/ontology/mo/>;
made = foaf:made :: xsd:anyURI ;
genre = made/mo:genre :: xsd:string ;

RESULT
{made: {<url1>: { genre: {pop, rock}}, <url2>: {genre: {techno}}}}

This can save many time to get a desired (needed) structure!

Thanks!