There were some changes recently to protect data being passed through clixon from being misinterpreted as XML and breaking the parser (thanks for that).

We are still having problems because you need to keep track of whether a string has been encoded or decoded already - for example if I start with the string "%" and using URI encoding the encoded result is "%25" if that gets encoded again, I get "%2525" and when I'm decoding I have no way to tell whether the original was "%25" or "%"... (the same thing can happen with XML encoding too)

We believe that the best way to deal with this is to use CDATA - which essentially "quotes" the data and tells the parser to skip on to the end marker. There's no laborious or inefficient en/decoding and we can pass it through as many time as we like and it's not altered.

So, we're requesting you implement CDATA in clixon,

thanks

Olof,

Thanks for your previous fix! It looks good.
Doing some more research and tests I found that GET requests currently will not give the data back as an array if there is only one object in the list.
For example, using this command curl -G http://localhost/restconf/data/interfaces-config While only having one interface in the data store

Should look like

{ "interfaces-config": { "interface": [ { "name": "local0" } ] } }
instead of
{ "interfaces-config": { "interface": { "name": "local0" } } }

I found some examples from the RFC: https://tools.ietf.org/html/rfc8040#section-5.3.2 and here https://tools.ietf.org/html/rfc8040#appendix-B.3.9

This would be helpful when looking at documents if you need to use an array index when parsing the returned JSON. Thank you again for your help on this.

Extra RPC arguments are not sanity checked, just ignored.

I have an RPC defined in my data model:

rpc dataplane-control {
    input {
        leaf operation {
            mandatory true;
            type ngcom:service-operation;
            description
                "The operation to perform";
        }
        leaf option {
            type string;
            description
                "Option on operation";
        }
    }
    output {
        leaf stdout {
            type string;
        }
    }
}

When I invoke this RPC via restconf, the reply that is generated by the backend callback that is registered for this RPC only contains <ok/>. Restconf interprets this as an invalid reply:

  {
    "ietf-restconf:errors" : {
    "error": {
      "error-type": "application",
      "error-tag": "unknown-element",
      "error-info": {
        "bad-element": "ok"
      },
      "error-severity": "error"
      }
    }
  }

RFC 7950 section 7.14.4 says:

    If the RPC operation invocation succeeded and no output parameters
    are returned, the <rpc-reply> contains a single <ok/> element

The output parameter for the RPC is not mandatory, so a reply of <ok/> seems RFC-compliant.

This would round out this project to support the current state of the art for APIs.

Hi Olof,

I have a simple yang like:
...
list filter {
key name;
ordered-by user;
description "Define a firewall filter";
leaf name {
description "Filter name";
type string;
}
...
When I do in cli
set filter %name%
it appends to the end of a list.
It would be great if if it will be possible to insert node in an arbitrary place (before or after existing nodes)
Thanks in advance!

Submodules cannot re-use a prefix in an import statement that is already used for another imported module in the module that the submodule belongs to.
In the example below, the submodule S is included by M, and therefore cannot use the prefix "p" for another module (M2) than the one imported by M (M1). It is OK if S imports M1 using p.

submodule S{
  belongs-to "M";
  import M2 { prefix p; } <<---- Must not be p since it is used by M
}
module M{
  include S;
  import M1 { prefix p; }
}

An rpc with an input parameter that is a string with a pattern "\w[\w_-]*" did not match the value "a-b". After stepping through with gdb, I found that regcomp() fails to compile this expression. The function regexp_xsd2posix() probably needs to replace '\w' with something that is valid in POSIX ERE.

When attempting to replace this with a regex that works with regcomp(), the pattern "[0-9A-Za-z-]" also didn't compile. Moving the '-' to the last character in the bracket expression ('[0-9A-Za-z-]') worked.

The Yang range statement does not support multiple values (RFC7895 sec 9.2.4).
For example

type uint8 {
      range "1..14 | 36  | 48";
}

is not supported

When example is run the following error is shown up

clixon_cli -f /usr/local/etc/example.xml
May 27 22:21:31: yang_parse_recurse: 1704: Yang error: No matching item-netconf-acm yang files found (expected module name or absolute filename)

many of the IETF YANG models are using these statements now - there is a work-around of recoding the IETF models, but it would be useful to be able to use them as is. we will probably have several models with different features, and rather than maintain separate YANG models, just use if-feature.

the ability to set a feature might be done in the configuration file like CLICON_NACM_MODE is set or maybe in the clixon startup code when a plugin is initialized it could possibly return the features it sets. If you use the latter, I don't think you need to validate the feature name when set, just note that it's set in a list.

Instead, it should check the spec for the type, convert the string to the correct type and do a numeric comparison for nodes with a numeric type.

Hi Olof,
I just had time to try out your latest changes. It seems I am getting a segfault when starting the clixon backend service. I haven't seen anyone else come across this issue and since I was the only one using the develop branch I think it has something to do with the most recent changes. Below is a clipping of my /var/log/messages after starting clixon backend I highlighted the part where it fails. Thank you for your help with this.

Feb 6 17:53:27 localhost systemd: Starting Clixon backend...
Feb 6 17:53:27 localhost clixon_backend[1494]: cfg_event_init: Config event processing is active
Feb 6 17:53:27 localhost clixon_backend: Version: v0.1.0-315-g8dc12a4
Feb 6 17:53:27 localhost clixon_backend: Build timestamp: Mon Feb 5 15:08:29 2018 EST
Feb 6 17:53:27 localhost clixon_backend: Git Commit: 0x8dc12a48
Feb 6 17:53:27 localhost clixon_backend[1494]: master: current caps: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,35,36+ep
Feb 6 17:53:27 localhost clixon_backend[1494]: master: Preserved capabilities
Feb 6 17:53:27 localhost clixon_backend[1494]: besd_init: plugin state data initialized
Feb 6 17:53:27 localhost clixon_backend[1494]: cfg_backend_state_read: Found last start_time 1517957602
Feb 6 17:53:27 localhost clixon_backend[1494]: cfg_backend_state_read: Found 0 active change
Feb 6 17:53:27 localhost clixon_backend[1494]: cfg_backend_check_start_time: clixon_backend start state: system already running (1)
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /acl-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /bridge-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /interfaces-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /memif-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /nat-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /neighbor-state
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /route-table-state
Feb 6 17:53:28 localhost vnet[955]: tap_inject_iface_isr:517: tap_inject_iface_isr: Enable: sw_if_index 0
Feb 6 17:53:28 localhost vnet[955]: tap_inject_iface_isr:539: tap_inject_iface_isr: sw_if_index 0 - skipping hardware interface class local
Feb 6 17:53:28 localhost clixon_backend[1494]: besd_register: handler registered for /sysctl-state

> Feb 6 17:53:28 localhost clixon_backend[1494]: plugin_load: 108: Plugins: dlopen: /usr/lib64/tnsr/backend/50-kea.so: undefined symbol: json_escape#012: Bad file descriptor

> Feb 6 17:53:28 localhost clixon_backend[1494]: clixon_backend: 1494 Terminated retval:-1

> Feb 6 17:53:28 localhost kernel: clixon_backend[1494]: segfault at 1018 ip 0000000000406ab2 sp 00007ffca27d4bb0 error 4 in clixon_backend[400000+b000]

> Feb 6 17:53:28 localhost systemd: clixon-backend.service: control process exited, code=killed status=11

Feb 6 17:53:28 localhost systemd: Failed to start Clixon backend.
Feb 6 17:53:28 localhost systemd: Unit clixon-backend.service entered failed state.
Feb 6 17:53:28 localhost systemd: clixon-backend.service failed.

Hi @olofhagsand

I have been using restconf for past some time. Then, I recompiled the apps folder and after that it stopped working.
I am using develop branch
ERROR IS

root@SHUBH:/home/shubham/RESTCONF_DEVELOP/clixon/example# sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data bind/listen: Address already in use
ADDRESS ALREADY IN USE

I ALSO CHECKED FOR
'root@SHUBH:/home/shubham/RESTCONF_DEVELOP/clixon/example# grep clicon /etc/group
clicon:x:1002:root,www-data
root@SHUBH:/home/shubham/RESTCONF_DEVELOP/clixon/example# '

PLEASE HELP

Is there constraint/enforcement for leafref type to prevent dangling reference? For example, I have a leaf with type leafref referencing to an interface, i.e. eth0, CLI allows me to delete the interface eth0 when there is reference to it. Shouldn't CLI validate and prevent it?

If you have a node in your scheme that has a name of one of
the axes, it leads to an invalid path. For instance, for the child axis:

/ipsec-config/tunnel[instance='1']/crypto/ike/child/name

Sorry, I don't have the exact error now but it was something like this:

... clixon_xpath_parseerror ... line 1 ... after or before "/"

At the same time, this path is considered as valid:

/ipsec-config/tunnel[instance='1']/crypto/ike/proposals/name

Both "child" and "proposals" are lists.

Expected only one URL per RPC.

POST /restconf/operations/example-ops:reboot HTTP/1.1

But currently I can do it like this without errors:

POST /restconf/operations/abc:reboot HTTP/1.1
POST /restconf/operations/example:reboot HTTP/1.1

Links:
rfc8040#section-3.6
rfc8040#section-3.6.1

Hi Olof,
After some testing, it was brought to my attention that the /operations endpoint should provide a list of possible RPC operations. Here is the RFC reference https://tools.ietf.org/html/rfc8040#section-3.3.2

Here is the example I was given of what it should look like.

The expected result for "operations":
$ curl -X GET http://localhost/restconf/operations

{
  "operations" : { 
     "netgate-system:system-running-to-startup" : [null],
     "netgate-frr:frr-control" : [null],
     "netgate-frr:frr-daemons-gen-config" : [null],
     "netgate-kea:kea-control" : [null],
     "netgate-vpp:vpp-control" : [null],
     "netgate-bgp:bgp-gen-config" : [null],
  }
}

Currently, it returns a 502 error.

There is an example of a request and response in this section https://tools.ietf.org/html/rfc8040#section-3.1

The client might send the following:

      GET /top/restconf/operations HTTP/1.1
      Host: example.com
      Accept: application/yang-data+json

   The server might respond as follows:

      HTTP/1.1 200 OK
      Date: Thu, 26 Jan 2017 20:56:30 GMT
      Server: example-server
      Cache-Control: no-cache
      Last-Modified: Thu, 26 Jan 2017 16:00:14 GMT
      Content-Type: application/yang-data+json

      { "operations" : { "example-jukebox:play" : [null] } }

Thank you for your help with this.

Hi Olof,

So I started making some tests for RPC calls that we have set-up. I have come across a few issues that maybe you can help with.

1. If no output is specified in the yang file it should not return a body. https://tools.ietf.org/html/rfc8040#section-3.6

If the RPC operation is invoked without errors and if the "rpc" or
  "action" statement has no "output" section, the response message
  MUST NOT include a message-body and MUST send a "204 No Content"
  status-line instead.

Currently this is what I see. On an RPC with no output defined in the yang.

sjones:~/Documents/restconf-tests  curl -X POST -d '{"input" : {"length":4096,"name":"key1","algorithm":"RSA"}}' http://169.254.0.5/restconf/operations/netgate-pki:pki-private-key-generate
{
  "output": {
    "ok": null
  }
}

2. It looks like it wraps errors in "output" as well. From https://tools.ietf.org/html/rfc8040#section-7 It seems like it should not have the "output" object just start with the "rpc-error" tag.
   Below is an example of what I currently see. I think the "output" should be taken out from the response body.

sjones:~/Documents/restconf-tests  curl -X POST -d '{}' http://169.254.0.5/restconf/operations/netgate-pki:pki-private-key-generate
{
  "output": {
    "rpc-error": {
      "error-tag": "missing-element",
      "error-type": "application",
      "error-severity": "error",
      "error-message": "No name provided",
      "error-info": {
        "bad-element": "name"
      }
    }
  }
}

Let me know if these may have been fixed or changed in the develop branch. I have not had a chance to test the new changes you have made in the develop branch as it requires some changes on our end. I haven't gotten around to setting up a test environment to run the new changes.

Let me know if you need more information. Thank you.

I am currently using the example docker image and had a question.
Is there w way to communicate with the database externally?

To elaborate further, is there way t expose some ports (such as 830 for netconf, and/or 8080 for rest) that would allow communication using an external netconf or rest client?

Ref: https://tools.ietf.org/html/rfc8040#section-4

For instance, the PUT request might return "201 Created" or
"204 No Content". For now, it always returns "200 OK" if successful.

Hi @olofhagsand !!!

I have two queries...
1.)
On doing a curl -G http://localhost/restconf/data, I'm only getting some hard coded values in example_backend.c from example_statedata() . I thing on curl it should get data from datastore and should also be able to manipulate the same.
2.)
Is it possible to run the command curl -G http://localhost/restconf/data from a remote host to get or post data. What should i need to change to achieve the same, current implementation is for local host.

With Regards,
Shubham

Wanted to add something similar to interfaces.
Added a new data model similar to interfaces and stored it in example directory.
changed the Makefile.in only to add this new yang file.

After this I was expecting the CLI to atleast show up the new one after i do "set ?"

Anything else I need to do as I dint find anything sepcific for other yang files, If i need to config and display (without any other functionality)

Yang RFC 7950 defines Yang sub-command cardinality (ie how many) and if they may appear. There is no such check currently. Example, a Yang module can have 0 or 1 description, but any number of descriptions are accepted by the the yang parse currently.

Hi Olof,
While testing some of the new changes PUT the whole datastore in particular I noticed some cases it was not doing what it should. If there is already data in the datastore it is not being replaced but just being added to or modified. Here is an example

[root@localhost ~]# curl -X PUT -d '{ "data" : { "interfaces-config" : { "interface" : { "name" : "t1"}}}}' http://localhost/restconf/data
[root@localhost ~]# curl -G http://localhost/restconf/data
{
  "data": {
    "acl-config": {
      "acl-table": {
        "acl-list": [
          {
            "acl-name": "block https",
            "acl-rules": {
              "acl-rule": [
                {
                  "sequence": 20,
                  "action": "permit"
                }
              ]
            }
          }
        ]
      }
    },
    "interfaces-config": {
      "interface": [
        {
          "name": "t1"
        }
      ]
    }
  }
}
[root@localhost ~]# 

As you can see it adds interfaces-config onto the datastore instead of replacing the whole datastore. (which would remove all of acl-config )
Here is the RFC reference It should replace the whole datastore with whatever the data is. It seems if there is already data in the datastore it doesn't get replaced properly. I believe I ran into this error on other areas as well such as within just interfaces-config so /restconf/data/interfaces-config as the endpoint if I did a PUT with an interface already defined in the data store and wanted it replaced with a new interface so a different name than the one in the datastore it didn't get replaced with what I sent. I had to run a DELETE command first to get it to remove the interface and then it would let me do a PUT.

Thanks again for your help on this.

Hi,
I built the example that is included in the distribution to test restconf and netconf. While restconf works correctly:

victor@lebox:/usr/local/var/example$ curl -G http://127.0.0.1/restconf/data/interfaces
{
    "interfaces": {
      "interface": [
        {
          "name": "lo",
          "type": "ex:loopback",
          "enabled": true
        },
        { 
          "name": "test",
          "type": "ex:eth",
          "enabled": true
        }
      ]
    }
  }

I can not achieve the same with netconf. Using a very simple ncclient code I managed to connect to Cisco sandbox netconf server and query it. While using the same code on clixon, it times out.

victor@lebox:~/luigi$ ssh root@localhopython discovery.py 
urn:ietf:params:xml:ns:netconf:base:1.0
urn:ietf:params:xml:ns:netconf:capability:candidate:1:0
urn:ietf:params:xml:ns:netconf:capability:validate:1.0
urn:ietf:params:netconf:capability:xpath:1.0
urn:ietf:params:netconf:capability:notification:1.0
urn:ietf:params:netconf:capability:startup:1.0



Traceback (most recent call last):
  File "discovery.py", line 11, in <module>
    netconf_reply = m.get_config(source = 'running')
  File "/home/victor/luigi/env/lib/python3.6/site-packages/ncclient/manager.py", line 170, in wrapper
    return self.execute(op_cls, *args, **kwds)
  File "/home/victor/luigi/env/lib/python3.6/site-packages/ncclient/manager.py", line 240, in execute
    raise_mode=self._raise_mode).request(*args, **kwds)
  File "/home/victor/luigi/env/lib/python3.6/site-packages/ncclient/operations/retrieve.py", line 133, in request
    return self._request(node)
  File "/home/victor/luigi/env/lib/python3.6/site-packages/ncclient/operations/rpc.py", line 342, in _request
    raise TimeoutExpiredError('ncclient timed out while waiting for an rpc reply.')
ncclient.operations.errors.TimeoutExpiredError: ncclient timed out while waiting for an rpc reply.

Out of curiosity I enabled debug, and I can see the following line in the log:

Sep  9 10:30:35 lebox clixon_backend[12644]: backend_notify CLICON
Sep  9 10:30:35 lebox clixon_backend[12644]: from_client retval=0
Sep  9 10:30:35 lebox clixon_backend[12644]: backend_notify CLICON
Sep  9 10:30:35 lebox clixon_netconf[15911]: clicon_rpc_msg retdata:<rpc-reply><data><interfaces><interface><name>lo</name><type>ex:loopback</type><enabled>true</enabled></interface><interface><name>test</name><type>ex:eth</type><enabled>true</enabled></interface></interfaces></data></rpc-reply>
Sep  9 10:30:35 lebox clixon_backend[12644]: event_loop: FD_ISSET: local netconf client socket
Sep  9 10:30:35 lebox clixon_backend[12644]: backend_notify CLICON
Sep  9 10:30:35 lebox clixon_backend[12644]: from_client
Sep  9 10:30:35 lebox clixon_backend[12644]: backend_notify CLICON
Sep  9 10:30:35 lebox clixon_backend[12644]: from_client retval=0
Sep  9 10:30:35 lebox clixon_backend[12644]: backend_notify CLICON

It suggests that response is sent (or at least generated), but my client can't see it. How can I diagnose it further?

regards
Jan

As per https://www.w3.org/TR/2008/REC-xml-20081126/#dt-attr, the value of an XML attribute can be enclosed between double or single quotes. Clixon XML parser only supports double quotes at the moment, it should support single quotes too.

Hi Olof,

Thanks for the previous fix it is working as expected.
After doing some more tests and broadening my configurations I noticed that all GET requests return everything as a type string. For example
{
"action": "deny",
"src-last-port": "443", // parsed as type string instead of number -> should be "src-last-port" : 443,
"dst-last-port": "443",
"protocol": "tcp"
}

They should return data based on the YANG type. (string, number, boolean);
Expected value

• Type - String:
  { "string" : "name" }

• Type - Number (uint16,uint32...)
  { "number" : 400 }

• Type - boolean
  { "boolean" : false }

Hello,

I'm just discovering this project, this is great work really!
Just wondering whether implementing RFC6022 (streaming of Yang models via Netconf) is somewhat planned?

Thanks,

Sylvain

Suppose we have this field:

leaf-list ip {
    type inet:ipv4-prefix;
    ...
}

1. Assign one valid IP via REST (PUT "ip": "1.1.1.1/32" or "ip": ["1.1.1.1/32"])
2. Assign empty IP via REST (PUT "ip": "" or "ip": [""])
3. Segmentation fault

#0  __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
#1  0x00007f63252be163 in xml_cmp1 () from /lib64/libclixon.so.3
#2  0x00007f63252be4c3 in xml_search1 () from /lib64/libclixon.so.3
#3  0x00007f63252be5ee in xml_search () from /lib64/libclixon.so.3
#4  0x00007f63252bed75 in match_base_child () from /lib64/libclixon.so.3
#5  0x00007f63252c0271 in xml_diff1 () from /lib64/libclixon.so.3
#6  0x00007f63252c039b in xml_diff1 () from /lib64/libclixon.so.3
#7  0x00007f63252c039b in xml_diff1 () from /lib64/libclixon.so.3
#8  0x00007f63252c039b in xml_diff1 () from /lib64/libclixon.so.3
#9  0x00007f63252c039b in xml_diff1 () from /lib64/libclixon.so.3
#10 0x00007f63252c056a in xml_diff () from /lib64/libclixon.so.3
#11 0x00000000004083cf in validate_common ()
#12 0x0000000000408680 in candidate_commit ()
#13 0x0000000000408876 in from_client_commit ()
#14 0x0000000000407cef in from_client_msg ()
#15 0x0000000000408094 in from_client ()
#16 0x00007f63252b9fe4 in event_loop () from /lib64/libclixon.so.3
#17 0x0000000000405102 in main ()

OR

#0  __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
#1  0x00007f4b296cdfef in xml_cmp () from /lib64/libclixon.so.3
#2  0x00007f4b290fdd59 in msort_with_tmp (p=0x7ffdfe1beb50, b=0x1664310, n=2) at msort.c:83
#3  0x00007f4b290fe04c in msort_with_tmp (n=2, b=0x1664310, p=0x7ffdfe1beb50) at msort.c:45
#4  __GI___qsort_r (b=0x1664310, n=2, s=8, cmp=0x7f4b296cdeae <xml_cmp>, arg=<optimized out>) at msort.c:297
#5  0x00007f4b296ce27a in xml_sort () from /lib64/libclixon.so.3
#6  0x00007f4b296cd75c in xml_apply () from /lib64/libclixon.so.3
#7  0x00007f4b296cd795 in xml_apply () from /lib64/libclixon.so.3
#8  0x00007f4b296cd795 in xml_apply () from /lib64/libclixon.so.3
#9  0x00007f4b296cd795 in xml_apply () from /lib64/libclixon.so.3
#10 0x00007f4b296cd852 in xml_apply0 () from /lib64/libclixon.so.3
#11 0x0000000000406421 in from_client_edit_config ()
#12 0x0000000000407a89 in from_client_msg ()
#13 0x0000000000408094 in from_client ()
#14 0x00007f4b296c9fe4 in event_loop () from /lib64/libclixon.so.3
#15 0x0000000000405102 in main ()

This patch seems to solve the problem:

diff --git a/lib/src/clixon_xml_sort.c b/lib/src/clixon_xml_sort.c
index e3aa4b2..9731909 100644
--- a/lib/src/clixon_xml_sort.c
+++ b/lib/src/clixon_xml_sort.c
@@ -144,6 +144,12 @@ xml_cmp(const void* arg1,
        return 0; /* Ordered by user: maintain existing order */
     switch (y1->ys_keyword){
     case Y_LEAF_LIST: /* Match with name and value */
+       b1 = xml_body(x1);
+       b2 = xml_body(x2);
+       if (b1 == NULL || b2 == NULL) {
+           equal = 1;
+           break;
+       }
        equal = strcmp(xml_body(x1), xml_body(x2));
        break;
     case Y_LIST: /* Match with key values 
@@ -203,7 +209,8 @@ xml_cmp1(cxobj        *x,
     case Y_LEAF_LIST: /* Match with name and value */
        if (userorder && yang_find((yang_node*)y, Y_ORDERED_BY, "user") != NULL)
            *userorder=1;
-       b=xml_body(x);
+       if ((b = xml_body(x)) == NULL)
+           break; /* error case */
        match = strcmp(keyval[0], b);
        break;
     case Y_LIST: /* Match with array of key values */

After installing all dependencies, running ./configure && make, I get this error. Looks like it's looking for header file in the installed path?

bison -y -l -d -p clixon_json_parse clixon_json_parse.y # -t is debug
mv y.tab.c clixon_json_parse.tab.c
mv y.tab.h clixon_json_parse.tab.h
flex -Pclixon_json_parse clixon_json_parse.l # -d is debug
gcc -I.  -I../../lib/clixon -I../../include -I../.. -I/usr/local/include -DHAVE_CONFIG_H  -fPIC -O2 -Wall  -Wno-error -c lex.clixon_json_parse.c
gcc -I.  -I../../lib/clixon -I../../include -I../.. -I/usr/local/include -DHAVE_CONFIG_H  -fPIC -O2 -Wall  -c clixon_json_parse.tab.c
clixon_json_parse.tab.c:180:27: fatal error: clixon/clixon.h: No such file or directory
compilation terminated.
Makefile:145: recipe for target 'clixon_json_parse.tab.o' failed
make[2]: *** [clixon_json_parse.tab.o] Error 1
make[2]: Leaving directory '/home/jocke/tmp/clixon/lib/src'
Makefile:42: recipe for target 'src' failed
make[1]: *** [src] Error 2
make[1]: Leaving directory '/home/jocke/tmp/clixon/lib'
Makefile:49: recipe for target 'lib' failed
make: *** [lib] Error 2

I'm building on Ubuntu 16.04, x86_64.

see @dmittryvvv comment on closed #30

RFC-8040 states the proper specification for an operation is

POST /restconf/operations/<module_name>:<rpc_procedure> HTTP/1.1

but clixon requires

POST /restconf/operations/<module_prefix>:<rpc_procedure> HTTP/1.1

the type statements at lines 122 & 129 are missing the yang: prefix which breaks validating.

the latest revision of this file seems to be [email protected], but this file bears little resemblance to the version supplied.

i can't find a source for the supplied version, so i don't know if this is an error in the source or in the clixon repo. a fix to the supplied version is trivially to add "yang:" to add the date-and-time types, but since this is in ietf namespace, in theory the latest version should be used.

passing a string containing a "<" character will break the RPC parsing:

Apr 17 10:08:53 ng1 clixon_backend[31155]: clixon_xml_parseerror: 188: XML error: xml_parse: line 0: syntax error: at or before: .

It appears that the character is not escaped and confuses the RPC parser. It is possible to require the sender to encode and receiver to decode the data, but this would not protect against a malicious injection.

Hi @olofhagsand

I have been using restconf for past some time and everything working fine. I compile the package with all other dependencies on openwrt platform for ARM. And I'm able to compile it, then problem is when I'm trying to run the clixon_backend on the target it hangs unexpectedly even when I'm trying the same with -F (foreground) options.

You can have a look.

'
root@OpenWrt:/www-data# clixon_backend -F -s init -D 4 -lf/www-data/.backend.log -f /www-data/configure_restconf/conf.xml -y /www-data/configure_restconf/example.yang

'
Not able to see anything on stdout or on log file.

Any kind of help would be appreciable.

Regards,
Shubh

Hi!

I tried running below command, it prompted clixon_backend: invalid option -- 'I', when I did clixon_backend --help there is no option as "-I"

clixon_backend -f /usr/local/etc/example.xml -I

Is it wrong??? what should I do to test restconf on localhost???

sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data

And the above command is prompting bind/listen: Address already in use

Regards,
Shubham

Hi All,

I've faced with a strange behaviour, I've changed example_cli.cli:

CLICON_MODE="example:main";
CLICON_PROMPT="%U@%H> ";
CLICON_PLUGIN="example_cli";

treename="main";
example("This is a comment") <var:int32>("Just a random number"), mycallback("myarg");

treename="example";
# Translate variable "value" by incrementing its characters
translate value (<value:string translate:incstr()>),cli_set("/translate/value");

and also changed line <CLICON_CLI_MODE>main</CLICON_CLI_MODE> in example.xml

after standart running #clixon_cli -f example.xml
I see "example" function in menu as expected, but when I execute it, callback function "mycallback" is NOT invocked.

Does somebody know why it's happened?

Thank you,
Maksym

Got latest clone.....
root@ubuntu> set interface

root@ubuntu> set interface testing
root@ubuntu> commit
root@ubuntu> show config
root@ubuntu>

This used to work earlier.

Backend is running fine.
root 27411 1 0 14:47 ? 00:00:00 clixon_backend -f /usr/local/etc/example.xml

An example of this problem exists in the ietf-interfaces YANG.

In clixon it seems I can put anything into /interfaces/interface=/type field and it is happy. clixon should validate those values to those allowed by the YANG model.

For interface types you can look at iana-if-type.yang.

I have compiled cligen and clixon and configured to install in my home directory. I have successfully compiled and installed "example" as well. The compilation and install was successful, however when I try to connect to clixon_backend I get an error example.sock not found. I do not see it getting generated in var/example/ folder as well. Is the file packaged in clixon or auto generated?

clixon is removing end-of-line characters from XML data improperly

according to the XML spec sec 2.11 (https://www.w3.org/TR/REC-xml/#sec-line-ends) the end-of-line markers should normalize the end-of-line characters to a single #xA, but not remove them.

Following the discussion on PR #45.

Referring to https://www.w3.org/TR/2008/REC-xml-20081126/#syntax
In that document, the authors always put spaces before and after double quote signs, so we should ignore these spaces.
Look at other sentences like:

The string " AT&T; " expands to " AT&T; " and the remaining ampersand is not recognized as an entity-reference delimiter.

Hi @olofhagsand !!!

I'm trying to run RESTCONF on my localhost, but on doing a curl, it is giving 502 Bad Gateway
.Can you figure out, what am I missing on configuring restconf???? Some output for running processes are as follows (for clixon & for nginx)...

AS ROOT : 01:31 PM : yang :) ps aux | grep [c]lixon
root 10189 0.0 0.0 28408 396 pts/0 Ss+ Jul11 0:00 /usr/sbin/clixon_backend -F -f /data/clixon.conf
root 10470 0.0 0.0 28412 484 pts/0 Ss+ Jul11 0:00 /usr/sbin/clixon_backend -F -f /data/clixon.conf
root 11228 0.0 0.0 28412 488 pts/0 Ss+ Jul11 0:00 /usr/sbin/clixon_backend -F -f /data/clixon.conf
root 16151 0.0 0.1 65612 4332 pts/4 S+ 11:26 0:00 sudo su -c /www-data/clixon_restconf -f /usr/local/etc/example.xml -s /bin/sh root
root 16152 0.0 0.1 65140 4036 pts/4 S+ 11:26 0:00 su -c /www-data/clixon_restconf -f /usr/local/etc/example.xml -s /bin/sh root
root 16165 0.0 0.0 4628 808 ? Ss 11:26 0:00 sh -c /www-data/clixon_restconf -f /usr/local/etc/example.xml
root 16166 0.0 0.0 23648 3300 ? S 11:26 0:00 /www-data/clixon_restconf -f /usr/local/etc/example.xml
root 18001 0.0 0.0 30528 3392 pts/1 S+ 12:49 0:00 clixon_backend -Ff /usr/local/etc/example.xml
root 18015 0.0 0.0 24376 3412 pts/2 S+ 12:50 0:00 clixon_netconf -f /usr/local/etc/example.xml
AS ROOT : 01:31 PM : yang :) ps aux | grep [n]ginx
root 16416 0.0 0.0 140656 1504 ? Ss 11:37 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 16417 0.0 0.1 143272 6272 ? S 11:37 0:00 nginx: worker process
www-data 16418 0.0 0.1 143272 6272 ? S 11:37 0:00 nginx: worker process
AS ROOT : 01:31 PM : yang :)
AS ROOT : 01:31 PM : yang :)
AS ROOT : 01:31 PM : yang :) curl -G http://127.0.0.1/restconf/data

502 Bad Gateway

502 Bad Gateway

Regards,
Shubham

When returning state XML nodes to the FE, they should not be sorted.

RFC 7950 Section 7.7.7 says:

7.7.7. The "ordered-by" Statement

The "ordered-by" statement defines whether the order of entries
within a list are determined by the user or the system. The argument
is one of the strings "system" or "user". If not present, ordering
defaults to "system".

This statement is ignored if the list represents state data, RPC
output parameters, or notification content.

Hi Olof,

Doing some tests with POST and RESTCONF we found that if a commit does not validate against the backend it still gets stored in the candidate DB but not in the running DB. POST/DELETE/PUT should roll back the candidate DB if it does not validate.

Here is output from /var/log/messages

Jan 22 17:40:09 localhost clixon_backend[1335]: transaction_commit: 158: Config error: Invalid sw_if_index
Jan 22 17:40:09 localhost clixon_restconf[21077]: Commit failed. Edit and try again or discard changes: protocol invalid-value Invalid sw_if_index
This makes it so any HTTP request fails until the DB is cleared or fixed.

When converting from XML to JSON formats, CDATA should be unwrapped and and escaped characters should be decoded prior to encoding the string for JSON.

the inverse conversion is trickier, but should also be resolved.

Following RFC6241, it would be great:

• to support writable-running capability (being able to write directly into the running datastore with an edit-config or copy-config operation, and to have this processed by the backend callbacks)
• to support confirmed-commit capability.

As discussed in #39, the namespaces should be properly tagged using the xmlns attributes in the various XML representations, based on the corresponding Yang models.

As discussed in #39, and as described in RFC6242, section 4, Netconf defines two framing modes. Clixon only support the legacy one (based on ]]>]]> sequence). It would be great to support equally the new one (and thus have the base:1.1 capability).

Hi,
I have been learning clixon for last few days and it is an impressive project. I am following the readme, and I got to the Datastore section. I am probably missing something, but my current mental model on how datastore_client should work is as follows:

1. I modify data in any datastore using the datastore_client app
2. Changes are reflected in cli

At this point it seems that the above assumption is not valid:

1. I created a basic configuration using cli:
   root@lebox> show configuration interfaces { interface { name eth1; type ex:loopback; enabled true; } }
2. I can retrieve that configuration using datastore_client:
   sudo ./datastore_client -d running -b /usr/local/var/example/ -p /home/victor/clixon/datastore/text/text.so -y /usr/local/share/example/yang/ -m example get /
<config><interfaces><interface><name>eth1</name><type>ex:loopback</type><enabled>true</enabled></interface></interfaces></config>
3. I can add an item to the configuration:
   victor@lebox:~/clixon/datastore$ sudo ./datastore_client -d running -b /usr/local/var/example/ -p /home/victor/clixon/datastore/text/text.so -y /usr/local/share/example/yang/ -m ietf-ip put merge '<config><interfaces><interface><name>eth2</name><type>ex:loopback</type><enabled>true</enabled></interface></interfaces></config>' victor@lebox:~/clixon/datastore$ sudo ./datastore_client -d running -b /usr/local/var/example/ -p /home/victor/clixon/datastore/text/text.so -y /usr/local/share/example/yang/ -m example get / <config><interfaces><interface><name>eth1</name><type>ex:loopback</type><enabled>true</enabled></interface><interface><name>eth2</name><type>ex:loopback</type><enabled>true</enabled></interface></interfaces></config>
4. The configuration change made in step 3 is not reflected in cli.
   Please advise what did I miss in that workflow, or is my understanding on how datastore_client should work flawed.

Regards
Jan