Comments (10)
sjd-xlnx
commented on July 1, 2024
1
Hi @michalvasko , I'm not finding it very easy to reproduce with the existing examples. Let's put this one on hold for now, and I'll try to get a standalone test case pulled together.
from sysrepo.
sjd-xlnx
commented on July 1, 2024
1
@michalvasko I'm going to close this one. I've not been able to reproduce, since cleaning sysrepo database and reinstalling.
from sysrepo.
michalvasko
commented on July 1, 2024
Please mention what exactly are the issues, post the server output. Recovery should be automatic.
from sysrepo.
sjd-xlnx
commented on July 1, 2024
I'll try to put together an example using the sysrepo/examples, but until I do that here is the basic problem I have.
I have a bunch of data, e.g. ietf-hardware, ietf-interfaces, o-ran-interfaces, etc. configured in "running" datastore. My application also pushes some operational data.
If I abnormally exit (e.g. kill -9, or exit when debugging in VS code) then next time to try to run the application, my push operational data fails, e.g.
SysrepoValidationFailedError('Invalid leafref value "eth2" - no target instance "/hw:hardware/hw:component/hw:name" with the same value., Validation failed.')
It fails on the the "validate" (even before I try to configure the operational data), so it seems the operational data was left in odd state.
Note, in above case, the "eth2" leaf it is referring to does exist in the data. If I delete that module, it just complains about something else.
If I delete the entire nearly everything, it works again.
Note, you fixed the following (#3137) for me recently, but I wonder if there is still something not right about validating operational pushed data.
PS: I'm using latest code from devel branch. I'm using the Python bindings.
from sysrepo.
michalvasko
commented on July 1, 2024
Note, in above case, the "eth2" leaf it is referring to does exist in the data.
Does it exist in the operational datastore? It being in running is not enough.
Other than that I am not sure why the validation would fail but that is the main issue here and what needs to be discovered.
from sysrepo.
sjd-xlnx
commented on July 1, 2024
Might be on to something...
My application provides operational data for the status of interfaces in the ietf-interfaces model.
My understanding is that the operational data created within a sysrepo connection is only valid for the lifetime of the connection.
Consider the following "normal" scenario...
- Start application
- In another shell, use
sysrepocfg -X -d operational -m ietf-interfaces. I can see the operational data for ietf-interfaces. - Exit application normally (Ctrl-C, SIGTERM, etc.)
- In another shell, use
sysrepocfg -X -d operational -m ietf-interfaces. There is no operational data for ietf-interfaces, because the connection was closed. - I can restart application and the "validate" succeeds.
Now consider the following "abnormal exit" scenario...
- In another shell, use
sysrepocfg -X -d operational -m ietf-interfaces. I can see the operational data for ietf-interfaces. - Exit application abnormally (SIGKILL)
- In another shell, use
sysrepocfg -X -d operational -m ietf-interfaces. I can still see the operational data for ietf-interfaces. (I guess because the connection was not closed normally) - If I restart the application, the "validate" fails.
Here is the data for ietf-interfaces from the "running" datastore...
sysrepocfg -X -m ietf-interfaces
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
<interface>
<name>eth0</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth1</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">65:70:32:ef:80:2f</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth1</port-name>
<port-number>1</port-number>
</port-reference>
</interface>
<interface>
<name>eth2</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">cc:e7:2a:11:f4:b1</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth2</port-name>
<port-number>2</port-number>
</port-reference>
</interface>
<interface>
<name>eth0.10</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
<address>
<ip>192.168.0.1</ip>
<prefix-length>24</prefix-length>
</address>
</ipv4>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth0</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">10</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth0.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth0</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth1.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth1</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">65:70:32:ef:80:2f</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth1</port-name>
<port-number>1</port-number>
</port-reference>
</interface>
<interface>
<name>eth2.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth2</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">cc:e7:2a:11:f4:b1</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth2</port-name>
<port-number>2</port-number>
</port-reference>
</interface>
</interfaces>
Here is the data for ietf-interfaces from the "operational" datastore (note, no application running here - this is the data left over after the application was killed)...
sysrepocfg -X -d operational -m ietf-interfaces
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
<interface>
<name>eth0</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth1</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">65:70:32:ef:80:2f</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth1</port-name>
<port-number>1</port-number>
</port-reference>
</interface>
<interface>
<name>eth2</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
<mac-address xmlns="urn:o-ran:interfaces:1.0">cc:e7:2a:11:f4:b1</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth2</port-name>
<port-number>2</port-number>
</port-reference>
</interface>
<interface>
<name>eth0.10</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
<address>
<ip>192.168.0.1</ip>
<prefix-length>24</prefix-length>
</address>
</ipv4>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth0</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">10</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth0.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth0</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">a1:ee:49:82:b6:78</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth0</port-name>
<port-number>0</port-number>
</port-reference>
</interface>
<interface>
<name>eth1.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth1</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">65:70:32:ef:80:2f</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth1</port-name>
<port-number>1</port-number>
</port-reference>
</interface>
<interface>
<name>eth2.20</name>
<type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:l2vlan</type>
<base-interface xmlns="urn:o-ran:interfaces:1.0">eth2</base-interface>
<vlan-id xmlns="urn:o-ran:interfaces:1.0">20</vlan-id>
<mac-address xmlns="urn:o-ran:interfaces:1.0">cc:e7:2a:11:f4:b1</mac-address>
<port-reference xmlns="urn:o-ran:interfaces:1.0">
<port-name>eth2</port-name>
<port-number>2</port-number>
</port-reference>
</interface>
</interfaces>
from sysrepo.
michalvasko
commented on July 1, 2024
Now consider the following "abnormal exit" scenario...
In another shell, use sysrepocfg -X -d operational -m ietf-interfaces. I can see the operational data for ietf-interfaces.
Exit application abnormally (SIGKILL)
In another shell, use sysrepocfg -X -d operational -m ietf-interfaces. I can still see the operational data for ietf-interfaces. (I guess because the connection was not closed normally)
If I restart the application, the "validate" fails.
I have just tried this and the data were recovered and disappeared in the second sysrepocfg call. What sysrepo version are you using?
from sysrepo.
sjd-xlnx
commented on July 1, 2024
I am using "so version" 7.19.4 from devel branch.
On Monday, I completely cleaned / re-installed sysrepo database and data, including shm_clean and removing all the directories associated with sysrepo, libyang, netopeer2, etc. and reinstalled "fresh", in case this fixed things.
Well, it does partially work now, in that it no longer fails the "validate" of operational datastore. However, I still do see the old operational data "stuck" in the system after an abnormal exit (SIGKILL).
For now at least, I can't reproduce the issue of "validate" failing (which was the original issue), but I am curious as to whether this "stuck" operational data will cause problems later on. The only way to remove it is to remove the ietf-interfaces (running config) data and this seems to cause the related ietf-interfaces operational data to get removed.
I do have some other operational data used by my system, but these don't seem to cause an issue. Only the ietf-interfaces seems to be problematic. This ietf-interfaces references ietf-hardware, and is itself referenced by a couple of other modules (o-ran-mplane-int, and o-ran-processing-elemenets). I wondered whether these leaf-refs were somehow causing issues, I tried a few things, but didn't really prove anything.
from sysrepo.
sjd-xlnx
commented on July 1, 2024
Additional information. The ietf-interfaces is augmented by o-ran-interfaces. Below is the example data that I have configured, in case there is something odd about the data (e.g. internal leaf-refs ?)
{
"ietf-interfaces:interfaces": {
"interface": [
{
"name": "eth0",
"type": "iana-if-type:ethernetCsmacd",
"o-ran-interfaces:l2-mtu": 6600,
"o-ran-interfaces:mac-address": "a1:ee:49:82:b6:78"
},
{
"name": "eth1",
"type": "iana-if-type:ethernetCsmacd",
"o-ran-interfaces:l2-mtu": 6600,
"o-ran-interfaces:mac-address": "65:70:32:ef:80:2f"
},
{
"name": "eth2",
"type": "iana-if-type:ethernetCsmacd",
"o-ran-interfaces:l2-mtu": 6600,
"o-ran-interfaces:mac-address": "cc:e7:2a:11:f4:b1"
},
{
"name": "eth1.10",
"type": "iana-if-type:l2vlan",
"ietf-ip:ipv4": {
"address": [
{
"ip": "192.168.0.1",
"prefix-length": 24
}
]
},
"o-ran-interfaces:base-interface": "eth1",
"o-ran-interfaces:vlan-id": 10,
"o-ran-interfaces:mac-address": "65:70:32:ef:80:2f"
},
{
"name": "eth1.20",
"type": "iana-if-type:l2vlan",
"o-ran-interfaces:base-interface": "eth1",
"o-ran-interfaces:vlan-id": 20,
"o-ran-interfaces:mac-address": "65:70:32:ef:80:2f"
},
{
"name": "eth2.20",
"type": "iana-if-type:l2vlan",
"o-ran-interfaces:base-interface": "eth2",
"o-ran-interfaces:vlan-id": 20,
"o-ran-interfaces:mac-address": "cc:e7:2a:11:f4:b1"
}
]
}
}
from sysrepo.
michalvasko
commented on July 1, 2024
Well, it does partially work now, in that it no longer fails the "validate" of operational datastore. However, I still do see the old operational data "stuck" in the system after an abnormal exit (SIGKILL).
Please include all the YANG modules, data, and steps to reproduce this, I could not. If you could use the examples (such as oper_data_push_example for providing the oper data), it would make it easier.
from sysrepo.
Related Issues (20)
- Unnecessary subscriptions triggered when writing in operational datastore HOT 6
- Changes applying order during commit request after edit-config to candidate broken when request contains changes for a few modules HOT 14
- Error when setting a value multiple times HOT 1
- Request for one last git tag on libyang1 branch HOT 2
- [ERR] Datastore plugin "JSON notif" not found HOT 8
- sr_shmmod_lock: Interrupted system call HOT 2
- sr_get_items crashes in libyang (lyd_lyb_data_length) HOT 3
- [ERR] JSON DS file: Retrieving GID "0" grp entry failed (No such file or directory) [ERR] Callback "init" of plugin "JSON DS file" for module "ietf netconf acm" failed HOT 5
- sr_get_items retrieve data elements that are not in the datastore. HOT 1
- Can we use APIs based on sr_val_t HOT 1
- ietf-factory-reset does not trigger module change callback HOT 1
- ThreadSanitizer: unlock of an unlocked mutex in `test_copy_config`'s `test_simple` HOT 4
- Callback "load" of plugin "JSON DS file" for module "name of the module" HOT 2
- edit-config parameter xml file invalid. HOT 3
- Merge functionality with sysrepocfg in devel HOT 1
- Recursive reader limit 255 reached HOT 2
- Detecting a close-session RPC HOT 2
- Malformed data in operational module, a huge sequence of `{"sysrepo:dup-inst-list-position":[null],"ietf-netconf:operation":"merge"}` HOT 5
- Action in augment: "Failed to send RPC" HOT 3
- Detecting subscription expiration HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sysrepo.