How to record a Value only if it has changed? about textfsm HOT 6 CLOSED

I was looking up a similar issue and yours intrigued me. How about this?

fh = StringIO(
"""Value Filldown VRF (\S+)
Value Required NEIGHBOR (\S+)
Value REMOTE_AS (\d+)
Value DESCRIPTION (.*)
Value ROUTE_MAP (\S+)
Value UPDATE_SRC (\S+)
Value MULTIHOP_NB (\d+)

Start
  ^ address-family ipv4 vrf ${VRF} -> NEIGHBORSTART

NEIGHBORSTART
  ^  neighbor ${NEIGHBOR} remote-as ${REMOTE_AS} -> NEIGHBOR

NEIGHBOR
  ^  neighbor ${NEIGHBOR} route-map ${ROUTE_MAP}
  ^  neighbor ${NEIGHBOR} description ${DESCRIPTION}
  ^  neighbor ${NEIGHBOR} update-source ${UPDATE_SRC}
  ^  neighbor ${NEIGHBOR} ebgp-multihop ${MULTIHOP_NB}
  ^  neighbor \S+ remote-as \d+ -> Continue.Record
  ^  neighbor ${NEIGHBOR} remote-as ${REMOTE_AS} 
  ^ exit-address-family -> Record Start""")

parser = textfsm.TextFSM(fh)

parser.ParseText(
""" address-family ipv4 vrf EASY-GOING
  no synchronization
  neighbor 10.42.80.2 remote-as 64442
  neighbor 10.42.80.2 inherit peer-session LTE-ASR
  neighbor 10.42.80.2 description *** some EASY-GOING router ***
  neighbor 10.42.80.2 activate
  neighbor 10.42.80.2 inherit peer-policy LTE-ASR
  neighbor 10.42.80.2 default-originate
 exit-address-family
 !
 address-family ipv4 vrf THE-MONTAIN
  import path selection all
  import path limit 8
  no synchronization
  redistribute static route-map advertise-primary
  redistribute connected route-map advertise-primary
  neighbor 10.144.139.165 remote-as 65021
  neighbor 10.144.139.165 description *** peering CASE DMZ MONTAINVPDMZ01 ***
  neighbor 10.144.139.165 route-map RESILIENCE_VIPS_EXT
  neighbor 10.144.139.166 remote-as 65021
  neighbor 10.144.139.166 ebgp-multihop 5
  neighbor 10.144.139.166 update-source Vlan508
  neighbor 10.144.139.166 route-map DENY_ALL out
 exit-address-family""")

This results in the following:

[['EASY-GOING', '10.42.80.2', '64442', '*** some EASY-GOING router ***', '', '', ''],
 ['THE-MONTAIN', '10.144.139.165', '65021', '*** peering CASE DMZ MONTAINVPDMZ01 ***', 'RESILIENCE_VIPS_EXT', '', ''],
 ['THE-MONTAIN', '10.144.139.166', '65021', '', 'DENY_ALL', 'Vlan508', '5']]

Changes to the FSM template:

Value Required NEIGHBOR (\S+)    # Make required so we don't end up with an empty row at the end due to exit-address-family triggering a record 
Value DESCRIPTION (.*)    # Wasn't matching the example descriptions you provided in config.

 ^ address-family ipv4 vrf ${VRF} -> NEIGHBORSTART    # Jump to the new state.

NEIGHBORSTART     # New state to ensure that the new neighbor attributes are scanned on the first pass.
  ^  neighbor ${NEIGHBOR} remote-as ${REMOTE_AS} -> NEIGHBOR     # New state to ensure that the new neighbor attributes are scanned on the first pass.

NEIGHBOR
  ^  neighbor \S+ remote-as \d+ -> Continue.Record    # When a new `remote-as` keyword encountered, a new neighbor record starts, but we continue to capture the new value after the current record (previous neighbor) is captured.
  ^  neighbor ${NEIGHBOR} remote-as ${REMOTE_AS}    # Moved to the bottom to make sure the first pass catches all of the first neighbor attributes. May not be necessary, but needs to at least be under the previous line.

from textfsm.

If it's not possible, could it be possible to make a PR on it ? I get that fine tunning data formatting is not the main purpose of TextFSM but it is not the first time I encounter this issue. I never figured it out and last time I just made my own parser to overcome the problem.

The thing is I like textFSM and I want to automate a lot of routers parsing using it, so if it's only a lack of understanding/skills on textFSM, feel free to explain me or at least maybe I could add some Option (e.g Target or whatever) for a given Value to skip Record unless a new Value's value is matched.

What do you think ?

from textfsm.

Thanks for the reply @jrschneider !

Your template works fine with my example indeed but I realize while reading your answer that I put a remote-as in both addess-family. Actually in the real data I am dealing with there are some address-families without remote-as. There is absolutely no pattern I can hang on to.

I hope you found an answer to your issue. I really think there should have some Value Option to record only if a Value's value has changed.

from textfsm.

Would this be a reference input for this problem? Where case 'B' has a 'remote-as' that is not the initial entry, and case 'C' is an instance with no remote-as.

A
A1 a1
A1
A2
A2 a2
A2
B
B1
B1 b1
C
C1
C1

With a desired output of:
A, A1, a1
A, A2, a2
B, B1, b1
C, C1

from textfsm.

I can see this as a useful feature to reduce template complexity in this situation and would allow a sensible output that the engine couldn't produce in certain other cases which would likely not occur in networking. Of course you could always have it record like crazy and wouldn't be too hard to parse it out in post (if I recall using lists and a post-processor was my solution to some of these problems).

Might it be useful to treat the feature request and help request separately?

For example, if we have something like you described where a/b is an IP, 1/2/3 is a non-recorded keyword, and a1 is "64442" , etc.

a, 1, a1
a, 2, a2
a, 3, a3
b, 1, b1
b, 2, b2
b, 3, b3

now consider:

a, 1, a1
a, 2, a2
b, 3, b3

How would we distinguish between A2 -> A3 versus A2 -> B3 when A3 and B3 are the same pattern: <IP> route-map <TEXT>?

I've rarely found this in actual router outputs because if B3 existed I've always seen either A3 or B1/B2 exist as well but there have been instances where numerous additional states were required to differentiate between A and B, where this feature would handle it in one state.

from textfsm.

i found this thread searching for similar sollution for networktocode/ntc-templates#784

as i understand the issue, there is no stable match for final line to reliably "-> record" and start new record

What is requires is a new action or Value e.g. Action=newOnChange , or Value=newTrigger)

Value newTrigger SEQ (\S+)
Value Filldown A (\S+)
Value Filldown B (\S+)
Value Filldown C (\S+)
Value Filldown D (\S+)
Value Filldown E (\S+)
Value Filldown F (\S+)

Start
  ^${SEQ} info A ${A} -> newOnChange ${SEQ}
  ^${SEQ} info B ${B} -> newOnChange ${SEQ}
  ^${SEQ} info C ${C} -> newOnChange ${SEQ}
   ...
  ^${SEQ} info F ${F} -> newOnChange ${SEQ}

#Another example to try and explain the issue.
Example pseudo data, only trigger for new "-> record" is change in 1st value
10 info A
10 info B
10 info C
20 info A
30 info C
40 info C
40 info E
40 info F

output
10, A=A, B=B, C=C, E="", F=""
20, A=A, B="", C="", E="", F=""
30, A="", B="", C=C, E="", F=""
40, A="", B="", C=C, E="", F=F

from textfsm.