Git Product home page Git Product logo

nd2-worker-plugin-aci's Introduction

nd2-worker-plugin-aci

Netdisco plugin to fetch mac and arp tables from APIC (Cisco ACI SDN controllers)

Limitations

Tested or reported to work on APIC 3.2, 4.x and up to 5.2(2f). You're very welcome to experiment and fork or contribute, but expect stuff to fail and support to be very limited. The current release requires Netdisco 2.062003 or newer.

Description

This module can talk to APIC to:

  • emulate the regular Netdisco SNMP-based macsuck to map fvCEp MAC entries to fabric switch ports
    • in the case of virtual port channel, map the VPC MAC relation onto the corresponding port-channels on the fabric switch
  • emulate the regular Netdisco SNMP-based arpnip to store fvCEp MAC/IP pairs like normal arp table entries
  • enhance Netdisco discovery to enable L2 and L3 layers on the controller, so it is eligible for arpnip and macsuck jobs
  • populate Netdisco custom_fields with ACI-specifc attributes
  • stores model and serial numbers of fabric extenders, which seems to be impossible with SNMP-only means

Installation

Prerequisites

cpanm JSON LWP::Protocol::https File::Slurp URL::Encode REST::Client Regexp::Common

The installed Netdisco version should be at least 2.039031.

Clone the this git repository

Can be in any location, e.g:

cd /home/netdisco 
git clone https://github.com/rc9000/nd2-worker-plugin-aci.git

Configure Netdisco

Add the following settings to deployment.yml to activate the extension

include_paths: [ '/home/netdisco/nd2-worker-plugin-aci/lib' ]
extra_worker_plugins: ['X::Macsuck::Nodes', 'X::Arpnip::Nodes', 'X::Discover::Properties', 'X::Discover::FabricDevices']

Then for each controller, add an entry to device_auth

device_auth:
  - tag: aci
    driver: netconf
    only:
      - 10.5.199.5 
    user: 'aciuser'
    password: 'topsecret'
    # preferred hostname to use in https api connection, to match cert, cnames, AAAA records etc.
    # connection will use the ip address if not specified
    https_hostname: 'aci-prod.example.local'

The setting aci_ignore_uplink_re allows to specify a regex matching a remote_type. If the match succeeds, macsuck will collect entries from the port even if it is discovered as uplink. To see the current values in the database:

 $ netdisco-do psql
netdisco=> select ip, port, remote_type from device_port where ip =  '10.219.2.76' and remote_type is not null order by port;

     ip      |      port       |                          remote_type
-------------+-----------------+----------------------------------------------------------------
 10.219.2.76 | Ethernet1/2     | NetApp HCI H410S-1 Storage Node, Release Element Software 12.7
 10.219.2.76 | Ethernet1/51    | topology/pod-1/node-101
 10.219.2.76 | Ethernet1/52    | topology/pod-1/node-102
 10.219.2.76 | Ethernet103/1/6 | NetApp HCI H410S-1 Storage Node, Release Element Software 12.7

Some examples to apply a regex to these:

# Treat FlexFabric and NetApp trunks as connection to nodes:
aci_ignore_uplink_re: "(NetApp|FlexFabric)"

# Only treat ACI internal connections as uplinks:
aci_ignore_uplink_re: "^(?!topology).*$"

To display some gathered ACI attributes also in the Netdisco UI, enable these custom fields:

custom_fields:
  device:
    - editable: false
      name: APIC
    - editable: false
      name: topSystem_dn
    - editable: false
      name: topSystem_role
  device_port:
    - editable: false
      name: l1PhysIf_dn
    - editable: false
      name: l1PhysIf_name
    - editable: false
      name: pcAggrIf_dn
    - editable: false
      name: pcAggrIf_name
    - editable: false
      name: eqptExtCh
    - editable: false
      name: epgs

Controller Configuration

Since we'll want to keep the controller as a managed device in Netdisco, it needs to respond to SNMP and be successfully discovered. Also, all the switches in the fabric must be discovered in Netdisco as normal SNMP-based devices, so that the mac and arp entries can be mapped to the correct device port. See the documentation at https://github.com/netdisco/netdisco/ if unsure how to achieve this.

Running, Validation

To test if everything works, try a manual disovery. It should look like this:

$ netdisco-do discover -d 10.5.199.5
[7888] 2019-02-28 20:17:38  info App::Netdisco version 2.039031 loaded.
[7888] 2019-02-28 20:17:39  info discover: [10.5.199.5] started at Thu Feb 28 21:17:39 2019
[7888] 2019-02-28 20:17:41  info  [10.5.199.5] NetdiscoX::Properties found an ACIController - running device.layers fixup
[7888] 2019-02-28 20:17:41  info discover: finished at Thu Feb 28 21:17:41 2019
[7888] 2019-02-28 20:17:41  info discover: status done: Ended discover for 10.5.199.5

As a next step, manual arpnip and macusck can be run:

$ netdisco-do macsuck -d 10.5.199.5
[9379] 2019-02-28 20:21:34  info App::Netdisco version 2.039031 loaded.
[9379] 2019-02-28 20:21:35  info macsuck: [10.5.199.5] started at Thu Feb 28 21:21:35 2019
[9379] 2019-02-28 20:21:35  info  [10.5.199.5] cisco aci macsuck - fetching data from  https://10.5.199.5:443/api
[9379] 2019-02-28 20:21:38  info  [10.5.199.5] cisco aci macsuck - updating node table
[9379] 2019-02-28 20:22:09  info  [10.5.199.5] cisco aci macsuck - stored 2762 node entries
[9379] 2019-02-28 20:22:10  info macsuck: finished at Thu Feb 28 21:22:10 2019
[9379] 2019-02-28 20:22:10  info macsuck: status done: OK


$ netdisco-do arpnip -d 10.5.199.5
[11310] 2019-02-28 20:23:24  info App::Netdisco version 2.039031 loaded.
[11310] 2019-02-28 20:23:25  info arpnip: [10.5.199.5] started at Thu Feb 28 21:23:25 2019
[11310] 2019-02-28 20:23:25  info  [10.5.199.5] cisco aci arpnip - fetching data from  https://10.5.199.5:443/api
[11310] 2019-02-28 20:23:26  info  [10.5.199.5] cisco aci arpnip - updating node_ip table
[11310] 2019-02-28 20:23:29  info  [10.5.199.5] cisco aci arpnip - processed 0 node_ip entries
[11310] 2019-02-28 20:23:30  info arpnip: finished at Thu Feb 28 21:23:30 2019
[11310] 2019-02-28 20:23:30  info arpnip: status done: OK

Once this has proven successful, Netdisco will poll the controller in exactly the same way as an SNMP based router or switch, according to the schedule settings.

Storing device and port dn, EPG and other ACI attributes

  • when running discover, the Discover::FabricDevices module will store various ACI information in the custom_fields structure of Netdisco
  • macsuck also stores the EPG an interface participates in, in the device_port.custom_fields.epgs array

Debugging & Troubleshooting

You can run netdisco-do with these additional variables and flags:

 NETDISCOX_EPFILTER=protpaths-1067 NETDISCOX_DUMPJSON=1 ND2_LOG_PLUGINS=1 netdisco-do macsuck -D -d 10.5.199.5 

This will produce a lot of debug output (-D), show details regarding the plugin loading process (ND2_LOG_PLUGINS), as well as store the JSON replies received from ACI in files in /tmp (NETDISCOX_DUMPJSON). NETDISCOX_EPFILTER is a regex that will be matched against the fvCEp tDn, so you can limit the run to certain objects or nodes.

Also check out the ./t and ./t/testdata directories, you can run offline tests against the output produced by your fabric (with NETDISCOX_DUMPJSON). To contribute test data, ./t/testdata/anon.pl can be used to remove all identifying information from the files.

In case of errors like SSL connect attempt failed error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed at LWP/Protocol/http.pm line 50., either install the proper CA cert or set the environment variable PERL_LWP_SSL_VERIFY_HOSTNAME=0 (not recommended). Also consider the device_auth.https_hostname to match the hostname in the certificate.

nd2-worker-plugin-aci's People

Contributors

rc9000 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

haugsvar

nd2-worker-plugin-aci's Issues

Use of uninitialized value

Hey,

upon use, this error is thrown. The device is snmp reachable, netdisco has the correct snmp credentials.

[516674] 2022-10-25 06:33:48  info App::Netdisco version 2.057008 loaded.
[516674] 2022-10-25 06:33:49  info discover: [10.10.10.10] started at Tue Oct 25 08:33:49 2022
Use of uninitialized value in string eq at /data/netdisco/nd2-worker-plugin-aci/lib/App/NetdiscoX/Worker/Plugin/Discover/Properties.pm line 14.
[516674] 2022-10-25 06:33:49  info  [10.10.10.10] NetdiscoX::Properties - not an ACIController, ignored
[516674] 2022-10-25 06:33:49  info discover: finished at Tue Oct 25 08:33:49 2022
[516674] 2022-10-25 06:33:49  info discover: status info: skip: driver or action not applicable

MACs not learned from APIC

Hi,

I installed this plugin and I can see ACI Switches and Controller in Netdisco. Also the custom fields are filled with correct information.
But I can't see the MAC-Adresses of connected devices in Netdisco.
I'm facing two problems I think.
First of all, if Unicast-Routing (L3) is inactive for a Bridge Domain, Learned MACs do not appear in any way (not in netdisco, nor in a macsuck-log)
If Unicast Routing is acitve, I can see the MAC in macsuck output:
debug [apic] cisco aci macsuck - xx:xx:xx:24:B8:2E fabric switch xxx.xxx.xxx.xxx port Ethernet1/5 is an uplink, skipped
If I check the database, every connected Leaf-Port is marked as uplink.
In this case, there is the setting aci_ignore_uplink_re.

Can someone give me an example value for this setting? I would like to mark only remote_types starting with "topology" as uplinks.

Thanks in advance.

macsuck: status error: Can't call method "make_column_dirty"

Hello,

Getting this when trying to macsuck now

Unsure if a recent netdisco upgrade broke it as it was previously working Any ideas?

ACI Version 5.2(7g) - Was previously working with this
Netdisco version - 2.072003

[3668107] 2024-02-16 18:00:27  info App::Netdisco version 2.072003 loaded.
[3668107] 2024-02-16 18:00:27  info macsuck: [172.27.129.2] started at Fri Feb 16 18:00:27 2024
[3668107] 2024-02-16 18:00:27  info  [172.27.129.2] cisco aci macsuck - fetching data from  https://172.27.129.2:443/api
Redundant argument in sprintf at /home/netdisco/nd2-worker-plugin-aci/lib/App/NetdiscoX/Worker/Plugin/Macsuck/Nodes.pm line 64.
[3668107] 2024-02-16 18:00:30  info  [172.27.129.2] cisco aci macsuck - updating epg participation in device_port
[3668107] 2024-02-16 18:00:30  info macsuck: finished at Fri Feb 16 18:00:30 2024
[3668107] 2024-02-16 18:00:30  info macsuck: status error: Can't call method "make_column_dirty" on an undefined value at /home/netdisco/nd2-worker-plugin-aci/lib/App/NetdiscoX/Worker/Plugin/Macsuck/Nodes.pm line 24.

APIC cookie requirement

Good afternoon, thank you for writing this Netdisco plugin, I have gotten it to work with our newer 5.x deployment somewhat but have run into some roadblocks I was hoping you could give some insight on.

When querying it is looking for a cookie and not finding it so failing on some things.

"text" : "Need a valid webtoken cookie (named APIC-Cookie) or a signed request with signature in the cookie APIC-Request-Signature for all REST AP
I requests",
"code" : "403"

Is there anything I can adjust to make it get and use a cookie?

Thank you

Possible fix: use rDNS to avoid "certificate verify failed" errors

Hi :)

I noticed something after providing our APICs with corret certificates and setting the correct HTTPS_CA_FILE ENV Variable to use the correct CA. The macsuck/arpnip would STILL fail, because we do not include IP Addresses in the certificates, and the current code will always build the "URL" using the IP address of the APIC node. see:

return "https://" . $self->{host} . ":" . $self->{port} . "/api";

I have rewritten this a litttle bit for our purpose, to use rDNS to resolve the hostname to that IP and use THAT to connect, to avoid the otherwise unavoidable certificate error:

diff --git a/lib/App/NetdiscoX/Util/ACI.pm b/lib/App/NetdiscoX/Util/ACI.pm
index 15bf3cd..a743f86 100644
--- a/lib/App/NetdiscoX/Util/ACI.pm
+++ b/lib/App/NetdiscoX/Util/ACI.pm
@@ -12,7 +12,7 @@ use URL::Encode ':all';
 use Hash::Merge qw(merge);
 use Moo;
 use namespace::clean;
-
+use Socket;
 
 has port => (
   is  => 'rw',
@@ -31,7 +31,7 @@ has [qw(host user password)] => (
 
 sub url {
   my $self = shift;
-  return "https://" . $self->{host} .  ":" . $self->{port} . "/api";
+  return "https://" . gethostbyaddr(inet_aton($self->{host}), 2) .  ":" . $self->{port} . "/api";
 }
 
 sub nodeinfo {

This of course required that the IP have a correct PTR in the DNS, but fixes the other issue.
I wrote this instead of a direct pull request to try and explain my reasoning first :)

Maybe someone comes up with a better Idea.
Additional Suggestion: Mention HTTPS_CA_FILE Env variable (maybe even default it to /etc/ssl/certs/ca-certificates.crt) as correct way to add certificate in README.md. Other methods (at least under Ubuntu) seem to be entirely useless.

arpnip not working

Thank you so such for this plugin, it really help us integrating the cisco ACI to netdisco.

It seems like the json has changed, the `$d->{fvCEp}->{attributes}->{ip} doesn't exist, and the ip information is now inside @{$d->{fvCEp}->{children}}{fvIp}->{attributes}->{addr}

I made the following changes to the code that seems to work, and also works for nodes with multiple ip (ip alias) too:

diff --git a/lib/App/NetdiscoX/Util/ACI.pm b/lib/App/NetdiscoX/Util/ACI.pm
index fa73dbe..3241bbf 100644
--- a/lib/App/NetdiscoX/Util/ACI.pm
+++ b/lib/App/NetdiscoX/Util/ACI.pm

@@ -71,15 +72,21 @@ sub read_info_from_json {
 
   my $ts = $tsj->{imdata};
   my @node_records; 
-  my @nodeip_records; 
+  my @nodeip_records;
 
   foreach my $d (@{$resp->{imdata}}){
     my $dn = $d->{fvCEp}->{attributes}->{dn};
     my $mac = $d->{fvCEp}->{attributes}->{mac};
-    my $ip = $d->{fvCEp}->{attributes}->{ip};
+#    my $ip = $d->{fvCEp}->{attributes}->{ip};
     my $vlan = $d->{fvCEp}->{attributes}->{encap} ? $d->{fvCEp}->{attributes}->{encap} : 0;
     $vlan =~ s/vlan-//;
 
+
+    foreach my $child_fvCEp ( @{$d->{fvCEp}->{children}} ) {
+       #       print Dumper($child_fvCEp->{fvIp}->{attributes}->{addr});
+       my $ip = $child_fvCEp->{fvIp}->{attributes}->{addr};
+       next unless $ip;
+
     my @child_tdns = map { $_->{fvRsCEpToPathEp}->{attributes}->{tDn} } @{$d->{fvCEp}->{children}};
     
     foreach my $c (@child_tdns){
@@ -161,7 +168,7 @@ sub read_info_from_json {
       }
     }
   }
-
+  }
   return { nodes => \@node_records, node_ips => \@nodeip_records };
 }

Plugin not getting fabric switch ips

Hi,

great plugin but in our infrastructure it is is not working ok as I get these messages (-D option used on macsuck):
...
cisco aci macsuck - F8:72:EA:66:F2:8D fabric switch 0.0.0.0 port Ethernet1/2 is unknown to netdisco, skipped
...
I can also see that this is due to plugin/netdisco¿? not getting devips of the fabric switches:
...
NetdiscoX::Util::ACI mac_arp_info - pod 1 node 123 port eth1/2 mac F8:72:EA:66:F2:8D vlan 41 arpip 10.105.240.8 devname LRF-B-R03-1 devip 0.0.0.0
...
Maybe we miss some configuration or something has changed in Version 4.2(7f)???

If you need more detailed debugging information or us doing some specific tests just ask.

Congrats for the plugin again.

Bug in stanza merge?

From IRC: having the ACI controller matched in a netconf and snmpv3 stanza potentially causes the netconf credentials not to work:

09:31 <dietmar> Hi, i do not have the config anymore, but that is what i had in use, about the aci user/passwd null values :
09:33 <dietmar> device_auth:
09:33 <dietmar>   - tag: aci_prod
09:33 <dietmar>     driver: netconf
09:33 <dietmar>     only:
09:33 <dietmar>       - 1.1.1.11
09:33 <dietmar>     user: 'read'
09:33 <dietmar>     password: 'read'
09:33 <dietmar>   - tag: aci_switches_prod
09:33 <dietmar>     user: user
09:33 <dietmar>     auth:
09:33 <dietmar>       pass: 'xxxxxx'
09:33 <dietmar>       proto: SHA
09:33 <dietmar>     priv:
09:33 <dietmar>       pass: 'xxxxxx'
09:33 <dietmar>       proto: AES128
09:33 <dietmar>     only:
09:33 <dietmar>       - 1.1.1.21-254
09:33 <dietmar>       - 1.1.1.11

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.