ripe-ncc / bgpdump Goto Github PK
View Code? Open in Web Editor NEWUtility and C Library for parsing MRT files
License: Other
Utility and C Library for parsing MRT files
License: Other
== libBGPdump == A C library designed to help with analyzing dump files produced by Zebra/Quagga or MRT. Documentation is available at: https://github.com/RIPE-NCC/bgpdump/wiki
Hi bgpdump maintainers,
I ran into some weird issues while parsing a RouteViews MRT file.
The file in question is http://routeviews.org/route-views3/bgpdata/2021.04/UPDATES/updates.20210416.1515.bz2
.
I'm using the latest bgpdump (commit bd89016) with the following switches: -v
, -m
, -p
. It looks as if the file contains prefixes that don't make any sense. Here are a few examples:
BGP4MP_ET|7680|1618586116.561183|A|104.218.61.166|213262|250.0.0.0/208|213262 38008 6939 32098 13999 262916|IGP|104.218.61.166|0|0|38008:103 64600:103 64601:38008|AG|262916 187.245.92.32|
BGP4MP_ET|13162|1618586139.260903|A|104.218.61.166|213262|0.0.0.0/238|213262 38008 6939 64049 55836 17488|IGP|104.218.61.166|0|0|38008:103 64600:103 64601:38008 65521:20|NAG||
BGP4MP_ET|21837|1618586171.006855|W|104.218.61.166|213262|160.0.0.0/160
BGP4MP_ET|258040|1618586913.681759|A|104.218.61.166|213262|80.0.0.0/127|213262 38008 6939 7738 8167 28292|IGP|104.218.61.166|0|0|38008:103 64600:103 64601:38008 65521:20|NAG||
...and many more
Note the corrupt NLRI - 250.0.0.0/208
, 0.0.0.0/238
, 160.0.0.0/160
, 80.0.0.0/127
Is this a bgpdump bug? A corrupt file? Something else?
And just to make sure we're talking about the same file, here's its hash:
> sha256sum updates.20210416.1515.bz2
5c693ec797d32efb1189bd522ce65fca419d914364c167c1158271f55a12ed83 updates.20210416.1515.bz2
> sha512sum updates.20210416.1515.bz2
3624be266dbfe40947dc4754ec82fcf5f8a632bb90af4dcf704f7fcd961380d4baa559e0dedfb01ced70619b1a8d8220ba1e07b0c62c32b7b364cd9489e6c819 updates.20210416.1515.bz2
When I was running bgpdump I got the Too many prefixes error for some files. For example:
(base) root@devcloud:/data/anycast_monitor/routeviews# /usr/bin/bgpdump -m route-views2.updates.20220308.0015.bz2 | head
2022-03-08 08:33:37 [info] logging to syslog
/var/log/messages-20220307:Mar 7 03:17:09 VM-66-201-centos bgpdump: too many prefixes (1851628600 > 88)
(base) root@devcloud:/data/anycast_monitor/routeviews# date
Tue Mar 8 08:34:37 CST 2022
(base) root@devcloud:/data/anycast_monitor/routeviews# date -u
Tue Mar 8 00:34:43 UTC 2022
But it works ok with another file
(base) root@devcloud:/data/anycast_monitor/routeviews# /usr/bin/bgpdump -m route-views.saopaulo.updates.20220308.0015.bz2 | head
2022-03-08 08:35:22 [info] logging to syslog
BGP4MP|1646698500|A|2001:12f8::223:254|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698500|A|2001:12f8::223:254|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::254|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::254|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::218:121|52720|2804:fec:2000::/36|52720 262493|IGP|2001:12f8::218:121|0|0||NAG||
BGP4MP|1646698501|A|2001:12f8::218:121|52720|2804:fec:9000::/36|52720 262493|IGP|2001:12f8::218:121|0|0||NAG||
BGP4MP|1646698501|A|2001:12f8::253|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::253|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::223:253|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
BGP4MP|1646698501|A|2001:12f8::223:253|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||
This is running on a Centos machine. I also noticed that the max prefix is changing, could you help me to understand what happened here?
Also, I tried on another machine with the same two files, there's no such problem. Both of the two files can be parsed.
Would you accept a CI (e.g. build using multiple compilers, maybe different operating systems, built-in selftest, maybe some code analysis) via GitHub Actions if provided via pull request? GitHub Actions usage is free for standard GitHub-hosted runners in public repositories, and for self-hosted runners.
Now that the RFC is out it would be nice if bgpdump supports it.
Currently when parsing a file longer than 4k it is confused about the number of prefixes in the MP_UNREACH_NLRI attribute (for example), reporting
[error] too many prefixes (20428 > 2050)
Please add support for parsing RFC 9234 OTC Attribute!
Requested by @mitradir
The 1.6.0 tag from bitbucket is used in the homebrew-core formula for bgpdump, but I don't see that tag on this git repository.
When parsing an MRT with this base64 representation (which appears in http://data.ris.ripe.net/rrc00/2010.11/updates.20101107.2220.gz with index 5118):
TNcm+AAQAAEAAABMG2oxbgAAAAEMAAE/wQAEHP////////////////////8APAIAAAAhQAEBAEACDAIFG2oN3TBlvJPHZEADBAwAAT/ACAQbCw0LDQsNCw==
these bytes appear after the path attribute:
0d 0b 0d 0b
and should represent NLRI prefixes.
Parsing these bytes should result in a prefix with length 13 (first byte - 0x0d), where the start address is the first 13 bits of the following two bytes - 0x0b0d, as stated in https://tools.ietf.org/html/rfc4271#section-4.3:
The Prefix field contains an IP address prefix, followed by
the minimum number of trailing bits needed to make the end
of the field fall on an octet boundary. Note that the value
of trailing bits is irrelevant.
So, ignoring the trailing bits, the prefix should be 11.8.0.0/13.
However, bgpdump outputs this:
TIME: 11/07/10 22:23:52
TYPE: BGP4MP/MESSAGE/Update
FROM: 12.0.1.63 AS7018
TO: 193.0.4.28 AS12654
ORIGIN: IGP
ASPATH: 7018 3549 12389 48275 51044
NEXT_HOP: 12.0.1.63
COMMUNITY: 6923:3339
ANNOUNCE
11.13.0.0/13
so it appears bgpdump uses full two bytes as the start address, and does not ignore trailing bits.
Furthermore, there is still one byte left in the message, which is not enough to represent a prefix, but bgpdump does not provide any indication of that.
Migration of this enhancement request from Bitbucket:
https://bitbucket.org/ripencc/bgpdump-hg/issues/23
Possibly add support for parsing Extended Communities.
We already parse Communities and Large Communities, it would be a useful addition.
I'm the Fedora and EPEL package maintainer of bgpdump since 2019. As of writing, there still doesn't seem to be proper upstream .so name versioning in libbgpdump.
Yes, I'm aware that something like SOFLAGS="-shared -Wl,-soname,libbgpdump.so.0"
could be passed while building (even the correct symlinks need to be generated manually then), but this still doesn't let downstreams know whether the old and the new version are ABI compatible or not, thus this should be IMHO really handled by upstream.
Quoting from section "SONAME handling" from https://docs.fedoraproject.org/en-US/packaging-guidelines/#_downstream_so_name_versioning:
The SONAME field is written to the shared object by linker, using (at least in case of ld) the -soname SONAME flags. This can be passed as an option to gcc like this:
$ gcc $CFLAGS -Wl,-soname,libfoo.so.0.n -o libfoo.so.0.n
If you want to check if the SONAME field is set and what value it has, use the objdump command (from binutils):
$ objdump -p /path/to/libfoo.so.0.n | grep 'SONAME'
Please note that this issue is also meant as very kind try to convince you as upstream to start versioning your .so library (as our Fedora Packaging Guidelines request).
Does bgpdump support the resolution of vpnv4, vpnv6, evpn and other address families?
make dist fails because License file is missing
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.