Discussion:
[pmacct-discussion] Tips on debugging IPFIX/v10 on 1.5.2?
Inge Bjørnvall Arnesen
2016-06-03 14:00:35 UTC
Permalink
Hi all,

We've changed one edge router to a more modern Juniper MX and I'm trying to get IPFIX working on my 1.5.2 installation. Since Juniper only allows a single destination, we have set up a splitter to duplicate traffic to the various flow destinations. The other destination appliances decode the v10 packets without problems and doing a tcpdump and Wireshark check on the nfacct host indicates that all the IPFIX packets are received correctly. No data is entered into the MySQL or memory plugins from this flow source however. With debugging enabled, I see (after the initial IPFIX packets before templates are received):

DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [0]
DEBUG ( default/core ): NfV10 agent : a.b.c.d:524288
DEBUG ( default/core ): NfV10 template type : flow
DEBUG ( default/core ): NfV10 template ID : 256
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): | pen | field type | offset | size |
DEBUG ( default/core ): | 0 | IPv4 src addr | 0 | 4 |
DEBUG ( default/core ): | 0 | IPv4 dst addr | 4 | 4 |
DEBUG ( default/core ): | 0 | tos | 8 | 1 |
DEBUG ( default/core ): | 0 | L4 protocol | 9 | 1 |
DEBUG ( default/core ): | 0 | L4 src port | 10 | 2 |
DEBUG ( default/core ): | 0 | L4 dst port | 12 | 2 |
DEBUG ( default/core ): | 0 | icmp type | 14 | 2 |
DEBUG ( default/core ): | 0 | input snmp | 16 | 4 |
DEBUG ( default/core ): | 0 | 58 | 20 | 2 |
DEBUG ( default/core ): | 0 | IPv4 src mask | 22 | 1 |
DEBUG ( default/core ): | 0 | IPv4 dst mask | 23 | 1 |
DEBUG ( default/core ): | 0 | src as | 24 | 4 |
DEBUG ( default/core ): | 0 | dst as | 28 | 4 |
DEBUG ( default/core ): | 0 | IPv4 next hop | 32 | 4 |
DEBUG ( default/core ): | 0 | tcp flags | 36 | 1 |
DEBUG ( default/core ): | 0 | output snmp | 37 | 4 |
DEBUG ( default/core ): | 0 | in bytes | 41 | 8 |
DEBUG ( default/core ): | 0 | in packets | 49 | 8 |
DEBUG ( default/core ): | 0 | 52 | 57 | 1 |
DEBUG ( default/core ): | 0 | 53 | 58 | 1 |
DEBUG ( default/core ): | 0 | 152 | 59 | 8 |
DEBUG ( default/core ): | 0 | 153 | 67 | 8 |
DEBUG ( default/core ): | 0 | 136 | 75 | 1 |
DEBUG ( default/core ): | 0 | 243 | 76 | 2 |
DEBUG ( default/core ): | 0 | 245 | 78 | 2 |
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 80
DEBUG ( default/core ):
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50103] version [10] seqno [434178]
DEBUG ( default/core ): NfV10 agent : a.b.c.d:524288
DEBUG ( default/core ): NfV10 template type : options
DEBUG ( default/core ): NfV10 template ID : 512
DEBUG ( default/core ): ----------------------------------------
DEBUG ( default/core ): | field type | offset | size |
DEBUG ( default/core ): | 144 | 0 | 4 |
DEBUG ( default/core ): | 160 | 4 | 8 |
DEBUG ( default/core ): | 130 | 12 | 4 |
DEBUG ( default/core ): | 131 | 16 | 16 |
DEBUG ( default/core ): | 214 | 32 | 1 |
DEBUG ( default/core ): | 215 | 33 | 1 |
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 34
DEBUG ( default/core ):
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443061]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443066]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443071]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443076]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443081]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443086]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443091]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443096]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443101]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443106]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443111]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443116]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443121]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443126]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443131]

and so on. All looks good, but nothing ends up in the plugins. Any idea on how to debug further? Is it possible to get more detail on the actual parsing of the IPFIX packets?

Regards,


n Inge
Paolo Lucente
2016-06-04 10:14:24 UTC
Permalink
Hi Inge,

Any chance you have some aggregate_filter or any other filtering in place
via pre_tag_map? Another option could be the new MX box is exporting less
data than the previous one (ie. as a result of a different configured
sampling rate) and buffers (plugin_buffer_size mainly) are set too high?

If none of this rings a bell, you can post here or privately your config;
if that also does not bring anything, you could send me privately a brief
trace of your IPFIX traffic so that i can try to reproduce the issue in
lab.

Cheers,
Paolo
Post by Inge Bjørnvall Arnesen
Hi all,
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [0]
DEBUG ( default/core ): NfV10 agent : a.b.c.d:524288
DEBUG ( default/core ): NfV10 template type : flow
DEBUG ( default/core ): NfV10 template ID : 256
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): | pen | field type | offset | size |
DEBUG ( default/core ): | 0 | IPv4 src addr | 0 | 4 |
DEBUG ( default/core ): | 0 | IPv4 dst addr | 4 | 4 |
DEBUG ( default/core ): | 0 | tos | 8 | 1 |
DEBUG ( default/core ): | 0 | L4 protocol | 9 | 1 |
DEBUG ( default/core ): | 0 | L4 src port | 10 | 2 |
DEBUG ( default/core ): | 0 | L4 dst port | 12 | 2 |
DEBUG ( default/core ): | 0 | icmp type | 14 | 2 |
DEBUG ( default/core ): | 0 | input snmp | 16 | 4 |
DEBUG ( default/core ): | 0 | 58 | 20 | 2 |
DEBUG ( default/core ): | 0 | IPv4 src mask | 22 | 1 |
DEBUG ( default/core ): | 0 | IPv4 dst mask | 23 | 1 |
DEBUG ( default/core ): | 0 | src as | 24 | 4 |
DEBUG ( default/core ): | 0 | dst as | 28 | 4 |
DEBUG ( default/core ): | 0 | IPv4 next hop | 32 | 4 |
DEBUG ( default/core ): | 0 | tcp flags | 36 | 1 |
DEBUG ( default/core ): | 0 | output snmp | 37 | 4 |
DEBUG ( default/core ): | 0 | in bytes | 41 | 8 |
DEBUG ( default/core ): | 0 | in packets | 49 | 8 |
DEBUG ( default/core ): | 0 | 52 | 57 | 1 |
DEBUG ( default/core ): | 0 | 53 | 58 | 1 |
DEBUG ( default/core ): | 0 | 152 | 59 | 8 |
DEBUG ( default/core ): | 0 | 153 | 67 | 8 |
DEBUG ( default/core ): | 0 | 136 | 75 | 1 |
DEBUG ( default/core ): | 0 | 243 | 76 | 2 |
DEBUG ( default/core ): | 0 | 245 | 78 | 2 |
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 80
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50103] version [10] seqno [434178]
DEBUG ( default/core ): NfV10 agent : a.b.c.d:524288
DEBUG ( default/core ): NfV10 template type : options
DEBUG ( default/core ): NfV10 template ID : 512
DEBUG ( default/core ): ----------------------------------------
DEBUG ( default/core ): | field type | offset | size |
DEBUG ( default/core ): | 144 | 0 | 4 |
DEBUG ( default/core ): | 160 | 4 | 8 |
DEBUG ( default/core ): | 130 | 12 | 4 |
DEBUG ( default/core ): | 131 | 16 | 16 |
DEBUG ( default/core ): | 214 | 32 | 1 |
DEBUG ( default/core ): | 215 | 33 | 1 |
DEBUG ( default/core ): -----------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 34
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443061]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443066]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443071]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443076]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443081]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443086]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443091]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443096]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443101]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443106]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443111]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443116]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443121]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443126]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from [a.b.c.d:50101] version [10] seqno [738443131]
and so on. All looks good, but nothing ends up in the plugins. Any idea on how to debug further? Is it possible to get more detail on the actual parsing of the IPFIX packets?
Regards,
n Inge
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Loading...