Skip to content
This repository was archived by the owner on Oct 13, 2025. It is now read-only.

Commit 93802f5

Browse files
awelzelawelzel
committed
devel: Add VXLAN VNI FIVETUPLE ConnKey plugin
1 parent dadcfc9 commit 93802f5

File tree

15 files changed

+594
-0
lines changed

15 files changed

+594
-0
lines changed

‎devel/connkey-plugin.rst‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
.. _connkey-plugin:
2+
3+
===============================
4+
Writing a Connection Key Plugin
5+
===============================
6+
7+
.. versionadded:: 8.0
8+
9+
Zeek's plugin API allows adding support for custom connection keys. By default,
10+
Zeek uses connections keys based on the classic five tuple consisting of IP address
11+
and port pairs and protocol identifier.
12+
This document describes the new functionality in form of a tutorial.
13+
We'll be using a custom connection key implementation to scope connections
14+
transported within VXLAN encapsulation by the VXLAN Network Identifier (VNI).
15+
16+
If you're not familiar with plugin development, head over to the
17+
:ref:`Writing Plugins <writing-plugins>` section.
18+
19+
As a test case, the `HTTP GET trace <https://github.com/zeek/zeek/raw/refs/heads/master/testing/btest/Traces/http/get.trace>`_ from the Zeek
20+
repository is encapsulated twice with VXLAN using VNIs 4711 and 4242, respectively.
21+
Then, we merge the original PCAP with these two PCAPs.
22+
It's useful to fetch the
23+
:download:`resulting PCAP <connkey-vxlan-fivetuple-plugin-src/Traces/vxlan-overlapping-http-get.pcap>`
24+
for testing.
25+
26+
By default, Zeek will create the same connection key for the original and
27+
encapsulated HTTP connections as they have identical five tuples. The fact
28+
that two of the connections are VXLAN encapsulated is ignored.
29+
Therefore, only a single ``http.log`` entry and two ``conn.log`` entries
30+
are created.
31+
32+
.. code-block:: shell
33+
34+
$ zeek -C -r Traces/vxlan-overlapping-http-get.pcap
35+
$ zeek-cut -m uid method host uri < http.log
36+
uid method host uri
37+
CpWF5etn1l2rpaLu3 GET bro.org /download/CHANGES.bro-aux.txt
38+
39+
$ zeek-cut -m uid service history orig_pkts resp_pkts < conn.log
40+
uid service history orig_pkts resp_pkts
41+
Cq2CY245oGGbibJ8k9 http ShADTadtFf 21 21
42+
CMleDu4xANIMzePYd7 vxlan D 28 0
43+
44+
Note that just two of the HTTP connections are encapsulated.
45+
That is why the VXLAN connection shows only 28 packets.
46+
Each HTTP connection has 14 packets total, 7 in each direction. All are
47+
aggregated in the single HTTP connection, but only 28 of these packets were
48+
transported within the VXLAN tunnel connection. Note also the ``t`` and ``T``
49+
flags in the :zeek:field:`Conn::Info$history` field. These stand for retransmissions
50+
and caused by Zeek not discriminating between the different HTTP connections.
51+
52+
The plugin we'll be developing adds the VXLAN VNI to the connection key.
53+
The result is that instead of a single HTTP connection, there'll be three HTTP
54+
connections tracked and logged separately by Zeek. The VNI is also added as
55+
:zeek:field:`vxlan_vni` to the :zeek:see:`conn_id` record and therefore available
56+
in the ``http.log`` and ``conn.log`` as part of the ``id.*`` fields.
57+
58+
59+
Implementation
60+
==============
61+
62+
Adding alternative connection keys involves implementing two classes.
63+
First, a factory class producing ``zeek::ConnKey`` instances. This
64+
is the class created through the added ``zeek::conn_key::Component``.
65+
Second, a custom connection key class derived from ``zeek::ConnKey``.
66+
Instances of this class are created by the factory. This is a typical
67+
abstract factory pattern.
68+
Zeek currently only supports IP-based connection tracking.
69+
While the ``zeek::ConnKey`` class is technically the base class,
70+
in this tutorial we'll derive from ``zeek::IPBasedConnKey`` as required
71+
by the ``IPBasedAnalyzer``.
72+
73+
Our plugin's ``Configure()`` method follows the standard pattern of setting up
74+
basic information about the plugin and then registering the ``ConnKey`` component.
75+
76+
.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Plugin.cc
77+
:caption: Plugin.cc
78+
:language: cpp
79+
:lines: 16-
80+
:linenos:
81+
:tab-width: 4
82+
83+
84+
Next, in the ``Factory.cc`` file, we're implementing a custom ``zeek::ConnKey`` class.
85+
This class is named ``VxlanVniConnKey`` and inherits from ``zeek::IPBasedConnKey``.
86+
As noted before, this is a requirement due to its sole usage by the ``IPBasedAnalyzer``
87+
today.
88+
89+
.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Factory.cc
90+
:caption: VxlanVniConnKey class in Factory.cc
91+
:language: cpp
92+
:linenos:
93+
:lines: 18-65
94+
:tab-width: 4
95+
96+
The current pattern for custom connection keys is to embed the bytes used for
97+
the ``zeek::session::detail::Key`` as a packed struct within a ``ConnKey`` instance.
98+
We override ``DoPopulateConnIdVal()`` to set the :zeek:field:`vxlan_vni` field
99+
of a :zeek:see:`conn_id` record value to the extracted VXLAN VNI. A small trick
100+
employed is that we default the most significant byte of ``vxlan_vni`` to 0xFF.
101+
As the VNI is only 24bit, this allows us to determine if a VNI was actually
102+
extracted, or whether it is unset.
103+
104+
The ``DoInit()`` implementation is the actual place for connection key customization.
105+
This is where we extract the VXLAN VNI. To do so, we're using the relatively
106+
new ``GetAnalyzerData()`` API of the packet analysis manager.
107+
This API allows generic access to the layers analyzed for give packet analyzer.
108+
For our use-case, we take the most outer VXLAN layer, if any, and extract the VNI
109+
into ``key.vxlan_vni``.
110+
111+
.. note::
112+
113+
The ``GetAnalyzerData()`` API comes with a certain overhead. While generic,
114+
it might be more efficient to track VXLAN VNIs explicitly. Whether that's a
115+
required optimization can usually be answered by profiling.
116+
117+
There's no requirement to use the ``GetAnalyzerData()`` API. If the ``zeek::Packet``
118+
instance passed to ``DoInit()`` contains the needed information, e.g. VLAN identifiers
119+
or information from the packet's raw bytes, this can be used as well.
120+
Using other Zeek APIs ways to determine connection key information is of
121+
course also possible.
122+
123+
The next part shown concerns the ``Factory`` class itself. The
124+
``DoConnKeyFromVal()`` method contains logic to produce a ``VxlanVniConnKey``
125+
instance from an existing :zeek:see:`conn_id` record.
126+
This is needed in order for the :zeek:see:`lookup_connection` builtin function to work properly.
127+
The implementation re-uses the ``DoConnKeyFromVal()`` implementation of the
128+
default ``fivetuple::Factory`` that our factory inherits from.
129+
130+
.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/src/Factory.cc
131+
:caption: Factory class in Factory.cc
132+
:language: cpp
133+
:linenos:
134+
:lines: 67-86
135+
:tab-width: 4
136+
137+
The implementation assumes exclusive use of :zeek:see:`conn_id` values, mostly
138+
for brevity reasons. Implementations may also accept other
139+
:zeek:see:`conn_id`-like values.
140+
141+
Last, the plugin's ``__load__.zeek`` file is shown. It includes the extension
142+
of the :zeek:see:`conn_id` identifier by the :zeek:field:`vxlan_vni` field.
143+
144+
.. literalinclude:: connkey-vxlan-fivetuple-plugin-src/scripts/__load__.zeek
145+
:caption: The conn_id redefinition in __load__.zeek
146+
:language: zeek
147+
:linenos:
148+
:tab-width: 4
149+
150+
151+
Using the custom Connection Key
152+
===============================
153+
154+
After installing the plugin, the new connection key implementation can be
155+
selected by redefining the script-level ``ConnKey::factory`` variable. This
156+
can either be done in a separate script, but we do it directly on the
157+
command-line for simplicity.
158+
159+
.. code-block:: shell
160+
161+
$ zeek -C -r Traces/vxlan-overlapping-http-get.pcap ConnKey::factory=ConnKey::CONNKEY_VXLAN_VNI_FIVETUPLE
162+
163+
Viewing the ``http.log`` and ``conn.log`` now shows three separate HTTP connections,
164+
two of which have a ``vxlan_vni`` value set.
165+
166+
167+
.. code-block:: shell
168+
169+
$ zeek-cut -m uid method host uri id.vxlan_vni < http.log
170+
uid method host uri id.vxlan_vni
171+
CyZiAc2lEt5DAZseQl GET bro.org /download/CHANGES.bro-aux.txt 4711
172+
CIwCdr1G7sTtHRZ8y4 GET bro.org /download/CHANGES.bro-aux.txt 4242
173+
CWBNgn3JYHzXJZjXKc GET bro.org /download/CHANGES.bro-aux.txt -
174+
175+
$ zeek-cut -m uid service history orig_pkts resp_pkts id.vxlan_vni < conn.log
176+
uid service history orig_pkts resp_pkts id.vxlan_vni
177+
CWBNgn3JYHzXJZjXKc http ShADadFf 7 7 -
178+
CIwCdr1G7sTtHRZ8y4 http ShADadFf 7 7 4242
179+
CyZiAc2lEt5DAZseQl http ShADadFf 7 7 4711
180+
C24p8iCAjprR6LFn8 vxlan D 28 0 -
181+
182+
Pretty cool, isn't it?

‎devel/connkey-vxlan-fivetuple-plugin-src/CMakeLists.txt‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
cmake_minimum_required(VERSION 3.15 FATAL_ERROR)
2+
3+
project(ZeekPluginConnKeyVxlanVniFivetuple)
4+
5+
include(ZeekPlugin)
6+
7+
zeek_add_plugin(
8+
Zeek
9+
ConnKey_Vxlan_Vni_Fivetuple
10+
SOURCES
11+
src/Factory.cc
12+
src/Plugin.cc
13+
SCRIPT_FILES scripts/__load__.zeek
14+
)

‎devel/connkey-vxlan-fivetuple-plugin-src/COPYING‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Copyright (c) 2025 by the Zeek Project. All rights reserved.
2+
3+
Redistribution and use in source and binary forms, with or without modification,
4+
are permitted provided that the following conditions are met:
5+
6+
1. Redistributions of source code must retain the above copyright notice, this
7+
list of conditions and the following disclaimer.
8+
9+
2. Redistributions in binary form must reproduce the above copyright notice,
10+
this list of conditions and the following disclaimer in the documentation
11+
and/or other materials provided with the distribution.
12+
13+
3. Neither the name of the copyright holder nor the names of its contributors
14+
may be used to endorse or promote products derived from this software
15+
without specific prior written permission.
16+
17+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
18+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
19+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
20+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
21+
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
22+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
23+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
24+
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
26+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎devel/connkey-vxlan-fivetuple-plugin-src/Makefile‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#
2+
# Convenience Makefile providing a few common top-level targets.
3+
#
4+
5+
cmake_build_dir=build
6+
arch=`uname -s | tr A-Z a-z`-`uname -m`
7+
8+
all: build-it
9+
10+
build-it:
11+
( cd $(cmake_build_dir) && make )
12+
13+
install:
14+
( cd $(cmake_build_dir) && make install )
15+
16+
clean:
17+
( cd $(cmake_build_dir) && make clean )
18+
19+
distclean:
20+
rm -rf $(cmake_build_dir)
21+
22+
test:
23+
make -C tests

‎devel/connkey-vxlan-fivetuple-plugin-src/README‎

Whitespace-only changes.

‎devel/connkey-vxlan-fivetuple-plugin-src/Traces/vxlan-overlapping-http-get.pcap‎

19.9 KB
Binary file not shown.

‎devel/connkey-vxlan-fivetuple-plugin-src/VERSION‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
0.1.0

0 commit comments

Comments
 (0)