Merge remote-tracking branch 'origin/topic/awelzel/move-tracing-events-to-scripting'

* origin/topic/awelzel/move-tracing-events-to-scripting:
  Move tracing-events into ./scripting

Author: awelzel

quickstart.rst

@@ -372,92 +372,6 @@ on the command line:
   mkdir output_directory ; zeek -r mypackets.trace Log::default_logdir=output_directory
 
 
-Tracing Events
---------------
-
-Zeek provides a mechanism for recording the events that occur during
-an execution run (on live traffic, or from a pcap) in a manner that you
-can then later replay to get the same effect but without the traffic source.
-You can also edit the recording to introduce differences between the original,
-such as introducing corner-cases to aid in testing, or anonymizing sensitive
-information.
-
-You create a trace using:
-
-.. code-block:: console
-
-  zeek --event-trace=mytrace.zeek <traffic-option> <other-options> <scripts...>
-
-or, equivalently:
-
-.. code-block:: console
-
-  zeek -E mytrace.zeek <traffic-option> <other-options> <scripts...>
-
-Here, the *traffic-option* would be ``-i`` or ``-r`` to arrange for
-a source of network traffic.  The trace will be written to the file
-``mytrace.zeek`` which, as the extension suggests, is itself a Zeek script.
-You can then replay the events using:
-
-.. code-block:: console
-
-  zeek <other-options> <scripts...> mytrace.zeek
-
-One use case for event-tracing is to turn a sensitive PCAP that can't
-be shared into a reflection of that same activity that - with some editing, for
-example to change IP addresses - is safe to share.  To facilitate such
-editing, the generated script includes at the end a summary of all of
-the constants present in the script that might be sensitive and require
-editing (such as addresses and strings), to make it easier to know what
-to search for and edit in the script.  The generated script also includes
-a global ``__base_time`` that's used to make it easy to alter (most of)
-the times in the trace without altering their relative offsets.
-
-The generated script aims to ensure that event values that were related
-during the original run stay related when replayed; re-execution should
-proceed in a manner identical to how it did originally.  There are however
-several considerations:
-
-* Zeek is unable to accurately trace events that include values that cannot
-  be faithfully recreated in a Zeek script, namely those having types of
-  ``opaque``, ``file``, or ``any``.  Upon encountering these, it generates
-  variables reflecting their unsupported nature, such as ``global
-  __UNSUPPORTED21: opaque of x509;``, and initializes them with code like
-  ``__UNSUPPORTED21 = UNSUPPORTED opaque of x509;``.  The generated script
-  is meant to produce syntax errors if run directly, and the names make
-  it easy to search for the elements that need to somehow be addressed.
-
-* Zeek only traces events that reflect traffic processing, i.e., those
-  occurring after :zeek:id:`network_time` is set.  Even if you don't include
-  a network traffic source, it skips the :zeek:id:`zeek_init` event
-  (since it is always automatically generated).
-
-* The trace does *not* include events generated by scripts, only those
-  generated by the "event engine".
-
-* The trace is generated upon Zeek cleanly exiting, so if Zeek crashes,
-  no trace will be produced. Stopping Zeek via *ctrl-c* does trigger a
-  clean exit.
-
-* A subtle issue arises regarding any changes that the scripts in the
-  original execution made to values present in subsequent events.  If
-  you re-run using the event trace script as well as those scripts,
-  the changes the scripts make during the re-run will be discarded and
-  instead replaced with the changes made during the original execution.
-  This generally won't matter if you're using the exact same scripts for
-  replay as originally, but if you've made changes to those scripts, then
-  it could.  If you need the replay script to "respond" to changes made
-  during the re-execution, you can delete from the replay script every
-  line marked with the comment ``# from script``.
-
-.. note::
-
-  It's possible that some timers will behave differently upon replay
-  than originally.  If you encounter this and it creates a problem, we
-  would be interested to hear about it so we can consider whether the
-  problem can be remedied.
-
-
 Telling Zeek Which Scripts to Load
 ----------------------------------
 

scripting/index.rst

@@ -9,5 +9,6 @@ Introduction to Scripting
    basics
    usage
    event-groups
+   tracing-events
    optimization
    javascript

scripting/tracing-events.rst

@@ -0,0 +1,87 @@
+.. _tracing_events:
+
+==============
+Tracing Events
+==============
+
+Zeek provides a mechanism for recording the events that occur during
+an execution run (on live traffic, or from a pcap) in a manner that you
+can then later replay to get the same effect but without the traffic source.
+You can also edit the recording to introduce differences between the original,
+such as introducing corner-cases to aid in testing, or anonymizing sensitive
+information.
+
+You create a trace using:
+
+.. code-block:: console
+
+  zeek --event-trace=mytrace.zeek <traffic-option> <other-options> <scripts...>
+
+or, equivalently:
+
+.. code-block:: console
+
+  zeek -E mytrace.zeek <traffic-option> <other-options> <scripts...>
+
+Here, the *traffic-option* would be ``-i`` or ``-r`` to arrange for
+a source of network traffic.  The trace will be written to the file
+``mytrace.zeek`` which, as the extension suggests, is itself a Zeek script.
+You can then replay the events using:
+
+.. code-block:: console
+
+  zeek <other-options> <scripts...> mytrace.zeek
+
+One use case for event-tracing is to turn a sensitive PCAP that can't
+be shared into a reflection of that same activity that - with some editing, for
+example to change IP addresses - is safe to share.  To facilitate such
+editing, the generated script includes at the end a summary of all of
+the constants present in the script that might be sensitive and require
+editing (such as addresses and strings), to make it easier to know what
+to search for and edit in the script.  The generated script also includes
+a global ``__base_time`` that's used to make it easy to alter (most of)
+the times in the trace without altering their relative offsets.
+
+The generated script aims to ensure that event values that were related
+during the original run stay related when replayed; re-execution should
+proceed in a manner identical to how it did originally.  There are however
+several considerations:
+
+* Zeek is unable to accurately trace events that include values that cannot
+  be faithfully recreated in a Zeek script, namely those having types of
+  ``opaque``, ``file``, or ``any``.  Upon encountering these, it generates
+  variables reflecting their unsupported nature, such as ``global
+  __UNSUPPORTED21: opaque of x509;``, and initializes them with code like
+  ``__UNSUPPORTED21 = UNSUPPORTED opaque of x509;``.  The generated script
+  is meant to produce syntax errors if run directly, and the names make
+  it easy to search for the elements that need to somehow be addressed.
+
+* Zeek only traces events that reflect traffic processing, i.e., those
+  occurring after :zeek:id:`network_time` is set.  Even if you don't include
+  a network traffic source, it skips the :zeek:id:`zeek_init` event
+  (since it is always automatically generated).
+
+* The trace does *not* include events generated by scripts, only those
+  generated by the "event engine".
+
+* The trace is generated upon Zeek cleanly exiting, so if Zeek crashes,
+  no trace will be produced. Stopping Zeek via *ctrl-c* does trigger a
+  clean exit.
+
+* A subtle issue arises regarding any changes that the scripts in the
+  original execution made to values present in subsequent events.  If
+  you re-run using the event trace script as well as those scripts,
+  the changes the scripts make during the re-run will be discarded and
+  instead replaced with the changes made during the original execution.
+  This generally won't matter if you're using the exact same scripts for
+  replay as originally, but if you've made changes to those scripts, then
+  it could.  If you need the replay script to "respond" to changes made
+  during the re-execution, you can delete from the replay script every
+  line marked with the comment ``# from script``.
+
+.. note::
+
+  It's possible that some timers will behave differently upon replay
+  than originally.  If you encounter this and it creates a problem, we
+  would be interested to hear about it so we can consider whether the
+  problem can be remedied.