---

Manual Reference Pages  - SHOKI.TOPOLOGY (7)

---

shoki_sez - overview of shoki NIDS topology

CONTENTS

Description
     Functional Nomenclature
     The Sensor
     The Aggregator and Database
Authors
Bugs
See Also

This is an overview of the basic topology of the shoki NIDS. It is intended to give the analyst an idea of how data is processed by shoki from the time it is pulled off the wire to when it shows up as an alert.

There are three main groups of functions in shoki. These functions may be shared on the same machine, or may reside in different machines.

Sensor	This is the machine (or machines) on which data is collected.
Aggregator
	A centralised server to which data are copied. This is were most of the shoki widgets live.
Database	The server running Postgres. Data from all the interesting packets ends up in various database tables here.

Flow of data on the sensor machine is straightforward:

     sensor(8) -> raw pcap data -> lexer(1) -> pcap data

1. Packets are captured off the wire by the sensor(8) widget and written to pcap dumpfiles
2. Each packet dump is run through lexer(1) with a simplified set of filter rules to produce a smaller dumpfile
3. The aggregator machine collects the reduced dumpfiles via rsync(1).

Data flow through the aggregator machine and database are somewhat more complicated:

                          --> alerts --> shoki_sez(8) --> IM client
                          |
                     --> ooda(8) --> doctrine_filters
                     |                     |
   --> lexer(1) --> events <--             |
   |                         |             |
   -- pcap data --------> dlex(8) <---------

1. Dumpfiles collected from the sensor machine are passed through lexer(1) running with a full set of NIDS signatures, logging matches to the events table of the shoki database.
2. The ooda(8) widget listens to the events table, and passes new events through its doctrines (see shoki.doctrine(5) for details), potentially writing new filters to the doctrine_filters table or writing alerts to the alerts table.
3. The dlex(8) widget listens to the doctrine_filters table, and runs any new filters against the same packet dumps read by lexer(1) in step 1 above, potentially writing new events to the events table (which will be seen by ooda(8), effectively looping back to step 2 above).
4. The shoki_sez(8) widget listens to the alerts table, forwarding any new alerts to its contact list via the Jabber protocol.

.An Stephen P. Berry <spb@meshuggeneh.net>

More information can be found at the shoki homepage:

Check the README at the top of the source tree.

shoki_sez(8)