Working with Snort's unified output

In the last Snort Report we looked at output methods for Snort. These included several ways to write data directly to disk, along with techniques for sending alerts via Syslog and even performing direct database inserts. I recommended not configuring Snort to log directly to a database because Snort is prone to drop packets while performing database inserts. In this edition of the Snort Report I demonstrate how to use unified output, the preferred method for high performance Snort operation.

Before continuing I should mention that the definitive reference on unified output is the chapter Mucking Around with Barnyard in the Syngress books Snort 2.1 (2004) and Snort Intrusion Detection and Prevention Toolkit (2007) by Barnyard author Andrew Baker. Reviewing the table of contents for each book shows the material to be identical.

Support for unified output first appeared in Snort 1.8.0, released in July 2001. Unified output is essentially a means for Snort to write sets of data to the hard drive of a sensor. Writing to the hard drive, instead of performing database inserts, allows Snort to operate much faster and minimize packet loss.

Snort provides two forms of unified output: alert and log. At one point Snort offered two additional forms, namely stream-stat and an experimental version. The first two forms exist today, and the second two are no longer available. (Code to process the stream-stat form can still be found in some unified spool readers like Barnyard, however.)

It's important to understand the data present in unified alert and log records, which I list below. Where necessary I explain the meaning of the field.

Alert records

Packet data (the actual packet, from layer 2 to layer 7)

Consider the important differences between these two formats. Generally, when you want access to alert details, you want to see the packet that triggered the alert. Unified alert output provides easy access to key packet elements like source and destination IP addresses and ports, plus protocol (TCP, UDP, ICMP, etc.), but nothing else. This isn't sufficient for most investigations. (Actually, an alert even with full packet data is almost never sufficient for investigation, but that's a story for another article.)

Unified log output might solve this problem. Unified log output includes the important non-packet details (like Signature information and so on) present in unified alert data. Unified log output also contains the entire packet that triggered the alert. That would seem to solve the problem, right? Unfortunately for those who want to see packets in human-readable form, unified log format records the packet in hexadecimal format. This means elements must be parsed by a program that understands this format. Unified log format is most likely going to be the unified form used in production environments.

Working with unified output

 Introduction
 Examining unified output
 Unified output readers
 Barnyard processing alerts
 Barnyard processing logs
 Barnyard working with databases

About the author
Richard Bejtlich is founder of TaoSecurity, author of several books on network security monitoring, including Extrusion Detection: Security Monitoring for Internal Intrusions, and operator of the TaoSecurity blog.

Rate this Tip To rate tips, you must be a member of SearchSecurityChannel.com. Register now to start rating these tips. Log in if you are already a member. Digg This!     StumbleUpon     Del.icio.us    RELATED CONTENT Open Source Security Tools Using SnortSP and Snort 2.8.2 OSSEC Host-Based Intrusion Detection Guide How to find new features in Snort 2.8.2 How to use shared object rules in Snort Snort frequently asked questions How to test Snort How to run IDS Snort on Red Hat Enterprise Linux 5 Output options for Snort data Snort IDS installation basics and tips for security resellers Snort IDS upgrade and tips on the Snort.conf file Snort Report Using SnortSP and Snort 2.8.2 The power of Snort 3.0 How to find new features in Snort 2.8.2 Top security tips for solutions providers When Snort is not enough Justifying Snort Network session data analysis with Snort and Argus How to use shared object rules in Snort Snort frequently asked questions Snort limitations RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary DISCLAIMER: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.