A Practical Guide to Honeypots

In this section, we describe Network Intrusion Detection Systems (IDS)—the traditional approach to network security.
We also introduce A Practical Guide to honeypots, providing a brief history of their development. Finally, we discuss the general advantages and disadvantages of honeypots in modern cybersecurity.

1.1 Network Intrusion Detection System

The goal of an Intrusion Detection System (IDS) is to:

“Identify, preferably in real time, unauthorized use, misuse, and abuse of computer systems by both system insiders and external penetrators.”

Unlike building a shield around the network (which often fails to prevent insider attacks), IDSs serve as an alternative or complement by actively monitoring and detecting intrusions as they happen.

Early research laid the groundwork for IDS development, but it wasn't until 1998 that real-time network intrusion detection became viable. Paxson’s system translated packet streams into high-level network events, which could then be analyzed using a defined security policy.

Since then, IDS technology has evolved with:

• Machine learning techniques to enhance detection accuracy
• Better protection against Denial-of-Service (DoS) attacks

However, as IDS systems advance, evasion techniques have also improved. Notably:

• Earlier researchers highlighted evasion vulnerabilities in IDSs
• Mimicry attacks were introduced that simulate legitimate behavior to bypass detection

Additionally, the increasing use of encrypted communication makes traditional IDS approaches less effective. These limitations have led to growing interest in alternative solutions—like honeypots.

1.2 Honeypots

The exact definition of a honeypot varies, but most interpretations center around this idea:

A honeypot is an information system resource whose value lies in unauthorized or illicit use of that resource.

A more practical (though limited) definition describes a honeypot as:

A server configured to detect intruders by mimicking a real production system. It appears to operate normally, but all data and transactions are fake. Located either inside or outside the firewall, it is used to study intruder techniques and uncover vulnerabilities in real systems.

In practice, honeypots are computers that pretend to be vulnerable systems. They log all interactions with users, and since they don’t offer any real services, any access is unauthorized—and often malicious. Honeypots have been compared to using wet cement to detect intruders—leaving behind visible traces of unwanted access.

1.3 Honeypot History

The first publicly available honeypot was the Deception ToolKit, created by Fred Cohen in 1998. Its purpose was to make systems appear vulnerable by simulating known security flaws.

Throughout the late 1990s, honeypots became increasingly common—both in open-source and commercial forms. As worms and automated malware started to spread in the early 2000s, honeypots became crucial in capturing, analyzing, and understanding them.

A key development came in 2004 with the introduction of virtual honeypots, allowing multiple honeypots to run on a single physical machine, improving scalability and efficiency.

1.4 Types of Honeypots

Honeypots can be categorized into two main types based on the level of interaction they offer to potential attackers:

1. High-Interaction Honeypots

• Simulate real operating systems and allow full interaction.
• Attackers can execute commands and run applications just like on a legitimate system.
• The goal is to collect detailed data about attack techniques and behavior.
• Minimal restrictions are placed on the attacker once they gain access.

2. Low-Interaction Honeypots

• Simulate limited services (e.g., fake HTTP or FTP).
• Interaction is restricted to detect known exploit attempts.
• Ideal for early detection and monitoring of unauthorized activity with less risk.

Some researchers have proposed a third type: Medium-Interaction Honeypots, which offer more functionality than low-interaction systems but still lack the full capabilities of high-interaction ones. However, in practice, these are not widely implemented. For simplicity, medium-interaction honeypots can be considered a subset of low-interaction systems, offering partial service emulation without full system access.

1.5 General Honeypot Advantages and Disadvantages

Honeypots offer several advantages over traditional security systems like Intrusion Detection Systems (IDS):

✅ Advantages:

• Fewer false positives – since honeypots don’t serve legitimate traffic, any activity is suspicious.
• Smaller, high-value data – they log only malicious or unauthorized behavior.
• Work in encrypted environments – they monitor activity at the endpoint rather than relying on traffic analysis.
• No need for known attack signatures – unlike IDSs, honeypots can detect new and unknown threats.

❌ Disadvantages:

• Can be misused – attackers might leverage a compromised honeypot to attack other systems.
• Limited scope – they only monitor interactions made directly with the honeypot.
• Risk of detection – skilled attackers may recognize and avoid honeypots.

While traditional tools like IDS still play a vital role, modern and complex attacks demand more proactive approaches. Honeypots are valuable in detecting novel attack vectors, especially in encrypted environments. Virtualization technologies have further enhanced their deployment and scalability.
However, it's crucial to understand their limitations to maximize their effectiveness within a defensive security strategy.

2. Recent Trends and Advances

In this section, we explore how honeypots are evolving—particularly through grouping into centralized or distributed systems like honeynets and honeyfarms.
We also touch on advanced technologies like shadow honeypots and distributed honeypots.

2.1 Honeynets and Honeyfarms

Honeynets and honeyfarms are collections of multiple honeypots, often grouped for better coordination and efficiency.

• Honeyfarms are typically centralized and easier to manage at scale.
• Grouping honeypots helps overcome some traditional limitations. For example:
  • A standalone honeypot often restricts outbound traffic to prevent misuse.
  • However, this restriction can expose the honeypot’s nature to attackers.

To solve this, researchers have introduced redirection points within honeyfarms that route outbound traffic through other nodes that behave like real victims. This technique makes the honeypot appear more legitimate and helps bypass attacker detection.

2.2 Shadow Honeypots

Shadow honeypots are a hybrid approach that combines traditional honeypots with Anomaly Detection Systems (ADS). This makes them a powerful alternative to purely rule-based intrusion detection systems.

The process starts by segmenting anomalous traffic from normal traffic. The suspicious or anomalous traffic is then redirected to a shadow honeypot, which is essentially an isolated instance of a legitimate service.

• If the shadow honeypot detects an attack, the changes in state are discarded—ensuring the system remains unaffected.
• If the activity turns out to be legitimate, the transaction is processed normally, preserving user experience and functionality.

Although shadow honeypots require more computational overhead, they offer a major advantage:

They can detect attacks that depend on the internal state of a service, which many traditional IDS and basic honeypots might miss.

This method adds a stateful layer of security, helping defend against sophisticated and subtle intrusion techniques that are often missed by signature-based tools.

2.3 Distributed Honeypots

One limitation of traditional honeypots is that they need to occupy a large portion of IP address space to be effective—since they rely on being discovered by attackers or automated malware.

To address this, distributed honeypot systems have emerged:

• Some frameworks allow legitimate hosts to redirect suspicious traffic to a centralized honeypot, reducing the need to deploy multiple physical or virtual systems across the entire network.
• In another model, each client machine is assigned a single unused IP address, and their traffic is anonymously routed through the Tor network to a centralized collection of honeypots. This preserves privacy while distributing the honeypot presence.

Technologies like honeyfarms, honeynets, and distributed honeypots help cover a wider address space and improve effectiveness. As discussed earlier, grouping honeypots also enables features such as simulated outbound traffic, making them appear more realistic to attackers.

These innovations—alongside shadow honeypots and distributed honeynets—represent the forefront of honeypot research. For those interested, keeping up with the latest studies and journals is highly recommended.

3. Existing Honeypot Products

This section briefly reviews some of the most well-known honeypot solutions available today. Each tool offers unique features and serves different use cases.

3.1 Honeyd

Honeyd is a powerful honeypot tool for Linux/Unix systems, developed by security researcher Niels Provos.

What makes Honeyd groundbreaking is its ability to:

• Create multiple virtual hosts on a single network interface
• Emulate different operating systems and services by mimicking their behavior at the TCP/IP stack level
• Fool sophisticated scanning tools like Nmap
• Passively identify attackers based on their interaction patterns

Honeyd is lightweight, flexible, and widely respected for its contribution to honeypot research.

You can learn more at: www.honeyd.org

3.2 HoneyBOT

HoneyBOT is a medium-interaction honeypot for Windows, developed by Atomic Software Solutions.

Initially built to detect worms like Code Red and Nimda back in 2001, it was released for free public use in 2005.

Key features include:

• Allowing attackers to upload files to a quarantined zone for analysis
• Detecting trojans, rootkits, and other forms of malware
• Easy-to-use graphical interface for Windows users

HoneyBOT is particularly useful for individuals or organizations running Windows environments who want a user-friendly honeypot solution.

Official site: www.atomicsoftwaresolutions.com/honeybot.php

3.3 Specter

Specter is described by its creators as a "honeypot-based intrusion detection system." However, its core functionality is that of a honeypot designed to:

• Lure attackers away from real systems
• Log attacker activity
• Gather evidence that can potentially be used for investigative or legal purposes

Key Features:

• Provides decoy data for attackers to download. These files can leave traceable markers on the attacker's system.
• Can emulate systems in different states:
  • Poorly configured systems
  • Secure and hardened machines
  • Systems with failing hardware/software
  • Unpredictable or unstable behavior
• Actively attempts to collect intelligence about the attacker during the interaction

Specter includes a detailed control panel interface to monitor and manage honeypot activity. It is a commercial tool, available at specter.com.

3.4 Comparison

Here’s a quick comparison of the three honeypot tools discussed above:

Honeypot	Interaction Level	OS Emulation	Price	Source Type
Honeyd	Low	Yes	Free	Open Source
HoneyBOT	Low	No	Free	Closed Source
Specter	High	Yes	$599 (Light) / $899 (Full)	Closed Source

Summary:

• HoneyBOT is perfect for beginners — simple, lightweight, and free, but limited in features and closed-source.
• Honeyd is ideal for power users — open-source, extensible, and capable of simulating multiple hosts and services.
• Specter is a premium solution — not free, but packed with advanced features like attacker fingerprinting and system emulation options, backed by a polished user interface.

4. HoneyRJ: Low-Interaction Honeypot Case Study

HoneyRJ is a case study in implementing a low-interaction honeypot. As previously discussed, low-interaction honeypots simulate limited protocol behavior with the goal of capturing and logging malicious traffic. These honeypots are placed on IP addresses unused by legitimate services, so any incoming connection is assumed malicious.

HoneyRJ was designed to be extremely simple, modular, and easy to extend, making it accessible to users with minimal technical experience. It demonstrates key concepts in honeypot architecture and allows users to easily add new protocol support.

4.1 Features of HoneyRJ

HoneyRJ includes several important capabilities that make it a functional and educational honeypot system:

General Features:

• Multi-Protocol Support: Users can add any protocol by implementing a provided Java Interface. Custom protocol classes can define interaction logic.
• Unlimited Client Connections: Thanks to a multi-threaded design, HoneyRJ can handle many simultaneous connections.
• Logging: Each connection is logged in its own file, including all sent and received packets.
• Graphical User Interface (GUI): A simple GUI provides users control over the application.
• Configurable Options: Various settings can be adjusted during compile time for customization.

Denial-of-Service (DoS) Prevention:

4.2 Application Design Overview

This section outlines the architectural and development choices that shaped HoneyRJ.

4.2.1 Programming Language: Java

The team chose Java due to:

• Prior experience and familiarity
• A robust, high-level sockets API that simplifies network programming
• A built-in thread library for managing concurrent client connections

4.2.2 Integrated Development Environment (IDE): Eclipse

Eclipse was selected because:

• It's free, open-source, and fully supported by the university
• Offers modern development features like code completion, refactoring, and package management
• Supports JavaDoc for automatic source code documentation
• Familiarity with Eclipse allowed quick ramp-up without additional training

4.2.3 Design Decisions

We have outlined our thinking behind the design decisions that occurred while developing HoneyRJ. The result of each decision is outlined below, with detail provided as to how we reached that decision. Our decisions follow our desire to create a simple, yet extendable, low-interaction honeypot robust enough to run multiple protocols and talk with multiple clients simultaneously.

4.2.3.1 Multi-threaded to support multiple connections

HoneyRJ can listen on multiple protocols and can talk to multiple clients on each protocol at once. We decided to design HoneyRJ to support multiple connections because otherwise the application would be severely limited in terms of its usefulness as a honeypot: the application would only be able to log one hacker's connection at a time. With only one available connection, you would not be able to run multiple protocols or see multiple connections from one hacker. This would be a severe limitation on the usefulness of the data collected by HoneyRJ and thus we decided to implement HoneyRJ as a multi-threaded application.

Our multi-threaded design is loosely based on the design in the "Supporting Multiple Clients" section of the Java Sockets tutorial. Each protocol running in HoneyRJ has a thread listening for connections on a socket. As shown in figure 5, when a thread receives a connection, a worker thread is launched to speak with the connected client, while the main thread continues to listen for connections. This design allows the application to listen on multiple ports simultaneously, while talking with multiple clients.

4.2.3.2 Logging to Plain Text Files

HoneyRJ stores log files as plain text documents in a local directory, updating them continuously as connections progress. We chose plain text logs to make it easy for users to read and to allow third-party tools to parse the data.

An alternative—storing logs as serialized Java objects—would have required a special viewer and limited future parsing options. By updating logs in real time, users can monitor active connections live. This also prevents loss of data if the application crashes. Users interested only in completed sessions can simply open the log files of closed connections.

The log file format includes timestamps, sending and receiving IP addresses and ports, and the packet contents. This information is critical for identifying client locations and monitoring their actions during a connection.

4.2.3.3 Easy Creation of Additional Protocols

HoneyRJ is designed to allow users to add new protocols by implementing a provided Java Interface. This architecture enables users to write custom protocols beyond the sample ones included, such as proprietary protocols specific to a company’s product.

This design offers high customization and also simplified development of the sample protocols included with HoneyRJ.

4.2.3.4 Support Only for String-Based Protocols

HoneyRJ currently supports only string-based protocols and does not handle binary data transmission. This decision was primarily made to keep the implementation simple.

Additionally, there are security concerns with allowing binary uploads—malicious files could exploit vulnerabilities like buffer overflows in the host system.

Since many common protocols are text-based, this limitation still allows HoneyRJ to support a wide range of useful protocols.

4.2.3.5 Denial of Service (DoS) Attack Prevention

HoneyRJ includes two main safeguards against DoS attacks, which are important given its role in attracting potentially malicious traffic:

• Connection Timeout: Any connection left idle or active beyond a configurable timeout (default 2 minutes) is forcefully closed. This prevents attackers from exhausting system resources by leaving many open connections.
• Waiting Period Between Connections: After accepting a connection on a protocol, HoneyRJ enforces a waiting period (default 5 seconds) before accepting another. This stops rapid flooding of connections that could overwhelm the honeypot.

These features ensure HoneyRJ remains responsive and capable of capturing useful data even under attack.

4.3 Application Internals

This section describes the flow of events inside HoneyRJ when the application is launched and when a module is added and started. The focus here is on the non-GUI classes, as the graphical user interface only provides wrapper functionality to these core classes.

Included in the appendix is the JavaDoc for HoneyRJ, which provides a description of every method and variable used in the application. Anyone planning significant changes to HoneyRJ should read the JavaDoc before starting.

The HoneyRJ application consists of:

• Two main classes:
  • HoneyRJ – the main application controller.
  • LIModule – handles connection support for implemented protocols.
• Two helper classes:
  • LIModuleThread – manages communication with a connected client.
  • LIProtocol – an interface that defines the communication logic for protocols.

HoneyRJ controls multiple LIModule instances. Each LIModule contains a class implementing the LIProtocol interface, which handles communication with connected clients. When a client connects, the LIModule launches a LIModuleThread to manage the interaction.

4.3.1 HoneyRJ Launch and Initialization

When the application launches, the following sequence occurs:

1. Constructor Execution – The HoneyRJ class constructor is called.
2. Module Management Setup – A HashMap is created to store LIModules, mapping each port number to a module, ensuring only one module per port.
3. Logging Directory Creation – A logging directory is created, and a reference to it is stored for later use by modules.
4. Adding Modules –
   • A LIModule is initialized.
   • It is passed to the RegisterService() method, which adds it to the HashMap while enforcing the one-module-per-port rule.
   • The module’s registerParent() method is then called, giving it access to the logging directory.
5. Repeat for Additional Modules – The process is repeated for each new module.
6. Waiting for Start – HoneyRJ waits for the user to start the added modules.
7. Listening for Connections – Once started, HoneyRJ listens for incoming connections on the configured ports.

4.3.2 LIModule Initialization

This section describes the process of initializing a LIModule and the sequence of events that occur when it is started.

A LIModule is responsible for handling communication and logging for a single protocol.

Initialization Process:

1. The LIModule constructor is called with an initialized class that implements the LIProtocol interface (see section 4.3.3 for details on LIProtocol).
2. The provided LIProtocol instance is stored as a member variable.
3. The LIModule waits for the HoneyRJ class to register itself with the module by calling the registerParent() method.
4. Once registered, the LIModule stores a reference to the HoneyRJ instance for later access to its logging directory.
5. At this stage, the module is ready to be started by the user.

When Started:

1. The module launches itself into a separate thread.
2. It creates a ServerSocket that listens on the port specified by the contained LIProtocol.
3. When a client connects to the port:
   • The LIModule spawns a LIModuleThread worker thread, passing it the connected socket.
   • The LIModuleThread communicates with the connecting user (or attacker) according to the LIProtocol.
4. Meanwhile, the LIModule continues listening for additional incoming connections.

4.3.3 LIProtocol Interface Overview

The LIProtocol interface defines five methods that must be implemented by any protocol class. Of these, the processInput() method performs most of the work, while the other four methods provide protocol-specific information.

When a LIModuleThread is launched to handle a client connection, it creates an instance of the class implementing LIProtocol.

Message Handling Flow:

• Each packet received from the client socket is converted into a String object and passed as a parameter to the processInput() method.
• The processInput() method processes the incoming string and returns a response as a Vector of String objects.
• Each string in the returned vector is sent to the client as a separate line.
• For protocols that only need to return one string, the helper method LIHelper.vectorFromString() can be used to wrap it into a Vector object.

Other LIProtocol Methods:

1. whoTalksFirst()
   • Determines whether the client or server sends the first message when a connection is established.
   • Defined by the TALK_FIRST enum with two values: SVR_FIRST (server sends first) and CLIENT_FIRST (client sends first).
   • If the server talks first, processInput() will be called initially with a null parameter, and the implementation should return the first message.
2. getPort()
   • Returns the integer port number on which the protocol listens.
3. toString()
   • Returns the protocol name as a String.
   • Used for naming log files and identifying the protocol in the GUI (e.g., "FTP").
4. isConnectionOver()
   • Returns a Boolean indicating whether the connection is considered complete.
   • Checked after the messages returned by processInput() are sent to the client.
   • true means no more messages will be sent or received; false means communication is still active.

For full method specifications, including parameter and return types, refer to the JavaDoc provided in the appendix.

4.4 Implementing Additional Protocols

HoneyRJ allows users to develop and integrate new protocols into the system. This section outlines the steps required to implement a protocol, using FTP as an example.

The process begins with a few key design decisions, followed by creating a class that implements the five methods defined in the LIProtocol interface. The new protocol class is then registered in the main application by modifying the HoneyRJMain.java file.

Developing a protocol requires knowledge of Java, including common data structures and object-oriented programming. The instructions assume use of the Eclipse IDE and familiarity with the protocol being implemented.

4.4.1 Key Protocol Design Decisions

Before coding, determine the following:

1. Protocol Name
   • Choose a short name (preferably under 5 characters) for display in the GUI and for naming log files.
   • Example: "FTP" for the File Transfer Protocol.
2. Who Talks First
   • Decide whether the server or client sends the first message once a connection is established.
   • Example:
     • Server first: FTP, where the server sends "220 Service Ready" immediately after connection.
     • Client first: IRC, where the client sends "NICK username" upon connecting.
3. Port Number
   • Determine the port on which your protocol will operate.
   • Standard protocols have ports defined in their RFCs (e.g., FTP uses port 21).
   • For custom protocols, avoid reusing well-known ports to prevent conflicts.

4.4.2 Implementation

Once you have completed the key protocol design decisions, you can begin programming your protocol.

1. Project Setup
   • Import the HoneyRJ source code project into Eclipse.
   • Create a public class that implements the LIProtocol interface.
   • Recommended naming convention: [Name]Protocol Replace [Name] with your chosen protocol name, and place the class in the src/protocol package.
   Example: public class FtpProtocol implements LIProtocol The FTP protocol class is named FtpProtocol.
2. Generate Interface Methods
   • Use Eclipse's Override/Implement Methods function to auto-generate method skeletons for the interface.
   • Implement the easy methods first, based on your earlier design decisions:
   Example (FTP Protocol): public TALK_FIRST whoTalksFirst() { return TALK_FIRST.SVR_FIRST; } public String toString() { return "FTP"; } public int getPort() { return 21; }
3. Implement Connection State Tracking
   • The two challenging methods are isConnectionOver() and processInput().
   • Maintain a member variable (e.g., connectionState) to track the current state of the connected client.
   • In processInput(), use a switch statement (or equivalent logic) based on connectionState to:
     • Validate incoming messages.
     • Determine the correct response.
4. Implement Connection Termination
   • isConnectionOver() should return true when the connection is closed, for example: public boolean isConnectionOver() { return connectionState == KILLED; }
5. Sending Responses
   • The processInput() method must return a Vector<String> as the response to an incoming String message.If only one response string is required, use the helper method: LIHelper.vectorFromString("Response message"); This creates a Vector containing a single String.
   Example method stub: public Vector<String> processInput(String msg) { // Implementation logic here }

4.4.3 Adding the Protocol

After creating your protocol class, you need to register it with the HoneyRJ application.

Steps:

1. Open the main application file
   • Navigate to: src/honeyrj/HoneyRJMain.java
2. Modify the main() method
   • Locate the section immediately after: createSampleProtocols();
   • Add the following lines: LIProtocol yourProtocol = new YourProtocolClass(); LIModule yourModule = new LIModule(yourProtocol); gui.AddModule(yourModule);
   Example for FTP Protocol:LIProtocol ftpProtocol = new FtpProtocol(); LIModule ftpModule = new LIModule(ftpProtocol); gui.AddModule(ftpModule);
   • Line 1: Replace YourProtocolClass with the name of your class (e.g., FtpProtocol).
   • Line 2: Creates an LIModule that wraps your protocol and provides socket communication.
   • Line 3: Adds the protocol module to HoneyRJ and updates the GUI.
3. Run the application
   • Execute HoneyRJMain.java as a Java application.
   • Your protocol will now be available in the GUI and operational in HoneyRJ.

5. Summary

Intrusion Detection Systems (IDS) have traditionally been used by network administrators to monitor traffic for unauthorized activity. However, with the rise of encrypted network connections—which IDS cannot easily inspect—honeypots have become an attractive alternative for identifying malicious sources.

Honeypots, first introduced in 1998, operate by recording all connections and connection attempts. They are typically deployed on unused IP addresses so that no legitimate traffic should ever reach them.

There are two main types of honeypots:

• Low-interaction honeypots – Provide partial implementations of common protocols. Their goal is to log the source of malicious traffic while keeping the system simple and safe.
• High-interaction honeypots – Often full-fledged servers with advanced monitoring tools, designed to help researchers understand an attacker’s behavior and decision-making process.

The field of honeypots continues to evolve, with modern developments including honeynets (networks of interconnected honeypots) and distributed honeypots that operate across multiple systems or locations.

This work presented a case study of a low-interaction honeypot written in Java, called HoneyRJ. HoneyRJ offers basic logging capabilities and is easily extendable, making it a flexible choice for modern low-interaction honeypot deployments.