Frequently Asked Questions
Table of Contents
General Questions
How should Tcpreplay be capitalized?
When refering to the Tcpreplay suite of tools (tcpreplay, tcpprep, tcprewrite, etc) then the T is capitalized. When only refering to the tcpreplay standalone utility, then it's not. There is no such thing as 'TCPreplay' or 'TCP replay' as far as I'm concerned.
Is there a Microsoft Windows port?
Old Answer: Not really. We had one user port the code over for an old version of tcpreplay to Windows. Now we're looking for someone to help merge and maintain the code in to the main development tree. If you're interested in helping with this please contact Aaron Turner or the tcpreplay-users list. Other then that, you can download the tcpreplay-win32.zip file from the downloads section and give it a go. Please understand that the Win32 port of tcpreplay comes with no support whatsoever, so if you run into a problem you're on your own.
New Answer: As of 3.0.beta13, Windows 2000 and better are supported under Cygwin. My goal is to in the future natively support Windows, but I will most likely need some help with this. For more information, please read the Win32Readme.txt file.
How is Tcpreplay licensed?
Tcpreplay is licensed under a three clause BSD-style license. For details see the docs/LICENSE file included with the source code.
Running Tcpreplay
Does tcpreplay support sending traffic to a server?
If by server you mean a daemon (Unix) or service (Windows) which listens on a port (a web or mail server would be common examples), then probably not. The biggest issue is that tcpreplay doesn't understand the state of common protocols like TCP. This means it is unable to syncronize Syn/Ack's to create valid TCP sessions.
In theory, ICMP and UDP based protocols should work fine if the MAC and IP addresses are correct. You can fix them using tcprewrite. However, some ICMP and UDP based protocols also carry Layer 3/4 protocol information in the protocol itself (SIP is one example), so if you change the IP addresses, it may not be a valid SIP packet anymore. In such cases, you may want to try using NetDude to edit the payload of the packets.
In the future, I hope to release flowreplay to allow you to replay pcap files to servers.
Can I send packets on the same computer running tcpreplay?
Generally speaking no. When tcpreplay sends packets, it injects them between the TCP/IP stack of the system and the device driver of the network card. The result is the TCP/IP stack system running tcpreplay never sees the packets.
One suggestion that has been made is using something like VMWare, Parallels or Xen. Running tcpreplay in the virtual machine (guest) would allow packets to be seen by the host operating system.
Older versions of tcpreplay allowed me to edit packets. What happened?
Version 2.x of tcpreplay allowed you to edit packets as they were being sent. This functionality was moved to tcprewrite and disabled by default in 3.0 due to performance reasons. If you're not as worried about performance and would like to edit packets as they are being sent, you can configure tcpreplay with --enable-tcpreplay-edit.
Why do I need to use tcpprep to create cache files? Can't this be done in tcprewrite?
Most of the tcpprep modes take a holistic view of a pcap and looks at all the packets before deciding which IP's are servers and which are clients. This requires the pcap file to be processed fully, sometimes twice. This for all practicality excludes including the tcpprep logic in tcpreplay.
Secondly, since both tcpreplay and tcprewrite use tcpprep cache file data, it made sense to separate it into a standalone utility so that the processing can be leveraged across multiple rewrites and the actual packet sending.
Why is tcpreplay not sending all the packets?
Every now and then, someone emails the tcpreplay-users list, asking if there is a bug in tcpreplay which causes it not to send all the packets. This usually happens when the user uses the -t flag or is replaying a high-speed pcap file (> 50Mbps, although this number is dependant on the hardware in use).
The short version of the answer is: no, we are not aware of any bugs which might cause a few packets to not be sent.
The longer version goes something like this:
If you are running tcpreplay multiple times and are using tcpdump or other packet sniffer to count the number packets sent and are getting different numbers, it's not tcpreplay's fault. The problem lies in one of two places:
- It is well known that tcpdump and other sniffers have a problem keeping up with high-speed traffic. Furthermore, the OS in many cases lies about how many packets were dropped. Tcpdump will repeat this lie to you. In other words, tcpdump isn't seeing all the packets. Usually this is a problem with the network card, driver or OS kernel which may or may not be fixable. Try another network card/driver.
- When tcpreplay sends a packet, it actually gets copied to a send buffer in the kernel. If this buffer is full, the kernel is supposed to tell tcpreplay that it didn't copy the packet to this buffer. If the kernel has a bug which squelches this error, tcpreplay will not keep trying to send the packet and will move on to the next one. Currently I am not aware of any OS kernels with this bug, but it is possible that it exists. If you find out that your OS has this problem, please let me know so I can list it here.
If for some reason, you still think its a bug in tcpreplay, by all means read the code and tell me how stupid I am. See sendpacket.c.
Why are tcpreplay timings all messed up?
Occasionally someone complains about timings to be "messed up". Usually this seems to be caused by a pcap which contains packets with non-sensical timestamps. More specifically, I have seen cases where a packet has a timestamp before the previous packet in the capture file. I'm not sure how such a pcap got created, but it seems to occasionally happen.
There may be other reasons for this to happen, such as a bug in tcpreplay, so be sure you're running the latest version of code. If after checking these two possibilities, please contact the author so that he can help debug the issue.
Does tcpreplay support dual NIC's like Tomahawk?
Yes! tcpreplay has for many years supported high-speed playback using two interfaces (long before Tomahawk existed). For more information about how traffic is split between the two interfaces, see the tcpprep manpage.
Can tcpreplay read gzip/bzip2 compressed files?
Yes, but not directly. Since tcpreplay can read data via STDIN, you can decompress the file on the fly like this:
gzcat myfile.pcap.gz | tcpreplay -i eth0 -
Note that decompressing on the fly will require additional CPU time and will likely reduce the overall performance of tcpreplay.
How fast can tcpreplay send packets?
First, if performance is important to you, then upgrading to tcpreplay 3.x is worthwhile since it is more optimized then the 2.x series. After that, there are a number of variables which effect performance, including on how you measure it (packets/sec or bytes/sec).
Anyways, here are some sample test runs from an Intel P4 3.4Ghz using the Intel e1000/Pro network card with a Linux 2.6.x kernel:
First run (larger packets):
tcpreplay -t -K -l 100000 -i eth2 -q --pktlen test/test.pcap
Actual: 10000000 packets (7020500000 bytes) sent in 58.95 seconds
Rated: 120843848.0 bps, 921.97 Mbps/sec, 172129.97 pps
Statistics for network device: eth2
Attempted packets: 10000000
Successful packets: 10000000
Failed packets: 0
Retried packets: 0
Second run (smaller packets):
tcpreplay -t -K -l 100000 -i eth2 -q test/test.pcap
Actual: 10000000 packets (1062900000 bytes) sent in 16.63 seconds
Rated: 63895116.0 bps, 487.48 Mbps/sec, 601139.50 pps
Statistics for network device: eth2
Attempted packets: 10000000
Successful packets: 10000000
Failed packets: 0
Retried packets: 0
The key thing here is that when sending smaller packets (the 2nd run) the packets/second is much higher (almost 4x), but the overall Mbps is lower (about 1/2). Using a much more powerful system (dual quad-core Xeon 2.66Ghz) the first run was virtually identical while the smaller packets showed a noticeable improvement:
tcpreplay -t -K -l 100000 -i eth1 -q test/test.pcap
Actual: 10000000 packets (1062900000 bytes) sent in 15.28 seconds
Rated: 69537776.0 bps, 530.53 Mbps/sec, 654226.88 pps
Statistics for network device: eth1
Attempted packets: 10000000
Successful packets: 10000000
Failed packets: 0
Retried packets: 0
Note Once you start limiting how quickly packets are sent (anything other then -t), there is a rather dramatic drop off in performance. I'm still working on how to create a low-overhead means of timing packets.
How can I make tcpreplay run faster?
Profiling tcpreplay has shown that a significant amount of time is spent writing packets to the network. Hence, your OS kernel implimentation of writing to raw sockets is one of the most important aspects since that is where tcpreplay spends most of it's time.
In no particular order:
- Hardware:
- Use a fast enough hardware. tcpreplay is very dependant on disk I/O, your network card and CPU.
- When selecting disk drives for storing pcap files, choose drives with higher RPM over lower seek times or larger caches.
- SCSI disks with a good controller should provide the best performance, since they generally require the CPU to do less work.
- For best performance, consider some sort of striping RAID (like level 0 or 5) to increase read throughput performance.
- Don't store pcap files on a remote server (NFS or SMB mounted). That just increases latency and CPU utilization. Directly attached storage like SCSI or FibreChannel is fine.
- Since tcpreplay is not multi-threaded, SMP or dual-core processors won't help very much. However, other processes can run on the other CPU(s).
- Turn off hyperthreading (HT) if your CPU supports it.
- A good network card/driver is important. Realtek cards are known to be crap. Intel EEPro/1000 cards are cheap and work well.
- Configuration:
- If you have enough RAM to store the pcap files in a RAM disk, that would be ideal.
- If you don't have enough RAM for a RAM disk, run tcpreplay against the file once in order to load parts of your file in your OS's disk cache. That will help improve performance the subsequent runs.
- If your computer is busy running a bunch of other processes (running X, downloading mail, etc) then it will impact tcpreplay's performance and ability to time packets correctly.
- Opening a bunch of small files repeatly will reduce performance. Consider using mergecap to generate a single large file.
- Like most network based I/O, it is faster to send the same amount of data in a few large packets then many small packets.
- If you are looping your pcap file, enabling the --enable-file-cache option should provide a significant performance boost assuming you have enough RAM.
Does tcpreplay support Endace DAG cards?
By default, tcpreplay does not support DAG cards. However, Endace has released a custom version of tcpreplay which does support their cards. Please note that the Tcpreplay developers do not support this custom version of tcpreplay, so if you have any questions, please contact Endace.
Can I use non-pcap capture files?
It turns out that there are quite a few capture file formats other then pcap. If you have a capture file created by a tool which uses one of these other formats (like Solaris snoop) you can convert it to pcap format by using Wireshark's tshark tool.
tshark -r blah.snoop -w blah.pcap
Can tcpreplay send packets over WiFi?
This turns out to be very OS/hardware dependent, but in many cases, the answer is yes. In order for things to work, you generally must do the following:
- Put the WiFi card in managed mode
- Your pcap files need to be DLT_EN10MB (Ethernet) and have a valid 803.2 header
- The source MAC of the packets need to match the MAC of your WiFi card
Compiling Tcpreplay
Are there binaries available for XXX operating system?
Maybe. I do not release binaries for ANY operating system. Many operating systems like Linux, *BSD, Solaris and OS X have teams which package open source applications like tcpreplay and release them in their package format (RPM, BSD/Mac ports, SunFreeware, etc). Google is your friend.
What if I ask you really nicely to build a binary for me?
If by "nice" you mean you'll buy me some Boddingtons or a DVD off my Amazon wish list, yeah, I'll prolly help you out if I can. If by "nice" you mean saying "pretty please with sugar on top", then start Googling.
Unable to find a supported method to send packets. Please upgrade your libpcap or enable libnet
Tcpreplay can use a variety of API's/libraries to send packets: BSD's BPF, Linux's PF_PACKET, libpcap and libnet. If you're not running on a platform which supports BPF or PF_PACKET, you'll need either a recent version of libpcap or libnet. Honestly, between the two, I strongly suggest upgrading libpcap since it tends to have fewer bugs and is actively developed. Libnet by contrast has known bugs and hasn't been updated in a few years. For all intents and purposes, libnet support has been deprecated and will likely be removed in a future version.
Right now there is one case where libnet is necessary: you're not running on Linux or *BSD and you want to use tcpbridge. Right now there are limitations in the libpcap API which prevents tcpbridge from working when BPF and PF_PACKET aren't available.
tcpedit_stub.def: Command not found
Making all in tcpedit make[2]: Entering directory `/home/acferen/tcpreplay-trunk/src/tcpedit' tcpedit_stub.def make[2]: tcpedit_stub.def: Command not found make[2]: *** [tcpedit_stub.h] Error 127 make[2]: Leaving directory `/home/acferen/tcpreplay-trunk/src/tcpedit' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/acferen/tcpreplay-trunk/src' make: *** [all] Error 2
You should only get this error if you're trying to build from SVN. The problem is that you do not have GNU Autogen installed on your system. Either install autogen or download one of the source tarballs.
libnet/libnet-types.h:36:23: error: ../config.h: No such file or directory
Libnet version 1.1.3-RC has a bug in one of the header files. If you get an error during compilation:
/usr/local/include/./libnet/libnet-types.h:36:23: error: ../config.h: No such file or directory
Then you should edit /usr/local/include/libnet/libnet-types.h (or wherever it is installed) and comment out or delete the three lines (should be lines 35-38):
#ifdef HAVE_CONFIG_H #include "../config.h" #endif
Feel free to bitch to libnet@… since they seem to be ignoring me.
configure: error: Unable to determine version of libnet
This is usually caused by one of two reasons:
- You have two versions of libnet installed on your system and configure can't figure out which to use. Upgrading to the latest 3.x version and using --with-libnet should fix that.
- You have not actually run 'make install' for libnet and installed it on your system. You can't point --with-libnet to the libnet source tree.
Issues with autogen/libopts
The Tcpreplay suite uses the GNU Autogen/libops library. This makes development much easier, but not without some cost. Basically, Autogen/libopts doesn't always provide good backwards compatibility, so if you have a different version of Autogen/libopts then the author does, you may have problems compiling tcpreplay. One example error would be:
make[3]: Entering directory `/root/services_from_source/tcpreplay-3.0.beta11/src/tcpedit' if gcc -DHAVE_CONFIG_H -I. -I. -I../../src -I.. -I../common -I../.. -g -O2 -Wall -O2 -funroll-loops -std=gnu99 -MT tcpedit.o -MD -MP -MF ".deps/tcpedit.Tpo" -c -o tcpedit.o tcpedit.c; \ then mv -f ".deps/tcpedit.Tpo" ".deps/tcpedit.Po"; else rm -f ".deps/tcpedit.Tpo"; exit 1; fi In file included from tcpedit.c:46: tcpedit_stub.h:18:30: autoopts/options.h: No such file or directory In file included from tcpedit.c:46: tcpedit_stub.h:50: error: syntax error before "const" tcpedit_stub.h:50: warning: type defaults to `int' in declaration of `tcpedit_tcpedit_optDesc_p' tcpedit.c:58: error: syntax error before "const" tcpedit.c:58: warning: type defaults to `int' in declaration of `tcpedit_tcpedit_optDesc_p' make[3]: *** [tcpedit.o] Error 1
The good news is that there is an easy fix: ./configure --enable-local-libopts
Problems with linking under recent Fedora Core/Red Hat
Newer versions of Fedora Core and Red Hat ES/WS do not ship static libraries at all and only have dynamic libraries. If you wish to compile tcpreplay from source, you will need to pass the --enable-dynamic-link to configure in order for tcpreplay to link to them.
Common Errors
Unable to send packet: Error with pcap_inject(packet #10): send: Message too long
There is a bug in OS X (at least up to 10.4.9) which causes problems for ethernet frames > 1500 bytes from being sent when you spoof source MAC addresses (like tcpreplay/tcpbridge does). Apple fixed this in 10.5 (Leopard). Currently there is no work around.
Can't open eth0: libnet_select_device(): Can't find interface eth0
Generally this occurs when the interface (eth0 in this example) is not up or doesn't have an IP address assigned to it.
Can't open lo: libnet_select_device(): Can't find interface lo
Version 1.1.0 of Libnet is unable to send traffic on the loopback device. Upgrade to a later release of the Libnet library to solve this problem.
Libnet can't open hme0: libnet_select_device(): ifaddrlist : SIOCGIFCONF: Invalid argument
Current versions of libnet don't support Solaris network interfaces with many subinterfaces (aka aliased interfaces). Either delete some subinterfaces, use a different operating system or complain to the libnet developers.
Can't open eth0: UID != 0
Tcpreplay requires that you run it as root.
100000 write attempts failed from full buffers and were repeated
When tcpreplay displays a message like "100000 write attempts failed from full buffers and were repeated", this usually means the kernel buffers were full and it had to wait until memory was available. This is quite common when replaying files as fast as possible with the "-R" option. See the tuning OS section in this document for suggestions on solving this problem.
Invalid mac address: 00:00:00:00:00:00
Oldver versions of tcpreplay (prior to 3.x) reserves the MAC address of 00:00:00:00:00:00 as reserved for internal use. Hence you can't rewrite the MAC address of packets to be all zeros.
Unable to process test.cache: cache file version missmatch
Cache files generated by tcpprep and read by tcpreplay are versioned to allow enhancements to the cache file format. Anytime the cache file format changes, the version is incremented. Since this occurs on a very rare basis, this is generally not an issue; however anytime there is a change, it breaks compatibility with previously created cache files. The solution for this problem is to use the same version of tcpreplay and tcpprep to read/write the cache files. Cache file versions match the following versions of tcpprep/tcpreplay:
- Version 1:
Prior to 1.3.beta1
- Version 2: 1.3.beta2 to 1.3.1/1.4.beta1
- Version 3: 1.3.2/1.4.beta2 to 2.0.3
- Version 4: 2.1.0 and above. Note that prior to version 2.3.0, tcpprep had a bug which broke cache file compatibility between big and little endian systems.
Skipping SLL loopback packet.
Your capture file was created on Linux with the 'any' parameter which then captured a packet on the loopback interface. However, tcpreplay doesn't have enough information to actual send the packet, so it skips it. Specifying a destination and source MAC address (-D and -S) will allow tcpreplay to send these packets.
Packet length (8892) is greater then MTU; skipping packet.
The packet length (in this case 8892 bytes) is greater then the maximum transmition unit (MTU) on the outgoing interface. Tcpreplay must skip the packet. Alternatively, you can specify the -T option and tcpreplay will truncate the packet to the MTU size, fix the checksums and send it.
tcpreplay doesn't send entire packet
You won't see any error from tcpreplay, but sometimes you'll open a pcap file in Ethereal/Wireshark and notice that a packet is something like 400 bytes but tcpreplay says it only sent 100 bytes. I've seen this 2 or 3 times and in each case the reason was the same: the pcap file was broken.
First a little background on the pcap file format. At the beginning of each file there is a file header which contains among other things the snaplen. The snaplen is the maximum amount of data that is stored in file per packet; if a packet is larger then this value, the packet is truncated. Each packet is also prefixed with a packet header which contains two values: the len and caplen. The len is the original packet size and the caplen is the amount of actual data which was stored.
A properly formatted pcap file will never have a caplen > snaplen. Some tools and applications unfortunately do not seem to enforce this. So when the libpcap library reads these files, it returns the snaplen as the actual data available. The reason Ethereal/Wireshark show the entire packet is because they do not use libpcap to read pcap files.
There are two ways to fix this:
- Fix your pcap file. (Best)
- Use the --pktlen flag under tcpreplay 3.x (May not always work)
Interesting Tickets
This is a list of tickets which people should be aware of since they affect the usage of tcpreplay.
- #130
- Gracefully handle broken pcaps where snaplen < caplen
- #167
- tcpreplay causes OS X WiFi to disassociate from AP
- #304
- Document building autogen/guile under Cygwin
- #142
- Unable to send full sized packets on OS X
- #114
- Selecting only some packets to send (filtering)
- #147
- --seed is fully random on some platforms
- #151
- Unable to spoof source MAC address on OS X
- #183
- tcprewrite says "skipping packet" but doesn't
- #321
- replaying packets over a slip connection in windows fails
