Opened 7 years ago

Last modified 6 years ago

#500 new enhancement

NewConn: replaying on live networks

Reported by: aturner Owned by: yhsiam
Priority: high Milestone: 3.5.0
Component: tcpreplay Version: 3.4.4
Keywords: Cc: yhsiam@…, pankab@…
Operating System: Add to FAQ?: no
Hardware: All
Output of tcpreplay -V: tcpreplay version: 3.4.5yhsiam (build 2551M) Copyright 2000-2010 by Aaron Turner <aturner at synfin dot net> Cache file supported: 04 Not compiled with libdnet. Compiled against libpcap: 1.2.1 64 bit packet counters: enabled Verbose printing via tcpdump: enabled Packet editing: disabled Fragroute engine: disabled Injection method: PF_PACKET send()

Description

Hi Aaron,

I am a member of a team of undergraduate students who are working on extending the functionality of Tcpreplay3.4.4. Panos has initiated a contact with you about 1.5 weeks ago regarding the Makefile.

We've been looking at the code to try to understand your code and what is it that we must do in order that we complete this exciting project. It is introducing tcpreplay_newconn which will allow pcaps to be replayed on live networks for testing purposes. This will involve replaying a packet capture and actively respond with the destination device while modifying seq and ack numbers.

At the moment, I have tried to add in a simple file to see how to integrate our code with the existing suite. I modified the Makefile.am but unsure if I did it correctly because it compiles but errors out on linking the object files.

I have modified some of the attached code to include simple functions for testing purposes. Look for " * Y.H.S. " that tags my additions.

Any suggestions as to why this is not compiling correctly?

Thank You

Attachments (1)

tcpreplaydeveltcpreplaynewconn.zip (15.7 KB) - added by aturner 7 years ago.
updated files

Download all attachments as: .zip

Change History (56)

Changed 7 years ago by aturner

updated files

comment:1 Changed 7 years ago by aturner

(In [2547]) create branch for Yazan Siam. refs #500

comment:2 Changed 7 years ago by aturner

(In [2548]) update to GNU Autogen 5.11.5
small update autoconf tools
mark version as part of yhsiam branch should anyone ever ask me about it
refs #500

comment:3 Changed 7 years ago by aturner

(In [2550]) oops... didn't mean to include these files in the last commit
refs #500

comment:4 Changed 7 years ago by yhsiam

(In [2552]) adding newconn test files refs #500

comment:5 Changed 7 years ago by yhsiam

(In [2553]) Makefile edits to include libraries refs #500

comment:6 Changed 7 years ago by ableonar

(In [2554]) refs #500
Test commit by andrew

comment:7 Changed 7 years ago by pkampana

(In [2555]) refs #500 Panos testing commit, insignificant code change

comment:8 Changed 7 years ago by pkampana

  • Cc pankab@… added
  • Output of tcpreplay -V modified (diff)
  • Type changed from defect to enhancement

comment:9 Changed 7 years ago by yhsiam

(In [2556]) I am adding in work we have so far on the reading+editing packets based on
command line args. Some of the current code still needs adjusting
like checksum and "Next Calculated local/remote packet SEQ/ACK numbers.
None of the replay, listening in on traffic or sending packets have been
implemented yet.

refs #500

comment:10 Changed 7 years ago by aturner

(In [2557]) fix building doxygen in this branch. refs #500

comment:11 Changed 7 years ago by aturner

Added Doxygen documentation for this branch here. Auto-updated daily.

http://tcpreplay.synfin.net/doxygen_yhsiam/

comment:12 Changed 7 years ago by yhsiam

(In [2558]) Checksum works, function now creates a new file called "newfile.pacp" with ONLY local traffic to be sent. Then function creates a schedule and
sets up the relevant data to be used for the rest of the program. A big portion of main (the portion of checking for flags) needs to be moved into a
separate function and should properly set the schedule based on the recevied and sent packets.

Next step:
-create function for checking received/to-be-sent data flag, data length & sequence numbers
-properly maintain live traffic handler while also handling schedule traffic
-implement the send/receive algorithms based: remote packet loss, local packet loss, normal operation

refs #500

comment:13 Changed 7 years ago by aturner

Just a quick comment. That last change to src/common.h in [2558] #includes headers from files outside of the src/common directory which is counter to my goals of seperation. If you want to include those 3 headers in your code, please include them manually in your own code... that will make it easier for me to merge your code into mainline/trunk later. Thanks!

comment:14 Changed 7 years ago by yhsiam

(In [2559]) Fixing src/common.h back to original format as per Aaron's request. Thank you Aaron!

refs #500

comment:15 Changed 7 years ago by yhsiam

(In [2560]) Hi Aaron, we are slowly making some progress on our project. At this point we are still testing various things of our design
after looking at what our project requires. But currently I am running into a compliation error that i cannot seem to figure out why.
Error: undefined reference to `didsig'

I have tried to define 'didsig' as you have specified in your sendpacket.c file but that doesnt fix it. Also, if you would like to look
at our code, we welcome any comments from you that can help us do this in the most efficient manner.

Thank You Always for your Time!

refs #500

comment:16 Changed 7 years ago by yhsiam

(In [2561]) complies, not yet tested.

refs #500

comment:17 Changed 7 years ago by yhsiam

(In [2562]) Segfault due to incorrect use of memcpy on line 280 of setup_sched() function. I am trying to copy the pointer state so that we are
able to re-reference to (re-send) the same data later on as we handle packet loss. I cannot figure out what's the issue with memcpy as i maybe
overlooking a mistake. Anyone knows what's going on?

refs #500

comment:18 Changed 7 years ago by yhsiam

(In [2563]) Added packet handeling functionality for packet loss. Time for testing sending/receiving function with a given pcap and using tcpreplay to replay remote packets only back to tcpreplaynewconn

refs #500

comment:19 Changed 7 years ago by yhsiam

(In [2564]) Segfault because of incorrect use of sending packets using sendpackets() and incorrect use of receiving from live handler (needs to be debugged).

But struct schedule has been updated and tested to work properly. All data pointers and pointers to each of the headers are there in the schedule. They are to be used for sending local data and editing remote packet expectations based on previous packet's statuses.

Once sending/receiving works properly, then we need to add in code to handel TCP flag checking so that we properly update the next expectation of remote packets within the schedule.

refs #500

comment:20 Changed 7 years ago by yhsiam

(In [2565]) sending is starting to work. Test is and see what it does. I read in the pcap file to act as a packet "received" when there is a remote packet scheduled. And I send local packets when local packets are scheduled.

Test it and take a packet capture to see what happens. You will see some remote packets mistakenly being sent.

refs #500

comment:21 Changed 7 years ago by yhsiam

(In [2566]) tcp filter is done and tested. Schedule is fixed.

refs #500

comment:22 Changed 7 years ago by yhsiam

(In [2567]) relative numbers fixed in schedule for remote SEQ & local ACK checking

refs #500

comment:23 Changed 7 years ago by yhsiam

(In [2579]) more filter testing

refs #500

comment:24 Changed 7 years ago by ableonar

(In [2580]) refs #500 handshake works, after that seq and ack #s incorrect

comment:25 Changed 7 years ago by ableonar

(In [2581]) refs #500 begin migration to pcap_loop

comment:26 Changed 7 years ago by yhsiam

(In [2582]) pcap_loop callbak

refs #500

comment:27 Changed 7 years ago by ableonar

(In [2583]) refs #500 prep to move loop handling code into pcap_loop

comment:28 Changed 7 years ago by yhsiam

(In [2584]) Still working on pcap_loop to continuously sniff packets and place them in a buffer. Must fix got_packet callback function.

refs #500

comment:29 Changed 7 years ago by yhsiam

(In [2585]) random number generation added for testing

refs #500

comment:30 Changed 7 years ago by ableonar

(In [2586]) refs #500 pcap_loop bug testing

comment:31 Changed 7 years ago by yhsiam

(In [2587]) minor changes. Still debugging. Initial handshake completes but a remote packet is not sent from remote host at all
when it is expected to be sent from remote and arrive at local host. This is being tested replaying a telnet pcap against
another vm.

refs #500.

comment:32 Changed 7 years ago by yhsiam

(In [2588]) I added minor functionality to change the sport to continue testing the consistent packet loss problem. Now add source port after the new remote mac argument.

Also, the entire schedule is now fixed to absolute (rather than relative) once a SYN-ACK is received.

refs #500.

comment:33 Changed 7 years ago by yhsiam

(In [2589]) Code update up to now, but still the same problem of packet response from remote host persists.

refs #500

comment:34 Changed 7 years ago by ableonar

(In [2590]) refs #500 retransmission/skipping acked packets

comment:35 Changed 7 years ago by yhsiam

(In [2591]) tcpreplaynewconn now WORKS with one TCP connection replay!

I tested http packet and telnet packet. They both replay now fine and no timing delay is needed as of now because the schedule does not proceed in sending packets until remote packets are received.

What to work on next:
-Testing packet loss code
-RST flag received, what to do?
-Figure out how to implement multiple TCP replays against multiple ports

refs #500

comment:36 Changed 7 years ago by yhsiam

(In [2592]) Packet loss, imbedded not yet tested. 'Same ACK packet skipped' function has been also added and tested.

refs #500

comment:37 Changed 7 years ago by yhsiam

(In [2593]) Minor changes that i updated since the last commit

refs #500

comment:38 Changed 6 years ago by yhsiam

(In [2594]) skipping function added and tested.

refs #500

comment:39 Changed 6 years ago by ableonar

(In [2595]) refs #500 added random_port()

comment:40 Changed 6 years ago by yhsiam

(In [2596]) Implemented the following:

  • fix for bug with stuck condition when server changes the packet order
  • random vs user specified source port
  • RST close conn and throw error
  • exit from any kind stuck condition after meeting a threshold, and throw message

refs #500

comment:41 Changed 6 years ago by yhsiam

(In [2608]) Implemented & Tested the following:

  • Early FIN-ACK global variable check
  • Packet loss handling
  • Differing packet payload than expectation from capture being replayed

refs #500

comment:42 Changed 6 years ago by yhsiam

(In [2609]) window update added

refs #500

comment:43 Changed 6 years ago by yhsiam

(In [2610]) Code is now ready for Cisco testing of one connection replay.

refs #500

comment:44 Changed 6 years ago by yhsiam

(In [2611]) minor changes/updates

refs #500

comment:45 Changed 6 years ago by yhsiam

(In [2612]) unknown error fixed

refs #500

comment:46 Changed 6 years ago by yhsiam

(In [2613]) proper send/receive print

refs #500

comment:47 Changed 6 years ago by yhsiam

(In [2614]) Project requirements completed for 1 tcp flow replay. Ready Cisco testing.

refs #500

comment:48 Changed 6 years ago by yhsiam

(In [2615]) Enhancement on better summary printouts for Cisco debugging purposes.

refs #500

comment:49 Changed 6 years ago by yhsiam

(In [2616]) printout summary adjustments

refs #500

comment:50 Changed 6 years ago by yhsiam

(In [2617]) warning and error printing to the console added to help in debugging replay

refs #500

comment:52 Changed 6 years ago by yhsiam

(In [2630]) The tool is now called tcpliveplay.

Next stage: Make sure all the code is commented appropriately. Code structure/design

must be documented.

refs #500

comment:53 Changed 6 years ago by yhsiam

(In [2631]) Done The Following:

  1. Intensive code comments added
  2. Code Design comments
  3. Scheduler function & structure commenets

Next to be done:

  1. If there is a missed packet by the server side, and you are not seeing the SEQ and ACK

you are expecting you need to dupACK, and not have the server wait and resend himself. FIX THIS!

  1. Add support for compiling under ./configure --enable-debug.
  1. Disable compiling on unsupported platforms.
  1. Write a man page for using it
  1. Document in relative detail what it does and what kind of pcap's

should/won't work on the wiki. Basically, set expectations with
potential users for when your tool is appropriate.

  1. Include at least one test case for 'make test'

refs #500

comment:54 Changed 6 years ago by yhsiam

(In [2632]) Done The Following:

  1. Added support for compiling under ./configure --enable-debug.
  2. Disabled compiling on unsupported platforms. Now it should only compile for linux.
  3. Wrote a man page explaining how to use the tool. Serves as a quick overview of the tool.
  4. Documented relative "Bugs" in the man page of the current state of the tool.

Next:

  1. If there is a missed packet by the server side, and you are not seeing the SEQ and ACK you are expecting you need to dupACK, and not have the server wait and resend himself. FIX THIS!
  1. Document in relative detail what it does and what kind of pcap's should/won't work on the wiki. Basically, set expectations with potential users for when your tool is appropriate.
  1. Include at least one test case for 'make test'

refs #500

comment:55 Changed 6 years ago by aturner

  • Milestone set to 3.5.0
  • Owner changed from aturner to yhsiam
Note: See TracTickets for help on using tickets.