[Linux] Network transfer rates

For my diploma thesis I need to transfer monitored data from a test PC running the L4/DROPS operating system to my Linux workstation. A kind of weird idea of my tutor was to write all the data to a hard disk with no file system or so and then switch the disk to the other workstation and have a program read all the data. The good thing about the idea would be a high transfer rate. The bad thing is, that no real-time monitoring could become possible.

As I've been developing a software network switch for L4 last year, I thought that transferring data through a network could work great. For this I ported a small IP stack implementation, uIP, to use L4 and my network switch. Transferring small amounts of log data is no problem with it, but for large transfers we encountered the limits of this stack. Some of its small size results from minimal TCP features implemented and the drawback for large transfers is uIP being able only to send one TCP packet at a time and then waiting until the packet gets acked by the receiver.

On my Linux machine I started using netcat to receive all the data and store it to a file. What I experienced there was a relatively fast start of transmission (~20 kB/s in the first 2 seconds) and then a sudden slowdown to only 2kB/s afterward. Ethereal can really be your friend when diagnosing network connections and so I found out that the Linux machine started sending acknowledgements fast and after some time considered the TCP-stream to be one-way and only sent one acknowledge message every 0.5 seconds. This is okay for a normal TCP stack, because TCP allows acknowledgement of several packets with one ACK message. uIP however needs the ACK to send the next packet and therefore everything messed up and became slow.

After some time of discussion (with even considering the option of porting yet another IP stack to L4) and searching, I found that there is an option for TCP sockets, putting this socket into the QUICKACK mode, where all packets are acked immediately. So I abandoned netcat and wrote my own little socket application that puts its receive socket into this mode for each receive. This finally gave me data transfer rates of 2.2 MB/s - 1000 times better than before.

If I should find this throughput too slow in the future (monitoring produces a lot of data), I will port the lwIP stack to L4, but I hope this will not be until I receive my diploma.