teepeedee file server

teepeedee is a small, hopefully scalable, single-threaded file server. It can serve or receive multiple files over http, https and ftp simultaneously, listening on any number of ports and providing entirely different views of the filesystem to different users. It uses sendfile and is written in C++.

It supports most of the draft-ietf-ftpext-mlst-16.txt FTP extensions, EPRT, EPSV, AUTH TLS, and does HTTP 1.1 persistant connections, and HTTPS.

teepeedee certainly runs on FreeBSD i586 4.7-STABLE, Linux i686 2.4.20 with glibc 2.3, Debian i686 Woody (20021220), Debian PPC Unstable (20021220).

teepeedee is not designed to run as root - it's supposed to be a convenient configurable fast file-server for personal use.

Benchmarks

I took an old Alpha 233 MHz workstation (160 MB RAM) running Linux 2.4.20 (Debian). From a 1 GHz Duron over an ethernet 100 MBit link, I ran 512 wgets downloading a 1 MB file from teepeedee 0.2.0 on the Alpha. I timed how long it took for all 512 transfers to complete. For this to work I raised the maximum number of open filedescriptors for teepeedee with ulimit -n 4000. It took roughly 99 seconds. Then I did the same test on the same file with vsftpd 1.1.3 on inetd (with rate 1024 clients per minute). This took roughly 120 seconds, and it seemed transfers were much less balanced (one client would get 9MB/s bandwidth, finish, then let the next get all the bandwidth and so on). For 100 transfers they finish in about the same time.

The test

start=`date +%s`;for i in `seq 1 512`; do wget -nv ftp://eurus:21/onemeg -O /dev/null& done; wait; echo $((`date +%s`-$start))
    

Conclusion

So vsftpd is 20% slower than teepeedee for 512 transfers, and will probably scale even worse as the number of transfers rises. OTOH both servers were not saturating the link, so possibly the problem was with the wget clients or something wasn't tweaked in the network stacks of the two machines.

teepeedee will probably scale even better on single CPU boxes, because of its single threaded design. On multiprocessor boxes vsftpd might have an advantage. Of course it would be fairly easy to get teepeedee to fork enough times to exploit the multiple processors but I don't have an SMP box so I haven't done it.

John Fremlin

Last modified: Sun May 25 17:34:14 BST 2003