FSP - File Service Protocol FAQ

Sven "Hoaxter" Hoexter

Radim "HSN" Kolar

Revision History
Revision 0.22005.01.03

This document is a major rewrite of the old FSP FAQ. I started with this FAQ from scratch cause nearly all parts of the old FAQ where outdated. This new FAQ is maintained in DocBook SGML. Send patches to the fsp-devel Mailinglist (FIXME link to id gethelpml) or directly to me . You should be able to find the latest SGML version of this FAQ in the fsp CVS at sf.net or here http://sven.stormbind.net/fsp/fsp-doc/fsp-faq.sgml (be aware the sf.net public CVS is often about a week behind the "real" CVS :( )


Table of Contents
1. Introduction
1.1. What is FSP?
1.2. FSP Project
1.3. Why and for what should I use FSP?
1.4. Comparison between fsp and other protocols
1.5. Benchmark results
1.5.1. FSP vs. http
1.5.2. FSP vs. ftp
1.5.3. FSP vs. tftp
1.6. Why should I run fspd?
1.7. Where can I download FSP?
1.8. Where can I find help?
1.8.1. RTFM - Read the fine Manuals
1.8.2. The Mailinglists
1.9. Licence, copyright and redistribution of FSP
1.10. Project history
1.10.1. The past
1.10.2. Today
1.11. Developers and contributors
2. Installation instructions
2.1. How to get FSP
2.1.1. How to obtain the source distribution
2.1.2. How to obtain a binary copy
2.2. How to compile the source code
3. How to set up your FSP server
3.1. How to install fsp server
3.2. How to quickly install fsp server
3.3. How to setup a basic fspd.conf
3.4. How can I start the fspd automagicly on reboot?
3.5. How to setup restrictions on directories?
4. How to use the fsp client tools
4.1. The fsp tool collection
4.2. fspclient
4.3. FSP PROXY: Using FSP in browser
4.4. Download Machine
4.5. FSP Win32 Suite
5. FSP technology in-depth
5.1. Protocol definition document
5.2. Transport mechanism used by FSP
5.3. Keying: Network bandwidth protection
5.4. Client side locking
5.5. Why not use bigger packet size?
5.6. Using FSP in your programs
5.7. How can I make FSP faster?
5.8. How can I make FSP slower?

1. Introduction

1.1. What is FSP?

FSP stands for File Service Protocol. In general FSP is what anonymous ftp should be: a reliable and bandwidth friendly way to access publicly available data.

FSP is a lightweight UDP based protocol for transferring files around. It has many benefits over FTP, mainly for running anonymous archives. FSP protocol is valuable in all kinds of environments because it is one of the only TCP/IP protocols that is not aggressive about bandwidth, while still being sufficiently fault tolerant.

1.2. FSP Project

FSP is (and always has been) open source project with MIT/X11 source code license. See Section 1.9 for more details.

In the past various people maintained the FSP code base. At the moment Radim Kolar is maintaining FSP. FSP project is now hosted on Source Forge.NET.

If you like to get involved take a look at the FSP Homepage and the FSP project page.

1.3. Why and for what should I use FSP?

The File Service Protocol has its strengths on slow lines and connections with a high packets lose like a Wireless LAN or radio packet networks. Because FSP is simple and lightweight, it is very well suited for use in embedded devices area.

FSP protocol is not aggressive about bandwidth and was designed to resist against users which want to do some bandwidth-related attacks. It can support more users, because the same available bandwidth is divided into smaller parts. Because FSP uses less bandwidth, FSP transfers takes about 2-3 times longer than with TCP-based protocols.

FSP protocol chooses reliability and simplicity over raw transfer speed. File transfer takes longer, but there are much more resistant against network failure.

Entire connection management is moved from server to client side, which solves problems with aborted and timed out transfer. Because FSP is stateless protocol, it can survive even server reboot without interruption.

More information about FSP use can be found in INFO document.

1.4. Comparison between fsp and other protocols

We did some benchmarking so that you can see how fsp performance compared to various other command protocols used to transfer files.

1.5. Benchmark results

You should understand that FSP is slower than TCP based protocols by design. TCP protocol can have more than one packet flying in network (usually 2-3) while FSP was designed to make it impossible (See Section 5.3).

All TCP protocols have on normal networks (max 15% packet loss) higher transfer rate than FSP. FSP is clean winner when it comes to overloaded sites or lines. These benchmarks were performed on 10Mbit LAN with zero traffic. Transfer rate is quite slow even for 10Mb because one of testing machines was old AMD K5/90 and both machines used cheap clone ISA NE2000 Ethernet cards.

Test file was 3656535 bytes long (FreeBSD kernel image). Test results shows, that FSP is about 2-2.5 times slower than TCP.

        tftp  15.2 secs 240.6 kB/s
        ftp   5.18 secs 688.7 kB/s
        http  5.32 secs 685.4 KB/s 
        sftp  8.0  secs 446.4 KB/s
        fsp   12.7 secs 287.7 kB/s
        

1.5.1. FSP vs. http

HTTP: widely supported by servers, clients and proxies. De-facto standard protocol for accessing information today. HTTP is also primary protocol used for file distribution today.

FSP: Can provide directory listings, which are not supported by standard HTTP/1.1 protocol. You need to have configured WebDAV server-side extension for that.

1.5.2. FSP vs. ftp

FTP: widely supported by servers and clients, more difficult to proxy than HTTP, uses 2 TCP connections, can provide directory listings in site-specific format, server side is more heavy service than HTTP. FTP is being replaced by HTTP today.

FSP: do not needs to make any connection, server side is very lightweight service, directory listings are standardized by protocol and very quick, no problems with NAT.

1.5.3. FSP vs. tftp

tftp is simple UDP (but TCP version also exists) protocol used mainly for loading boot and flash ROM images from network. Standard tftp uses 512 bytes long packets.

TFTP: more widely used, standardized in RFC, most network cards can boot image via TFTP.

FSP: about 1.5-times faster because of 1k packet size, supports directory listings, can query file date and size, can provide random access to file and supports opening more than one file, supports access control, timeout management moved to client side, stateless.

1.6. Why should I run fspd?

If you are providing anonymous archive (FTP,WWW), you should also offer FSP access. FSP daemon is a very lightweight process, never forks and you will never have more than one copy running around. On todays machines, it has zero performance impact.

Running fspd allows people with overloaded or lousy lines to get files from your archive. It also helps people during spikes periods when archive is very busy.

There are also some people which likes slower FSP downloads better, because it saves their bandwidth for interactive tasks. This is often preferred way of downloading large files, like CD-ROM images.

It is not true that UDP based protocols are unsecure. File Service Protocol provides the same security level as anonymous FTP and is much more resistant against denial of service attacks. It is nearly impossible to overload fsp server by flooding. On other side, TCP-based protocols are very simple to SYN flood.

1.7. Where can I download FSP?

You can Download the FSP source Distribution from the sf.net mirror network or from ibblio. If you have the FSP tools already you can access the latest FSP release via fsp on hxt.homelinux.org or wrack.telelev.net both running fspd on port 2221.

When you're interested in the latest, and of course greatest, development done in FSP please try a CVS checkout. Explanation and all needed information to do this can be found on the sf.net CVS page.

1.8. Where can I find help?

First of all various people took the time to write nice documentation on FSP including the fspd Server and the client tools. Please read those fine documents before you start writing to the mailinglists.

1.8.2. The Mailinglists

For developing issue write to the fsp-dev mainlinglist. For end-user questions and questions on how to configure your server/client write to the fsp-user Mailinglist. You can find more information on how to subscribe/unsubscribe to the mailinglist and the archive on the FSP mailinglist page on sf.net. Please before you write to one of the provided mailinglists read and understand ESRs HowTo ask smart questions and learn.to/quote

1.9. Licence, copyright and redistribution of FSP

FSP code base uses MIT/X11 license (also known as 2-point BSD license). This license is very friendly to users in commercional area. You can freely develop, use and sell your commercional implementation of File Service Protocol.

  1. Do not claim that you have wrote this software.

  2. Do not sue us if something breaks.

Summary: As long as you will not remove our copyrights from source code and will not go with your complains to the court, you can do with this software anything you want.

From the COPYRIGHT file in the source distribution (e-mail addresses removed):

          Authors:
          Radim Kolar  (Current FSP maintainer)
          Andrew Doherty
          Michael Fischbein
          Cimarron D. Taylor
          Guido van Rossum
          Wen-King Su
          Philip G. Richards
          Michael Meskes
          Rich $alz
          gjc@mitech.com
          Joseph_Traub
          Sven (VMS port maintainer)
          Very Mad Students, University of Karlsruhe, FRG (VMS port)
          
          Copyright:
          All of the FSP code is free software.  Most of fsp falls under two
          copyrights, the 2-point BSD copyright and one by Wen-King Su:
          
          Copyright (c) 1991 by Wen-King Su (wen-king@vlsi.cs.caltech.edu)
          You may copy or modify this file in any manner you wish, provided
          that this notice is always included, and that you hold the author
          harmless for any loss or damage resulting from the installation or
          use of this software.
          
          Other contributions to fsp fall under different copyrights:
          
          This file is Copyright 1992 by Philip G. Richards.  All Rights Reserved.
          See the file README that came with this distribution for permissions on
          code usage, copying, and distribution.  It comes with absolutely no warranty.
          
          Copyright (c) 1993 by Michael Meskes
          You may copy or modify this file in any manner you wish, provided
          that this notice is always included, and that you hold the author
          harmless for any loss or damage resulting from the installation or
          use of this software.
        

1.10. Project history

This should give you a short overview over the development of the File Service Protocol in the past and today.

1.10.1. The past

FSP development started in a very old days. From source code and man page time stamps we can see that it was working and alive in 1988. FSP versions 1.0 and 2.0 was born in Dec 1991. Release 2.0 was just bufixed 1.0 but includes man pages. After that two first releases FSP goes to the active life. People starts using it because it was superior to anonymous FTP at that time.

FSP active development era ends in May 1993 when last official version 2.7.1 was released. It was quite popular at that time, mainly because it was superior way for transferring warez around when compared with FTP. Because of this, FSP protocol got a bad name "Underground file sharing protocol" and suffers from it even today. I see that FSP warez era as practical demonstration how can FSP perform nicely on overloaded sites and lines.

After that then was some maintenance work on FSP in 1995 which ends with famous last official beta FSP version 2.8.1b3 in March 1996, which was coded by original FSP developers team. This version is used on some servers today because Debian Woody has it.

Last official stable FSP version was still 2.7.1 from May '93. This is still used on some forgotten machines in universities today. It has also known security issue (fspd follows all symlinks), but at this time this was viewed as feature, not a bug.

Radim Kolar released 2.8.1b4 in 2001, which was just some bugfixed version of 2.8.1b3 untouched from 1997 when I ported it to OS/2.

With wireless networks Wi-Fi boom in 2002, it becomes clear that large wireless networks can have quite a high packet loss rate and TCP protocol is not very suitable for them.

Active working on FSP again starts on 25 June 2003 when Version 2.8.1b5 goes out. Major parts of FSP server code was rewrited for higher performance and many old bugs in FSP code fixed. Some bugs gets fixed after more than 10 years.

In the past various people worked on the File Service Protocol, see Section 1.9

1.10.2. Today

Radim Kolar started to work on the File Service Protocol again in June 2003. Since that time he's maintaining the source distribution. You can reach him on the mailinglists or via e-mail .

Since September 2003 Sven Hoexter is working on parts of the documentation and code. He is also doing release management and RPM, deb packaging. You can reach him on the mailinglists or via e-mail .