spider.pl is a program for fetching documnts from a web server, and outputs the documents to STDOUT in a special format designed to be read by Swish-e. The spider can index non-text documents such as ...