This is a project I worked quite a bit on (mostly because of the sh*tty GUI). This webcrawler can only search for hyperlinks on a webpage by looking for 'href', 'src' and 'action'. It might actually ...

This repo will contain drafts and updates towards making a modern, efficient webcrawler and text processor in Python. TextProcessorRoughDraft.py contains a class and is an overall program that will ...

WebCrawler, probably the most bought and sold search engine on the web – and one of the first, celebrates its 10th birthday today. WebCrawler was developed by Brian Pinkerton in his spare time while a ...