Can I read Design and Implementation of A High-Performance Distributed Web Crawler on EtoBox?

Design and Implementation of A High-Performance Distributed Web Crawler by Amritpal Singh is a document available to read on EtoBox.

What is Design and Implementation of A High-Performance Distributed Web Crawler about?

The document describes the design and implementation of a high-performance distributed web crawler. It discusses the challenges of building a system that can crawl hundreds of millions of web pages over several weeks at high speeds while being robust, manageable, and considerate of resources. The proposed crawler architecture scales to crawl several hundred pages per second using a network of workstations.

Author

Amritpal Singh

Language

EN