Webbots, Spiders, and Screen Scrapers, 2nd Edition

First of all, I thought this was a great book; once I picked it up, I hardly put it down. The examples, projects and ideas for extending those projects have really inspired me and I now find myself thinking of wacky ways to use the web to its true potential.

The projects range from downloading all of the images on a website to creating snipers to purchase auction items when the time is right and spiders which crawl large portions of the web searching for specific things that interest you. The book even focuses on targeting modern websites which heavily use JavaScript and render traditional webbots useless.

The author provides an excellent companion website which provides a controlled environment where you can test your webbots without getting into trouble out in the real world. The website also includes all of the general good stuff, such as source code and libraries.

At first one of my concerns was that the libraries, which hide low level functions so that the book can actually focus on learning how to code webbots, spiders and screen scrapers, would hide the gory details and actually decrease general understanding. But the content of the libraries is explained well and make the scripts more simple and robust rather than providing a magical black box.

For those who are interested, the book uses PHP, or more specifically PHP/CURL for its examples, although I believe that all of the techniques could be implemented in the language of your choice, as long as it has a curl implementation or another strong networking library. The database choice is MySQL, although again, the code is written in such a way that the data layer can easily be changed.

The book also focuses on real world details, such as keeping yourself out of trouble, getting the most out of your deployment environment and what really adds value to a project. The author gives a few war stories about his experiences and gives you the chance to learn from his mistakes.

On top of this, the author writes in an easy to follow and clear style and has a sense of humour; I didn't fall asleep with the book on my face once. But even if you disagree with me on that, the projects still open up a lot of opportunities and the author obviously has experience.

Bottom line, I would definitely recommend this to a friend.


Matthew Bates


External Links

Compsoc Wiki
Compsoc Library

Upcoming Events

There are no upcoming events planned. Check the CompSoc Wiki in case of emergency

RSS | iCal


No Starch Press
Durham Students Union

Random Poll

How would you describe this poll?

View results
Submit a new poll
All polls

This section exists to trap prefetching clients. Please just ignore it if you have css disabled and thus can see this. Do not click this link unless you want us to think you are a bot