Version 2.7.2 of screen-scraper available

This is just a minor bug fix release, but anyone invoking screen-scraper from the command line should upgrade. Somehow a semi-critical bug slipped through our radar on the 2.7 release. In 2.7 if you have the workbench open, then run screen-scraper from the command line, when the command line instance ends it will close screen-scraper’s … Read moreVersion 2.7.2 of screen-scraper available

Three common methods for data extraction

Building off of my earlier posting on data discovery vs. data extraction, in the data extraction phase of the web scraping process you’ve already arrived at the page containing the data you’re interested in, and you now need to pull it out of the HTML. Probably the most common technique used traditionally to do this … Read moreThree common methods for data extraction

New screen-scraper tutorial available

We’ve just released a new screen-scraper tutorial: It’s just received the blessing from our project manager and aspiring professional writer/editor, Jason Bellows, so it should be ready for public consumption. Here’s a snippet from the tutorial introduction: “It’s often the case in screen-scraping that you want to submit a form multiple times using different … Read moreNew screen-scraper tutorial available

Untrusted Server Certificate Chain fix

Some of you in the past may have run into this dreaded message when trying to access a site that uses HTTPS: Untrusted Server Certificate Chain I’m happy to report that we’ve just issued a fix for that in version See this FAQ if you run into any trouble upgrading.

Data discovery vs. data extraction

Looking at screen-scraping at a simplified level, there are two primary stages involved: data discovery and data extraction. Data discovery deals with navigating a web site to arrive at the pages containing the data you want, and data extraction deals with actually pulling that data off of those pages. Generally when people think of screen-scraping … Read moreData discovery vs. data extraction