|
If you've done screen-scraping in code before you likely know that it often involves dealing with a lot of details such as cookies, HTTP, parsing HTML, and regular expressions.
screen-scraper handles all of the details of HTTP for you. This means you don't need to worry about tracking cookies or setting HTTP headers. Because of this screen-scraper can easily handle logging in to a site, and will maintain the login session while data is extracted.
Once you've mined web pages that contain the data you need extracted you're faced with having to parse out the HTML to download only the individual pieces you want. This often implies complex parsing of text and working with large regular expressions. screen-scraper hides the details of scraping text for you by providing extractor patterns. Extractor patterns are snippets of HTML with special tokens that indicate the data points you want scraped from the page. Behind the scenes screen-scraper generates regular expressions to parse the text such that you only need to worry about working with the extracted data. At the same time, if you have the expertise, screen-scraper also allows you full control over regular epxressions, allowing you to be very precise about collecting data out of a page.
|
