Screen-Scaper - Capture the Web

Technology

screen-scraper automates:

  • Copying text from a web page.
  • Clicking links.
  • Entering data into forms and submitting them.
  • Iterating through search results pages.
  • Downloading files (PDF, MS Word, images, etc.).

Rock solid

  • Very mature - has been under continuous development since 2002.
  • Can handle just about any website you throw at it, including sites that use AJAX.
  • Scraping processes can run uninterrupted for weeks or months.

Integrates with existing systems

  • Invoke screen-scraper from .NET, Java, Ruby, PHP, or just about any other language.
  • Runs on Windows, Linux, Mac OS X, and any other OS that supports Java.
  • Can call out to external applications using a variety of methods and APIs.