screen-scraper can be invoked from software applications written in most modern programming languages, including Java, Active Server Pages, PHP, .NET, and anything that supports SOAP. In this tutorial we'll give some examples of applications that do just that.

Our application will pass parameters to screen-scraper corresponding to login information as well as a key phrase for which to search. As in the third tutorial, we're going to pretend that the web site requires us to log in before we can search, for the sake of providing an example, even though it actually doesn't. Once we pass the parameters to screen-scraper we'll tell it to start scraping. screen-scraper will then run the scraping session using the parameters we gave it, extracting out the data it normally does. Once it's done, we'll ask it for the extracted information, then output it for the user to see.

Before we begin we'll first need to make a couple of minor changes to the e-commerce scraping session from the third tutorial. If you haven't already, start up screen-scraper. Under the "Shopping Site" scraping session click on the "Login" scrapeable file, then on the "Parameters" tab. We're going to alter the "email_address" and "password" POST parameters so that we can pass those parameters in rather than hard-coding them. For the "email_address" parameter change the value "test@test.com" to ~#EMAIL_ADDRESS#~, and change the "testing" value for the "password" parameter to ~#PASSWORD#~. You might remember from Tutorial 3 that tokens surrounded by the ~# #~ delimiters indicate that the value of a session variable should be inserted. For example, in our case we're going to create an "EMAIL_ADDRESS" session variable and give it the value "test@test.com" such that screen-scraper substitutes it in for the corresponding POST parameter at runtime.

The code that we'll be writing in our external application will also be essentially taking the place of the current "Shopping Site--initialize session" script. Let's disable that since it would otherwise overwrite the values we'll be passing in externally. To do that click on the "Shopping Site" scraping session in the tree on the left, then on the "Scripts" tab. In the scripts table, un-check the "Enabled?" check box for the "Shopping Site--initialize session" script. Save your changes and exit screen-scraper.

Where you go next depends on which programming language you're interested in. Use one of the links below according to your preference.

Tutorial Overview

Invoking screen-scraper from ASP Invoking screen-scraper from C#.NET Invoking screen-scraper from Cold Fusion Invoking screen-scraper from Java Invoking screen-scraper from PHP Invoking screen-scraper from VB.NET