Complex Forms

There are some sites that have some pretty complex forms–sometimes in the sheer number of parameters, or sometimes by being incomprehensible to humans. In such cases we have a method to get all the form elements for you.

On the page with the forum, you need to extract the whole form, including the “<form” and “</form>”. I will make an extractor with the token named ~@_FORM@~, and use the RegEx in the token properties to define which form I need. An example RegEx:

<form id="firstForm".*?</form>

Once I get it extracted, there is a script to run on each pattern application. Therein you need to set any fields, selection, radio buttons, etc, and set as a session variable.

import com.screenscraper.util.form.*;

Form form = scrapeableFile.buildForm(dataRecord.get("_FORM"));
form.setValue("SESSION_TOKEN", session.getv("SESSION_TOKEN")); // Set a field. Add any number needed
form.setValueChecked("values", session.getv("TO_CHECK")); // Set a checkbox as selected. Add any number needed
session.setv("_FORM", form);

Then you request the next scrapeableFile, and on that file you run a script before the file is scraped, and it will clear any current URL and parameters, and replace them with those from the _FORM. I rarely change this script.

import com.screenscraper.util.form.*;

Form form = session.getv("_FORM");

Leave a Comment