Web scraping is the use of automated software to interact with a web page. The term “web scraping” is often used interchangeably with “screen scraping” although the latter can also refer to scraping from a terminal screen–like if you need to save data from an old 3270 mainframe.

Mainframe scraping is older than web scraping, but they accomplish similar things for different systems. In the golden age of mainframes, the computer you sat at was a “terminal” that didn’t have much computing power at all. As PCs gained power and prevalence, it became desirable to save data from the mainframe to your computer to analyse it or to make a visually pleasing representation of it. Mainframes are renowned for their lack of visual pleasantness.

Such systems are still in use. In the rare case when someone reaches out to us seeking a dose of this elder brother to web scraping, we will refer them to Jagacy.

Web scraping refers to a routine that accesses a web page (or any web resource) through HTTP. Like the other type of scraping, it is used to get information and make it more accessible–in this case it’s any one of the millions of computers that make up the internet. Web scraping can be run from your personal computer, or if you have a huge job, and more horsepower is needed, it can be run across high performance server farms running together to save huge swaths of information.