Thoughts, tips, and updates on screen-scraping
Fixed a number of bugs in this one: Made a bug fix that arose when available anonymous proxy servers was depleted to zero. Now disallowing running multiple screen-scraper interfaces simultaneously. For example, previously the screen-scraper workbench could be run concurrently with the server. This ended up causing database corruption in some cases, though, so we’re … Read moreVersion 5.0.26a of screen-scraper Released
This one’s just a minor update with a couple of bug fixes: With the update of JRE 1.6.0_22 on Mac OS X the “sss” file extension was being trunctaed when exporting. In some cases anonymous proxies would spawn, but never become available. Added code to handle this situation. Anyone using anonymization with an alpha version … Read moreVersion 5.0.25a of screen-scraper Released
We’ve actually had a 30-day money-back guarantee since almost the beginning, but recently decided to highlight it a bit more. Any time in a software development project you incorporate a new library or application you’re incurring a certain amount of risk. The new application may appear to do just what you want it to, but … Read moreMoney-Back Guarantee for screen-scraper
Here we go for the second installment. The topic for today is online video. Online video You may be familiar with certain sites that allow you to view your favorite TV episodes or watch a poor squirrel being launched into the woods off of some guys deck via a salad strainer and 20 feet of … Read moreOh, the possibilities (screen-scraping online video)
Due to some recent pricing changes Amazon has effected in their EC2 service I’m happy to report that we’re likewise able to reduce the price of our anonymization service. Whereas previously the cost per hour per proxy was 25 cents, it will now be 10. The one hitch to this is that you’ll need to … Read morePrice Drop in Anonymization Service
This is the first installment in what will hopefully become a series. Here at screen-scraper we handle a variety of projects for a myriad of different clients. All of our work is centered around our core software, screen-scraper, but is often complimented by third-party software such as PHP, Tomcat, Lucene, Google Web Toolkit, mySQL, along … Read moreOh, the possibilities (ScrapbookFinds.com)
As Scott pointed out, we were featured in a Wall Street Journal article yesterday. I thought it might be worthwhile to share my point of view on what information it presents. On the whole, I think the article largely misrepresents the type of work we do. The tone of the article seems to be fairly … Read moreMy Take on the Wall Street Journal Article
On September 5th, 2010 we received a call from a reporter from the Wall Street Journal, Steve Stecklow (2007 Pulitzer Prize winner). He was calling to speak with someone at our company for a story he was doing related to our industry. He and I talked for about 40 minutes where he asked a lot … Read morescreen-scraper.com in the Wall Street Journal
I get a lot of requests for help to configure and run screen-scraper to scrape at an optimal rate. As is often the case with optimization, it is often as much art as science since the many variables that can affect the speed of a scrape are impossible to catalog. While these steps will help … Read moreScaling & Optimizing screen-scraper
Here I go tooting our horn again–Oracle has just posted a podcast on integrating screen-scraper with their Oracle Secure Enterprise Search product. The page that links to the podcast is here, and the MP3 file of the same can be found here. Might be worth a listen if only to get a better sense for … Read moreOracle/screen-scraper Podcast