Scraping not working or returning no results

Scraping not working or returning no results

If you are scraping a engine, especially Google and you are not getting any results, its generally due to one of a couple reasons.

1.) The terms you are scraping simply do not have any results to return.  For example if you manually go to Google.com and search for:
inurl:car inurl:house inurl:cat site:purple.com purple cows

Google is then going to return this:
Your search - inurl:car inurl:house inurl:cat site:purple.com purple cows - did not match any documents.

So if you put that same string into scrapebox its not going to harvest any results either.

2.) Your proxies are blocked or some other error is happening.  The easiest way to check this is go to the settings menu.  Uncheck "use multi threaded harvester".   Then try to harvest.  Scrapebox will display each query, the proxy used and the result, including any error messages.

If you see lines that say:
Results 0 completed using proxy xxx.xxx.xxx.xxx:xxx   - then it means that the request finished, but the engine returned no results, same as it would do if you actually went to the engine and manually searched for it.

If you see:
Results 0 Error xxx received using proxy xxx.xxx.xxx.xxx:xxx - Then you can look at the error message and generally determine the problem.  Most common for this one are:

Error 302 - Your IP is blocked
Error 404 - the proxy is bad or was never found
Error 407 - your proxy need authentication, if your using private proxies you need to get with your provider to see if you need to use IP authentication and how to do it, or if you need a username and password

This entry was posted in Uncategorized. Bookmark the permalink.