Hey guys. We built ParseHub so you can get data from super complicated and interactive websites. As I like to say - websites partying like its 1999. You can give it a try here.
Here are a few things ParseHub can do:
- Get data from infinitely scrolling pages
- Get data from behind an log-in. All you have to do is input your email and password
- Automatically fill our forms and send them
- Download images and files to DropBox
- Handle pagination - even AJAX pagination and crawl through millions of pages
- Get HTML, attributes and clean up data with RegEx
- Make the data structured the way you want to with for loops and conditionals to filter our results and text
- Click through a bunch of nested drop downs and get data that loads dynamically
- Enter thousands of search queries into a search box to get results
- Jump from one website to another so you can do things like scraping wages and then converting the currency on another website
- Open tabs, pop-ups and hidden elements on hover
- Get data from maps
- Input thousands of links for ParseHub to crawl through
If you have a super shitty site - like a government site for example send it my way and I will make sure we can get data from it.
You can give it a try here.