Testbed for Web Scraping

This is a testbed for web crawling using firebase functions. It only works for authenticated users.

Test Web Crawler

Note that when the metadata of your returned request says installed_browser: true then the request was much longer than it will be in production. Make the same request a second time immediately after and you will see the speed we can achieve in production where installed_browser: false

The url will be scraped by a browser running on a Google Cloud Function, and that browser's request is routed through a global IP proxy network so that you can scrape many websites without getting blocked by the website.

Scrape a WG-Gesucht WG Zimmer Listing

If you just want to try it out, you could find a link to a current listing in Berlin on this link

Please make sure you select a WG zimmer and not for example a 1-zimmer-wohnung, as those can't be scraped yet

Metadata:

No form submitted yet

Scraped Website Content:

No form submitted yet

Generate an email to the WG: