Basic Web Scraper Built With Power Automate for Desktop

A functional web scraper with complete flow actions. Showing how to scrape websites, traverse links and download content.
How It Works

This is a non-interactive web scraper, meaning that it does not use browser automation (Chrome, Edge, Firefox) for scraping. Instead, all web page requests are sent with the Download from web flow action.

Instructions
  • Line 1: Insert the url to be scraped.
  • Line 2: Set a folder where files should be downloaded to.
  • Line 3: (Optional) Change file extensions that should be downloaded.
  • Line 4: (Optional) Change the custom user agent header that is sent to websites.
  • Line 5: (Optional) Change MaxPageRequests (default is 30)
  • Line 6: (Optional) Change MaxFileRequests (default is 50)
Limits

The scaper has following limits:

  • Scraping only static html content (so no JavaScript content).
  • The target website must use absolute urls (could be improved to understand relative urls).
Notes
  • This scraper does not obey robots.txt. If you plan to use this on a website you don’t have ownership to, then you should comply with their robots.txt. Do not scrape Disallowed Pages and obey Crawl-Delay.
  • Do not perform malicious activities.
  • Wait 1 flow action have been added after each webpage request (the flow will wait for one second after each request). Many websites do run on shared hosts with limited resources and they deserve our understanding. This is the minimum wait value that should be used.
PAG Admin
Flow Variables
Flow Datatypes
Also See These Flows
Flow Action
Create Animated Doughnut in Notepad

This example shows how to read JSON data, loop items and print output to screen (Notepad in this case).

PAG Admin

338

Flow Action
Run Flow From Command Prompt or Windows Task Scheduler

A simple way to run flows from Command Line, Windows Task Scheduler, shortcut keys or by using a URL.

PAG Admin

8,477

Flow Action
Write to Google Sheets API Using OAuth 2.0 Power Automate for Desktop

Write cell values to spreadsheets. Includes basic logic for handling refresh tokens for repetitive tasks.

PAG Admin

2,748

Flow Action
Read Google Sheets API With Power Automate for Desktop

This example uses Invoke Web Service flow action to send a request to Google Sheets API, the response contains spreadsheet values.

PAG Admin

1,512

Flow Action
Open or Close an Application When Bluetooth Device Is Connected/Disconnected

This example opens and closes Spotify automatically when Bose NC 700 HP headphones are turned on/off. Discover how you can use Bluetooth as a trigger with Power Automate for Desktop. You can use any Bluetooth device and application.

PAG Admin

647