Scraping with Selenium and vpn

VPN
1

I am working on this project where i need to access truepeople.com but for some i get the message that I am blocked to access the website. Nonetheless, the only way to access it is using a vpn connected to usa , how can i do that with Selenium?
Im using edge because i face alot of issues with chrome and i find it rather slow

2

I have a project that does this. I just open sourced:

Basically using Tor Network, Selenium, and Tor Driver (Firefox and geckodriver)

3

If you run Selenium on your own computer, you could simply install something like ProtonVPN and start it, and all your browser traffic goes through the VPN. If Selenium runs on some remote server, then you probably need a VPN that can be controlled via CLI commands, which is a pain in the ass

4

I have had some success scraping this site using selenium and a proxy service called webshare. But they are not cheap. Selenium has an option to add a proxy, using that and we shared rotating proxy endpoint I would scrape until the ip got blocked. Once it did the browser would close and reopen until the rotation occurred then continue scraping with the new ip

5

I am running selenium on my computer and i did try a vpn turbovpn and 1.1.1.1 but for some reason my websites weren’t loading when i turned them on so i installed Zen mate vpn extension on edge and i was able to access the website

6

May i ask how many proxy addresses would i need for rotating or how long does a proxy address can be useful

7

One more thing to account for, even if you don’t get blocked you can reach a total amount of searches for that day, so rotating proxy would help there too

8

I think you’ll need to replace the socks proxy configuration with the vpn information to control coming out in USA specifically. Otherwise, it’ll be random Tor exit nodes.

9

I never tried those VPNs but I am pretty happy with the free version of ProtonVPN, it also has free servers in America. But if Zen Mate works for you, I would stay with it.

10

the issue with zenmate is that i can only use its extension, cant download for all browsing activities. It’ll try the protonvpn tho