Reformulating the topic not to violate the rules of the forum:
How to get html source code on sites that require Javascript without using Selenium WebDriver? Is there any other way?
Skipmode A1 Wrote:You can run javascript with nodejs: see here https://github.com/Anorov/cloudflare-scrape
This however requires nodejs present on the computer you running your kodi-add on.
I use Windows, and I installed cfscrape in python 2.7:
python -m pip install -U pip cfscrape
And installed Node.JS
https://nodejs.org/dist/v7.9.0/node-v7.9.0-x86.msi
I’m trying to get the source code of the site where javascript is required, eg sites hosted on Cloudflare
first test in Python IDLE (Windows):
import cfscrape
scraper = cfscrape.create_scraper()
print scraper.get('http://somesite.com.Cloudflare-anti-bot').content
But it still did not open Javascript. Could you give me an example of how to use cfscrape with nodejs?
Thanks