elroid
01/28/2021, 5:29 PMmode = Mode.DOM
is no longer available. According to the docs on Github, it looks like all I should need to do is pass BrowserFetcher
to the skrape function as an argument, but that doesn’t seem to do the trick. I tried setting jsExecution to true, e.g. something like this:
skrape(BrowserFetcher) {
request { url = urlToScrape }
}
extract {
htmlDocument(html = responseBody, baseUri = baseUri, jsExecution = true) {
val t = title { findFirst { text } }
i { "Got title:$t" }
}
}
…which looks like it’s trying to do something, but I’m getting an error loading one of the included js files:
java.io.IOException: Unable to download JavaScript from '<https://cdn>...Stub.js' (status 400).
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1075)
Not sure why it’s a 400, as the file in question seems to download fine with a browser or other http clients. Any assistance is much appreciated…