A modern programming language that makes developers happier.

kotlinlang

FWIW, after some investigation, the issue seems to lie in the <https://github.com/skrapeit/skrape.it/blob/master/core/src/main/kotlin/it/skrape/core/Parser.kt|Parser object>. Specifically - the `toUriScheme()` method was causing a massive URL, that later when added as a `Referer` header had around 44kb of content - which was too large for the server to accept, hence the 400. Truncating it to 200 bytes meant there were no more errors, but unfortunately the generated dom content was incomplete, so at this point I don’t see a way of using <http://Skrape.it|Skrape.it> for JS-based sites such as <https://stores.sainsburys.co.uk/4946/bristol-hippodrome-local|this one>. If any fixes are made to make parsing this page viable with <http://Skrape.it|Skrape.it> I will definitely try again!