What HTTP lib would you suggest for screenscraping...
# announcements
a
What HTTP lib would you suggest for screenscraping (saving session cookies etc)? I’m looking for something like “requests” for python and scala. Ease of use would be the biggest factor
c
maybe #C0A974TJ9?
m
You could use a java lib like jsoup (https://jsoup.org/) or jaunt (https://jaunt-api.com/)
a
Thanks! Jaunt looks interesting. ktor in combination with jsoup could also work
a
I used JSoup to write an app for accessing DSL modem values. Worked well but it's a pain to parse the html. Probably going to be the same with any lib.
z
Selenium with the chromedriver works pretty well. I've had good luck using Kotlin with it as well.