Helllo, i started experimenting with coroutines an...
# coroutinesb
Bruno Ortiz
12/08/2017, 6:21 PMHelllo, i started experimenting with coroutines and wrote a simple Crawler(https://github.com/bruno-ortiz/skraper). It's not complete yet, but i already have some questions.
The idea is really simple. I use a channel and start N coroutines to listen from this channel. This coroutines downloads the HTML's and calls the method 'parse' from the Crawler object, the result is either a item, or a object indicating that a new page must be sent to the channel for processing.
What i realized is that if i start 20 coroutines with a pool of 10 threads, the performance is inferior then if i had started with a pool of 20 threads. I am sure that i'm doing something wrong, but the only way to overcome this was creating a separated pool. One for reading from the channel and parsing the pages, and another for downloading the pages, this is because when i call Jsoup to download the pages, from what i understand is blocking the threads. Is there any way that i can do this without blocking the threads?