You're right - I used to get notified (and still do in other workspaces) when someone replies to a followed thread, but the Slack icon has a blue dot on it. But now, on Kotlin Slack only, the icon has a red dot on it. And a red dot means direct mentions. It means someone is waiting for me to answer. I have to drop everything. Which shouldn't be the case when someone replies to a thread that I find interesting to follow but in which I'm not active.