Yeah, to me it’s just interesting that things that I process without any conscious effort (language, vision) are best solved by the black box of a neural network. It’ll be interesting to see how the utilization of NNs continues to evolve since we have only had practically useful neural networks for a few years. It’s been interesting to see NNs applied to video game AI as well.

I'm a little bit more cynical about neural networks. They are not new and been around for decades. The availability of data allowed them to hit some low-hanging fruit but still have diminishing returns in other domains. As I've said before, I'm seeing a growing number of articles warning about another AI Winter this year: https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-on-its-way/

I have no idea if we’ve hit a hard wall, I’m not really on the data science side of things (unfortunately). I think it’s interesting that by default NNs answer the question of “is this a picture of a dog?” the same way a human might-- ie. with a high value if it’s very much a dog, or a lower value if it’s “dog-like” or a very low value if it doesn’t seem like a dog at all. And while Bayesian classifier and other such things do something similar, I usually think of a lot of algorithms as providing discrete responses, which NNs generally do not provide.

@dalexander surprisingly, categorization (discrete) and regression (continuous) are pretty tied together in a lot of machine learning models. Neural networks and Naive Bayes produce probabilities for the likelihood of each category, and choose the highest one. This translates a probability into a discrete decision.

I think that’s because a lot of other systems want to act on discrete outcomes. Even a lot of humans wouldn’t want to interpret a continuous value when they could get a yes/no answer instead. And there’s not often a meaningful action that could be taken with something like “this is probably spam” compared to “spam / not spam”. Maybe put it in the user’s inbox and flag it, but it’s harder to come up with actions, and most of the value is already captured in the yes/no output.