Comments on: When Less is More

By: molmod

molmod — Wed, 15 Jul 2015 13:28:31 +0000

That’s definitely the case : an interpretable model can be much more useful. Linear regression models certainly win in that sense over NNs. However, sometimes the descriptors that go into such a model have an obscure relationship with molecular structure and give no real insight into how to optimise the activity of interest. I don’t think I’d dare ask a chemist to reduce his Balaban index for example.

I love the images generated by the Google NNs in your link. Particularly the one that sees animals in the pictures of clouds.

By: Amethyst

Amethyst — Wed, 08 Jul 2015 16:56:40 +0000

Hey Molmod, I like your "less is more" neural net thoughts. Do you think that the interpretability of a model might also be a factor in the choice of method used? I have found that some chemistry clients favour models that give them a meaningful reason to why a compound is being predicted as active. I have just stumbled across this interesting blog article published last month about neural networks in image recognition. Has some really eye catching images of what you get when you take a peek inside a neural network. For example take a look at the magical animals that are found in the sky.

By: molmod

molmod — Sun, 05 Jul 2015 13:35:37 +0000

Hi there Amethyst. I remember this well when I was playing around with neural nets a lot. Add too many neurons and it was great at telling you what you already knew but the predictions were hopeless. The other interesting thing was that you could train them too much. If you curtailed the training, the predictions were usually better than if if let the training run on longer, whereas the error in the test set just got smaller and smaller. Definitely less is more there…
Neural nets seem to have fallen out of fashion these days (at least in chemistry applications) although there have been one or two more recent papers that may have got around some of these issues. I wait to see if they rise again in popularity to the levels we saw in the ’90s.