try it for yourself - my team did, initially I sent the guy doing the tests away for screwing up the sums because I didn't believe it could be as good as he said. I was wrong - it was. This was a relatively early CNN as well. AGNN and RNN are revolutions in themselves and everyone is now scrambling to understand and harness them.
Hand someone ideally a 3-5 year old an average digital camera and have them walk around taking random pictures where half of them are a blurry mess and the results are much worse. Or drive around with a camera on you bumper etc. Public Datasets have style of photo that represent it's own bias.
Our data set was from surveys collected for human use using a variety of digital cameras - not DSLRs though. The surveyors did take specific images, blurry images would have meant not getting paid so none of those. We used transfer learning from the imagenet network vs a few hundred of the images well labelled and we trained it on a 6 gpu machine (titan x's) This is no place to claim results or do the detailed write up so all I would say is that do try for yourself if you haven't already, I was shocked by the quality of the classifier. Maybe you will be too!
I started from the dataset bias position myself - billions of family snaps and selfies can't provide the reference for arbitrary images of stuff from arbitrary angles. My reading of the results that we got is that the claims of lower level features extracted by the cnn are correct, and that a network trained on massive public data can be the basis for specialised tools trained on more constrained proprietary datasets.
> Hand someone ideally a 3-5 year old an average digital camera and have them walk around taking random pictures where half of them are a blurry mess and the results are much worse.
Garbage in, garbage out. When was it any different?
The question is: can a human do much better on that garbage?
> Public Datasets have style of photo that represent it's own bias.
The problem is self driving cars need to work with Garbage when their sensor is covered by slush from a puddle, scratches on the glass etc etc. The problem is we are not training them on Garbage which is it's own problem.
Scoring is also critical, you want to gracefully degrade classification systems not getting the correct dog breed is insignificant vs calling a dog a sofa. Further, people base real world classification on stereoscopic video footage. Training robots to classify photos is a handicap for building robots to operate in the real world.
Our GPU kit has got 6 times better in 2 years.
Exciting times.