Without knowing much about ML, it seems that using two (or more) very different methods could be a reasonable defense; if the methods are sufficiently different then it will get exponentially harder to find a gradient that fools all the methods; what to do when the outputs strongly disagree is a good question, but switching to a failsafe mode seems better than what we have now.
I was thinking use the same training set but different ML techniques; It's been almost 20 years since I took an AI class, but RNNs weren't the only thing in there...
