People not responding in time doesn't have to do with how they read their mail but only with how they prioritize their life. You consider mail important ,they probably not as much. I follow the same "full inbox" approach and answer quite fast to mails, in part because i do have email notifications everwhere. I do answer mails with a certain importance at once and keep others on unread or starred.

I know people that archive all mails and do respond slow in general and people that respond fast.