Bufferbloat.net talks about this. The problem is that many home routers do a poor job allocating their resources so if I'm streaming video and you're VOIPing, your latency will go up dramatically since my download is pushing you out. In reality there's enough bandwidth for both of us, but it's getting used poorly. On my setup, loaded latency was over one second (to google.com) compared to under 50ms for unloaded! (Fast.com gives more mild numbers for loaded latency than I observed manually.)
Latency is a measure of how long it takes a packet to go from your computer to some other computer and back. People usually measure this by a ping test
Unloaded latency (when there isn't much traffic on your connection through your link to the ISP) generally ranges from 5 msec to 50 msec, depending on what kind of connection you have (fiber, cable, dsl, cellular).
Loaded latency is measured when there is traffic on the link. A good router will minimize the additional latency, but many routers will add a lot - between two and five seconds (2000 to 5000 msec). That's why you lag out in games.
