We got the same problem, when we play with up to 4 people. If just one of the guys have lag, everybody lags. I understand that this is not the exact same issue you discuss here, but it sucks. In my opinion the biggest minus point when it comes to multiplayer experience, sadly.