[Crimsonite]

It might be that you have internet connectivity problems and when you go near other people, your internet can't load the other players up that fast, which causes lag...