Add a grid acceleration structure that IS fast to rebuild every frame but have same method bvh has for rollback style netcode