panfrost: Move BO cache related fields to a sub-struct