glsl: stop adding pointers from bindless structs to the cache