egl/wayland: Don't invalidate drawable on swap buffers