gitlab-ci: Only pull/push cache contents in build+test stage jobs