glsl: Avoid making a temporary for lower_mat_op_to_vec if not needed.