From 8ec5b16a9a3dbd6d825596c22f1bc32646de28fe Mon Sep 17 00:00:00 2001 From: Richard Sandiford Date: Wed, 6 Nov 2019 12:29:47 +0000 Subject: [PATCH] Check the VF is small enough for an epilogue loop The number of iterations of an epilogue loop is always smaller than the VF of the main loop. vect_analyze_loop_costing was taking this into account when deciding whether the loop is cheap enough to vectorise, but that has no effect with the unlimited cost model. We need to use a separate check for correctness as well. This can happen if the sizes returned by autovectorize_vector_sizes happen to be out of order, e.g. because the target prefers smaller vectors. It can also happen with later patches if two vectorisation attempts happen to end up with the same VF. 2019-11-06 Richard Sandiford gcc/ * tree-vect-loop.c (vect_analyze_loop_2): When vectorizing an epilogue loop, make sure that the VF is small enough or that the epilogue loop can be fully-masked. From-SVN: r277880 --- gcc/ChangeLog | 6 ++++++ gcc/tree-vect-loop.c | 10 ++++++++++ 2 files changed, 16 insertions(+) diff --git a/gcc/ChangeLog b/gcc/ChangeLog index a8a8968c9bc..d04b6bea18b 100644 --- a/gcc/ChangeLog +++ b/gcc/ChangeLog @@ -1,3 +1,9 @@ +2019-11-06 Richard Sandiford + + * tree-vect-loop.c (vect_analyze_loop_2): When vectorizing an + epilogue loop, make sure that the VF is small enough or that + the epilogue loop can be fully-masked. + 2019-11-06 Richard Sandiford * tree-vect-loop.c (vect_analyze_loop): Break out of the main diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index f8049047a8e..ec0bd2c3875 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -2143,6 +2143,16 @@ start_over: " support peeling for gaps.\n"); } + /* If we're vectorizing an epilogue loop, we either need a fully-masked + loop or a loop that has a lower VF than the main loop. */ + if (LOOP_VINFO_EPILOGUE_P (loop_vinfo) + && !LOOP_VINFO_FULLY_MASKED_P (loop_vinfo) + && maybe_ge (LOOP_VINFO_VECT_FACTOR (loop_vinfo), + LOOP_VINFO_VECT_FACTOR (orig_loop_vinfo))) + return opt_result::failure_at (vect_location, + "Vectorization factor too high for" + " epilogue loop.\n"); + /* Check the costings of the loop make vectorizing worthwhile. */ res = vect_analyze_loop_costing (loop_vinfo); if (res < 0) -- 2.30.2