ac/nir: use llvm.amdgcn.rsq for nir_op_frsq

Instead of emitting 1.0 / sqrt(x) which includes a slow division that
LLVM doesn't always optimize even if the metadata is correctly set.

pipeline-db (VEGA10/LLVM 9):
Totals from affected shaders:
SGPRS: 16872 -> 16864 (-0.05 %)
VGPRS: 15320 -> 15464 (0.94 %)
Spilled SGPRs: 2021 -> 2133 (5.54 %)
Code Size: 1915464 -> 1917476 (0.11 %) bytes
Max Waves: 641 -> 639 (-0.31 %)

pipeline-db (VEGA10/LLVM 10):
Totals from affected shaders:
SGPRS: 43936 -> 44120 (0.42 %)
VGPRS: 41776 -> 41972 (0.47 %)
Spilled SGPRs: 875 -> 875 (0.00 %)
Code Size: 4468164 -> 4468120 (-0.00 %) bytes
Max Waves: 2412 -> 2414 (0.08 %)

pipeline-db (VEGA10/LLVM 11 - 92744f62478):
Totals from affected shaders:
SGPRS: 60096 -> 60096 (0.00 %)
VGPRS: 63552 -> 63648 (0.15 %)
Spilled SGPRs: 6135 -> 6117 (-0.29 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Code Size: 6252996 -> 6249772 (-0.05 %) bytes
Max Waves: 2324 -> 2337 (0.56 %)

LLVM 11 (master) is more affected than previous versions, but
based on the small impact with LLVM 9/10, I decided to emit it
unconditionally.

Cc: 20.0 <mesa-stable@lists.freedesktop.org>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4326>

diff --git a/src/amd/llvm/ac_nir_to_llvm.c b/src/amd/llvm/ac_nir_to_llvm.c
index 1e7e95116935a3090f4c8ccf3670ec83e0b23d47..d1c333ac73d252004c6a54a003cf9ce3394f77e5 100644 (file)
--- a/src/amd/llvm/ac_nir_to_llvm.c
+++ b/src/amd/llvm/ac_nir_to_llvm.c
@@ -834,9 +834,8 @@ static void visit_alu(struct ac_nir_context *ctx, const nir_alu_instr *instr)
                                              ac_to_float_type(&ctx->ac, def_type), src[0]);
                break;
        case nir_op_frsq:
                                              ac_to_float_type(&ctx->ac, def_type), src[0]);
                break;
        case nir_op_frsq:

-               result = emit_intrin_1f_param(&ctx->ac, "llvm.sqrt",
-                                             ac_to_float_type(&ctx->ac, def_type), src[0]);
-               result = ac_build_fdiv(&ctx->ac, LLVMConstReal(LLVMTypeOf(result), 1.0), result);
+               result = emit_intrin_1f_param(&ctx->ac, "llvm.amdgcn.rsq",
+                                             ac_to_float_type(&ctx->ac, def_type), src[0]);

                break;
        case nir_op_frexp_exp:
                src[0] = ac_to_float(&ctx->ac, src[0]);
                break;
        case nir_op_frexp_exp:
                src[0] = ac_to_float(&ctx->ac, src[0]);