i965/fs: Emit BRW_AOP_INC or BRW_AOP_DEC for imageAtomicAdd of +1 or -1

v2: Refactor selection of atomic opcode to a separate function.
Suggested by Jason.

No changes on any other Intel platforms.

Skylake
total instructions in shared programs: 14304261 -> 14304241 (<.01%)
instructions in affected programs: 1625 -> 1605 (-1.23%)
helped: 4
HURT: 0
helped stats (abs) min: 1 max: 8 x̄: 5.00 x̃: 5
helped stats (rel) min: 1.01% max: 14.29% x̄: 5.86% x̃: 4.07%
95% mean confidence interval for instructions value: -10.66 0.66
95% mean confidence interval for instructions %-change: -15.91% 4.19%
Inconclusive result (value mean confidence interval includes 0).

total cycles in shared programs: 527531226 -> 527531194 (<.01%)
cycles in affected programs: 92204 -> 92172 (-0.03%)
helped: 2
HURT: 0

Haswell and Broadwell had similar results. (Broadwell shown)
total instructions in shared programs: 14615730 -> 14615710 (<.01%)
instructions in affected programs: 1838 -> 1818 (-1.09%)
helped: 4
HURT: 0
helped stats (abs) min: 1 max: 8 x̄: 5.00 x̃: 5
helped stats (rel) min: 0.89% max: 13.04% x̄: 5.37% x̃: 3.78%
95% mean confidence interval for instructions value: -10.66 0.66
95% mean confidence interval for instructions %-change: -14.59% 3.85%
Inconclusive result (value mean confidence interval includes 0).

Signed-off-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

diff --git a/src/intel/compiler/brw_fs_nir.cpp b/src/intel/compiler/brw_fs_nir.cpp
index 68d16966eaedb2d45d7698ce16899b5ba596b8b9..67c0bee7acdfe66e3cdd36e4db65b5c2a2800663 100644 (file)
--- a/src/intel/compiler/brw_fs_nir.cpp
+++ b/src/intel/compiler/brw_fs_nir.cpp
@@ -3899,10 +3899,16 @@ fs_visitor::nir_emit_intrinsic(const fs_builder &bld, nir_intrinsic_instr *instr
                           var->data.image.write_only ? GL_NONE : format);
       } else {
          int op;
+         unsigned num_srcs = info->num_srcs;
 
          switch (instr->intrinsic) {
          case nir_intrinsic_image_deref_atomic_add:
-            op = BRW_AOP_ADD;
+            assert(num_srcs == 4);
+
+            op = get_op_for_atomic_add(instr, 3);
+
+            if (op != BRW_AOP_ADD)
+               num_srcs = 3;
             break;
          case nir_intrinsic_image_deref_atomic_min:
             op = (get_image_base_type(type) == BRW_REGISTER_TYPE_D ?
@@ -3931,10 +3937,10 @@ fs_visitor::nir_emit_intrinsic(const fs_builder &bld, nir_intrinsic_instr *instr
             unreachable("Not reachable.");
          }
 
-         const fs_reg src0 = (info->num_srcs >= 4 ?
+         const fs_reg src0 = (num_srcs >= 4 ?
                               retype(get_nir_src(instr->src[3]), base_type) :
                               fs_reg());
-         const fs_reg src1 = (info->num_srcs >= 5 ?
+         const fs_reg src1 = (num_srcs >= 5 ?
                               retype(get_nir_src(instr->src[4]), base_type) :
                               fs_reg());