I also tried splitting ubfe instructions with one or zero constants,
and zero shaders in shader-db were affected.
The "lost" shader is a compute shader that was promoted from SIMD8 to
SIMD16, so is also counted as the gained shader.
v2: Further restrict bfe splitting. bfe with multiple constants is
better on at least some Radeon GPUs. Use -x instead of 32-x in shift
counts.
v3: Fix the outer shift count for ibfe lowering. Add c=0 optimizations
to prevent bad lowering. Both suggested by Rhys. Add shift by -32
optimizations.
Tiger Lake
total instructions in shared programs:
17608764 ->
17596316 (-0.07%)
instructions in affected programs: 303765 -> 291317 (-4.10%)
helped: 113
HURT: 46
helped stats (abs) min: 1 max: 458 x̄: 120.67 x̃: 21
helped stats (rel) min: 0.09% max: 11.23% x̄: 3.47% x̃: 1.39%
HURT stats (abs) min: 1 max: 201 x̄: 25.83 x̃: 6
HURT stats (rel) min: 0.23% max: 5.18% x̄: 1.53% x̃: 1.11%
95% mean confidence interval for instructions value: -101.13 -55.45
95% mean confidence interval for instructions %-change: -2.61% -1.44%
Instructions are helped.
total cycles in shared programs:
338390770 ->
333530868 (-1.44%)
cycles in affected programs:
79438330 ->
74578428 (-6.12%)
helped: 112
HURT: 64
helped stats (abs) min: 2 max: 268955 x̄: 44261.93 x̃: 1452
helped stats (rel) min: <.01% max: 29.51% x̄: 4.72% x̃: 2.23%
HURT stats (abs) min: 2 max: 17618 x̄: 1522.41 x̃: 84
HURT stats (rel) min: <.01% max: 7.34% x̄: 1.35% x̃: 0.34%
95% mean confidence interval for cycles value: -37232.47 -17993.69
95% mean confidence interval for cycles %-change: -3.37% -1.65%
Cycles are helped.
total spills in shared programs: 8944 -> 8138 (-9.01%)
spills in affected programs: 3240 -> 2434 (-24.88%)
helped: 67
HURT: 0
total fills in shared programs: 9373 -> 7842 (-16.33%)
fills in affected programs: 4736 -> 3205 (-32.33%)
helped: 67
HURT: 0
LOST: 1
GAINED: 2
Ice Lake and Skylake had similar results. (Ice Lake shown)
total instructions in shared programs:
16123288 ->
16116876 (-0.04%)
instructions in affected programs: 241155 -> 234743 (-2.66%)
helped: 126
HURT: 2
helped stats (abs) min: 1 max: 209 x̄: 50.90 x̃: 7
helped stats (rel) min: 0.07% max: 5.94% x̄: 1.76% x̃: 0.65%
HURT stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel) min: 0.05% max: 0.24% x̄: 0.15% x̃: 0.15%
95% mean confidence interval for instructions value: -61.29 -38.89
95% mean confidence interval for instructions %-change: -2.05% -1.42%
Instructions are helped.
total cycles in shared programs:
335419163 ->
330438819 (-1.48%)
cycles in affected programs:
77515502 ->
72535158 (-6.42%)
helped: 139
HURT: 37
helped stats (abs) min: 2 max: 269140 x̄: 36374.19 x̃: 597
helped stats (rel) min: <.01% max: 28.60% x̄: 3.67% x̃: 1.31%
HURT stats (abs) min: 4 max: 17618 x̄: 2045.08 x̃: 174
HURT stats (rel) min: 0.02% max: 8.32% x̄: 2.61% x̃: 0.62%
95% mean confidence interval for cycles value: -37799.30 -18795.51
95% mean confidence interval for cycles %-change: -3.13% -1.57%
Cycles are helped.
total spills in shared programs: 8065 -> 7306 (-9.41%)
spills in affected programs: 3153 -> 2394 (-24.07%)
helped: 67
HURT: 0
total fills in shared programs: 8710 -> 7412 (-14.90%)
fills in affected programs: 4466 -> 3168 (-29.06%)
helped: 67
HURT: 0
LOST: 1
GAINED: 1
Broadwell
total instructions in shared programs:
14970538 ->
14965967 (-0.03%)
instructions in affected programs: 227040 -> 222469 (-2.01%)
helped: 126
HURT: 2
helped stats (abs) min: 1 max: 136 x̄: 36.29 x̃: 8
helped stats (rel) min: 0.07% max: 6.02% x̄: 1.47% x̃: 0.89%
HURT stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel) min: 0.05% max: 0.24% x̄: 0.14% x̃: 0.14%
95% mean confidence interval for instructions value: -43.05 -28.37
95% mean confidence interval for instructions %-change: -1.69% -1.19%
Instructions are helped.
total cycles in shared programs:
336237662 ->
333035960 (-0.95%)
cycles in affected programs:
72066394 ->
68864692 (-4.44%)
helped: 134
HURT: 42
helped stats (abs) min: 4 max: 122640 x̄: 24344.54 x̃: 1833
helped stats (rel) min: <.01% max: 26.93% x̄: 4.02% x̃: 2.38%
HURT stats (abs) min: 1 max: 17205 x̄: 1439.69 x̃: 92
HURT stats (rel) min: <.01% max: 7.12% x̄: 1.34% x̃: 0.62%
95% mean confidence interval for cycles value: -23753.58 -12629.40
95% mean confidence interval for cycles %-change: -3.50% -1.98%
Cycles are helped.
total spills in shared programs: 21122 -> 20204 (-4.35%)
spills in affected programs: 3644 -> 2726 (-25.19%)
helped: 67
HURT: 0
total fills in shared programs: 24879 -> 23460 (-5.70%)
fills in affected programs: 4883 -> 3464 (-29.06%)
helped: 67
HURT: 0
Haswell
total instructions in shared programs:
13148269 ->
13145444 (-0.02%)
instructions in affected programs: 137046 -> 134221 (-2.06%)
helped: 97
HURT: 3
helped stats (abs) min: 1 max: 137 x̄: 30.58 x̃: 3
helped stats (rel) min: 0.14% max: 4.38% x̄: 1.38% x̃: 0.44%
HURT stats (abs) min: 1 max: 70 x̄: 47.00 x̃: 70
HURT stats (rel) min: 0.05% max: 5.82% x̄: 3.90% x̃: 5.82%
95% mean confidence interval for instructions value: -37.15 -19.35
95% mean confidence interval for instructions %-change: -1.56% -0.89%
Instructions are helped.
total cycles in shared programs:
321221834 ->
318333159 (-0.90%)
cycles in affected programs:
54932349 ->
52043674 (-5.26%)
helped: 95
HURT: 53
helped stats (abs) min: 4 max: 123390 x̄: 30648.39 x̃: 702
helped stats (rel) min: <.01% max: 28.87% x̄: 4.27% x̃: 2.87%
HURT stats (abs) min: 4 max: 2357 x̄: 432.49 x̃: 113
HURT stats (rel) min: <.01% max: 3.44% x̄: 1.03% x̃: 0.54%
95% mean confidence interval for cycles value: -26154.16 -12881.99
95% mean confidence interval for cycles %-change: -3.20% -1.55%
Cycles are helped.
total spills in shared programs: 19878 -> 19293 (-2.94%)
spills in affected programs: 3020 -> 2435 (-19.37%)
helped: 41
HURT: 2
total fills in shared programs: 20918 -> 19875 (-4.99%)
fills in affected programs: 3968 -> 2925 (-26.29%)
helped: 41
HURT: 2
LOST: 0
GAINED: 1
Ivy Bridge
total instructions in shared programs:
11875585 ->
11873641 (-0.02%)
instructions in affected programs: 78065 -> 76121 (-2.49%)
helped: 27
HURT: 0
helped stats (abs) min: 8 max: 134 x̄: 72.00 x̃: 72
helped stats (rel) min: 0.36% max: 4.23% x̄: 2.42% x̃: 2.42%
95% mean confidence interval for instructions value: -83.68 -60.32
95% mean confidence interval for instructions %-change: -2.78% -2.07%
Instructions are helped.
total cycles in shared programs:
178232734 ->
175769085 (-1.38%)
cycles in affected programs:
50018707 ->
47555058 (-4.93%)
helped: 27
HURT: 0
helped stats (abs) min: 82035 max: 99953 x̄: 91246.26 x̃: 92278
helped stats (rel) min: 4.40% max: 5.69% x̄: 4.93% x̃: 4.95%
95% mean confidence interval for cycles value: -93674.20 -88818.32
95% mean confidence interval for cycles %-change: -5.09% -4.78%
Cycles are helped.
total spills in shared programs: 4182 -> 3739 (-10.59%)
spills in affected programs: 1089 -> 646 (-40.68%)
helped: 27
HURT: 0
total fills in shared programs: 5216 -> 4345 (-16.70%)
fills in affected programs: 1874 -> 1003 (-46.48%)
helped: 27
HURT: 0
No changes on any earlier Intel platforms.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4156>
*/
bool intel_vec4;
+ /** Lower nir_op_ibfe and nir_op_ubfe that have two constant sources. */
+ bool lower_bfe_with_two_constants;
+
/** Whether 8-bit ALU is supported. */
bool support_8bit_alu;
# any conversions that could have been removed will have been removed in
# nir_opt_algebraic so any remaining ones are required.
(('f2fmp', a), ('f2f16', a)),
+
+ # Section 8.8 (Integer Functions) of the GLSL 4.60 spec says:
+ #
+ # If bits is zero, the result will be zero.
+ #
+ # These prevent the next two lowerings generating incorrect results when
+ # count is zero.
+ (('ubfe', a, b, 0), 0),
+ (('ibfe', a, b, 0), 0),
+
+ # On Intel GPUs, BFE is a 3-source instruction. Like all 3-source
+ # instructions on Intel GPUs, it cannot have an immediate values as
+ # sources. There are also limitations on source register strides. As a
+ # result, it is very easy for 3-source instruction combined with either
+ # loads of immediate values or copies from weird register strides to be
+ # more expensive than the primitive instructions it represents.
+ (('ubfe', a, '#b', '#c'), ('iand', ('ushr', 0xffffffff, ('ineg', c)), ('ushr', a, b)), 'options->lower_bfe_with_two_constants'),
+
+ # b is the lowest order bit to be extracted and c is the number of bits to
+ # extract. The inner shift removes the bits above b + c by shifting left
+ # 32 - (b + c). ishl only sees the low 5 bits of the shift count, which is
+ # -(b + c). The outer shift moves the bit that was at b to bit zero.
+ # After the first shift, that bit is now at b + (32 - (b + c)) or 32 - c.
+ # This means that it must be shifted right by 32 - c or -c bits.
+ (('ibfe', a, '#b', '#c'), ('ishr', ('ishl', a, ('ineg', ('iadd', b, c))), ('ineg', c)), 'options->lower_bfe_with_two_constants'),
+
+ # Clean up no-op shifts that may result from the bfe lowerings.
+ (('ishl', a, 0), a),
+ (('ishl', a, -32), a),
+ (('ishr', a, 0), a),
+ (('ishr', a, -32), a),
+ (('ushr', a, 0), a),
]
for op in ['fadd']:
.lower_unpack_unorm_4x8 = true, \
.lower_usub_sat64 = true, \
.lower_hadd64 = true, \
+ .lower_bfe_with_two_constants = true, \
.max_unroll_iterations = 32
static const struct nir_shader_compiler_options scalar_nir_options = {