nir/algebraic: Recognize open-coded byte or word extract from bfe

v2: Move word-extract patterns up near the byte-extract patterns.
Suggested by Rhys.

Tiger Lake
total instructions in shared programs: 21369236 -> 21368712 (<.01%)
instructions in affected programs: 913104 -> 912580 (-0.06%)
helped: 209
HURT: 165
helped stats (abs) min: 1 max: 30 x̄: 5.35 x̃: 3
helped stats (rel) min: 0.03% max: 6.92% x̄: 0.28% x̃: 0.12%
HURT stats (abs)   min: 1 max: 18 x̄: 3.61 x̃: 3
HURT stats (rel)   min: 0.04% max: 0.87% x̄: 0.16% x̃: 0.12%
95% mean confidence interval for instructions value: -2.04 -0.76
95% mean confidence interval for instructions %-change: -0.14% -0.04%
Instructions are helped.

total cycles in shared programs: 490161481 -> 490175959 (<.01%)
cycles in affected programs: 72557244 -> 72571722 (0.02%)
helped: 193
HURT: 189
helped stats (abs) min: 1 max: 14240 x̄: 509.16 x̃: 71
helped stats (rel) min: <.01% max: 13.71% x̄: 0.44% x̃: 0.05%
HURT stats (abs)   min: 2 max: 4210 x̄: 596.53 x̃: 173
HURT stats (rel)   min: <.01% max: 5.59% x̄: 0.54% x̃: 0.14%
95% mean confidence interval for cycles value: -96.33 172.13
95% mean confidence interval for cycles %-change: -0.07% 0.16%
Inconclusive result (value mean confidence interval includes 0).

total spills in shared programs: 10780 -> 10782 (0.02%)
spills in affected programs: 18 -> 20 (11.11%)
helped: 0
HURT: 1

total fills in shared programs: 10396 -> 10370 (-0.25%)
fills in affected programs: 2292 -> 2266 (-1.13%)
helped: 27
HURT: 1

Ice Lake
total instructions in shared programs: 19556356 -> 19555446 (<.01%)
instructions in affected programs: 833336 -> 832426 (-0.11%)
helped: 400
HURT: 0
helped stats (abs) min: 1 max: 20 x̄: 2.27 x̃: 2
helped stats (rel) min: 0.07% max: 4.42% x̄: 0.14% x̃: 0.10%
95% mean confidence interval for instructions value: -2.42 -2.13
95% mean confidence interval for instructions %-change: -0.18% -0.11%
Instructions are helped.

total cycles in shared programs: 488026481 -> 488008714 (<.01%)
cycles in affected programs: 81581708 -> 81563941 (-0.02%)
helped: 193
HURT: 206
helped stats (abs) min: 1 max: 3615 x̄: 576.35 x̃: 131
helped stats (rel) min: <.01% max: 4.50% x̄: 0.49% x̃: 0.22%
HURT stats (abs)   min: 1 max: 2244 x̄: 453.73 x̃: 170
HURT stats (rel)   min: <.01% max: 5.71% x̄: 0.36% x̃: 0.14%
95% mean confidence interval for cycles value: -127.23 38.17
95% mean confidence interval for cycles %-change: -0.12% 0.03%
Inconclusive result (value mean confidence interval includes 0).

total fills in shared programs: 9935 -> 9908 (-0.27%)
fills in affected programs: 2208 -> 2181 (-1.22%)
helped: 27
HURT: 0

Skylake
total instructions in shared programs: 17766078 -> 17765186 (<.01%)
instructions in affected programs: 822017 -> 821125 (-0.11%)
helped: 399
HURT: 1
helped stats (abs) min: 1 max: 20 x̄: 2.27 x̃: 2
helped stats (rel) min: 0.07% max: 4.46% x̄: 0.15% x̃: 0.10%
HURT stats (abs)   min: 12 max: 12 x̄: 12.00 x̃: 12
HURT stats (rel)   min: 0.50% max: 0.50% x̄: 0.50% x̃: 0.50%
95% mean confidence interval for instructions value: -2.39 -2.07
95% mean confidence interval for instructions %-change: -0.18% -0.11%
Instructions are helped.

total cycles in shared programs: 470905548 -> 470907497 (<.01%)
cycles in affected programs: 78598491 -> 78600440 (<.01%)
helped: 202
HURT: 192
helped stats (abs) min: 1 max: 3690 x̄: 228.98 x̃: 60
helped stats (rel) min: <.01% max: 4.51% x̄: 0.24% x̃: 0.03%
HURT stats (abs)   min: 1 max: 2260 x̄: 251.05 x̃: 77
HURT stats (rel)   min: <.01% max: 5.31% x̄: 0.24% x̃: 0.06%
95% mean confidence interval for cycles value: -45.01 54.90
95% mean confidence interval for cycles %-change: -0.07% 0.05%
Inconclusive result (value mean confidence interval includes 0).

total spills in shared programs: 9941 -> 9943 (0.02%)
spills in affected programs: 26 -> 28 (7.69%)
helped: 0
HURT: 1

total fills in shared programs: 10293 -> 10268 (-0.24%)
fills in affected programs: 2391 -> 2366 (-1.05%)
helped: 27
HURT: 1

Broadwell
total instructions in shared programs: 17463211 -> 17462366 (<.01%)
instructions in affected programs: 861444 -> 860599 (-0.10%)
helped: 399
HURT: 1
helped stats (abs) min: 1 max: 20 x̄: 2.14 x̃: 2
helped stats (rel) min: 0.03% max: 4.46% x̄: 0.14% x̃: 0.09%
HURT stats (abs)   min: 7 max: 7 x̄: 7.00 x̃: 7
HURT stats (rel)   min: 0.33% max: 0.33% x̄: 0.33% x̃: 0.33%
95% mean confidence interval for instructions value: -2.26 -1.97
95% mean confidence interval for instructions %-change: -0.17% -0.10%
Instructions are helped.

total cycles in shared programs: 507048912 -> 506898243 (-0.03%)
cycles in affected programs: 79806433 -> 79655764 (-0.19%)
helped: 248
HURT: 136
helped stats (abs) min: 1 max: 8450 x̄: 1124.18 x̃: 64
helped stats (rel) min: <.01% max: 5.91% x̄: 0.83% x̃: 0.05%
HURT stats (abs)   min: 2 max: 7632 x̄: 942.12 x̃: 103
HURT stats (rel)   min: <.01% max: 5.62% x̄: 0.71% x̃: 0.08%
95% mean confidence interval for cycles value: -647.01 -137.73
95% mean confidence interval for cycles %-change: -0.47% -0.10%
Cycles are helped.

total spills in shared programs: 22996 -> 22998 (<.01%)
spills in affected programs: 31 -> 33 (6.45%)
helped: 0
HURT: 1

total fills in shared programs: 25951 -> 25923 (-0.11%)
fills in affected programs: 2444 -> 2416 (-1.15%)
helped: 29
HURT: 1

Haswell
total instructions in shared programs: 15841325 -> 15840554 (<.01%)
instructions in affected programs: 869679 -> 868908 (-0.09%)
helped: 394
HURT: 6
helped stats (abs) min: 1 max: 20 x̄: 2.15 x̃: 2
helped stats (rel) min: 0.06% max: 4.46% x̄: 0.14% x̃: 0.09%
HURT stats (abs)   min: 7 max: 18 x̄: 12.83 x̃: 13
HURT stats (rel)   min: 0.32% max: 0.82% x̄: 0.59% x̃: 0.61%
95% mean confidence interval for instructions value: -2.16 -1.69
95% mean confidence interval for instructions %-change: -0.16% -0.09%
Instructions are helped.

total cycles in shared programs: 520417167 -> 520279766 (-0.03%)
cycles in affected programs: 80949963 -> 80812562 (-0.17%)
helped: 246
HURT: 139
helped stats (abs) min: 1 max: 8152 x̄: 790.08 x̃: 129
helped stats (rel) min: <.01% max: 11.46% x̄: 0.70% x̃: 0.09%
HURT stats (abs)   min: 1 max: 7085 x̄: 409.78 x̃: 80
HURT stats (rel)   min: <.01% max: 5.25% x̄: 0.31% x̃: 0.06%
95% mean confidence interval for cycles value: -526.34 -187.43
95% mean confidence interval for cycles %-change: -0.49% -0.18%
Cycles are helped.

total spills in shared programs: 21714 -> 21729 (0.07%)
spills in affected programs: 174 -> 189 (8.62%)
helped: 0
HURT: 6

total fills in shared programs: 22136 -> 22132 (-0.02%)
fills in affected programs: 2848 -> 2844 (-0.14%)
helped: 31
HURT: 6

Ivy Bridge
total instructions in shared programs: 15177059 -> 15177003 (<.01%)
instructions in affected programs: 79370 -> 79314 (-0.07%)
helped: 29
HURT: 0
helped stats (abs) min: 1 max: 2 x̄: 1.93 x̃: 2
helped stats (rel) min: 0.06% max: 0.16% x̄: 0.08% x̃: 0.07%
95% mean confidence interval for instructions value: -2.03 -1.83
95% mean confidence interval for instructions %-change: -0.09% -0.07%
Instructions are helped.

total cycles in shared programs: 420424359 -> 420417254 (<.01%)
cycles in affected programs: 29562648 -> 29555543 (-0.02%)
helped: 23
HURT: 6
helped stats (abs) min: 2 max: 2741 x̄: 432.57 x̃: 142
helped stats (rel) min: <.01% max: 0.26% x̄: 0.04% x̃: 0.02%
HURT stats (abs)   min: 4 max: 1184 x̄: 474.00 x̃: 226
HURT stats (rel)   min: <.01% max: 0.11% x̄: 0.05% x̃: 0.05%
95% mean confidence interval for cycles value: -553.48 63.48
95% mean confidence interval for cycles %-change: -0.05% <.01%
Inconclusive result (value mean confidence interval includes 0).

total fills in shared programs: 6420 -> 6393 (-0.42%)
fills in affected programs: 1901 -> 1874 (-1.42%)
helped: 27
HURT: 0

No changes on any earlier Intel platforms.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4156>

diff --git a/src/compiler/nir/nir_opt_algebraic.py b/src/compiler/nir/nir_opt_algebraic.py
index 4112ccb0aa48893559497cebdb2b3a2a7d4f3d85..1f8f6fd1d6f1b1c648c0da1138e7205900da0bb8 100644 (file)
--- a/src/compiler/nir/nir_opt_algebraic.py
+++ b/src/compiler/nir/nir_opt_algebraic.py
@@ -999,6 +999,27 @@ optimizations.extend([
    (('ishr', 'a@64', 56), ('extract_i8', a, 7), '!options->lower_extract_byte'),
    (('iand', 0xff, a), ('extract_u8', a, 0), '!options->lower_extract_byte'),
 
+   (('ubfe', a,  0, 8), ('extract_u8', a, 0), '!options->lower_extract_byte'),
+   (('ubfe', a,  8, 8), ('extract_u8', a, 1), '!options->lower_extract_byte'),
+   (('ubfe', a, 16, 8), ('extract_u8', a, 2), '!options->lower_extract_byte'),
+   (('ubfe', a, 24, 8), ('extract_u8', a, 3), '!options->lower_extract_byte'),
+   (('ibfe', a,  0, 8), ('extract_i8', a, 0), '!options->lower_extract_byte'),
+   (('ibfe', a,  8, 8), ('extract_i8', a, 1), '!options->lower_extract_byte'),
+   (('ibfe', a, 16, 8), ('extract_i8', a, 2), '!options->lower_extract_byte'),
+   (('ibfe', a, 24, 8), ('extract_i8', a, 3), '!options->lower_extract_byte'),
+
+    # Word extraction
+   (('ushr', ('ishl', 'a@32', 16), 16), ('extract_u16', a, 0), '!options->lower_extract_word'),
+   (('ushr', 'a@32', 16), ('extract_u16', a, 1), '!options->lower_extract_word'),
+   (('ishr', ('ishl', 'a@32', 16), 16), ('extract_i16', a, 0), '!options->lower_extract_word'),
+   (('ishr', 'a@32', 16), ('extract_i16', a, 1), '!options->lower_extract_word'),
+   (('iand', 0xffff, a), ('extract_u16', a, 0), '!options->lower_extract_word'),
+
+   (('ubfe', a,  0, 16), ('extract_u16', a, 0), '!options->lower_extract_word'),
+   (('ubfe', a, 16, 16), ('extract_u16', a, 1), '!options->lower_extract_word'),
+   (('ibfe', a,  0, 16), ('extract_i16', a, 0), '!options->lower_extract_word'),
+   (('ibfe', a, 16, 16), ('extract_i16', a, 1), '!options->lower_extract_word'),
+
    # Useless masking before unpacking
    (('unpack_half_2x16_split_x', ('iand', a, 0xffff)), ('unpack_half_2x16_split_x', a)),
    (('unpack_32_2x16_split_x', ('iand', a, 0xffff)), ('unpack_32_2x16_split_x', a)),
@@ -1034,13 +1055,6 @@ for op in ('extract_u8', 'extract_i8'):
    optimizations.extend([((op, ('ishl', 'a@64', 56 - 8 * i), 7), (op, a, i)) for i in range(6, -1, -1)])
 
 optimizations.extend([
-    # Word extraction
-   (('ushr', ('ishl', 'a@32', 16), 16), ('extract_u16', a, 0), '!options->lower_extract_word'),
-   (('ushr', 'a@32', 16), ('extract_u16', a, 1), '!options->lower_extract_word'),
-   (('ishr', ('ishl', 'a@32', 16), 16), ('extract_i16', a, 0), '!options->lower_extract_word'),
-   (('ishr', 'a@32', 16), ('extract_i16', a, 1), '!options->lower_extract_word'),
-   (('iand', 0xffff, a), ('extract_u16', a, 0), '!options->lower_extract_word'),
-
    # Subtracts
    (('ussub_4x8', a, 0), a),
    (('ussub_4x8', a, ~0), 0),