radeonsi: skip vs output optimizations for some outputs

If PT_SPRITE_TEX is enabled, PS inputs are overriden at runtime so
we can't apply the vs output optim.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2747
Fixes: 3ec9975555d ("radeonsi: eliminate trivial constant VS outputs")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4559>

nir/gcm: dont move movs unless we can replace them later with their src

This helps us avoid moving the movs outside if branches when there
src can't be scalarized.

For example it avoids:

   vec4 32 ssa_7 = tex ssa_6 (coord), 0 (texture), 0 (sampler),
   if ... {
      r0 = imov ssa_7.z
      r1 = imov ssa_7.y
      r2 = imov ssa_7.x
      r3 = imov ssa_7.w
      ...
   } else {
      ...
      if ... {
         r0 = imov ssa_7.x
         r1 = imov ssa_7.w
         ...
      else {
         r0 = imov ssa_7.z
         r1 = imov ssa_7.y
         ...
      }
      r2 = imov ssa_7.x
      r3 = imov ssa_7.w
   }
   ...
   vec4 32 ssa_36 = vec4 r0, r1, r2, r3

Becoming something like:

   vec4 32 ssa_7 = tex ssa_6 (coord), 0 (texture), 0 (sampler),
   r0 = imov ssa_7.z
   r1 = imov ssa_7.y
   r2 = imov ssa_7.x
   r3 = imov ssa_7.w

   if ... {
      ...
   } else {
      if ... {
         r0 = imov r2
         r1 = imov r3
         ...
      else {
         ...
      }
      ...
   }

While this is has a smaller instruction count it requires more work
for the same result. With more complex examples we can also end up
shuffling the registers around in a way that requires more registers
to use as temps so that we don't overwrite our original values along
the way.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: be more conservative about moving instructions from loops

Here we only pull instructions further up control flow if they are
constant or texture instructions. See the code comment for more
information.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: allow derivative dependent intrinisics to be moved earlier

We can't move them later as we could move them into non-uniform
control flow, but moving them earlier should be fine.

This helps avoid a bunch of spilling in unigine shaders due to
moving the tex instructions sources earlier (outside if branches)
but not the instruction itself.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Prefer the instruction's original block

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Delete dead instructions

Classically, global code motion is also a dead code pass.  However, in
the initial implementation, the decision was made to place every
instruction and let conventional DCE clean up the dead ones.  Because
any uses of a dead instruction are unreachable, we have no late block
and the dead instructions are always scheduled early.  The problem is
that, because we place the dead instruction early, it  pushes the
placement of any dependencies of the dead instruction earlier than they
may need to be placed.  In order prevent dead instructions from
affecting the placement of live ones, we need to delete them.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Add a real concept of "progress"

Now that the GCM pass is more conservative and only moves instructions
to different blocks when it's advantageous to do so, we can have a
proper notion of what it means to make progress.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Move block choosing into a helper function

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Use an array for storing the early block

We are about to adjust our instruction block assignment algorithm and we
will want to know the current block that the instruction lives in.  In
order to allow for this, we can't overwrite nir_instr::block in the
early scheduling pass.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/gcm: Loop over blocks in pin_instructions

Now that we have the new block iterators, we can simplify things a bit.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

nir/dominance: Better handle unreachable blocks

v2: Fix minor comments (Ken)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4636>

aco: use v_subrev_f32 for fsub with an sgpr operand in src1

This fixes an accidentally introduced regression.

Fixes: 9be4be515f2a08b9c9e5ae1fc4c5dc9a830c2337 ('aco: implement 16-bit nir_op_fsub/nir_op_fadd')
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4633>

CI: Disable Lima jobs due to lab unhealthiness

The BayLibre LAVA host appears to be down.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4631>

ci/windows: Make Chocolatey installs more reliable

Chocolatey installs depend on downloading binaries from SourceForge,
which is an unreliable host: container builds often fail because it
cannot pick up winflexbison.

Add a loop to retry chocolatey installs if any installs have failed, and
ensure Python is in the accessible PowerShell path rather than relying
on the path being externally refreshed.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4631>

nir: Lower returns correctly inside nested loops

Inside nested flow control, nir_lower_returns inserts predicated breaks
in the outer block. However, it would omit doing this if the remainder
of the outer block (after the inner block) was empty. This is not
correct in the case of loops, as execution just wraps back around to the
start of the loop, so this change doesn't skip the predication inside
loops.

Fixes: 79dec93ead6e (nir: Add return lowering pass)
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2724
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4603>

anv: Apply any needed PIPE_CONTROLs before emitting state

Push constants in particular can get picked up by the hardware at weird
times that happen *before* 3DPRIMITIVE.  Therefore, we need to flush
before we emit all our state to ensure that any data they may pick up is
in memory in time.  This fixes an app which does vkCmdCopyBuffers
immediately followed by a vkCmdBeginRenderPass and vkCmdDraw which uses
the destination of the copy as a UBO which we push.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4601>

anv: Move vb_emit setup closer to where it's used in flush_state

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4601>

Fix promotion of floats to doubles

Use the f variants of the math functions if the input parameter is a
float, saves converting from float to double and running the double
variant of the math function for gaining no precision at all

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3969>

ir3/ra: Fix off-by-one issues with live-range extension

The intersects() function assumes that inside each instruction values
always die before they are defined, so that if the end of one range is
the same instruction as the beginning of the next then they don't
intersect. However, this isn't the case for values that become live at
the beginning of a basic block, which become live *before* the first
instruction, or instructions that die at the end of a basic block which
die after the last instruction.

For example, imagine that we have two values, A which is defined earlier
in the block and B which is defined in the last instruction of the block
and both die at the end of the basic block (e.g. are used in the next
iteration of a loop). We would compute a range for A of, say, (10, 20)
and for B of (20, 20) since each block's end_ip is the same as the ip of
the last instruction, and RA would consider them to not interfere.
There's a similar problem with values that become live at the beginning.

The fix is to offset the block's start_ip and end_ip by one so that they
don't correspond to any actual instruction. One way to think about this
is that we're adding fake instructions at the beginning and end of a
block where values become live & die. We could invert the order, so that
values consumed by each instruction are considered dead at the end of
the previous instruction, but then values that become dead at the
beginning of the basic block would incorrectly have an empty live range,
with a similar problem at the end of the basic block if we try to say
that values are defined at the beginning of the next instruction. So
the extra padding instructions are unavoidable.

This fixes an accidental infinite loop in the shader for
dEQP-VK.spirv_assembly.type.scalar.u32.switch_vert.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4614>

util/sparse_free_list: manipulate node pointers using atomic primitives

Probably doesn't fix anything but those should be accessed in an
atomic way just like the head pointer.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: e4f01eca3b3cd1 ("util: Add a free list structure for use with util_sparse_array")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4613>

glsl: only set stage ref when uniforms referenced in stage

This updates the NIR uniform linker to behave like the GLSL IR
linker and fixes a number of CTS tests once we enable the NIR
linker for glsl.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: pull mark_array_elements_referenced() out into common helper

We will reuse this helper in the NIR linker in the following
patches.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: fix block index in NIR uniform linker

We only want to set the index for the first block of an array. Also
add a comment about why we do not break here.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: error check max user assignable uniform locations

This adds the error check to the NIR uniform linker.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: fix explicit locations for the glsl linker

We already reserved explicit locations in the GLSL linker.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

Revert "glsl: fix resizing of the uniform remap table"

This reverts commit e0aa0a839f9c168784a1f50013c83877cc876094.

Instead we fix it correctly in the following patch.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: tidy up uniform storage value count code in NIR linker

This makes the code cleaner and better reflects what the existing
glsl IR  linker does possibly fixing subtle bugs.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

glsl: fix struct offsets in the nir uniform linker

This change properly applies layouts to structs of uniforms in a
similar way to the GLSL IR linker.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

nir: add matrix_layout to nir_variable data

This will be used by the following patch.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4623>

anv: skip writing perfcntr in results on Gen12+

We were not capturing the register already so don't bother writing the
delta in the results (we were previously doing a delta between two 0
values).

v2: Fix unused function warning

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4586>

intel/perf: Enable MDAPI queries for Gen12

We're missing the cases for gen12 leading to those metrics going
missing.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 15b7b56eb2fb41 ("intel/perf: add TGL support")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4586>

pan/bit: Add fp16 min/max tests

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add constants test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add fexp2_fast test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add fexp2_fast interp

Kind of a hack and not at all how the h/w does it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add FMA_MSCALE test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: _MSCALE interp

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add BI_TABLE test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add log2 helper interp

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add FMA_REDUCE test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add BI_REDUCE_FMA interp

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add frexp_log test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bit: Add FREXP interp support

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Lower special ops to 32-bit

We don't have 16-bit tables. We could probably do a bit better to avoid
so many conversions but hey.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Round constants to 32-bit

We can only access lo/hi at 32-bit intervals.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Dump extra bits for disasm

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Pack MAX.v2f16

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Pack ADD.v2f16

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Structify add and min/max fp16 ADD

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Workaround constant packing errata

Incomplete fix.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Try to reuse constants in ALU

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Handle st_vary with <4 components

Still no writemasks.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Fix vec2/3 handling

Otherwise we get moves from null.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Implement flog2

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add fexp2 implementation

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Fix lower_combine swizzle rewrite

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Fix packing with low-nibble-set on hi constant

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Fix packing with multiple constants

Need to use bottom nibble of the 64, not the half.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Fix bi_get_immediate with multiple imms

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Ensure CONSTANT srcs have types

So the next commit is valid.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Split src/dest index printing

So we can handle constant printing correctly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add fexp2_fast packing

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Pack FMA_MSCALE

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Structify FMA_MSCALE

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add _MSCALE flag for FMA/ADD

So we can bias by exponents.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add log2_help packing

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Pack ADD_FREXPM

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add bi_pack_fma_2src helper

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add frexp_log packing

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add log_frexpe op to IR

As part of BI_FREXP

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add FLOG2_U op to disassembler

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add op for ADD_FREXPM

Used in log2. Needs a new class as well due to scheduling silliness.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add special op for exp2

Needs some extra help but basically exp2_fast

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add BI_TABLE for fast table accesses

Used to implement SPECIAL ops. Separate class since they are faster
which means you can pair them with actual work on FMA.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Disable FMA scheduling for CONVERT

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

pan/bi: Add disasm for ADD.i8

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4615>

spirv,nir: Move the SPIR-V vector insert code to NIR

This also makes spirv_to_nir a bit simpler because the new
nir_vector_insert helper automatically handles a constant component
selector like nir_vector_extract does.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

spirv: Call nir_builder directly for vector_extract

The nir_builder helper already handles checking if the component
selector is an immediate and returns an undef in the OOB case.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

nir/builder: Handle any bit-size selector in nir_extract

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

spirv: Error if OpCompositeInsert/Extract has OOB indices

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

spirv,nir: Add a better vector_insert

The old one in spirv_to_nir was besel'ing the whole vector for every
component.  If we think about this as a vector operation, we can do it
way more efficiently.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

spirv: Handle OOB vector extract operations

We use vtn_vector_extract to handle vector component level derefs.  This
makes us gracefully handle the case where your vector component is OOB
and give you an undef.  The SPIR-V working group is still working out
whether or not this is technically legal but it's very little code for
us to handle it so we may as well.

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4495>

util/sparse_array: don't stomp head's counter on pop operations

By temporarily storing the new_head by a uint32_t, we wipe out the
counter section of the head pointer.

Fixes: e4f01eca ("util: Add a free list structure for use with util_sparse_array")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4612>

st/mesa: Update shader info of ffvp/ARB_vp after translation to NIR

We must update stp->Base.info after translation and before
st_prepare_vertex_program is called, because inputs_read
may become outdated after NIR optimization passes.

For ffvp/ARB_vp inputs_read is populated based on declared
attributes without taking their usage into consideration.
When creating shader variants we expect that their inputs_read
would match the base ones for input mapping to work properly.

Cc: <mesa-stable@lists.freedesktop.org>
Fixes: 8a0dd0af3f1a6c0310a08daf4220132ec6815b31
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2758
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4598>

aco: fix exporting the viewport index if the fragment shader needs it

It's like the layer, it has to be exported via the pos and also
as a varying if the fragment shader reads it.

Fixes dEQP-VK.draw.shader_viewport_index.fragment_shader_*

Cc: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4564>

radv/llvm: fix exporting the viewport index if the fragment shader needs it

It's like the layer, it has to be exported via the pos and also
as a varying if the fragment shader reads it.

Fixes dEQP-VK.draw.shader_viewport_index.fragment_shader_*

Cc: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4564>

radv: set missing SHARED_VGPR_CNT for NGG VS and ACO

shuffle is implemented with shared VGPRs with ACO and Wave64.

Fixes dEQP-VK.subgroups.shuffle.framebuffer.subgroupshuffle*_vertex
with Wave64.

Fixes: c24d9522dae ("radv: Enable ACO for NGG VS/TES, but disable NGG for ACO GS.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4595>

radv: fix geometry shader primitives query with ACO on GFX10

Fixes
dEQP-VK.query_pool.statistics_query.*.geometry_shader_primitives.*.

Fixes: c24d9522dae ("radv: Enable ACO for NGG VS/TES, but disable NGG for ACO GS.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4593>

intel/compiler: Fixup operands in fs_builder::emit() that takes array

The versions that take a specific number of operands will do various
fixups depending on the platform and the opcode.  However, the version
that takes an array of sources did not.  This makes all version operate
similarly.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/compiler: CSEL can do saturate

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/compiler: Only GE and L modifiers are commutative for SEL

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/compiler: Silence unused parameter warning in update_inst_scoreboard

src/intel/compiler/brw_fs_scoreboard.cpp: In function ‘void {anonymous}::update_inst_scoreboard(const fs_visitor*, const ordered_address*, const fs_inst*, unsigned int, {anonymous}::scoreboard&)’:
src/intel/compiler/brw_fs_scoreboard.cpp:793:45: warning: unused parameter ‘shader’ [-Wunused-parameter]
  793 |    update_inst_scoreboard(const fs_visitor *shader, const ordered_address *jps,
      |                           ~~~~~~~~~~~~~~~~~~^~~~~~

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/compiler: Silence unused parameter warning in fs_live_variables::setup_one_read

src/intel/compiler/brw_fs_live_variables.cpp: In member function ‘void brw::fs_live_variables::setup_one_read(brw::fs_live_variables::block_data*, fs_inst*, int, const fs_reg&)’:
src/intel/compiler/brw_fs_live_variables.cpp:56:67: warning: unused parameter ‘inst’ [-Wunused-parameter]
   56 | fs_live_variables::setup_one_read(struct block_data *bd, fs_inst *inst,
      |                                                          ~~~~~~~~~^~~~

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/compiler: Silence unused parameter warnings in vec4_tcs_visitor

In file included from src/intel/compiler/brw_vec4_tcs.cpp:31:
src/intel/compiler/brw_vec4_tcs.h: In member function ‘virtual void brw::vec4_tcs_visitor::emit_urb_write_header(int)’:
src/intel/compiler/brw_vec4_tcs.h:74:43: warning: unused parameter ‘mrf’ [-Wunused-parameter]
   74 |    virtual void emit_urb_write_header(int mrf) {}
      |                                       ~~~~^~~
src/intel/compiler/brw_vec4_tcs.h: In member function ‘virtual brw::vec4_instruction* brw::vec4_tcs_visitor::emit_urb_write_opcode(bool)’:
src/intel/compiler/brw_vec4_tcs.h:75:57: warning: unused parameter ‘complete’ [-Wunused-parameter]
   75 |    virtual vec4_instruction *emit_urb_write_opcode(bool complete) { return NULL; }
      |                                                    ~~~~~^~~~~~~~

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4582>

intel/blorp: Delete an unused enum

This was lying around from back when BLORP write to fs_visitor directly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4606>

anv: Emit pushed UBO bounds checking code in the back-end compiler

This commit fixes performance regressions introduced by e03f9652801ad7
in which we started bounds checking our push constants.  This added a
LOT of shader code to shaders which use the robustBufferAccess feature
and led to substantial spilling.  The checking we just added to the FS
back-end is far more efficient for two reasons:

 1. It can be done at a whole register granularity rather than per-
    scalar and so we emit one SIMD8 SEL per 32B GRF rather than one
    SIMD16 SEL (executed as two SELs) for each component loaded.

 2. Because we do it with NoMask instructions, we can do it on whole
    pushed GRFs without splatting them out to SIMD8 or SIME16 values.
    This means that robust buffer access no longer explodes our register
    pressure for no good reason.

As a tiny side-benefit, we're now using can use AND instead of SEL which
means no need for the flag and better scheduling.

Vulkan pipeline database results on ICL:

    Instructions in all programs: 293586059 -> 238009118 (-18.9%)
    SENDs in all programs: 13568515 -> 13568515 (+0.0%)
    Loops in all programs: 149720 -> 149720 (+0.0%)
    Cycles in all programs: 88499234498 -> 84348917496 (-4.7%)
    Spills in all programs: 1229018 -> 184339 (-85.0%)
    Fills in all programs: 1348397 -> 246061 (-81.8%)

This also improves the performance of a few apps:

 - Shadow of the Tomb Raider: +4%
 - Witcher 3: +3.5%
 - UE4 Shooter demo: +2%

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4447>

intel/cfg: Add first/last_block helpers

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4447>

tu: Use tu_cs_add_entries() with non-render-pass secondaries

Even though vkCmdRenderPassBegin() isn't allowed inside a secondary
command buffer, vkCmdDispatch() is, and we emit an IB with compute
dispatches, which means that if the secondary command buffer records a
vkCmdDispatch() then we'll have an IB inside an IB, which is illegal.
Fixes hangs in e.g.
dEQP-VK.api.command_buffers.record_simul_use_secondary_one_primary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4605>

mesa: add interaction between compute derivatives and variable local sizes

This is an added interaction in NV_compute_shader_derivatives added in
Sep 2019.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4583>

st/mesa: properly guard fallback_copy_texsubimage aginst failed maps

Fixes random crashes in some packed_pixel GLES CTS tests

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4592>