mesa: add Const.BufferCreateMapUnsynchronizedThreadSafe & MESA_MAP_THREAD_SAFE

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

gallium: add PIPE_CAP_MAP_UNSYNCHRONIZED_THREAD_SAFE for glthread

and add radeonsi support.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

glthread: sort variables in marshal structures to pack them optimally

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

glthread: use GLenum16 in batch buffers to save space

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

glthread: reduce dereferences of the next batch

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

glthread: use 32-bit align instead of 64-bit ALIGN

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

mesa: remove exec="dynamic" from Draw functions that are not really dynamic

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

mesa: reset primitive restart state in glClientAttribDefaultEXT

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

mesa: replace _NEW_EVAL with vbo_exec_update_eval_maps

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4758>

ac: reassociate FP expressions for inexact instructions for radeonsi

Totals:
SGPRS: 2591784 -> 2590696 (-0.04 %)
VGPRS: 1666888 -> 1666736 (-0.01 %)
Spilled SGPRs: 4131 -> 4107 (-0.58 %)
Spilled VGPRs: 38 -> 38 (0.00 %)
Private memory VGPRs: 2176 -> 2176 (0.00 %)
Scratch size: 2228 -> 2228 (0.00 %) dwords per thread
Code Size: 52715468 -> 52693584 (-0.04 %) bytes
LDS: 92 -> 92 (0.00 %) blocks
Max Waves: 479897 -> 479892 (-0.00 %)
Wait states: 0 -> 0 (0.00 %)

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4696>

ac: generate FMA for inexact instructions for radeonsi

NIR mostly does this already.

Totals:
SGPRS: 2588520 -> 2591784 (0.13 %)
VGPRS: 1666984 -> 1666888 (-0.01 %)
Spilled SGPRs: 4074 -> 4131 (1.40 %)
Spilled VGPRs: 38 -> 38 (0.00 %)
Private memory VGPRs: 2176 -> 2176 (0.00 %)
Scratch size: 2228 -> 2228 (0.00 %) dwords per thread
Code Size: 52726872 -> 52715468 (-0.02 %) bytes
LDS: 92 -> 92 (0.00 %) blocks
Max Waves: 479872 -> 479897 (0.01 %)
Wait states: 0 -> 0 (0.00 %)

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4696>

ac: update and document fast math flags used by radeonsi

This should have no effect, because we never use FP division, but
it's safer for the future.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4696>

ac: force enable -structurizecfg-skip-uniform-regions for LLVM 11

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4696>

st/mesa: Treat vertex inputs absent in inputMapping as zero in mesa_to_tgsi

After updating vertex inputs being read based on optimized NIR, they may go out
of sync with inputs in mesa IR. Which is translated to TGSI and used together
with NIR if draw doesn't have llvm.

It's much easier to treat such inputs as zero because there is no pass to
entirely get rid of them and they don't contribute to shader's output.

Fixes: d684fb37bfbc47d098158cb03c0672119a4469fe
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2815
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4705>

gitlab-ci: add lists of expected failures for RADV CI

Currently only supports PITCAIRN, POLARIS10, VEGA10 and NAVI10
with ACO only, but it's a start.

Unfortunately, we have to duplicate and we will have to try to
keep these lists up-to-date, but it's better than nothing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4689>

radv: fix robust_buffer_access if enabled via VkPhysicalDeviceFeatures2

It can be enabled via pEnabledFeatures or via vkPhysicalDeviceFeatures2.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4706>

gallivm: fix stencil border

Fixes:
dEQP-GLES31.functional.texture.border_clamp.unused_channels.depth32f_stencil8_sample_stencil
dEQP-GLES31.functional.texture.border_clamp.sampler.uint_stencil

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe: clamp color storage for integer types.

If storing to an integer for lower bit size (i.e. 16-bit uint to
10-bit uint), we need to clamp to the maximum value not truncate.

Fixes:
dEQP-VK.api.copy_and_blit.core.blit_image.all_formats.color.r16_uint.a2b10g10r10_uint_pack32.optimal_optimal_nearest

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe: enable stencil only formats. (v2)

This fixes two bugs, one in clearing and one in sign extensions
for S8 only types, and enables it for use.

These are useful for vulkan support later.

v2: move casting to same place as Z casting.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe/setup: add point size clamping

Fixes
dEQP-GLES2.functional.rasterization.limits.points
dEQP-VK.rasterization.primitive_size.points.point_size*

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe: fix d32 unorm depth conversions.

When the depth value was 1.0 and was being converted to Z32_UNORM
the conversion would scale it up to INT32_MAX + 1 which would
cause FPToSI to give incorrect results, changing it to use
FPToUI for the unsigned 32-bit case only fixes it.

Fixes:
GTF-GL45.gtf30.GL3Tests.depth_texture.depth_texture_fbo_clear

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

draw/tess: fix TES patch vertices in.

Fixes CTS KHR-GL45.tessellation_shader.single.max_patch_vertices

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe: fix ssbo alignment

KHR-GL45.geometry_shader.api.max_shader_storage_blocks

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

llvmpipe: bump max images to 16

This is needed to make some tests run, and helps for vulkan later.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

util/indirect: handle stride less than number of parameters.

It's legal to have a stride less than the num of parameters,
in this case no need to try and over map the buffer which asserts

Fixes:
GTF-GL45.gtf43.GL3Tests.multi_draw_indirect.multi_draw_indirect_stride

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

gallivm/nir: add helper invocation support

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

gallivm/nir: fix image store conversions

This fixes a few of the image store paths, to do the
correct clamping of unsigned/signed values

Fixes: KHR-GLES31.core.layout_binding.block_layout_binding_block_ComputeShader
KHR-GL45.shader_image_load_store.basic-allFormats-store
KHR-GL46.shader_image_load_store.multiple-uniforms

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4574>

tu: Don't invert point coords

We shouldn't need to invert them, and the Vulkan blob doesn't either.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4733>

ir3: Remove VARYING_SLOT_PNTC remapping hack

The st now does this for us.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4732>

st/nir: Fix assigning PointCoord location with !PIPE_CAP_TEXCOORD

This was trying to emulate the effect of mapping GL -> TGSI -> NIR,
but failed to handle VARYING_SLOT_PNTC which led to a kludgy workaround
in freedreno.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4732>

freedreno/a6xx: Implement PrimID passthrough

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4704>

tu: Implement PrimID passthrough

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4704>

ir3: Skip missing VS outputs in VS out map when linking

The hardware is capable of automatically filling in certain values in
the VPC without writing them from the last geometry stage, like
gl_PointCoord or gl_PrimitiveID when there is no GS. However, we *do*
have to enable these outputs (i.e. set the VPC_VAR_DISABLE bit to 0) as
VPC_VAR_DISABLE is really about FS inputs rather than VS outputs. To do
this, we move the computation of the enable bits to ir3_link_add(),
which is also a nice refactor anyway. In addition we detect the PrimID
case specifically so that the driver can program the location.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4704>

freedreno/a6xx: Document PrimID passthrough registers

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4704>

radv: Pass logical device to si_emit_graphics

We'll need this in order to retrieve the va of a bo for a future ext.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4728>

freedreno/ir3: Print @tex write mask using 0x%x

That way we can parse it again with the assembler.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4741>

freedreno/ir3: Reset lex line number when we start parsing

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4741>

freedreno/ir3: Parse, but ignore @in, @out and @tex headers

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4741>

freedreno/ir3: Move ir3 assembler to backend compiler

For easier reuse.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4741>

freedreno/computerator: Decouple ir3 assembler

Specifically, don't include ir3_asm.h in the parser as that's
computerator specific.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4741>

Revert "meson,ci: Disable sparse_array tests on windows"

The Wine version in the build image has been upgraded.

This reverts commit 6be65b077743fc80efe061b1e05cb13b2ff1a6b1.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4678>

gitlab-ci: install winehq-stable to get 5.0 instead of 4.0

Additionally, purge the winehq-stable package and its dependencies to
avoid crashing when building for s390x.

v2:
  - Remove winehq-stable and dependencies for s390x.

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2657
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Daniel Stone <daniels@collabora.com> [v1]
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4678>

etnaviv: Fix depth stencil ops on GC880/GC2000

This patch fixes depth stencil ops on MX6S GC880 and MX6Q GC2000.
The following dEQPs now pass:
  dEQP-GLES2.functional.fragment_ops.depth_stencil.stencil_depth_funcs.*
  dEQP-GLES2.functional.fragment_ops.depth_stencil.stencil_ops.*
which is roughly 600 fixed dEQP tests.

The problem is that if the front-facing stencil has a value mask 0x00 and
the back-facing stencil has some non-zero value mask, then the stencil part
of the depth stencil buffer is written with 0x00 unconditionally. The blob
replicates the value mask of the back-facing stencil to the value mask of
the front-facing stencil to achieve correct rendering, replicate the same
behavior here.

Signed-off-by: Marek Vasut <marex@denx.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4275>

radv/aco: enable 8/16-bit storage and int8/int16 on GFX8+

With this, Doom Eternal should now run with ACO on GFX8+.

The generated 8/16-bit storage code is okay but the generated int8/int16
code is currently pretty bad but it works and apparently Doom Eternal
doesn't actually use it (even though it requires it).

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4707>

aco: lower 8/16-bit integer arithmetic

dEQP-VK.spirv_assembly.type.* passes with the features and extensions
enabled.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4707>

aco: improve sub-dword emit_split_vector() with sgprs

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: clobber scc in s_bfe_u32 in get_alu_src()

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: handle undef p_create_vector operands in the optimizer

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: vectorize global loads/stores

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: allow 8/16-bit shared loads

These should work now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: add and use get_buffer_store_op() helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor visit_store_scratch() to use new helpers

Should support 8/16-bit stores now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor visit_store_global() to use new helpers

Should support 8/16-bit stores now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor visit_store_ssbo() to use new helpers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor store_vmem_mubuf() to use new helpers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor store_lds() to use new helpers

It should also work correctly for 8/16-bit stores

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: add helpers for splitting stores

split_store_data() splits a vector and p_as_uniforms it if needed.

scan_write_mask()/advance_write_mask() are similar to
u_bit_scan_consecutive_range(), but makes it easier to only clear part of
the range and will also give ranges for zero'd bits.

split_buffer_store() is a helper for splitting VMEM/SMEM stores.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: use emit_load helper for VMEM/SMEM loads

Also implements 8/16-bit loads for scratch/global.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: refactor load_lds to use new helpers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: add emit_load helper

This helper is used for recombining split loads, passing the result to
p_as_uniform, aligning the offset down and shifting it right if needed and
handling large constant offsets.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: add and use RegClass::get() helper

Eventually, we'll probably want to replace the current
RegClass(type, size) constructor with this.

This has a functional change in that get_reg_class() now creates v1/v2
instead of v4b/v8b.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: be more careful about using SMEM for load_global

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

radv: allocate larger shader memory slabs if needed

Fixes dEQP-VK.ssbo.phys.layout.random.16bit.scalar.13 hang with ACO
(features needed for the test are implemented in a later commit)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

radv: align buffer descriptor sizes to dword

This is needed to prevent bounds checking issues when load 8/16-bit values
with dword loads.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>

aco: Move s_setprio to correct place after the gs_alloc_req.

Previously the setprio was inside the branch, so it would only reset
the priority on the first wave, but not the others.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Use 24-bit multiplication for NGG wave id and thread id.

Both of them should always fit 24 bits anyway.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Use 24-bit multiplication in TCS I/O

The TCS inputs and outputs must always fit into the LDS,
which implies that their addresses also always fit 24 bits.

On AMD GPUs, 24-bit multiplication is much faster than 32-bit
multiplication, so we can take the opportunity to use that
for TCS I/O instead.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Const correctness for aco_print_ir.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Const correctness for get_barrier_interaction.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Abort when RA can't find a register.

Previously, it was just unreachable, which means it will generate
invalid shaders when it encounters a situation when it can't allocate
registers for eg. a large load.

This commit makes it slightly easier to notice such problems without
triggering a GPU hang.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Increase barrier_count to 7 to include barrier_barrier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

aco: Only store TCS outputs to VMEM when they are read by TES.

Totals from affected shaders (GFX10):
Code Size: 10832 -> 10736 (-0.89 %) bytes

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

radv: Add inputs read by TES to radv_shader_info.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>

turnip: add adreno 650

Tile alignment is 96, with gmem alignment of 0x6000

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4608>

turnip: use RESOLVE_TS event

This is required on a650 to flush the GMEM store.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4608>

turnip: remove unused RB_UNKNOWN_8E04_blit

New blit code doesn't change this value, and different values seem to be
related to the driver version and not the GPU version.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4608>

zink: set UBO alignments in nir_intrinsic_load_uniform lowering

resolves this error
  error: nir_intrinsic_align_offset(instr) < nir_intrinsic_align_mul(instr) (../src/compiler/nir/nir_validate.c:582)
in ext_packed_depth_stencil-readdrawpixels piglit test

port of f5b14d983e5afa1b8f75e6f3692830a1ee46d1df

Fixes: fb64954d9dd ("nir: Validate that memory load/store ops work on whole bytes")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4711>

freedreno: allow FMT6_8_UNORM as a UBWC format

FMT6_8_UNORM is necessary for NV12 textures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4722>

spirv: Rewrite CFG construction

This commit completely rewrites the way we extract a structured CFG from
SPIR-V.  The new approach is different in a few ways:

 1. It does a breadth-first search instead of depth-first.  This means
    that we've visited the merge node for a construct before we visit
    any of the nodes inside the construct.  This makes it easier to
    validate things like loop and switch nesting.

 2. We record more information in the CFG.  Earlier commits added a
    parent pointer to vtn_cf_node but we now record all of the merge and
    other special blocks for each CFG node.  This lets us validate
    things more precisely.

 3. It makes heavy use of merge blocks for walking the CFG.  Previously,
    we sort of used them as hints for trying to guess the CFG structure
    but things got dicey whenever a merge was missing.  We had some
    heuristics for how to handle short-circuiting if statements but it
    was a bunch of special cases.

    Now, we make them a fundamental part of walking the CFG.  When we
    encounter a control-flow construct, we add the body components of
    the construct to the BFS work list and then jump to the merge block
    if one exists to continue scanning the current CFG nesting level.
    If no merge block exists, we assume that means that control-flow
    never re-converges in a normal way and that the only way to get back
    to normality is with a direct jump such as a loop break or continue.
    This should make things far more robust when trying to deal with the
    more creative placement (or lack thereof) of merge instructions.

Reviewed-by: Alan Baker <alanbaker@google.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3820>
Closes: #2760
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4446>

anv: Add support for HiZ+CCS

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

intel/isl: Refactor isl_surf_get_ccs_surf

This refactor breaks out a new isl_surf_supports_ccs function which does
most of the validity checking.  The isl_surf_get_ccs_surf function calls
this function and then dives right into constructing the CCS aux_surf.

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

intel/isl: Delete a misleading comment

Untyped messages are only use on Gen9+ for UBOs and SSBOs.  They will
never be used on anything using an isl_surf.

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv/cmd_buffer: Move anv_image_init_aux_tt higher

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Simplify a case in layout_to_aux_usage

If it's depth, the only possible value of planes[plane].aux_usage is
ISL_AUX_USAGE_HIZ at least right now.

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

intel/blorp: Allow more HiZ usages in hiz_clear_depth_stencil

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Generalize some aux usage checks

For the checks dealing with fast-clear values, we change them to check
for the depth aspect because the distinction there really is between
color and depth more than between HiZ and CCS.

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv/blorp: Do less hard-coding of aux usages

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Rework anv_layout_to_aux_state

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Be more conservative about image view usage

We were ORing together the image and stencil usage rather than actually
following the formula in the spec.  This can lead to assertions in other
parts of the driver if we're not careful.

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Move vk_image_layout_is_read_only higher

While we're at it, we drop some _KHR suffixes

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Add a vk_image_layout_to_usage_flags helper

Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4100>

anv: Enable HiZ on multi-layer depth buffers.

Improves The Witcher 3 fps by 2-10% on ICL (depending on the configs and
system).

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4661>

etnaviv: support for using generic blit path

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1641>

etnaviv: call util_blitter_save_fragment_constant_buffer_slot(..)

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1641>

etnaviv: drop default state for FE_HALTI5_ID_CONFIG

It gets emitted when needed - see emit_halti5_only_state(..).

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4729>

docs/features: mark GL_ARB_texture_filter_anisotropic as done for etnaviv

Needs GPUs with HALT0.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4725>

freedreno/ir3: fix incorrect conversion folding

Fixes dEQP-VK.glsl.builtin.function.pack_unpack.unpackhalf2x16_compute

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4708>

freedreno/ir3: set even bit for f2f16_rtne

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4708>

freedreno/ir3: fix 16-bit ssbo access

Update cat6 instruction type, and shift 1 in lower_offset_for_ssbo.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4708>

aco: fix outdated label_vec from p_create_vector labelling

Fixes random dEQP-VK.transform_feedback.fuzz.* crashes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 2dc550202e82c5da198ad0a416a5d24dd89addd8
    ('aco: copy-propagate p_create_vector copies of vectors')

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4730>