freedreno: Add some missing a6xx address declarations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3455>

relnotes: Add GL_INTEL_shader_integer_functions2 and VK_INTEL_shader_integer_functions2

Suggested-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>

lima: use imul for calculations with intrinsic src

It's source is supposed to be int, so we have to use integer
multiplication otherwise we'll get undefined result.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3529>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3529>

nir: don't emit ishl in _nir_mul_imm() if backend doesn't support bitops

Otherwise we'll have to lower it later.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3529>

pan/decode: Rotate trace files

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

pan/decode: Dump to a file

The file name is taken from the environment variable
PANDECODE_DUMP_FILE, defaulting to pandecode.dump if it is not set.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

pan/decode: Support dumping to a file

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

pan/bifrost: Support disassembling to a file

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

pan/midgard: Support disassembling to a file

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

pan/midgard: Fix a memory leak in the disassembler

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3525>

turnip: Fix execution of secondary cmd bufs with nothing in primary.

We want to finish off cmd emission in the primary CS and add its entry to
the IB, but regardless of whether there had been anything in the primary
CS to emit, we still need a reserved CS entry for the loop below.

Fixes crashes in dEQP-VK.binding_model.shader_access.secondary_cmd_buf.*
and many more in dEQP-VK.renderpass*

Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3524>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3524>

panfrost: Drop mysterious zero=0xFFFF field

It doesn't seem to affect any results and it's not at all clear if/why
the blob sometimes(?) sets it? So let's clean this up since this
solution isn't correct anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3513>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3513>

pan/midgard: Fix bundle dynarray leak

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3496>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3496>

radeonsi: separate LLVM compilation from non-LLVM code

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: change prototypes of si_is_multi_part_shader & si_is_merged_shader

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: make si_compile_shader return bool

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: make si_compile_llvm return bool

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: move more LLVM functions into si_shader_llvm.c

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: fold si_shader_context_set_ir into si_build_main_function

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: move si_nir_build_llvm into si_shader_llvm.c

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: minor cleanup in si_shader_internal.h

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: move si_shader_llvm_build.c content into si_shader_llvm.c

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: move VS shader code into si_shader_llvm_vs.c

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: move non-LLVM code out of si_shader_llvm.c

There was also some redundant code in si_shader_nir.c

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

radeonsi: use ctx->ac. for types and integer constants

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3421>

turnip: simplify tu_physical_device_get_format_properties

Fixes the "bad VkImageTiling" error when tiling is
VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3485>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3485>

vulkan/wsi: remove unused image_get_modifier

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3485>

turnip: set linear tiling for scanout images

Fixes: 210e6887 "vulkan/wsi: Use the interface from the real modifiers extension"
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3485>

turnip: hook up GetImageDrmFormatModifierPropertiesEXT

Fixes: 210e6887 "vulkan/wsi: Use the interface from the real modifiers extension"
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3485>

freedreno/drm: Don't miscalculate timeout

The current code overflows (s * 1000000000) for s >= 5 but that is
e.g. used in msm_bo_cpu_prep.

Signed-off-by: Guido Günther <agx@sigxcpu.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3514>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3514>

turnip: Add support for fine derivatives.

This does appear to be the required instruction sequence (dsxpp_1 dst src;
dsxpp_1.p dst src) as dropping either instruction fails the testsuite.

Fixes dEQP-VK.glsl.derivate.*

Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3494>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3494>

freedreno/ir3: Plumb the ir3_shader_variant into legalize.

legalize is computing a lot of state that goes in the variant, let's just
store it directly instead of passing pointers around.  This leaves
max_bary in place, which is doing some surprising work (overwriting the
original total_in in some cases).

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3494>

util/hash_table: update users to use new optimal integer hash functions

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>

util/hash_table: added hash functions for integer types

A few hash_table users roll their own integer hash functions which
call _mesa_hash_data to perform the hashing which ultimately calls
into XXH32 with a dynamic key length. When using small keys with a
constant size the hash rate can be greatly improved by inlining
XXH32 and providing it a constant key length, see:
https://fastcompression.blogspot.com/2018/03/xxhash-for-small-keys-impressive-power.html

Additionally, this patch removes calls to _mesa_key_hash_string and
makes them instead call _mesa_has_string directly, matching the new
integer hash functions.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>

util/hash_table: replace _mesa_hash_data's fnv1a hash function with xxhash

For most key sizes, xxhash outperforms fnv1a's hash rate substantially (bug
2153). In particular, the V3D driver hashes multiple ~200 byte keys as part
of the shader cache lookup which can easily eat up 10-20% of the runtime on
the Raspberry Pi. Swapping over to xxhash drops this to ~1% of the runtime.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>

util: move fnv1a hash implementation into its own header

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>

util: import xxhash

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3475>

winsys/amdgpu: Close KMS handles for other DRM file descriptions

When a BO or amdgpu_screen_winsys is destroyed.

Should fix leaking such BOs in other DRM file descriptions.

v2:
* Pass the correct file descriptor to drmIoctl (Pierre-Eric
  Pelloux-Prayer)
* Use _mesa_hash_table_remove
v3:
* Close handles in amdgpu_winsys_unref as well
v4:
* Adapt to amdgpu_winsys::sws_list_lock.

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2270
Fixes: 11a3679e3aba "winsys/amdgpu: Make KMS handles valid for original
                     DRM file descriptor"

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

winsys/amdgpu: Re-use amdgpu_screen_winsys when possible

Namely, if os_same_file_description determined that the DRM file
descriptor references the same file description.

v2:
* Adapt to amdgpu_winsys::sws_list_lock.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

util: Add os_same_file_description helper

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

winsys/amdgpu: Only re-export KMS handles for different DRM FDs

When the amdgpu_screen_winsys uses the same FD as the amdgpu_winsys
(which is always the case for the first amdgpu_screen_winsys), we can
just use bo->u.real.kms_handle.

v2:
* Also only create the kms_handles hash table if the
  amdgpu_screen_winsys fd is different from the amdgpu_winsys one.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

winsys/amdgpu: Keep track of retrieved KMS handles using hash tables

The assumption being that KMS handles are only retrieved for relatively
few BOs, so hash tables should be efficient both in terms of performance
and memory consumption.

We use the address of struct amdgpu_winsys_bo as the key and its
kms_handle field (the KMS handle valid for the DRM file descriptor
passed to amdgpu_device_initialize) as the hash value.

v2:
* Add comment above amdgpu_screen_winsys::kms_handles (Pierre-Eric
  Pelloux-Prayer)
v3:
* Protect kms_handles hash table with amdgpu_winsys::sws_list_lock
  mutex.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

winsys/amdgpu: Keep a list of amdgpu_screen_winsyses in amdgpu_winsys

v2:
* Add dedicated mutex for the list.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3202>

aco: implement nir_op_f2i64/nir_op_f2u64 on GFX6

V_TRUNC_F64 and V_FLOOR_F64 needs to be lowered on GFX6.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement 64-bit nir_op_ffloor on GFX6

GFX6 doesn't have V_FLOOR_F64, it needs to be lowered. Loosely based
on the AMDGPU LLVM backend.

Introduce a new function because it will be useful for some other
64-bit operations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement 64-bit nir_op_fround_even on GFX6

GFX6 doesn't have V_RNDNE_F64, it needs to be lowered. Loosely based
on the AMDGPU LLVM backend.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement 64-bit nir_op_fceil on GFX6

GFX6 doesn't have V_CEIL_F64, it needs to be lowered. Loosely based
on the AMDGPU LLVM backend.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement 64-bit nir_op_ftrunc on GFX6

GFX6 doesn't have V_TRUNC_F64, it needs to be lowered. Loosely based
on the AMDGPU LLVM backend.

Introduce a new function because it will be useful for some other
64-bit operations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement nir_intrinsic_global_atomic_* on GFX6

GFX6 doesn't have FLAT instructions, use MUBUF instructions instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement nir_intrinsic_load_global on GFX6

GFX6 doesn't have FLAT instructions, use MUBUF instructions instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: implement nir_intrinsic_store_global on GFX6

GFX6 doesn't have FLAT instructions, use MUBUF instructions instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

aco: fix wrong IR in nir_intrinsic_load_barycentric_at_sample

Only GFX6 was affected, my mistake. The total number of SGPR operands
should be 4 when we want to create a vec4.

Fixes: dbdf3b3ef97 ("aco: implement nir_intrinsic_load_barycentric_at_sample on GFX6")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3477>

anv/iris: warn gen12 3DSTATE_HS restriction

This should never happen but better off documenting it in case someone
plays with max threads numbers.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3489>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3489>

gallium/swr: add option for static link

Set swr-shared to 'false' to link SWR statically into Mesa.
Only one swr arch can be specified if swr-shared is set to false.

Reviewed-by: Jan Zielinski <jan.zielinski@intel.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3510>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3510>

aco: fix printing assembly with CLRXdisasm on GFX6

We thought that CLRXdisasm allowed gfx600 as well as gfx700 but
it actually doesn't. Use the family for GFX6 chips instead.

Fixes: 0099f85232b ("aco: print assembly with CLRXdisasm for GFX6-GFX7 if found on the system")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3531>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3531>

clover/meson: Define OpenCL header macros

Rather than defining the macros any time right before including an
OpenCL header, set Meson to define them for the whole clover project.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Francisco Jerez <currojerez@riseup.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3137>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3137>

clover: Use the dispatch table type from the OpenCL headers

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2243
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3137>

include/CL: Update OpenCL headers to latest

This latest update contains a new header that defines the dispatch table
structure in order to avoid OpenCL implementations having to define it
themselves.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3137>

radv: advertise VK_AMD_shader_fragment_mask

Only for GFX8+ because it's untested on older generations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

aco: add support for nir_texop_fragment_{mask}_fetch

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

ac/nir: add support for nir_texop_fragment_{mask}_fetch

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

nir/lower_input_attachments: lower nir_texop_fragment_{mask}_fetch

These instructions are allowed to fetch from multisampled
subpass input attachments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

spirv: add support for SpvOpFragment{Mask}FetchAMD operations

nir_tex_src_ms_index is re-used for the fragment index with
nir_texop_fragment_fetch to avoid introducing a new texture source type.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

nir: add two new texture ops for multisample fragment color/mask fetches

This introduces:
   - nir_texop_fragment_mask_fetch (fetch a fragment mask from a
     compressed multisampled color surface)
   - nir_texop_fragment_fetch (fetch a color fragment for a
     particular sample at corresponding fragment mask index).

These two texture operations are necessary for implementing
SPV_AMD_shader_fragment_mask.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

spirv: add SpvCapabilityFragmentMaskAMD

This new capability is for SPV_AMD_shader_fragment_mask.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3304>

radv: handle missing implicit subpass dependencies

When a subpass doesn't declare an explicit dependency from/to
VK_SUBPASS_EXTERNAL, Vulkan says there is an implicit dependency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3330>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3330>

radv: add explicit external subpass dependencies to meta operations

No functional changes because a subpass dependency with dstStageMask
set to VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT is a no-op.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3330>

gallivm: fix find lsb

the GLSL return value is different than the llvm intrinsic.

Fixes arb gpu shader5 tests

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3528>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3528>

galllivm: fix gather offset casting

cast texture offsets to 32-bit integers

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3528>

llvmpipe: fix some integer instruction lowering.

We want to lower to shifts for bitfields, and lower ifind_msb.

Fixes a bunch of gpu shader5 tests.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3528>

gallivm: fix gather component handling.

Fixes the extended gather test for gpu shader5

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3528>

turnip: Add support for uniform texel buffers.

Pretty straightforward: Port texture descriptor code from freedreno, fill
in alignment limits from closed vk, and tu_cmd_buffer.c was already
uploading the texture descriptor.

This doesn't implement storage texel buffers (required in the compute
pipeline) yet, since those will need an IBO descriptor for the store path.
Still, making the load path be connected to the texture descriptor won't
hurt.

Part of #2237

Fixes dEQP-VK.binding_model.shader_access.primary_cmd_buf.uniform_texel_buffer.*

Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3522>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3522>

intel: Fix aux map alignments on 32-bit builds.

ALIGN() brilliantly uses uintptr_t, making it unsafe for use with 64-bit
GPU addresses in 32-bit builds of the driver.  Use align64() instead,
which uses uint64_t.

Fixes assertion failures when running any 32-bit program on Tigerlake.

Fixes: 2e6a7ced4db ("iris/gen12: Write GFX_AUX_TABLE base address register")
Fixes: 0d0290bb3f7 ("intel/common: Add surface to aux map translation table support")
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3507>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3507>

util: Remove tmp argument from BITSET_FOREACH_SET macro

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3499>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3499>

util: Explain BITSET_FOREACH_SET params

__size, in particular, makes this macro rather confusing to understand
how to use. Hopefully this comment saves future users the headache.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3499>

lima: implement invalidate_resource()

We don't need to resolve invalidated resources, so it should
improve performance for applications that are doing this hint.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3476>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3476>

glsl_to_nir: update interface type properly

Since 76ba225184ed the member variable types were being redefined
but we assigned the old interface type to the variable.

In a following patch series we will use the types to check if we
are dealing with an interface instance when apply GLSL linking
rules.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3468>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3468>

glsl: count uniform components and storage better in nir linking

This helps avoid incorrect validation error when linking glsl
shaders and avoids assigning uniform storage slots that will
never be used.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3468>

glsl: fix check for matrices in blocks when using nir uniform linker

We need to stripe any arrays before checking the type. Here we
just use the uniform type which has already be stripped.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3468>

glsl: remove bogus assert in nir uniform linking

I'm not sure why this was first added but it causes an assert
on any uniform matrix.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3468>

nir/algebraic: Optimize some 64-bit integer comparisons involving zero

I noticed that we can do better for these kinds of comparisons while
working on the lowering for iadd_sat@64 and isub_sat@64.  This
eliminated 11 instruction from the fs-addSaturate-int64.shader_test.

My hope is that this will improve the run-time of int64 tests on Ice
Lake.  I have no data to support or refute this.

Unsurprisingly, no changes on shader-db.

v2: Condition the min and max patterns with nir_lower_minmax64.
Suggested by Caio.  Very long discussion in the MR. :)

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

anv: Enable SPV_INTEL_shader_integer_functions2 and VK_INTEL_shader_integer_functions2

Currently only implemented in the scalar backend, so only enable for
Gen8+.  If support for the other opcodes is added to the vec4 backend,
Gen7 could be supported.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

iris: Enable INTEL_shader_integer_functions2

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

gallium: Add a cap bit for integer multiplication between 32-bit and 16-bit

Driver supports integer multiplication between a 32-bit integer and a
16-bit integer.  If the second operand is 32-bits, the upper 16-bits are
ignored, and the low 16-bits are possibly sign extended as necessary.

Iris will eventually enable this.  Not sure about other drivers.

v2: Add default value to u_screen.c.  Suggested by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

gallium: Add a cap bit for OpenCL-style extended integer functions

Iris will eventually enable this.  Looking at the header files, it looks
like Midgard could also enable it.  Basically, any GPU that fully
supports OpenCL can.

v2: Add default value to u_screen.c.  Suggested by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

i965: Enable INTEL_shader_integer_functions2 on Gen8+

v2: Use new lower_hadd64 and lower_usub_sat64 flags.

v3: Enable SPIR-V capability.

v4: Move lowering options to COMMON_SCALAR_OPTIONS.  Suggested by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

spirv: Add support for IntegerFunctions2INTEL capability

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

spirv: Silence a bunch of unused parameter warnings

The change to get_uniform_nir_atomic_op make it look like the other
get_*_nir_atomic_op functions.  The rest just add UNUSED or ASSERTED
to parameters required for some of the interfaces.

src/compiler/spirv/spirv_to_nir.c: In function ‘struct_member_decoration_cb’:
src/compiler/spirv/spirv_to_nir.c:673:47: warning: unused parameter ‘val’ [-Wunused-parameter]
                             struct vtn_value *val, int member,
                                               ^~~
src/compiler/spirv/spirv_to_nir.c: In function ‘struct_member_matrix_stride_cb’:
src/compiler/spirv/spirv_to_nir.c:778:50: warning: unused parameter ‘val’ [-Wunused-parameter]
                                struct vtn_value *val, int member,
                                                  ^~~
src/compiler/spirv/spirv_to_nir.c: In function ‘type_decoration_cb’:
src/compiler/spirv/spirv_to_nir.c:805:61: warning: unused parameter ‘ctx’ [-Wunused-parameter]
                     const struct vtn_decoration *dec, void *ctx)
                                                             ^~~
src/compiler/spirv/spirv_to_nir.c: In function ‘spec_constant_decoration_cb’:
src/compiler/spirv/spirv_to_nir.c:1359:70: warning: unused parameter ‘v’ [-Wunused-parameter]
 spec_constant_decoration_cb(struct vtn_builder *b, struct vtn_value *v,
                                                                      ^
src/compiler/spirv/spirv_to_nir.c: In function ‘handle_workgroup_size_decoration_cb’:
src/compiler/spirv/spirv_to_nir.c:1407:43: warning: unused parameter ‘data’ [-Wunused-parameter]
                                     void *data)
                                           ^~~~
src/compiler/spirv/spirv_to_nir.c: In function ‘vtn_handle_function_call’:
src/compiler/spirv/spirv_to_nir.c:1806:55: warning: unused parameter ‘opcode’ [-Wunused-parameter]
 vtn_handle_function_call(struct vtn_builder *b, SpvOp opcode,
                                                       ^~~~~~
src/compiler/spirv/spirv_to_nir.c:1807:54: warning: unused parameter ‘count’ [-Wunused-parameter]
                          const uint32_t *w, unsigned count)
                                                      ^~~~~
src/compiler/spirv/spirv_to_nir.c: In function ‘get_uniform_nir_atomic_op’:
src/compiler/spirv/spirv_to_nir.c:2548:47: warning: unused parameter ‘b’ [-Wunused-parameter]
 get_uniform_nir_atomic_op(struct vtn_builder *b, SpvOp opcode)
                                               ^
src/compiler/spirv/spirv_to_nir.c: In function ‘vtn_handle_atomics’:
src/compiler/spirv/spirv_to_nir.c:2633:48: warning: unused parameter ‘count’ [-Wunused-parameter]
                    const uint32_t *w, unsigned count)
                                                ^~~~~
src/compiler/spirv/spirv_to_nir.c: In function ‘vtn_handle_barrier’:
src/compiler/spirv/spirv_to_nir.c:3197:48: warning: unused parameter ‘count’ [-Wunused-parameter]
                    const uint32_t *w, unsigned count)
                                                ^~~~~
src/compiler/spirv/spirv_to_nir.c: In function ‘vtn_handle_execution_mode’:
src/compiler/spirv/spirv_to_nir.c:3618:68: warning: unused parameter ‘data’ [-Wunused-parameter]
                           const struct vtn_decoration *mode, void *data)
                                                                    ^~~~

Acked-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/spirv: Translate SPIR-V to NIR for new INTEL_shader_integer_functions2 opcodes

v2: Rebase on 272e927d0e9 ("nir/spirv: initial handling of OpenCL.std
extension opcodes")

v3: Add missing SpvOpUCountTrailingZerosINTEL case to switch in
vtn_handle_body_instruction. Remove stray semicolon in
vtn_nir_alu_op_for_spirv_opcode. Use umin instead of umax for
SpvOpUCountTrailingZerosINTEL "lowering" in vtn_handle_alu.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

intel/fs: Implement support for NIR opcodes for INTEL_shader_integer_functions2

v2: Remove smashing type to D for nir_op_irhadd.  Caio noticed it was
odd, and removing it fixes an assertion failure in the crucible
func.shader.averageRounded.int64_t test (because the source should be
W).

v3: Emit BRW_OPCODE_MUL directly for nir_op_umul_32x16 and
nir_op_imul_32x16.  Suggested by Curro.

v4: Smash types of MUL instruction generated for nir_op_umul_32x16 and
nir_op_imul_32x16.  With this change, I get the same assembly now as I
did with v2.

v5: Remove support for pre-Gen7.  The integer multiply path was
incorrect, and, since the extension isn't enabled pre-Gen7, there's no
way to test it.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

intel/fs: Add SHADER_OPCODE_[IU]SUB_SAT pseudo-ops

v2: Add a big comment explaining the [IU]SUB_SAT lowering.  Suggested by
Caio.

v3: Use get_fpu_lowered_simd_width in get_lowered_simd_width.  Suggested
by Ken on IRC.

v4: Fix a typo in a comment.  Noticed by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

intel/fs: Don't lower integer multiplies that don't need lowering

v2: Move the check to fs_visitor::lower_integer_multiplication.
Previously the cases where lowering was skipped, the original
instruction was removed by fs_visitor::lower_integer_multiplication.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

compiler: Translate GLSL IR to NIR for new INTEL_shader_integer_functions2 expressions

v2: Rebase on 272e927d0e9 ("nir/spirv: initial handling of OpenCL.std
extension opcodes")

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/algebraic: Add lowering for 64-bit iadd_sat and isub_sat

v2: Rearranged and expand the comment about the optimizations applied to
the lowering.  Suggested by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/algebraic: Add lowering for 64-bit uadd_sat

Fixes piglit fs-addsaturate-uint64 and vs-addsaturate-uint64 on Ice
Lake.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/algebraic: Add lowering for 64-bit usub_sat

v2: Rebase on 272e927d0e9 ("nir/spirv: initial handling of OpenCL.std
extension opcodes")

v3: Add a new lower_usub_sat64 flag that only applies to the 64-bit
version of the nir_op_usub_sat instruction.

v4: Also enable the lowering when nir_lower_iadd64 is set.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> [v3]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/algebraic: Add lowering for 64-bit hadd and rhadd

v2: Rebase on 272e927d0e9 ("nir/spirv: initial handling of OpenCL.std
extension opcodes")

v3: Add a new lower_hadd64 flag that only applies to the 64-bit versions
of the instructions.

v4: Also enable the lowering when nir_lower_iadd64 is set.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> [v3]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir/algebraic: Add lowering for uabs_usub and uabs_isub

v2: Remove some rebase failures noticed by Caio.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

nir: Add new instructions for INTEL_shader_integer_functions2

uctz isn't added because it will implemented in the GLSL path and the
SPIR-V path using other pre-existing instructions.

v2: Avoid signed integer overflow for uabs_isub(0, INT_MIN).  Noticed by
Caio.

v3: Alternate fix for signed integer overflow for abs_sub(0, INT_MIN).
I tried the previous methon in a small test program with -ftrapv, and it
failed.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> [v1]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>

glsl: Add built-in functions for INTEL_shader_integer_functions2

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/767>