llvmpipe/cs: add shader caching

As for fragment shader, skip compilation step if we have the shaders

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

llvmpipe/fs: add caching support

Serialize and check if the object is in the cache, it there is
a cached object skip compilation code once we've constructed
the function interface.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: don't cache shaders that use fetch functions.

This needs to be reworked, but it's a bit messy as we have to store
all the fetch pointers to be added as globals later once gallivm
has been initialised further. For now just refuse to cache shaders
that hit these paths (mainly ETC1 and BPTC).

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

llvmpipe: add infrastructure for disk cache support

This hooks up the gallium API and adds the APIs needed
for shader stages to search and add things to the cache.

It also adds cache stats debug printing.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: add cache interface to mcjit

MCJIT uses an ObjectCache object to implement the cache,
this creates and instances of it and adds it to the MCJIT
instances, it stores the cached object for later use by
the outer layers.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: skip operations if we have a cached object.

If the object is loaded from the cache, a bunch of gallivm/llvm
interactions can be skipped.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: add support for a cache object

This plumbs the cache object into the gallivm API, nothing uses
it yet.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: rework debug printf hook to use global mapping.

Cached shaders require relinking, so hardcoding the pointer
can't work. This switches out the printf code to use new
proper API.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

gallivm: rework coroutine malloc/free callouts.

When using cached shaders we have to relink the shader with
external symbols when it's loaded. However the way gallivm does
function calls now hardcodes the function pointer into the shader.

LLVM had a mechanism for doing this properly using global mappings,
this switches the coroutine alloc/free code to use a global mapping.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

llvmpipe/draw: drop variant number from function names.

When we use an object cache for the MCJIT we can have identical
cache entries from the same shader variant in different shaders,
but the JIT objcache uses the function name to relink things,
so it has to be consistent. Just drop the variants from the
function names.

Note the modules still have the variant info.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

llvmpipe/cs: overhaul cs variant key state.

This just realigns it with the fs state, and fixes some issues
where shaders weren't getting cached correctly.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

util/disk_cache: add fallback for disk_cache_get_function_identifier

Otherwise drivers need to have a ifdef on windows, easier to fix
here hopefully.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5049>

ci: fix possible spuriously run of jobs

Need to list arm_test-base here as well, or jobs using this
template may spuriously run if the arm_test-base job fails or
is cancelled.

Suggested-by: Michel Dänzer <mdaenzer@redhat.com>
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5405>

ac/surface: cache DCC retile maps (v2)

This reduces overhead when resizing windows or when allocating
similar image sizes over and over again.

v2: optimize the memory footprint of the cache

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5398>

ac/surface: add a wrapper structure to hold ADDR_HANDLE

and more things in the future.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5398>

amd/addrlib: remove unused members of ADDR2_COMPUTE_DCC_ADDRFROMCOORD_INPUT

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5398>

amd/addrlib: don't recompute DCC info for every ComputeDccAddrFromCoord call

This decreases the DCC retile map overhead from 23% to 18%.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5398>

ac/surface: don't recompute the DCC retile map for imported textures

The retile map is not used in this case, and the retile map computation
takes 39% of CPU time when resizing a window.

This brings it down to 23%.

The dcc_retile_use_uint16 setting has to be derived from DCC sizes.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5398>

aco: fix moving sub-dword values out of a register for a fixed definition

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: use Info::definition_size instead of definition's regclass

16-bit abs/neg creates v_xor_b32/v_and_b32 with v2b definitions. These
instructions never do partial writes without SDWA.

No shader-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: add Info::{operand_size,definition_size}

No shader-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: prefer 4-byte aligned definitions

shader-db (Navi, fp16 enabled):
Totals from 42 (0.03% of 127638) affected shaders:
CodeSize: 811984 -> 806224 (-0.71%)
Instrs: 155733 -> 155939 (+0.13%); split: -0.04%, +0.18%
Cycles: 1982568 -> 1984400 (+0.09%); split: -0.06%, +0.15%
VMEM: 7187 -> 7121 (-0.92%); split: +0.86%, -1.78%
SMEM: 1770 -> 1769 (-0.06%)
VClause: 1475 -> 1476 (+0.07%)
Copies: 12406 -> 12606 (+1.61%); split: -0.46%, +2.07%
Branches: 5901 -> 5900 (-0.02%); split: -0.25%, +0.24%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: allow reading/writing upper halves/bytes when possible

Use SDWA, opsel or a different opcode to achieve this.

shader-db (Navi, fp16 enabled):
Totals from 42 (0.03% of 127638) affected shaders:
VGPRs: 3424 -> 3416 (-0.23%)
CodeSize: 811124 -> 811984 (+0.11%); split: -0.12%, +0.23%
Instrs: 156638 -> 155733 (-0.58%)
Cycles: 1994180 -> 1982568 (-0.58%); split: -0.59%, +0.00%
VMEM: 7019 -> 7187 (+2.39%); split: +3.45%, -1.05%
SMEM: 1771 -> 1770 (-0.06%); split: +0.06%, -0.11%
VClause: 1477 -> 1475 (-0.14%)
Copies: 13216 -> 12406 (-6.13%)
Branches: 5942 -> 5901 (-0.69%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: p_extract_vector in 64-bit u2f16/i2f16

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

aco: validate instructions reading/writing upper halves/bytes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>

panfrost: Add writes_stencil to the EARLY_Z disable list

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Print writeout sources in mir_print_instruction

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Add new depth store lowering

This uses the new nir_intrinsic_store_combined_output_pan intrinsic,
which can write depth, stencil and color in a single instruction. If
there are no color writes, the "depth RT" is written to.

Fixes the dEQP GLES3 depth write tests, as well as the piglit tests
fragdepth_gles2, glsl-1.10-fragdepth and when modified to not rely
on depth/stencil reload, glsl-fs-shader-stencil-export.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Add depth/stencil support to emit_fragment_store

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Move search_var to earlier in midgard_compile.c

It will be needed by the new zs lowering.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Add new depth writeout code

We schedule depth writeout to smul and stencil to vlut, so scheduling
to smul has to be disabled in these cases.

When only writing stencil, scheduling to smul is still disabled to
prevent stencil writeout from being scheduled there.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Replace writeout booleans with a single value

A single value is easier to deal with than three separate booleans.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

nir: Replace the zs_output_pan intrinsic with combined_output_pan

Depth and stencil writes are combined with color writes, so we need
this intrinsic which has sources for color, RT, depth and stencil.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Remove writeout case from bytemask_of_read_components

By setting the swizzle for the fragment color, and setting qmask to ~0
for branches, the special case for writeout branches can be removed
from mir_bytemask_of_read_components_index.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Remove old depth writeout code

We need to be able to do color writeout at the same time as depth
writeout. The old code can't do that, so needs to be removed.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Remove old zs store lowering

It is broken for when there are also color writes, and will be
replaced with a new lowering which takes that into account.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Move r1.w writeout to branch->dest

There will need to be sources for depth and stencil writeout, so
something has to be moved to the dest of the writeout branch.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

pan/mdg: Add a macro for printing instruction source information

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5065>

nir: Remove nir_intrinsic_output_u8_as_fp16_pan

Now unused in favour of nir_intrinsic_load_output, happily.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5287>

ac/surface: fix epitch when modifying surf_pitch

This is needed otherwise it can cause bad rendering of UYVY files.
The align(..., 256 / surf->bpe) constraint comes from addrlib.

Fixes: 69aadc49331 ("radeonsi: fix surf_pitch for subsampled surface")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5314>

ac/surface: set SCANOUT if surf->is_displayable

Fixes: ba10fb3f7f4 ("radeonsi: preserve the scanout flag for shared resources on gfx9 and gfx10")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5314>

zink: only report device-local memory as video-memory

While the definition of "video memory" isn't super clear, I think it's
pretty reasonable to assume host-memory isn't meant to be included. So
let's only count dedicated memory here.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3107
Reviewed-by: Witold Baryluk <witold.baryluk@gmail.com>
Tested-by: Witold Baryluk <witold.baryluk@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5409>

ac/nir: fix integer comparisons with pointers

If we get a comparison between a pointer and an integer, LLVM
complains if the operands aren't of the same type.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3085
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5397>

radeonsi/ngg: try GS multi-cycling mode if default mode failed

If gsprim_lds_size is larger than target_lds_size then gfx10_ngg_calculate_subgroup_info
will fail.

This commit adds a logic to try the multi-cycling in this case because it's
using less memory.

This fix glsl-1.50-gs-max-output when using NGG.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5401>

radeonsi: add return value to gfx10_ngg_calculate_subgroup_info

gfx10_ngg_calculate_subgroup_info uses assert to detect invalid configuration,
but if asserts are disabled it will continue its execution.

This commits adds a boolean return value to let the caller know that something
went wrong and that the results mustn't be used.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3103
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5401>

glsl: fix crash on glsl macro redefinition

In case shader contains two equal macro defines, first one with trailing spaces
and the second one without.
`#define A 1   `
`#define A 1`
The parser crashes

Fixes: 0346ad37741b11d640c1c4970b275c1f0c7f9e75 ("glsl: ignore trailing whitespace when define redefined")
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Andrii Simiklit <andrii.simiklit@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5312>

anv/allocator: Compare to start_offset in state_pool_free_no_vg

In d11e4738a86ec, we started using a start_offset to allow us to
allocate pools where the base address isn't at the start of the pool.
This is useful for binding table pools which want to be relative to
surface state base address (more or less), among other things.  However,
we had a bug where, if you have a negative offset, everything returned
to the pool would end up being returned to the "back" of the pool.  This
isn't what we want for binding tables in the softpin world.  This was
causing us to never actually re-use any binding table blocks.  How this
passed CTS, I have no idea.

Closes: #3100
Fixes: d11e4738a86ec "anv/allocator: Add a start_offset to anv_state_pool"
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5395>

panfrost: Ensure we have ro before using it

Even through the resouce requested has a BIND_SCANOUT or related tag,
this does not mean that we have a render-only driver.

This can trivially happen as one requests such resource from GBM, while
using the panfrost fd (and hence panfrost_dri.so)

Forward port of !3000

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Closes: #2664
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5410>

radv/aco: enable shaderInt8 and VK_KHR_shader_float16_int8 on GFX6-GFX7

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

radv/aco: enable shaderInt16 on GFX6-GFX7

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

radv/aco: enable 8-bit/16-bit storage on GFX6-GFX7

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: remove unnecessary split- and create_vector instructions for subdword loads

This helps GFX6/7 by removing unnecessary shuffle code.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: fix alignment of vectors with 4 elements

I think this case was just missing.

This fixes a bunch of 16-bit storage related CTS failures like
dEQP-VK.ssbo.phys.layout.single_basic_type.std430.u16vec4.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: implement 8-bit/16-bit conversions on GFX6-GFX7

Use v_bfe to implement small bitsize conversions because the
compiler probably optimizes this better.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: optimize packing of 16bit subdword registers on GFX6/7

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: skip partial copies on first iteration when lowering to hw

Helps some Detroit : Become Human shaders.

Totals from affected shaders: (VEGA)
Code Size: 47693912 -> 47670212 (-0.05 %) bytes
Instructions: 9183788 -> 9177863 (-0.06 %)
Copies: 910052 -> 904127 (-0.65 %)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: coalesce copies more aggressively when lowering to hw

Helps some Detroit : Become Human shaders.

Totals from affected shaders: (VEGA)
Code Size: 9880420 -> 9879088 (-0.01 %) bytes
Instructions: 1918553 -> 1918220 (-0.02 %)
Copies: 177783 -> 177450 (-0.19 %)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: add and use scratch SGPR to lower subdword p_create_vector on GFX6/7

This is needed to lower some corner cases correctly,
in case the same operand occurs multiple times:
e.g. v0 = p_create_vector(v0[0:8], v0[0:8], v0[0:8], v0[0:8])

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: adjust GFX6 subdword lowering workarounds for 8bit

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: Workarounds subdword lowering on GFX6/7

As there are no SDWA instructions, we need to take care not to overwrite
the upper bits of other copy_operation's operands.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: use full-register instructions to implement subdword packing on GFX6/7

On GFX6/7, there are no SDWA instructions.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: simplify statistics collection for copies

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

aco: fix register assignment for p_create_vector on GFX6/7

In case, some operand was already placed in the definition space,
it could happen that it wasn't considered for live-range splits.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5226>

zink: emit interpolation decorations for ntv outputs

this matches up with nir internal states pre/post ntv

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5384>

zink: track program usages for each shader

when shaders are created and destroyed in large numbers, the same pointers
get reused for different shaders, which can lead to bad lookups in the
program_cache hash table.

now each shader tracks its program usage to automatically remove itself from
that program in order to avoid hash collisions

fixes mesa/mesa#3053

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5315>

zink: assert that image-view format isn't undefined

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5407>

zink: emulate B8G8R8X8_SRGB with B8G8R8A8_SRGB

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5407>

turnip: Expose robustBufferAccess.

It is a required device feature, and all enabled tests in
dEQP-VK.robustness.* pass.

Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

ci: Use rsync for initial nfsroot population on cheza.

rm -rf and then copying over all the contents again is a waste of time
when we'll almost always be using the same rootfs.  Saves about 30s of job
time.

Closes: #3065
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

ci: Enable pre-merge fractional vulkan CTS runs on the turnip driver.

Test 1/50th of the CTS on a630 pre-merge, since we've got hardware that
can do it and infrastructure that should handle instability with a
less-mature driver.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

ci: Build the full VK CTS for baremetal testing.

I'm going to enable the VK CTS on cheza, so swap the deqp we have in the
container.  build-deqp-vk already included GLES deqp binaries and data,
and is a newer branch than the last opengl-es-cts tag.

This brings a few things back over from build-deqp-gl for testlog
extraction, and copyes out the GLES mustpass lists.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

ci: Disable shader cache on vulkan CI runs.

I found it to be flaky in freedreno CI, and tracked down the issue to
parallel-deqp-runner needing to manage the shader cache
(https://gitlab.freedesktop.org/mesa/parallel-deqp-runner/-/merge_requests/13).
Until we fix that in the runner, disable it.  This should matter less now
that we prebuild the SPIRV, though.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

ci: Bump up to the current version of the VK CTS.

For enabling VK CTS on freedreno, I've heard there were important
stability fixes in the CTS recently.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

turnip: Fix crashes in compute with no descriptors to load.

Found when trying to rebase cheza VK CI on top of this change.

Fixes: 334204823eee ("tu: Fix context faults loading unused descriptor sets")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>

frontends/vdpau: Default destination rect to source rect

mpv is passing in a NULL destination_video_rect, which results in a
black screen when playing videos using VDPAU in some cases.

Signed-off-by: Thong Thai <thong.thai@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5386>

radeonsi: require LLVM 11 for gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: add support for Sienna Cichlid

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

ac: align num_vgprs for gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: don't set any XNACK options on gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: set BIG_PAGE fields on gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: move L2_CACHE_CONTROL registers into si_emit_framebuffer_state

the next commit will set more fields.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: implement R9G9B9E5 render target and image store support on gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: enable larger SDMA clears and copies on gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: honor a user-specified pitch on gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

ac/surface: add displayable DCC code for gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

ac,radeonsi: start adding support for gfx10.3

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

ac,radeonsi: replace == GFX10 with >= GFX10 where it's needed

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>

radeonsi: enable ARB_sparse_buffer

This seems to be working now, but it wasn't working before.
I don't know what fixed this. Tested on Raven and Navi14.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5402>

tu: Fix context faults loading unused descriptor sets

The app is allowed to never bind descriptor sets that are statically
unused by the pipeline, which would've caused a context fault since
CP_LOAD_STATE6 would try to load the descriptors that don't exist. Fix
this by not preloading descriptors from unused descriptor sets. We could
do more fine-grained accounting of which descriptors are used, but this
is enough to fix the problem.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5400>

i965: Work around incorrect usage of glDrawRangeElements in UE4

Unreal Engine 4 has a bug in usage of glDrawRangeElements,
causing it to be called with a number of vertices in place
of "end" parameter (which specifies the maximum array index
contained in indices).

Since there is unknown amount of games affected and we
could not identify that a game is built with UE4 - we are
forced to make a blanket workaround, disregarding max_index
in range calculations. Fortunately all such calls look like:
  glDrawRangeElements(GL_TRIANGLES, 0, 3, 3, ...);
So we are able to narrow down this workaround.

This was uncovered after b684030c3a656ffdbc93581856034e0982db46fd
broke a bunch of UE4 games.

Cc: 20.1 <mesa-stable@lists.freedesktop.org>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2917
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5203>

tu: Rewrite flushing to use barriers

Replace the various ad-hoc flushes that we've inserted, copied from
freedreno, etc. with a unified system that uses the user-supplied
information via vkCmdPipelineBarrier() and subpass dependencies.

There are a few notable differences in behavior:

- We now move setting RB_CCU_CNTL up a little in the gmem case, but
hopefully that won't matter too much. This matches what the Vulkan blob
does.

- We properly implement delayed setting of events, completing our
implementaton of events.

- Finally, of course, we should be a lot less flush-happy. We won't emit
useless CCU/cache flushes with multiple copies, renderpasses, etc. that
don't depend on each other, and also won't flush/invalidate the cache
around renderpasses unless we actually need to.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4964>

tu: Remove useless event_write helpers

tu6_emit_cache_flush() was wrongly named, and with the removal of the
last parameter both are useless.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4964>

tu: Don't actually track seqno's for events

We just dropped the last user which actually cared about the seqno.
This never worked anyway, since the seqno was never reset between
multiple executions of the same command buffer. Turn the part of the
control buffer which used to track the seqno into a dummy dword, and
figure out automatically whether we need to include it. We will
implement seqnos again eventually, with timline semaphores, but that
will likely be totally different.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4964>

tu: Remove useless post-binning flushes

The Vulkan blob doesn't do this, and based on my understanding of how
the blob works this is unnecessary. CACHE_FLUSH is already serialized
against all 3d commands so you don't need to wait for rendering commands
to finish before issuing it, and the subsequent wfi + WAIT_FOR_ME will
cause the CP to wait for the CACHE_FLUSH to finish, so there's also no
need to wait for it to complete. The CACHE_INVALIDATE also seems
unnecessary, and also isn't done by the blob.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4964>

panfrost: Mark PIPE_BUFFER BOs as not renderable

Without this, memory usage explodes by 16x due to height alignment.

Closes: #2715
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4451>

winsys/radeon: do not cast bo->va as void*

Using a util_hash_table_create_ptr_keys to store bo->va address doesn't
work on 32 bits.
This commit makes radeon_drm_winsys::bo_vas a hash_table_u64 instead.

Tested by Miklós Máté.

CC: 20.1 <mesa-stable@lists.freedesktop.org>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3056
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5380>

ci: use separate docker images for baremetal builds

Using arm_test-base as a separate base layer as well for storage &
network bandwidth efficiency.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5381>

ci: add arm_test-base docker image

Similar to x86_build-base.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5381>

radv/llvm: expose VK_EXT_shader_demote_to_helper_invocation with LLVM 9+

It should already work with the LLVM backend.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5361>

glthread: sync in glFlush for multiple contexts

See the code comment.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5382>