nir: Add a helper for general instruction-modifying passes.

The nir_shader_lower_instructions() is really nice, but it's only for SSA
operations, and sometimes you want something more general.  I've put it in
nir_builder.h so it can be inlined and retain the same performance
characteristics we're used to in our lowering passes even in the absence
of LTO.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>

nir/opt_copy_prop_vars: Quiet valgrind warning about overlapping memcpy.

The warning is kind of silly:

Test case 'dEQP-GLES2.functional.shaders.indexing.tmp_array.vec3_const_write_static_read_vertex'..
==1874780== Source and destination overlap in memcpy(0xa261690, 0xa261690, 160)
==1874780==    at 0x484D498: __GI_memcpy (vg_replace_strmem.c:1037)
==1874780==    by 0x596FC07: copy_entry_remove (nir_opt_copy_prop_vars.c:296)

The "memcpy is undefined if they overlap" thing is surely meant to be
"memcpy with *partial* overlap is undefined", but let's keep anyone else
from having to debug this.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6178>

egl: document which driver hooks are only required by extensions

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

egl: move extension driver functions after core functions

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: correct/update DRI2 mention with the shiny new DRI3

That initial sentence must have been written ages ago ^^'

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: add some more documentation

Inspired by `src/egl/main/README.txt`, which was severely outdated, but
still contained valid information.

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: add haiku driver

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: complete list of dri2 platforms

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: move section around

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

docs/egl: fix typo

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6130>

nir/vtn: SPIR-V bit count opcodes (core and extension) dest size mismatches nir

SPIR-V dest sizes match the input, while nir is always int32. Insert
casts from the nir op to the expected SPIR-V dest.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir: Add bit_count to lower_int64 pass

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir: Remove 32bit restriction for uadd_carry optimization

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir: Implement mul_high lowering for bit sizes other than 32

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir_lower_bit_size: Support lowering ops with differing source/dest sizes

Specifically the bit-finding routines always return int32. Don't complain
about the dest already being 32 bits when lowering to 32 bits, and
don't bother casting the dest if it's already right.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir: Relax opt_if logic to prevent re-merging 64bit phis for loop headers

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

nir: Add a lowering pass to split 64bit phis

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>

iris: Drop stale syncobj references in fence_server_sync

When calling glWaitSync (fence_server_sync), we added dependencies
in all batches (render and compute) on existing work.  Even if
applications don't use compute at all, they theoretically could,
so we record that the compute batch depends on the render batch.

But if the application truly doesn't use compute, or rarely uses
it, we ended up recording dependencies on _all_ previous render
batches, racking up a massive list of syncobjs.  Not only is this
pointless, it also meant that we never allowed the kernel to free
the underlying i915_request objects.

There are a number of solutions to this problem, but for now, we
take a simple one: when recording a new syncobj dependency, we
walk the list and see if any of them have already passed.  If so,
that dependency has been fulfilled.  We no longer need to track it,
and can simply drop it from the list, unreferencing the syncobj.

Android's SurfaceFlinger in particular was hitting this issue,
as it uses glWaitSync, doesn't typically use compute shaders,
and runs for long durations.

Thanks to Yang A Shi <yang.a.shi@intel.com> and
Kefei Yao <kefei.yao@intel.com> for their excellent work in
tracking down this issue!

Fixes: f459c56be6b ("iris: Add fence support using drm_syncobj")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Tapani Pälli <tapani.palli@intel.com>
Tested-by: Yang A Shi <yang.a.shi@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6463>

iris: Reorder the loops in iris_fence_await() for clarity.

Swapping the order of the loops makes the logic much easier to follow:
for each point in our fence, if it hasn't gone by, make future work in
all batches depend on it.  Both loops are necessary, and now it's
clearer why.

(This doesn't actually fix a bug but needs to be cherry-picked for
the next patch to apply, which does fix a bug.)

Fixes: f459c56be6b ("iris: Add fence support using drm_syncobj")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Tapani Pälli <tapani.palli@intel.com>
Tested-by: Yang A Shi <yang.a.shi@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6463>

ci: Fix up rules for post-merge / main project branch pipelines

Pre-merge pipelines for MRs appear to run in the target project
namespace now, so we have to explicitly rule those out.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6481>

aco: add ACO_DEBUG=novn,noopt,nosched for debugging purposes

To disable value numbering, optimizations and scheduling.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6470>

radv: Fix 3d blits.

- the offsets are inclusive-exclusive so the +1 was wrong
- Since the GPU doesn't do the interpolation on depth (as we render
  per layer), we have to add an offset for the pixel center.

CC: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3073
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6458>

st/mesa: remove useless code for lowered IO in st_nir_assign_vs_in_locations

It's not needed.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6465>

ci: Test the KHR-GL* CTS cases with softpipe.

I've been hacking on softpipe in the process of trying to delete a bunch
of core Mesa code, and want to make sure I don't regress desktop GL
either.  The run takes under a minute and a half.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6466>

aco: sink get_alu_src() in bfe lowering

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6424>

aco: fix sgpr ubfe/ibfe if the offset is too large

If the offset is large enough, it could affect the width. I'm also not
sure if the hardware masks the offset by 0x1f.

Found by inspection. No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6424>

aco: remove 64-bit SGPR ubfe/ibfe

ubfe/ibfe is always 32-bit.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6424>

aco/tests: add tests for long jumps

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: shorten disassembly for repeated instructions

Future tests will do this.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco/tests: add test for GFX10 0x3f bug

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: create long jumps

When the branch offset can't be encoded, we have to use s_setpc_b64.

Fixes hang in RPCS3 vertex ubershader.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3231
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: reserve 2 sgprs for each branch

We'll need two sgprs for the possibility of a long jump.

fossil-db (Navi):
Totals from 10197 (7.50% of 135946) affected shaders:
SGPRs: 946268 -> 946468 (+0.02%)
VGPRs: 705884 -> 707956 (+0.29%); split: -0.00%, +0.30%
SpillSGPRs: 31485 -> 36212 (+15.01%); split: -0.04%, +15.05%
CodeSize: 88296484 -> 88384604 (+0.10%); split: -0.01%, +0.11%
MaxWaves: 81379 -> 81171 (-0.26%)
Instrs: 17219111 -> 17231682 (+0.07%); split: -0.03%, +0.10%
Cycles: 1594875900 -> 1596450136 (+0.10%); split: -0.05%, +0.15%
VMEM: 1687263 -> 1689080 (+0.11%); split: +0.14%, -0.03%
SMEM: 657726 -> 660262 (+0.39%); split: +0.61%, -0.22%
VClause: 294806 -> 294638 (-0.06%); split: -0.08%, +0.02%
SClause: 556702 -> 556210 (-0.09%); split: -0.12%, +0.03%
Copies: 1466323 -> 1469349 (+0.21%); split: -0.57%, +0.78%
Branches: 619793 -> 618556 (-0.20%); split: -0.28%, +0.08%
PreSGPRs: 806364 -> 811477 (+0.63%); split: -0.14%, +0.77%
PreVGPRs: 655845 -> 657174 (+0.20%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: keep loop live-through variables spilled

fossil-db (Navi):
Totals from 3149 (2.32% of 135946) affected shaders:
VGPRs: 280928 -> 280932 (+0.00%)
SpillSGPRs: 51133 -> 30042 (-41.25%)
CodeSize: 43063076 -> 41377252 (-3.91%); split: -3.92%, +0.00%
Instrs: 8278435 -> 8037133 (-2.91%); split: -2.92%, +0.00%
Cycles: 709575456 -> 683366172 (-3.69%); split: -3.69%, +0.00%
VMEM: 542887 -> 542937 (+0.01%); split: +0.05%, -0.04%
SMEM: 210255 -> 206368 (-1.85%); split: +0.12%, -1.97%
SClause: 258847 -> 258019 (-0.32%); split: -0.52%, +0.20%
Copies: 731836 -> 684784 (-6.43%); split: -6.44%, +0.01%
Branches: 305422 -> 292844 (-4.12%); split: -4.12%, +0.00%
PreSGPRs: 333103 -> 332701 (-0.12%)
PreVGPRs: 280086 -> 280089 (+0.00%)

Helps mostly Detroit: Become Human and the single spilling Doom Eternal
shader.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: fix spills_entry heuristic for branch blocks in init_live_in_vars()

fossil-db (Navi):
Totals from 222 (0.16% of 135946) affected shaders:
SpillSGPRs: 9121 -> 9117 (-0.04%)
SpillVGPRs: 2820 -> 1821 (-35.43%)
CodeSize: 5134264 -> 5053336 (-1.58%); split: -1.63%, +0.05%
Instrs: 953435 -> 938761 (-1.54%); split: -1.59%, +0.05%
Cycles: 100567688 -> 97252432 (-3.30%); split: -3.34%, +0.04%
VMEM: 40752 -> 39219 (-3.76%); split: +0.04%, -3.80%
SMEM: 15416 -> 15509 (+0.60%); split: +0.64%, -0.03%
VClause: 20120 -> 19091 (-5.11%)
SClause: 23540 -> 23544 (+0.02%); split: -0.11%, +0.12%
Copies: 125912 -> 122017 (-3.09%); split: -3.36%, +0.26%
Branches: 31131 -> 30009 (-3.60%)

Mostly affects parallel-rdp ubershaders.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: fix regclass checks when fixing to vcc/exec with Builder

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: don't fix break condition for break+discard to exec

This would move the old exec mask back into exec. This also fixes the
live_out_exec.

Issue found in dEQP-VK.graphicsfuzz.cosh-return-inf-unused

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: don't consider the first partial spill if it's the wrong type

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

aco: consider branch definitions in spiller

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: 20.2 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>

radv: print a warning when RADV_TRAP_HANDLER is used

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6468>

etnaviv: Add lock around pending_ctx

The content of rsc->pending_ctx could be changed from multiple contexts
and thus from multiple threads. The per-context lock is not sufficient
to protect this list. Add per-resource lock to protect this list.

Fixes: e5cc66dfad0 ("etnaviv: Rework locking")
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6454>

etnaviv: Remove etna_resource_get_status()

This function is not used, remove it.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6454>

etnaviv: Fix disabling early-z rejection on GC7000L (HALTI5)

The VIVS_PE_DEPTH_CONFIG_DISABLE_ZS in PE_DEPTH_CONFIG caused depth
write hangs on HALTI5.
This is because the 0x11000000 bits in RA have to be toggled on
when setting this bit to zero. This combination will disable
early-z rejection on GC7000L, which was previously done through
a different bit.
Tested only on GC7000L so far.

Signed-off-by: Lukas F. Hartmann <lukas@mntre.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5456>

radv: fix setting EXCP_EN for different shader stages

While TRAP_PRESENT is always at the same place, EXCP_EN can be
different between shader stages. This sets it properly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6452>

v3d: set instance id to 0 at start of tile

PTB assumes that base instance to be 0 at start of tile, but hw would
not do that, we need to set it. It is worth to note that the opcode
name is somewhat confusing as what it really sets is the base
instance. We could rename the opcode, but then the name would be
different to the original Broadcom name, so confusing in any case.

This fixes several dEQP-GLES3 and dEQP-GLES31 tests that passes
individually, but started to fail depending on other tests running
before using base instance different to zero.

This is the backport of a Vulkan patch that fixed some Vulkan CTS
tests that start to fails after some other tests used an instance id.

CC: 20.2 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6447>

v3d/packet: fix typo on Set InstanceID/PrimitiveID packet

Fixes: 276d22c52 ("v3d: Add some more new packets for V3D 4.x.")
CC: 20.2 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6447>

freedreno/a5xx: Don't set the VARYING flag for fragcoord-only programs.

As with a6xx (commits beb02a78, 5785bcc8), the blob doesn't set this flag
for a5xx when fragcoords are used but not proper varyings.  See for
example dEQP-GLES2.functional.shaders.builtin_variable.fragcoord_xyz.

The hope was that this would clear up separate_shader fails/flakes like it
helped with a6xx's flakes, but that didn't happen.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6464>

gallium/dri2: Report correct YUYV and UYVY plane count

Return the actual number of planes in these formats (one) instead of the
number of planes used for lowering (two).

Fixes: d5c857837aa ("gallium/dri2: Fix creation of multi-planar modifier images")
Acked-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6449>

ci: Mark the rest of compswap as flaky on freedreno.

We got another flake, this time on
dEQP-GLES31.functional.compute.shared_var.atomic.compswap.highp_uint,
which blocked !4162 from merging.  Mark the rest flaky so we don't have to
keep firefighting one test at a time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6459>

nir: Report progress properly in nir_lower_bool_to_*

All three passes have the same bug where, in the mov/vec case they
unconditionally return true even if they don't change anything.  Throw
in a bit size check so they return false properly.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6435>

freedreno/a6xx: fix hang with large render target

It seems we do have some limits.  Similar to older gens, # of tiles per
pipe cannot be more than 32.  But I could not trigger any hangs with 16
or more tiles per pipe in either X or Y direction, so that limit does
not seem to apply.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6461>

freedreno: add env var to override tiles-per-pipe

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6461>

freedreno: add env var to override GMEM size

Sometimes it is useful to force a smaller size while debugging.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6461>

freedreno/gmemtool: add tile_alignw/h and a650

Fixes: f6f8a190920 ("freedreno/a6xx: split up gmem/tile alignment requirements")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6461>

radeonsi: fix tess levels coming as scalar arrays from SPIR-V

This fixes: KHR-GL45.gl_spirv.spirv_modules_positive_test

Fixes: 75ce078a0af "radeonsi: enable NIR by default and document GL 4.6 support"
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6460>

mesa: add NV_copy_depth_to_color support for nir

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6289>

nvc0: Add shader disk caching

Adds shader disk caching for nvc0 to reduce the need to every time compile
shaders. Shaders are saved into disk_shader_cache from nvc0_screen structure.

It serializes the input nv50_ir_prog_info to compute the hash key and
also to do a byte compare between the original nv50_ir_prog_info and the one
saved in the cache. If keys match and also the byte compare returns they
are equal, shaders are same, and the compiled nv50_ir_prog_info_out from the
cache can be used instead of compiling input info.

Seems to be significantly improving loading times, these are the results
from running bunch of shaders:
cache off
real 2m58.574s
user 21m34.018s
sys 0m8.055s

cache on, first run
real 3m32.617s
user 24m52.701s
sys 0m20.400s

cache on, second run
real 0m23.745s
user 2m43.566s
sys 0m4.532s

Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: Add nv50_ir_prog_info serialize

Adds a function for serializing a nv50_ir_prog_info structure, which is
needed for shader caching.

v2 (Karol): strip nir when serializing

Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: Add prog_info_out print

Adds a function for printing nv50_ir_prog_info_out structure
in JSON-like format, which could be used in debugging.

Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: Add nv50_ir_prog_info_out serialize and deserialize

Adds functions for serializing and deserializing
nv50_ir_prog_info_out structure, which are needed for shader caching.

Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: Use a bit field in info_out structure

This will decrease structure size.

Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: add nv50_ir_prog_info_out

Split out the output relevant fields from the nv50_ir_prog_info struct
in order to have a cleaner separation between the input and output of
the compilation.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

nv50/ir: remove symbol table support for compute shaders

The initial plan was to use this for OpenCL kernels, but back then the
plan was to convert from LLVM to TGSI. As it turns out, we didn't went
that way.

Right now for OpenCL we don't reqiure supporting multiple entry points
inside the same binary and if we want to support it later, we can add
this back.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4264>

radv: Update CTS version.

CC: 20.2 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6444>

radv: Remove conformance warnings with ACO.

Nobody complained about our conformance submissions.

CC: 20.2 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6444>

nir: Move new edgeflag assert into the io_lowered case

We only need to assert this in the `io_lowered` case, which actually
uses num_outputs.  This assert also doesn't appear to hold on iris,
where num_outputs is showing up as 0 (because it's likely not yet set).

Fixes assertion failures in edgeflag related tests on iris, which
doesn't use the io_lowered path currently.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3456
Fixes: 484a60d5474 ("nir: generate lowered IO in nir_lower_passthrough_edgeflags")
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6450>

nir: Copy semantics to nir_intrinsic_load_fs_input_interp_deltas

When using nir_lower_interpolation, we need to propagate the IO
semantics from the load_interpolated_input to the new
load_fs_input_interp_deltas intrinsics.  nir_lower_io assumes
they will be filled out.

This fixes assertions in most tests on iris since commit
01ab308edc78cda777bc66f2e8110fbd8c21aa18, where nir_lower_io
started reading this field.

Fixes: 01ab308edc7 ("nir: update IO semantics in nir_io_add_const_offset_to_base")
Fixes: 502abfce7f5 ("nir: save IO semantics in lowered IO intrinsics")
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6450>

panfrost: Drop mali_shader_meta

Now completely replaced by XML versions.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

pan/decode: Use unpacks for state descriptor

This is a win in terms of line of decoder code, but it's a regression in
terms of verbosity. That will be fixed when we teach the decode
autogeneration about defaults and non-canonical fields.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use pack for fragment shaders

The heart of the series! Build up all the structures separately and
combine with an OPAQUE pack. Groups state naturally and eliminates the
memcpy. And a nice cleanup too.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use opaque pack for vertex shaders

Not much difference but eliminates a reference to shader_meta (and
avoids a memcpy but who cares).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Add optional opaque packs to GenXML

These special OPAQUE packs use packed structs in the struct template,
instead of struct templates. The use case is packing nested structs
out-of-band, to fit into the CSO model.

A more conventional GenXML solution would be an overlapping uint, but
this breaks our assumptions about struct packing which are otherwise
correct, so this seemed less intrusive than risk disrupting the main
pack routines.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use pack for Bifrost test state

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use pack for blit shaders

This is quite a bit cleaner, I think, and validates the XML in
preparation for moving over the main driver.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Inherit default values from structs

Nesting can be useful sometimes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: XMLify the rest of shader_meta

This contains a bit of everything, so just XML for this commit. The rest
of the series will be slowly moving over to this representation.

The one noteworthy addition is the rename of "No MSAA" to
"Single-sampled lines". This came about due to a buggy branch that
forgot to set this bit. Ths worked, with the caveat of the following
tests failing with a single-sampled framebuffer:

   dEQP-GLES2.functional.rasterization.interpolation.basic.line_loop_wide
   dEQP-GLES2.functional.rasterization.interpolation.basic.line_strip_wide
   dEQP-GLES2.functional.rasterization.interpolation.basic.lines_wide
   dEQP-GLES2.functional.rasterization.interpolation.projected.line_loop_wide
   dEQP-GLES2.functional.rasterization.interpolation.projected.line_strip_wide
   dEQP-GLES2.functional.rasterization.interpolation.projected.lines_wide
   dEQP-GLES2.functional.rasterization.primitives.line_loop
   dEQP-GLES2.functional.rasterization.primitives.line_loop_wide
   dEQP-GLES2.functional.rasterization.primitives.line_strip
   dEQP-GLES2.functional.rasterization.primitives.line_strip_wide
   dEQP-GLES2.functional.rasterization.primitives.lines
   dEQP-GLES2.functional.rasterization.primitives.lines_wide

That is, this bit controls the behaviour of line rasterization with
multisampling. This is required to implement the divergent behaviours
described in the OpenGL ES 3.2 specification sections 13.6.1 ("Basic
Line Segment Rasterization") and 13.6.4 ("Line Multisample
Rasterization"), where setting this bit corresponds to the former
(single-sampled) behaviour.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use preuploaded shader descriptors

For non-fragment shaders, we can just use the preuploaded BO as-is and
do no packing/copying at draw-time. Fragment shaders are still a bit of
an edge case, in having rasterizer/zsa/blend state in the same
descriptor, but now we can specialize that path further for
fragment-only.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Upload shader descriptors at CSO create

Now that we've fixed all the implicit state dependencies, these don't
change after the variant is created.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Allocate a state uploader

We'd like to uploader some descriptors at CSO time.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Ensure shader-db state is zero-initialized

Otherwise the next commit will invoke undefined behaviour.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Simplify shaderless packing

Let's keep all the shader descriptor stuff together.

Note that the packing here can be constant folded entirely in any
reasonable compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Prepack fragment properties/preload

This isn't as clean as vertex shaders, since some of this state is only
known at draw-time (e.g. some reasons we might disable early-Z). But we
can still pack as much as we can ahead-of-time and then OR together
what's left at draw-time.

Thank you to Kristian for this one crazy trick (blob developers hate
it! er...)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Pack vertex properties when compiling

They only depend on shader properties so we can do all this work at CSO
create time, reducing draw-time overhead.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Derive UBO count from shader_info

Rather than checking against the bound constant buffers. Like
with num_textures, this eliminates a dependency of the shader descriptor
on the context, which matters especially for vertex/compute shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: XMLify beginning of shader descriptor

We have just enough abstracted now to pack this ahead-of-time, during
the CSO create, instead of at draw-time. But it's only a start.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Derive texture/sampler_count from shader

This avoids a dependency of the shader descriptor on the texture/sampler
state, which simplifies state management. It also fixes pandecode
warnings where textures/samplers are specified but not referenced.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Quiet pandecode error

The smallest job descriptor is smaller than 256, and with the
tighter packing, pandecode can sometimes error nowadays.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Support SHADERLESS mode everywhere

Now that the missing bits on SFBD are identified, and we have a
reasonable way to pack the properties for Bifrost, we can probably do
this everywhere to generalize the optimization and drop the quirk.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Identify additional SFBD flags

These are analogues to the flags we have in the blend structure on
T760+, which enables us to fix shaderless and sRGB operation on T720.

Closes #2771

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Fixes: a64599a303e ("panfrost: Pass the sampler view format when creating a tex descriptor")
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: XMLify Bifrost preload

There's a lot of code here since the meaning of this field changes
depending on shader state. The good news is that our careful handling
allows preload registers to be decoded now, which pandecode could not
previously do. Likewise, the cmdstream code to emit this is now much
more obvious.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Group SFBD code tighter

Allows us to drop the redundant check by reordering, and will match the
conventions we'll use once the whole structure is XMLified.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Drop redundant NULL check

ctx->blend is already dereferenced by this point anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: XMLify bifrost1

It's so poorly understood there's not much to do in this commit, sadly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Remove midgard1 bitfield

It is now entirely XML, including for decoding. Woo!

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Simplify bind_blend_state

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Fold work_count packing for blend shaders

It's annoying this is needed at all, but that's life for you.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use pack for shaderless

We'll need a more aggressive refactor for this soon, I think.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Use packs for fragment properties

A bit more complicated, since there are dependencies on other state (for
early-z and discard handling). We separate this cleanly to help defer
packing later.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Pack compute Midgard properties

Now that we have XML for it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: XMLify Midgard properties

In a long journey to a full XML representation of mali_shader_meta,
let's start with the fourth word, containing some shader properties.
This is a translation from panfrost-job.h, with the exception of
widening the uniform buffer count field [1]

The other noteworthy change is combining the unknown 0x20 flag with the
WRITES_Z flag to form a 2-bit depth source. This papers over the fact
that the blob zeroes this field for non-fragment shaders. Given the
proximity, this is a reasonable guess and avoids an ugly "is_fragment"
bit.

[1] Justified by the increased limit advertised by the Vulkan blob
(maxDescriptorSetUniformBuffers on
https://vulkan.gpuinfo.org/displayreport.php?id=5602#limits). Not
actually supported in Panfrost right now.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>

panfrost: Group SFBD state together

By proximity with the other fields.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6440>