pan/bi: Add float-only mode to condition fusing

Useful for discards.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Emit discard (not if)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Handle discard_if in NIR->BIR naively

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Unwrap BRANCH into CONDITIONAL class

We can simplify the IR considerably and unify more conditions, which
gives conditional discard for free.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Remove BI_GENERIC

Goofy.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Structify DISCARD

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Fix DISCARD ops in disasm

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Disable CSEL4 emit for now

We need proper scheduling for 4-src ops to work, so for now disable
condition fusing so we cap at 3-src at a performance penalty. A bit of a
hack but I'd rather not build hacks into a scheduler that will be
rewritten soon anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Fix incorrectly flipped swizzle

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Fix missing swizzle

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Fix double-abs flipping

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Set clause type for gl_FragCoord.z

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Lower gl_FragCoord

We accept a sysval and emit various forms for each component.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Passthrough direct ld_var addresses

Don't bother wasting a constant.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Print bad instruction on src packing fail

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Futureproof COMBINE lowering against non-u32

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Abort on unhandled intrinsics

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Abort on unknown op packing

We're stable enough this is better than just nop'ing it out.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

pan/bi: Add clause type for gl_FragCoord.zw load

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

panfrost: Setup gl_FragCoord as sysval on Bifrost

..rather than a varying.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4883>

etnaviv: do not use int filter when anisotropic filtering is used

The blob does not use this combination. This change moves the
decision if int filter gets used to state emit time.

Fixes: 7aaa0e59086 ("etnaviv: add anisotropic filter support")
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4872>

etnaviv: fix SAMP_ANISOTROPY register value

This caused some serious problems like shredded output, ~1fps and GPU hungs.

Fixes: 7aaa0e59086 ("etnaviv: add anisotropic filter support")
Reported-by: Lukas F. Hartmann <lukas@mntmn.com>
Tested-by: Lukas F. Hartmann <lukas@mntmn.com>
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4872>

vulkan/wsi: Make wsi_swapchain inherit from vk_object_base

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

vulkan: Add run-time object type asserts in handle casts

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

anv: Refactor setting descriptors with immutable sampler

Don't call anv_sampler_from_handle if the handle may be invalid.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

vulkan,anv: Move the DEFINE_HANDLE_CASTS macros to vk_object.h

We've already got these duplicated a bunch of places.  They should
really probably live in common code.  The new versions take two more
arguments:

 1. The struct member which gets you from __driver_type to the
    vk_object_base.  This requires drivers which use this to also use
    vk_object_base.

 2. The VkObjectType enum which represents that object type.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

vulkan,anv: Add a base object struct type

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

anv: Allocate CPU-side memory for events

As discrete graphics looms, we really need to stop storing CPU data
structures in GPU memory.  One of the most egregious instances of this
was VkEvent where we had a CPU data structure living inside a dynamic
state pool allocation.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

anv: Stop clflushing events

They're allocated out of the dynamic state pool which is snooped.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

vulkan,anv: Add a common base object type for VkDevice

We should keep this very minimal; I don't know that we need to go all
struct gl_context on it.  However, this gives us at least a tiny base on
which we can start building some common functionality.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

vulkan: Allow destroying NULL debug report callbacks

Fixes: 086cfa5652 "anv: implementation of VK_EXT_debug_report extension"
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4690>

st/mesa: destroy only own program variants when program is released

Earlier commit tried to achieve this but actually did more. This makes
sure the variants for other contexts continue to live.

Fixes: de3d7dbed52 ("mesa/st: release variants for active programs before unref")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2865
Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4831>

radeonsi: fix export count

Fixes: 17acff01a00 ("radeonsi: skip vs output optimizations for some outputs")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2877
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4871>

vtn/opencl: native sqrt support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: native rsqrt support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: native recip support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: native powr support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: native divide support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: native variants of sin/cos

These obviously map directly to nir opcodes.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

vtn/opencl: add native_tan-support

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

compiler/nir: move tan-calculation to helper

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-By: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4811>

mesa: check draw buffer completeness on glClearBufferfv/glClearBufferuiv

From OpenGL 4.6, section 9.4.4 "Effects of Framebuffer Completeness on
Framebuffer Operations", page 332:

"An INVALID_FRAMEBUFFER_OPERATION error is generated by attempts to render
to or read from a framebuffer which is not framebuffer complete.
This error is generated regardless of whether fragments are actually read
from or written to the framebuffer. For example, it is generated when a
rendering command is called and the framebuffer is incomplete, even if
RASTERIZER_DISCARD is enabled."

Signed-off-by: Dmytro Nester <dmytro.nester@globallogic.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4833>

Revert "ac/surface: remove RADEON_SURF_TC_COMPATIBLE_HTILE and assume it's always set"

This reverts commit f6d87ec8a908250af5e805c8042524ac360094a2.

It breaks RADV.

Fixes: f6d87ec8a908250af5e805c8042524ac360094a2 "ac/surface: remove RADEON_SURF_TC_COMPATIBLE_HTILE and assume it's always set"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4864>

i965: disable shadow batches when batch debugging.

If you want to dump batch state, it needs to have the relocs processed
but the relocs don't get processed on the shadow batch.

Choose debugging over speed here.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4846>

i965: add support for gen 5 pipelined pointers to dump

I wanted to see inside these, so added support to the dumper.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4846>

radv: Extend tiling flags to 64-bit.

SCANOUT is bit 63 ....

Fixes: bfd9e7ff243 "radv: Use new scanout gfx9 metadata flag."
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2879
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4859>

aco: add message to static_assert

static_assert without a message is only supported with C++17 and later.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: c99107ece02b64916031883889b9e010c99e2435
    ('aco: add explicit padding for all Instruction sub-structs')

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4850>

aco: remove use of f-strings

f-strings require Python 3.6 but 3.5 is still maintained and used.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2839
Fixes: 2ab45f41 ("aco: implement sub-dword swaps")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4850>

anv: Disable extensions based on Android versions

This extends commit 2243f0cd for anv with additional
extensions for Pie and Q versions.

Fixes tests with 9_R11 CTS:
dEQP-VK.api.info.android#no_unknown_extensions
dEQP-VK.api.info.device#extensions.

v2: Use snake_case function name (Jason Ekstrand)
    Drop Change-Id in commit (Kristian H. Kristensen)

v3: Resolve meson-clang error for ANDROID_API_LEVEL.

Signed-off-by: Nataraj Deshpande <nataraj.deshpande@intel.com>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4827>

anv: Limit vulkan version to 1.1 for Android

Current Android dessert versions such as Pie, Q reject
vulkan version > 1.1. Clamp the vulkan versions to 1.1
for platforms running these Android desserts.

Fixes android.graphics.cts.VulkanFeaturesTest and
dEQP-VK.api.info.device#properties.

v2: Limit version with '!ANDROID' (Eric Engestrom and Tapani Pälli)

Signed-off-by: Nataraj Deshpande <nataraj.deshpande@intel.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4781>

iris: Implement ARB_compute_variable_group_size

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4794>

intel: Let drivers call brw_nir_lower_cs_intrinsics()

The motivating factor is: this lowering may cause
nir_intrinsic_load_local_group_size intrinsics to be added to the
shader, and by moving this around we make possible for the drivers to
lower that intrinsic by themselves.

Iris will do just that in a later patch for implementing variable
group size.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4794>

intel/fs: Add and use a new load_simd_width_intel intrinsic

Intrinsic to get the SIMD width, which not always the same as subgroup
size.  Starting with a small scope (Intel), but we can rename it later
to generalize if this turns out useful for other drivers.

Change brw_nir_lower_cs_intrinsics() to use this intrinsic instead of
a width will be passed as argument.  The pass also used to optimized
load_subgroup_id for the case that the workgroup fitted into a single
thread (it will be constant zero).  This optimization moved together
with lowering of the SIMD.

This is a preparation for letting the drivers call it before the
brw_compile_cs() step.

No shader-db changes in BDW, SKL, ICL and TGL.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4794>

intel/fs: Add an option to lower variable group size in backend

Adding this since Iris will handle variable group size parameters by
itself.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4794>

intel/fs: Clean up variable group size handling in backend

Just use the information from NIR shader_info.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4794>

iris: Implement PIPE_FLUSH_DEFERRED support.

(Co-authored with Chris Wilson.)

Frequently, games create fences and later check them with a timeout of
0 to see if that work has completed yet.  They do not want the work to
be flushed immediately upon fence creation.

This is what PIPE_FLUSH_DEFERRED does - it inhibits the flush at fence
creation time, but still guarantees that a flush will occur later on
once fence_finish() is called.

Since syncpts can only occur at batch boundaries, when deferring a
flush, we have to wait for the syncpt at the end of the batch being
constructed.  This is later than desired, but safe if blocking.  To
avoid extra delays, we additionally insert a PIPE_CONTROL to write an
availability bit at the exact point of the fence.  We can poll this
on the CPU, allowing us to check whether the fence has gone by, even
if the batch hasn't completed.  It can also let us skip kernel calls.

Improves performance in Bioshock Infinite by 10% on Icelake GT2 on
-ForceCompatLevel=5 settings.  Thanks to Felix Degrood and Mark Janes
for helping notice the extraneous stalls and batches, Marek Olšák for
adding deferred flush support to Gallium to solve this issue, and
Chris Wilson for reworking a lot of the internals of this work.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Detect DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT kernel support

We will use this for implementing deferred flushes in the next commit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

intel: Move anv_gem_supports_syncobj_wait to common code.

This will let me use this in iris.

We leave the existing anv function for anv_gem_stubs.c faking, but
move the contents to a helper in a new src/intel/common/gen_gem.c file.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Flush any current work in iris_fence_await before adding deps

Receiving a fence_server_sync (iris_fence_await) means that any future
work needs to wait for the fence.  But previous work doesn't need to.

So flush it now, to avoid delaying it arbitrarily.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Store a seqno for each batch in the fence

In the next patch, we will introduce deferred fences where we will need
to flush a fence later. To do this, we need to know which batch requires
flushing, so keep a 1:1 mapping between seqno[] and the associated
batch.

It's also substantially less confusing to have a 1:1 mapping.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Convert fences to using lightweight seqno

By using the breadcrumbs we inject into the batch, we can build a
lightweight fence - that can be evaluated in userspace without having to
check in the kernel. In order to pass the fences between processes, and
to wait efficiently, we continue to track the syncobj for each batch and
use that as a terminator for the fence, and for passing coarse
scheduling decisions to the kernel on execbuf.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Place a seqno at the end of every batch

We can use seqno as a basic for fast userspace fences: where we can
check a value directly to test for fence completion without having to
query using the kernel. To do so we need to write a breadcrumb from the
batch and track those writes as the basis for our lightweight fences.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Destroy transfer slab after batches

Batches are going to have an uploader in the next commit, so destroying
batches will destroy uploaders, which will unmap transfers, which will
return things to the slab allocator.  So we need to reorder destroying
the slab allocator to the end to avoid crashing.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Give up on not passing ice to iris_init_batch

We're going to need it to create a uploader in the batch soon.  We still
avoid storing it, to maintain the charade of separation, and make people
think twice about fetching random fields from there and intertwining
things even worse.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Rename iris_syncpt to iris_syncobj for clarity.

This is just a refcounted wrapper around a drm_syncobj.  There is
enough terminology going on in the area of synchronization (sync
objects, sync files, ...) that I'd rather not invent our own.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

anv: Include linux/sync_file.h instead of cut and pasting contents

Linux 4.7 has been out for a long time, this is probably safe to
depend on at this point, rather than cut and pasting the contents.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

iris: Include linux/sync_file.h instead of cut and pasting contents

Lets us drop some cut and pasted kernel header contents.

Linux 4.7 came out 4 years before we the first officially supported
release of this driver; iris won't run on kernels older than 4.16,
and 4.18.11+ is strongly recommended.  So I suspect it's safe to
assume that a kernel header from 4.7 will exist at build time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3802>

panfrost: Update dEQP expectation list

These tests were recently fixed.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Enable nir_opt_algebraic_distribute_src_mods

Helps cleanup some issues otherwise missed by the new source mod
handling. (Noticed a double negative)

total instructions in shared programs: 3606 -> 3605 (-0.03%)
instructions in affected programs: 41 -> 40 (-2.44%)
helped: 1
HURT: 0

total bundles in shared programs: 1883 -> 1883 (0.00%)
bundles in affected programs: 0 -> 0
helped: 0
HURT: 0

total quadwords in shared programs: 3296 -> 3324 (0.85%)
quadwords in affected programs: 596 -> 624 (4.70%)
helped: 0
HURT: 2

total registers in shared programs: 337 -> 336 (-0.30%)
registers in affected programs: 6 -> 5 (-16.67%)
helped: 1
HURT: 0

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Drop `opt` in name of midgard_opt_cull_dead_branch

It's necessary for conformance - not an optimization.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Drop forever todo

Not much to be done.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Move constant switch opts to algebraic pass

No shader-db changes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Rename .one to .sat_signed

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

pan/mdg: Ingest actual isub ops

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4852>

glthread: Add GLAPIENTRY to _mesa_marshal_MultiDrawArrays.

Fixes MSVC build.  Trivial.

Fixes: 2840bc3065b9e991b2c5880a2ee02e2458a758c4

intel/dev: Bail when INTEL_DEVID_OVERRIDE is not valid

Avoids surprises where you set an OVERRIDE but it gets ignored and the
system PCI ID is used.

Also fixes the bug that the error of invalid platform name being
printed too early, even when the passed platform was a PCI ID (which
is also supported).

For the case where euid != uid, a warning was added but the behavior
wasn't changed: it is still going to fallback to system PCI ID.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4841>

anv,iris: Fix input vertex max for tcs on gen12

gen12 does away with the single patch dispatch mode for tcs, and
increases some limits so that 8_patch mode can always work. Make the
necessary changes so we don't try to fall back to single patch mode.

Fixes KHR-GL46.tessellation_shader.single.max_patch_vertices and others

Fixes: 44754279ace7 ("intel/fs/gen12: Use TCS 8_PATCH mode.")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4843>

freedreno/ir3: Set the FS .msaa flag to true during precompiles.

If you're going out of your way to do per-sample interpolation, you are
almost surely going to be doing so to an MSAA framebuffer.  Should reduce
recompiles with MSAA enabled.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno: Immediately compile a default variant of shaders.

Now that we normalize our keys fairly well, build a variant at shader
state creation time so that hopefully you don't have to call the compiler
at draw time (as is now the case with glmark2 ES and most of the humus GL
demos).

Fixes: #2782
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Set up outputs for multi-slot varyings.

Necessary to avoid compiler assertion failures in:

dEQP-GLES31.functional.program_interface_query.program_output.type.interface_blocks.out.named_block_explicit_location.struct.mat3x2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Stop initializing regid of so->outputs during setup.

It's unused and overwritten by ir3_compile_shader_nir().

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Improve shader key normalization.

We can remove a bunch of conditional code at key comparison time by
computing a bitmask of used key bits at ir3_shader creation time.  This
also gives us a nice place to put additional key simplification to reduce
how many variants we create (like skipping rastflat if we don't read
colors in the FS, or skipping vclamp_color if we don't write colors).

It does mean walking the whole key to AND it, but the key is just 28 bytes
so far so that seems pretty fine.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno: Emit debug messages when doing draw-time recompiles of shaders.

Right now that's "always" unless you have shaderdb set.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Remove unused half precision shader key flag.

The code using it was removed in 4af86bd0b933 ("freedreno/ir3: remove
half-precision output")

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno: Fix assertion failures on GS/tess shaders with shader-db enabled.

We weren't filling in the tess mode of the key, or setting has_gs on GS
shaders, resulting in assertion failures when NIR intrinsics didn't get
lowered.

We have to make a guess at prim mode for TCS, but it should be better to
have some shader-db coverage than none, and it will avoid these failures
happening when we start precompiling shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Skip tess epilogue if the program is missing stores.

Some of the negative API tests make shaders for tess stages that don't do
all the stores they need to.  Once we start precompiling (or doing
shader-db of tess), we need to at least not segfault when generating them.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno: Stop doing binning shaders other than the VS in shader-db.

ir3_cache.c only ever asks for binning variants for VS.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Fix register allocation assertion failures.

We were failing to tell the allocator about the restriction that scalar
texture instructions (allocated as scalar regs) couldn't be allocated such
that the start of the full unwritemasked vector started before r0.  There
was a patch in select_reg_callback on a6xx that tried to work around that,
but you could still end up backed into a corner you shouldn't be because
we didn't tell the RA what it needed.

Fixes compiler assertion failures on a300-a400's blit_z shader, used for
Z32F gmem blits.

Looks like as a result we get tighter register allocation but more nops:

instructions in affected programs: 757945 -> 760356 (0.32%)
nops in affected programs: 317983 -> 320468 (0.78%)
non-nops in affected programs: 27525 -> 27451 (-0.27%)
mov in affected programs: 3098 -> 3023 (-2.42%)
dwords in affected programs: 109664 -> 110656 (0.90%)
last-baryf in affected programs: 112701 -> 112847 (0.13%)
full in affected programs: 4326 -> 4011 (-7.28%)
sstall in affected programs: 120550 -> 120836 (0.24%)
(ss) in affected programs: 13939 -> 13918 (-0.15%)
(sy) in affected programs: 3006 -> 2786 (-7.32%)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Drop hack to clean up split vars

When the GS lowering was working on store_output intrinsics, we had to
clean up the split vars to avoid getting confused.  Now that we shadow
the output vars instead, there's no confusion and we can drop this
hack.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Lower GS builtins before lowering IO

We mostly got away with replacing a store_output with a store_var, but
for complex types like structs, that doesn't work. Once the IO has
been lowered from vars to intrinsic, we've lost the deref chains and
can't properly shadow the outputs.

This commits moves the GS lowering up so we do it before the output
variables get lowered to store_output.  This way the pass works much
like nir_lower_io_to_temporaries() and cleanly shadows the outputs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Add ir3_nir_lower_to_explicit_input() pass

This pass lowers per-vertex input intrinsics to load_shared_ir3. This
was open coded in the TCS and GS lowering passes before - this way we
can share it. Furthermore, we'll need to run the rest of the GS
lowering earlier (before lowering IO) so we need to split off this
part that operates on the IO intrinsics first.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Rename ir3_nir_lower_to_explicit_io

We rename it to ir3_nir_lower_to_explicit_output, since it only
handles output and we'll add a lowering pass for input next.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Pass stream output info to ir3_shader_from_nir

We need shader->stream_output filled out when we layout the push
constants in ir3_setup_const_state(). Otherwise
const_state->offsets.tfbo ends up as ~0, which doesn't work.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Fix the a3xx TF outputs stores.

We were trying to deref the vector-collected outputs[] array before it's
been set up, but we want the per-component outputs anyway.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

freedreno/ir3: Set up the block predecessors for a3xx TF

Fixes a segfault in ir3_legalize.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4562>

intel/fs: Update location of Render Target Array Index for gen12

Render Target Array Index has moved from R0.0[26:16] to
R1.1[26:16] on gen12.

Fixes dEQP-VK.multiview.input_attachments.*

Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4836>

pan/decode: Properly print tripped zeroes

The "%" got lost.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Fixes: 6148d1be4bb5 ("panfrost: Fix size of bifrost sampler descriptor")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4832>

panfrost: Add Bifrost texture trampoline BO to batch

Fixes: d3eb23adb50c ("panfrost: Emit sampler descriptor on bifrost")
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4832>

pan/bi: Lower for now sincos

Will be implemented later.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4832>

panfrost: mali_attr_meta.unknown1 is zero on Bifrost

For unknown1 reasons :)

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4832>