turnip: Update maxGeometryShaderInvocations to match blob

Geometry shaders support an invocations parameter up to a limit
defined by maxGeometryShaderInvocations. This was set to 127, but
executing with invocations > 32 causes a crash. As it turns out, the
blob only advertises a max of 32 invocations, so we set that in
turnip as well.

Fixes dEQP-VK.geometry.instanced.draw_*_instances_{127, 64}_geometry_invocations

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Selectively configure GRAS_LAYER_CNTL

One of the features of geometry shaders is the ability to render to
different layers by assigning to the gl_Layer (Layer in SPIR-V)
builtin.

While have already plumbed the layer regid to the geometry shader,
we also need to GRAS_LAYER_CNTL to actually use layered rendering.
In addition, gmem does not support layered rendering, so we need to
force sysmem.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Set up REG_A6XX_SP_GS_CONFIG

Updates GS_CONFIG and HLSQ_GS_CNTL registers to match those emitted
by the blob and fd.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Configure VFD_CONTROL with gsheader and primitiveid

This commit updates VFD_CONTROL to use the GS header and primitive
ID sysvals if a geometry shader stage is present in the pipeline.
Like in the case of VPC, the code here is adapted from fd6_program.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Configure VPC for geometry shaders

This commit updates tu6_emit_vpc to selectively emit GS-specifc
configuration. Most of this is repurposed from fd6_program.c.

This also refactors `link_geometry_stages` to ir3_nir_lower_tess.c
so it can be shared between fd and tu.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Emit geometry shader obj and related consts

Like with other shader types, we need to emit the geometry shader
object and the consts it uses. In addition, we need to emit
additional geometry-specific consts that link primitive/vertex stride
between the vs and gs. In conjunction with the gsheader, these are
used by the vs to determine where to stlw outputs and used by the gs
to determine where to ldlw those outputs from.

FD emits these consts in the draw call because in GL, you can mix
and match shaders in different programs. In Vulkan, however, we
compile and link the shaders at pipeline creation, so we can emit
these in the pipeline IB instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

turnip: Set has_gs in ir3_shader_key

The ir3 compiler only lowers the VS and GS for geometry shading if
the corresponding has_gs key is set in the shader key. Without it,
GS-specific intrinsics like load_per_vertex_input won't get lowered
and the GS header will be initialized with invalid values.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4436>

radv: Print shader stage before disassembly.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Print shader stage in aco_print_program.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

radv: Enable ACO for NGG VS/TES, but disable NGG for ACO GS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Run GS_ALLOC_REQ on priority 3 for NGG VS and TES.

It is recommended to do this as quickly as possible.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Schedule position exports of NGG VS/TES.

Similarly to the HW VS stage, the HW NGG GS stage also
benefits from executing these exports as early as possible.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Implement NGG VS and TES.

When NGG is used, vertex and tess eval shaders are executed on the
hardware NGG geometry stage. There is a series of steps they
must perform:

* Request GS space using GS_ALLOC_REQ
* Export the primitive
* Finally, export the normal VS outputs

In this commit, two modes are implemented:

* "late" which matches what the RADV LLVM backend currently does
* "early" which is an optimized version as seen in radeonsi

Vulkan doesn't allow the shader to write the edge flags, so we can
currently always use the "early" mode.

Exporting the primitive ID is also supported by having the GS threads
write that into LDS and reading them from LDS in the ES threads.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Setup NGG VS and TES stages.

ngg_vertex_gs and ngg_tess_eval_gs work very similarly to
vertex_vs and tess_eval_vs, but they run on the HW NGG GS stage.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Fix exports for NGG VS and TES.

The exports in NGG VS and TES work just like VS exports,
so the assembler needs to fix these too in the same manner.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Initialize exec mask for NGG VS and TES.

They behave like merged ESGS shaders, so the exec mask needs
to be manually initialized for these NGG shaders too.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco/ngg: Add new stage for hw_ngg_gs.

This is needed to distinguish between NGG and legacy.
Otherwise, vertex_geometry_gs and ngg_vertex_geometry_gs
have the same value, which we want to avoid.

Also, there is no such thing as ngg_vertex_tess_control_hs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Treat s_setprio as a scheduling barrier.

We want to execute instructions after s_setprio in the given
priority, so we must prevent the scheduler from scheduling beyond
s_setprio, otherwise some instructions could be executed in a
different priority.

Rename hazard_fail_memtime to hazard_fail_unreorderable and include
s_setprio in the list of unreorderable opcodes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Extract merged_wave_info_to_mask to its own function.

Currently we only use this at the beginning of merged shader parts,
but we are going to need to use it with some NGG code as well.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Print block_kind_export_end.

Useful when debugging issues with exports.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Extract uniform if handling to separate functions.

Currently we only use this for uniform ifs that come from NIR,
but we are going to need to use it with some NGG parts as well.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576>

aco: Fix crash in insert_wait_states.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4465>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4465>

pan/bit: Wire up add/add op+test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Add fmin/max16 tests

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Enable more debug for `run`

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Add min/max support to interpreter

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Unify test frontends

Random.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Force ADD scheduling for MINMAX

Might be GPU version specific.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Fix incorrect abs flip in fma/fadd16

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Set BI_MODS for MINMAX

We support it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Add ADD add/min/max fp32 packing

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Structify ADD unit add/min/max

..since it's missing for FMA

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Implement min/max on FMA

Unfortunately, while this looks fine to the disasm, it's raising
INSTR_INVALID_ENC on my g31 board here. Looks like it might be ADD only
on newer Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Add special unit test

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bit: Add special op interpreting

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Add fp16 support for frcp/frsq

More ops.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Add 32-bit _FAST packing

For frcp/frsq on newer Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

pan/bi: Remove nontrivial SPECIAL ops

These require a lot more handholding in the IR than we can deal with at
this stage; we need to restrict ourselves to frcp/sqrt.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4470>

aco: zero-initialize Temp

Fixes dEQP-VK.transform_feedback.* crashes from accesses garbage
temporaries in emit_extract_vector().

Fixes: 85521061 ("aco: prepare helper functions for subdword handling")
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4463>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4463>

aco: remove divergence check in sanitize_if()

We also need to do this if a side ends in a divergent break.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4461>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4461>

nir/lower_amul: fix slot calculation

Fixes incorrect indexing in
dEQP-GLES31.functional.ssbo.layout.instance_array_basic_type.packed.mat2x3

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4455>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4455>

nir: add some swizzle helpers

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4455>

nir: Assert memory loads are aligned

We've had alignment parameters on these operations for a while but a
bunch of places weren't setting them.  That should be resolved now so we
can start validating that they're always set.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4441>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4441>

util: move and adjust the vertex upload heuristic equation from u_vbuf

This will also be used by glthread.

The new equation is optimized for glthread.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

vbo: expose helper function vbo_get_minmax_index_mapped for glthread

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: split _mesa_primitive_restart_index into a function without gl_context

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: precompute _mesa_primitive_restart_index during state changes

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: remove no longer needed _mesa_is_bufferobj function

All buffers have Name != 0.

Note that there is no longer the pointer dereference to get Name,
so it's faster.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: remove NullBufferObj

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: don't ever bind NullBufferObj to glBindBuffer(Base,Range) slots

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: don't ever bind NullBufferObj for glBindBuffer targets

Since VAOs don't use NullBufferObj for vertex attribs anymore, let's remove
more uses of NullBufferObj.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: don't ever set NullBufferObj in gl_vertex_array_binding

This improves performance by 5% in the game "torcs", FPS: 98.83 -> 103.73

It does a lot of glPush/PopClientAttrib, which exacerbates the overhead
of setting NullBufferObj.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

mesa: optimize initialization of new VAOs

Precompute the default state in gl_context, and just copy it when we create
a VAO.

This also helps glPushClientAttrib function, which always creates a VAO,
which has a substantial CPU overhead in profiles.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4466>

android: aco: add various compiler statistics

Fixes a building error due to compiler/aco_statistics.cpp
missing in src/amd/Makefile.sources

Fixes: b1544352 ("aco: add various compiler statistics")
Signed-off-by: Mauro Rossi <issor.oruam@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>

nir: fix wrong assignment to buffer in xfb_varyings_info

Tested with dEQP-VK.transform_feedback.fuzz.various_buffers.buffers100_instance_array_vertex

Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Cc: mesa-stable@lists.freedesktop.org
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4459>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4459>

mesa/st: release variants for active programs before unref

Programs can be shared among many contexts and each program holds a
variant list which has context specific variants. When context gets
destroyed it must make sure it relases all variants, otherwise remaining
context that utilizes same program will attempt to save a zombie shader
for already deleted context when releasing program and its variants.

Fixes:
   dEQP-EGL.functional.sharing.gles2.program.render

and other flaky multihread dEQP-EGL failures.

v2: pass program pointer via & (Marek)

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Cc: mesa-stable@lists.freedesktop.org
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4386>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4386>

mesa/st: unbind shader state before deleting it

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Cc: mesa-stable@lists.freedesktop.org
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4386>

pan/bit: Add mode to run unit tests

Probably the most useful of the bunch going forward.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Make run more useful

..by printing some output.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add csel tests

..and pull out common instruction generation to reduce duplication in
tests a bit.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add CSEL to interpreter

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add FMA tests

Now that the earlier reg ctrl issue is fixed these should pass.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add 16-bit fmod tests

These raise another set of issues -- indeed, not all of these tests are
passing, since it turns out I have an actual bug in the packing code. So
after all this work, test bringup has identified an actual issue :)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add verbose printing for tests

We'd like to dump both the generated IR (so we know exactly what's being
tested) as well as the compiled program (so we know what's running for
comparison).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add helper for generating floating mod tests

We can iterate them, isn't that adorable!

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add packing test framework

Given an instruction, we'd like to wrap it in a clause with some I/O on
each end so we can pack it up and send it to the hardware to compare
against the simulator.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Implement floating source mods

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Implement outmods

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Add preliminary FMA/ADD/MOV implementations

Missing some details about modifiers but the core structure should
look like this for 32 and 16-bit, I think. My sincerest apologies for
the macro magic, I tried to make it the least bad I could but trying to
keep down repitition.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Handle read/write

We case the various sources and destinations to model register file
access and passthrough in particular.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bit: Stub out BIR interpreter

We'd like to step through a BIR program to evaluate it for testing.
Let's stub out some infrastructure for modeling Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Match CSEL argument order with hw

It turns out ports need to be in order of the arguments of an
instruction (port 3, that is), which breaks on instructions whose IR
argument order is different from the packed order, like csel. So fix
that.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Add helper to debug port assignment

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Handle BIFROST_FIRST_WRITE_FMA_P2_READ_P3

It's a special case for unclear reasons, and if you mess it up you get
INSTR_INVALID_ENC. Isn't hardware fun?

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Allow BI_FMA to take mods

It doesn't take abs but it can take outmod/neg.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Don't gobble zero ports

In case we've reading/writing R0.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Fix negation in ADD.v2f16

When we flip the sources we need to flip the negates as well to stay
consistent.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Fix duplicated source in ADD.v2f16

Typo.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

pan/bi: Export bi_class_name

For use in naming tests.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4458>

lima: avoid situations when scissor minx > maxx or miny > maxy

Clip scissor to viewport and then to framebuffer to avoid that since
hardware can't handle this case.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4427>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4427>

etnaviv: convert perfmon queries to acc queries

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: move generic perfmon functionality into own file

This change removes the basic infrastructure to work with perfmon
from the perfmon query impl and puts it into its own place.
Makes the whole series easier to review and ends smaller changes.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: extend acc sample provide with an allocate(..)

We might be able to sub-class etna_acc_query.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: extend result(..) to return if data is ready

For the upcoming conversion of perfmon queries to the acc query
framework we need a way to tell that the data is not ready.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: make use of a fixed size array to track of all acc query provider

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: extend acc query provider with supports(..) function

Move the logic if a query provider supports a query_type
directly to the provider.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: rework wait/flush logic

Saves us from doing an extra flush in !wait case and seems more
logical now. Also evaluate etna_bo_cpu_prep(..) retun value.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: reset no_wait_cnt after triggered flush

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: explicitly call resource_written(..)

We might end in cases where etna_acc_get_query_result(..) gets called
within one draw call (aka before flushing). At this point the status
of the resource was not set but gets used in etna_acc_get_query_result(..)
to handle different wait cases. Fix this issue by calling resource_written(..)
explicitly.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: rework etna_acc_sample_provider

Simplify the interface a sampler provider needs to implement. The start(..)
and stop(..) functions got called by resume(..) and suspend(..) so lets
get rid of start(..) and stop(..). Also the way we count and use samples
is much easier to follow now.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

etnaviv: rename hw queries to acc queries

The name hw queries was choosen as occlusion queries are 'feeling'
like nothing special. It is possible to interact with them only
via the command stream - unlike perfom queries where some kernel
magic is needed.

Accumulated HW queries is a much better name for this type of queries.
We read some hardware values over some draw calls and need to accumulate
them to get the final result.

This is some prep work for the following perfmon changes.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1530>

glx: use anonymous namespace to avoid -Wodr issues when building with LTO enabled

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2597>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2597>

glx: fix 630 times -Wlto-type-mismatch when building with LTO enabled

The prototypes are simply copied from include/GL/gl*.h

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2597>

Revert "spirv: Rewrite CFG construction"

This reverts commit fa5a36dbd474fb3c755da51553c6ca18dab76a06.

Revert "gallivm: disable rgtc/latc SNORM accellerated fetches"

This reverts commit 4897e70ccd3987d470ec8622d473ee3405f6e96f.

Fixed in previous commits.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4425>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4425>

gallivm/rgtc: enable fast path for snorm types.

As per Roland's suggestions it should be easy to enable the
fast path fetch for rgtc snorm as well here.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4425>

gallivm/rgtc: fix the truncation to 8-bit

The 8 bit type wasn't 8-bit so when doing signed work we lost
the sign bit. This fixes it to use a proper vector type,
even if we just end up in here with the 1-wide path for now.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4425>

glsl: don't limit fp16 lowering to frag

This restriction doesn't belong in core code.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4423>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4423>

freedreno: limit fp16 to frag and compute

To avoid dealing w/ some backend varying linking issues when mixing
precision vs geom/tess.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4423>

freedreno/ir3: also precompile compute shaders for shaderdb

Similar as with draw shaders, nothing will trigger the final variant in
shader-db.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4423>