radv: Enable ACO on all stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Enable running TES as ES, including merged TES+GS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

radv: Enable ACO for TES when there is no GS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement loading TES inputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Enable streamout when TES runs on the HW VS stage.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Store TES outputs when TES runs on the HW VS stage.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Use TES output info when TES runs on the VS stage.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Setup tessellation evaluation shader variables.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

radv: Enable ACO for tessellation control shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Fix LS VGPR init bug on affected hardware.

Vega 10 and Raven have a HW bug: when the HS thread count is zero,
the LS input arguments are loaded in the wrong registers. This commit
works around this by using the registers where the data actually is,
for the affected arguments.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Store VS outputs correctly when tessellation is used.

When tessellation is used, the VS runs on the HW LS stage (merged
into HS on GFX9-10). This commit enables such VS to store its
outputs properly in LDS so that the TCS can load them as its
per-vertex inputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement tessellation control shader input/output.

Tessellation control shaders can have per-vertex inputs,
and both per-vertex and per-patch outputs. TCS can not only store,
but also load their outputs.

The TCS outputs are stored in RING_HS_TESS_OFFCHIP in VMEM, which
is where the TES reads them from. Additionally, the are also stored
in LDS to make sure they can be loaded fast when read by the TCS.

Tessellation factors are always just stored in LDS.
At the end of the shader, the first shader invocation reads these
from LDS and writes them to RING_HS_TESS_FACTOR in VMEM, and
additionally to RING_HS_TESS_OFFCHIP when they are read by
the Tessellation Evaluation Shader.

This implementation matches the memory layouts used by radv_nir_to_llvm.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Fix combining DS additions in the optimizer.

Previously, it was calculated incorrectly for 64-bit writes and reads.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Slight fix to lds_store and lds_load.

This commit fixes lds_store and lds_load so that they can
properly support 32 and 64-bit loads and stores; and makes
them a little more reusable so they can be used by
tessellation control shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Refactor VS output stores in preparation for tessellation.

This commit takes the new helpers into use by the VS output
store function. This function is also where the VS outputs will
be handled when the VS runs on the HW LS stage.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Refactor load_per_vertex_input in preparation for tessellation.

This commit carves out the GS per-vertex input load, and takes the
new helper functions into use.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Introduce new helpers for calculating address offsets.

These helpers are going to make it unnecessary to reimplement the
(almost) same address offset calculation in mulitple places.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Introduce new VMEM load/store helpers.

These are going to be used for loading and storing inputs
and outputs in various stages, such as GS, TCS and TES.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Remove esgs_itemsize from LDS alignment calculation.

It was problematic to have it, because some shader stages might
not even know about the esgs_itemsize, for example TCS and
the merged VS+TCS stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Extract LDS alignment calculation to a separate function.

This function is going to be reused in multiple functions when
storing or loading something in the LDS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Remove vertex_geometry_gs assertion from merged shaders.

We are going to support more kinds of merged shaders, such
as vertex_tess_control_hs and tess_eval_geometry_gs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Use mesa shader stage when loading inputs.

This makes it more clear which stages should load these inputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Setup correct HW stages when tessellation is used.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement load_view_index for TCS and TES.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement memory_barrier_tcs_patch.

TCS outputs are going to be written to LDS, so it
has to use memory_barrier_shared in order to ensure
that it waits for LDS writes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement control_barrier for tessellation control shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement load_invocation_id for tessellation control shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement load_patch_vertices_in.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement load_primitive_id for tessellation shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Implement load_tess_coord.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Setup tessellation control shader variables.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

aco: Extract setup_gs_variables into a separate function.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

radv: Move some helper functions to the radv_shader.h header file.

Move calculate_tess_lds_size and get_tcs_num_patches to radv_shader.h
ACO will need to call these functions too.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3964>

vdpau: remove bogus assert

The assert introduced by 24f2b0a8560 triggers when an application
requests a chroma_type that's different to the one from the
PIPE_VIDEO_CAP_PREFERED_FORMAT (before this change the chroma_type
was set but ignored).

So restore this behavior and ignore the chroma_type.

Reported-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Fixes: 24f2b0a8560 ("gallium/video: remove pipe_video_buffer.chroma_format")
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4104>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4104>

radv: do not recursively begin/end render pass for meta operations

To avoid breaking SQTT user markers that are emitted to report
barriers and layout transitions to RGP.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4136>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4136>

lima/gpir: print acc ops even if we have only one source

floor and sign have only one source, so we need to print acc ops
even if src1 is unused.

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4110>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4110>

lima/gpir: improve disassembler output

Print each op at new line and add unit name suffix for each op.
It improves readability a bit and gives us a hint what unit
was used for particular op.

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4110>

lima: print gp uniforms if gp debug is enabled

Since we keep other constants there as well it's useful for reading disassembly.

Reviewed-by: Andreas Baierl <ichgeh@imkreisrum.de>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4110>

gitlab-ci: add rules:changes for RADV

Including mesa_core_file_list is probably not the best but it's
better than nothing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4117>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4117>

gallium: hud_context: Fix scalar initializer warning.

When trying to build mesa/master under AOSP, I've run into the
following error:

external/mesa3d/src/gallium/auxiliary/hud/hud_context.c:1821:31: error: braces around scalar initializer [-Werror,-Wbraced-scalar-init]
   struct sigaction action = {{0}};
                              ^~~
1 error generated.

This patch addresses this by switching to using memset instead of
using an initializer.

Signed-off-by: John Stultz <john.stultz@linaro.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4141>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4141>

panfrost: Move pan_afbc.c file to the the right Makefile.source file

It seems pan_afbc.c was added to the wrong Makefile.sources file.

So fix this, so we don't run into build issues with mesa/master
trying to build under AOSP.

Signed-off-by: John Stultz <john.stultz@linaro.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4141>

freedreno: Add ir3_cf.c and ir3_delay.c to Makefile.sources

This patch adds missing ir3_cf.c and ir3_delay.c files to
the Makefile.sources file to address build issues seen
when trying to build mesa/master on AOSP

Signed-off-by: John Stultz <john.stultz@linaro.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4141>

gallium/cso_context: remove cso_delete_xxx_shader helpers to fix the live cache

With the live shader cache, equivalent shaders can be backed by the same
CSO. This breaks the logic that identifies whether the shader being deleted
is bound.

For example, having shaders A and B, you can bind shader A and delete
shader B. Deleting shader B will unbind shader A if they are equivalent.

Pierre-Eric figured out the root cause for this issue.

Fixes: 0db74f479b9 - radeonsi: use the live shader cache
Closes: #2596
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4078>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4078>

vulkan/wsi: fix cleanup when dup() fails

Fixes: f5433e4d6ce247b86dae ("vulkan/wsi: Add modifiers support to wsi_create_native_image")
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4137>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4137>

Revert "gallium: make handles of set_global_binding 64 bit"

This reverts commit e1ffb72a05f9b50ee47767aaadbab3e47896ee14

gallium: make handles of set_global_binding 64 bit

needed by CL

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4072>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4072>

pan/bi: Implement comparison opcodes via BI_CMP

Pretty straightforward for the moment. Ideally these would be fused into
csel/branches but that will come a bit later.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Print source types unconditionally

We track them all now, let's use them.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Specify comparison op for BI_CMP

...and adjust printing so we can use it as an op name.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Lower b2f to bcsel

Since we can get a zero for free and a one inlined into the constant,
the obvious turns out to be efficient (while allowing flexibility for
boolean size).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Implement nir_op_bcsel

No condition fusing yet.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Import algebraic pass from midgard

We'll need some of these at least.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Add isub op

Can't be a regular ADD since there's no negate modifier for integers
(it's a different opcode entirely).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Disable lower_sub

For float, fixing up the modifier ourselves is easy. For int, we have a
dedicated isub instruction anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Implement fabs, fneg as fmov with mods

Fusing will come later with the appropriate NIR support.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Handle special ops in NIR->BIR

Only on supported GPUs at the moment; for older Bifrost that don't
support these, I'm not sure yet where the right place to do the lowering
is. NIR algebraic rules would be "nice" but probably impractical -- but
it wouldn't be hard to do it directly in BIR (as a lowering pass or
alternative implementation).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Add BI_SPECIAL_* enum

To disambiguate the different special ops from each other.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Add a bunch of ALU ops

These are all regular ALU ops found in GLES2 which makes them
particularly nice targets at the moment. Just translate straight to our
IR.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Implement fsat as mov.sat

Soon we'll have a NIR support to handle this the Right Way along with
pos and sat_signed support, but we'll always need the fallback anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Allow inlining constants

This will allow us to optimize out the constant moves (although that
will require a DCE pass which has yet to be written).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Add initial handling of ALU ops

We do the bare minimum translation, just enough for fmov/fadd/fmul right
now with no modifiers / inlined constants / etc. The rest is to come!
But hopefully I got bitsize handling right this time around.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Lower vec* to writemasks in NIR

I was hoping not to tread down this path but it seems inevitable now.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Remove bi_load

This is now made redundant with writemasks, so let's regularize the IR.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Introduce writemasks

I feel so dirty. But this will let the IR be a lot more flexible seeing
as we really are vector in a certain sense (I/O, small types)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

pan/bi: Generalize swizzles to avoid extracts

We'd really rather not emit extracts. We are approaching on a vector IR
anyway which is annoying but really necessary to handle I/O and fp16
correctly. So let's just go all the way and deal with swizzles and masks
within reason; it'll still be somewhat saner in the long-term.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

panfrost: Move mir_to_bytemask to common code

...also so we can start sharing code properly between the panfrost
compilers.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4139>

freedreno/fdperf: set locale

Set local to get numbers printed w/ commas.. much easier to read that
way.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4119>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4119>

freedreno/computerator: add performance counter support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4119>

vulkan/wsi: Return an error if dup() fails

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4135>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4135>

vulkan/wsi: Don't leak the FD when GetImageDrmFormatModifierProperties fails

Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4135>

freedreno/ir3: try to avoid syncs

Update postsched to be better aware of where costly (ss) syncs would
result.  Sometimes it is better to allow a nop or two, to avoid a
sync quickly after an SFU.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: round-robin RA

In the second (scalar pass) use the information about # of registers
used in the first pass as the target max, and round-robin within that
range.  This generally gives the post-RA sched pass more opportunities
to re-order instructions to remove nop's.

Also, we can be a bit clever when assigning dest registers for SFU
instructions, by picking the register used for it's src (if available
and already assigned).  This avoids some (ss) syncs caused by write
after read hazards.  (Ie. the SFU instruction will read it's own src
before writing dest.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: track register usage in first RA pass

We'll use the feedback from the first pass to select a target register
usage in the second pass.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: fix has_latency_to_hide

Also count tex-prefetch instructions.  And only let the no-latency rule
kick in for frag shaders.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: split out has_latency_to_hide()

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

util/ra: move NO_REG to header

In the select_reg callback, I want to be able to determine if a given
node is already assigned, and if so what physical register has been
assigned.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

util/ra: spiff out select_reg_callback

Add a parameter so the callback can know which node it is selecting a
register for.  And remove the graph parameter, as it is unused by
existing users, and somewhat unnecessary (ie. the callback data could
be used instead).

And add a comment so $future_me remembers how this works.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno: fix FD_MESA_DEBUG=inorder

Fixes: 2c07e03b792 ("freedreno: allow ctx->batch to be NULL")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: add simplified stall estimation

Doesn't take into account stalls that result from a register written in
a different block, etc.  But this should be more useful than just using
number of (ss)'s by trying to estimate how costly a given sync is.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: remove extra nops inserted in scheduler

They were inserting a nop between back to back SFU instrucions.  But
that doesn't actually appear to be required.  And they get stripped out
later anyways before legalize.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/computerator: add hrsq/hlog2/hexp2

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

freedreno/ir3: also lower lowp frag outputs

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

nir/print: show variable precision

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4071>

intel/tools: Fix compilation with UBSan

Compilation failed with several similar errors:

../src/intel/tools/aub_read.c:322:4: error: case label does not reduce to an integer constant
  322 |    case MAKE_HEADER(TYPE_AUB, OPCODE_AUB, SUBOPCODE_HEADER):

Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4132>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4132>

i965: Use gl_vertex_format in brw_vertex_element.

State upload needs to cope with the vertex format
rather than with the full attribute data.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Make use of the vertex format functions in i965.

v2: Style fixes.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

mesa: Provide gl_vertex_format accessors.

Provide the same set of VAO and current value gl_vertex_format
accessor functions like we have for the gl_array_attributes.
For most purpose the vertex format is what we need.

v2: Style fixes.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

mesa: Remove now unused _mesa_draw_attrib.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

mesa: Remove now unused _mesa_draw_attrib_and_binding.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Remove glbinding from brw_vertex_element.

v2: Rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Reorder workaround flags computation.

Vertex processing workaround flags can be split into
array and current vertex attributes. By that we
can use specific access functions for these different
vertex attribute kinds. This finally obsoletes
some access functions that I introduced last winter
for a smooth transition.

v2: Style fixes.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Split merge_inputs and clear_buffers.

The merge_inputs function handles that part that changes when the
inputs change. The clear_buffers function triggers when we may need
a new upload. Thus the merge_inputs can be limited to be once
per brw_draw_prims.

v2: Move declaration of attribute index into the for scope.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Test original vertex array pointer to skip array upload.

Rather than do a NULL pointer check on a pointer that may be offset by the
min-max index range of an GL draw operation, execute the NULL test on the
original vertex array pointer.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Use the VAOs binding information in array setup.

The change basically reimplements array setup by walking
the gl_contex::Array._DrawVAO on a per binding sequence.
In this way we can make direct use of the application
provided minimum set of buffer objects and emit fewer relocs.

v2: Rebase onto:
    compiler: Move double_inputs to gl_program::DualSlotInputs
v3: Rebase onto introduction of gl_vertex_format
v4: Reorder and extend patch series.
v5: Split out two hunks into seperate patches.
v6: Avoid using GL* types.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Use 32 bit u_bit_scan for vertex attribute setup.

The vertex array object contains 32 vertex arrays. By that we cannot
reference more then these in the vertex shader inputs. So, we can use
the 32 bits u_bit_scan function to iterate the vertex shader inputs
and place an assert that only these are present. Also place an other
assert that the vertex array setup in i965 does not overrun the
enabled array in brw_context::vs::enabled.

v2: Style fixes.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

iris: Move down iris_emit_sbe_swiz in profiles.

Harvest the information gathered in the previous patch
inside of iris.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

i965: Move down genX_upload_sbe in profiles.

Avoid looping over all VARYING_SLOT_MAX urb_setup array
entries from genX_upload_sbe. Prepare an array indirection
to the active entries of urb_setup already in the compile
step. On upload only walk the active arrays.

v2: Use uint8_t to store the attribute numbers.
v3: Change loop to build up the array indirection.
v4: Rebase.
v5: Style fix.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/308>

panfrost: Get rid of ctx->payloads[]

Now that vertex/tiler payloads are re-initialized at draw/launch_grid
time we can get of of the ctx->payloads[] field and allocate those
payload templates on the stack.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4083>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4083>

panfrost: Use ctx->active_prim in panfrost_writes_point_size()

Check ctx->active_prim instead of prefix.draw_mode so we can eventually
get rid of ctx->payloads.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4083>

panfrost: Re-init the VT payloads at draw/launch_grid() time

Doing that should help us avoiding state leaks between draw/launch_grid
calls.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4083>