freedreno: move a4xx specific layout code to a4xx code

Every other gen has its own setup_slices

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4611>

tests: Make tests aware of meson test wrapper

Meson 0.55.0 will set the MESON_EXE_WRAPPER environment variable to the
joined version of that wrapper if it is needed. Our tests that take
compiled targets as arguments can use that information to run cross
built binaries, or if there isn't a wrapper and we get an ENOEXEC, we
can skip the tests gracefully.

We try to use mesonlib.split_args, which handles windows arguments
better than python's builtin shlex module, but fall back to that if the
meson module isn't available for some reason.

Cc: 20.0 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5103>

gallium/auxiliary/vl: Fix compute shader scale_y for interlaced videos

Signed-off-by: Thong Thai <thong.thai@amd.com>
Fixes: 494b7ef0c1a ("gallium/auxiliary/vl: Fix compute shader scaling for non-square pixels")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5121>

pan/mdg: Optimize liveness computation in DCE

Rather than recompute liveness every block, compute it just once for the
whole shader, which ends up more efficient.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5123>

pan/mdg: Precompute mir_special_index

Rather than O(N) each call, we can precompute the whole set - also O(N)
- and then subsequent checks are O(1).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5123>

pan/mdg: Optimize pipelining logic

The test and rewrite were both accidentally O(N) to the shader size when
they should be O(1), so overall this takes the pass from O(N^2) to O(N).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5123>

pan/mdg: Emit fcsel when beneficial

If there are floating point modifiers, we emit fcsel instead of icsel
(and likewise if integer modifiers, icsel instead of fcsel) to minimize
redundant instructions.

total instructions in shared programs: 3628 -> 3626 (-0.06%)
instructions in affected programs: 139 -> 137 (-1.44%)
helped: 2
HURT: 0

total bundles in shared programs: 1886 -> 1885 (-0.05%)
bundles in affected programs: 19 -> 18 (-5.26%)
helped: 1
HURT: 0

total quadwords in shared programs: 3319 -> 3317 (-0.06%)
quadwords in affected programs: 127 -> 125 (-1.57%)
helped: 2
HURT: 0

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5123>

intel/aub_error_decoder: print driver identifier if found

You can find it right before the application batch :

HuC firmware: i915/kbl_huc_ver02_00_1810.bin
        status: fetch NONE, load NONE
        version: wanted 2.0, found 0.0
        header: offset 0, size 0
        uCode: offset 0, size 0
        RSA: offset 0, size 0
Driver identifier: i965 20.0.0-devel
--- batch buffer (rcs0 (submitted by glxgears [44455])) at 0x0000fffe ec000000
0xfffeec000000:  0x70000007:  MEDIA_VFE_STATE
0xfffeec000000:  0x70000007 : Dword 0
    DWord Length: 7
0xfffeec000004:  0x00000000 : Dword 1
    Per Thread Scratch Space: 0

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

anv: add identifier BO

A buffer added to all execbufs so that we can attribute a batch that
caused a hang to a particular driver.

v2: Reuse workaround BO

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

i965: add identifier BO

A buffer added to all execbufs so that we can attribute a batch that
caused a hang to a particular driver.

v2: Reuse workaround BO

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

iris: add identifier BO

A buffer added to all execbufs so that we can attribute a batch that
caused a hang to a particular driver.

v2: Reuse workaround BO

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

intel: add identifier for debug purposes

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

i965: store workaround_bo offset

This offset store the location where we read/write into the
workaround_bo. It will allow to select a different address later,
leaving the beginning of the buffer to some other use.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

iris: store workaround address

This will allow to select a different address later, leaving the
beginning of the buffer to some other use.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

anv: store the workaround address

This will allow to select a different address later, leaving the
beginning of the buffer to some other use.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

blorp: rename workaround address function

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

anv: fixup unwinding of device create failure

We appear to have the ordering mixed up a bit.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3203>

panfrost: Enable PIPE_CAP_VERTEX_COLOR_UNCLAMPED

This tells Mesa to clamp vertex colours in the vertex shader.

This improves rendering in a number of games such as Extreme Tux
Racer and H-Craft Championships.

Cc: 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5075>

glsl_type: don't serialize padding bytes from glsl_struct_field

This should fix such valgrind warnings:
==37417== Uninitialised byte(s) found during client check request
==37417==    at 0x6183471: blob_write_bytes (blob.c:163)
==37417==    by 0x629785B: encode_type_to_blob (glsl_types.cpp:2760)
==37417==    by 0x61E68D8: write_variable (nir_serialize.c:293)
==37417==    by 0x61E6F6A: write_var_list (nir_serialize.c:421)
==37417==    by 0x61EBA7A: nir_serialize (nir_serialize.c:2018)
==37417==    by 0x5B5E007: serialize_nir_part (brw_program_binary.c:135)
==37417==    by 0x5B5E7F3: brw_serialize_program_binary (brw_program_binary.c:299)
==37417==    by 0x5FEF5FF: write_program_payload (program_binary.c:177)
==37417==    by 0x5FEF7BB: _mesa_get_program_binary_length (program_binary.c:225)
==37417==    by 0x5E3D31D: get_programiv (shaderapi.c:912)
==37417==    by 0x5E3F730: _mesa_GetProgramiv (shaderapi.c:1827)
==37417==    by 0x111DA0: program_binary_save_restore (shader_runner.c:686)
==37417==  Address 0x8f59481 is 81 bytes inside a block of size 480 alloc'd
==37417==    at 0x483B7F3: malloc (vg_replace_malloc.c:309)
==37417==    by 0x618CE67: ralloc_size (ralloc.c:123)
==37417==    by 0x618CF35: rzalloc_size (ralloc.c:155)
==37417==    by 0x618D245: rzalloc_array_size (ralloc.c:234)
==37417==    by 0x629041D: glsl_type::glsl_type(glsl_struct_field const*, unsigned int, glsl_interface_packing, bool, char const*) (glsl_types.cpp:148)
==37417==    by 0x6293EC3: glsl_type::get_interface_instance(glsl_struct_field const*, unsigned int, glsl_interface_packing, bool, char const*) (glsl_types.cpp:1271)
==37417==    by 0x604C878: (anonymous namespace)::per_vertex_accumulator::construct_interface_instance() const (builtin_variables.cpp:365)
==37417==    by 0x6050722: (anonymous namespace)::builtin_variable_generator::generate_varyings() (builtin_variables.cpp:1568)
==37417==    by 0x60509CA: _mesa_glsl_initialize_variables(exec_list*, _mesa_glsl_parse_state*) (builtin_variables.cpp:1600)
==37417==    by 0x6149AE9: _mesa_ast_to_hir(exec_list*, _mesa_glsl_parse_state*) (ast_to_hir.cpp:131)
==37417==    by 0x60706D6: _mesa_glsl_compile_shader (glsl_parser_extras.cpp:2222)
==37417==    by 0x5E3DC16: _mesa_compile_shader (shaderapi.c:1211)

==37417== Use of uninitialised value of size 8
==37417==    at 0x529AE13: ??? (in /usr/lib/x86_64-linux-gnu/libz.so.1.2.11)
==37417==    by 0x6184075: util_hash_crc32 (crc32.c:127)
==37417==    by 0x5FEF401: write_program_binary (program_binary.c:95)
==37417==    by 0x5FEF8BC: _mesa_get_program_binary (program_binary.c:252)
==37417==    by 0x5E40E22: _mesa_GetProgramBinary (shaderapi.c:2411)
==37417==    by 0x4914057: stub_glGetProgramBinary (piglit-dispatch-gen.c:24737)
==37417==    by 0x111E4A: program_binary_save_restore (shader_runner.c:704)
==37417==    by 0x11F765: piglit_display (shader_runner.c:5112)
==37417==    by 0x499082F: run_test (piglit_fbo_framework.c:52)
==37417==    by 0x4980E89: piglit_gl_test_run (piglit-framework-gl.c:229)
==37417==    by 0x110DA9: main (shader_runner.c:72)

v2: - decode_glsl_struct_field_from_blob and
    encode_glsl_struct_field should be `static`
    ( Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> )
v3: - we can get rid of `struct packed_struct_field_flags`
    ( Tapani Pälli <tapani.palli@intel.com> )
    - we can get rid of `unsigned __pad: 15` bitfield
    ( Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> )

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Signed-off-by: Andrii Simiklit <asimiklit.work@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5054>

turnip: enable 422_UNORM formats

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4590>

turnip: implement VK_KHR_sampler_ycbcr_conversion

Most changes based on radv, some simplification, since we don't need to
sample multiple planes, 422_UNORM/420_UNORM formats will be supported
directly using the hardware formats for those.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4590>

freedreno/registers: document 422_UNORM and 420_UNORM formats

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4590>

util/format: translate 422_UNORM and 420_UNORM vulkan formats

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4590>

intel/perf: repurpose INTEL_DEBUG=no-oaconfig

We initially used this debug option to mean "don't bother registering
the OA configuration into the kernel".

This change makes this option suppress any interaction with the
i915/perf interface. This is useful when debugging self modifying
batches with performance queries while running on the intel_mi_runner.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

anv: Implement VK_KHR_performance_query

This has the same kernel requirements are VK_INTEL_performance_query

v2: Fix empty queue submit (Lionel)

v3: Fix autotool build issue (Piotr Byszewski)

v4: Fix Reset & Begin/End in same command buffer, using soft-pin &
    relocation on the same buffer won't work currently. This version
    uses a somewhat dirty trick in anv_execbuf_add_bo (Piotr Byszewski)

v5: Fix enumeration with null pointers for either pCounters or
    pCounterDescriptions (Piotr)
    Fix return condition on enumeration (Lionel)
    Set counter uuid using sha1 hashes (Lionel)

v6: Fix counters scope, should be COMMAND_KHR not COMMAND_BUFFER_KHR (Lionel)

v7: Rebase (Lionel)

v8: Rework checking for loaded queries (Lionel)

v9: Use new i915-perf interface

v10: Use anv_multialloc (Jason)

v11: Implement perf query passes using self modifying batches (Lionel)
     Limit support to softpin/gen8

v12: Remove spurious changes (Jason)

v13: Drop relocs (Jason)

v14: Avoid overwritting .sType in
     VkPerformanceCounterKHR/VkPerformanceCounterDescriptionKHR (Lionel)

v15: Don't copy the entire
     VkPerformanceCounterKHR/VkPerformanceCounterDescriptionKHR (Jason)
     Reuse anv_batch rather than custom packing (Jason)

v16: Fix missing MI_BB_END in reconfiguration batch
     Only report the extension with kernel support (perf_version >= 3)

v17: Some cleanup of unused stuff

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: reuse offset specified in the query

The current code relies on the order of the function
gen_perf_query_result_accumulate() to match the descriptions written
by gen_perf.py. Let's just reuse the offset specified in the python
script.

v2: Use accumlator offsets more (Jason)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

anv: use a query filled by the perf code

We're about to use the offset fields from the query object. We can't
just use a made up object.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: report whether the platform supported

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: add counter category to generated code

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: add helper to compute metrics from counters

The produced array tells use what metric to enable for a given pass.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: emit counter units in generated code

We'll use this coming extension.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: compute number of passes for a set of counters

We want to compute the number of passes required to gather performance
data about a set of counters.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: create a unique list of counters

For a future extension we want to be able to list the counters. Our
existing sets counters might contain the same counters multiple times.
This is a side effect of the fixed OA counters in the HW. We track
thoses with a mask so that we know when a counter is available from
multiple metrics.

v2: Use BITFIELD64_BIT() (Jason)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: update generated code to ralloc all data

Previously counter descriptions as well register values were written
in global static variables. This isn't really thread safe so instead
ralloc all the data back under the gen_perf_config object.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: store the appropriate OA formats in queries

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/perf: make pipeline statistic query loading optional

On Vulkan most of those are already covered by standard queries so
add the ability to skip them.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/genxml: add PIPE_CONTROL command cache invalidate bit

This new bit invalidates the cache/prefetch of commands in the command
streamer. This will be useful for self modifying batches.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

anv: add a new execution mode for secondary command buffers

This change adds a call/return execution mode for secondary command
buffer rather than the existing copy into the primary batch mode.

v2: Rework convention to avoid burning an ALU register (Jason)

v3: Use anv_address_add() (Jason)

v4: Move command emissions to anv_batch_chain.c (Jason)

v5: Also move last MI_BBS emission in secondary command buffer to
    anv_batch_chain.c (Jason)

v6: Fix end secondary command buffer end (Jason)

v7: Refactor anv_batch_address() to remove additional emit functions

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

anv: don't reserve a particular register for draw count

By using the same mi_builder throughout the draw call, we can just
allocate a register from the mi_builder and unref it when we're done.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/mi-builder: add framework for self modifying batches

v2: Use Jason's idea to store addresses to modify

v3: Add ALU flushes (Jason)

v4: Remove ALU flush from gen_mi_self_mod_barrier() (Jason)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v2)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

intel/genxml: fix bits generation for MI_LOAD_REGISTER_IMM

This instruction has a group with the same name than another field above :

  <field name="Data DWord" start="64" end="95" type="uint"/>
  <group count="0" start="96" size="64">
    <field name="Register Offset" start="2" end="22" type="offset"/>
    <field name="Data DWord" start="32" end="63" type="uint"/>
  </group>

The script was replacing the offset of the field first with the second
one in the group.

This change ignore anything a group within an instruction.

v2: Drop unused variable (Rafael)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rafael Antognolli <rafael.antognolli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2775>

gitlab: Ask about reproduction rate in the issue template

Reviewed-by: <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5055>

nir: Add const to nir_intrinsic_src_components

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5108>

pan/mdg: Apply outmods

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Use helpers for branch/discard inversion

Doesn't come up on glmark but would covered by the old passes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Remove invert optimizations

Unused since last commit.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Treat inot as a modifier

With this, we may remove all invert passes and simply look at the src
modifier on NIR->MIR and fixup at pack time. No shader-db changes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Apply abs/neg modifiers

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Ingest fsat_signed/fclamp_pos

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Prepare for modifier helpers

We have to restructure to ensure NIR->MIR does not mutate the NIR and to
allow passing around dest/outmods for the new helpers. If NIR->MIR were
better designed this would be easier. Sigh.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Drop nir_lower_to_source_mods

shader-db regressions fixed shortly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

pan/mdg: Remove .pos propagation pass

Will be replaced later in the series. shader-db regressions but those
fixed momentarily.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

panfrost: Add modifier detection helpers

With the goal of removing modifiers from NIR, these helpers let us
detect modifier patterns without mutating the underlying NIR. These were
intended for upstream, but due to various issues are being (temporarily)
vendored.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

nir: Add fclamp_pos opcode

Corresponds to the .pos modifier on all Mali GPUs (lima and panfrost).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

nir: Add fsat_signed opcode

Exists on later Mali. Equivalent to clamp(x, -1.0, 1.0)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5102>

tu: Support VK_FORMAT_FEATURE_BLIT_SRC_BIT for texture-only formats

It turns out this is required for compressed formats, and we might as
well enable it for the one other texture-only format too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5098>

tu: Fix buffer compressed pitch calculation with unaligned sizes

We can just set the extent and not bufferRowLength/bufferImageHeight,
and the extent may not be a multiple of the block size if it covers the
entire image. In this case we have to first divide to get the
width/height in terms of blocks, and then multiply by the block size to
get the buffer's pitch and layer size. Multiplying and dividing instead
won't get the correct result when the extent covers the entire image and
isn't a multiple of the block size. This also makes the code easier to
follow because we don't calculate a pitch in non-sensical units (bytes
times the block width) as an intermediate step.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5098>

tu: Fall back to 3d blit path for BC1_RGB_* formats

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5098>

tu: Always initialize image_view fields for blit sources

Previously we only supported BLIT_SRC_BIT and BLIT_DEST_BIT together, so
we didn't have to worry about initializing blit-related fields for
texture-only formats, but it turns out that 2d blits work out just fine
with these formats and we'll need to enable BLIT_SRC_BIT for
texture-only formats due to a Vulkan requirement on compressed formats.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5098>

nir: Add a store_reg helper and use the builder in phis_to_regs

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5094>

nir: Add a new helper for iterating phi sources leaving a block

This takes the same callback as nir_foreach_src except it walks all phi
sources which leave a given block.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5094>

nir/clone: Re-use clone_alu for nir_alu_instr_clone

All it takes are a couple small tweaks to the clone infrastructure to
allow us to use it without any remap table at all.  This reduces code
duplication and the chances for bugs that come with it.  In particular,
the hand-rolled nir_alu_instr_clone didn't preserve no_[un]signed_wrap,
or source/destination modifiers.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5094>

radv/winsys: Finish mapping for sparse residency.

This adds the part that disables pagefaults when unbacked sparse
textures get accessed.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5079>

intel/drm-shim: Return correct values for I915_PARAM_HAS_ALIASING_PPGTT

It sure looks like it should be a Boolean value, but it's not.  The
values that we really want for later platforms are either 2 or 3.  The
old intel_stub.c in shader-db just always returns 3
(I915_GEM_PPGTT_FULL).  This returns the same set of values per platform
that kernel 5.6.13 would.

When using the shim for ICL with i965 driver, this fixes:

    i965 requires softpin (Kernel 4.5) on Gen10+.

Fixes: 0f4f1d70bfe ("intel: add stub_gpu tool")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5061>

intel/drm-shim: Add noop ioctl handler for set_tiling

When using the shim for HSW and earlier, this fixes:

    DRM_SHIM: unhandled driver DRM ioctl 33 (0xc0106461)

Fixes: 0f4f1d70bfe ("intel: add stub_gpu tool")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5061>

radv: Expose VK_EXT_pipeline_creation_cache_control.

Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2972
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5072>

radv: Support VK_PIPELINE_CACHE_CREATE_EXTERNALLY_SYNCHRONIZED_BIT_EXT.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5072>

radv: Support VK_PIPELINE_CREATE_EARLY_RETURN_ON_FAILURE_BIT_EXT.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5072>

radv: Support VK_PIPELINE_COMPILE_REQUIRED_EXT.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5072>

panfrost: Enable AFBC for Z24X8

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5069>

panfrost: Fix Z24 vs Z32 mixup

We don't actually support Z32_UNORM; the format we've been using as such
is in fact Z24X8 / Z24S8. Fix that and drop Z32_UNORM.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5069>

panfrost: Switch formats to table

Rather than heuristically guessing what PIPE formats correspond to what
in the hardware, hardcode a table. This is more verbose, but a lot more
obvious -- the previous format support code was a source of endless
silent bugs.

v2: Don't report RGB233 (icecream95). Allow RGB5 for texturing
(icecream95).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5069>

pan/mfbd: Add format codes for PIPE_FORMAT_B5G5R5A1_UNORM

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5069>

nir/opt_if: use nir_src_as_bool in opt_peel_loop_initial_if helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4757>

nir/opt_if: run opt_peel_loop_initial_if after all other optimizations

Fixes dEQP-VK.graphicsfuzz.loops-ifs-continues-call with RADV.

opt_if_loop_terminator can cause this optimization or
opt_if_simplification to be run on the non-SSA code.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Fixes: 52c8bc0130a ('nir: make opt_if_loop_terminator() less strict')
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2943
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4757>

nir: Add documentation for each jump instruction type

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>

nir: Use a switch statement in nir_handle_add_jump

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>

nir: Validate jump instructions as an instruction type

This has the downside of putting block successor validation in two
places that are a bit further apart.  However, handling them as a
special case makes the code more confusing than needed.  At least two
different people have not noticed that we don't have jump instruction
validation in the last week or two and added it.  Being able to search
for validate_jump_instr is useful.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>

radv/aco: enable storageInputOutput16 on GFX9+

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: fix off-by-one error with 16-bit MTBUF opcodes on GFX10

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: implement 16-bit interp

For 16-bit bank LDS (ie. Kabini/Stoney) we need a slightly different
path. It's completely untested though because I don't have these
chips but according to vkpipeline-db the generated assembly seems fine.

Note that 16-bit I/O is currently only exposed on GFX9+ for both
compiler backends.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: emit v_interp_*_f16 instructions as VOP3 instead of VINTRP

This adds a separate emission path in the assembly for the 16-bit
interp instructions.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: validate v_interp_*_f16 as VOP3 instructions instead of VINTRP

16-bit interp instructions are considered VINTRP by the compiler
but they are emitted as VOP3 by the assembler.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: implement 16-bit vertex fetches with tbuffer_load_format_d16_*

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: implement 8-bit/16-bit mov's with p_create_vector

ACO doesn't lower 8-bit/16-bit mov's in NIR.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2997
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: allow to load/store 16-bit values in VMEM for tess and geom

We only have to adjust some assertions to allow storing/loading
16-bit values.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: convert 16-bit values before exporting MRTs

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

aco: store 16-bit temporary outputs as v2b

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>

Expose EGL_KHR_platform_* when EXT is supported

On EGL 1.4, one had to check for the existence of EGL_EXT_platform_base
before querying the eglGetPlatformDisplayEXT() and
eglCreatePlatformWindowSurfaceEXT() symbols, to then use them if the
EGL_EXT_platform_* extension for the given platform was exposed.

Since EGL 1.5, the platform functionality was made core, which means we
can obtain the symbols unconditionally, but we can't know the EGL
version before having created a display, at which point we've already
done a platform selection by passing an EGLNativeDisplay.  The
EGL_KHR_platform_* extensions thus are used by clients to know whether
it's safe or not to dlsym() the EGL 1.5 symbols.

This commit adds those extensions when the given platform is enabled.

Acked-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5052>

pan/decode: Fix min/max_tile_coord mixup

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5099>

pan/decode: Use a page table for tracking mmaps

We create a hash table mapping GPU va's to mmap structures, such that
searching for a mapped address is effectively O(1) rather than O(N) to
the number of mapped entries as with the previous linked list approach.
This is a memory-time tradeoff, but the speed-up is tracing is notable.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5099>

freedreno/ir3/validate: add checking for types and opcodes

For cases where instructions have a src and/or dst type, validate that
it matches the src/dst register types.  And for cases where there are
different opcodes for half vs full, validate that the opcode matches.

Now that we maintain this properly throughout the stages of the ir, we
can drop the fixups from the RA pass.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3: add helpers to deal with src/dst types

Add some helpers to properly maintain src/dst types, and in the cases
where opcode depends on src or dst type, maintain that as well.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3: add simple validate pass

We can add to this as we notice other things that are worth validating
between ir3 passes.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3: fix mismatched wrmask for overlapping VS inputs

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3/cp: fix cmps folding

When we start doing cp iteratively, we hit the case that we've already
`cmps.s.*` into a `cmps.s.ne p0.x, ...`..  when we try to do that again
we can invert the logic condition.  So check specifically the condition
to prevent this.

TODO we could maybe be more clever about this to combine conditions.
But why isn't that happening in nir?  For example, see
dEQP-GLES31.functional.ssbo.layout.single_basic_array.packed.bool

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3/print: print cat2 condition

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3: fix immed type in create_addr0()

We can also remove a bunch of manual src/dst flag munging, since the
instruction builders handle this automatically now.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3/cf: handle multiple cov's properly

There can be multiple (for ex.) f32f16's from a single source, in
particular appearing in different blocks.  We need to update all uses
of the src which had conversion folded in, not all the uses of the
individual cov.  Also, to avoid invalidating the ssa use info that was
gathered at the beginning of the pass, don't actually eliminate the
cov, but instead change it to a simple mov that the cp pass can gobble
up.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>

freedreno/ir3: fix mismatched flags on split

We have to fixup the meta:split half flag, because `ir3_split_dest()` is
called before we fixup the dest type.  But we should fixup both the
split src and dest, as well as the thing it is splitting.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5048>