radv: Keep shader info when needed.

This allows enabling the shader info keeping on a per shader basis.
Also disables the cache on a per shader basis.

Reviewed-by: Dave Airlie <airlied@redhat.com>

radv: Add VK_KHR_pipeline_executable_properties in disabled state.

So we can add the functions.

Reviewed-by: Dave Airlie <airlied@redhat.com>

radv: Use string for nir dumping.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Allows us to easily dump all nir shaders for combined variants in
vega and simplifies ownership.

radv: Get max workgroup size without nir.

Reviewed-by: Dave Airlie <airlied@redhat.com>

radv: Add utility function to calculate max waves.

Not AC because a lot of it is data extraction out of radv structs.

Reviewed-by: Dave Airlie <airlied@redhat.com>

iris/gen9: Optimize slice and subslice load balancing behavior.

See "i965/gen9: Optimize slice and subslice load balancing behavior."
for the rationale.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

intel/genxml: Add GT_MODE hashing defs for Gen9.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/gen9: Optimize slice and subslice load balancing behavior.

The default pixel hashing mode settings used for slice and subslice
load balancing are far from optimal under certain conditions (see the
comments below for the gory details).  The top-of-the-line GT4 parts
suffer from a particularly severe performance problem currently due to
a subslice load balancing issue.  Fixing this seems to improve
graphics performance across the board for most of the benchmarks in my
test set, up to ~20% in some cases, e.g. from SKL GT4:

unigine/valley:                3.44% ±0.11%
gfxbench/gl_manhattan31:       3.99% ±0.13%
gputest/pixmark_piano:         7.95% ±0.33%
synmark/OglTexFilterAniso:    15.22% ±0.07%
synmark/OglTexMem128:         22.26% ±0.06%

Lower-end platforms are also affected by some subslice load imbalance
to a lesser degree, especially during CCS resolve and fast clear
operations, which are handled specially here due to rasterization
ocurring in reduced CCS coordinates, which changes the semantics of
the pixel hashing mode settings.

No regressions seen during my tests on some SKL, KBL and BXT
configurations.  Additional benchmark reports welcome on any Gen9
platforms (that includes anything with Skylake, Broxton, Kabylake,
Geminilake, Coffeelake, Whiskey Lake, Comet Lake or Amber Lake in your
renderer string).

P.S.: A similar problem is likely to be present on other non-Gen9
      platforms, especially for CCS resolve and fast clear operations.
      Will follow-up with additional patches fixing the hashing mode
      for those once I have enough performance data to justify it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

pan/midgard: Handle 64-bit address in mir_mask_of_read_components

This is a bit of a hack, but it'll hold us over until we have 64-bit
support wired through.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Allocate separate spill indices for lowered moves

This helps RA be slightly more reasonable.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Extend liveness analysis to trinary ops

Fixes RA fails with multiple indirect SSBO writes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Fix load/store pairing

This used a delicate hack to try to find indirect inputs and skip them
as candidates for pairing. Let's use a better criterion -- no sources --
and pair based on that.

We could do better, but that would require more complex data flow
analysis than we're interested in doing here.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Implement nir_intrinsic_load_num_work_groups

Just a sysval to route through.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Implement some compute builtins

We implement gl_WorkGroupID and gl_LocalInvocationID, which map to
ld_compute_id with special sources.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Rename ld_global_id -> ld_compute_id

It's used for more general loads within a compute shader.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Handle partial writes in liveness analysis

This allows liveness analysis within a loop to be more fine grained,
fixing RA failures with partial spilled movs within a loop, as well as
enabling a slight reduction of register pressure more generally:

total registers in shared programs: 350 -> 347 (-0.86%)
registers in affected programs: 12 -> 9 (-25.00%)
helped: 3
HURT: 0
helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1
helped stats (rel) min: 25.00% max: 25.00% x̄: 25.00% x̃: 25.00%

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Dump "no spill"?

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Absorb nonexistance sources

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Pretty-print destinations

They're not "sources" but they follow the same conventions.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Pretty-print units

Since we are seeing some use of MIR post-scheduling, let's get this
printed right.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Print mask in dumped MIR

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Add no_spill flag

Hint for the RA to avoid infinite spilling loops.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Generalize mir_mask_of_read_components

This now works for load/store and texture instructions as well as ALU.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Implement SSBO access

Just laying the groundwork. Reads and writes should be supported (both
direct and indirect, either int or float, vec1/2/3/4), but no bounds
checking is done at the moment.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Pipe uniform mask through when spilling

This is a corner case that happens a lot with SSBOs. Basically, if we
only read a few components of a uniform, we need to only spill a few
components or otherwise we try to spill what we spilled and RA hangs.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Clamp sysval component count

We don't want to load a 128-bit sysval when 64-bits will do. Fixes RA
failures with SSBO indirect writes.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Pass uploaded midgard_instruction through

We want to edit it after emission in some cases.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Allow sysval destination override

Sometimes a sysval is used to facilitate an instruction but is not the
instruction itself.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Force flush every compute job

This is of course suboptimal for performance, forcing each
glDispatchCompute call to be submitted separately to the kernel and
finish to completion. However, for the initial bring-up of compute jobs,
this simplifies quite a bit.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Add SSBO system value

For each SSBO index we get from Gallium/NIR, we need two pieces of
information in the shader:

1. The address of the SSBO in GPU memory. Within the shader, we'll be
accessing it with raw memory load/store, so we need the actual address,
not just an index.

2. The size of the SSBO. This is not strictly necessary, but at some
point, we may like to do bounds checking on SSBO accesses.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

gallium/util: Add u_stream_outputs_for_vertices helper

This u_prim.h helper determines the number of outputs for stream output,
given a particular primitive type and a vertex count. This is useful for
statically calculating sizes of stream output buffers (i.e. when there
is no geometry/tessellation shader in use).

This helper will be used in Panfrost's transform feedback
implementation, as you can probably guess since why else would I be
submitting it....

See also dEQP's getTransformFeedbackOutputCount routine.

v2: Simplify definition using new helpers, which also extends to non-ES2
primitive types (Eric).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>

radeonsi: remove the always_nir option

tgsi_to_nir is no longer optional if NIR is enabled.

radeonsi/nir: implement default tess level system values

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

compiler: add SYSTEM_VALUE_TESS_LEVEL_OUTER/INNER_DEFAULT

TCS system values for internal passthru TCS, needed by radeonsi NIR support

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

gallium: add TGSI_SEMANTIC_DEFAULT_OUTER/INNER_LEVEL

for radeonsi NIR support.

tgsi_to_nir: handle tess level inner/outer varyings

for internal radeonsi shaders

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

tgsi_to_nir: add support for the stencil FS output

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

tgsi_to_nir: add support for TEX_LZ

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>

compiler: add SYSTEM_VALUE_USER_DATA_AMD

for internal radeonsi shaders

compiler: add shader_info.cs.user_data_components_amd

tgsi_to_nir: add basic compute shader support

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

tgsi_to_nir: add support for LOAD & STORE with SSBOs and images

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

tgsi_to_nir: make setup_texture_info reusable

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>

tgsi_to_nir: add support for TXF_LZ

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

compiler: add shader_info.vs.blit_sgprs_amd

for internal radeonsi shaders

tgsi_to_nir: be careful about not losing any TGSI properties silently (v2)

v2: squash with Timur Kristof's commit

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

tgsi/scan: don't set GS_INVOCATIONS for all shader stages

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

compiler: add ACCESS_STREAM_CACHE_POLICY

radeonsi will use this.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>

gallium: add AMD-specific compute TGSI enums

for tgsi_to_nir

gallium: add TGSI_PROPERTY_VS_BLIT_SGPRS_AMD for tgsi_to_nir

needed by radeonsi NIR support

st/mesa: don't allocate mipmapped texture for NEAREST_MIPMAP_LINEAR

Reviewed-by: Brian Paul <brianp@vmware.com>

glsl: Optimize the SoftFP64 shader when first creating it.

By optimizing the shader before inlining, we avoid having to redo this
work for each inlined copy of a function.  It should also reduce the
memory consumption a bit.

This cuts the KHR-GL46.arrays_of_arrays_gl.SubroutineFunctionCalls2
runtime by 25% on my Icelake.  That test compiles many shaders, which
contain large types (dmat4) and division (expensive operations).

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>

etnaviv: fix compile warnings in release build

[27/31] Compiling C object 'src/gallium/drivers/etnaviv/df32d18@@etnaviv@sta/etnaviv_compiler_nir.c.o'.
In file included from ../../src/gitlab_mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir.c:552:
../../src/gitlab_mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir_emit.h: In function 'ra_assign':
../../src/gitlab_mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir_emit.h:903:9: warning: unused variable 'ok' [-Wunused-variable]
    bool ok = ra_allocate(g);
         ^~
../../src/gitlab_mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir.c: In function 'etna_compile_shader_nir':
../../src/gitlab_mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir.c:663:9: warning: unused variable 'ok' [-Wunused-variable]
    bool ok = emit_shader(c->nir, &options, &v->num_temps, &num_consts);
         ^~

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>

radv: Do not setup attachments without a framebuffer.

Test that found this: dEQP-VK.geometry.layered.1d_array.secondary_cmd_buffer

Fixes: 49e6c2fb78c "radv: Store color/depth surface info in attachment info instead of framebuffer."
Reviewed-by: Dave Airlie <airlied@redhat.com>

anv: Implement VK_EXT_subgroup_size_control version 2

The version bump adds a proper features struct.

Fixes: d10de253097 "anv: Implement VK_EXT_subgroup_size_control"
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>

vulkan: Update the XML and headers to 1.1.119

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>

radv: Hash Wave32 settings in shader key.

Can result in different shaders.

Fixes: 8a86908e9a7 "radv/gfx10: add Wave32 support for vertex, tessellation and geometry shaders"
Reviewed-by: Dave Airlie <airlied@redhat.com>

radv: Properly use Wave64 for non-NGG GS and copy shader.

Fixes: 8a86908e9a7 "radv/gfx10: add Wave32 support for vertex, tessellation and geometry shaders"
Reviewed-by: Dave Airlie <airlied@redhat.com>

radv: Put wave size in shader options/info.

Instead of having the three values everywhere. This is also more
future proof if we want the driver to make those decisions eventually.

Reviewed-by: Dave Airlie <airlied@redhat.com>

relnotes: Make entries for radv more consistent.

Always use 'on' as for the rest of the drivers.

Reviewed-by: Dave Airlie <airlied@redhat.com>

relnotes: Add new exts on radv for 19.2.

Reviewed-by: Dave Airlie <airlied@redhat.com>

iris: reorder arguments as expected by the function

CID: 1452262
Fixes: b4c54894bba "iris: Handle vertex shader with window space position"
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>

iris/android: move iris_query.c to 'per gen' LIBIRIS_SRC_FILES

Fixes Iris build on Android.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

iris: Free query on error path

CID: 1452276

iris: Add missing 'break'

We don't want to fall through to unreachable().

CID: 1452277

spirv: Drop lower_workgroup_access_to_offsets

Intel drivers are not using this anymore, and turnip still don't have
Compute Shaders, so won't make a difference to stop using this option.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Rob Clark <robdclark@chromium.org>

i965/spirv: Lower shared memory later

Instead of asking spirv_to_nir to lower the workgroup (shared memory)
to offsets, keep them as derefs longer, then lower it later on.

Because Workgroup memory doesn't have explicit offsets, we need to set
those using nir_lower_vars_to_explicit_types before calling the I/O
lowering pass.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

i965: Use force_compat_profile driconf option

Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: fix mem leak in error path

Fixes: 8ae6667992ccca41d088 ("intel/perf: move query_object into perf")
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>

gitlab-ci: simplify $CROSS option

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

iris: minor restyling

iris/query: enable amd performance monitors

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

iris/perf: get monitor results

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

iris/perf: add begin/end hooks

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

iris/perf: add delete query

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

iris/perf: implement iris_create_monitor_object

This is the first call that provides the iris context to the monitor
implementation.  On the first call, use the iris context to initialize
the monitor context.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

iris/perf: implement routines to return counter info

With this commit, Iris will report that AMD_performance_monitor is
supported, and will allow the caller to query the available metrics.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

anv: add missing `break`

Fixes: f6e7de41d7b15185b746 ("anv: Implement VK_EXT_line_rasterization")
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

util: drop final reference to p_compiler.h

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

util: os_misc: drop p_compiler.h include

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

util: u_math: drop p_compiler.h include

This file was moved from gallium so drop depending on gallium headers.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

vc4: prepare for p_compiler.h dependency removal

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

amd: prepare dropping include of p_compiler.h

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

mesa: be consistent on GL_TRUE/GL_FALSE & TRUE/FALSE

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

mesa: drop some p_compiler.h types

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

mesa: add stddef include in preparation for dropping p_compiler.h

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

panfrost: prepare for p_compiler.h dependency removal

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

i965: don't use p_compiler.h types

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>

gitlab-ci: generate meson cross-files earlier

Suggested-by: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>

panfrost: Assign varying buffers dynamically

Rather than hardcoding certain varying buffer indices "by convention",
work it out at draw time. This added flexibility is needed for
futureproofing and will be enable streamout.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Assign indices at draw-time

This will allow us to shuffle buffers.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Break out pan_varyings.c

This code is fairly self-contained, so let's factor it out of the giant
pan_context.c monster.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Enable PIPE_CAP_STREAM_OUTPUT_INTERLEAVE_BUFFERS

Just as easy/hard as the rest of XFB.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

panfrost: Import streamout data structures

Pretty much copypasted from v3d to jumpstart us.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/midgard: Account for swizzle/mask in st_vary

Register allocation for varying stores is a bit different, since the
instructions ignore the writemask (varyings are normalized
packed/vectorized..)

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

pan/decode: Resolve crash with NULL attr/varyings

This case needs more investigation, but this was found with geometry
shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

gallium/swr: Fix glClear when it's used with glEnable/glDisable GL_SCISSOR_TEST

When GL_SCISSOR_TEST is enabled glClear is handled by state tracker
and there is no need to do this in gallium driver.

Reviewed-by: Alok Hota alok.hota@intel.com

util: Revert "util: added missing headers in anon-file"

This reverts commit c73988300f943e185a50aaba015f2f114ffcb262.

Reason: Made a fix for this, then saw @eric's change
        ("util/anon_file: add missing"), but some sequence of events
        I don't really remember caused this to get merged. So revert ;-)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>

etnaviv: Remove etna_bo_from_handle() prototype

Remove etna_bo_from_handle() as there are no known users.

Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>

anv: drop unused code

We stopped using this when we moved to Jason's mi_builder.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>