freedreno: Trim num_ubos to just the ones we haven't lowered to constbuf.

With the upcoming LDC usage in the GL driver, we don't want to be
uploading descriptors for every UBO when they aren't actually in use.
Trimming NIR's num_ubos will avoid that, and cleans up num_ubo handling
elsewhere right now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4858>

freedreno/ir3: Move i/o offset lowering after analyze_ubo_ranges.

I found that when moving more UBOs to load_ubo_ir3, analyze_ubo_ranges
would move things back in a broken way.  We can just run this pass later
and drop the _ir3 path.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4858>

freedreno/ir3: Leave the cursor alone during ir3_nir_try_propagate_bit_shift.

Otherwise, we might end up inserting the nir_intrinsic_load_ubo_ir3()
after the non-offset src's definition, leading to nir_validate() failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4858>

freedreno/ir3: Clean up a silly nir_src_for_ssa(src.ssa).

Just copy the src through.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4858>

nir: Include num_ubos in the printed shader (if nonzero).

I keep wanting this number for debugging shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4858>

util/ra: Add [de]serialization support

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5019>

util/vma: Add a debug print helper

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5019>

util/vma: Add an option to configure high/low preference

The vma_heap allocator was originally designed to prefer high addresses
in order to find bugs in ANV's high address handling.  However, there
are cases where you might want the allocator to prefer lower addresses
for some reason.  This provides a configure bit for exactly this
purpose.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5019>

util/list: Add list_foreach_entry_from_safe

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5019>

util/list: Add a list pair iterator

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5019>

anv: Implement VK_EXT_custom_border_color

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4898>

anv: Add a way to reserve states from a pool

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4898>

anv: Disable B5G6R5_UNORM_PACK16

It's not a required format and it causes issues with some features.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4898>

anv: use the correct format on Android

Per https://android.googlesource.com/platform/frameworks/native/+/master/vulkan/libvulkan/swapchain.cpp#745
the format Android requires is R5G6B5, and we have it backwards here.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4898>

drirc: Enable glthread for PCSX2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5023>

post_version.py: stop adding release candidates to the index and relnotes

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2870
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4929>

post_version.py: invert `is_point` into `is_first_release` to make its purpose clearer

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4929>

post_version.py: fix branch name construction for release candidates

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2870
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4929>

glthread: stop using GLenum16 to get correct GL errors for out-of-bounds enums

Reported by Ian Romanick.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5016>

radeonsi: also enable tgsi_to_nir caching for compute shaders

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4993>

radeonsi: Enable tgsi to nir disk cache

Enable the tgsi to nir cache for radeonsi.

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4993>

st/nine: Enable ttn cache

A trace of a Hat in Time, which builds thousands of shaders
takes 339 seconds to run the second time without this patch,
and 41 seconds with it (basically there is no more loading times).

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4993>

ttn: Implement disk cache

ttn is slow, let's disk cache it.

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4993>

ttn: Add new allow_disk_cache parameter

For now this parameter doesn't do anything.
It means the implementation is allowed to use
a cache on disk.

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4993>

freedreno/a6xx: Fix UBWC mipmapping height alignment.

After fixing the power of two sizing, pitches worked, but 1-pixel high and
unaligned height miplevels were off.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno/a6xx: Fix UBWC mipmap sizing.

The HW requires a log2 width/height of the level 0 meta_* size in the
descriptors, making it pretty clear that UBWC mipmapping is all
power-of-two sized.  Fixes a bunch of failures in the upcoming unit UBWC
layout unit tests.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno/a6xx: Fix UBWC blockheight for RG8.

Using texturator on a P3A at 1024x1024, RG8 has log2w/h of 6x7 instead of
R16I/UI's 6x8.  The other blockw/h I verified other than cpp=1
(R8/R8I/R8UI didn't use UBWC) and 32 (would need a bigger type).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno: Pull the tile_alignment lookup for a layout to a helper.

The r8g8 case UBWC alignment will be changing in the next commit, so
fdl6_get_ubwc_blockwidth needs to start paying attention to r8g8 too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno/a6xx: Add a testcase for UBWC buffer sharing.

These offsets are hand-computed referencing msm_media_info.h, and match
our driver's current behavior.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno/a6xx: Improve layout testcase logging for UBWC fails.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

freedreno/a4xx+: Increase max texture size to 16384.

Noticed when poking around with texture layouts and found that my big
texture layout from the blob buffer overflowed.  Values come from
http://vulkan.gpuinfo.org for Adreno 418, 512, 630.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4931>

nir: reset ssa-defs as non-divergent during divergence analysis instead of upfront

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

nir: simplify phi handling in divergence analysis

This patch adds some control flow information to the
state to keep track whether a loop contains divergent
continue or break statements to not having to
recalculate this property for every phi.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

nir: rework phi handling in divergence analysis

This patch splits the visit_phi() function into
three different ones according to the kind of phi
(merge-node, loop-header or loop-exit) and calls
them when visiting the cf_nodes.
This allows to revisit loops if the loop header's
phis have changed, only.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

nir: refactor divergence analysis state

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

nir: add nir_intrinsic_elect to divergence analysis

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

nir: Make "divergent" a property of an SSA value

v2: fix usage in ACO (by Daniel Schürmann)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4062>

gallium: remove more "state tracker" occurences

Trivial.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4902>

gallium: rename PIPE_RESOURCE_FLAG_ST_PRIV to FRONTEND_PRIV

Acked-by: Eric Anholt <eric@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4902>

gallium: change comments to remove 'state tracker'

Acked-by: Eric Anholt <eric@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4902>

gallium: rename 'state tracker' to 'frontend'

Acked-by: Eric Anholt <eric@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4902>

tu: Implement fallback linear staging blit for CopyImage

Also, rewrite the format decision code so that we correctly decide when
the linear fallback is needed, even if UBWC is disabled. As part of
that, I also moved around some of the code to handle compressed formats
to make sure that copying compressed formats with a linear staging blit
works (this is now possible since we started allowing tiled compressed
textures).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5007>

tu: Add noubwc debug flag to disable UBWC

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5007>

tu: Add a "scratch bo" allocation mechanism

This is simpler than a full-blown memory reuse mechanism, but is good
enough to make sure that repeatedly doing a copy that requires the
linear staging buffer workaround won't use excessive memory or be slowed
down due to repeated allocations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5007>

aco: improve phi affinities with p_split_vector

Totals from 5860 (4.59% of 127638) affected shaders:
VGPRs: 460212 -> 460216 (+0.00%)
CodeSize: 65554356 -> 65464816 (-0.14%)
Instrs: 12655972 -> 12633578 (-0.18%)
Copies: 1309994 -> 1292163 (-1.36%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4990>

aco: consider affinities when creating v_mac_f32

Totals from 8487 (6.65% of 127638) affected shaders:
CodeSize: 62061988 -> 62058020 (-0.01%); split: -0.01%, +0.01%
Instrs: 11910757 -> 11885409 (-0.21%); split: -0.21%, +0.00%
Copies: 1065244 -> 1040945 (-2.28%); split: -2.30%, +0.02%
Branches: 349665 -> 348914 (-0.21%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4990>

aco: mark phi definitions as last-seen phi operands

Totals from 14340 (11.23% of 127638) affected shaders:
SGPRs: 1251648 -> 1251512 (-0.01%)
VGPRs: 994556 -> 994104 (-0.05%); split: -0.06%, +0.01%
CodeSize: 122894528 -> 121099604 (-1.46%); split: -1.49%, +0.03%
MaxWaves: 106039 -> 106103 (+0.06%); split: +0.06%, -0.00%
Instrs: 23860066 -> 23414317 (-1.87%); split: -1.90%, +0.03%
Copies: 2448228 -> 2049305 (-16.29%); split: -16.37%, +0.07%
Branches: 789381 -> 757921 (-3.99%); split: -4.62%, +0.64%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4990>

aco: fix consecutively written vgprs from vmem instructions

If one VMEM instruction uses a sampler and the other doesn't, we can't do
this optimization.

Totals from 47 (0.04% of 127638) affected shaders:
CodeSize: 271744 -> 271656 (-0.03%); split: -0.04%, +0.01%
Instrs: 52783 -> 52761 (-0.04%); split: -0.05%, +0.01%
Cycles: 5547040 -> 5546952 (-0.00%); split: -0.00%, +0.00%
VMEM: 10022 -> 9887 (-1.35%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4949>

aco: simplify consecutive ordered vmem/lds writes optimization

This was unnecessary and messed with statistics

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4949>

gitlab-ci: correct tracie behavior with replay errors

[dump_trace_images] Info: Dumping trace /tmp/tracie.test.ap5pshYcsg/traces-db/trace1/magenta.testtrace... ERROR
[dump_trace_images] Debug: === Failure log start ===
invalid literal for int() with base 16: 'in'
[dump_trace_images] Debug: === Failure log end ===
[check_image] Trace /tmp/tracie.test.ap5pshYcsg/traces-db/trace1/magenta.testtrace couldn't be replayed. See above logs for more information.
Traceback (most recent call last):
  File "/tmp/tracie.test.ap5pshYcsg/tracie.py", line 176, in <module>
    main()
  File "/tmp/tracie.test.ap5pshYcsg/tracie.py", line 164, in main
    ok, result = gitlab_check_trace(project_url, commit_id, args.device_name, trace, expectation)
TypeError: cannot unpack non-iterable bool object

Fixes: efbbf8bb81e ("tracie: Print results in a machine readable format")
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4839>

gitlab-ci: create always the "results" directory with tracie

Otherwise, we will fail when the traces description file doesn't
contain any checksum for the specified device.

Fixes: efbbf8bb81e ("tracie: Print results in a machine readable format")
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4839>

radv: add a LLVM version string workaround for SotTR and ACO

When the LLVM version is too old or missing, SotTR applies shader
workarounds and that reduces performance by 2-5% with ACO.

SotTR workarounds are applied with LLVM 8 and older, so reporting
LLVM 9.0.1 should be fine.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Edmondo Tommasina <edmondo.tommasina@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4984>

turnip: use the common code for generating extensions and dispatch tables

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4987>

anv: use the common code for generating extensions and dispatch tables

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4987>

radv: use the common code for generating extensions and dispatch tables

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4987>

vulkan: import common code for generating extensions

ANV and RADV have similar Python code for generating extensions
and dispatch tables.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4987>

radv: implement VK_EXT_private_data

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4886>

radv: use the base object struct types

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4886>

radv: use the common base object type for VkDevice

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4886>

etnaviv: Disable seamless cube map on GC880

The GC880 on iMX6DL indicates in it's minorFeatures2 register that it
does support SEAMLESS_CUBE_MAP, however when the TE.SAMPLER_CONFIG1
VIVS_TE_SAMPLER_CONFIG1_SEAMLESS_CUBE_MAP bit is set on GC880 on iMX6DL,
the result is corrupted image. In particular, the following ~112 dEQPs
are affected and fail:

  dEQP-GLES2.functional.texture.filtering.cube.*

This only happens on MX6DL GC880, MX6Q GC2000 and STM32MP1 GC400(GCnano)
do not report the minorFeatures2 SEAMLESS_CUBE_MAP bit and ignore the
TE_SAMPLER_CONFIG1 VIVS_TE_SAMPLER_CONFIG1_SEAMLESS_CUBE_MAP bit (note
that ss->seamless_cube_map is unconditionally set by mesa at times even
PIPE_CAP_SEAMLESS_CUBE_MAP_PER_TEXTURE returns 0), so there is no visible
problem and there are no failing dEQP tests on the GC2000 and GCnano.

This might imply that the minorFeatures2 SEAMLESS_CUBE_MAP has some
different meaning on GC880 or the SEAMLESS_CUBE_MAP behaves differently
on the GC880.

This patch does not set the SEAMLESS_CUBE_MAP bit on hardware which does
not indicate support for seamless cube map and on GC880, which results
in reduction in failed dEQPs: 635 to 186 on GC880, 274 to 270 on GC2000
and no change on GC400(GCnano).

Fixes: 8dd26fa2f06 ("etnaviv: support GL_ARB_seamless_cubemap_per_texture")
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Signed-off-by: Marek Vasut <marex@denx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4865>

freedreno/a6xx: fix max-scissor opt

On a6xx we need a 0,0 based scissor in the binning pass, but can use the
blit-scissor to avoid restore/resolve of untouched pixels, and use the
conditional execution if the IB to bin to skip bins with no geometry
(due to the scissor).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5021>

freedreno/ir3/sched: try to avoid syncs

Similar to what we do in postsched.  It is useful for pre-RA sched to be
a bit aware of things that would cause syncs.  In particular for the tex
fetches, since the vecN src/dst tends to limit postsched's ability to
re-order them.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4923>

freedreno/ir3/sched: avoid scheduling outputs

If an instruction's only use is as an output, and it increases register
pressure, then try to avoid scheduling it until there are no other
options.

A semi-common pattern is `fragcolN.a = 1.0`, this pushes all these
immed loads to the end of the shader.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4923>

freedreno/ir3/postsched: try to avoid (sy) syncs

Similar to avoidance of `(ss)` syncs, it turns out to be helpful to
avoid `(sy)` syncs as well.  This helps us turn an tex, (sy)alu, tex,
(sy)alu sequence into tex, tex, (sy)alu, alu, which is a big win in
gfxbench gl_fill2.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4923>

freedreno/ir3/postsched: reset sfu_delay on sync

Once we schedule an instruction that will require an `(ss)` sync flag,
there is no need to delay any further instructions that consume an
SFU result (until the next SFU instruction is scheduled).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4923>

freedreno/ir3: limit # of tex prefetch by shader size

It seems for short frag shaders, too much prefetch can be detrimental.
I think what we *really* want to do is decide after pre-RA sched, when
we also know about nop's and what the actual ir3 instruction count is.
But that will require re-working how prefetch lowering works.  For now
this is a super crude heuristic to attempt to approximate a good
solution.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4923>

freedreno/ir3: fix indirect cb0 load_ubo lowering

We can no longer assume that `state->ranges[0]` is block 0.  It *often*
is, but when we encounter a "real" ubo that we lower to `load_uniform`
before a block 0 `load_ubo`, it could end up another entry in the table.
Resulting in the second pass after gathering ubo ranges, not finding a
valid range.  Which results in a `load_ubo` for a thing that is not
actually a ubo making it's way into ir3 frontend.  Resulting in grabbing
what we think is a ubo address out of some unrelated const register, and
trying to dereference that.  Which as you can imagine, fails in amusing
ways.

Fixes: fc850080ee3 ("ir3: Rewrite UBO push analysis to support bindless")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4954>

freedreno/ir3: don't allow negative const_offset

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4954>

panfrost: Run dEQP-GLES3.functional.shaders.derivate.* on CI

Should be stable now, and should pass except for MSAA tests
(multisampling is still a todo overall).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Fix derivative swizzle

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Set types for derivatives

Closes #2900

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Remove texture_op_count

Was used as a crude approximation of the terminate flag, which we now
can do properly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Use analysis to set .cont/.last flags

Corresponds roughly to what we analyze. Note that "terminate AND
execute" is a contradiction (rather: it's equivalent to just
terminating), hence why there are only three possibilities for the
states of the flags:

   .cont = continue, don't execute
   .last = don't continue, don't execute
   .cont.last = continue and execute

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Use the helper invo analyze passes

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Analyze helper execution requirements

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Analyze helper invocation termination

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

pan/mdg: Explain helper invocations dataflow theory

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5014>

intel/compiler: fix alignment assert in nir_emit_intrinsic

Fixes: c643979228 (intel/fs: Choose memory message type based on bit size)
Fixes: dEQP-VK.subgroups.ballot_broadcast.compute.subgroupbroadcast_i8vec2
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5000>

freedreno: Skip taking the lock for resource usage if it's already flagged.

Improves nohw drawoverhead 8-ubos update throughput by 13.493% +/-
0.391444% (n=15).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5011>

freedreno: Move the resource_read early out to an inline.

Looking at perf, the drawoverhead test case was now spending 13% CPU (89%
in that function) on stack management.

nohw drawoverhead throughput 1.03902% +/- 0.380257% (n=13).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4996>

freedreno: Add an early out for preparing to read a resource.

nohw drawoverhead 8 UBOs test throughput 1.06093% +/- 0.363376% (n=10).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4996>

freedreno: Split the fd_batch_resource_used by read vs write.

This is for an optimization I plan in a following commit.  I found I had
to add likely()s to avoid a perf regression from branch prediction.

On the drawoverhead 8 UBOs test, the HW can't quite keep up with the CPU,
but if I set nohw then this change is 1.32023% +/- 0.373053% (n=10).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4996>

freedreno: Add a nohw flag to skip submitting to the kernel.

For some CPU-side-only optimizations, it can be nice to disable rendering
so that we can see what the impact is even on cases where the GPU can't
quite keep up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4996>

turnip: Execute ir3_nir_lower_gs pass again

This commit fixes a GS regression introduced in !4562 where
ir3's GS lowering pass was moved from common code (ir3_nir) to
freedreno-specific code (ir3_shader). For GS support in turnip, we
need to add the GS lowering pass back in, this time in tu_shader.

As for the nir_gather_info change, the GS lowering pass has always
introduced a discard_if intrinsic into the GS. Previously, we simply
ran nir_shader_gather_info before GS lowering, but now since we lower
the GS before we need to remove the assertion that only a FS can use
the discard_if intrinsic.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4892>

freedreno/gmem: rework gmem layout algo

And try a bit harder to find an optimal layout.  Improves on a sub-
optimal layout we arrive at in the 4 MRT pass in manhattan, picking
up a bit more than 3%.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno/gmem: relax alignment on a6xx

The blob only uses single page alignment, and empirically that appears
to work just fine.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno: add gmemtool

A simple standalone thing to run through a bunch of GMEM layouts for a
given gpu.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno/gmem: add helper to dump GMEM layout

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno/gmem: add div_align() helper

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno: initialize max_scissor

Somehow the initialization of this got lost somewhere along the way,
resulting in assuming minx/miny are always zero.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

freedreno/gmem: don't assume scissor opt when estimating # of bins

We potentially don't know yet what the resulting scissor bounds are, so
we can't assume this when estimating number of bins per pipe for VSC
size calculations.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4976>

vulkan: Handle vkGet/SetPrivateDataEXT on Android swapchains

There is an annoying spec corner on Android.  Because VkSwapchain is
implemented in the Vulkan loader on Android which may not know about
this extension, we have to handle it as a special case inside the
driver.  We only have to do this on Android and only for VkSwapchainKHR.

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4882>

anv,vulkan: Implement VK_EXT_private_data

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4882>

turnip: enable tiling for compressed formats

Now that layout code supports this, we can enable it.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5009>

turnip: update "fetchsize" value to match fdl6_layout changes

It seems this is actually a "minimum pitch" value. For example
TFETCH6_2_BYTE means a minimum pitch of 128 bytes for mipmap levels.

This fixes breakage with compressed formats. For example this test:

dEQP-VK.pipeline.sampler.view_type.2d.format.eac_r11_snorm_block.mipmap.linear.lod.equal_min_3_max_3

Fixes: a34b3fa198a4f ("freedreno/fdl: Align after dividing by block size")
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5009>

freedreno: Fix non-constbuf-upload UBO block indices and count.

The nir_analyze_ubo_ranges pass removes all UBO block 0 loads to reverse
what nir_lower_uniforms_to_ubo() had done, and we only upload UBO pointers
to the HW for UBO block 1-N, so let's just fix up the shader state.

Fixes an off by one in const state layout setup, and some really dodgy
register addressing trying to deal with dynamic UBO indices when the UBO
pointers happen to be at the start of the constbuf.

There's no fixes tag, though this fixes a bug from September, because it
would require the num_ubos fix in nir_lower_uniforms_to_ubo.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4992>

nir: Fix count when we didn't lower load_uniforms but did shift load_ubos.

The fixed commit was really nice in mostly fixing num_ubos to reflect the
shader after lowering, but for
dEQP-GLES31.functional.compute.basic.ubo_to_ssbo_single_invocation there
are no default uniforms and so we skipped the increment, even though we
shifted the block index up.

Fixes: 4777ee1a62f0 ("nir: Always create UBO variable when lowering uniforms to ubo")
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4992>

freedreno: Drop the "write" arg to emit_const_bo now relocs don't care.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4967>

freedreno: Replace OUT_RELOCW with OUT_RELOC.

Final cleanup commit now that they're the same.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4967>

freedreno: Tell the kernel that all BOs are for writing.

Using non-write flags is pretty dubious -- it means the kernel tracking an
array of read-only consumers of the BO and having exclusive consumers wait
on each reader's fence.  It allows multiple readers through dma-bufs to do
work in parallel, but at the cost of kernel CPU time and memory management
of the shared array.  Other drivers have dropped this distinction since
dma-buf sharing is usually producer-consumer, not producer-two-consumers,
and the userspace and kernel space tracking is expensive.

For us, this lets us drop the flags passed in for relocs and tracked in
the ringbuffer reloc lists.  The end result of the flags reduction work is
drawoverhead uniforms test throughput 2.37195% +/- 0.365579% (n=15)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4967>