i965/HACK: Build brw_cs into libcompiler

We need it for CS push constants

gen8/cmd_buffer: Don't push CS constants if there aren't any

Issuing MEDIA_CURB_LOAD with a size of zero causes GPU hangs on BDW.

anv: Use 4 instead of sizeof(gl_constant_value)

We no longer have access to gl_constant_value and, really, it's 4 because
our uniform layout code works entirely in dwords.

nir/spirv: Put SSBO store writemasks in the right index

It moved with the nir_intrinsic_load/store update.

Merge remote-tracking branch 'mesa-public/master' into vulkan

This pulls in nir_intrinsic_load/store changes and the switch of all
uniforms in i965 to bytes.  This accounts for the Vulkan changes.

anv/icd: Advertise the right ABI version

nir: Get rid of *_indirect variants of input/output load/store intrinsics

There is some special-casing needed in a competent back-end.  However, they
can do their special-casing easily enough based on whether or not the
offset is a constant.  In the mean time, having the *_indirect variants
adds special cases a number of places where they don't need to be and, in
general, only complicates things.  To complicate matters, NIR had no way to
convdert an indirect load/store to a direct one in the case that the
indirect was a constant so we would still not really get what the back-ends
wanted.  The best solution seems to be to get rid of the *_indirect
variants entirely.

This commit is a bunch of different changes squashed together:

 - nir: Get rid of *_indirect variants of input/output load/store intrinsics
 - nir/glsl: Stop handling UBO/SSBO load/stores differently depending on indirect
 - nir/lower_io: Get rid of load/store_foo_indirect
 - i965/fs: Get rid of load/store_foo_indirect
 - i965/vec4: Get rid of load/store_foo_indirect
 - tgsi_to_nir: Get rid of load/store_foo_indirect
 - ir3/nir: Use the new unified io intrinsics
 - vc4: Do all uniform loads with byte offsets
 - vc4/nir: Use the new unified io intrinsics
 - vc4: Fix load_user_clip_plane crash
 - vc4: add missing src for store outputs
 - vc4: Fix state uniforms
 - nir/lower_clip: Update to the new load/store intrinsics
 - nir/lower_two_sided_color: Update to the new load intrinsic

NIR and i965 changes are

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
NIR indirect declarations and vc4 changes are

Reviewed-by: Eric Anholt <eric@anholt.net>
ir3 changes are

Reviewed-by: Rob Clark <robdclark@gmail.com>
NIR changes are

Acked-by: Rob Clark <robdclark@gmail.com>

i965/fs_nir: Refactor store_output, load_input, and load_uniform

There was way too much incrementing of things going on.  Instead, let's
just start everything off at the right base location, and then increment in
the loop.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

gallium/util: return correct number of bound vertex buffers

In case a state tracker unbinds every slot by a seperate
pipe->set_vertex_buffers() call, starting from slot zero, the number
of bound buffers would not reach zero at all.
The current algorithm does not account for pre-existing holes in the
buffer list.

Unbinding all buffers at once or starting at the top-most slot results
in correct behaviour.

Calculating the correct number of bound buffers fixes a NULL pointer
dereference in nvc0_validate_vertex_buffers_shared().

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93004
Signed-off-by: Patrick Rudolph <siro@das-labor.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>

blit: Don't take into account the Mesa format when checking MSRT blit

According to the GLES3 spec, blitting between multisample FBOs with
different internal formats should not be allowed. The
compatible_resolve_formats function implements this check. Previously
it had a shortcut where if the Mesa formats of the two renderbuffers
were the same then it would assume the blit is ok. However some
drivers map different internal formats to the same Mesa format, for
example it might implement both GL_RGB and GL_RGBA textures with
MESA_FORMAT_R8G8B8A_UNORM. The function is used to generate a GL error
according to what the GL spec requires so the blit should not be
allowed in that case. This patch just removes the shortcut so that it
only ever looks at the internal format.

Note that I posted a related patch to disable this check altogether
for desktop GL. However this function is still used on GLES3 because
there are conformance tests that require this behaviour so this patch
is still useful.

Cc: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>

i965: Check base format to determine whether to use tiled memcpy

The tiled memcpy doesn't work for copying from RGBX to RGBA because it
doesn't override the alpha component to 1.0. Commit 2cebaac479d4 added
a check to disable it for RGBX formats by looking at the TexFormat.
However a lot of the rest of the code base is written with the
assumption that an RGBA texture can be used internally to implement a
GL_RGB texture. If that is done then this check breaks. This patch
makes it instead check the base format of the texture which I think
more directly matches the intention.

Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>

i965/gen8: Allow rendering to B8G8R8X8

Since Gen8 this is allowed as a rendering target so we don't need to
override it to B8G8R8A8. This is helpful on Gen9+ where using this
override causes fast clears not to work.

Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Ben Widawsky <benjamin.widawsky@intel.com>

i965/gen9: Allow fast clear for MSRT formats matching render

Previously fast clear was disallowed on Gen9 for MSRTs with the claim
that some formats don't work but we didn't understand why. On further
investigation it seems the formats that don't work are the ones where
the render surface format is being overriden to a different format
than the one used for texturing. The one used for texturing is not
actually a renderable format. It arguably makes sense that the sampler
hardware doesn't handle the fast color correctly in these cases
because it shouldn't be possible to end up with a fast cleared surface
that is non-renderable.

This patch changes the limitation to prevent fast clear for surfaces
where the format for rendering is overriden.

Reviewed-by: Ben Widawsky <benjamin.widawsky@intel.com>

i965/gen9/fast-clear: Handle linear→SRGB conversion

If GL_FRAMEBUFFER_SRGB is enabled when writing to an SRGB-capable
framebuffer then the color will be converted from linear to SRGB
before being written. There is no chance for the hardware to do this
itself because it can't modify the clear color that is programmed in
the surface state so it seems pretty clear that the driver should be
handling this itself.

Note that this wasn't a problem before Gen9 because previously we were
only able to do fast clears to 0 or 1 and those values are the same in
linear and SRGB space.

Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>

docs: Add ARB_compute_shader to 11.2.0 release notes

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

docs: Mark ARB_compute_shader as done for i965

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Enable ARB_compute_shader extension on supported hardware

Enable ARB_compute_shader on gen7+, on hardware that supports the
OpenGL 4.3 requirements of a local group size of 1024.

With SIMD16 support, this is limited to Ivy Bridge and Haswell.

Broadwell will work with a local group size up to 896 on SIMD16
meaning programs that use this size or lower should run when setting
MESA_EXTENSION_OVERRIDE=GL_ARB_compute_shader.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965/nir: Implement shared variable atomic operations

v3:
 * Update based on latest SSBO code (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

nir: Add nir intrinsics for shared variable atomic operations

v3:
 * Update min/max based on latest SSBO code (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Disable several optimizations on shared variables

Shared variables can be accessed by other threads within the same
local workgroup. This prevents us from performing certain
optimizations with shared variables.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Buffer atomics are supported for compute shaders

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Translate atomic intrinsic functions on shared variables

When an intrinsic atomic operation is used on a shared variable, we
translate it to a new 'shared variable' specific intrinsic function
call.

For example, a call to __intrinsic_atomic_add when used on a shared
variable will be translated to a call to
__intrinsic_atomic_add_shared.

v3:
 * Fix stale comments copied from SSBOs (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Check for SSBO variable in check_for_ssbo_store

The compiler probably already blocks this earlier on, but we should be
checking for an SSBO here.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Check for SSBO variable in SSBO atomic lowering

When an atomic function is called, we need to check to see if it is
for an SSBO variable before lowering it to the SSBO specific intrinsic
function.

v2:
 * is_in_buffer_block => is_in_shader_storage_block (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Replace atomic_ssbo and ssbo_atomic with atomic

The atomic functions can also be used with shared variables in compute
shaders.

When lowering the intrinsic in lower_ubo_reference, we still create an
SSBO specific intrinsic since SSBO accesses can be indirectly
addressed, whereas all compute shader shared variable live in a single
shared variable area.

v2:
 * Also remove the _internal suffix from ssbo atomic intrinsic names (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Allow atomic functions to be used with shared variables

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Lower shared variable references to intrinsic calls

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Enable shared local memory for CS shared variables

v3:
 * Check shared variable size at link time

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965/fs: Handle nir shared variable store intrinsic

v4:
 * Apply similar optimization for shared variable stores as
   0cb7d7b4b7c32246d4c4225a1d17d7ff79a7526d. This was causing a
   OpenGLES 3.1 CTS failure, but
   867c436ca841b4196b4dde4786f5086c76b20dd7 fixes that.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965/fs: Handle nir shared variable load intrinsic

v3:
 * Remove extra #includes (Iago)
 * Use recently added GEN7_BTI_SLM instead of BRW_SLM_SURFACE_INDEX (curro)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Disable vector splitting on shared variables

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

nir: Translate glsl shared var store intrinsic to nir intrinsic

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

nir: Translate glsl shared var load intrinsic to nir intrinsic

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Add lowering pass for shared variable references

In this lowering pass, shared variables are decomposed into intrinsic
calls.

v2:
 * Send mem_ctx as a parameter (Iago)

v3:
 * Shared variables don't have an associated interface block (Iago)
 * Always use 430 packing (Iago)
 * Comment / whitespace cleanup (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Don't assert on shared variable matrices with 'inherited' layout

We use column-major for shared variable matrices.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Don't lower_variable_index_to_cond_assign for shared variables

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: Remove mem_ctx as member variable in lower_ubo_reference_visitor

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl ubo/ssbo: Move common code into lower_buffer_access::setup_buffer_access

This code will also be usable by the pass to lower shared variables.

Note, that *const_offset is adjusted by setup_buffer_access so it must
be initialized before calling setup_buffer_access.

v2:
 * Add comment for lower_buffer_access::setup_buffer_access

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl ubo/ssbo: Move is_dereferenced_thing_row_major into lower_buffer_access

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl ubo/ssbo: Add lower_buffer_access class

This class has code that will be shared by lower_ubo_reference and
lower_shared_reference. (lower_shared_reference will be used to
support compute shader shared variables.)

v2:
 * Add lower_buffer_access.h to makefile (Emil)
 * Remove static is_dereferenced_thing_row_major from
   lower_buffer_access.cpp. This will become a lower_buffer_access
   method in the next commit.
 * Pass mem_ctx as parameter rather than using a member variable (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl ubo/ssbo: Split buffer access to insert_buffer_access

This allows the code in emit_access to be generic enough to also be
for lowering shared variables.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl ubo/ssbo: Use enum to track current buffer access type

v2:
 * Rename ssbo_get_array_length to ssbo_unsized_array_length_access (Iago)
 * Use always use this-> when referencing buffer_access_type (Iago)

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

glsl: do not loose always_active_io when packing varyings

Otherwise packed and inactive varyings get optimized away. This needs
to be prevented when using separate shader objects where interface
needs to be preserved.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>

mesa: invalidate pipeline status after glUseProgramStages

This will cause validation to run during next draw, this is done
because possible changes in used stages and programs can cause
invalid pipeline state.

This fixes a subtest in following CTS test:
ES31-CTS.sepshaderobjs.StateInteraction

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>

mesa/varray: set double arrays to non-normalised.

Doesn't have any effect in practice I don't think, but
CTS reads back using GetVertexAttrib.

This fixes: GL41-CTS.vertex_attrib_64bit.get_vertex_attrib

Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

clover: Fix build against LLVM 3.8 SVN >= r255078

Reviewed-by: Francisco Jerez <currojerez@riseup.net>

anv/image: Remove some vkCreateImage validation

Don't validate the baseArrayLayer and layerCount of cube images.  This
allows us to remove a bloated lookup table and an unneeded struct
definition (anv_image_view_info).

anv/image: Drop unused halign, valign lookup tables

mesa: fix ID usage for buffer warnings

We need a different ID pointer for each call site.

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>

docs: remove stray <ul> tag from 11.0.5.html file to fix indentation

freedreno: little clean up in fd_create_surface

in order to avoid returing invalid adress if CALLOC_STRUCT return NULL.

Signed-off-by: Rob Clark <robclark@freedesktop.org>

freedreno: change to goto fail

in fd_resource_transfer_map, like the others error cases

Signed-off-by: Rob Clark <robclark@freedesktop.org>

freedreno: fix bind_sampler_states when hwcso is NULL

src/gallium/tests/trivial/compute.c expects samplers to be cleaned
when the samplers list is NULL.
Like in radeon, the function behave like when the number of samplers
parameter is set to 0.

[small s/hwsco/hwcso/ typo fix]
Signed-off-by: Rob Clark <robclark@freedesktop.org>

gallium/util: Make u_prims_for_vertices() safe

Let us avoid trapping in hardware from a SIGFPE and instead
assert on a zero divisor.

Hint: This can occur if a PIPE_PRIM_? is not handled in
      u_prim_vertex_count() that results in ' info ' not
      being initialized in the expected manner.

Further, we also fix a possibly NULL pointer dereference
from ' info ' being NULL from a u_prim_vertex_count() call.

Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>
Signed-off-by: Marek Olšák <marek.olsak@amd.com>

docs: add news item for mesa-demos 8.3.0 release

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>

vulkan: Pull in the 0.210.1 vk_platform header

Somehow this got missed in the API update.

gen8/compute: Setup push constants and local ids

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>

anv: Add anv_cmd_buffer_cs_push_constants

Similar to anv_cmd_buffer_push_constants, but handles the compute
pipeline, which requires different setup from the other stages.

This also handles initializing the compute shader local IDs.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>

nv50,nvc0: fix use-after-free when vertex buffers are unbound

Always reset the vertex bufctx to make sure there's no pointer to
an already freed pipe_resource left after unbinding buffers.
Fixes use after free crash in nvc0_bufctx_fence().

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93004
Signed-off-by: Patrick Rudolph <siro@das-labor.org>
[imirkin: simplify nvc0 fix, apply to nv50]
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>

mesa: Fix a typo in a comment

s/suports/supports/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

glx: Fix a typo in a comment

s/suports/supports/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

st/osmesa: Fix a typo in a comment

s/suport/support/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

meta: Fix a typo in a print message

s/Unkown/Unknown/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

mesa: Fix typos in print messages

s/inconsistant/inconsistent/
s/occurences/occurrences/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

glsl: Fix a typo in a comment

s/suports/supports/

Signed-off-by: Andreas Boll <andreas.boll.dev@gmail.com>
Reviewed-by: Brian Paul <brianp@vmware.com>

svga: initialize pipe_driver_query_info entries with a macro

To be safe, set all the fields in case the enums ordering/values
ever change.

Reviewed-by: Charmaine Lee <charmainel@vmware.com>

mesa: detect inefficient buffer use and report through debug output

When a buffer is created with GL_STATIC_DRAW, its contents should not
be changed frequently.  But that's exactly what one application I'm
debugging does.  This patch adds code to try to detect inefficient
buffer use in a couple places.  The GL_ARB_debug_output mechanism is
used to report the issue.

NVIDIA's driver detects these sort of things too.

Other types of inefficient buffer use could also be detected in the
future.

Reviewed-by: José Fonseca <jfonseca@vmware.com>

docs: add news item and link release notes for 11.0.7

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>

docs: add sha256 checksums for 11.0.7

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
(cherry picked from commit f9715bc449d6b2cc5693e44bb82a9d5305a32ab5)

docs: add release notes for 11.0.7

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
(cherry picked from commit bec983b738a7f149478ee692ba0e1d26fcc9fd8e)

i965: Resolve color and flush for all active shader images in intel_update_state().

Fixes arb_shader_image_load_store/execution/load-from-cleared-image.shader_test.

Couldn't reproduce any significant FPS regression in CPU-bound
benchmarks from the Finnish benchmarking system on neither VLV nor BSW
after 30 runs with 95% confidence level.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92849
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jason Ekstrand <jason.ekstrand@intel.com>
Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Tested-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Document inconsistent units the URB size is represented in.

Every other gen the representation of the URB size was changed and
previous ones weren't updated.  I'd be willing to write a series
normalizing this to be KB on all generations if anybody else cares.

i965: Hook up L3 partitioning state atom.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Work around L3 state leaks during context switches.

This is going to require some rather intrusive kernel changes to fix
properly, in the meantime (and forever on at least pre-v4.1 kernels)
we'll have to restore the hardware defaults at the end of every batch
in which the L3 configuration was changed to avoid interfering with
the DDX and GL clients that use an older non-L3-aware version of Mesa.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
v2: Optimize look-up of the default configuration by assuming it's the
    first entry of the L3 config array in order to avoid an FPS
    regression in GpuTest Triangle and SynMark OglBatch2-7 on most
    affected platforms.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: Add debug flag to print out the new L3 state during transitions.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Implement L3 state atom.

The L3 state atom calculates the target L3 partition weights when the
program bound to some shader stage is modified, and in case they are
far enough from the current partitioning it makes sure that the L3
state is re-emitted.

v2: Fix for inconsistent units the context URB size is expressed in.
    Clamp URB size to 1008 KB on SKL due to FF hardware limitation.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Calculate appropriate L3 partition weights for the current pipeline state.

This calculates a rather conservative partitioning of the L3 cache
based on the shaders currently bound to the pipeline and whether they
use SLM, atomics, images or scratch space.  The result is intended to
be fine-tuned later on based on other pipeline state.

Note that the L3 partitioning calculated for VLV in the non-SLM non-DC
case differs from the hardware defaults in that it doesn't include a
DC partition and has twice as much RO cache space -- This is an
intentional functional change that improves performance in several
bandwidth-bound benchmarks on VLV (5% significance): SynMark
OglTexFilterAniso by 14.18%, SynMark OglTexFilterTri by 7.15%, Unigine
Heaven by 4.91%, SynMark OglShMapPcf by 2.15%, GpuTest Fur by 1.83%,
SynMark OglDrvRes by 1.80%, SynMark OglVsTangent by 1.71%, and a few
other benchmarks from the Finnish system by less than 1%.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Implement selection of the closest L3 configuration based on a vector of weights.

The input of the L3 set-up code is a vector giving the approximate
desired relative size of each partition.  This implements logic to
compare the input vector against the table of validated configurations
for the device and pick the closest compatible one.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Define and use REG_MASK macro to make masked MMIO writes slightly more readable.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965/hsw: Enable L3 atomics.

Improves performance of the arb_shader_image_load_store-atomicity
piglit test by over 25x (which isn't a real benchmark it's just heavy
on atomics -- the improvement in a microbenchmark I wrote a while ago
seemed to be even greater).  The drawback is one needs to be
extra-careful not to hang the GPU (in fact the whole system).  A DC
partition must have been allocated on L3, the "convert L3 cycle for DC
to UC" bit may not be set, the MOCS L3 cacheability bit must be set
for all surfaces accessed using DC atomics, and the SCRATCH1 and
ROW_CHICKEN3 bits must be kept in sync.

A fairly recent kernel is required for the command parser to allow
writes to these registers.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Implement programming of the L3 configuration.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Import tables enumerating the set of validated L3 configurations.

It should be possible to use additional L3 configurations other than
the ones listed in the tables of validated allocations ("BSpec »
3D-Media-GPGPU Engine » L3 Cache and URB [IVB+] » L3 Cache and URB [*]
» L3 Allocation and Programming"), but it seems sensible for now to
hard-code the tables in order to stick to the hardware docs.  Instead
of setting up the arbitrary L3 partitioning given as input, the
closest validated L3 configuration will be looked up in these tables
and used to program the hardware.

The included tables should work for Gen7-9.  Note that the quantities
are specified in ways rather than in KB, this is because the L3
control registers expect the value in ways, and because by doing that
we can re-use a single table for all GT variants of the same
generation (and in the case of IVB/HSW and CHV/SKL across different
generations) which generally have different L3 way sizes but allow the
same combinations of way allocations.

v2: Use slice count from the devinfo structure instead of the gt
    number to implement get_l3_way_size().

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Add slice count to the brw_device_info structure.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965/gen8: Don't add workaround bits to PIPE_CONTROL stalls if DC flush is set.

According to the hardware docs a DC flush is sufficient to make
CS_STALL happy, there's no need to add STALL_AT_SCOREBOARD whenever
it's present.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Define state flag to signal that the URB size has been altered.

This will make sure that we recalculate the URB layout anytime the URB
size is modified by the L3 partitioning code.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Keep track of whether LRI is allowed in the context struct.

This stores the result of can_do_pipelined_register_writes() in the
context struct so we can find out later whether LRI can be used to
program the L3 configuration.

v2:
 * Split change of gen check in can_do_pipelined_register_writes (jljusten)

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Adjust gen check in can_do_pipelined_register_writes

Allow for pipelined register writes for gen < 7.

v2:
 * Split from another patch and adjust comment (jljusten)

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Define symbolic constants for some useful L3 cache control registers.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

radeonsi: handle loading doubles as geometry shader inputs.

This adds the double code to the geometry shader input handling.

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

radeonsi: handle doubles in lds load path.

This handles loading doubles from LDS properly.

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Cc: "11.0 11.1" <mesa-stable@lists.fedoraproject.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

r600: handle geometry dynamic input array index

This fixes:
glsl-1.50/execution/geometry/dynamic_input_array_index.shader_test
my profanity.

We need to load the AR register with the value from the index reg

Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

r600g: fix geom shader input indirect indexing.

This fixes:
gs-input-array-vec4-index-rd

The others run out of gprs unfortunately.

Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

r600g: fix outputing to non-0 buffers for stream 0.

This fixes:
arb_transform_feedback3-ext_interleaved_two_bufs_gs
arb_transform_feedback3-ext_interleaved_two_bufs_gs_max
transform-feedback-builtins

If we are only emitting one ring, then emit all output
buffers on it.

Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

r600: Add ARB_copy_image support

[airlied: update relnotes]

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

r600g: allow copying between compatible un/compressed formats

See: `commit e82c527f1fc2f8ddc64954ecd06b0de3cea92e93`

which is where a block in src maps to a pixel in dst and vice versa.
    e.g. DXT1 <-> R32G32_UINT
         DXT5 <-> R32G32B32A32_UINT

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

nv50/ir: fix cutoff for using r63 vs r127 when replacing zero

The only effect here is a space savings - 822 programs in shader-db
affected with the following overall change:

total bytes used in shared programs   : 44154976 -> 44139880 (-0.03%)

Fixes: 641eda0c (nv50/ir: r63 is only 0 if we are using less than 63 registers)
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Cc: "11.0 11.1" <mesa-stable@lists.freedesktop.org>

nv50/ir: prefer to color mad def and src2 with the same color

This allows us to use the short encoding, and potentially fold
immediates in later on.

total instructions in shared programs : 6379731 -> 6367861 (-0.19%)
total gprs used in shared programs    : 728502 -> 728683 (0.02%)
total local used in shared programs   : 9904 -> 9904 (0.00%)
total bytes used in shared programs   : 44661008 -> 44154976 (-1.13%)

                local        gpr       inst      bytes
    helped           0          51        7267       20306
      hurt           0         232         125         274

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>

nv50/ir: reduce degree limit on ops that can't encode large reg dests

Operations that take immediates can only encode registers up to 64. This
fixes a shader in a "Powered by Unity" intro.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>

nv50/ir: only unspill once ahead of a group of instructions

We already semi-did this but the list of uses as unsorted, so it was
unreliable. Sort the uses by bb and serial, and don't unspill for each
instruction in a sequence. (And also don't unspill multiple times for a
single instruction that uses the value in question multiple times.)

This causes a minor reduction in generated instructions for shader-db
(as few programs spill) but more importantly it brings determinism to
each run's output.

On SM10:

total instructions in shared programs : 6387945 -> 6379359 (-0.13%)
total gprs used in shared programs    : 728544 -> 728544 (0.00%)
total local used in shared programs   : 9904 -> 9904 (0.00%)

                local        gpr       inst      bytes
    helped           0           0         322         322
      hurt           0           0           0           0

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>

nv50/ir: check if the target supports the new offset before inlining

Fixes: abd326e81b (nv50/ir: propagate indirect loads into instructions)
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93300
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>