freedreno: prep work for timestamp queries

We need "NULL" state to be a valid bit in the bitmask, because timestamp
queries are not restricted to draw/etc stages (ie. the only commands to
submit may just be to read the timestamp).  And just because there are
no draws, isn't a reason to skip the flush and return zero.

Signed-off-by: Rob Clark <robdclark@gmail.com>

radeonsi: ensure sample locations are set for line and polygon smoothing

Since commit d938b8c, the sample locations are no longer set unconditionally,
so we need to set the atom to dirty on all chips, not just Polaris.

Cc: 12.0 <mesa-stable@lists.freedesktop.org>

radeonsi: fix Polaris MSAA regression

The regression was introduced by commit d938b8c. The problem here is that in
order to use the small primitive filter, we need to explicitly set the sample
locations to 0. But the DB doesn't properly process the change of sample
locations without a flush, and so we can end up with incorrect Z values.

Instead of doing a flush, just disable the small primitive filter when MSAA
is force-disabled.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=96908
Cc: 12.0 <mesa-stable@lists.freedesktop.org>

freedreno/ir3: Add missing braces in initializer

Signed-off-by: Rob Clark <robdclark@gmail.com>

freedreno/a2xx: silence missing case 'SHADER_COMPUTE' warning (v2)

v2: no need for break after an unreachable (Matt Turner)

Signed-off-by: Francesco Ansanelli <francians@gmail.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

radeonsi: implement buffer_subdata without indirect calls

There is less noise in CPU profile data now.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/util: don't modify usage in pipe_buffer_write

All drivers were already doing it except virgl.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium: split transfer_inline_write into buffer and texture callbacks

to reduce the call indirections with u_resource_vtbl.

The worst call tree you could get was:
  - u_transfer_inline_write_vtbl
    - u_default_transfer_inline_write
      - u_transfer_map_vtbl
        - driver_transfer_map
      - u_transfer_unmap_vtbl
        - driver_transfer_unmap

That's 6 indirect calls. Some drivers only had 5. The goal is to have
1 indirect call for drivers that care. The resource type can be determined
statically at most call sites.

The new interface is:
  pipe_context::buffer_subdata(ctx, resource, usage, offset, size, data)
  pipe_context::texture_subdata(ctx, resource, level, usage, box, data,
                                stride, layer_stride)

v2: fix whitespace, correct ilo's behavior

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Acked-by: Roland Scheidegger <sroland@vmware.com>

nir: Lower interp_var_at_* like a normal load_var for flat inputs.

"flat centroid" and "flat sample" both just mean "flat", so we should
ignore interpolateAtCentroid/Sample and just return the flat value.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97032
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>

mesa: Don't call GenerateMipmap if Width or Height == 0.

One of the WebGL 2.0 conformance tests is trying to call
glGenerateMipmaps with a width and height of 0.  With the meta
implementation, this generates a "framebuffer attachment incomplete"
status, and falls back to the CPU path, calling MapTextureImage.

Except that there's no actual texture to map, and we assert fail.

There's no work to do in this case.  The test expects it to succeed,
so just return early with no error and avoid hassling the driver.

Cc: mesa-stable@lists.freedesktop.org
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=96911
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>

anv/pipeline: Set up point coord enables

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: "12.0" <mesa-stable@lists.freedesktop.org>

spirv/nir: Add support for ImageQuerySamples

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Handle texture projectors

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

nir/spirv: Refactor coordinate handling in handle_texture

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Refactor type handling in handle_texture

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Move opcode selection higher up in handle_texture

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

anv/image: Assert that the image format is actually supported

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Don't increment coord_components for array lod queries

For lod query instructions, we really don't care whether or not the sampler
is an array type because that doesn't factor into the LOD.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

i965: Get rid of the do_lower_unnormalized_offsets pass

We can do this in NIR now.  No need to keep a GLSL pass lying around for
it.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

i965/nir: Enable NIR lowering of txf and rect offsets

This fixes the following piglit tests on gen6+:

tex-miplevel-selection textureProjGradOffset 2DRect
tex-miplevel-selection textureGradOffset 2DRect
tex-miplevel-selection textureGradOffset 2DRectShadow
tex-miplevel-selection textureProjGradOffset 2DRect_ProjVec4
tex-miplevel-selection textureProjGradOffset 2DRectShadow

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

nir/lower_tex: Add support for lowering coordinate offsets

On i965, we can't support coordinate offsets for texelFetch or rectangle
textures.  Previously, we were doing this with a GLSL pass but we need to
do it in NIR if we want those workarounds for SPIR-V.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

nir/lower_tex: Add some helpers for working with tex sources

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

nir: Add a helper for determining the type of a texture source

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

anv/pipeline: Set binding_table.gather_texture_start

This should get texture gather working on gen8+ and mostly working on gen7.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Properly handle gather components

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Add support for shadow samplers that return vec4

While SPIR-V technically doesn't support "old style" shadow, the
shadow-compare gather instruction does return a vec4 so we need to be able
to set the old_style_shadow bit in NIR.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

spirv/nir: Fix some texture opcode asserts

We can't get an lod with txf_ms and SPIR-V considers textureGrad to be an
explicit-LOD texturing instruction.

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-dev@lists.freedesktop.org>

nv50/ir: allow to swap sources for OP_SUB

This allows the load-propagation pass to swap the sources in presence
of immediate values.

Maxwell (GM107):

total instructions in shared programs :1928187 -> 1927634 (-0.03%)
total gprs used in shared programs    :330741 -> 330154 (-0.18%)
total local used in shared programs   :28032 -> 28032 (0.00%)

                local        gpr       inst      bytes
    helped           0         271         425         425
      hurt           0           0         194         194

Fermi (GF114):

total instructions in shared programs :2334474 -> 2333829 (-0.03%)
total gprs used in shared programs    :380934 -> 380215 (-0.19%)
total local used in shared programs   :33304 -> 33264 (-0.12%)

                local        gpr       inst      bytes
    helped           5         314         521         521
      hurt           0           4         195         195

No regressions on GM107 and GF114 with full piglit.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>

gallium/radeon: make deferred flushes asynchronous

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

gallium: add PIPE_FLUSH_DEFERRED

There are 2 uses:
- Asynchronous flushing for multithreaded drivers.
- Return a fence without flushing (mid-command-buffer fence). The driver
  can defer flushing until fence_finish is called.

This is required to make Bioshock Infinite faster, which creates
1000 fences (flushes) per frame.

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Rob Clark <robdclark@gmail.com>

gallium/os: use CLOCK_MONOTONIC for sleeps (v2)

v2: handle EINTR, remove backslashes

Reviewed-by: Eric Engestrom <eric.engestrom@imgtec.com>

mapi: fix typo in macro name

Fixes: 5ec140c17b54c2592009 ("mapi: Massage code to allow clang to compile.")
Reported-by: Alexandre Demers <alexandre.f.demers@gmail.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Eric Engestrom <eric.engestrom@imgtec.com>

docs: Put swr back on the GL_ARB_texture_buffer_object_rgb32 list.

Looks like this was lost when resolving merge conflicts in
commit d1fbd4cdb1bdb8041362a8e5f05833c43a39c9a6.

glsl: subroutine types cannot be compared

subroutine variables are to be used just in the way functions are
called. Although the spec doesn't say it explicitely, this means that
these variables are not to be used in any other way than those left
for function calls. Therefore, a comparison between 2 subroutine
variables should also cause a compilation error.

From The OpenGL® Shading Language 4.40, page 117:

  "  To use subroutines, a subroutine type is declared, one or more
     functions are associated with that subroutine type, and a
     subroutine variable of that type is declared. The function
     currently assigned to the variable function is then called by
     using function calling syntax replacing a function name with the
     name of the subroutine variable. Subroutine variables are
     uniforms, and are assigned to specific functions only through
     commands (UniformSubroutinesuiv) in the OpenGL API."

From The OpenGL® Shading Language 4.40, page 118:

  "  Subroutine uniform variables are called the same way functions
     are called. When a subroutine variable (or an element of a
     subroutine variable array) is associated with a particular
     function, all function calls through that variable will call that
     particular function."

Fixes GL44-CTS.shader_subroutine.subroutines_cannot_be_assigned_float_int_values_or_be_compared

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

i965: fix varying output setup

Since 7f53fead5c we treat every location as using all
four components so we only need special handling for
doubles when they cross multiple locations.

This fixes a crash in GL45-CTS.enhanced_layouts.varying_locations
where the outputs array would overflow when a dmat2 was stored at
the max varying location i.e 30.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>

nvc0/mme: fix offsets used for indirect draws

This fixes a regression introduced in
1da704a94c57aa0b0cf8faaa3236fe47dfb8f88c because the offset has moved
from 0x180 to 0x1a0, and the macros have to be re-compiled.

Fixes: 1da704a ("nvc0: increase the tex handles area size in the driver")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>

nvc0: fix offsets of MP perf counters input parameters

This fixes a regression introduced in
1da704a94c57aa0b0cf8faaa3236fe47dfb8f88c because the offset has moved
from 0x600 to 0x620, and the kernels used for reading MP perf counters
have to be re-assembled.

This also fixes amd_performance_monitor_measure piglit.

Fixes: 1da704a ("nvc0: increase the tex handles area size in the driver")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>

mesa: Add GL_BGRA_EXT to the list of GenerateMipmap internal formats.

The GL_EXT_texture_format_BGRA8888 extension specification defines a
GL_BGRA_EXT unsized internal format (which is a little odd - usually
BGRA is a pixel transfer format).  The extension is written against
the ES 1.0 specification, so it's a little hard to map, but I believe
it's effectively adding it to the table used here, so we should allow
it here as well.

Note that GL_EXT_texture_format_BGRA8888 is always enabled (dummy_true),
so we don't need to check if it's enabled here.

This fixes mipmap generation in Skia and ChromeOS.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
References: https://bugs.chromium.org/p/chromium/issues/detail?id=630371
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reported-by: Stéphane Marchesin <marcheu@chromium.org>
Cc: mesa-stable@lists.freedesktop.org

i965: Fix "operation operation" in comment.

From the redundant redundant department.

Reported-by: Michael Schellenberger Costa <mschellenbergercosta@googlemail.com>

i965: Fix shared atomic intrinsics to pay attention to base.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>

nir: Add a base const_index to shared atomic intrinsics.

Commit 52e75dcb8c04c0dde989970c4c587cbe8313f7cf made nir_lower_io
start using nir_intrinsic_set_base instead of writing const_index[0]
directly.  However, those intrinsics apparently don't /have/ a base,
so this caused assert failures.

However, the old code was happily setting non-existent const_index
fields, so it was pretty bogus too.

Jason pointed out that load_shared and store_shared have a base,
and that the i965 driver uses that field.  So presumably atomics
should have one as well, so that loads/stores/atomics all refer
to variables with consistent addressing.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>

glsl: re-enable varying packing in GL4.4+

We can still do packing we just need to get the packing type from the consumer
rather than the producer.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97033

i965: Include VUE handles for GS with invocations > 1.

We always resort to the pull model for instanced GS inputs.  So, we'd
better include the VUE handles, or else we can't actually pull anything.

Ian reports that on his branch with OES_geometry_shader enabled,
this fixes a bunch of dEQP-GLES31.functional.geometry_shading tests::

- instanced.draw_2_instances_geometry_2_invocations
- instanced.draw_2_instances_geometry_8_invocations
- instanced.draw_4_instances_geometry_2_invocations
- instanced.draw_4_instances_geometry_8_invocations
- instanced.draw_8_instances_geometry_2_invocations
- instanced.draw_8_instances_geometry_8_invocations
- instanced.geometry_2_invocations
- instanced.geometry_32_invocations
- instanced.geometry_8_invocations
- instanced.geometry_max_invocations
- instanced.geometry_output_different_2_invocations
- instanced.geometry_output_different_32_invocations
- instanced.geometry_output_different_8_invocations
- instanced.geometry_output_different_max_invocations
- instanced.invocation_output_vary_by_attribute
- instanced.invocation_output_vary_by_texture
- instanced.invocation_output_vary_by_uniform
- query.primitives_generated_instanced

Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Tested-by: Ian Romanick <ian.d.romanick@intel.com>

mesa: Add -fno-math-errno -fno-trapping-math to CXXFLAGS.

Not sure why I forgot to add them to CXXFLAGS in commit f55c408067 or
commit 875458b778. Cuts about 1k of .text.

   text     data      bss      dec      hex  filename
5806354   287816    29384  6123554   5d7022  i965_dri.so before
5805497   287744    29384  6122625   5d6c81  i965_dri.so after

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>

mesa: Drop -fno-builtin-memcmp.

According to the referenced bug report, gcc-4.5 and newer do not inline
memcmp(). I see no difference in performance of ipers with llvmpipe on a
Sandybridge (which does not have "Enhanced REP MOVSB/STOSB") by removing
this flag.

I attempted to confirm the problem with gcc-4.4, but it fails to compile
for quite a few different reasons.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

mapi: Massage code to allow clang to compile.

According to https://llvm.org/bugs/show_bug.cgi?id=19778#c3 this code
was violating the spec, resulting in it failing to compile.

Cc: mesa-stable@lists.freedesktop.org
Co-authored-by: Tomasz Paweł Gajc <tpgxyz@gmail.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89599
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>

docs: Add extensions not part of any GL or GL ES version

Based loosely on patches submitted ages ago by Thomas Helland.

v2: Add lots of missing data provided by Ilia.  Fix sort order of
GL_ARB_sparse_texture extensions suggested by Ilia.

v3: Note that Dave Airlie has started work on GL_ARB_bindless_texture.

Signed-off-by: Ian Romanick <ian.d.romanick@intel.com>

docs: Update GL3.txt for OpenGL 4.0 on i965-ish hardware

Signed-off-by: Ian Romanick <ian.d.romanick@intel.com>

docs: Update GL3.txt for OpenGL ES on i965-ish hardware

Signed-off-by: Ian Romanick <ian.d.romanick@intel.com>

i965: print error messages if gs fails to compile

We do this for all other stages.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: enable GL4.4 for Gen8+

Acked-by: Kenneth Graunke <kenneth@whitecape.org>

i965: enable ARB_enhanced_layouts for gen6+

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965/vec4: add packing support for tcs load outputs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965/vec4: add support for packing tes inputs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

i965/vec4: add support for packing tcs outputs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965/vec4: support packing tcs inputs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965/vec4: add component packing for gs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965/vec4: add support for packing vs/gs/tes outputs

Here we create a new output_generic_reg array with the ability to
store the dst_reg for each component of user defined varyings.
This is needed as the previous code only stored the dst_reg based
on the varying location which meant packed varyings would overwrite
each other.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

i965/vec4: add support for packing inputs

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965: add helper for creating packing writemask

For example where n=3 first_component=1 this will give us
0xE (WRITEMASK_YZW).

V2:
Add assert to check first component is <= 4 (Suggested by Ken)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965: add helpers for creating component layout swizzle

This will be used to swizzle components to the beginning or end
of the vector based on the component layout qualifier and whether
we are doing a load or store.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

vc4: Return V3D version details in the GL renderer info.

This is as close as we get to a name for the 3D blocks.

vc4: Check the V3D version reported by the kernel.

We don't want to bring up an old userspace driver on a kernel for
newer hardware.  We'll also want to look at the other ident fields in
the future.

vc4: Detect and report kernel support for branching.

vc4: Switch to using the libdrm-provided vc4_drm.h.

The required version is set to .69 for the getparam ioctl that will be
used in the next commit.

docs: mark ARB_enhanced_layouts as DONE for i965

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>

i965: enable ARB_enhanced_layouts for gen8+

Acked-by: Edward O'Callaghan <funfunctor@folklore1984.net>

nir: add doubles component packing support

This makes sure we give the correct driver location
for doubles when using component packing. Specifically
it handles packing a dvec3 with a double which is the
only packing scenario allowed which spans across two
locations.

Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>

i965: add component packing support for load_output intrinsics

Here we use the component qualifier (which is the first component)
as an offset when loading output varyings.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: enable component packing for vs and fs

Rather than trying to work out the total number of components
used at a location we simply treat all outputs as vec4s. This
removes the need for complex code looping over varyings to match
packed locations and the need for storing the total number of
components used at each location.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: bring back type_size_vec4_times_4()

We will use this for output varyings. To make component
packing simpler we will just treat all varyings as vec4s.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

nir/inline: Constant-initialize local variables in the callee if needed

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-stable@lists.freedesktop.org>

nir: Add a nir_deref_foreach_leaf helper

Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: "12.0" <mesa-stable@lists.freedesktop.org>

clover: Re-order includes in invocation.cpp to fix build

The build was failing because the official CL headers have a few defines,
like: # define cl_khr_gl_sharing 1

Which have the same name as some class members of clang's OpenCLOptions class.
If we include the cl headers first, this breaks the build because the member
names of this class are replaced by the literal 1.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Vedran Miletić <vedran@miletic.net>

clover: Add missing include v2

clang commit r275822 removed unnecessary includes from header files,
so we now need to explicitly include clang/Lex/PreprocessorOptions.h

v2:
  - Use <> instead of "" for the include path.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Vedran Miletić <vedran@miletic.net>

i965: Move VS load_input handling to nir_emit_vs_intrinsic().

TCS/TES/GS and now FS all handle these in stage-specific functions.
CS don't have inputs, so VS was the only one left using this code.

Move it to the VS-specific function for clarity.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

i965: Delete the FS_OPCODE_INTERPOLATE_AT_CENTROID virtual opcode.

We no longer use this message.  As far as I can tell, it's fairly
useless - the equivalent information is provided in the payload.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

i965: Rewrite FS input handling to use the new NIR intrinsics.

This eliminates the need to walk the list of input variables, recurse
into their types (via logic largely redundant with nir_lower_io), and
interpolate all possible inputs up front.  The backend no longer has
to care about variables at all, which eliminates complications from
trying to pack multiple variables into the same location.  Instead,
each intrinsic specifies exactly what's needed.

This should unblock Timothy's work on GL_ARB_enhanced_layouts.

Each load_interpolated_input intrinsic corresponds to PLN instructions,
while load_barycentric_at_* intrinsics correspond to pixel interpolator
messages.  The pixel/centroid/sample barycentric intrinsics simply refer
to payload fields (delta_xy[]), and don't actually generate any code.

Because we use a single intrinsic for both centroid-qualified variables
and interpolateAtCentroid(), they become indistinguishable.  We stop
sending pixel interpolator messages for those, and instead use the
payload provided data, which should be considerably faster.

On Broadwell:

total instructions in shared programs: 9067751 -> 9067570 (-0.00%)
instructions in affected programs: 145902 -> 145721 (-0.12%)
helped: 422
HURT: 209

total spills in shared programs: 2849 -> 2899 (1.76%)
spills in affected programs: 760 -> 810 (6.58%)
helped: 0
HURT: 10

total fills in shared programs: 3910 -> 3950 (1.02%)
fills in affected programs: 617 -> 657 (6.48%)
helped: 0
HURT: 10

LOST:   3
GAINED: 3

The differences mostly appear to be slight changes in MOVs.

v2: Use nir_shader_compiler_options::use_interpolated_input_intrinsics
    flag rather than passing it directly to nir_lower_io.  Use the
    unreachable() macro rather than assert in one place.  (Review
    feedback from Chris Forbes.)

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>

i965: Move load_interpolated_input/barycentric_* intrinsics to the top.

Currently, i965 interpolates all FS inputs at the top of the program.
This has advantages and disadvantages, but I'd like to keep that policy
while reworking this code.  We can consider changing it independently.

The next patch will make the compiler generate PLN instructions "on the
fly", when it encounters an input load intrinsic, rather than doing it
for all inputs at the start of the program.

To emulate this behavior, we introduce an ugly pass to move all NIR
load_interpolated_input and payload-based (not interpolator message)
load_barycentric_* intrinsics to the shader's start block.

This helps avoid regressions in shader-db for cases such as:

   if (...) {
      ...load some input...
   } else {
      ...load that same input...
   }

which CSE can't handle, because there's no dominance relationship
between the two loads.  Because the start block dominates all others,
we can CSE all inputs and emit PLNs exactly once, as we did before.

Ideally, global value numbering would eliminate these redundant loads,
while not forcing them all the way to the start block.  When that lands,
we should consider dropping this hacky pass.

Again, this pass currently does nothing, as i965 doesn't generate these
intrinsics yet.  But it will shortly, and I figured I'd separate this
code as it's relatively self-contained.

v2: Dramatically simplify pass - instead of creating new instructions,
    just remove/re-insert their list nodes (suggested by Jason Ekstrand).

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com> [v1]
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

i965: Add a pass to demote sample interpolation intrinsics.

When working with a non-multisampled render target, asking for "sample"
interpolation locations doesn't make sense.  We demote them to centroid.

In a couple of patches, brw_compute_barycentric_modes will begin looking
at these intrinsics to determine the barycentric modes.  fs_visitor also
will use them to code-generate pixel interpolator messages or payload
references.  Handling the "but what if it's not MSAA?" logic ahead of
time in a NIR pass simplifies things and prevents duplicated logic.

This patch doesn't actually do anything useful yet as we don't generate
these intrinsics.  I decided to keep it separate as it's self-contained,
in the hopes of shrinking the "convert everything" patch for reviewers.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

nir: Add nir_load_interpolated_input lowering code.

Now nir_lower_io can optionally produce load_interpolated_input
and load_barycentric_* intrinsics for fragment shader inputs.

flat inputs continue using regular load_input.

v2: Use a nir_shader_compiler_options flag rather than ad-hoc boolean
    passing (in response to review feedback from Chris Forbes).

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

nir: Add new intrinsics for fragment shader input interpolation.

Backends can normally handle shader inputs solely by looking at
load_input intrinsics, and ignore the nir_variables in nir->inputs.

One exception is fragment shader inputs.  load_input doesn't capture
the necessary interpolation information - flat, smooth, noperspective
mode, and centroid, sample, or pixel for the location.  This means
that backends have to interpolate based on the nir_variables, then
associate those with the load_input intrinsics (say, by storing a
map of which variables are at which locations).

With GL_ARB_enhanced_layouts, we're going to have multiple varyings
packed into a single vec4 location.  The intrinsics make this easy:
simply load N components from location <loc, component>.  However,
working with variables and correlating the two is very awkward; we'd
much rather have intrinsics capture all the necessary information.

Fragment shader input interpolation typically works by producing a
set of barycentric coordinates, then using those to do a linear
interpolation between the values at the triangle's corners.

We represent this by introducing five new load_barycentric_* intrinsics:

- load_barycentric_pixel     (ordinary variable)
- load_barycentric_centroid  (centroid qualified variable)
- load_barycentric_sample    (sample qualified variable)
- load_barycentric_at_sample (ARB_gpu_shader5's interpolateAtSample())
- load_barycentric_at_offset (ARB_gpu_shader5's interpolateAtOffset())

Each of these take the interpolation mode (smooth or noperspective only)
as a const_index, and produce a vec2.  The last two also take a sample
or offset source.

We then introduce a new load_interpolated_input intrinsic, which
is like a normal load_input intrinsic, but with an additional
barycentric coordinate source.

The intention is that flat inputs will still use regular load_input
intrinsics.  This makes them distinguishable from normal inputs that
need fancy interpolation, while also providing all the necessary data.

This nicely unifies regular inputs and interpolateAt functions.
Qualifiers and variables become irrelevant; there are just
load_barycentric intrinsics that determine the interpolation.

v2: Document the interp_mode const_index value, define a new
    BARYCENTRIC() helper rather than using SYSTEM_VALUE() for
    some of them (requested by Jason Ekstrand).

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Chris Forbes <chrisforbes@google.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv: Properly call gen75_emit_state_base_address on Haswell.

This should fix MOCS values.  Caught by Coverity.

CID: 1364155

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

genxml: Rename "API Rendering Disable" to "Rendering Disable".

Gen7/7.5 call it "Rendering Disable" while Gen8/9 prefix it with "API".

Pick one for consistency, and so we can share code between generations.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv: Unify 3DSTATE_CLIP code across generations.

The bulk of this is the same.  There are just a couple fields that only
exist on one generation or another, and we can easily handle those with
an #ifdef.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv: Enable early culling on Gen7.

We set the cull mode, but forgot the enable bit.  Gen8 uses this.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv: Fix near plane clipping on Gen7/7.5.

The Gen7/7.5 clip code used APIMODE_OGL, while the Gen8+ clip code used
APIMODE_D3D.  The meaning hasn't changed, so one of these must be wrong.

It appears that the hardware documentation is completely wrong.  It
claims that the "API Mode" bit means:

   0h    APIMODE_OGL    NEAR_VP boundary == 0.0 (NDC)
   1h    APIMODE_D3D    NEAR_VP boundary == -1.0 (NDC)

However, DirectX typically uses 0.0 for the near plane, while unextended
OpenGL uses -1.0.  i965's gen6_clip_state.c uses APIMODE_D3D for the
GL_ZERO_TO_ONE case, so I believe the meanings are backwards from what
the documentation says.

Section 23.2 ("Primitive Clipping") of the Vulkan 1.0.21 specification
contains the following equations:

   -w_c <= x_c <= w_c
   -w_c <= y_c <= w_c
      0 <= z_c <= w_c

This means that Vulkan follows D3D semantics.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

genxml: Add APIMODE_D3D missing enum values and improve consistency.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

genxml: Add CLIPMODE_* prefix to 3DSTATE_CLIP's "Clip Mode" enum values.

Gen6-7.5 use CLIPMODE_REJECT_ALL, while Gen8+ just used REJECT_ALL.
Being consistent will let me unify code, and I prefer having the prefix.

Cc: "12.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

swr: [rasterizer core] introduce simd16intrin.h

Refactoring to leave existing simd_* intrinsics in "simdintrin.h" unchanged,
adding corresponding simd16_* intrinsics in "simd16intrin.h" on the side,
with emulation, that we can use piecemeal, rather than the all-or-nothing
approach to bring up avx512.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] fix for possible int32 overflow condition

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] rename *_MAX enum values to *_COUNT

Makes these names semantically correct.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] centroid correction

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] support range of values in TemplateArgUnroller

Fixes Linux warnings.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] ensure adjacent topologies use the cut-aware PA

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer] attribute swizzling and linkage

Add support for enhanced attribute swizzling. Currently supports constant
source overrides to handle PrimitiveID support. No support yet for input
select swizzling or wrap shortest. Removes obsoleted linkageMask and
associated code.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer common] icc declspec definitions

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer jitter] rework vertex/instance ID storage in fetch

Moved the setting into the existing component control code. Fixes bad
interaction between attribute/component setting for vertex/instance ID
and component packing.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] avx512 simd utility work

Enabling KNOB_SIMD_WIDTH = 16 for AVX512 pre-work and low level simd utils

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>

swr: [rasterizer core] viewport rounding for disabled scissor

Adjust viewport rounding when scissor rect is disabled during macro
tile scissor setup.

Signed-off-by: Tim Rowley <timothy.o.rowley@intel.com>