gallium/radeon: don't flush the GFX IB if DMA doesn't depend on it

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

radeonsi: consolidate radeon_add_to_buffer_list calls for DMA

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: add a heuristic for better (S)DMA performance

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: flush if DMA IB memory usage is too high

This prevents IB rejections due to insane memory usage from
many concecutive texture uploads.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

radeonsi: add new SDMA texture copy code

This implements:
- Linear-to-linear partial copies. (unaligned)
- Tiled-to-linear and linear-to-tiled partial copies.
  (unaligned except 1-2 Bpp)
- Tiled-to-tiled partial copies aligned to 8x8.

v2: Extend the SDMA L2T VM fault workaround to T2L.
    - Same algorithm, just applied to T2L.
      (and using a 0-based address and surface.bo_size instead of buf->size)

Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: fix (S)DMA read-after-write hazards

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

radeonsi: raise the max size for SDMA buffer copies

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

radeonsi: remove SDMA texture copy code

Most of this has never worked according to the new test.

The new code will be radically different.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

radeonsi: only expose *_init_*dma_functions from (S)DMA files

just normalizing the interfaces

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: implement randomized SDMA texture copy testing (v2)

v2: - adjustments for exercising all important SDMA code paths
    - decrease the probability of getting huge sizes (faster testing)
    - increase the probability of getting power-of-two dimensions
    - change the memory cap to 128MB (faster testing)
    - better detect which engine has been used

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: discard CMASK or DCC if overwriting a whole texture by DMA

v2: simplify the conditionals

Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: use a common function for DMA blit preparation

this is more robust and probably fixes some bugs already

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: split out code for discarding DCC

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: rename r600_texture_disable_cmask -> discard_cmask

because it doesn't decompress

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

st/mesa: use transfer_inline_write for memcpy TexSubImage path

This allows drivers to use their own fast path for texture uploads.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: fix partial layered transfers of cube (array) textures

a staging cube texture with array_size % 6 != 0 doesn't work very well

just use 2D_ARRAY or 2D for all staging textures

Cc: 11.1 11.2 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: align alignments for better buffer reuse

It's for the buffer cache.

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallium/radeon: use gart_page_size instead of hardcoded 4096

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

winsys/radeon: use gart_page_size instead of private size_align

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

winsys/amdgpu: move gart_page_size to struct radeon_winsys

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

gallivm: print declarations of intrinsics with GALLIVM_DEBUG=ir

Those aren't really interesting, however outputting them is helpful when
trying to feed the IR to llvm llc (or opt) for debugging.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>

gallivm: use InternalLinkage instead of PrivateLinkage for texture functions

At least with MCJIT the disassembler will crash otherwise when trying to
disassemble such functions.

Reviewed-by: Jose Fonseca <jfonseca@vmware.com>

gallivm: disable avx512 features

We don't target this yet, and some llvm versions incorrectly enable it based
on cpu string, causing crashes.
(Albeit this is a losing battle, it is pretty much guaranteed when the next
new feature comes along llvm will mistakenly enable it on some future cpu,
thus we would have to proactively disable all new features as llvm adds them.)

This should fix https://bugs.freedesktop.org/show_bug.cgi?id=94291 (untested)

Tested-by: Timo Aaltonen <tjaalton@ubuntu.com>
Reviewed-by: Jose Fonseca <jfonseca@vmware.com
CC: <mesa-stable@lists.freedesktop.org>

Revert "nir: Try to warn when C99 extensions are used in nir headers."

This reverts commit 99474dc29b0f45413d0e538d7321da9ad4c9f6f5.

-Wpedantic is too verbose, even when applied to just a few includes.

We'll just have to deal with the issues as they come.

Reviewed-by: Brian Paul <brianp@vmware.com>

i965/fs: fix MOV_INDIRECT exec_size for doubles

In that case, the writes need two times the size of a 32-bit value.
We need to adjust the exec_size, so it is not breaking any hardware
rule.

v2:
  - Add an assert to verify type size is not less than 4 bytes (Jordan).

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: take into account doubles when calculating read_size for MOV_INDIRECT

v2:
- Fix assert's line width (Topi).

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965/fs: demote_pull_constants() did not take into account double types

The constants could be double, and it was allocating size for float types
for the destination register of varying pull constant loads.

Then the fs_visitor::validate() will complain.

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965/fs: push first double-based uniforms in push constant buffer

When there is a mix of definitions of uniforms with 32-bit or 64-bit
data type sizes, the driver ends up doing misaligned access to double
based variables in the push constant buffer.

To fix this, this patch pushes first all the 64-bit variables and
then the rest. Then, all the variables would be aligned to
its data type size.

v2:
- Fix typo and improve comment (Jordan).
- Use ralloc(NULL,...) instead of rzalloc(mem_ctx,...) (Jordan).
- Fix typo (Topi).
- Use pointers instead of references in set_push_pull_constant_loc() (Topi).

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>

i965/fs: recognize writes with a subreg_offset > 0 as partial

Usually, writes to a subreg_offset > 0 would also have a stride > 1
and we would recognize them as partial, however, there is one case
where this does not happen, that is when we generate code for 64-bit
imemdiates in gen7, where we produce something like this:

mov(8) vgrf10:UD, <low 32-bit>
mov(8) vgrf10+0.4:UD, <high 32-bit>

and then we use the result with a stride of 0, as in:

mov(8) vgrf13:DF, vgrf10<0>:DF

Although we could try to avoid this issue by producing different code
for this by using writes with a stride of 2, that runs into other
problems affecting gen7 and the fact is that any instruction that
writes to a subreg_offset > 0 is a partial write so we should really
recognize them as such.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs/lower_simd_width: Fix registers written for split instructions

When the original instruction had a stride > 1, the combined registers
written by the split instructions won't amount to the same register space
written by the original instruction because the split instructions will
use a stride of 1. The current code assumed otherwise and computed the
number of registers written by split instructions as an equal share based
on the relation between the lowered width and the original execution size
of the instruction.

It is only after the split, when we interleave the components of the result
from the lowered instructions back into the original dst register, that the
original stride takes effect and we write all the registers specified by
the original instruction.

Just make the number of register written the same as the vgrf space we
allocate for the dst of the split instruction.

Fixes crashes in fp64 tests produced as a result of assigning incorrectly the
number of registers written by split instructions, which led to incorrect
validation of the size of the writes against the allocated vgrf space.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: rename our lower_d2f pass to lower_d2x

Since it no longer handles conversions from double to float but from
double to various other 32-bit types.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: implement i2d and u2d

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: implement d2i and d2u

These need the same treatment as d2f, so generalize our d2f lowering to cover
these too.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: implement d2b

v2: Use subscript() instead of stride() (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: implement fsign() for doubles

v2 (Sam):
  - Fix indentation (Kenneth)
  - Simplify code (Kenneth)

v3: Use subscript() instead of stride() (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add null_reg_df

Probably not needed since we fix the dst type of comparisons
automatically, but for consistency with the rest of null_reg_*
functions.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: We only support 32-bit integer ALU operations for now

Add asserts so we remember to address this when we enable 64-bit
integer support, as suggested by Connor and Jason.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: handle fp64 opcodes in brw_do_channel_expressions

In the case of the pack opcode we are already doing the
lowering in NIR, so no need to do it here. The unpack opcode
operates on scalars, so it should not be lowered.

In the case of frexp_sig and frexp_exp, they are lowered in
lower_instructions, so we don't have to care about them.

All the remaining opcodes involve conversions from and to doubles
and are business as usual.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add support for f2d and d2f

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add a pass for legalizing d2f

We need to do this late, in order to avoid partial writes during the
optimization loop.

v2: Use subscript() instead of stride().

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: fix dst width calculation in CSE

v2 (Sam):
- Fix line width (Topi).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965/fs: fix regs_written in LOAD_PAYLOAD for doubles

v2: Account for the stride of the dst (Iago)

Signed-off-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965/fs: fix is_copy_payload() for doubles

v2 (Sam):
- LOAD_PAYLOAD treats each header source as a 32B block
  regardless of the datatype. Drop the change (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965/fs: fix compares for doubles

The destination has to have the same source as the type, or else the
simulator will complain. As a result, we need to emit a CMP that
outputs a 64-bit wide result and then do a strided MOV to pick out the
low 32 bits of each channel.

v2: Use subscript() instead of stride() (Curro)

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: extend exec_size halving in the generator

The HW has a restriction that only vertical stride may cross register
boundaries. Previously, this only mattered for SIMD16 instructions where
we needed to use the same regioning parameters as the equivalent SIMD8
instruction but double the exec size. But we need to do the same
splitting for 64-bit instructions as well as instructions with a stride
of 2 (which effectively consume 64 bits per element). Fix up the code to
do the right thing instead of special-casing SIMD16.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: fix assign_constant_locations() for doubles

Uniform doubles will read two registers, in which case we need to mark
both as being live.

v2 (Sam):
  - Use a formula to get the number of registers read with proper
    units (Curro).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965/fs: use byte_offset() in offset() for uniforms

This makes things more consistent, and also fixes the offset calculation
for double uniforms.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: handle uniforms in byte_offset()

v2: Do it only for uniforms (Iago)

Signed-off-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: fix type_size() for doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: optimize unpack double

When we are actually unpacking from a double that we have previously
packed from its 32-bit components we can bypass the pack operation
and source from its arguments directly.

v2 (Sam):
- Fix line overflow (Topi)
- Bail if the parent instruction's source is not SSA (Connor)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: optimize pack double

When we are actually creating a double using values obtained from a
previous unpack operation we can bypass the unpack and source from
the original double value directly.

v2:
- Style changes (Topi)
- Bail is parent instruction's src is not SSA (Connor)

v3: Use subscript() instead of stride() (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs/nir: translate double pack/unpack

v2 (Sam):
- Fix line overflow (Topi).

v3: Use subscript() instead of stride() (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add a pass for lowering PACK opcodes

v2: Use subscript() instead of stride() (Curro)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add PACK opcode

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: Introduce helper to extract a field from each channel of a register.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>

i965/fs: always pass the bitsize to brw_type_for_nir_type()

v2 (Sam):
- Add bitsize to brw_type_for_nir_type() in optimize_extract_to_float()

v3 (Sam):
- Fix line width (Topi).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: add support for printing double immediates

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: don't propagate 64-bit immediates

They can only be used with 1-src instructions, which practically (since
we should've constant-propagated away all 1-src instructions with 64-bit
immediates in NIR) means that they must be kept in separate MOV's and
can't be propagated.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965/fs: use the NIR bit size when creating registers

v2 (Iago):
  - Squashed bits from 'support double precission constant operands for
    the implementation of 64-bit emit_load_const'.
  - Do not use BRW_REGISTER_TYPE_D for all 32-bit registers since that breaks
    asserts and functionality for some piglit tests. Just keep 32-bit types
    untouched and add 64-bit support.
  - Use DF instead of Q for 64-bit registers. Otherwise the code we generate
    will use Q sometimes and DF others and we hit unwanted DF/Q conversions,
    so always use DF.

v3 (Sam):
  - Mark 'reg_type' occurrences as const (Topi).

Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Signed-off-by: Tapani Palli <tapani.palli@intel.com>
Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
Signed-off-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: fixup uniform setup for doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: two-argument instructions can only use 32-bit immediates

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: fix brw_abs_immediate() for doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: fix brw_saturate_immediate() for doubles

v2 (Sam):
  - Mark 'size' as const (Topi).
  - Add comment to explain that we do copies 64-bits regardless of the
    type (Topi)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: fix is_zero(), is_one() and is_negative_one() for doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: fix brw_negate_immediate() for doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965/eu: add support for DF immediates

v2 (Sam):
  - Remove 'however' from the comment (Topi)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: add support for disassembling DF immediates

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: add support for getting/setting DF immediates

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: add brw_imm_df

v2 (Iago)
  - Fixup accessibility in backend_reg

Signed-off-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965/eu: Allow 3-src float ops with doubles

v2:
  - set 3src_src_type for BRW_REGISTER_TYPE_DF (Connor)

Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965/disasm: fix disasm of 3-src doubles

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: Tell backend register about double precision type

Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Signed-off-by: Tapani P\344lli <tapani.palli@intel.com>
Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: Determine size of double precision float register

This is used to determine how many registers an instruction reads and
writes as well as for offseting register region into a desired component.

v2 (Connor): rebase on master

Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Signed-off-by: Tapani P\344lli <tapani.palli@intel.com>
Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: Lower DFRACEXP/DLDEXP

v2 (Connor): rebase on master which moved this to brw_link.cpp
v3 (Sam):
- Only enable DFREXP_DLDEXP_TO_ARITH in process_glsl_ir(). This is
used for doubles. Single floating point op is lowered by NIR.

Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: use pack/unpackDouble lowering

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

i965: use double lowering pass

v2: also lower trunc, ceil, floor, fract and roundEven (Iago)
v3: also lower mod for doubles (Sam)

Signed-off-by: Iago Toral Quiroga <itoral@igalia.com>
Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

freedreno/ir3: lower lrp when operating with double operands

Lower lrp when operating with double operands because float version of
lrp is also lowered.

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: enable lrp lowering for doubles

Broadwell and previous generations does not support lrp instruction
operating with doubles.

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>

st/glsl_to_tgsi: brown paper bag for the input offsets fix.

Oops, thanks compiler.

Signed-off-by: Dave Airlie <airlied@redhat.com>

glsl: check geometry output vertices limits.

This fixes:
GL45-CTS.geometry_shader.limits.max_output_vertices

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

mesa/vbo: fix check for zero aliases with 2/10/10/10

This fixes:
GL33-CTS.gtf33.GL3Tests.vertex_type_2_10_10_10_rev.vertex_type_2_10_10_10_rev_attrib

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Dave Airlie <airlied@redhat.com>

nir/print: Print memory qualifiers in a variable declaration

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

glsl: Apply memory qualifiers to vars inside named block interfaces

This is missing and memory qualifiers are currently being ignored for SSBOs.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

st/glsl_to_tgsi: handle offsets from inputs

This fixes:
GL45-CTS.gpu_shader5.texture_gather_offset_color_repeat

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

scripts: bump git_reviewer.pl --git-min-percent default

Bump up default percentage of commits required to be auto-picked for CC.
Seems from a bit of trial-and-error to come up with a more reasonable
list of CC's this way.

Signed-off-by: Rob Clark <robclark@freedesktop.org>

Revert "Revert "i965: Switch to scalar TCS by default.""

This reverts commit bd326c229c528a214c9fda705e7a961cfa49ac9e.

Now that we've fixed the GPU hangs, let's turn it back on.

Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Actually assign binding table offsets for the TCS.

As far as I can tell, this was just entirely missing...honestly, I'm
not sure how anything worked at all.

Caught by noticing GPU hangs in image load store tests with scalar TCS,
but probably has broader implications.

Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>

i965: Clamp "Maximum VP Index" to 1 when gl_ViewportIndex isn't written.

fs_visitor::emit_urb_writes skips writing the VUE header for shaders
that don't write gl_PointSize, gl_Layer, or gl_ViewportIndex.  This
leaves their values uninitialized.  Kristian's nearby comment says:

"But often none of the special varyings that live there are written
 and in that case we can skip writing to the vue header, provided the
 corresponding state properly clamps the values further down the
 pipeline."

However, we were clamping gl_ViewportIndex to [0, 15], so we would end
up using a random viewport.  To fix this, detect when the shader doesn't
write gl_ViewportIndex, and clamp it to [0, 0].

The vec4 backend always writes zeros to the VUE header, so it doesn't
suffer from this problem.  With vec4-style HWord writes, we can write
the header and position together in a single message.  In the FS world,
we would need 4 extra MOVs of 0 and a longer message, or a separate
OWord write.  It's likely cheaper to just clamp the value.

Fixes DiRT Showdown and Bioshock Infinite, which only rendered half of
the screen - the lower left of two triangles.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93054
Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>

i965/hsw: Fix brw_store_data_imm*

For Gen6 through Haswell dword 1 is MBZ. In gen 8 it becomes part of
the 64-bit address.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

i965: Reimplement ARB_transform_feedback2 on Haswell and later.

My old implementation accumulated <start, end> pairs in a buffer,
and eventually processed that data on the CPU.  This meant flushing
the batchbuffer and waiting for it to completely execute before we
could map it, resulting in really long stalls.  We could also run out
of space in the buffer, and have to do this early.

Instead, we can use Haswell's MI_MATH command to do the (end - start)
subtraction, as well as the multiplication by 2 or 3 to convert from
the number of primitives written to the number of vertices written.
We still need to CS stall to read the counters, but otherwise everything
is completely pipelined - there's no CPU<->GPU synchronization required.
It also uses only 80 bytes in the buffer, no matter what.

Improves performance in Manhattan on Skylake GT3e at 800x600 by
6.1086% +/- 0.954166% (n=9).  At 1920x1080, improves performance
by 2.82103% +/- 0.148596% (n=84).

v2: Fix number of primitives -> number of vertices calculation for
    GL_TRIANGLES (I was multiplying by 4 instead of 3.)  Caught by
    Jordan Justen.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: Add a brw_load_register_reg64 helper.

It appears that we can't do this in a single command (like we do for
MI_LOAD_REGISTER_IMM) - the Skylake simulator gets rather grumpy about
the command length if I try to combine them.  No matter.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

i965: Only enable ARB_query_buffer_object for newer kernels on Haswell.

On Haswell, we need version 6 of the kernel command parser in order to
write the math registers.  Our implementation of ARB_query_buffer_object
heavily relies on MI_MATH, so we should only advertise it when MI_MATH
is available.  We also need MI_LOAD_REGISTER_REG, which requires version
7 of the command parser.

To make these checks easier, introduce a screen->has_mi_math_and_lrr
flag that will be set when both commands are supported.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

mesa/objectlabel: don't return info on genned but never bound textures.

This fixes some cases in the CTS KHR debug tests where it uses
glIsTexture to find an invalid ID and then call GetObjectLabel.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

mesa: don't use genned but unnamed xfb objects.

If we try to draw or query an XFB object that hasn't been bound,
we shouldn't return any information.

This fixes a couple if cases in:
GL33-CTS.transform_feedback.api_errors_test

The ObjectLabel test is inspired by another test.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

nv50/ir: silence unsupported TGSI_PROPERTY_CS_FIXED_BLOCK_*

We don't need them for compute shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>

mesa/compute: Fix indirect dispatch buffer size check on 32-bit systems

2655265fcba9017e793026c76e490e04db088c8f, but for compute.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

freedreno/ir3: fix fallout from new block iterators

Since this is potentially modifying the block structure of the shader,
it needs the _safe() version of the iterator.

Signed-off-by: Rob Clark <robclark@freedesktop.org>

radeonsi: workaround for tesselation on SI

We request more than 32KB of LDS here, which SI doesn't have. Since LLVM
recently started checking the size of declared LDS allocations, all shaders
involved in tesselation fail to compile on SI.

Note that the entire calculation here seems wrong, given how we calculate
indices for generic attributes, so the number ends up wrong on CI+ as well.
A proper solution is clearly needed, but this patch should serve as a band-aid
for SI in the meantime.

Also note that the real size of the LDS allocation in hardware is independent
from what we tell LLVM, so this is really more of a "cosmetic" change.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95198
Cc: "11.2" <mesa-stable@lists.freedesktop.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>

radeonsi: always allocate export memory for pixel shaders

Experiments with framebuffer-no-attachments type draw calls have shown that
NULL exports stall terribly unless we ensure that export memory is allocated
by the SPI.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>

radeonsi: expose performance counters as 64 bit

This is useful for shader-related counters, since they tend to quickly
exceed 32 bits.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>