src/glsl/README

   1 Welcome to Mesa's GLSL compiler.  A brief overview of how things flow:
   2
   3 1) lex and yacc-based preprocessor takes the incoming shader string
   4 and produces a new string containing the preprocessed shader.  This
   5 takes care of things like #if, #ifdef, #define, and preprocessor macro
   6 invocations.  Note that #version, #extension, and some others are
   7 passed straight through.  See glcpp/*
   8
   9 2) lex and yacc-based parser takes the preprocessed string and
  10 generates the AST (abstract syntax tree).  Almost no checking is
  11 performed in this stage.  See glsl_lexer.lpp and glsl_parser.ypp.
  12
  13 3) The AST is converted to "HIR".  This is the intermediate
  14 representation of the compiler.  Constructors are generated, function
  15 calls are resolved to particular function signatures, and all the
  16 semantic checking is performed.  See ast_*.cpp for the conversion, and
  17 ir.h for the IR structures.
  18
  19 4) The driver (Mesa, or main.cpp for the standalone binary) performs
  20 optimizations.  These include copy propagation, dead code elimination,
  21 constant folding, and others.  Generally the driver will call
  22 optimizations in a loop, as each may open up opportunities for other
  23 optimizations to do additional work.  See most files called ir_*.cpp
  24
  25 5) linking is performed.  This does checking to ensure that the
  26 outputs of the vertex shader match the inputs of the fragment shader,
  27 and assigns locations to uniforms, attributes, and varyings.  See
  28 linker.cpp.
  29
  30 6) The driver may perform additional optimization at this point, as
  31 for example dead code elimination previously couldn't remove functions
  32 or global variable usage when we didn't know what other code would be
  33 linked in.
  34
  35 7) The driver performs code generation out of the IR, taking a linked
  36 shader program and producing a compiled program for each stage.  See
  37 ir_to_mesa.cpp for Mesa IR code generation.
  38
  39 FAQ:
  40
  41 Q: What is HIR versus IR versus LIR?
  42
  43 A: The idea behind the naming was that ast_to_hir would produce a
  44 high-level IR ("HIR"), with things like matrix operations, structure
  45 assignments, etc., present.  A series of lowering passes would occur
  46 that do things like break matrix multiplication into a series of dot
  47 products/MADs, make structure assignment be a series of assignment of
  48 components, flatten if statements into conditional moves, and such,
  49 producing a low level IR ("LIR").
  50
  51 However, it now appears that each driver will have different
  52 requirements from a LIR.  A 915-generation chipset wants all functions
  53 inlined, all loops unrolled, all ifs flattened, no variable array
  54 accesses, and matrix multiplication broken down.  The Mesa IR backend
  55 for swrast would like matrices and structure assignment broken down,
  56 but it can support function calls and dynamic branching.  A 965 vertex
  57 shader IR backend could potentially even handle some matrix operations
  58 without breaking them down, but the 965 fragment shader IR backend
  59 would want to break to have (almost) all operations down channel-wise
  60 and perform optimization on that.  As a result, there's no single
  61 low-level IR that will make everyone happy.  So that usage has fallen
  62 out of favor, and each driver will perform a series of lowering passes
  63 to take the HIR down to whatever restrictions it wants to impose
  64 before doing codegen.
  65
  66 Q: How is the IR structured?
  67
  68 A: The best way to get started seeing it would be to run the
  69 standalone compiler against a shader:
  70
  71 ./glsl --dump-lir ~/src/piglit/tests/shaders/glsl-orangebook-ch06-bump.frag
  72
  73 So for example one of the ir_instructions in main() contains:
  74
  75 (assign (constant bool (1)) (var_ref litColor)  (expression vec3 * (var_ref Surf
  76 aceColor) (var_ref __retval) ) )
  77
  78 Or more visually:
  79                      (assign)
  80                  /       |        \
  81         (var_ref)  (expression *)  (constant bool 1)
  82          /          /           \
  83 (litColor)      (var_ref)    (var_ref)
  84                   /                  \
  85            (SurfaceColor)          (__retval)
  86
  87 which came from:
  88
  89 litColor = SurfaceColor * max(dot(normDelta, LightDir), 0.0);
  90
  91 (the max call is not represented in this expression tree, as it was a
  92 function call that got inlined but not brought into this expression
  93 tree)
  94
  95 Each of those nodes is a subclass of ir_instruction.  A particular
  96 ir_instruction instance may only appear once in the whole IR tree with
  97 the exception of ir_variables, which appear once as variable
  98 declarations:
  99
 100 (declare () vec3 normDelta)
 101
 102 and multiple times as the targets of variable dereferences:
 103 ...
 104 (assign (constant bool (1)) (var_ref __retval) (expression float dot
 105  (var_ref normDelta) (var_ref LightDir) ) )
 106 ...
 107 (assign (constant bool (1)) (var_ref __retval) (expression vec3 -
 108  (var_ref LightDir) (expression vec3 * (constant float (2.000000))
 109  (expression vec3 * (expression float dot (var_ref normDelta) (var_ref
 110  LightDir) ) (var_ref normDelta) ) ) ) )
 111 ...
 112
 113 Each node has a type.  Expressions may involve several different types:
 114 (declare (uniform ) mat4 gl_ModelViewMatrix)
 115 ((assign (constant bool (1)) (var_ref constructor_tmp) (expression
 116  vec4 * (var_ref gl_ModelViewMatrix) (var_ref gl_Vertex) ) )
 117
 118 An expression tree can be arbitrarily deep, and the compiler tries to
 119 keep them structured like that so that things like algebraic
 120 optimizations ((color * 1.0 == color) and ((mat1 * mat2) * vec == mat1
 121 * (mat2 * vec))) or recognizing operation patterns for code generation
 122 (vec1 * vec2 + vec3 == mad(vec1, vec2, vec3)) are easier.  This comes
 123 at the expense of additional trickery in implementing some
 124 optimizations like CSE where one must navigate an expression tree.
 125
 126 Q: Why no SSA representation?
 127
 128 A: Converting an IR tree to SSA form makes dead code elmimination,
 129 common subexpression elimination, and many other optimizations much
 130 easier.  However, in our primarily vector-based language, there's some
 131 major questions as to how it would work.  Do we do SSA on the scalar
 132 or vector level?  If we do it at the vector level, we're going to end
 133 up with many different versions of the variable when encountering code
 134 like:
 135
 136 (assign (constant bool (1)) (swiz x (var_ref __retval) ) (var_ref a) )
 137 (assign (constant bool (1)) (swiz y (var_ref __retval) ) (var_ref b) )
 138 (assign (constant bool (1)) (swiz z (var_ref __retval) ) (var_ref c) )
 139
 140 If every masked update of a component relies on the previous value of
 141 the variable, then we're probably going to be quite limited in our
 142 dead code elimination wins, and recognizing common expressions may
 143 just not happen.  On the other hand, if we operate channel-wise, then
 144 we'll be prone to optimizing the operation on one of the channels at
 145 the expense of making its instruction flow different from the other
 146 channels, and a vector-based GPU would end up with worse code than if
 147 we didn't optimize operations on that channel!
 148
 149 Once again, it appears that our optimization requirements are driven
 150 significantly by the target architecture.  For now, targeting the Mesa
 151 IR backend, SSA does not appear to be that important to producing
 152 excellent code, but we do expect to do some SSA-based optimizations
 153 for the 965 fragment shader backend when that is developed.