.gitlab-ci/tracie/README.md

   1 Tracie - Mesa Traces Continuous Integration System
   2 ==================================================
   3
   4 Home of the Mesa trace testing effort.
   5
   6 ### Traces definition file
   7
   8 The trace definition file contains information about the git repo/commit to get
   9 the traces from, and a list of the traces to run along with their expected image
  10 checksums on each device. An example:
  11
  12 ```yaml
  13 traces-db:
  14   repo: https://gitlab.freedesktop.org/gfx-ci/tracie/traces-db
  15   commit: master
  16
  17 traces:
  18   - path: glmark2/jellyfish.rdc
  19     expectations:
  20       - device: gl-intel-0x3185
  21         checksum: 58359ea4caf6ad44c6b65526881bbd17
  22       - device: gl-vmware-llvmpipe
  23         checksum: d82267c25a0decdad7b563c56bb81106
  24   - path: supertuxkart/supertuxkart-antediluvian-abyss.rdc
  25     expectations:
  26       - device: gl-intel-0x3185
  27         checksum: ff827f7eb069afd87cc305a422cba939
  28 ```
  29
  30 The traces-db entry can be absent, in which case it is assumed that the
  31 current directory is the traces-db directory.
  32
  33 Traces that don't have an expectation for the current device are skipped
  34 during trace replay.
  35
  36 Adding a new trace to the list involves commiting the trace to the git repo and
  37 adding an entry to the `traces` list. The reference checksums can be calculated
  38 with the [image_checksum.py](.gitlab-ci/tracie/image_checksum.py) script.
  39 Alternatively, an arbitrary checksum can be used, and during replay (see below)
  40 the scripts will report the mismatch and expected checksum.
  41
  42 ### Trace-db repos
  43
  44 The trace-db repos are assumed to be git repositories using LFS for their trace
  45 files. This is so that trace files can be potentially checked out and replayed
  46 individually, thus reducing storage requirements during CI runs.
  47
  48 ### Enabling trace testing on a new device
  49
  50 To enable trace testing on a new device:
  51
  52 1. Create a new job in .gitlab-ci.yml. The job will need to be tagged
  53    to run on runners with the appropriate hardware.
  54
  55    1. If you mean to test GL traces, use the `.traces-test-gl`
  56       template jobs as a base, and make sure you set a unique value for the
  57      `DEVICE_NAME` variable:
  58
  59    ```yaml
  60    my-hardware-gl-traces:
  61      extends: .traces-test-gl
  62      variables:
  63        DEVICE_NAME: "gl-myhardware"
  64    ```
  65
  66    2. If you mean to test Vulkan traces, use the `.traces-test-vk`
  67       template jobs as a base, set the `VK_DRIVER` variable, and make
  68       sure you set a unique value for the `DEVICE_NAME` variable:
  69
  70    ```yaml
  71    my-hardware-vk-traces:
  72      extends: .traces-test-vk
  73      variables:
  74        VK_DRIVER: "radeon"
  75        DEVICE_NAME: "vk-myhardware"
  76    ```
  77
  78 2. Update the .gitlab-ci/traces.yml file with expectations for the new device.
  79    Ensure that the device name used in the expectations matches the one
  80    set in the job. For more information, and tips about how to calculate
  81    the checksums, see the section describing the trace definition files.
  82
  83 ### Trace files
  84
  85 Tracie supports renderdoc (.rdc), apitrace (.trace) and gfxreconstruct
  86 (.gfxr) files. Trace files need to have the correct extension so that
  87 tracie can detect them properly.
  88
  89 The trace files that are contained in public traces-db repositories must be
  90 legally redistributable. This is typically true for FOSS games and
  91 applications. Traces for proprietary games and application are typically not
  92 redistributable, unless specific redistribution rights have been granted by the
  93 publisher.
  94
  95 ### Replaying traces
  96
  97 Mesa traces CI uses a set of scripts to replay traces and check the output
  98 against reference checksums.
  99
 100 The high level script [tracie.sh](.gitlab-ci/tracie/tracie.sh) accepts
 101 a traces definition file and the type of traces
 102 (apitrace/renderdoc/gfxreconstruct) to run:
 103
 104     tracie.sh .gitlab-ci/traces.yml renderdoc
 105
 106 tracie.sh copies produced artifacts to the `$CI_PROJECT_DIR/result`
 107 directory. By default, created images from traces are only stored in case of a
 108 checksum mismatch. The `TRACIE_STORE_IMAGES` CI/environment variable can be set
 109 to `1` to force storing images, e.g., to get a complete set of reference
 110 images.
 111
 112 The `tracie.sh` script requires that the environment variable `DEVICE_NAME` is
 113 properly set for the target machine, and matches the `device` field of the
 114 relevant trace expectations in the used `traces.yml` file.
 115
 116 At a lower level the
 117 [dump_trace_images.py](.gitlab-ci/tracie/dump_trace_images.py) script is
 118 called, which replays a trace, dumping a set of images in the process. By
 119 default only the image corresponding to the last frame of the trace is dumped,
 120 but this can be changed with the `--calls` parameter. The dumped images are
 121 stored in a subdirectory `test/<device-name>` next to the trace file itself,
 122 with names of the form `tracefilename-callnum.png`.  The full log of any
 123 commands used while dumping the images is also saved in a file in the
 124 'test/<device-name>' subdirectory, named after the trace name with '.log'
 125 appended.
 126
 127 Examples:
 128
 129     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe mytrace.trace
 130     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe --calls=2075,3300 mytrace.trace
 131
 132 ### Running the replay scripts locally
 133
 134 It's often useful, especially during development, to be able to run the scripts
 135 locally.
 136
 137 Depending on the target 3D API, the scripts require a recent version
 138 of apitrace being in the path, and also the renderdoc python module
 139 being available, for GL traces.
 140
 141 To ensure python3 can find the renderdoc python module you need to set
 142 `PYTHONPATH` to point to the location of `renderdoc.so` (binary python modules)
 143 and `LD_LIBRARY_PATH` to point to the location of `librenderdoc.so`. In the
 144 renderdoc build tree, both of these are in `renderdoc/<builddir>/lib`. Note
 145 that renderdoc doesn't install the `renderdoc.so` python module.
 146
 147 In the case of Vulkan traces, the scripts need a recent version of
 148 gfxrecon-replay being in the path, and also the
 149 `VK_LAYER_LUNARG_screenshot` Vulkan layer from LunarG's VulkanTools.
 150
 151 To ensure that this layer can be found when running the trace you need
 152 to set `VK_LAYER_PATH` to point to the location of
 153 `VkLayer_screenshot.json` and `LD_LIBRARY_PATH` to point to the
 154 location of `libVkLayer_screenshot.so`.
 155
 156 In the case of DXGI traces, the scripts require Wine, a recent version
 157 of DXVK installed in the default `WINEPREFIX`, and a recent binary
 158 version of apitrace for Windows which should be reachable through
 159 Windows' `PATH` environment variable.