.gitlab-ci/tracie/README.md

   1 Tracie - Mesa Traces Continuous Integration System
   2 ==================================================
   3
   4 Home of the Mesa trace testing effort.
   5
   6 ### Traces definition file
   7
   8 The trace definition file contains information about the traces to run along
   9 with their expected image checksums on each device, and optionally from where to
  10 download them. An example:
  11
  12 ```yaml
  13 traces-db:
  14   download-url: https://minio-packet.freedesktop.org/mesa-tracie-public/
  15
  16 traces:
  17   - path: glmark2/jellyfish.rdc
  18     expectations:
  19       - device: gl-intel-0x3185
  20         checksum: 58359ea4caf6ad44c6b65526881bbd17
  21       - device: gl-vmware-llvmpipe
  22         checksum: d82267c25a0decdad7b563c56bb81106
  23   - path: supertuxkart/supertuxkart-antediluvian-abyss.rdc
  24     expectations:
  25       - device: gl-intel-0x3185
  26         checksum: ff827f7eb069afd87cc305a422cba939
  27 ```
  28
  29 The `traces-db` entry can be absent, in which case it is assumed that
  30 the traces can be found in the `CWD/traces-db` directory.
  31
  32 Traces that don't have an expectation for the current device are skipped
  33 during trace replay.
  34
  35 Adding a new trace to the list involves commiting the trace to the git repo and
  36 adding an entry to the `traces` list. The reference checksums can be calculated
  37 with the [image_checksum.py](.gitlab-ci/tracie/image_checksum.py) script.
  38 Alternatively, an arbitrary checksum can be used, and during replay (see below)
  39 the scripts will report the mismatch and expected checksum.
  40
  41 ### Trace-db download urls
  42
  43 The trace-db:download-url property contains an HTTPS url from which traces can
  44 be downloaded, by appending traces:path properties to it.
  45
  46 ### Enabling trace testing on a new device
  47
  48 To enable trace testing on a new device:
  49
  50 1. Create a new job in .gitlab-ci.yml. The job will need to be tagged
  51    to run on runners with the appropriate hardware.
  52
  53    1. If you mean to test GL traces, use the `.traces-test-gl`
  54       template jobs as a base, and make sure you set a unique value for the
  55      `DEVICE_NAME` variable and the name of the Mesa driver as `DRIVER_NAME`:
  56
  57    ```yaml
  58    my-hardware-gl-traces:
  59      extends: .traces-test-gl
  60      variables:
  61        DEVICE_NAME: "gl-myhardware"
  62        DRIVER_NAME: "mydriver"
  63    ```
  64
  65    2. If you mean to test Vulkan traces, use the `.traces-test-vk`
  66       template jobs as a base, set the `VK_DRIVER` variable, and make
  67       sure you set a unique value for the `DEVICE_NAME` variable:
  68
  69    ```yaml
  70    my-hardware-vk-traces:
  71      extends: .traces-test-vk
  72      variables:
  73        VK_DRIVER: "radeon"
  74        DEVICE_NAME: "vk-myhardware"
  75        DRIVER_NAME: "radv"
  76    ```
  77
  78 2. Update the .gitlab-ci/traces-$DRIVER_NAME.yml file with expectations for
  79    the new device. Ensure that the device name used in the expectations
  80    matches the one set in the job. For more information, and tips about how to
  81    calculate the checksums, see the section describing the trace definition
  82    files.
  83
  84 ### Trace files
  85
  86 Tracie supports renderdoc (.rdc), apitrace (.trace) and gfxreconstruct
  87 (.gfxr) files. Trace files need to have the correct extension so that
  88 tracie can detect them properly.
  89
  90 The trace files that are contained in public traces-db repositories must be
  91 legally redistributable. This is typically true for FOSS games and
  92 applications. Traces for proprietary games and application are typically not
  93 redistributable, unless specific redistribution rights have been granted by the
  94 publisher.
  95
  96 Trace files in a given repository are expected to be immutable once committed
  97 for the first time, so any changes need to be accompanied by a change in the
  98 file name (eg. by appending a _v2 suffix to the file).
  99
 100 ### Replaying traces
 101
 102 Mesa traces CI uses a set of scripts to replay traces and check the output
 103 against reference checksums.
 104
 105 The high level script [tracie.py](.gitlab-ci/tracie/tracie.py) accepts
 106 a traces definition file and the name of the device to be tested:
 107
 108     tracie.py --file .gitlab-ci/traces-llvmpipe.yml --device-name gl-vmware-llvmpipe
 109
 110 tracie.py copies the produced artifacts to the `$CI_PROJECT_DIR/result`
 111 directory. By default, created images from traces are only stored in case of a
 112 checksum mismatch. The `TRACIE_STORE_IMAGES` CI/environment variable can be set
 113 to `1` to force storing images, e.g., to get a complete set of reference
 114 images.
 115
 116 At a lower level the
 117 [dump_trace_images.py](.gitlab-ci/tracie/dump_trace_images.py) script is
 118 called, which replays a trace, dumping a set of images in the process. By
 119 default only the image corresponding to the last frame of the trace is dumped,
 120 but this can be changed with the `--calls` parameter. The dumped images are
 121 stored in a subdirectory `test/<device-name>` next to the trace file itself,
 122 with names of the form `tracefilename-callnum.png`.  The full log of any
 123 commands used while dumping the images is also saved in a file in the
 124 'test/<device-name>' subdirectory, named after the trace name with '.log'
 125 appended.
 126
 127 Examples:
 128
 129     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe mytrace.trace
 130     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe --calls=2075,3300 mytrace.trace
 131
 132 ### Running the replay scripts locally
 133
 134 It's often useful, especially during development, to be able to run the scripts
 135 locally.
 136
 137 Depending on the target 3D API, the scripts require a recent version
 138 of apitrace being in the path, and also the renderdoc python module
 139 being available, for GL traces.
 140
 141 To ensure python3 can find the renderdoc python module you need to set
 142 `PYTHONPATH` to point to the location of `renderdoc.so` (binary python modules)
 143 and `LD_LIBRARY_PATH` to point to the location of `librenderdoc.so`. In the
 144 renderdoc build tree, both of these are in `renderdoc/<builddir>/lib`. Note
 145 that renderdoc doesn't install the `renderdoc.so` python module.
 146
 147 In the case of Vulkan traces, the scripts need a recent version of
 148 gfxrecon-replay being in the path, and also the
 149 `VK_LAYER_LUNARG_screenshot` Vulkan layer from LunarG's VulkanTools.
 150
 151 To ensure that this layer can be found when running the trace you need
 152 to set `VK_LAYER_PATH` to point to the location of
 153 `VkLayer_screenshot.json` and `LD_LIBRARY_PATH` to point to the
 154 location of `libVkLayer_screenshot.so`.
 155
 156 In the case of DXGI traces, the scripts require Wine, a recent version
 157 of DXVK installed in the default `WINEPREFIX`, and a recent binary
 158 version of apitrace for Windows which should be reachable through
 159 Windows' `PATH` environment variable.