.gitlab-ci/tracie/README.md

   1 Tracie - Mesa Traces Continuous Integration System
   2 ==================================================
   3
   4 Home of the Mesa trace testing effort.
   5
   6 ### Traces definition file
   7
   8 The trace definition file contains information about the traces to run along
   9 with their expected image checksums on each device, and optionally from where to
  10 download them. An example:
  11
  12 ```yaml
  13 traces-db:
  14   download-url: https://minio-packet.freedesktop.org/mesa-tracie-public/
  15
  16 traces:
  17   - path: glmark2/jellyfish.rdc
  18     expectations:
  19       - device: gl-intel-0x3185
  20         checksum: 58359ea4caf6ad44c6b65526881bbd17
  21       - device: gl-vmware-llvmpipe
  22         checksum: d82267c25a0decdad7b563c56bb81106
  23   - path: supertuxkart/supertuxkart-antediluvian-abyss.rdc
  24     expectations:
  25       - device: gl-intel-0x3185
  26         checksum: ff827f7eb069afd87cc305a422cba939
  27 ```
  28
  29 The `traces-db` entry can be absent, in which case it is assumed that
  30 the traces can be found in the `CWD/traces-db` directory.
  31
  32 Traces that don't have an expectation for the current device are skipped
  33 during trace replay.
  34
  35 Adding a new trace to the list involves commiting the trace to the git repo and
  36 adding an entry to the `traces` list. The reference checksums can be calculated
  37 with the [image_checksum.py](.gitlab-ci/tracie/image_checksum.py) script.
  38 Alternatively, an arbitrary checksum can be used, and during replay (see below)
  39 the scripts will report the mismatch and expected checksum.
  40
  41 ### Trace-db download urls
  42
  43 The trace-db:download-url property contains an HTTPS url from which traces can
  44 be downloaded, by appending traces:path properties to it.
  45
  46 ### Enabling trace testing on a new device
  47
  48 To enable trace testing on a new device:
  49
  50 1. Create a new job in .gitlab-ci.yml. The job will need to be tagged
  51    to run on runners with the appropriate hardware.
  52
  53    1. If you mean to test GL traces, use the `.traces-test-gl`
  54       template jobs as a base, and make sure you set a unique value for the
  55      `DEVICE_NAME` variable:
  56
  57    ```yaml
  58    my-hardware-gl-traces:
  59      extends: .traces-test-gl
  60      variables:
  61        DEVICE_NAME: "gl-myhardware"
  62    ```
  63
  64    2. If you mean to test Vulkan traces, use the `.traces-test-vk`
  65       template jobs as a base, set the `VK_DRIVER` variable, and make
  66       sure you set a unique value for the `DEVICE_NAME` variable:
  67
  68    ```yaml
  69    my-hardware-vk-traces:
  70      extends: .traces-test-vk
  71      variables:
  72        VK_DRIVER: "radeon"
  73        DEVICE_NAME: "vk-myhardware"
  74    ```
  75
  76 2. Update the .gitlab-ci/traces.yml file with expectations for the new device.
  77    Ensure that the device name used in the expectations matches the one
  78    set in the job. For more information, and tips about how to calculate
  79    the checksums, see the section describing the trace definition files.
  80
  81 ### Trace files
  82
  83 Tracie supports renderdoc (.rdc), apitrace (.trace) and gfxreconstruct
  84 (.gfxr) files. Trace files need to have the correct extension so that
  85 tracie can detect them properly.
  86
  87 The trace files that are contained in public traces-db repositories must be
  88 legally redistributable. This is typically true for FOSS games and
  89 applications. Traces for proprietary games and application are typically not
  90 redistributable, unless specific redistribution rights have been granted by the
  91 publisher.
  92
  93 Trace files in a given repository are expected to be immutable once committed
  94 for the first time, so any changes need to be accompanied by a change in the
  95 file name (eg. by appending a _v2 suffix to the file).
  96
  97 ### Replaying traces
  98
  99 Mesa traces CI uses a set of scripts to replay traces and check the output
 100 against reference checksums.
 101
 102 The high level script [tracie.py](.gitlab-ci/tracie/tracie.py) accepts
 103 a traces definition file and the name of the device to be tested:
 104
 105     tracie.py --file .gitlab-ci/traces.yml --device-name gl-vmware-llvmpipe
 106
 107 tracie.py copies the produced artifacts to the `$CI_PROJECT_DIR/result`
 108 directory. By default, created images from traces are only stored in case of a
 109 checksum mismatch. The `TRACIE_STORE_IMAGES` CI/environment variable can be set
 110 to `1` to force storing images, e.g., to get a complete set of reference
 111 images.
 112
 113 At a lower level the
 114 [dump_trace_images.py](.gitlab-ci/tracie/dump_trace_images.py) script is
 115 called, which replays a trace, dumping a set of images in the process. By
 116 default only the image corresponding to the last frame of the trace is dumped,
 117 but this can be changed with the `--calls` parameter. The dumped images are
 118 stored in a subdirectory `test/<device-name>` next to the trace file itself,
 119 with names of the form `tracefilename-callnum.png`.  The full log of any
 120 commands used while dumping the images is also saved in a file in the
 121 'test/<device-name>' subdirectory, named after the trace name with '.log'
 122 appended.
 123
 124 Examples:
 125
 126     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe mytrace.trace
 127     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe --calls=2075,3300 mytrace.trace
 128
 129 ### Running the replay scripts locally
 130
 131 It's often useful, especially during development, to be able to run the scripts
 132 locally.
 133
 134 Depending on the target 3D API, the scripts require a recent version
 135 of apitrace being in the path, and also the renderdoc python module
 136 being available, for GL traces.
 137
 138 To ensure python3 can find the renderdoc python module you need to set
 139 `PYTHONPATH` to point to the location of `renderdoc.so` (binary python modules)
 140 and `LD_LIBRARY_PATH` to point to the location of `librenderdoc.so`. In the
 141 renderdoc build tree, both of these are in `renderdoc/<builddir>/lib`. Note
 142 that renderdoc doesn't install the `renderdoc.so` python module.
 143
 144 In the case of Vulkan traces, the scripts need a recent version of
 145 gfxrecon-replay being in the path, and also the
 146 `VK_LAYER_LUNARG_screenshot` Vulkan layer from LunarG's VulkanTools.
 147
 148 To ensure that this layer can be found when running the trace you need
 149 to set `VK_LAYER_PATH` to point to the location of
 150 `VkLayer_screenshot.json` and `LD_LIBRARY_PATH` to point to the
 151 location of `libVkLayer_screenshot.so`.
 152
 153 In the case of DXGI traces, the scripts require Wine, a recent version
 154 of DXVK installed in the default `WINEPREFIX`, and a recent binary
 155 version of apitrace for Windows which should be reachable through
 156 Windows' `PATH` environment variable.