.gitlab-ci/tracie/README.md

   1 Tracie - Mesa Traces Continuous Integration System
   2 ==================================================
   3
   4 Home of the Mesa trace testing effort.
   5
   6 ### Traces definition file
   7
   8 The trace definition file contains information about the GitLab
   9 project and git commit to get the traces from, and a list of the
  10 traces to run along with their expected image checksums on each
  11 device. An example:
  12
  13 ```yaml
  14 traces-db:
  15   gitlab-project-url: https://gitlab.freedesktop.org/gfx-ci/tracie/traces-db
  16   commit: master
  17
  18 traces:
  19   - path: glmark2/jellyfish.rdc
  20     expectations:
  21       - device: gl-intel-0x3185
  22         checksum: 58359ea4caf6ad44c6b65526881bbd17
  23       - device: gl-vmware-llvmpipe
  24         checksum: d82267c25a0decdad7b563c56bb81106
  25   - path: supertuxkart/supertuxkart-antediluvian-abyss.rdc
  26     expectations:
  27       - device: gl-intel-0x3185
  28         checksum: ff827f7eb069afd87cc305a422cba939
  29 ```
  30
  31 The `traces-db` entry can be absent, in which case it is assumed that
  32 the traces can be found in the `CWD/traces-db` directory.
  33
  34 Traces that don't have an expectation for the current device are skipped
  35 during trace replay.
  36
  37 Adding a new trace to the list involves commiting the trace to the git repo and
  38 adding an entry to the `traces` list. The reference checksums can be calculated
  39 with the [image_checksum.py](.gitlab-ci/tracie/image_checksum.py) script.
  40 Alternatively, an arbitrary checksum can be used, and during replay (see below)
  41 the scripts will report the mismatch and expected checksum.
  42
  43 ### Trace-db GitLab projects
  44
  45 The trace-db GitLab projects are assumed to have git repositories
  46 using LFS for their trace files. This is so that trace files can be
  47 potentially checked out and replayed individually, thus reducing
  48 storage requirements during CI runs.
  49
  50 ### Enabling trace testing on a new device
  51
  52 To enable trace testing on a new device:
  53
  54 1. Create a new job in .gitlab-ci.yml. The job will need to be tagged
  55    to run on runners with the appropriate hardware.
  56
  57    1. If you mean to test GL traces, use the `.traces-test-gl`
  58       template jobs as a base, and make sure you set a unique value for the
  59      `DEVICE_NAME` variable:
  60
  61    ```yaml
  62    my-hardware-gl-traces:
  63      extends: .traces-test-gl
  64      variables:
  65        DEVICE_NAME: "gl-myhardware"
  66    ```
  67
  68    2. If you mean to test Vulkan traces, use the `.traces-test-vk`
  69       template jobs as a base, set the `VK_DRIVER` variable, and make
  70       sure you set a unique value for the `DEVICE_NAME` variable:
  71
  72    ```yaml
  73    my-hardware-vk-traces:
  74      extends: .traces-test-vk
  75      variables:
  76        VK_DRIVER: "radeon"
  77        DEVICE_NAME: "vk-myhardware"
  78    ```
  79
  80 2. Update the .gitlab-ci/traces.yml file with expectations for the new device.
  81    Ensure that the device name used in the expectations matches the one
  82    set in the job. For more information, and tips about how to calculate
  83    the checksums, see the section describing the trace definition files.
  84
  85 ### Trace files
  86
  87 Tracie supports renderdoc (.rdc), apitrace (.trace) and gfxreconstruct
  88 (.gfxr) files. Trace files need to have the correct extension so that
  89 tracie can detect them properly.
  90
  91 The trace files that are contained in public traces-db repositories must be
  92 legally redistributable. This is typically true for FOSS games and
  93 applications. Traces for proprietary games and application are typically not
  94 redistributable, unless specific redistribution rights have been granted by the
  95 publisher.
  96
  97 ### Replaying traces
  98
  99 Mesa traces CI uses a set of scripts to replay traces and check the output
 100 against reference checksums.
 101
 102 The high level script [tracie.py](.gitlab-ci/tracie/tracie.py) accepts
 103 a traces definition file and the name of the device to be tested:
 104
 105     tracie.py --file .gitlab-ci/traces.yml --device-name gl-vmware-llvmpipe
 106
 107 tracie.py copies the produced artifacts to the `$CI_PROJECT_DIR/result`
 108 directory. By default, created images from traces are only stored in case of a
 109 checksum mismatch. The `TRACIE_STORE_IMAGES` CI/environment variable can be set
 110 to `1` to force storing images, e.g., to get a complete set of reference
 111 images.
 112
 113 At a lower level the
 114 [dump_trace_images.py](.gitlab-ci/tracie/dump_trace_images.py) script is
 115 called, which replays a trace, dumping a set of images in the process. By
 116 default only the image corresponding to the last frame of the trace is dumped,
 117 but this can be changed with the `--calls` parameter. The dumped images are
 118 stored in a subdirectory `test/<device-name>` next to the trace file itself,
 119 with names of the form `tracefilename-callnum.png`.  The full log of any
 120 commands used while dumping the images is also saved in a file in the
 121 'test/<device-name>' subdirectory, named after the trace name with '.log'
 122 appended.
 123
 124 Examples:
 125
 126     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe mytrace.trace
 127     python3 dump_traces_images.py --device-name=gl-vmware-llvmpipe --calls=2075,3300 mytrace.trace
 128
 129 ### Running the replay scripts locally
 130
 131 It's often useful, especially during development, to be able to run the scripts
 132 locally.
 133
 134 Depending on the target 3D API, the scripts require a recent version
 135 of apitrace being in the path, and also the renderdoc python module
 136 being available, for GL traces.
 137
 138 To ensure python3 can find the renderdoc python module you need to set
 139 `PYTHONPATH` to point to the location of `renderdoc.so` (binary python modules)
 140 and `LD_LIBRARY_PATH` to point to the location of `librenderdoc.so`. In the
 141 renderdoc build tree, both of these are in `renderdoc/<builddir>/lib`. Note
 142 that renderdoc doesn't install the `renderdoc.so` python module.
 143
 144 In the case of Vulkan traces, the scripts need a recent version of
 145 gfxrecon-replay being in the path, and also the
 146 `VK_LAYER_LUNARG_screenshot` Vulkan layer from LunarG's VulkanTools.
 147
 148 To ensure that this layer can be found when running the trace you need
 149 to set `VK_LAYER_PATH` to point to the location of
 150 `VkLayer_screenshot.json` and `LD_LIBRARY_PATH` to point to the
 151 location of `libVkLayer_screenshot.so`.
 152
 153 In the case of DXGI traces, the scripts require Wine, a recent version
 154 of DXVK installed in the default `WINEPREFIX`, and a recent binary
 155 version of apitrace for Windows which should be reachable through
 156 Windows' `PATH` environment variable.