1 This is a collection of tests for GDB.
3 The file gdb/README contains basic instructions on how to run the
4 testsuite, while this file documents additional options and controls
5 that are available. The GDB wiki may also have some pages with ideas
12 There are two ways to run the testsuite and pass additional parameters
13 to DejaGnu. The first is to do `make check' in the main build
14 directory and specifying the makefile variable `RUNTESTFLAGS':
16 make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp'
18 The second is to cd to the testsuite directory and invoke the DejaGnu
19 `runtest' command directly.
23 runtest GDB=/usr/bin/gdb
25 (The `site.exp' file contains a handful of useful variables like host
26 and target triplets, and pathnames.)
31 If not testing with a remote host (in DejaGnu's sense), you can run
32 the GDB test suite in a fully parallel mode. In this mode, each .exp
33 file runs separately and maybe simultaneously. The test suite ensures
34 that all the temporary files created by the test suite do not clash,
35 by putting them into separate directories. This mode is primarily
36 intended for use by the Makefile.
38 For GNU make, the Makefile tries to run the tests in parallel mode if
39 any -j option is given. For a non-GNU make, tests are not
42 If RUNTESTFLAGS is not empty, then by default the tests are
43 serialized. This can be overridden by either using the
44 `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
45 to any non-empty value:
47 make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
48 make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
50 If you want to use runtest directly instead of using the Makefile, see
51 the description of GDB_PARALLEL below.
56 Sometimes, new testcases are added to the testsuite that are not
57 entirely deterministic, and can randomly pass or fail. We call them
58 "racy testcases", and they can be bothersome when one is comparing
59 different testsuite runs. In order to help identifying them, it is
60 possible to run the tests several times in a row and ask the testsuite
61 machinery to analyze the results. To do that, you need to specify the
62 RACY_ITER environment variable to make:
64 make check RACY_ITER=5 -j4
66 The value assigned to RACY_ITER represents the number of times you
67 wish to run the tests in sequence (in the example above, the entire
68 testsuite will be executed 5 times in a row, in parallel). It is also
69 possible to check just a specific test:
71 make check TESTS='gdb.base/default.exp' RACY_ITER=3
73 One can also decide to call the Makefile rules by hand inside the
74 gdb/testsuite directory, e.g.:
76 make check-parallel-racy -j4
78 In which case the value of the DEFAULT_RACY_ITER variable (inside
79 gdb/testsuite/Makefile.in) will be used to determine how many
80 iterations will be run.
82 After running the tests, you shall see a file name 'racy.sum' in the
83 gdb/testsuite directory. You can also inspect the generated *.log and
84 *.sum files by looking into the gdb/testsuite/racy_ouputs directory.
86 If you already have *.sum files generated from previous testsuite runs
87 and you would like to analyze them without having to run the testsuite
88 again, you can also use the 'analyze-racy-logs.py' script directly.
89 It is located in the gdb/testsuite/ directory, and it expects a list
90 of two or more *.sum files to be provided as its argument. For
93 ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
94 testsuite-02/gdb.sum testsuite-03/gdb.sum
96 The script will output its analysis report to the standard output.
98 Re-running Tests Outside The Testsuite
99 **************************************
101 When running a test, the arguments used to run GDB are saved to gdb.cmd and
102 all commands sent to GDB are saved to gdb.in. As well as being a reference
103 of the commands run, they can be used to manually re-run a test by using
104 the gdb.in file as a batch file to a GDB launched with the arguments in the
105 gdb.cmd file, for example:
106 $(cat outputs/gdb.base/store/gdb.cmd) -x outputs/gdb.base/store/gdb.in
108 Tests that run GDB multiple times will append .1, .2, .3 etc to the end
109 of each .cmd and .in file.
111 When gdbserver is launched as part of a test, a gdbserver.cmd will be created.
112 To re-run these tests, run the contents of gdbserver.cmd in a separate
113 terminal before running gdb, for example:
114 $(cat outputs/gdb.base/store/gdbserver.cmd)
115 Alternatively, if the test is run with GDBSERVER_DEBUG="replay", then this
116 will create a gdbserver.replay file which can be used with the gdbreplay tool,
117 instead of launching gdbserver.
119 Running the Performance Tests
120 *****************************
122 GDB Testsuite includes performance test cases, which are not run together
123 with other test cases, because performance test cases are slow and need
124 a quiet system. There are two ways to run the performance test cases.
125 The first is to do `make check-perf' in the main build directory:
127 make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
129 The second is to cd to the testsuite directory and invoke the DejaGnu
130 `runtest' command directly.
134 runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
136 Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They
137 stand for "compile tests only", "run tests only", and "compile and run
138 tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT
139 specify the timeout, which is 3000 in default. The result of
140 performance test is appended in `testsuite/perftest.log'.
145 The following parameters are DejaGNU variables that you can set to
146 affect the testsuite run globally.
150 By default, the testsuite exercises the GDB in the build directory,
151 but you can set GDB to be a pathname to a different version. For
154 make check RUNTESTFLAGS=GDB=/usr/bin/gdb
156 runs the testsuite on the GDB in /usr/bin.
160 You can set GDBSERVER to be a particular GDBserver of interest, so for
163 make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
165 checks both the installed GDB and GDBserver.
169 Command line options passed to all GDB invocations.
171 The default is "-nw -nx".
173 `-nw' disables any of the windowed interfaces.
174 `-nx' disables ~/.gdbinit, so that it doesn't interfere with
177 This is actually considered an internal variable, and you
178 won't normally want to change it. However, in some situations,
179 this may be tweaked as a last resort if the testsuite doesn't
180 have direct support for the specifics of your environment.
181 The testsuite does not override a value provided by the user.
183 As an example, when testing an installed GDB that has been
184 configured with `--with-system-gdbinit', like by default,
185 you do not want ~/.gdbinit to interfere with tests, but, you
186 may want the system .gdbinit file loaded. As there's no way to
187 ask the testsuite, or GDB, to load the system gdbinit but
188 not ~/.gdbinit, a workaround is then to remove `-nx' from
189 INTERNAL_GDBFLAGS, and point $HOME at a directory without
190 a .gdbinit. For example:
195 GDBSERVER=/usr/bin/gdbserver \
196 INTERNAL_GDBFLAGS=-nw
200 To use parallel testing mode without using the Makefile, set
201 GDB_PARALLEL on the runtest command line to "yes". Before starting
202 the tests, you must ensure that the directories cache, outputs, and
203 temp in the test suite build directory are either empty or have been
204 deleted. cache in particular is used to share data across invocations
205 of runtest, and files there may affect the test results. The Makefile
206 automatically does these deletions.
210 Setting FORCE_PARALLEL to any non-empty value forces parallel testing
211 mode even if RUNTESTFLAGS is not empty.
213 FORCE_SEPARATE_MI_TTY
215 Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
216 in console mode, with MI running on a separate TTY, on a secondary UI
217 started with "new-ui".
221 For debugging parallel mode, it is handy to be able to see when a test
222 case writes to a file outside of its designated output directory.
224 If you have the inotify-tools package installed, you can set the
225 GDB_INOTIFY variable on the runtest command line. This will cause the
226 test suite to watch for parallel-unsafe file creations and report
227 them, both to stdout and in the test suite log file.
229 This setting is only meaningful in conjunction with GDB_PARALLEL.
233 This variable is used to specify which set of tests to run.
234 It is passed to make (not runtest) and its contents are a space separated
235 list of tests to run.
237 If using GNU make then the contents are wildcard-expanded using
238 GNU make's $(wildcard) function. Test paths must be fully specified,
239 relative to the "testsuite" subdirectory. This allows one to run all
240 tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
241 by using the check-gdb.subdir target in the Makefile.
243 If for some strange reason one wanted to run all tests that begin with
244 the letter "d" that is also possible: TESTS="*/d*.exp".
246 Do not write */*.exp to specify all tests (assuming all tests are only
247 nested one level deep, which is not necessarily true). This will pick up
248 .exp files in ancillary directories like "lib" and "config".
249 Instead write gdb.*/*.exp.
253 make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
255 If not using GNU make then the value is passed directly to runtest.
256 If not specified, all tests are run.
260 This make (not runtest) variable is used to specify whether the
261 testsuite preloads the read1.so library into expect. Any non-empty
262 value means true. See "Race detection" below.
266 This variable can provide the hostname/address that should be used
267 when performing GDBserver-related tests. This is useful in some
268 situations, e.g., when you want to test the IPv6 connectivity of GDB
269 and GDBserver, or when using a different hostname/address is needed.
270 For example, to make GDB and GDBserver use IPv6-only connections, you
273 make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]'
275 Note that only a hostname/address can be provided, without a port
280 This variable turns on the timestamp printing for each line of "make
281 check". Note that the timestamp will be printed on stdout output
282 only. In other words, there will be no timestamp output on either
283 gdb.sum and gdb.log files. If you would like to enable timestamp
284 printing, you can do:
290 You can provide a custom format for timestamp printing with this
291 variable. The format must be a string compatible with "strftime".
292 This variable is only useful when the TS variable is also provided.
293 If you would like to change the output format of the timestamp, you
296 make check TS=1 TS_FORMAT='[%b %H:%S]'
300 When set gdb debug is sent to the file gdb.debug in the test output
301 directory. It should be set to a comma separated list of gdb debug
303 For example, to turn on debugging for infrun and target, you can do:
305 make check GDB_DEBUG="infrun,target"
309 When set gdbserver debug is sent to the a file in the test output directory.
310 It should be set to a comma separated list of the following options:
311 debug - write gdbserver debug to gdbserver.debug.
312 remote - write gdbserver remote debug to gdbserver.debug.
313 replay - write a replay log to the file gdbserver.replay for use
315 Alternatively, it can be set to "all" to turn on all the above
316 For example, to turn on gdbserver debugging, you can do:
318 make check GDBSERVER_DEBUG="debug,replay"
323 The testsuite includes a mechanism that helps detect test races.
325 For example, say the program running under expect outputs "abcd", and
326 a test does something like this:
337 Which case happens to match depends on what expect manages to read
338 into its internal buffer in one go. If it manages to read three bytes
339 or more, then the first case matches. If it manages to read two
340 bytes, then the second case matches. If it manages to read only one
341 byte, then the third case matches.
343 To help detect these cases, the race detection mechanism preloads a
344 library into expect that forces the `read' system call to always
345 return at most 1 byte.
347 To enable this, either pass a non-empty value in the READ1 make
348 variable, or use the check-read1 make target instead of check.
352 make -j10 check-read1 TESTS="*/paginate-*.exp"
354 If you've already built the read1 support code, either via a previous
355 'check-read1' run, or by using "make read1", you can use:
357 make -j10 check READ1="1"
359 Note: While the intention is to detect races and make otherwise passing tests
360 fail, it can also have the effect of making otherwise failing tests pass.
361 This happens f.i. if the test is trying to match a gdb prompt using an end of
362 input marker "${gdb_prompt} $" and there is output after the gdb prompt. This
363 may either pass or fail in normal operation, but using check-read1 will ensure
364 that it passes. Use check-readmore to detect this type of failure.
366 Testsuite Configuration
367 ***********************
369 It is possible to adjust the behavior of the testsuite by defining
370 the global variables listed below, either in a `site.exp' file,
375 Defining this variable changes the default timeout duration used
376 during communication with GDB. More specifically, the global variable
377 used during testing is `timeout', but this variable gets reset to
378 `gdb_test_timeout' at the beginning of each testcase, which ensures
379 that any local change to `timeout' in a testcase does not affect
380 subsequent testcases.
382 This global variable comes in handy when the debugger is slower than
383 normal due to the testing environment, triggering unexpected `TIMEOUT'
384 test failures. Examples include when testing on a remote machine, or
385 against a system where communications are slow.
387 If not specifically defined, this variable gets automatically defined
388 to the same value as `timeout' during the testsuite initialization.
389 The default value of the timeout is defined in the file
390 `testsuite/config/unix.exp' (at least for Unix hosts; board files may
391 have their own values).
395 Defining this variable changes the default timeout duration when tests
396 under gdb.reverse directory are running. Process record and reverse
397 debugging is so slow that its tests have unexpected `TIMEOUT' test
398 failures. This global variable is useful to bump up the value of
399 `timeout' for gdb.reverse tests and doesn't cause any delay where
400 actual failures happen in the rest of the testsuite.
406 DejaGNU includes the concept of a "board file", which specifies
407 testing details for a particular target (which are often bare circuit
408 boards, thus the name).
410 In the GDB testsuite specifically, the board file may include a
411 number of "board settings" that test cases may check before deciding
412 whether to exercise a particular feature. For instance, a board
413 lacking any I/O devices, or perhaps simply having its I/O devices
414 not wired up, should set `noinferiorio'.
416 Here are the supported board settings:
418 gdb,cannot_call_functions
420 The board does not support inferior call, that is, invoking inferior
425 The board supports reverse execution.
427 gdb,no_hardware_watchpoints
429 The board does not support hardware watchpoints.
433 GDB is unable to intercept target file operations in remote and
434 perform them on the host.
438 The board is unable to provide I/O capability to the inferior.
442 A program will not return an exit code or result code (or the value
443 of the result is undefined, and should not be looked at).
447 The board does not support signals.
451 Skip time-consuming tests on the board with slow connection.
455 Skip tests related to floating point.
459 The board supports process record.
464 Commands to send to GDB every time a program is about to be run. The
465 first of these settings defines a single command as a string. The
466 second defines a TCL list of commands being a string each. The commands
467 are sent one by one in a sequence, first from `gdb_init_command', if any,
468 followed by individual commands from `gdb_init_command', if any, in this
473 The location of GDBserver. If GDBserver somewhere other than its
474 default location is used in test, specify the location of GDBserver in
475 this variable. The location is a file name for GDBserver, and may be
476 either absolute or relative to the testsuite subdirectory of the build
481 The location of the in-process agent (used for fast tracepoints and
482 other special tests). If the in-process agent of interest is anywhere
483 other than its default location, set this variable. The location is a
484 filename, and may be either absolute or relative to the testsuite
485 subdirectory of the build directory.
489 GDB does not support argument passing for inferior.
493 The board does not support type long long.
497 The board is running the monitor Cygmon.
501 The tests are running with a GDB stub.
505 Set to true if GDB can assume that letting the program run to end
506 reliably results in program exits being reported as such, as opposed
507 to, e.g., the program ending in an infinite loop or the board
508 crashing/resetting. If not set, this defaults to $use_gdb_stub. In
509 other words, native targets are assumed reliable by default, and
510 remote stubs assumed unreliable.
514 The predefined trace state variables the board has.
518 The target doesn't support thread names.
522 The flag required to force the compiler to produce position-independent
527 The flag required to force the linker to produce position-independent
532 The flag required to force the compiler to produce non-position-independent
537 The flag required to force the linker to produce non-position-independent
542 When set gdb debug is sent to the file gdb.debug in the test output
543 directory. It should be set to a comma separated list of gdb debug
544 components. For example, to turn on debugging for infrun and target, set to
549 When set gdbserver debug is sent to the file gdbserver.debug in the test
550 output directory. For valid values see the entry for GDBSERVER_DEBUG.
552 Testsuite Organization
553 **********************
555 The testsuite is entirely contained in `gdb/testsuite'. The main
556 directory of the testsuite includes some makefiles and configury, but
557 these are minimal, and used for little besides cleaning up, since the
558 tests themselves handle the compilation of the programs that GDB will
561 The file `testsuite/lib/gdb.exp' contains common utility procs useful
562 for all GDB tests, while the directory testsuite/config contains
563 configuration-specific files, typically used for special-purpose
564 definitions of procs like `gdb_load' and `gdb_start'.
566 The tests themselves are to be found in directories named
567 'testsuite/gdb.* and subdirectories of those. The names of the test
568 files must always end with ".exp". DejaGNU collects the test files by
569 wildcarding in the test directories, so both subdirectories and
570 individual files typically get chosen and run in alphabetical order.
572 The following lists some notable types of subdirectories and what they
573 are for. Since DejaGNU finds test files no matter where they are
574 located, and since each test file sets up its own compilation and
575 execution environment, this organization is simply for convenience and
580 This is the base testsuite. The tests in it should apply to all
581 configurations of GDB (but generic native-only tests may live here).
582 The test programs should be in the subset of C that is both valid
587 Language-specific tests for any language besides C. Examples are
588 gdb.cp for C++ and gdb.rust for Rust.
592 Non-portable tests. The tests are specific to a specific
593 configuration (host or target), such as eCos.
597 Architecture-specific tests that are (usually) cross-platform.
601 Tests that exercise a specific GDB subsystem in more depth. For
602 instance, gdb.disasm exercises various disassemblers, while
603 gdb.stabs tests pathways through the stabs symbol reader.
607 GDB performance tests.
612 In many areas, the GDB tests are already quite comprehensive; you
613 should be able to copy existing tests to handle new cases. Be aware
614 that older tests may use obsolete practices but have not yet been
617 You should try to use `gdb_test' whenever possible, since it includes
618 cases to handle all the unexpected errors that might happen. However,
619 it doesn't cost anything to add new test procedures; for instance,
620 gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
623 Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even
624 if GDB has several valid responses to a command, you can use
625 `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes
626 internal errors and unexpected prompts.
628 Do not write tests which expect a literal tab character from GDB. On
629 some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
630 spaces, so by the time GDB's output reaches `expect' the tab is gone.
632 The source language programs do *not* need to be in a consistent
633 style. Since GDB is used to debug programs written in many different
634 styles, it's worth having a mix of styles in the testsuite; for
635 instance, some GDB bugs involving the display of source lines might
636 never manifest themselves if the test programs used GNU coding style
639 Some testcase results need more detailed explanation:
643 Use KFAIL for known problem of GDB itself. You must specify the GDB
644 bug report number, as in these sample tests:
646 kfail "gdb/13392" "continue to marker 2"
650 setup_kfail gdb/13392 "*-*-*"
651 kfail "continue to marker 2"
656 Short for "expected failure", this indicates a known problem with the
657 environment. This could include limitations of the operating system,
658 compiler version, and other components.
660 This example from gdb.base/attach-pie-misread.exp is a sanity check
661 for the target environment:
663 # On x86_64 it is commonly about 4MB.
664 if {$stub_size > 25000000} {
665 xfail "stub size $stub_size is too large"
669 You should provide bug report number for the failing component of the
670 environment, if such bug report is available, as with this example
671 referring to a GCC problem:
673 if {[test_compiler_info {gcc-[0-3]-*}]
674 || [test_compiler_info {gcc-4-[0-5]-*}]} {
675 setup_xfail "gcc/46955" *-*-*
677 gdb_test "python print ttype.template_argument(2)" "&C::c"
679 Note that it is also acceptable, and often preferable, to avoid
680 running the test at all. This is the better option if the limitation
681 is intrinsic to the environment, rather than a bug expected to be
682 fixed in the near future.
684 Local vs Remote vs Native
685 *************************
687 It's unfortunately easy to get confused in the testsuite about what's
688 native and what's not, what's remote and what's not. The confusion is
689 caused by the overlap in vocabulary between DejaGnu and GDB.
691 From a DejaGnu point of view:
693 - native: the host or target board is considered native if the its
694 triplet is the same as the build system's triplet,
696 - remote: the host or target board is considered remote if it's
697 running on a different machine, and thus require ssh, for example,
698 to run commands, versus simply running commands directly.
700 Note that they are not mutually exclusive, as you can have a remote
701 machine that has the same triplet as the build machine.
703 From a GDB point of view:
705 - native: when GDB uses system calls such as ptrace to interact
706 directly with processes on the same system its running on,
708 - remote: when GDB speaks the RSP (Remote Serial Protocol) with
709 another program doing the ptrace stuff.
711 Note that they are mutually exclusive. An inferior can only be either
712 debugged with the native target, or with the remote target a specific
715 That means that there are cases where the target is not remote for
716 DejaGnu, but is remote for GDB (e.g. running GDBserver on the same
719 You can also have a remote target for DejaGnu, but native for GDB
720 (e.g. building on x86 a GDB that runs on ARM and running the
721 testsuite with a remote host).
723 Therefore, care must be taken to check for the right kind of remote.
724 Use [is_remote target] to check whether the DejaGnu target board is
725 remote. When what you really want to know is whether GDB is using the
726 remote protocol, because feature X is only available when GDB debugs
727 natively, check gdb_protocol instead.