From b866c52d3b2d22bafeb2902c7ebdf1f04d626750 Mon Sep 17 00:00:00 2001 From: Stan Shebs Date: Thu, 12 Sep 2013 22:51:16 +0000 Subject: [PATCH] * README: New file. --- gdb/testsuite/ChangeLog | 4 + gdb/testsuite/README | 382 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 386 insertions(+) create mode 100644 gdb/testsuite/README diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog index d24e9250777..87af6587c73 100644 --- a/gdb/testsuite/ChangeLog +++ b/gdb/testsuite/ChangeLog @@ -1,3 +1,7 @@ +2013-09-12 Stan Shebs + + * README: New file. + 2013-09-12 Doug Evans * gdb.python/py-events.py (new_objfile_handler): Remove accidentally diff --git a/gdb/testsuite/README b/gdb/testsuite/README new file mode 100644 index 00000000000..4d369c554ce --- /dev/null +++ b/gdb/testsuite/README @@ -0,0 +1,382 @@ +This is a collection of tests for GDB. + +The file gdb/README contains basic instructions on how to run the +testsuite, while this file documents additional options and controls +that are available. The GDB wiki may also have some pages with ideas +and suggestions. + + +Running the Testsuite +********************* + +There are two ways to run the testsuite and pass additional parameters +to DejaGnu. The first is to do `make check' in the main build +directory and specifying the makefile variable `RUNTESTFLAGS': + + make check RUNTESTFLAGS='TRANSCRIPT=y gdb.base/a2-run.exp' + +The second is to cd to the testsuite directory and invoke the DejaGnu +`runtest' command directly. + + cd testsuite + make site.exp + runtest TRANSCRIPT=y + +(The `site.exp' file contains a handful of useful variables like host +and target triplets, and pathnames.) + +Testsuite Parameters +******************** + +The following parameters are DejaGNU variables that you can set to +affect the testsuite run globally. + +TRANSCRIPT + +You may find it useful to have a transcript of the commands that the +testsuite sends to GDB, for instance if GDB crashes during the run, +and you want to reconstruct the sequence of commands. + +If the DejaGNU variable TRANSCRIPT is set (to any value), each +invocation of GDB during the test run will get a transcript file +written into the DejaGNU output directory. The file will have the +name transcript., where is an integer. The first line of the +file shows the invocation command with all the options passed to it, +while subsequent lines are the GDB commands. A `make check' might +look like this: + + make check RUNTESTFLAGS=TRANSCRIPT=y + +The transcript may not be complete, as for instance tests of command +completion may show only partial command lines. + +GDB + +By default, the testsuite exercises the GDB in the build directory, +but you can set GDB to be a pathname to a different version. For +instance, + + make check RUNTESTFLAGS=GDB=/usr/bin/gdb + +runs the testsuite on the GDB in /usr/bin. + +GDBSERVER + +You can set GDBSERVER to be a particular GDBserver of interest, so for +instance + + make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver" + +checks both the installed GDB and GDBserver. + +INTERNAL_GDBFLAGS + +Command line options passed to all GDB invocations. + +The default is "-nw -nx". + +`-nw' disables any of the windowed interfaces. +`-nx' disables ~/.gdbinit, so that it doesn't interfere with +the tests. + +This is actually considered an internal variable, and you +won't normally want to change it. However, in some situations, +this may be tweaked as a last resort if the testsuite doesn't +have direct support for the specifics of your environment. +The testsuite does not override a value provided by the user. + +As an example, when testing an installed GDB that has been +configured with `--with-system-gdbinit', like by default, +you do not want ~/.gdbinit to interfere with tests, but, you +may want the system .gdbinit file loaded. As there's no way to +ask the testsuite, or GDB, to load the system gdbinit but +not ~/.gdbinit, a workaround is then to remove `-nx' from +INTERNAL_GDBFLAGS, and point $HOME at a directory without +a .gdbinit. For example: + + cd testsuite + HOME=`pwd` runtest \ + GDB=/usr/bin/gdb \ + GDBSERVER=/usr/bin/gdbserver \ + INTERNAL_GDBFLAGS=-nw + +GDB_PARALLEL + +When testing natively (that is, not with a remote host), you can run +the GDB test suite in a fully parallel mode. In this mode, each .exp +file runs separately and maybe simultaneously. The test suite will +ensure that all the temporary files created by the test suite do not +clash, by putting them into separate directories. This mode is +primarily intended for use by the Makefile. + +To use this mode, set the GDB_PARALLEL on the runtest command line. +Before starting the tests, you must ensure that the directories cache, +outputs, and temp in the test suite build directory are either empty +or have been deleted. cache in particular is used to share data +across invocations of runtest, and files there may affect the test +results. Note that the Makefile automatically does these deletions. + +GDB_INOTIFY + +For debugging parallel mode, it is handy to be able to see when a test +case writes to a file outside of its designated output directory. + +If you have the inotify-tools package installed, you can set the +GDB_INOTIFY variable on the runtest command line. This will cause the +test suite to watch for parallel-unsafe file creations and report +them, both to stdout and in the test suite log file. + +This setting is only meaningful in conjunction with GDB_PARALLEL. + + +Testsuite Configuration +*********************** + +It is possible to adjust the behavior of the testsuite by defining +the global variables listed below, either in a `site.exp' file, +or in a board file. + +gdb_test_timeout + +Defining this variable changes the default timeout duration used +during communication with GDB. More specifically, the global variable +used during testing is `timeout', but this variable gets reset to +`gdb_test_timeout' at the beginning of each testcase, which ensures +that any local change to `timeout' in a testcase does not affect +subsequent testcases. + +This global variable comes in handy when the debugger is slower than +normal due to the testing environment, triggering unexpected `TIMEOUT' +test failures. Examples include when testing on a remote machine, or +against a system where communications are slow. + +If not specifically defined, this variable gets automatically defined +to the same value as `timeout' during the testsuite initialization. +The default value of the timeout is defined in the file +`testsuite/config/unix.exp' (at least for Unix hosts; board files may +have their own values). + + +Board Settings +************** + +DejaGNU includes the concept of a "board file", which specifies +testing details for a particular target (which are often bare circuit +boards, thus the name). + +In the GDB testsuite specifically, the board file may include a +number of "board settings" that test cases may check before deciding +whether to exercise a particular feature. For instance, a board +lacking any I/O devices, or perhaps simply having its I/O devices +not wired up, should set `noinferiorio'. + +Here are the supported board settings: + +gdb,cannot_call_functions + + The board does not support inferior call, that is, invoking inferior + functions in GDB. + +gdb,can_reverse + + The board supports reverse execution. + +gdb,no_hardware_watchpoints + + The board does not support hardware watchpoints. + +gdb,nofileio + + GDB is unable to intercept target file operations in remote and + perform them on the host. + +gdb,noinferiorio + + The board is unable to provide I/O capability to the inferior. + +gdb,noresults + + A program will not return an exit code or result code (or the value + of the result is undefined, and should not be looked at). + +gdb,nosignals + + The board does not support signals. + +gdb,skip_huge_test + + Skip time-consuming tests on the board with slow connection. + +gdb,skip_float_tests + + Skip tests related to floating point. + +gdb,use_precord + + The board supports process record. + +gdb_server_prog + + The location of GDBserver. If GDBserver somewhere other than its + default location is used in test, specify the location of GDBserver in + this variable. The location is a file name for GDBserver, and may be + either absolute or relative to the testsuite subdirectory of the build + directory. + +in_proc_agent + + The location of the in-process agent (used for fast tracepoints and + other special tests). If the in-process agent of interest is anywhere + other than its default location, set this variable. The location is a + filename, and may be either absolute or relative to the testsuite + subdirectory of the build directory. + +noargs + + GDB does not support argument passing for inferior. + +no_long_long + + The board does not support type long long. + +use_cygmon + + The board is running the monitor Cygmon. + +use_gdb_stub + + The tests are running with a GDB stub. + +gdb,predefined_tsv + + The predefined trace state variables the board has. + + +Testsuite Organization +********************** + +The testsuite is entirely contained in `gdb/testsuite'. The main +directory of the testsuite includes some makefiles and configury, but +these are minimal, and used for little besides cleaning up, since the +tests themselves handle the compilation of the programs that GDB will +run. + +The file `testsuite/lib/gdb.exp' contains common utility procs useful +for all GDB tests, while the directory testsuite/config contains +configuration-specific files, typically used for special-purpose +definitions of procs like `gdb_load' and `gdb_start'. + +The tests themselves are to be found in directories named +'testsuite/gdb.* and subdirectories of those. The names of the test +files must always end with ".exp". DejaGNU collects the test files by +wildcarding in the test directories, so both subdirectories and +individual files typically get chosen and run in alphabetical order. + +The following lists some notable types of subdirectories and what they +are for. Since DejaGNU finds test files no matter where they are +located, and since each test file sets up its own compilation and +execution environment, this organization is simply for convenience and +intelligibility. + +gdb.base + +This is the base testsuite. The tests in it should apply to all +configurations of GDB (but generic native-only tests may live here). +The test programs should be in the subset of C that is both valid +ANSI/ISO C, and C++. + +gdb. + +Language-specific tests for any language besides C. Examples are +gdb.cp for C++ and gdb.java for Java. + +gdb. + +Non-portable tests. The tests are specific to a specific +configuration (host or target), such as HP-UX or eCos. Example is +gdb.hp, for HP-UX. + +gdb.arch + +Architecture-specific tests that are (usually) cross-platform. + +gdb. + +Tests that exercise a specific GDB subsystem in more depth. For +instance, gdb.disasm exercises various disassemblers, while +gdb.stabs tests pathways through the stabs symbol reader. + +Writing Tests +************* + +In many areas, the GDB tests are already quite comprehensive; you +should be able to copy existing tests to handle new cases. Be aware +that older tests may use obsolete practices but have not yet been +updated. + +You should try to use `gdb_test' whenever possible, since it includes +cases to handle all the unexpected errors that might happen. However, +it doesn't cost anything to add new test procedures; for instance, +gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test' +multiple times. + +Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even +if GDB has several valid responses to a command, you can use +`gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes +internal errors and unexpected prompts. + +Do not write tests which expect a literal tab character from GDB. On +some operating systems (e.g. OpenBSD) the TTY layer expands tabs to +spaces, so by the time GDB's output reaches `expect' the tab is gone. + +The source language programs do *not* need to be in a consistent +style. Since GDB is used to debug programs written in many different +styles, it's worth having a mix of styles in the testsuite; for +instance, some GDB bugs involving the display of source lines might +never manifest themselves if the test programs used GNU coding style +uniformly. + +Some testcase results need more detailed explanation: + +KFAIL + +Use KFAIL for known problem of GDB itself. You must specify the GDB +bug report number, as in these sample tests: + + kfail "gdb/13392" "continue to marker 2" + +or + + setup_kfail gdb/13392 "*-*-*" + kfail "continue to marker 2" + + +XFAIL + +Short for "expected failure", this indicates a known problem with the +environment. This could include limitations of the operating system, +compiler version, and other components. + +This example from gdb.base/attach-pie-misread.exp is a sanity check +for the target environment: + + # On x86_64 it is commonly about 4MB. + if {$stub_size > 25000000} { + xfail "stub size $stub_size is too large" + return + } + +You should provide bug report number for the failing component of the +environment, if such bug report is available, as with this example +referring to a GCC problem: + + if {[test_compiler_info {gcc-[0-3]-*}] + || [test_compiler_info {gcc-4-[0-5]-*}]} { + setup_xfail "gcc/46955" *-*-* + } + gdb_test "python print ttype.template_argument(2)" "&C::c" + +Note that it is also acceptable, and often preferable, to avoid +running the test at all. This is the better option if the limitation +is intrinsic to the environment, rather than a bug expected to be +fixed in the near future. -- 2.30.2