From: Simon Marchi Date: Tue, 7 Aug 2018 22:14:20 +0000 (-0400) Subject: Add DWARF index cache X-Git-Url: https://git.libre-soc.org/?a=commitdiff_plain;h=87d6a7aa931f2bd4cfe784344b6a2cd595f6f2c9;p=binutils-gdb.git Add DWARF index cache New in v3: - Remove things related to the dwarf-5 format. - Fix compilation on mingw (scoped_mmap.c). GDB can generate indexes for DWARF debug information, which, when integrated in the original binary, can speed up loading object files. This can be done using the gdb-add-index script or directly by the linker itself. However, not many people know about this. And even among those who do, because it requires additional steps, I don't know a lot of people who actually go through that trouble. To help make using the DWARF index more transparent, this patch introduces a DWARF index cache. When enabled, loading an index-less binary in GDB will automatically save an index file in ~/.cache/gdb. When loading that same object file again, the index file will be looked up and used to load the DWARF index. You therefore get the benefit of the DWARF index without having to do additional manual steps or modifying your build system. When an index section is already present in the file, GDB will prefer that one over looking up the cache. When doing my edit-compile-debug cycle, I often debug multiple times the same build, so the cache helps reducing the load time of the debug sessions after the first one. - The saved index file is exactly the same as the output of the "save gdb-index" command. It is therefore the exact same content that would be found in the .gdb_index or .debug_names section. We just leave it as a standalone file instead of merging it in the binary. - The cache is just a directory with files named after the object file's build-id. It is not possible to save/load the index for an object file without build-id in the cache. - The cache uses the gdb index format. The problem with the dwarf-5 format is that we can generate an addendum to the .debug_str section that you're supposed to integrate to the original binary. This complicates a little bit loading the data from the cached index files, so I would leave this for later. - The size taken up by ~/.cache/gdb is not limited. I was thinking we could add configurable limit (like ccache does), but that would come after. Also, maybe a command to flush the cache. - The cache is disabled by default. I think once it's been out there and tested for a while, it could be turned on by default, so that everybody can enjoy it. - The code was made to follow the XDG specification: if the XDG_CACHE_HOME environment variable, it is used, otherwise it falls back to ~/.cache/gdb. It is possible to change it using "set index-cache directory". On other OSes than GNU/Linux, ~/.cache may not be the best place to put such data. On macOS it should probably default to ~/Library/Caches/... On Windows, %LocalAppData%/... I don't intend to do this part, but further patches are welcome. - I think that we need to be careful that multiple instances of GDB don't interfere with each other (not far fetched at all if you run GDB in some automated script) and the cache is always coherent (either the file is not found, or it is found and entirely valid). Writing the file directly to its final location seems like a recipe for failure. One GDB could read a file in the index while it is being written by another GDB. To mitigate this, I made write_psymtabs_to_index write to temporary files and rename them once it's done. Two GDB instances writing the index for the same file should not step on each other's toes (the last file to be renamed will stay). A GDB looking up a file will only see a complete file or no file. Also, if GDB crashes while generating the index file, it will leave a work-in-progress file, but it won't be picked up by other instances looking up in the cache. gdb/ChangeLog: * common/pathstuff.h (get_standard_cache_dir): New. * common/pathstuff.c (get_standard_cache_dir): New. * build-id.h (build_id_to_string): New. * dwarf-index-common.h (INDEX4_SUFFIX, INDEX5_SUFFIX, DEBUG_STR_SUFFIX): Move to here. * dwarf-index-write.c (INDEX4_SUFFIX, INDEX5_SUFFIX, DEBUG_STR_SUFFIX): Move from there. (write_psymtabs_to_index): Make non-static, add basename parameter. Write to temporary files, rename when done. (save_gdb_index_command): Adjust call to write_psymtabs_to_index. * dwarf2read.h (dwarf2_per_objfile) : New field. * dwarf2read.c (dwz_file) : New field. (get_gdb_index_contents_from_cache): New. (get_gdb_index_contents_from_cache_dwz): New. (dwarf2_initialize_objfile): Read index from cache. (dwarf2_build_psymtabs): Save to index. * dwarf-index-cache.h: New file. * dwarf-index-cache.c: New file. * dwarf-index-write.h: New file. gdb/testsuite/ChangeLog: * boards/index-cache-gdb.exp: New file. * gdb.dwarf2/index-cache.exp: New file. * gdb.dwarf2/index-cache.c: New file. * gdb.base/maint.exp: Check if we are using the index cache. --- diff --git a/gdb/ChangeLog b/gdb/ChangeLog index b73f90b0333..d6e3b71faa0 100644 --- a/gdb/ChangeLog +++ b/gdb/ChangeLog @@ -1,3 +1,27 @@ +2018-08-07 Simon Marchi + + * common/pathstuff.h (get_standard_cache_dir): New. + * common/pathstuff.c (get_standard_cache_dir): New. + * build-id.h (build_id_to_string): New. + * dwarf-index-common.h (INDEX4_SUFFIX, INDEX5_SUFFIX, + DEBUG_STR_SUFFIX): Move to here. + * dwarf-index-write.c (INDEX4_SUFFIX, INDEX5_SUFFIX, + DEBUG_STR_SUFFIX): Move from there. + (write_psymtabs_to_index): Make non-static, add basename + parameter. Write to temporary files, rename when done. + (save_gdb_index_command): Adjust call to + write_psymtabs_to_index. + * dwarf2read.h (dwarf2_per_objfile) : New + field. + * dwarf2read.c (dwz_file) : New field. + (get_gdb_index_contents_from_cache): New. + (get_gdb_index_contents_from_cache_dwz): New. + (dwarf2_initialize_objfile): Read index from cache. + (dwarf2_build_psymtabs): Save to index. + * dwarf-index-cache.h: New file. + * dwarf-index-cache.c: New file. + * dwarf-index-write.h: New file. + 2018-08-07 Simon Marchi * gnulib/aclocal.m4: Re-generate. diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 915e88c5fda..9e90a4608d6 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -993,6 +993,7 @@ COMMON_SFILES = \ disasm.c \ disasm-selftests.c \ dummy-frame.c \ + dwarf-index-cache.c \ dwarf-index-common.c \ dwarf-index-write.c \ dwarf2-frame.c \ @@ -1217,6 +1218,7 @@ HFILES_NO_SRCDIR = \ dictionary.h \ disasm.h \ dummy-frame.h \ + dwarf-index-cache.h \ dwarf-index-common.h \ dwarf2-frame.h \ dwarf2-frame-tailcall.h \ diff --git a/gdb/build-id.h b/gdb/build-id.h index 15fb6094097..796e5ac2aff 100644 --- a/gdb/build-id.h +++ b/gdb/build-id.h @@ -21,6 +21,7 @@ #define BUILD_ID_H #include "gdb_bfd.h" +#include "common/rsp-low.h" /* Locate NT_GNU_BUILD_ID from ABFD and return its content. */ @@ -47,4 +48,14 @@ extern gdb_bfd_ref_ptr build_id_to_debug_bfd (size_t build_id_len, extern std::string find_separate_debug_file_by_buildid (struct objfile *objfile); +/* Return an hex-string representation of BUILD_ID. */ + +static inline std::string +build_id_to_string (const bfd_build_id *build_id) +{ + gdb_assert (build_id != NULL); + + return bin2hex (build_id->data, build_id->size); +} + #endif /* BUILD_ID_H */ diff --git a/gdb/common/pathstuff.c b/gdb/common/pathstuff.c index 8c4093fc38d..2d9dcb25d40 100644 --- a/gdb/common/pathstuff.c +++ b/gdb/common/pathstuff.c @@ -158,3 +158,27 @@ contains_dir_separator (const char *path) return false; } + +/* See common/pathstuff.h. */ + +std::string +get_standard_cache_dir () +{ + char *xdg_cache_home = getenv ("XDG_CACHE_HOME"); + if (xdg_cache_home != NULL) + { + /* Make sure the path is absolute and tilde-expanded. */ + gdb::unique_xmalloc_ptr abs (gdb_abspath (xdg_cache_home)); + return string_printf ("%s/gdb", abs.get ()); + } + + char *home = getenv ("HOME"); + if (home != NULL) + { + /* Make sure the path is absolute and tilde-expanded. */ + gdb::unique_xmalloc_ptr abs (gdb_abspath (home)); + return string_printf ("%s/.cache/gdb", abs.get ()); + } + + return {}; +} diff --git a/gdb/common/pathstuff.h b/gdb/common/pathstuff.h index 9f261273e68..d1aa6b306c3 100644 --- a/gdb/common/pathstuff.h +++ b/gdb/common/pathstuff.h @@ -50,4 +50,14 @@ extern gdb::unique_xmalloc_ptr gdb_abspath (const char *path); extern bool contains_dir_separator (const char *path); +/* Get the usual user cache directory for the current platform. + + On Linux, it follows the XDG Base Directory specification: use + $XDG_CACHE_HOME/gdb if the XDG_CACHE_HOME environment variable is defined, + otherwise $HOME/.cache. The return value is absolute and tilde-expanded. + + Return an empty string if neither XDG_CACHE_HOME or HOME are defined. */ + +extern std::string get_standard_cache_dir (); + #endif /* PATHSTUFF_H */ diff --git a/gdb/dwarf-index-cache.c b/gdb/dwarf-index-cache.c new file mode 100644 index 00000000000..57118d62cd1 --- /dev/null +++ b/gdb/dwarf-index-cache.c @@ -0,0 +1,463 @@ +/* Caching of GDB/DWARF index files. + + Copyright (C) 1994-2018 Free Software Foundation, Inc. + + This file is part of GDB. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include "defs.h" +#include "dwarf-index-cache.h" + +#include "build-id.h" +#include "cli/cli-cmds.h" +#include "command.h" +#include "common/scoped_mmap.h" +#include "common/pathstuff.h" +#include "dwarf-index-write.h" +#include "dwarf2read.h" +#include "objfiles.h" +#include "selftest.h" +#include +#include + +/* When set to 1, show debug messages about the index cache. */ +static int debug_index_cache = 0; + +/* The index cache directory, used for "set/show index-cache directory". */ +static char *index_cache_directory = NULL; + +/* See dwarf-index.cache.h. */ +index_cache global_index_cache; + +/* set/show index-cache commands. */ +static cmd_list_element *set_index_cache_prefix_list; +static cmd_list_element *show_index_cache_prefix_list; + +/* A cheap (as in low-quality) recursive mkdir. Try to create all the parents + directories up to DIR and DIR itself. Stop if we hit an error along the way. + There is no attempt to remove created directories in case of failure. */ + +static void +mkdir_recursive (const char *dir) +{ + gdb::unique_xmalloc_ptr holder (xstrdup (dir)); + char * const start = holder.get (); + char *component_start = start; + char *component_end = start; + + while (1) + { + /* Find the beginning of the next component. */ + while (*component_start == '/') + component_start++; + + /* Are we done? */ + if (*component_start == '\0') + return; + + /* Find the slash or null-terminator after this component. */ + component_end = component_start; + while (*component_end != '/' && *component_end != '\0') + component_end++; + + /* Temporarily replace the slash with a null terminator, so we can create + the directory up to this component. */ + char saved_char = *component_end; + *component_end = '\0'; + + /* If we get EEXIST and the existing path is a directory, then we're + happy. If it exists, but it's a regular file and this is not the last + component, we'll fail at the next component. If this is the last + component, the caller will fail with ENOTDIR when trying to + open/create a file under that path. */ + if (mkdir (start, 0700) != 0) + if (errno != EEXIST) + return; + + /* Restore the overwritten char. */ + *component_end = saved_char; + component_start = component_end; + } +} + +/* Default destructor of index_cache_resource. */ +index_cache_resource::~index_cache_resource () = default; + +/* See dwarf-index-cache.h. */ + +void +index_cache::set_directory (std::string dir) +{ + gdb_assert (!dir.empty ()); + + m_dir = std::move (dir); + + if (debug_index_cache) + printf_unfiltered ("index cache: now using directory %s\n", m_dir.c_str ()); +} + +/* See dwarf-index-cache.h. */ + +void +index_cache::enable () +{ + if (debug_index_cache) + printf_unfiltered ("index cache: enabling (%s)\n", m_dir.c_str ()); + + m_enabled = true; +} + +/* See dwarf-index-cache.h. */ + +void +index_cache::disable () +{ + if (debug_index_cache) + printf_unfiltered ("index cache: disabling\n"); + + m_enabled = false; +} + +/* See dwarf-index-cache.h. */ + +void +index_cache::store (struct dwarf2_per_objfile *dwarf2_per_objfile) +{ + objfile *obj = dwarf2_per_objfile->objfile; + + if (!enabled ()) + return; + + const bfd_build_id *build_id = build_id_bfd_get (obj->obfd); + if (build_id == nullptr) + { + if (debug_index_cache) + printf_unfiltered ("index cache: objfile %s has no build id\n", + objfile_name (obj)); + return; + } + + if (m_dir.empty ()) + { + warning (_("The index cache directory name is empty, skipping store.")); + return; + } + + std::string build_id_str = build_id_to_string (build_id); + + TRY + { + /* Try to create the containing directory. */ + mkdir_recursive (m_dir.c_str ()); + + if (debug_index_cache) + printf_unfiltered ("index cache: writing index cache for objfile %s\n", + objfile_name (obj)); + + /* Write the index itself to the directory, using the build id as the + filename. */ + write_psymtabs_to_index (dwarf2_per_objfile, m_dir.c_str (), + build_id_str.c_str (), dw_index_kind::GDB_INDEX); + } + CATCH (except, RETURN_MASK_ERROR) + { + if (debug_index_cache) + printf_unfiltered ("index cache: couldn't store index cache for objfile " + "%s: %s", objfile_name (obj), except.message); + } + END_CATCH +} + +#if HAVE_SYS_MMAN_H + +/* Hold the resources for an mmapped index file. */ + +struct index_cache_resource_mmap final : public index_cache_resource +{ + /* Try to mmap FILENAME. Throw an exception on failure, including if the + file doesn't exist. */ + index_cache_resource_mmap (const char *filename) + : mapping (mmap_file (filename)) + {} + + scoped_mmap mapping; +}; + +/* See dwarf-index-cache.h. */ + +gdb::array_view +index_cache::lookup_gdb_index (const bfd_build_id *build_id, + std::unique_ptr *resource) +{ + if (!enabled ()) + return {}; + + if (m_dir.empty ()) + { + warning (_("The index cache directory name is empty, skipping cache " + "lookup.")); + return {}; + } + + /* Compute where we would expect a gdb index file for this build id to be. */ + std::string filename = make_index_filename (build_id, INDEX4_SUFFIX); + + TRY + { + if (debug_index_cache) + printf_unfiltered ("index cache: trying to read %s\n", + filename.c_str ()); + + /* Try to map that file. */ + index_cache_resource_mmap *mmap_resource + = new index_cache_resource_mmap (filename.c_str ()); + + /* Yay, it worked! Hand the resource to the caller. */ + resource->reset (mmap_resource); + + return gdb::array_view + ((const gdb_byte *) mmap_resource->mapping.get (), + mmap_resource->mapping.size ()); + } + CATCH (except, RETURN_MASK_ERROR) + { + if (debug_index_cache) + printf_unfiltered ("index cache: couldn't read %s: %s\n", + filename.c_str (), except.message); + } + END_CATCH + + return {}; +} + +#else /* !HAVE_SYS_MMAN_H */ + +/* See dwarf-index-cache.h. This is a no-op on unsupported systems. */ + +gdb::array_view +index_cache::lookup_gdb_index (const bfd_build_id *build_id, + std::unique_ptr *resource) +{ + return {}; +} + +#endif + +/* See dwarf-index-cache.h. */ + +std::string +index_cache::make_index_filename (const bfd_build_id *build_id, + const char *suffix) const +{ + std::string build_id_str = build_id_to_string (build_id); + + return m_dir + SLASH_STRING + build_id_str + suffix; +} + +/* "set index-cache" handler. */ + +static void +set_index_cache_command (const char *arg, int from_tty) +{ + printf_unfiltered (_("\ +Missing arguments. See \"help set index-cache\" for help.\n")); +} + +/* True when we are executing "show index-cache". This is used to improve the + printout a little bit. */ +static bool in_show_index_cache_command = false; + +/* "show index-cache" handler. */ + +static void +show_index_cache_command (const char *arg, int from_tty) +{ + /* Note that we are executing "show index-cache". */ + auto restore_flag = make_scoped_restore (&in_show_index_cache_command, true); + + /* Call all "show index-cache" subcommands. */ + cmd_show_list (show_index_cache_prefix_list, from_tty, ""); + + printf_unfiltered ("\n"); + printf_unfiltered + (_("The index cache is currently %s.\n"), + global_index_cache.enabled () ? _("enabled") : _("disabled")); +} + +/* "set index-cache on" handler. */ + +static void +set_index_cache_on_command (const char *arg, int from_tty) +{ + global_index_cache.enable (); +} + +/* "set index-cache off" handler. */ + +static void +set_index_cache_off_command (const char *arg, int from_tty) +{ + global_index_cache.disable (); +} + +/* "set index-cache directory" handler. */ + +static void +set_index_cache_directory_command (const char *arg, int from_tty, + cmd_list_element *element) +{ + /* Make sure the index cache directory is absolute and tilde-expanded. */ + gdb::unique_xmalloc_ptr abs (gdb_abspath (index_cache_directory)); + xfree (index_cache_directory); + index_cache_directory = abs.release (); + global_index_cache.set_directory (index_cache_directory); +} + +/* "show index-cache stats" handler. */ + +static void +show_index_cache_stats_command (const char *arg, int from_tty) +{ + const char *indent = ""; + + /* If this command is invoked through "show index-cache", make the display a + bit nicer. */ + if (in_show_index_cache_command) + { + indent = " "; + printf_unfiltered ("\n"); + } + + printf_unfiltered (_("%s Cache hits (this session): %u\n"), + indent, global_index_cache.n_hits ()); + printf_unfiltered (_("%sCache misses (this session): %u\n"), + indent, global_index_cache.n_misses ()); +} + +#if GDB_SELF_TEST && defined (HAVE_MKDTEMP) +namespace selftests +{ + +/* Try to create DIR using mkdir_recursive and make sure it exists. */ + +static bool +create_dir_and_check (const char *dir) +{ + mkdir_recursive (dir); + + struct stat st; + if (stat (dir, &st) != 0) + perror_with_name ("stat"); + + return (st.st_mode & S_IFDIR) != 0; +} + +/* Test mkdir_recursive. */ + +static void +test_mkdir_recursive () +{ + char base[] = "/tmp/gdb-selftests-XXXXXX"; + + if (mkdtemp (base) == NULL) + perror_with_name ("mkdtemp"); + + /* Try not to leave leftover directories. */ + struct cleanup_dirs { + cleanup_dirs (const char *base) + : m_base (base) + {} + + ~cleanup_dirs () { + rmdir (string_printf ("%s/a/b/c/d/e", m_base).c_str ()); + rmdir (string_printf ("%s/a/b/c/d", m_base).c_str ()); + rmdir (string_printf ("%s/a/b/c", m_base).c_str ()); + rmdir (string_printf ("%s/a/b", m_base).c_str ()); + rmdir (string_printf ("%s/a", m_base).c_str ()); + rmdir (m_base); + } + + private: + const char *m_base; + } cleanup_dirs (base); + + std::string dir = string_printf ("%s/a/b", base); + SELF_CHECK (create_dir_and_check (dir.c_str ())); + + dir = string_printf ("%s/a/b/c//d/e/", base); + SELF_CHECK (create_dir_and_check (dir.c_str ())); +} +} +#endif /* GDB_SELF_TEST && defined (HAVE_MKDTEMP) */ + +void +_initialize_index_cache () +{ + /* Set the default index cache directory. */ + std::string cache_dir = get_standard_cache_dir (); + if (!cache_dir.empty ()) + { + index_cache_directory = xstrdup (cache_dir.c_str ()); + global_index_cache.set_directory (std::move (cache_dir)); + } + else + warning (_("Couldn't determine a path for the index cache directory.")); + + /* set index-cache */ + add_prefix_cmd ("index-cache", class_files, set_index_cache_command, + _("Set index-cache options"), &set_index_cache_prefix_list, + "set index-cache ", false, &setlist); + + /* show index-cache */ + add_prefix_cmd ("index-cache", class_files, show_index_cache_command, + _("Show index-cache options"), &show_index_cache_prefix_list, + "show index-cache ", false, &showlist); + + /* set index-cache on */ + add_cmd ("on", class_files, set_index_cache_on_command, + _("Enable the index cache."), &set_index_cache_prefix_list); + + /* set index-cache off */ + add_cmd ("off", class_files, set_index_cache_off_command, + _("Disable the index cache."), &set_index_cache_prefix_list); + + /* set index-cache directory */ + add_setshow_filename_cmd ("directory", class_files, &index_cache_directory, + _("Set the directory of the index cache."), + _("Show the directory of the index cache."), + NULL, + set_index_cache_directory_command, NULL, + &set_index_cache_prefix_list, + &show_index_cache_prefix_list); + + /* show index-cache stats */ + add_cmd ("stats", class_files, show_index_cache_stats_command, + _("Show some stats about the index cache."), + &show_index_cache_prefix_list); + + /* set debug index-cache */ + add_setshow_boolean_cmd ("index-cache", class_maintenance, + &debug_index_cache, + _("Set display of index-cache debug messages."), + _("Show display of index-cache debug messages."), + _("\ +When non-zero, debugging output for the index cache is displayed."), + NULL, NULL, + &setdebuglist, &showdebuglist); + +#if GDB_SELF_TEST && defined (HAVE_MKDTEMP) + selftests::register_test ("mkdir_recursive", selftests::test_mkdir_recursive); +#endif +} diff --git a/gdb/dwarf-index-cache.h b/gdb/dwarf-index-cache.h new file mode 100644 index 00000000000..b4fa8c4a635 --- /dev/null +++ b/gdb/dwarf-index-cache.h @@ -0,0 +1,112 @@ +/* Caching of GDB/DWARF index files. + + Copyright (C) 2018 Free Software Foundation, Inc. + + This file is part of GDB. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#ifndef DWARF_INDEX_CACHE_H +#define DWARF_INDEX_CACHE_H + +#include "dwarf-index-common.h" +#include "common/array-view.h" +#include "symfile.h" + +/* Base of the classes used to hold the resources of the indices loaded from + the cache (e.g. mmapped files). */ + +struct index_cache_resource +{ + virtual ~index_cache_resource () = 0; +}; + +/* Class to manage the access to the DWARF index cache. */ + +class index_cache +{ +public: + /* Change the directory used to save/load index files. */ + void set_directory (std::string dir); + + /* Return true if the usage of the cache is enabled. */ + bool enabled () const + { + return m_enabled; + } + + /* Enable the cache. */ + void enable (); + + /* Disable the cache. */ + void disable (); + + /* Store an index for the specified object file in the cache. */ + void store (struct dwarf2_per_objfile *dwarf2_per_objfile); + + /* Look for an index file matching BUILD_ID. If found, return the contents + as an array_view and store the underlying resources (allocated memory, + mapped file, etc) in RESOURCE. The returned array_view is valid as long + as RESOURCE is not destroyed. + + If no matching index file is found, return an empty array view. */ + gdb::array_view + lookup_gdb_index (const bfd_build_id *build_id, + std::unique_ptr *resource); + + /* Return the number of cache hits. */ + unsigned int n_hits () const + { return m_n_hits; } + + /* Record a cache hit. */ + void hit () + { + if (enabled ()) + m_n_hits++; + } + + /* Return the number of cache misses. */ + unsigned int n_misses () const + { return m_n_misses; } + + /* Record a cache miss. */ + void miss () + { + if (enabled ()) + m_n_misses++; + } + +private: + + /* Compute the absolute filename where the index of the objfile with build + id BUILD_ID will be stored. SUFFIX is appended at the end of the + filename. */ + std::string make_index_filename (const bfd_build_id *build_id, + const char *suffix) const; + + /* The base directory where we are storing and looking up index files. */ + std::string m_dir; + + /* Whether the cache is enabled. */ + bool m_enabled = false; + + /* Number of cache hits and misses during this GDB session. */ + unsigned int m_n_hits = 0; + unsigned int m_n_misses = 0; +}; + +/* The global instance of the index cache. */ +extern index_cache global_index_cache; + +#endif /* DWARF_INDEX_CACHE_H */ diff --git a/gdb/dwarf-index-common.h b/gdb/dwarf-index-common.h index 32774c7bf62..742eba6887b 100644 --- a/gdb/dwarf-index-common.h +++ b/gdb/dwarf-index-common.h @@ -20,6 +20,11 @@ #ifndef DWARF_INDEX_COMMON_H #define DWARF_INDEX_COMMON_H +/* The suffix for an index file. */ +#define INDEX4_SUFFIX ".gdb-index" +#define INDEX5_SUFFIX ".debug_names" +#define DEBUG_STR_SUFFIX ".debug_str" + /* All offsets in the index are of this type. It must be architecture-independent. */ typedef uint32_t offset_type; diff --git a/gdb/dwarf-index-write.c b/gdb/dwarf-index-write.c index fd245a09d89..100619d7ad9 100644 --- a/gdb/dwarf-index-write.c +++ b/gdb/dwarf-index-write.c @@ -24,6 +24,7 @@ #include "common/byte-vector.h" #include "common/filestuff.h" #include "common/gdb_unlinker.h" +#include "common/scoped_fd.h" #include "complaints.h" #include "dwarf-index-common.h" #include "dwarf2.h" @@ -39,11 +40,6 @@ #include #include -/* The suffix for an index file. */ -#define INDEX4_SUFFIX ".gdb-index" -#define INDEX5_SUFFIX ".debug_names" -#define DEBUG_STR_SUFFIX ".debug_str" - /* Ensure only legit values are used. */ #define DW2_GDB_INDEX_SYMBOL_STATIC_SET_VALUE(cu_index, value) \ do { \ @@ -1541,11 +1537,11 @@ assert_file_size (FILE *file, const char *filename, size_t expected_size) gdb_assert (file_size == expected_size); } -/* Create an index file for OBJFILE in the directory DIR. */ +/* See dwarf-index-write.h. */ -static void +void write_psymtabs_to_index (struct dwarf2_per_objfile *dwarf2_per_objfile, - const char *dir, + const char *dir, const char *basename, dw_index_kind index_kind) { struct objfile *objfile = dwarf2_per_objfile->objfile; @@ -1563,50 +1559,85 @@ write_psymtabs_to_index (struct dwarf2_per_objfile *dwarf2_per_objfile, if (stat (objfile_name (objfile), &st) < 0) perror_with_name (objfile_name (objfile)); - std::string filename (std::string (dir) + SLASH_STRING - + lbasename (objfile_name (objfile)) + /* Make a filename suitable to pass to mkstemp based on F (e.g. + /tmp/foo -> /tmp/foo-XXXXXX). */ + auto make_temp_filename = [] (const std::string &f) -> gdb::char_vector + { + gdb::char_vector filename_temp (f.length () + 8); + strcpy (filename_temp.data (), f.c_str ()); + strcat (filename_temp.data () + f.size (), "-XXXXXX"); + return filename_temp; + }; + + std::string filename (std::string (dir) + SLASH_STRING + basename + (index_kind == dw_index_kind::DEBUG_NAMES ? INDEX5_SUFFIX : INDEX4_SUFFIX)); + gdb::char_vector filename_temp = make_temp_filename (filename); - FILE *out_file = gdb_fopen_cloexec (filename.c_str (), "wb").release (); - if (!out_file) - error (_("Can't open `%s' for writing"), filename.c_str ()); + gdb::optional out_file_fd + (gdb::in_place, mkstemp (filename_temp.data ())); + if (out_file_fd->get () == -1) + perror_with_name ("mkstemp"); - /* Order matters here; we want FILE to be closed before FILENAME is + FILE *out_file = gdb_fopen_cloexec (filename_temp.data (), "wb").release (); + if (out_file == nullptr) + error (_("Can't open `%s' for writing"), filename_temp.data ()); + + /* Order matters here; we want FILE to be closed before FILENAME_TEMP is unlinked, because on MS-Windows one cannot delete a file that is still open. (Don't call anything here that might throw until - file_closer is created.) */ - gdb::unlinker unlink_file (filename.c_str ()); + file_closer is created.) We don't need OUT_FILE_FD anymore, so we might + as well close it now. */ + out_file_fd.reset (); + gdb::unlinker unlink_file (filename_temp.data ()); gdb_file_up close_out_file (out_file); if (index_kind == dw_index_kind::DEBUG_NAMES) { std::string filename_str (std::string (dir) + SLASH_STRING - + lbasename (objfile_name (objfile)) - + DEBUG_STR_SUFFIX); + + basename + DEBUG_STR_SUFFIX); + gdb::char_vector filename_str_temp = make_temp_filename (filename_str); + + gdb::optional out_file_str_fd + (gdb::in_place, mkstemp (filename_str_temp.data ())); + if (out_file_str_fd->get () == -1) + perror_with_name ("mkstemp"); + FILE *out_file_str - = gdb_fopen_cloexec (filename_str.c_str (), "wb").release (); - if (!out_file_str) - error (_("Can't open `%s' for writing"), filename_str.c_str ()); - gdb::unlinker unlink_file_str (filename_str.c_str ()); + = gdb_fopen_cloexec (filename_str_temp.data (), "wb").release (); + if (out_file_str == nullptr) + error (_("Can't open `%s' for writing"), filename_str_temp.data ()); + + out_file_str_fd.reset (); + gdb::unlinker unlink_file_str (filename_str_temp.data ()); gdb_file_up close_out_file_str (out_file_str); const size_t total_len = write_debug_names (dwarf2_per_objfile, out_file, out_file_str); - assert_file_size (out_file, filename.c_str (), total_len); + assert_file_size (out_file, filename_temp.data (), total_len); /* We want to keep the file .debug_str file too. */ unlink_file_str.keep (); + + /* Close and move the str file in place. */ + close_out_file_str.reset (); + if (rename (filename_str_temp.data (), filename_str.c_str ()) != 0) + perror_with_name ("rename"); } else { const size_t total_len = write_gdbindex (dwarf2_per_objfile, out_file); - assert_file_size (out_file, filename.c_str (), total_len); + assert_file_size (out_file, filename_temp.data (), total_len); } /* We want to keep the file. */ unlink_file.keep (); + + /* Close and move the file in place. */ + close_out_file.reset (); + if (rename (filename_temp.data (), filename.c_str ()) != 0) + perror_with_name ("rename"); } /* Implementation of the `save gdb-index' command. @@ -1651,7 +1682,9 @@ save_gdb_index_command (const char *arg, int from_tty) { TRY { - write_psymtabs_to_index (dwarf2_per_objfile, arg, index_kind); + const char *basename = lbasename (objfile_name (objfile)); + write_psymtabs_to_index (dwarf2_per_objfile, arg, basename, + index_kind); } CATCH (except, RETURN_MASK_ERROR) { diff --git a/gdb/dwarf-index-write.h b/gdb/dwarf-index-write.h new file mode 100644 index 00000000000..53b5f385603 --- /dev/null +++ b/gdb/dwarf-index-write.h @@ -0,0 +1,34 @@ +/* DWARF index writing support for GDB. + + Copyright (C) 2018 Free Software Foundation, Inc. + + This file is part of GDB. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#ifndef DWARF_INDEX_WRITE_H +#define DWARF_INDEX_WRITE_H + +#include "symfile.h" +#include "dwarf2read.h" + +/* Create an index file for OBJFILE in the directory DIR. BASENAME is the + desired filename, minus the extension, which gets added by this function + based on INDEX_KIND. */ + +extern void write_psymtabs_to_index + (struct dwarf2_per_objfile *dwarf2_per_objfile, const char *dir, + const char *basename, dw_index_kind index_kind); + +#endif /* DWARF_INDEX_WRITE_H */ diff --git a/gdb/dwarf2read.c b/gdb/dwarf2read.c index 296b50bc3c5..81a0087c26d 100644 --- a/gdb/dwarf2read.c +++ b/gdb/dwarf2read.c @@ -30,6 +30,7 @@ #include "defs.h" #include "dwarf2read.h" +#include "dwarf-index-cache.h" #include "dwarf-index-common.h" #include "bfd.h" #include "elf-bfd.h" @@ -868,6 +869,10 @@ struct dwz_file /* The dwz's BFD. */ gdb_bfd_ref_ptr dwz_bfd; + + /* If we loaded the index from an external file, this contains the + resources associated to the open file, memory mapping, etc. */ + std::unique_ptr index_cache_res; }; /* Struct used to pass misc. parameters to read_die_and_children, et @@ -6186,6 +6191,32 @@ get_gdb_index_contents_from_section (objfile *obj, T *section_owner) return {section->buffer, section->size}; } +/* Lookup the index cache for the contents of the index associated to + DWARF2_OBJ. */ + +static gdb::array_view +get_gdb_index_contents_from_cache (objfile *obj, dwarf2_per_objfile *dwarf2_obj) +{ + const bfd_build_id *build_id = build_id_bfd_get (obj->obfd); + if (build_id == nullptr) + return {}; + + return global_index_cache.lookup_gdb_index (build_id, + &dwarf2_obj->index_cache_res); +} + +/* Same as the above, but for DWZ. */ + +static gdb::array_view +get_gdb_index_contents_from_cache_dwz (objfile *obj, dwz_file *dwz) +{ + const bfd_build_id *build_id = build_id_bfd_get (dwz->dwz_bfd.get ()); + if (build_id == nullptr) + return {}; + + return global_index_cache.lookup_gdb_index (build_id, &dwz->index_cache_res); +} + /* See symfile.h. */ bool @@ -6237,6 +6268,17 @@ dwarf2_initialize_objfile (struct objfile *objfile, dw_index_kind *index_kind) return true; } + /* ... otherwise, try to find the index in the index cache. */ + if (dwarf2_read_gdb_index (dwarf2_per_objfile, + get_gdb_index_contents_from_cache, + get_gdb_index_contents_from_cache_dwz)) + { + global_index_cache.hit (); + *index_kind = dw_index_kind::GDB_INDEX; + return true; + } + + global_index_cache.miss (); return false; } @@ -6262,6 +6304,9 @@ dwarf2_build_psymtabs (struct objfile *objfile) psymtab_discarder psymtabs (objfile); dwarf2_build_psymtabs_hard (dwarf2_per_objfile); psymtabs.keep (); + + /* (maybe) store an index in the cache. */ + global_index_cache.store (dwarf2_per_objfile); } CATCH (except, RETURN_MASK_ERROR) { diff --git a/gdb/dwarf2read.h b/gdb/dwarf2read.h index 6e37c5d06f5..13855bcd547 100644 --- a/gdb/dwarf2read.h +++ b/gdb/dwarf2read.h @@ -20,6 +20,7 @@ #ifndef DWARF2READ_H #define DWARF2READ_H +#include "dwarf-index-cache.h" #include "filename-seen-cache.h" #include "gdb_obstack.h" @@ -245,6 +246,10 @@ public: /* Table containing all filenames. This is an optional because the table is lazily constructed on first access. */ gdb::optional filenames_cache; + + /* If we loaded the index from an external file, this contains the + resources associated to the open file, memory mapping, etc. */ + std::unique_ptr index_cache_res; }; /* Get the dwarf2_per_objfile associated to OBJFILE. */ diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog index f86e8fb2741..349aac9ef9b 100644 --- a/gdb/testsuite/ChangeLog +++ b/gdb/testsuite/ChangeLog @@ -1,3 +1,10 @@ +2018-08-07 Simon Marchi + + * boards/index-cache-gdb.exp: New file. + * gdb.dwarf2/index-cache.exp: New file. + * gdb.dwarf2/index-cache.c: New file. + * gdb.base/maint.exp: Check if we are using the index cache. + 2018-08-07 Rainer Orth * Makefile.in (saw_dash_j): Allow for GNU make 4.2+ passing -jN in diff --git a/gdb/testsuite/gdb.base/index-cache.c b/gdb/testsuite/gdb.base/index-cache.c new file mode 100644 index 00000000000..460d70ef336 --- /dev/null +++ b/gdb/testsuite/gdb.base/index-cache.c @@ -0,0 +1,23 @@ +/* This testcase is part of GDB, the GNU debugger. + + Copyright 2018 Free Software Foundation, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +int +main () +{ + return 0; +} + diff --git a/gdb/testsuite/gdb.base/index-cache.exp b/gdb/testsuite/gdb.base/index-cache.exp new file mode 100644 index 00000000000..87f3c91c070 --- /dev/null +++ b/gdb/testsuite/gdb.base/index-cache.exp @@ -0,0 +1,193 @@ +# Copyright 2018 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# This test checks that the index-cache feature generates the expected files at +# the expected location. + +standard_testfile + +if { [prepare_for_testing "failed to prepare" $testfile $srcfile] } { + return +} + +# List the files in DIR on the host (where GDB-under-test runs). +# Return a list of two elements: +# - 0 on success, -1 on failure +# - the list of files on success, empty on failure + +proc ls_host { dir } { + lassign [remote_exec host ls "-1 $dir"] ret output + + if { $ret != 0 } { + fail "failed to list files on host in $dir" + return -1 + } + + # ls -1 returns a list separated by \r\n. split will return a bunch of + # empty entries (it treats a sequence of split characters as separate + # fields, plus there is a \r\n at the end of the result). Ignore empty + # list elements. + set filtered {} + set files [split $output \r\n] + + foreach file $files { + if { $file != "" } { + lappend filtered $file + } + } + + return "0 $filtered" +} + +# Execute "show index-cache stats" and verify the output against expected +# values. + +proc check_cache_stats { expected_hits expected_misses } { + set re [multi_line \ + " Cache hits .this session.: $expected_hits" \ + "Cache misses .this session.: $expected_misses" \ + ] + + gdb_test "show index-cache stats" $re "check index-cache stats" +} + +# Run CODE using a fresh GDB configured based on the other parameters. + +proc run_test_with_flags { cache_dir cache_enabled code } { + global GDBFLAGS testfile + + save_vars { GDBFLAGS } { + set GDBFLAGS "$GDBFLAGS -iex \"set index-cache directory $cache_dir\"" + set GDBFLAGS "$GDBFLAGS -iex \"set index-cache $cache_enabled\"" + + clean_restart ${testfile} + + uplevel 1 $code + } +} + +# Test administrative stuff. + +proc_with_prefix test_basic_stuff { } { + global testfile + + clean_restart ${testfile} + + # Check that the index cache is disabled by default. + gdb_test \ + "show index-cache" \ + " is currently disabled." \ + "index-cache is disabled by default" + + # Test that we can enable it and "show index-cache" reflects that. + gdb_test_no_output "set index-cache on" "enable index cache" + gdb_test \ + "show index-cache" \ + " is currently enabled." \ + "index-cache is now enabled" + + # Test the "set/show index-cache directory" commands. + gdb_test "set index-cache directory" "Argument required.*" "set index-cache directory without arg" + gdb_test_no_output "set index-cache directory /tmp" "change the index cache directory" + gdb_test \ + "show index-cache directory" \ + "The directory of the index cache is \"/tmp\"." \ + "show index cache directory" +} + +# Test loading a binary with the cache disabled. No file should be created. + +proc_with_prefix test_cache_disabled { cache_dir } { + lassign [ls_host $cache_dir] ret files_before + + run_test_with_flags $cache_dir off { + lassign [ls_host $cache_dir] ret files_after + + set nfiles_created [expr [llength $files_after] - [llength $files_before]] + gdb_assert "$nfiles_created == 0" "no files were created" + + check_cache_stats 0 0 + } +} + +# Test with the cache enabled, we expect to have exactly one file created. + +proc_with_prefix test_cache_enabled_miss { cache_dir } { + global testfile + + lassign [ls_host $cache_dir] ret files_before + + run_test_with_flags $cache_dir on { + + lassign [ls_host $cache_dir] ret files_after + set nfiles_created [expr [llength $files_after] - [llength $files_before]] + gdb_assert "$nfiles_created > 0" "at least one file was created" + + set build_id [get_build_id [standard_output_file ${testfile}]] + if { $build_id == "" } { + fail "couldn't get executable build id" + return + } + + set expected_created_file [list "${build_id}.gdb-index"] + set found_idx [lsearch -exact $files_after $expected_created_file] + gdb_assert "$found_idx >= 0" "expected file is there" + + remote_exec host rm "-f $cache_dir/$expected_created_file" + + check_cache_stats 0 1 + } +} + + +# Test with the cache enabled, this time we should have one file (the +# same), but one cache read hit. + +proc_with_prefix test_cache_enabled_hit { cache_dir } { + # Just to populate the cache. + run_test_with_flags $cache_dir on {} + + lassign [ls_host $cache_dir] ret files_before + + run_test_with_flags $cache_dir on { + lassign [ls_host $cache_dir] ret files_after + set nfiles_created [expr [llength $files_after] - [llength $files_before]] + gdb_assert "$nfiles_created == 0" "no files were created" + + check_cache_stats 1 0 + } +} + +test_basic_stuff + +# The cache dir should be on the host (possibly remote), so we can't use the +# standard output directory for that (it's on the build machine). +lassign [remote_exec host mktemp -d] ret cache_dir + +if { $ret != 0 } { + fail "couldn't create temporary cache dir" + return +} + +# The ouput of mktemp contains an end of line, remove it. +set cache_dir [string trimright $cache_dir \r\n] + +test_cache_disabled $cache_dir +test_cache_enabled_miss $cache_dir +test_cache_enabled_hit $cache_dir + +# Test again with the cache disabled, now that it is populated. +test_cache_disabled $cache_dir + diff --git a/gdb/testsuite/gdb.base/maint.exp b/gdb/testsuite/gdb.base/maint.exp index ee0d08927bb..746081aee68 100644 --- a/gdb/testsuite/gdb.base/maint.exp +++ b/gdb/testsuite/gdb.base/maint.exp @@ -123,6 +123,30 @@ gdb_test_multiple "maint info sections .gdb_index .debug_names" "check for .gdb_ } } +# There also won't be any psymtabs if we read the index from the index cache. +# We can detect this by looking if the index-cache is enabled and if the number +# of cache misses is 0. +set index_cache_misses -1 +gdb_test_multiple "show index-cache stats" "check index cache stats" { + -re ".*Cache misses \\(this session\\): (\\d+)\r\n.*$gdb_prompt $" { + set index_cache_misses $expect_out(1,string) + } +} + +set using_index_cache 0 +gdb_test_multiple "show index-cache" "check index cache status" { + -re ".*is currently disabled.\r\n$gdb_prompt $" { + set using_index_cache 0 + } + -re ".*is currently enabled.\r\n$gdb_prompt $" { + set using_index_cache 1 + } +} + +if { $index_cache_misses == 0 && $using_index_cache } { + set have_gdb_index 1 +} + # # this command does not produce any output # unless there is some problem with the symtabs and psymtabs