By inspection, I noticed a path where we return without discarding the
cleanups.
gdb/ChangeLog:
2015-04-01  Pedro Alves  <palves@redhat.com>
	* infrun.c (keep_going): Also discard cleanups if inserting
	breakpoints fails.
+2015-04-01  Pedro Alves  <palves@redhat.com>
+
+       * infrun.c (keep_going): Also discard cleanups if inserting
+       breakpoints fails.
+
 2015-04-01  Pedro Alves  <palves@redhat.com>
 
        * infrun.c (wait_for_inferior): Install the
 
        {
          exception_print (gdb_stderr, e);
          stop_waiting (ecs);
+         discard_cleanups (old_cleanups);
          return;
        }
       END_CATCH