cpu-kvm,arch-arm: Improve KvmCPU tick event scheduling

The memory-mapped timer emulated by gem5 is driven by the underlying
gem5 tick, which means that we must align the tick with the host time
to make the timer interrupt fire at a nearly native rate.

In each KVM execution round, the number of ticks incremented is
directly calculated from the number of instructions executed. However,
when a guest CPU switches to idle state, KVM seems to stay in kernel-
space until the POSIX timer set up in user-space raises an expiration
signal, instead of trapping to user-space immediately; and somehow the
instruction count is just too low to match the elapsed host time. This
makes the gem5 tick increment very slowly when the guest is idle and
drastically slow down workloads being sensitive to the guest time which
is driven by timer interrupt.

Before switching to KVM to execute the guest code, gem5 programs the
POSIX timer to expire according to the remaining ticks before the next
event in the event queue. Based on this, we can come up with the
following solution: If KVM returns to user-space due to POSIX timer
expiration, it must be time to process the next gem5 event, so we just
fast-forward the tick (by scheduling the next CPU tick event) to that
event directly without calculating from the instruction count.

There is one more related issue needed to be solved. The KVM exit
reason, KVM_EXIT_INTR, was treated as the case where the KVM execution
was disturbed by POSIX timer expiration. However, there exists a case
where the exit reason is KVM_EXIT_INTR but the POSIX timer has not
expired. Its cause is still unknown, but it can be observed via the
"old_value" argument returned by timer_settime() when disarming the
POSIX timer. In addition, it seems to happen often when a guest CPU is
not in idle state. When this happens, the above tick event scheduling
incorrectly treats KVM_EXIT_INTR as POSIX timer expiration and fast-
forwards the tick to process the next event too early. This makes the
guest feel external events come too fast, and will sometimes cause
trouble. One example is the VSYNC interrupt from HDLCD. The guest seems
to get stuck in VSYNC handling if the KVM CPU is not given enough time
between each VSYNC interrupt to complete a service. (Honestly I did not
dig in to see how the guest handled the VSYNC interrupt and how the
above situation became trouble. I just observed from the debug trace of
GIC & HDLCD & timer, and made this conclusion.) This change also uses
a workaround to detect POSIX timer expiration correctly to make the
guest work with HDLCD.

JIRA: https://gem5.atlassian.net/browse/GEM5-663

Change-Id: I6159238a36fc18c0c881d177a742d8a7745a23ca
Signed-off-by: Hsuan Hsu <hsuan.hsu@mediatek.com>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30919
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

diff --git a/src/cpu/kvm/base.cc b/src/cpu/kvm/base.cc
index 18aead84eec983221e48eae2a10c89b2c3999961..0afab1e8415f1e563abf0e7fee30ffb13ca09882 100644 (file)
--- a/src/cpu/kvm/base.cc
+++ b/src/cpu/kvm/base.cc
@@ -691,8 +691,13 @@ BaseKvmCPU::tick()
     }
 
     // Schedule a new tick if we are still running
-    if (_status != Idle && _status != RunningMMIOPending)
-        schedule(tickEvent, clockEdge(ticksToCycles(delay)));
+    if (_status != Idle && _status != RunningMMIOPending) {
+        if (_kvmRun->exit_reason == KVM_EXIT_INTR && runTimer->expired())
+            schedule(tickEvent, clockEdge(ticksToCycles(
+                     curEventQueue()->nextTick() - curTick() + 1)));
+        else
+            schedule(tickEvent, clockEdge(ticksToCycles(delay)));
+    }
 }
 
 Tick

diff --git a/src/cpu/kvm/timer.cc b/src/cpu/kvm/timer.cc
index e7b185f5dab1ac995109bd83cbf5d844852d730a..538fd569f40ee7e49a4e1824f345ab40ada1a673 100644 (file)
--- a/src/cpu/kvm/timer.cc
+++ b/src/cpu/kvm/timer.cc
@@ -122,10 +122,19 @@ PosixKvmTimer::disarm()
     struct itimerspec ts;
     memset(&ts, 0, sizeof(ts));
 
-    DPRINTF(KvmTimer, "Disarming POSIX timer\n");
-
-    if (timer_settime(timer, 0, &ts, NULL) == -1)
+    if (timer_settime(timer, 0, &ts, &prevTimerSpec) == -1)
         panic("PosixKvmTimer: Failed to disarm timer\n");
+
+    DPRINTF(KvmTimer, "Disarmed POSIX timer: %is%ins left\n",
+            prevTimerSpec.it_value.tv_sec,
+            prevTimerSpec.it_value.tv_nsec);
+}
+
+bool
+PosixKvmTimer::expired()
+{
+    return (prevTimerSpec.it_value.tv_nsec == 0 &&
+            prevTimerSpec.it_value.tv_sec == 0);
 }
 
 Tick

diff --git a/src/cpu/kvm/timer.hh b/src/cpu/kvm/timer.hh
index 32222e53b8327aa0358e9e9f313783321a34cd12..376ba7c1adeecde2870b22cfdecbc1e6bec31ddb 100644 (file)
--- a/src/cpu/kvm/timer.hh
+++ b/src/cpu/kvm/timer.hh
@@ -93,6 +93,9 @@ class BaseKvmTimer
      * signals upon timeout.
      */
     virtual void disarm() = 0;
+    virtual bool expired() {
+        return true;
+    }
 
     /**
      * Determine the resolution of the timer in ticks. This method is
@@ -193,6 +196,7 @@ class PosixKvmTimer : public BaseKvmTimer
 
     void arm(Tick ticks);
     void disarm();
+    bool expired() override;
 
   protected:
     Tick calcResolution();
@@ -200,6 +204,7 @@ class PosixKvmTimer : public BaseKvmTimer
   private:
     clockid_t clockID;
     timer_t timer;
+    struct itimerspec prevTimerSpec;
 };
 
 /**