add WIP atomics spec

diff --git a/openpower/atomics.mdwn b/openpower/atomics.mdwn
new file mode 100644 (file)
index 0000000..cde1ff5
--- /dev/null
+++ b/openpower/atomics.mdwn
@@ -0,0 +1,113 @@
+# Draft proposal for improved atomic operations for the Power ISA
+
+<https://bugs.libre-soc.org/show_bug.cgi?id=236>
+
+## Motivation
+
+Power ISA currently has some issues with its atomic operations support:
+
+### Power ISA's current atomic operations are inefficient
+Implementations have a hard time recognizing existing atomic operations via macro-op fusion because they would often have to detect and fuse a large number of instructions, including branches.
+
+There is also the issue that PowerISA's memory fences are unnecessarily strong, particularly `isync` which is used for a lot of `acquire` and stronger fences. `isync` forces the cpu to do a full pipeline flush, which is unnecessary when all that is needed is a memory barrier.
+
+`atomic_fetch_add_seq_cst` is 6 instructions including a loop:
+```
+# address in r4, addend in r5
+    sync
+loop:
+    ldarx 3, 0, 4
+    add 6, 5, 3
+    stdcx. 6, 0, 4
+    bne 0, loop
+    lwsync
+# output in r3
+```
+
+`atomic_load_seq_cst` is 5 instructions, including a branch, and an unnecessarily-strong memory fence:
+```
+# address in r3
+    sync
+    ld 3, 0(3)
+    cmpw 0, 3, 3
+    bne- 0, skip
+    isync
+skip:
+# output in r3
+```
+
+`atomic_compare_exchange_strong_seq_cst` is 7 instructions, including a loop with 2 branches, and an unnecessarily-strong memory fence:
+```
+# address in r4, compared-to value in r5, replacement value in r6
+    sync
+loop:
+    ldarx 3, 0, 4
+    cmpd 0, 3, 5
+    bne 0, not_eq
+    stdcx. 6, 0, 4
+    bne 0, loop
+not_eq:
+    isync
+# output loaded value in r3, store-occurred flag in cr0.eq
+```
+
+`atomic_load_acquire` is 4 instructions, including a branch and an unnecessarily-strong memory fence:
+```
+    ld 3, 0(3)
+    cmpw 0, 3, 3
+    bne- skip
+    isync
+skip:
+```
+
+Having single atomic operations is useful for implementations that want to send atomic operations to a shared cache since they can be more efficient to execute there, rather than having to move a whole cache block. Relying exclusively on 
+
+### Power ISA doesn't align well with C++11 atomics
+
+[P0668R5: Revising the C++ memory model](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0668r5.html):
+
+> Existing implementation schemes on Power and ARM are not correct with
+> respect to the current memory model definition. These implementation
+> schemes can lead to results that are disallowed by the current memory
+> model when the user combines acquire/release ordering with seq_cst
+> ordering. On some architectures, especially Power and Nvidia GPUs, it
+> is expensive to repair the implementations to satisfy the existing
+> memory model. Details are discussed in (Lahav et al)
+> http://plv.mpi-sws.org/scfix/paper.pdf (which this discussion relies
+> on heavily).
+
+### Power ISA's Atomic-Memory-Operations have issues
+
+PowerISA v3.1 Book II section 4.5: Atomic Memory Operations
+
+They are still missing better fences, combined operation/fence instructions, and operations on 8/16-bit values, as well as issues with unnecessary restrictions:
+
+it has only 32-bit and 64-bit atomic operations.
+
+the operations it has that I was going to propose:
+fetch_add
+fetch_xor
+fetch_or
+fetch_and
+fetch_umax
+fetch_smax
+fetch_umin
+fetch_smin
+exchange
+
+as well as a few I wasn't going to propose (they seem less useful to me):
+compare-and-swap-not-equal
+fetch-and-increment-bounded
+fetch-and-increment-equal
+fetch-and-decrement-bounded
+store-twin
+
+The spec also basically says that the atomic memory operations are only intended for when you want to do atomic operations on memory, but don't want that memory to be loaded into your L1 cache.
+
+imho that restriction is specifically *not* wanted, because there are plenty of cases where atomic operations should happen in your L1 cache.
+
+I'd guess that part of why those atomic operations weren't included in gcc or clang as the default implementation of atomic operations (when the appropriate ISA feature is enabled) is because of that restriction.
+
+imho the cpu should be able to (but not required to) predict whether to send an atomic operation to L2-cache/L3-cache/etc./memory or to execute it directly in the L1 cache. The prediction could be based on how often that cache block was accessed from different cpus, e.g. by having a small saturating counter and a last-accessing-cpu field, where it would count how many times the same cpu accessed it in a row, sending it to the L1 cache if that's more than some limit, otherwise doing the operation in the L2/L3/etc.-cache if the limit wasn't reached or a different cpu tried to access it.
+
+# TODO: add list of proposed instructions
\ No newline at end of file