Can't reproduce cpu cache-miss - cpu-cache

Question

Good day!
I'm reading this wonderful article: What every programmer should know about memory. Right now I'm trying to figure out how CPU caches work and reproduce the experiment with cache misses. The aim is reproduce performance degradation when amount of accessed data rises (figure 3.4). I wrote a little program, that should reproduce degradation, but it doesn't. The performance degradation appears after I allocate more than 4Gb of memory, and I don't understand why. I think it should appear when 12 or maybe 100 of MB allocated. Maybe the program is wrong and I miss something? I use
Intel Core i7-2630QM
L1: 256Kb
L2: 1Mb
L3: 6Mb
Here is the GO listing.
main.go
package main
import (
"fmt"
"math/rand"
)
const (
n0 = 1000
n1 = 100000
)
func readInt64Time(slice []int64, idx int) int64
func main() {
ss := make([][]int64, n0)
for i := range ss {
ss[i] = make([]int64, n1)
for j := range ss[i] {
ss[i][j] = int64(i + j)
}
}
var t int64
for i := 0; i < n0; i++ {
for j := 0; j < n1; j++ {
t0 := readInt64Time(ss[i], rand.Intn(n1))
if t0 <= 0 {
panic(t0)
}
t += t0
}
}
fmt.Println("Avg time:", t/int64(n0*n1))
}
main.s
// func readInt64Time(slice []int64, idx int) int64
TEXT ·readInt64Time(SB),$0-40
MOVQ slice+0(FP), R8
MOVQ idx+24(FP), R9
RDTSC
SHLQ $32, DX
ORQ DX, AX
MOVQ AX, R10
MOVQ (R8)(R9*8), R8 // Here I'm reading the memory
RDTSC
SHLQ $32, DX
ORQ DX, AX
SUBQ R10, AX
MOVQ AX, ret+32(FP)
RET

Answer 1

For those who interested. I reproduced the 'cache-miss' behavior. But the performance degradation was not so dramatic as the article describes. Here is the final benchmark listing:
main.go
package main
import (
"fmt"
"math/rand"
"runtime"
"runtime/debug"
)
func readInt64Time(slice []int64, idx int) int64
const count = 2 << 25
func measure(np uint) {
n := 2 << np
s := make([]int64, n)
for i := range s {
s[i] = int64(i)
}
t := int64(0)
n8 := n >> 3
for i := 0; i < count; i++ {
// Intex is 64 byte aligned, since cache line is 64 byte
t0 := readInt64Time(s, rand.Intn(n8)<<3)
t += t0
}
fmt.Printf("Allocated %d Kb. Avg time: %v\n",
n/128, t/count)
}
func main() {
debug.SetGCPercent(-1) // To eliminate GC influence
for i := uint(10); i < 27; i++ {
measure(i)
runtime.GC()
}
}
main_amd64.s
// func readInt64Time(slice []int64, idx int) int64
TEXT ·readInt64Time(SB),$0-40
MOVQ slice+0(FP), R8
MOVQ idx+24(FP), R9
RDTSC
SHLQ $32, DX
ORQ DX, AX
MOVQ AX, R10
MOVQ (R8)(R9*8), R11 // Read memory
MOVQ $0, (R8)(R9*8) // Write memory
RDTSC
SHLQ $32, DX
ORQ DX, AX
SUBQ R10, AX
MOVQ AX, ret+32(FP)
RET
I disabled garbadge collector to illiminate its infuence and made 64B index alignment, since my processor has 64B cache line.
The benchmark result is:
Allocated 16 Kb. Avg time: 22
Allocated 32 Kb. Avg time: 22
Allocated 64 Kb. Avg time: 22
Allocated 128 Kb. Avg time: 22
Allocated 256 Kb. Avg time: 22
Allocated 512 Kb. Avg time: 23
Allocated 1024 Kb. Avg time: 23
Allocated 2048 Kb. Avg time: 24
Allocated 4096 Kb. Avg time: 25
Allocated 8192 Kb. Avg time: 29
Allocated 16384 Kb. Avg time: 31
Allocated 32768 Kb. Avg time: 33
Allocated 65536 Kb. Avg time: 34
Allocated 131072 Kb. Avg time: 34
Allocated 262144 Kb. Avg time: 35
Allocated 524288 Kb. Avg time: 35
Allocated 1048576 Kb. Avg time: 39
I run this bench many times, and it gave me similar results for every run. If I removed read and write ops from asm code, then I got 22 cycles for all allocations, so this time difference is memory access time. As you can see, the first time shift is at 512 Kb allocation size. Just one cpu cycle, but it's very stable. The next time change at 2 Mb. At 8 Mb there is the most significant time change, but it still 4 cycles, and we out of cache at all.
After all that tests I see that there is no dramatic cost for the cache-miss. It still significant, since the time difference is 10-15 times, but not 50-500 times as we saw in the article. Maybe todays memory significantly faster, than 7 years ago? Looks promising =) Maybe after the next 7 years there will be architertures with no cpu caches at all. We'll see.
EDIT:
As #Leeor mentioned, RDTSC instruction has no serializing behaviour and there can be out of order execution. I tried RDTSCP instruction instead:
main_amd64.s
// func readInt64Time(slice []int64, idx int) int64
TEXT ·readInt64Time(SB),$0-40
MOVQ slice+0(FP), R8
MOVQ idx+24(FP), R9
BYTE $0x0F; BYTE $0x01; BYTE $0xF9; // RDTSCP
SHLQ $32, DX
ORQ DX, AX
MOVQ AX, R10
MOVQ (R8)(R9*8), R11 // Read memory
MOVQ $0, (R8)(R9*8) // Write memory
BYTE $0x0F; BYTE $0x01; BYTE $0xF9; // RDTSCP
SHLQ $32, DX
ORQ DX, AX
SUBQ R10, AX
MOVQ AX, ret+32(FP)
RET
Here what I got with this changes:
Allocated 16 Kb. Avg time: 27
Allocated 32 Kb. Avg time: 27
Allocated 64 Kb. Avg time: 28
Allocated 128 Kb. Avg time: 29
Allocated 256 Kb. Avg time: 30
Allocated 512 Kb. Avg time: 34
Allocated 1024 Kb. Avg time: 42
Allocated 2048 Kb. Avg time: 55
Allocated 4096 Kb. Avg time: 120
Allocated 8192 Kb. Avg time: 167
Allocated 16384 Kb. Avg time: 173
Allocated 32768 Kb. Avg time: 189
Allocated 65536 Kb. Avg time: 201
Allocated 131072 Kb. Avg time: 215
Allocated 262144 Kb. Avg time: 224
Allocated 524288 Kb. Avg time: 242
Allocated 1048576 Kb. Avg time: 281
Now I see that big difference between cahce and RAM access. The time is actualy 2 times lower than in the article, but it's predictable, since memory frequency is as twice higher.

Answer 2

This indeed doesn't produce the attempted observation, it's not clear exactly what your benchmark does - are you accessing randomly within the range? are you measuring access latency per single access?
It seems that your benchmark will incur a constant overhead per time measurement that is not amortized, so you're basically measuring the function call time (which is constant).
Only when the memory latency become large enough to pass that overhead (when you access the DRAM at 4GB), you actually start getting meaningful measurements.
You should switch to measuring the time over the entire loop (over count iterations) and divide.