Measuring CPU clock speed with NOPs and time(1)
Table of Contents
1. Introduction
Brendan Gregg published a simple CPU benchmark for checking your cpu speed. I decided to give it a go and see for myself. Even for a simple thing like that it turned out a little different than I expected.
2. Benchmark
2.1. Code
Instead of writing C-code and then editing asm/binary I opted for writing my own assembly. I don't need to lookup gcc flags or read through the boilerplate generated by GCC.
I prefer Intel asm syntax over AT&T, I think Intel is more readable for humans.
section .text global _start SYS_EXIT equ 1 ARG equ 1048576000 _start: mov eax, ARG ; store the number of loops in the register loop: nop ; do nothing, burn cycles nop nop ; <snip> ;total of 2048 NOPs nop add eax, -1 ; subtract 1 from loop counter cmp eax, 0 ; test if loop counter is 0 jne loop ; go back to the nops if not finished end: mov eax, SYS_EXIT ; we want exit(2) int 0x80 ; we want it now
Compiling and linking it:
nasm -f elf64 main.asm ld -o main main.o
2.2. Running it freely
I used taskset to ensure my workload won't get moved to a different CPU. I want to benchmark one core, have hot caches. Moving the task across CPUs could introduce cache misses or reduce core temperature. I imagine running on core #1 for a while could heat it up, which would result in core frequency reduction. This may cause the kernel to move move the workload to faster core #2, wait a bit, move to core #3 etc…
taskset 1 time ./main 2>&1
115.34user 0.00system 1:55.36elapsed 99%CPU (0avgtext+0avgdata 300maxresident)k 0inputs+0outputs (0major+14minor)pagefaults 0swaps
2.2.1. Results analysis:
| What | How to calculate | Result |
|---|---|---|
| Instructions | 2048 * 1048576000 | 2147483648000 |
| Time[s] | time(1) | 115.34 |
| NOPs/s/4 | 4654681000 |
How does that compare to my CPU?
head /proc/cpuinfo cpupower -c 0 frequency-info
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 140
model name : 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
stepping : 1
microcode : 0x9a
cpu MHz : 2800.000
cache size : 12288 KB
physical id : 0
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 400 MHz - 4.70 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 400 MHz and 4.70 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: Unable to call hardware
current CPU frequency: 1.25 GHz (asserted by call to kernel)
boost state support:
Supported: yes
Active: yes
Here's my CPU's specification on Intel page: https://ark.intel.com/content/www/us/en/ark/products/208921/intel-core-i71165g7-processor-12m-cache-up-to-4-70-ghz-with-ipu.html
2.2.2. Conclusions:
So a 4.65GHz clock speed is within plausible results but that would mean that:
- Top CPU speed range advertised by Intel is actually reachable
- A compact laptop with questionable heat tolerance can sustain top CPU speed for 2 minutes
This might be true, especially because NOPs don't need a lot of power power to run but I tend to not believe vendor-advertised top specs.
Maybe my CPU can execute 5 or 6 NOPs in parallel?
Let's fix the frequency at some low value and have a look at results.
2.3. Running it at fixed 1GHz speed
sudo cpupower -c 0 frequency-set --min 1000MHz --max 1000MHz cpupower -c 0 frequency-info taskset 1 time ./main 2>&1
Setting cpu: 0
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 400 MHz - 4.70 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 1000 MHz and 1000 MHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: Unable to call hardware
current CPU frequency: 2.35 GHz (asserted by call to kernel)
boost state support:
Supported: yes
Active: yes
511.81user 0.03system 8:32.55elapsed 99%CPU (0avgtext+0avgdata 244maxresident)k
0inputs+0outputs (0major+13minor)pagefaults 0swaps
2.3.1. Result analysis
Frequency artifacts
The same command says that
frequency should be within 1000 MHz and 1000 MHz.andcurrent CPU frequency: 2.35 GHz (asserted by call to kernel). What do I make out of this?I ran the command a couple of more times and got various results while the core was idle: 1.65GHz, 1.56GHz, 1.31GHz, 1.12GHz. It looks like the CPU does what it wants.
Sampling frequency once will not give precise results. You need to sample it a couple of times and run some statistics to get meaningful results. Let's see what happens under load.
Frequency is actually pretty stable
During the benchmark I ran
while sleep 2; do cpupower monitor; doneto monitor whether core 0 frequency is actually around 1GHz. I got something like that:| Nehalem || Mperf || Idle_Stats CPU| C3 | C6 | PC3 | PC6 || C0 | Cx | Freq || POLL | C1_A | C2_A | C3_A 0| 0.00| 0.00| 0.00| 0.00|| 99.60| 0.40| 1014|| 0.00| 0.00| 0.00| 0.00 1| 0.00| 0.00| 0.00| 0.00|| 0.60| 99.40| 2331|| 0.00| 0.04| 7.08| 92.26 2| 0.00| 0.00| 0.00| 0.00|| 0.30| 99.70| 855|| 0.00| 0.04| 0.21| 99.46 3| 0.00| 0.00| 0.00| 0.00|| 0.53| 99.47| 859|| 0.00| 0.06| 3.00| 96.43 4| 0.00| 0.00| 0.00| 0.00|| 0.75| 99.25| 909|| 0.00| 0.13| 3.64| 95.50 5| 0.00| 0.00| 0.00| 0.00|| 0.74| 99.26| 858|| 0.00| 0.44| 2.93| 95.93 6| 0.00| 0.00| 0.00| 0.00|| 0.26| 99.74| 882|| 0.00| 0.10| 0.00| 99.65 7| 0.00| 0.00| 0.00| 0.00|| 0.43| 99.57| 910|| 0.00| 0.78| 0.70| 98.12
It looked like core #0 was consistently clocking around 1GHz while under sustained load.
An interesting detail is that core #1 is clocking at over 2GHz. We've got hyperthreading so core #0 is tied to core #1. It might be that frequency scaler sees that there's a high demand on CPU power and ramps up the CPU speed, but core #0 has a limited speed. Note that thread #0 is busy and thread #1 is idle (C0 means "busy", Cx means "idle", as per linux source code).
- # of NOPs / time
| What | How to calculate | Result |
|---|---|---|
| Instructions | 2048 * 1048576000 | 2147483648000 |
| Time[s] | time(1) | 511.81 |
| NOPs/s/4 | 1048965300 |
2.3.2. Conclusions
it looks like we clocked on average a little over 1GHz and indeed we can still run 4 NOPs per second.
So indeed, two conclusions in the previous benchmaks proved to be true!
3. Bonus: bogomips
The Linux Kernel exports 'bogomips' through /proc/cpuinfo
head -n28 /proc/cpuinfo grep bogomips /proc/cpuinfo
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 140 model name : 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz stepping : 1 microcode : 0x9a cpu MHz : 2800.000 cache size : 12288 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 27 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l2 invpcid_single cdp_l2 ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear flush_l1d arch_capabilities vmx flags : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple pml ept_mode_based_exec tsc_scaling bugs : spectre_v1 spectre_v2 spec_store_bypass swapgs bogomips : 5608.00 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00 bogomips : 5608.00
It is closely related to the above benchmark and has a wikipedia article https://en.wikipedia.org/wiki/BogoMips
It was introduced in Linux-0.99.11 (July 17, 1993) in commit 5e736a6b1aac3994a22b08b6d7611ee2a39339ec