nvsl / optanestudy Goto Github PK

Dear code author:
What is the meaning of stripe size in compilation options？I do not understand it.
Hope to get your answer.
Ziyue, Xu
Nankai university

I have got the experiment running，and I want to reproduct the experiment result.
Run a test like this:
echo "task=1" > /proc/lattester
../kernel/gen_tasks /dev/pmem1 1
./parsing/10_parse_basic.py
and I want to know how to get the "Best-case latency" result in Figure 2 in your Paper?
Read Sequential -----> 'load-fence-64-seq'
Read Random -----> 'load-fence-64-rand'
Write(ntstore) -----> 'nstore-fence-64-seq'
Write(clwb) -----> 'store-clwb-64-seq'
Is this the correspondence?

Dear authors:
After I tried to run the test (using command "echo task=1,op=0 > /proc/lattester"), I encountered a problem when I wanted to remove the two modules.
sudo ./umount.sh
output:
umount: /mnt/latency: not mounted.
umount: /mnt/report: not mounted.
rmmod: ERROR: Module latfs is in use
rmmod: ERROR: Module repfs is in use by: latfs

lsmod | grep latfs
output:
latfs 61440 3
repfs 16384 1 latfs

If I tried to load modules again,
sudo ./mount.sh /dev/pmem0 /dev/pmem1
output:
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-62-generic'
gcc -O2 -o gen_tasks gen_tasks.c -mmmx -msse2 -mavx -mpreferred-stack-boundary=4 -DAEP_SUPPORTED -DFENCE
Unmounting existing ReportFS
umount: /mnt/latency: not mounted.
umount: /mnt/report: not mounted.
rmmod: ERROR: ../libkmod/libkmod-module.c:793 kmod_module_remove_module() could not remove 'latfs': Resource temporarily unavailable
rmmod: ERROR: could not remove module latfs: Resource temporarily unavailable
rmmod: ERROR: Module repfs is in use by: latfs
insmod: ERROR: could not insert module repfs.ko: File exists
insmod: ERROR: could not insert module latfs.ko: File exists
mount: /mnt/report: /dev/pmem0 already mounted on /mnt/pmem0.
mount: /mnt/latency: /dev/pmem1 already mounted on /mnt/pmem1.

I tried to find a solution online, but I failed. Can you tell me how to solve the problem.

When I use “dmesg|tail” to get the result of bandwidth, the result includes 10 numbers.
such as:
1 640(36)
2 640(36)
3 640(36)
4 640(36)
5 640(36)
6 640(36)
7 640(36)
8 640(36)
9 640(36)
10 640(36)
Should I Accumulate these data to get the final answer(6400=640*10)?
Hope to get your answer.
Ziyue, Xu
Nankai university

Hi authors,
I tried to find this in the repo, but I haven't found it, do you mind telling me how is effective write ratio measured? Thank you!

First, thank you for this work, the tasks are really useful.
I saw one recent commit is "Update lat.c and rep.c for linux kernel version 5.1", and I wonder if the current version supports kernel version 5.4.1 (https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.4.y&id=79438f37a69a80822333c86acb06a71abce1bce3)?
Also, I have tried the code on a slightly modified distribution (sorry I cannot find where I got this):

dpkg --list | grep linux-image
linux-image-unsigned-5.4.1-050401-lowlatency 5.4.1-050401.201911290555 amd64 Linux kernel image for version 5.4.1 on 64 bit x86 SMP

And whenever I try to run the "Strided access bandwidth" tests, my server would go down. Here are the cpu info

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Stepping: 7
CPU MHz: 1000.631
CPU max MHz: 3900.0000
CPU min MHz: 1000.0000
BogoMIPS: 4600.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 22528K
NUMA node0 CPU(s): 0-31
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities

Dear code author:
When I compile the benchmark code, there is a compile error in src/common.h. The error is
src/kernel/common.h: Unknown register name 'ymm0' appears in 'asm'. I can't how to solve this error.
Hope to get your answer.
Kewen, He
Nankai university

Dear code authors:
I’m using the OptaneStudy, but I have encountered some problems in using it.
I use it in Fedora 27 and linux kernel 4.13.16. I config the Octane use these commands:
1. Create pmem devices
ipmctl create -goal PersistentMemoryType=AppDirect
ndctl create-namespace
and get these regions:
[
{
"dev":"namespace3.0”,
"mode":"fsdax”,
"map":"dev”,
"size":133175443456,
"uuid":"8824fbe5-6b52-4463-97b3-ee1c76cdf685”,
"blockdev":"pmem3”
},
{
"dev":"namespace2.0”,
"mode":"fsdax”,
"map":"dev”,
"size":133175443456,
"uuid":"904229c9-0cc0-425e-a288-c90cfd9a1ba8”,
"blockdev":"pmem2”
}
]

2. Execute the mount.sh
sh /root/OptaneStudy/src/testscript/mount.sh /dev/pmem2 /dev/pmem3

3. Test Latency
echo task=1,op=0 > /proc/lattester

4. See the output using dmesg
dmesg | tail -n 100
and get the output like this:
[ 266.020088] {0}[0]load-fence-64 avg 2729, stddev^2 21261321, max 58924, min 864 
[ 266.064615] {0}Running ntload-fence-64 
[ 268.347241] {0}[1]ntload-fence-64 avg 2728, stddev^2 2328676, max 22524, min 866 
[ 268.391851] {0}Running store-fence-64 
[ 269.926788] {0}[2]store-fence-64 avg 18446744073709550438, stddev^2 5080516, max 58952, min 18446744069414592274 
[ 269.994989] {0}Running store-clflush-64 
[ 270.868930] {0}[3]store-clflush-64 avg 18446744073709548557, stddev^2 19686969, max 53560, min 18446744069414585292

I'm afraid this result is incorrect. May I ask what is wrong with my testing process.

Ubuntu 18.04, Linux 4.13.0, 18 cores per socket, 6 interleaved Optane DIMMs.

When I ran subtests/40_sizebw.sh with op=0 for random read bandwidth, I got the follwing result (MB/s):

64B: 9750
128B: 18950
256B: 34530
512B: 35000
1KB: 17000
2KB: 22500
4KB: 28500
8KB: 31200
16KB: 32500

According to the paper, the lowest performance should be at 4KB.
However, I got weird result at 1KB and 2KB.
(The random write bandwidth result is the same with the paper)

Dear Authors, I need a little support to run this code. I am interested to reproduce the work but I am struggling to run the code. Would you like to provide some details instructions that how can I reproduce the work? Thanks. I appreciate your help in advance/

Where do I get simd.h?

[root@localhost kernel]# make all
make -C /usr/lib/modules/3.10.0-1049.el7.x86_64/build M=`pwd`
make[1]: Entering directory `/usr/src/kernels/3.10.0-1062.el7.x86_64'
  CC [M]  /root/OptaneStudy/src/kernel/tasks.o
/root/OptaneStudy/src/kernel/tasks.c:19:22: fatal error: asm/simd.h: No such file or directory
 #include <asm/simd.h>
                      ^
compilation terminated.
make[2]: *** [/root/OptaneStudy/src/kernel/tasks.o] Error 1
make[1]: *** [_module_/root/OptaneStudy/src/kernel] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.10.0-1062.el7.x86_64'
make: *** [all] Error 2