heechul / palloc Goto Github PK

Hi,

thank you so much for making great research software publicly available. I really appreciate it.

mc-mapping.c uses a memory region starting from a physical address of 0x1000000080000000 for mmap'ing, but my machine somewhat uses this range thus the contents of list are changed immediately after they are written.

Is there any special reason to specifically use 0x1000000080000000? Or can I just use some different address in order to avoid memory regions used by the system? The machine has a single-socket Core i9-7900X (Skylake-X) and 4 DIMMs.

Thanks for your time and effort,

Soramichi

Update: the same happens even if I remove 3 of the DIMMs so that the machine has only 1 DIMM.

Update 2: I noticed that 0x1000000080000000 is approximately 10^18, which must not be used by the system because no machine has that large memory. I guess I'm still missing something.

1. option b is uesd for ? When the value range of b is 6~23, the following code results are the same, it seems to have no effect.

g_mem_size += (1 << page_shift);
g_mem_size = CEIL(g_mem_size, ENTRY_DIST);

2. What does the 4 in the code below mean?

/* initialize data */
int off_idx = (1<<page_shift) / 4;

3. What does the 4 in the code below mean?

for (i = 0; i < NUM_ENTRIES; i++) {
	int idx = i * ENTRY_DIST / 4;
	if (i == (NUM_ENTRIES - 1))
		list[idx] = 0;
	else
		list[idx] = (i+1) * ENTRY_DIST/4;
}

4. Finally, there are many problems while running mc-mapping on my machine, is there any good suggestion?

- ARM64 architecture
- 2cpus, 2 dies per CPU, 16 cores per die, so totally 16 * 2 * 2 = 64 cores
- 4 channels of DDR4， total 512GB of memory

I'm using nxp T1040D4RDB board with Linux. How do I use palloc on this board?

Conference on Real-Time and Embedded Technology and Applications Symposium (RTAS), 2014 test code could open ?

Maybe related to #9 but a different issue.

Since linux 4.15, mmap checks if the addresses given as parameters are not harmful in the sense that they do not intentionally set flag bits embedded in the addresses (such as NX bit-related ones).
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/arch/x86/mm/mmap.c?h=v4.15.8&id=be62a32044061cb4a3b70a10598e093f1319102e

Due to this change, mc-mapping with the -x option does not work for linux 4.15> because 0x1000000080000000 used as the offset parameter of mmap is condemned by this check and the mmap returns EINVAL.

I am trying to run detect-mc-mapping.sh as described in the readme to detect the bank bits. I first tried on kernel 4.15 (the default for Ubuntu 16.04 with HWE), but it fails as described in issue #10.

So I booted into 4.10:

Linux 9550-linux 4.10.0-42-generic #46~16.04.1-Ubuntu SMP Mon Dec 4 15:57:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

but now I get:

$ sudo ./detect-mc-mapping.sh 
mc-mapping: no process found
Run a background task on core1-3
./detect-mc-mapping.sh: line 7:  3982 Segmentation fault      ./mc-mapping -c 1 -i 100000000000 -b 0 -x &> /dev/null
./detect-mc-mapping.sh: line 7:  3983 Segmentation fault      ./mc-mapping -c 2 -i 100000000000 -b 0 -x &> /dev/null
./detect-mc-mapping.sh: line 7:  3984 Segmentation fault      ./mc-mapping -c 3 -i 100000000000 -b 0 -x &> /dev/null
Now run the test
Bit6: 141.80
Bit7: 140.23
Bit8: 143.74
Bit9: 144.22
Bit10: 141.69
Bit11: 144.26
Bit12: 141.70
Bit13: 140.32
Bit14: 139.21
Bit15: 145.27
Bit16: 141.87
Bit17: 140.40
Bit18: 145.22
Bit19: 143.68
Bit20: 141.45
Bit21: 145.47
Bit22: 142.03
Bit23: 139.57
mc-mapping: no process found

It seems the background processes fail to start due to segfault, which in turns makes the detection fail.

dmesg shows the following errors, corresponding to the segfaults:

[  176.603088] show_signal_msg: 15 callbacks suppressed
[  176.603090] mc-mapping[3984]: segfault at 7fdf07962ffc ip 0000000000400ca3 sp 00007ffc31b7acf0 error 4 in mc-mapping[400000+2000]
[  176.603107] x86/PAT: mc-mapping:3984 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  176.603189] mc-mapping[3983]: segfault at 7f8b2274affc ip 0000000000400ca3 sp 00007ffdcbaf1840 error 4 in mc-mapping[400000+2000]
[  176.603201] x86/PAT: mc-mapping:3983 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  176.603331] mc-mapping[3982]: segfault at 7f8e89419ffc ip 0000000000400ca3 sp 00007ffc97f08c70 error 4 in mc-mapping[400000+2000]
[  176.603341] x86/PAT: mc-mapping:3982 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  181.661912] x86/PAT: mc-mapping:3987 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  185.825739] x86/PAT: mc-mapping:3990 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  189.889824] x86/PAT: mc-mapping:3993 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  193.940241] x86/PAT: mc-mapping:3996 freeing invalid memtype [mem 0x80000000-0xa0ffffff]
[  198.062300] x86/PAT: mc-mapping:4001 freeing invalid memtype [mem 0x80000000-0xa0ffffff]

addr2line indicate the failure is at line 85 of mc-mapping:85:

addr2line -a 400ca3 -e mc-mapping
0x0000000000400ca3
.../mc-mapping.c:85

Hi,

we're having issues with CGROUP when trying use palloc. The commands specified in the documentation cannot be used to create a new partition. We've opened a thread on StackOverflow containing more information about the problem. We would appreciate any help that can be provided. To give more context, we want to use it to apply cache partitioning.

SO link: https://stackoverflow.com/questions/49668844/palloc-linux-kernel-4-4

Thanks in advance!

Hi,

In the PALLOC paper I understand the cache partitioning to be done as an additional partition after DRAM bank partitioning. That is, cache partitioning is considered to be those bits that overlap between DRAM bank bits and cache set bits.

Have you ever tested PALLOC to be used only for cache partitioning where the palloc_mask contains all the bits used for cache set selection? I am trying out the v4.4 patch on the v4.4.113 kernel on a system with a 1MB L2 cache with 1024 sets. I provide the ten bit set selection bitmask to palloc_mask (after increasing MAX_PALLOC_BITS to 10) but the system crashes in __rmqueue_smallest() soon after I enable PALLOC ("echo 1 > /sys/kernel/debug/palloc/use_palloc"). Is PALLOC intended to be used for such a case?

More details about the error:
[ 2418.583136] page:ffffea0001cc4540 count:0 mapcount:0 mapping: (null) index:0x2
[ 2418.583228] flags: 0x4000000000000000()
[ 2418.583261] page dumped because: VM_BUG_ON_PAGE(!PageBuddy(page))
[ 2418.583319] ------------[ cut here ]------------
[ 2418.583356] kernel BUG at SNIP/include/linux/page-flags.h:529!
[ 2418.583481] invalid opcode: 0000 [#1] PREEMPT SMP
[ 2418.583525] Dumping ftrace buffer:
[ 2418.583552] (ftrace buffer empty)
[ 2418.583577] Modules linked in: 8021q iptable_mangle ip6table_filter ip6_tables xt_tcpudp iptable_filter ip_tables x_tables x86_pkg_temp_thermal e
[ 2418.583907] CPU: 2 PID: 179 Comm: systemd-journal Not tainted 4.4.113-test-test+ #9
[ 2418.583957] Hardware name: Intel corporation NUC6CAYS/NUC6CAYB, BIOS AYAPLCEL.86A.0027.2016.1108.1529 11/08/2016
[ 2418.584028] task: ffff880075400000 ti: ffff880074818000 task.ti: ffff880074818000
[ 2418.584077] RIP: 0010:[] [] __rmqueue_smallest+0x406/0x8a0
[ 2418.584142] RSP: 0018:ffff88007481ba40 EFLAGS: 00010046
[ 2418.584178] RAX: 0000000000000035 RBX: ffffea0001cc4560 RCX: 0000000000000002
[ 2418.584224] RDX: 0000000080000002 RSI: 0000000000000000 RDI: 00000000ffffffff
[ 2418.584276] RBP: ffff88007481bb78 R08: 0000000000000001 R09: 0000000000000001
[ 2418.584322] R10: 0000000000000001 R11: 0000000000000001 R12: ffffffff8212cd38
[ 2418.584369] R13: 0000000073115000 R14: 0000000000000140 R15: ffffffff8212b380
[ 2418.584415] FS: 00007fec798e07c0(0000) GS:ffff880072d00000(0000) knlGS:0000000000000000
[ 2418.584468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2418.584506] CR2: 00007f4730c6a2e0 CR3: 0000000074e4c000 CR4: 0000000000340670
[ 2418.584555] Stack:
[ 2418.584570] 0000000100000000 ffffffffffffffc0 0000000000000020 ffffffff8212b548
[ 2418.584628] 0000000000000080 ffffea0001cc6920 ffffffff8212b588 ffffffff8212b380
[ 2418.584685] ffff88007481bad0 ffffea0001cc6920 0000000000000001 0000000000000001
[ 2418.584742] Call Trace:
[ 2418.584766] [] __rmqueue.isra.97+0x24/0x620
[ 2418.584811] [] get_page_from_freelist+0x274/0xae0
[ 2418.584855] [] __alloc_pages_nodemask+0x160/0xd40
[ 2418.584900] [] ? get_task_mm+0x41/0x50
[ 2418.584939] [] __get_free_pages+0x1d/0x50
[ 2418.584978] [] proc_pid_cmdline_read+0x7d/0x510
[ 2418.585022] [] ? __might_fault+0x95/0xa0
[ 2418.585066] [] __vfs_read+0x28/0xe0
[ 2418.585104] [] ? security_file_permission+0xa0/0xc0
[ 2418.585148] [] ? rw_verify_area+0x4f/0xf0
[ 2418.585188] [] vfs_read+0x94/0x150
[ 2418.585223] [] SyS_read+0x49/0xa0
[ 2418.585259] [] entry_SYSCALL_64_fastpath+0x18/0x8d
[ 2418.585308] Code: 89 85 08 ff ff ff 0f 85 5c fc ff ff e9 83 fe ff ff 89 ca e9 2f ff ff ff 48 8b bd 48 ff ff ff 48 c7 c6 40 03 d0 81 e8 1a a5 02
[ 2418.585592] RIP [] __rmqueue_smallest+0x406/0x8a0
[ 2418.585637] RSP
[ 2418.593127] ---[ end trace 5f8fc54dac6446dc ]---

Thank you

Reinette

Hi,

I'm trying to apply palloc version 4.14 and while building the kernel I got an error
which says CGROUP_PALLOC is new and the build is aborted.

In the init/Kconfig file, I added CGROUP_PALLOC right before the endif # CGROUPS

... 
config SOCK_CGROUP_DATA
             bool
             default n

config CGROUP_PALLOC
             bool "Enable PALLOC"
             default y
             help
                        Enable PALLOC. ...
endif # CGROUOPS
...

The error message is:

 -f /home/jchoi/shin/linux_android/sources/kernel/Makefile Image
| make -f /home/jchoi/shin/linux_android/sources/kernel/Makefile silentoldconfig
| make -f /home/jchoi/shin/linux_android/sources/kernel/scripts/Makefile.build obj=scripts/basic
| ln -fsn /home/jchoi/shin/linux_android/sources/kernel source
| /bin/bash /home/jchoi/shin/linux_android/sources/kernel/scripts/mkmakefile \
|     /home/jchoi/shin/linux_android/sources/kernel . 4 14
|   GEN     ./Makefile
| rm -f .tmp_quiet_recordmcount
| make -f /home/jchoi/shin/linux_android/sources/kernel/scripts/Makefile.build obj=scripts/kconfig silentoldconfig
| mkdir -p include/config include/generated
| test -e include/generated/autoksyms.h || \
|     touch   include/generated/autoksyms.h
| scripts/kconfig/conf  --silentoldconfig Kconfig
| *
| * Restart config...
| *
| *
| * Control Group support
| *
| Control Group support (CGROUPS) [Y/?] y
|   Memory controller (MEMCG) [Y/n/?] y
|     Swap controller (MEMCG_SWAP) [Y/n/?] y
|       Swap controller enabled by default (MEMCG_SWAP_ENABLED) [Y/n/?] y
|   IO controller (BLK_CGROUP) [N/y/?] n
|   PIDs controller (CGROUP_PIDS) [N/y/?] n
|   RDMA controller (CGROUP_RDMA) [N/y/?] n
|   Freezer controller (CGROUP_FREEZER) [N/y/?] n
|   Cpuset controller (CPUSETS) [Y/n/?] y
|     Include legacy /proc/<pid>/cpuset file (PROC_PID_CPUSET) [Y/n] y
|   Device controller (CGROUP_DEVICE) [N/y/?] n
|   Simple CPU accounting controller (CGROUP_CPUACCT) [Y/n/?] y
|   Perf controller (CGROUP_PERF) [Y/n/?] y
|   Debug controller (CGROUP_DEBUG) [N/y/?] n
|   Enable PALLOC (CGROUP_PALLOC) [Y/n/?] (NEW) aborted!
|
| Console input/output is redirected. Run 'make oldconfig' to update configuration.
|
| /home/jchoi/shin/linux_android/sources/kernel/scripts/kconfig/Makefile:38: recipe for target 'silentoldconfig' failed
| make[4]: *** [silentoldconfig] Error 1
| /home/jchoi/shin/linux_android/sources/kernel/Makefile:523: recipe for target 'silentoldconfig' failed
| make[3]: *** [silentoldconfig] Error 2
| make[2]: *** No rule to make target 'include/config/auto.conf', needed by 'include/config/kernel.release'.  Stop.
| Makefile:146: recipe for target 'sub-make' failed
| make[1]: *** [sub-make] Error 2
| Makefile:24: recipe for target '__sub-make' failed
| make: *** [__sub-make] Error 2
| ERROR: oe_runmake failed
| WARNING: exit code 1 from a shell command.

I'm using Yocto project to build the Linux kernel (if it is helpful to explain the reason..)
Do you know if there is any way to solve this problem?

Because detect-mc-mapping.sh doesn't work for me as described in #15, I tried ./detect-mc-mapping-pagemap.sh which I understand is the new "safe" method of detecting the bank bits.

However, this script fails since the mc-mapping-pagemap binary doesn't support the -b option that the script uses:

$ ./detect-mc-mapping-pagemap.sh 
mc-mapping-pagemap: no process found
Run a background task on core1-3
Now run the test
Bit12: ./mc-mapping-pagemap: invalid option -- 'b'
Bit13: ./mc-mapping-pagemap: invalid option -- 'b'
Bit14: ./mc-mapping-pagemap: invalid option -- 'b'
Bit15: ./mc-mapping-pagemap: invalid option -- 'b'
Bit16: ./mc-mapping-pagemap: invalid option -- 'b'
Bit17: ./mc-mapping-pagemap: invalid option -- 'b'

Hello, I was curious as to whether there had been plans to submit this upstream? I've not been able to track down any patch submissions for this project.

Thanks,

Sean

Hi!

Is it possible for you to provide a patch for the 3.12 kernel?

Thanks in advance,
Bogdan.