Git Product home page Git Product logo

tencentos-kernel's Introduction

tencentos-kernel's People

Contributors

alexshi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tencentos-kernel's Issues

dnf check-update

有这个日志 不知道 是不是在这里问,执行 dnf check-update时候出来的
kernel-devel.x86_64 5.4.32-19.0001.tl3 TencentOS
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.32-19.0003.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.87-19.0004.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.109-19.0005.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0006.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0007.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0008.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0009.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 Updates
kernel-devel.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.32-19.0001.tl3 TencentOS
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.32-19.0003.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.87-19.0004.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.109-19.0005.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0006.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0007.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0008.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0009.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 Updates
kernel-headers.x86_64 5.4.119-19.0009.1.tl3 @updates

Process become D state when do mv operation under overlayfs and xfs

I am using Linux 5.4 ( https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/tag/?h=Ubuntu-5.4.0-34.38 )to do the test. We have some containers using the base image and start script. Sometimes that many processes will come into D state.

Some of the process that come into D state are:


crash> foreach UN ps -m |sort|tail
[0 16:55:33.313] [UN]  PID: 148201  TASK: ffff89610e331ec0  CPU: 33  COMMAND: "checkCustom"
[0 16:55:33.885] [UN]  PID: 147963  TASK: ffff894bd4845c40  CPU: 16  COMMAND: "network_mon"
[0 16:55:33.941] [UN]  PID: 147952  TASK: ffff89667c833d80  CPU: 3   COMMAND: "disk_mon"
[0 16:55:34.831] [UN]  PID: 147702  TASK: ffff896225998000  CPU: 34  COMMAND: "mv"
[0 16:55:36.268] [UN]  PID: 146103  TASK: ffff896bc51bdc40  CPU: 7   COMMAND: "checkCPU"
[0 16:55:36.579] [UN]  PID: 2690   TASK: ffff896e75769ec0  CPU: 10  COMMAND: "containerd"
[0 16:55:37.063] [UN]  PID: 16429  TASK: ffff896e77039ec0  CPU: 39  COMMAND: "runsv"
[0 16:55:37.537] [UN]  PID: 147220  TASK: ffff8968e5305c40  CPU: 32  COMMAND: "mv"
[0 16:55:37.537] [UN]  PID: 147221  TASK: ffff895d140d0000  CPU: 10  COMMAND: "mv"
[0 16:55:37.540] [UN]  PID: 147222  TASK: ffff896278ee3d80  CPU: 26  COMMAND: "mv"

There are containers using the same image, and do mv /etc/security/access.conf /etc/security/access.conf.bak operations after start.

crash> ps -p 147222
PID: 0      TASK: ffffffff86a13780  CPU: 0   COMMAND: "swapper/0"
 PID: 1      TASK: ffff896737dedc40  CPU: 38  COMMAND: "systemd"
  PID: 25742  TASK: ffff895e24405c40  CPU: 13  COMMAND: "containerd"
   PID: 32242  TASK: ffff895337bb1ec0  CPU: 20  COMMAND: "containerd-shim"
    PID: 32317  TASK: ffff895c3b288000  CPU: 27  COMMAND: "run.sh"
     PID: 147222  TASK: ffff896278ee3d80  CPU: 26  COMMAND: "mv"

crash> ps -p 147221
PID: 0      TASK: ffffffff86a13780  CPU: 0   COMMAND: "swapper/0"
 PID: 1      TASK: ffff896737dedc40  CPU: 38  COMMAND: "systemd"
  PID: 17972  TASK: ffff8967372d0000  CPU: 14  COMMAND: "containerd"
   PID: 26289  TASK: ffff896e770d3d80  CPU: 16  COMMAND: "containerd-shim"
    PID: 26333  TASK: ffff8958a7c38000  CPU: 10  COMMAND: "run.sh"
     PID: 147221  TASK: ffff895d140d0000  CPU: 10  COMMAND: "mv"

crash> ps -p 147220
PID: 0      TASK: ffffffff86a13780  CPU: 0   COMMAND: "swapper/0"
 PID: 1      TASK: ffff896737dedc40  CPU: 38  COMMAND: "systemd"
  PID: 3315   TASK: ffff896ad717bd80  CPU: 28  COMMAND: "containerd"
   PID: 38825  TASK: ffff89654b693d80  CPU: 22  COMMAND: "containerd-shim"
    PID: 38950  TASK: ffff895c38c73d80  CPU: 32  COMMAND: "run.sh"
     PID: 147220  TASK: ffff8968e5305c40  CPU: 32  COMMAND: "mv"

crash> ps -p 16429
PID: 0      TASK: ffffffff86a13780  CPU: 0   COMMAND: "swapper/0"
 PID: 1      TASK: ffff896737dedc40  CPU: 38  COMMAND: "systemd"
  PID: 3315   TASK: ffff896ad717bd80  CPU: 28  COMMAND: "containerd"
   PID: 15723  TASK: ffff8958b5d23d80  CPU: 14  COMMAND: "containerd-shim"
    PID: 15747  TASK: ffff896e71c41ec0  CPU: 29  COMMAND: "my_init"
     PID: 16428  TASK: ffff896562fc8000  CPU: 34  COMMAND: "runsvdir"
      PID: 16429  TASK: ffff896e77039ec0  CPU: 39  COMMAND: "runsv"

The stacks:

crash> bt -t 147222
PID: 147222  TASK: ffff896278ee3d80  CPU: 26  COMMAND: "mv"
              START: __schedule at ffffffff85ea4cc6
  [ffff9e33ab0df6c0] __schedule at ffffffff85ea4cc6
  [ffff9e33ab0df700] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0df720] schedule at ffffffff85ea5103
  [ffff9e33ab0df738] schedule_timeout at ffffffff85ea8a65
  [ffff9e33ab0df798] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0df7b8] __down at ffffffff85ea7756
  [ffff9e33ab0df810] down at ffffffff85500f91
  [ffff9e33ab0df830] xfs_buf_lock at ffffffffc0456fa7 [xfs]
  [ffff9e33ab0df858] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0df8e8] xfs_buf_get_map at ffffffffc0457743 [xfs]
  [ffff9e33ab0df938] xfs_buf_read_map at ffffffffc0457fab [xfs]
  [ffff9e33ab0df980] xfs_trans_read_buf_map at ffffffffc04906e9 [xfs]
  [ffff9e33ab0df9c0] xfs_read_agi at ffffffffc0439783 [xfs]
  [ffff9e33ab0dfa20] xfs_iunlink at ffffffffc046ca7b [xfs]
  [ffff9e33ab0dfa30] current_time at ffffffff856f9921
  [ffff9e33ab0dfa80] xfs_droplink at ffffffffc046cc24 [xfs]
  [ffff9e33ab0dfaa0] xfs_rename at ffffffffc087fab2 [livepatch_93597ae8]
  [ffff9e33ab0dfb70] xfs_vn_rename at ffffffffc046a0b3 [xfs]
  [ffff9e33ab0dfbe8] vfs_rename at ffffffff856e9955
  [ffff9e33ab0dfc60] d_lookup at ffffffff856f88da
  [ffff9e33ab0dfca0] ovl_do_rename at ffffffffc0d0e2bc [overlay]
  [ffff9e33ab0dfce0] ovl_rename at ffffffffc0d0f557 [overlay]
  [ffff9e33ab0dfd98] vfs_rename at ffffffff856e9955
  [ffff9e33ab0dfe10] __lookup_hash at ffffffff856e6c94
  [ffff9e33ab0dfe50] do_renameat2 at ffffffff856ed168
  [ffff9e33ab0dff20] __x64_sys_rename at ffffffff856ed3e0
  [ffff9e33ab0dff30] do_syscall_64 at ffffffff85404447
  [ffff9e33ab0dff50] entry_SYSCALL_64_after_hwframe at ffffffff8600008c
    RIP: 00007fdb71618367  RSP: 00007ffce72ad108  RFLAGS: 00000202
    RAX: ffffffffffffffda  RBX: 0000000000008000  RCX: 00007fdb71618367
    RDX: 0000000000000000  RSI: 00007ffce72af1aa  RDI: 00007ffce72af19d
    RBP: 00007ffce72ad4d0   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000640  R11: 0000000000000202  R12: 00007ffce72ad201
    R13: 00007ffce72ad5c0  R14: 00007ffce72af19d  R15: 0000000000000000
    ORIG_RAX: 0000000000000052  CS: 0033  SS: 002b
crash> bt -t  147221
PID: 147221  TASK: ffff895d140d0000  CPU: 10  COMMAND: "mv"
              START: __schedule at ffffffff85ea4cc6
  [ffff9e33ab0af6c0] __schedule at ffffffff85ea4cc6
  [ffff9e33ab0af700] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0af720] schedule at ffffffff85ea5103
  [ffff9e33ab0af738] schedule_timeout at ffffffff85ea8a65
  [ffff9e33ab0af750] select_task_rq_fair at ffffffff854de761
  [ffff9e33ab0af798] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0af7b8] __down at ffffffff85ea7756
  [ffff9e33ab0af7d8] arch_haltpoll_enable.cold.25 at ffffffff85477200
  [ffff9e33ab0af810] down at ffffffff85500f91
  [ffff9e33ab0af830] xfs_buf_lock at ffffffffc0456fa7 [xfs]
  [ffff9e33ab0af858] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0af8e8] xfs_buf_get_map at ffffffffc0457743 [xfs]
  [ffff9e33ab0af938] xfs_buf_read_map at ffffffffc0457fab [xfs]
  [ffff9e33ab0af948] kmem_zone_alloc at ffffffffc047d34c [xfs]
  [ffff9e33ab0af980] xfs_trans_read_buf_map at ffffffffc04906e9 [xfs]
  [ffff9e33ab0af9c0] xfs_read_agi at ffffffffc0439783 [xfs]
  [ffff9e33ab0afa20] xfs_iunlink_remove at ffffffffc046d205 [xfs]
  [ffff9e33ab0afa48] _cond_resched at ffffffff85ea5399
  [ffff9e33ab0afaa0] xfs_rename at ffffffffc087f9e0 [livepatch_93597ae8]
  [ffff9e33ab0afb70] xfs_vn_rename at ffffffffc046a0b3 [xfs]
  [ffff9e33ab0afbe8] vfs_rename at ffffffff856e9955
  [ffff9e33ab0afc60] d_lookup at ffffffff856f88da
  [ffff9e33ab0afca0] ovl_do_rename at ffffffffc0d0e2bc [overlay]
  [ffff9e33ab0afce0] ovl_rename at ffffffffc0d0f557 [overlay]
  [ffff9e33ab0afd98] vfs_rename at ffffffff856e9955
  [ffff9e33ab0afe10] __lookup_hash at ffffffff856e6c94
  [ffff9e33ab0afe50] do_renameat2 at ffffffff856ed168
  [ffff9e33ab0aff20] __x64_sys_rename at ffffffff856ed3e0
  [ffff9e33ab0aff30] do_syscall_64 at ffffffff85404447
  [ffff9e33ab0aff50] entry_SYSCALL_64_after_hwframe at ffffffff8600008c
    RIP: 00007f85457cf367  RSP: 00007ffeb63a9778  RFLAGS: 00000206
    RAX: ffffffffffffffda  RBX: 0000000000008000  RCX: 00007f85457cf367
    RDX: 0000000000000000  RSI: 00007ffeb63ac1ad  RDI: 00007ffeb63ac1a0
    RBP: 00007ffeb63a9b40   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000640  R11: 0000000000000206  R12: 00007ffeb63a9901
    R13: 00007ffeb63a9c30  R14: 00007ffeb63ac1a0  R15: 0000000000000000
    ORIG_RAX: 0000000000000052  CS: 0033  SS: 002b
crash> bt -t 147220
PID: 147220  TASK: ffff8968e5305c40  CPU: 32  COMMAND: "mv"
              START: __schedule at ffffffff85ea4cc6
  [ffff9e33ab0cf6c0] __schedule at ffffffff85ea4cc6
  [ffff9e33ab0cf700] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0cf720] schedule at ffffffff85ea5103
  [ffff9e33ab0cf738] schedule_timeout at ffffffff85ea8a65
  [ffff9e33ab0cf750] select_task_rq_fair at ffffffff854de761
  [ffff9e33ab0cf798] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0cf7b8] __down at ffffffff85ea7756
  [ffff9e33ab0cf810] down at ffffffff85500f91
  [ffff9e33ab0cf830] xfs_buf_lock at ffffffffc0456fa7 [xfs]
  [ffff9e33ab0cf858] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33ab0cf8e8] xfs_buf_get_map at ffffffffc0457743 [xfs]
  [ffff9e33ab0cf938] xfs_buf_read_map at ffffffffc0457fab [xfs]
  [ffff9e33ab0cf980] xfs_trans_read_buf_map at ffffffffc04906e9 [xfs]
  [ffff9e33ab0cf9c0] xfs_read_agi at ffffffffc0439783 [xfs]
  [ffff9e33ab0cfa20] xfs_iunlink at ffffffffc046ca7b [xfs]
  [ffff9e33ab0cfa30] current_time at ffffffff856f9921
  [ffff9e33ab0cfa80] xfs_droplink at ffffffffc046cc24 [xfs]
  [ffff9e33ab0cfaa0] xfs_rename at ffffffffc087fab2 [livepatch_93597ae8]
  [ffff9e33ab0cfb70] xfs_vn_rename at ffffffffc046a0b3 [xfs]
  [ffff9e33ab0cfbe8] vfs_rename at ffffffff856e9955
  [ffff9e33ab0cfc60] d_lookup at ffffffff856f88da
  [ffff9e33ab0cfca0] ovl_do_rename at ffffffffc0d0e2bc [overlay]
  [ffff9e33ab0cfce0] ovl_rename at ffffffffc0d0f557 [overlay]
  [ffff9e33ab0cfd98] vfs_rename at ffffffff856e9955
  [ffff9e33ab0cfe10] __lookup_hash at ffffffff856e6c94
  [ffff9e33ab0cfe50] do_renameat2 at ffffffff856ed168
  [ffff9e33ab0cff20] __x64_sys_rename at ffffffff856ed3e0
  [ffff9e33ab0cff30] do_syscall_64 at ffffffff85404447
  [ffff9e33ab0cff50] entry_SYSCALL_64_after_hwframe at ffffffff8600008c
    RIP: 00007f0c25917367  RSP: 00007fff15e5e8f8  RFLAGS: 00000202
    RAX: ffffffffffffffda  RBX: 0000000000008000  RCX: 00007f0c25917367
    RDX: 0000000000000000  RSI: 00007fff15e60c91  RDI: 00007fff15e60c77
    RBP: 00007fff15e5ecc0   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000640  R11: 0000000000000202  R12: 00007fff15e5ea01
    R13: 00007fff15e5edb0  R14: 00007fff15e60c77  R15: 0000000000000000
    ORIG_RAX: 0000000000000052  CS: 0033  SS: 002b
crash> bt -t 16429
PID: 16429  TASK: ffff896e77039ec0  CPU: 39  COMMAND: "runsv"
              START: __schedule at ffffffff85ea4cc6
  [ffff9e33961d74c0] __schedule at ffffffff85ea4cc6
  [ffff9e33961d7500] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33961d7520] schedule at ffffffff85ea5103
  [ffff9e33961d7538] schedule_timeout at ffffffff85ea8a65
  [ffff9e33961d7550] _xfs_trans_bjoin at ffffffffc0490328 [xfs]
  [ffff9e33961d7578] xfs_trans_read_buf_map at ffffffffc0490729 [xfs]
  [ffff9e33961d7598] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33961d75b8] __down at ffffffff85ea7756
  [ffff9e33961d75e0] xfs_btree_lookup_get_block at ffffffffc0420932 [xfs]
  [ffff9e33961d7608] down at ffffffff85500f91
  [ffff9e33961d7628] xfs_buf_lock at ffffffffc0456fa7 [xfs]
  [ffff9e33961d7650] xfs_buf_find at ffffffffc0457435 [xfs]
  [ffff9e33961d76e0] xfs_buf_get_map at ffffffffc0457743 [xfs]
  [ffff9e33961d7730] xfs_buf_read_map at ffffffffc0457fab [xfs]
  [ffff9e33961d7778] xfs_trans_read_buf_map at ffffffffc04906e9 [xfs]
  [ffff9e33961d77b8] xfs_read_agi at ffffffffc0439783 [xfs]
  [ffff9e33961d7818] xfs_ialloc_read_agi at ffffffffc0439854 [xfs]
  [ffff9e33961d7850] xfs_dialloc at ffffffffc043a163 [xfs]
  [ffff9e33961d78c8] xfs_ialloc at ffffffffc046b8ab [xfs]
  [ffff9e33961d7948] xfs_dir_ialloc at ffffffffc046e1b5 [xfs]
  [ffff9e33961d79e0] xfs_create at ffffffffc046e556 [xfs]
  [ffff9e33961d7a78] xfs_generic_create at ffffffffc046b4b6 [xfs]
  [ffff9e33961d7af8] xfs_vn_mknod at ffffffffc046b5a4 [xfs]
  [ffff9e33961d7b08] xfs_vn_create at ffffffffc046b5e3 [xfs]
  [ffff9e33961d7b18] vfs_create at ffffffff856ea530
  [ffff9e33961d7b58] ovl_create_real at ffffffffc0d0ebab [overlay]
  [ffff9e33961d7b88] ovl_create_or_link at ffffffffc0d0fe8e [overlay]
  [ffff9e33961d7bb8] inode_init_always at ffffffff856f8e11
  [ffff9e33961d7be0] alloc_inode at ffffffff856f9226
  [ffff9e33961d7c00] ovl_fill_inode at ffffffffc0d0b99c [overlay]
  [ffff9e33961d7c68] ovl_create_object at ffffffffc0d105bb [overlay]
  [ffff9e33961d7cb8] ovl_create at ffffffffc0d10683 [overlay]
  [ffff9e33961d7cc8] path_openat at ffffffff856ec08f
  [ffff9e33961d7da8] do_filp_open at ffffffff856ed9d3
  [ffff9e33961d7e58] __alloc_fd at ffffffff856fd596
  [ffff9e33961d7eb0] do_sys_open at ffffffff856d6aa8
  [ffff9e33961d7f20] __x64_sys_openat at ffffffff856d6c40
  [ffff9e33961d7f30] do_syscall_64 at ffffffff85404447
  [ffff9e33961d7f50] entry_SYSCALL_64_after_hwframe at ffffffff8600008c
    RIP: 00007f8986361c8e  RSP: 00007ffd58c71540  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 000055a4fc1c27fe  RCX: 00007f8986361c8e
    RDX: 0000000000000a41  RSI: 000055a4fc1c27d6  RDI: 00000000ffffff9c
    RBP: 000055a4fc1c2828   R8: 00007ffd58c71678   R9: 00007ffd58c71670
    R10: 00000000000001a4  R11: 0000000000000246  R12: 000055a4fc1c27d6
    R13: 000055a4fc1c2815  R14: 000055a4fc1c27ec  R15: 000055a4fc3c5100
    ORIG_RAX: 0000000000000101  CS: 0033  SS: 002b
crash> bt -t 2690
PID: 2690   TASK: ffff896e75769ec0  CPU: 10  COMMAND: "containerd"
              START: __schedule at ffffffff85ea4cc6
  [ffff9e334e3e7ae0] __schedule at ffffffff85ea4cc6
  [ffff9e334e3e7b40] schedule at ffffffff85ea5103
  [ffff9e334e3e7b58] rwsem_down_read_slowpath at ffffffff85ea7ed3
  [ffff9e334e3e7bb0] lookup_fast at ffffffff856e6f1c
  [ffff9e334e3e7bd8] xfs_ilock_data_map_shared at ffffffffc046d5b0 [xfs]
  [ffff9e334e3e7bf0] down_read at ffffffff85ea8125
  [ffff9e334e3e7c08] xfs_ilock at ffffffffc046d4d2 [xfs]
  [ffff9e334e3e7c28] xfs_dir_open at ffffffffc045c0a0 [xfs]
  [ffff9e334e3e7c38] xfs_ilock_data_map_shared at ffffffffc046d5b0 [xfs]
  [ffff9e334e3e7c50] xfs_dir_open at ffffffffc045c0d9 [xfs]
  [ffff9e334e3e7c70] xfs_dir_open at ffffffffc045c0a0 [xfs]
  [ffff9e334e3e7c80] do_dentry_open at ffffffff856d4e43
  [ffff9e334e3e7cb8] vfs_open at ffffffff856d678d
  [ffff9e334e3e7cc8] path_openat at ffffffff856eb119
  [ffff9e334e3e7cd8] filename_lookup at ffffffff856ed500
  [ffff9e334e3e7da8] do_filp_open at ffffffff856ed9d3
  [ffff9e334e3e7e58] __alloc_fd at ffffffff856fd596
  [ffff9e334e3e7eb0] do_sys_open at ffffffff856d6aa8
  [ffff9e334e3e7f20] __x64_sys_openat at ffffffff856d6c40
  [ffff9e334e3e7f30] do_syscall_64 at ffffffff85404447
  [ffff9e334e3e7f50] entry_SYSCALL_64_after_hwframe at ffffffff8600008c
    RIP: 00005580fc05c8da  RSP: 000000c422afb540  RFLAGS: 00000202
    RAX: ffffffffffffffda  RBX: 0000000000000000  RCX: 00005580fc05c8da
    RDX: 0000000000080000  RSI: 000000c423733bc0  RDI: ffffffffffffff9c
    RBP: 000000c422afb5c0   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000202  R12: ffffffffffffffff
    R13: 000000000000004b  R14: 000000000000004a  R15: 0000000000000055
    ORIG_RAX: 0000000000000101  CS: 0033  SS: 002b

This version of kernel (( https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/tag/?h=Ubuntu-5.4.0-34.38 )) already have the patch: f8c4b7a#diff-59e0e215d80c58cdb9a2c9088d902660 but not have 44a72ef#diff-59e0e215d80c58cdb9a2c9088d902660. But I can still can see this issue after the patch applied.
From the stack, there is no AGF and AGI related lock stacks. I think this can be common issue so I open the issues here since you guys have much experience to debug this kind of issues. Appreciate for take a look, thanks. @kaixuxiakx

BT离线调度器不起作用

(使用master分支自行编译)

您好,我们在开启BT_SCHED之后,将对应的进程都设置成离线BT。
然后使用stress-ng进行压测,发现对应的p99/p95几乎和不开启离线没啥区别

是只需要设置完sched_setscheduler之后,bt的进程就几乎不会影响其他进程?

我们也通过kernel.sched_bt_runtime_us将离线服务压制到50%,但是已然干扰其他进制

kernel panic

can_migrate_bt_task里面对bt_statistics的统计如下:
p->se.bt_statistics->nr_failed_migrations_running
调用的是se.bt_statistics,这里se.bt_statistics没有赋值,是不是应该调用bt.bt_statistics?

为什么不直接往linux kernel贡献?

为什么不往linux kernel贡献?
独立开发,维护的成本非常高,且如果一直自己玩,离主干linux kernel只会差异越来越大。当差异达到百万行代码。。你如何保证能适时更新到官方最新的版本。

sysconf(_SC_LEVEL3_CACHE_SIZE) returns 0

We tested in following environment:

Cloud provider: Tencent Cloud.
Kubernetes: TKE 1.20.6-tke.27, containerd 1.4.3
Node: S3.8XLARGE64, Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz
OS/Kernel: tlinux2.6(tkernel4)x86_64 / 5.4.119-1-tlinux4-0010.1
Pod image: aspnet:7.0.4

$ cat /sys/devices/system/cpu/cpu0/cache/index*/size
32K
32K
4096K
36608K

We can't confirm that L2 is correct since we can't find any intel spec sheet of 8255C, but L3 size is coherent with many different sources.

But getconf (which is implemented using sysconf) says L2 is 2048K, and L3 is 0:

$ getconf -a | grep CACHE
LEVEL1_ICACHE_SIZE                 32768
LEVEL1_ICACHE_ASSOC                8
LEVEL1_ICACHE_LINESIZE             64
LEVEL1_DCACHE_SIZE                 32768
LEVEL1_DCACHE_ASSOC                8
LEVEL1_DCACHE_LINESIZE             64
LEVEL2_CACHE_SIZE                  2097152
LEVEL2_CACHE_ASSOC                 8
LEVEL2_CACHE_LINESIZE              64
LEVEL3_CACHE_SIZE                  0
LEVEL3_CACHE_ASSOC                 0
LEVEL3_CACHE_LINESIZE              0
LEVEL4_CACHE_SIZE                  0
LEVEL4_CACHE_ASSOC                 0
LEVEL4_CACHE_LINESIZE              0

后端real server被删除,conntrack仍存在该real server的记录并不断更新TTL

节点内核参数net.ipv4.vs.conn_reuse_mode=1 内核版本4.14.105-19-0021

流量在经过website-f10-proxy-gateway网关代理通过website-f10-jeebasic-web-finance服务的service访问该服务实例时,删除一个后端实例,导致nginx出现大量504超时且长时间无法恢复

复现现象:
网关所在节点的conntrack表中对应的记录的目标地址仍为删除的后端实例的ip,并不断被刷新老化时间。
image
nginx容器内存在大量端口不变的SYN-SENT状态顽固连接,
image

nginx容器内存在大量端口不变的SYN-SENT状态顽固连接,端口能够对应上

取其中一个端口的连接
image
image

在网关所在节点抓包发现63403该端口发出的报文被ipvs修改为了前往183上,然后从183返回syn+ack报文后,却被Reset了

image

sched_setscheduler接口调用失败

版本:4.14.105-19-0022
编译安装之后,开启offline_class和offline_group
重启后/proc/bt_stat以及cgroup下相关的bt文件存在

尝试使用sched_setscheduler()设置某个进程为离线进程,发现设置失败,求助~~~

[root@10 ~]# uname -r
4.14.105-19-0023+

./set_sched 2675
pid: 2675
result: 0
result: -1
errorno: 3

set_sched代码如下:
#include <sched.h>
#include <stdio.h>
#include <errno.h>

int main(int argc,char *argv[]) {
struct sched_param param;
param.sched_priority = 0;
if (argc < 2) {
printf("no pid specified\n");
return 0;
}
printf("pid: %s\n", argv[1]);

int n_pid = atoi(argv[1]);

int r = sched_setscheduler(n_pid, 3, &param);
printf("result: %d\n", r);

r = sched_setscheduler(n_pid, 7, &param);
printf("result: %d\n", r);
printf("errorno: %d\n", errno);
// printf("%s\n", strerror(errno));

return 0;
}

将进程设置policy=3是正常,但是设置为7的时候就不行了。

Tlinux 2.4 , mpirun --allow-run-as-root -np 8 xxx.sh , performance too low

OS : Tlinux 2.4 , 4.14.105-19-0018
docker: docker-ce 19.03.3
Case:
1,lauch a docker service:
$docker run -it --privileged=true --network host --name build-1 -v /data:/data tlinux/tlinux_2.4:latest
2,run mpirun in docker
$mpirun --allow-run-as-root -np 8 xxx.sh
3, check the fps , the fps value is only about 1/4 for normal test
4, check host top

find out kworker is always binded to cpu 0,

kworker/0:0
kworker/0:1
kworker/0:2
kworker/0:4
kworker/0:9
kworker/0:8
kworker/0:5
kworker/0:6

[Feature request] page cache limit for cgroup

针对整个系统做page cache limit是个很好的feature,但是是否有准备支持对某个memory cgroup做page cache的限制呢?

在我们的环境上发现如果memory 的limit和使用的memory rss比较接近的话,会导致大量的shrink slab发生, 不停的cache切换,disk的io utils会一直处于很高的状态。 如果该cgroup内的进程数量比较多,会产生很多的D进程,系统load就会比较高,而且会导致执行ps命令卡住很长时间。

所以在针对这种情况,是否可以不让产生cache? 所剩的memory不多,使用cache反而会导致问题。

编译出的内核ko版本与内核源码版本对应不上

·我想给内核安装dm9601的usb网卡驱动,使用的内核版本为5.4.119-19-0009.3, 然后我在git上下载了相同版本的源码进行驱动的ko编译,编译出的ko驱动的version magic则显示是5.4.119-1-0009.3,导致ko在insmod过程中不能通过内核版本检查,请教下这是什么原因谢谢~

system.slice not included in cpuset subsystem

Description
Install TencentOS-Server-3.1-for-x86_64-TK4-20210303.iso on my virtual machine,and couldn't find system.slice in /sys/fs/cgroup/cpuset/ directory.

[root@TENCENT64 cgroup]# find /sys/fs/cgroup/ -name "system.slice"
/sys/fs/cgroup/memory/system.slice
/sys/fs/cgroup/blkio/system.slice
/sys/fs/cgroup/devices/system.slice
/sys/fs/cgroup/pids/system.slice
/sys/fs/cgroup/cpu,cpuacct/system.slice
/sys/fs/cgroup/systemd/system.slice

Problem
please confirm whether the dir /sys/fs/cgroup/cpuset/system.slice exist?

关于容器增强功能

  1. Docker修改
    ​ >>业务如需在docker环境下使用该特性,需要自行修改docker添加相关的mount bind操作。

麻烦问一下
这部分的docker修改功能代码在哪能找到呢?貌似这部分修改没有放进你们的docker-xx.[s]rpm里?有什么样例代码可以提供的吗?谢谢。

kernel.container_cpuquota_aware not work

kernel.container_cpuquota_aware 这个开关有问题,无论是在主机还是在容器设置,无论是设置0还是1, 结果都一样,容器里面的/proc/cpuinfo都没有根据容器的cpuquota来换算。 客户期望都是要根据cpuquota来换算。

建议缺省要开启这个特性,容器里面的/proc/cpuinfo要根据容器的cpuquota来换算。

另外,这个开关可以去掉,目前没有这个需求。
或者作为系统级的开关,可读。 不需要作为容器级的开关。

tlinux默认没有添加bt的config

commit 7b0b2d84b956826329d20f01e1b605842c0db69f
Author: Xiaoming Gao <[email protected]>
Date:   Mon Sep 14 16:59:33 2020 +0800

    config: disable bt sched
    
    Signed-off-by: Xiaoming Gao <[email protected]>

diff --git a/package/default/config.default b/package/default/config.default
index 9a987dd..07a7d89 100644
--- a/package/default/config.default
+++ b/package/default/config.default
@@ -163,7 +163,6 @@ CONFIG_DEBUG_BLK_CGROUP=y
 CONFIG_CGROUP_WRITEBACK=y
 CONFIG_CGROUP_SCHED=y
 CONFIG_FAIR_GROUP_SCHED=y
-CONFIG_BT_GROUP_SCHED=y
 CONFIG_CFS_BANDWIDTH=y
 CONFIG_RT_GROUP_SCHED=y
 CONFIG_CGROUP_PIDS=y
@@ -185,7 +184,7 @@ CONFIG_IPC_NS=y
 CONFIG_USER_NS=y
 CONFIG_PID_NS=y
 CONFIG_NET_NS=y
-CONFIG_BT_SCHED=y
+# CONFIG_BT_SCHED is not set
 CONFIG_SCHED_AUTOGROUP=y
 # CONFIG_SYSFS_DEPRECATED is not set
 CONFIG_RELAY=y
@@ -1657,6 +1656,8 @@ CONFIG_BLK_DEV_SR=m
 CONFIG_BLK_DEV_SR_VENDOR=y
 CONFIG_CHR_DEV_SG=y
 CONFIG_CHR_DEV_SCH=y
+CONFIG_MEGARAID_LEGACY=y
+CONFIG_MEGARAID_SAS=y
 CONFIG_SCSI_CONSTANTS=y
 CONFIG_SCSI_LOGGING=y
 # CONFIG_SCSI_SCAN_ASYNC is not set
@@ -1708,8 +1709,6 @@ CONFIG_SCSI_MVUMI=y
 # CONFIG_SCSI_ADVANSYS is not set
 CONFIG_SCSI_ARCMSR=y
 # CONFIG_SCSI_ESAS2R is not set
-CONFIG_MEGARAID_SAS=y
-CONFIG_MEGARAID_LEGACY=y

tlinux现在默认没有添加bt的config,是担心bt的质量吗,如果我想把bt算法加回去,需要注意什么不?

兼容 Docker

希望可以兼容 docker 自动安装脚本,或支持修改 distribution

overlay2 dosesn't work with nfsv4 mountpoint subdir as lower-dir

nfsv4作为overlayfs lower层时,在overlayfs工作目录的子目录创建文件和目录没有权限。

原因:nfsv4 acl和posix acl不兼容,由于acl都是以xattr保存,overlayfs copy_up机制会将文件xattr中的nfsv4_acl copy_up到上层upper文件系统如ext4,其使用posix acl,导致没有权限创建文件和目录。当前5.7内核已修复该问题。https://github.com/torvalds/linux/blob/v5.7/fs/overlayfs/copy_up.c#L118

docker 安装问题

系统信息: tos -s
Machine type: kvm
Kernel version: 5.4.119-19-0007
TencentOS release: TencentOS Server 3.1 (Final)
Release version: 3.1
Release date: 20210604
Gcc version: gcc-8.4.1-1.tl3.x86_64
Glibc version: glibc-2.28-151.tl3.x86_64
Systemd version: systemd-239-45.tl3.1.x86_64

发行版 redhat-release
CentOS Linux release 8.4.2105 (Core)
发行版 centos-release
CentOS Linux release 8.4.2105 (Core)

当我尝试安装docker
$releasever 变量对应的是TencentOS发行版的版本 3.1
docker 并没有对应的版本

https://mirrors.cloud.tencent.com/docker-ce/linux/centos/

Errors during downloading metadata for repository 'docker-ce-stable':
  - Status code: 404 for https://mirrors.cloud.tencent.com/docker-ce/linux/centos/3.1/x86_64/stable/repodata/repomd.xml (IP: 42.194.157.107)
Error: Failed to download metadata for repo 'docker-ce-stable': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

试图修改$releasever 的值时 自身软件源 出现了问题

 Status code: 404 for http://mirrors.tencentyun.com/tlinux/3/TencentOS/x86_64/repodata/repomd.xml (IP: 0.0.0.0)
Error: Failed to download metadata for repo 'TencentOS': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

离线调度算法没有被启用生效

make menuconfig勾选了离线调度算法并成功编译,重启进入新内核后,cgroup目录下并没有出现离线调度相关的文件,调用sched_setschedluer()设置调度类型为7,提示参数不合法。
启用离线调度算法是否需要打开开关?

dracut:Failed to install module xen_blkfront

On aliyun centos7.8 with

rpm -i https://mirrors.tencent.com/tlinux/2.4/tlinux/x86_64/RPMS/kernel-headers-4.14.105-19.0012.tl2.x86_64.rpm

returns result dracut:Failed to install module xen_blkfront

usb模块缺失

请问腾讯内核是将usb模块给去除掉了吗?
另外在menuconfig内核配置中将usb重新启用后,也无法make进行配置更新

memory.vmstat cannot be mounted

Description:
Install TencentOS-Server-3.1 on my virtual machine and execute the following command
docker run -itd --mount type=bind,source=/sys/fs/cgroup/memory/system.slice/docker.service/memory.meminfo,destination=/proc/meminfo --mount type=bind,source=/sys/fs/cgroup/memory/system.slice/docker.service/memory.vmstat,destination=/proc/vmstat --mount type=bind,source=/sys/fs/cgroup/cpuset/docker/cpuset.stat,destination=/proc/stat --mount type=bind,source=/sys/fs/cgroup/cpuset/docker/cpuset.cpuinfo,destination=/proc/cpuinfo --mount type=bind,source=/sys/fs/cgroup/cpuset/docker/cpuset.loadavg,destination=/proc/loadavg --mount type=bind,source=/sys/fs/cgroup/cpuacct/system.slice/docker.service/cpuacct.uptime,destination=/proc/uptime --mount type=bind,source=/sys/fs/cgroup/blkio/system.slice/docker.service/blkio.diskstats,destination=/proc/diskstats <docker-image>

Error Message:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"rootfs_linux.go:58: mounting \\\"/sys/fs/cgroup/memory/system.slice/docker.service/memory.vmstat\\\" to rootfs \\\"/var/lib/docker/overlay2/0147401dcd448dfe8b9294637f812b88ad28db19a02529b1528b2e3ba7ea8433/merged\\\" at \\\"/proc/vmstat\\\" caused \\\"\\\\\\\"/var/lib/docker/overlay2/0147401dcd448dfe8b9294637f812b88ad28db19a02529b1528b2e3ba7ea8433/merged/proc/vmstat\\\\\\\" cannot be mounted because it is located inside \\\\\\\"/proc\\\\\\\"\\\"\"": unknown.

Problems
I wanna to confirm that if memory.vmstat can be mounted into docker container? Thx!

是否能获取 iops 合并后的数据?

我们是否能够取得顺序io在合并后的速率数据?

内核版本

4.14.105-19-0012

测试步骤

  1. 设置iops写阈值为10: echo "8:0 10" > blkio.throttle.write_iops_device
  2. 写数据: dd if=/dev/zero of=/tmp/iodemo.dat bs=4M count=100

预期结果

  1. blkio.throttle.io_serviced 每秒增加 10

实际结果

  1. blkio.throttle.io_serviced 每秒增加 300+
  2. blkio.io_merged 中的值似乎没有变化

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.