hewlettpackard / linuxki Goto Github PK

LinuxKI Toolset (Trace-based performance analysis tool)

License: Other

CSS 2.16% JavaScript 5.80% Makefile 0.47% HTML 19.73% Shell 3.81% PHP 0.24% C 67.29% Dockerfile 0.05% Batchfile 0.10% PowerShell 0.23% TSQL 0.11%

linuxki's People

Contributors

Stargazers

Watchers

linuxki's Issues

UNKNOWN - function / stack trace

data collected by ./runki -R -a ... what is missing to get anything meaningful for UNKNOWN? SLES15.3 kernel ..

Command line: /opt/linuxki/kiinfo -kiprof nsym=50,npid=20 -ts 0227_0602

kiinfo (7.6)

Linux host1 5.3.18-59.37-default #1 SMP Mon Nov 22 12:29:04 UTC 2021 (d10168e) x86_64 x86_64 x86_64 GNU/Linux

NOTE: idle hardclock traces are not always logged.


******** GLOBAL HARDCLOCK REPORT ********
  Count   USER%    SYS%   INTR%   IDLE%
1139513   2.36%   0.60%   0.00%  97.04%

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Kernel Functions executed during profile
   Count     Pct  State  Function
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   27505   2.41%  SYS    UNKNOWN
     351   0.03%  SYS    __schedule
     251   0.02%  SYS    cshook_systemcalltable_pre_compat_sys_ioctl
     241   0.02%  SYS    do_task_stat
     233   0.02%  SYS    lookup_fast
     172   0.02%  SYS    m_sys_openat
     161   0.01%  SYS    enc_threads_add_job
     135   0.01%  SYS    vsnprintf
     132   0.01%  SYS    __handle_mm_fault
     130   0.01%  SYS    cshook_security_inode_free_security
     126   0.01%  SYS    copyout
     115   0.01%  SYS    try_to_wake_up
     100   0.01%  SYS    __audit_syscall_exit

Error while installing linuxki on SLES 12 SP3

We are trying to install linuxki on SLES 12 SP3 and getting the following error
hostname:/tmp # rpm -ivh linuxki-5.7-1.noarch.rpm
Preparing... ################################# [100%]
Updating / installing...
1:linuxki-5.7-1 ################################# [100%]
Finishing installation of the Linux KI Toolset ...
Unable to find compatible module - attempting to build from source ...
hostkernel=4.4.155-94.50-default
hostdistro=sles
hostversion=12.3
EXTRA_CFLAGS=-g -O3 -Wall -DSLES12 -DHARDCLOCK_STACK_SKIP=0 -DOTHER_STACK_SKIP=3
make -C /lib/modules/4.4.155-94.50-default/build M=/tmp/tmp.l4T9oOUt7S modules
make[1]: *** /lib/modules/4.4.155-94.50-default/build: No such file or directory. Stop.
Makefile:150: recipe for target 'build' failed
make: *** [build] Error 2
insmod: ERROR: could not load module likit.ko: No such file or directory
Unable to build compatible LiKI module - you may still be able to use ftrace mode

Kernel version: 4.4.155-94.50-default

-sysignore switch doesn't seems to be working

Hello.
I am trying to filter out certain system calls for -kipid report. But to no avail.
I have prepared a file with syscalls names I would like to filter out:

[sergey@win7 0618_1119]$ cat sysignore.txt 
getrusage
times
stat

Afterward I run kiinfo like this:
kiinfo -ts 0618_1119 -kipid pid=32759,kitrace,subsys=syscalls,subsys=sched,sysignore=sysignore.txt | less

But syscalls are still in the output:

Command line: kiinfo -ts 0618_1119 -kipid pid=32759,kitrace,sysignore=sysignore.txt 

kiinfo (7.0)

Linux svacqapp-perf 5.3.18-24.64-default #1 SMP Tue May 4 10:05:02 UTC 2021 (0a8fae2) x86_64 x86_64 x86_64 GNU/Linux

KI Binary Version 9
    0.000161 cpu=22 pid=-1 tgid=-1 sched_wakeup target_pid=32759 prio=120 target_cpu=2 success=1
    0.000162 cpu=2 pid=0 tgid=0 sched_switch syscall=idle prio=n/a state=n/a next_pid=32759 next_prio=120 next_tgid=32759 policy=SCHED_NORMAL vss=0 rss=0
    0.000180 cpu=2 pid=32759 tgid=32759 write [1] entry fd=6 *buf=0x1b6efa6 count=13
    0.000189 cpu=2 pid=32759 tgid=32759 write [1] ret=0xd syscallbeg=    0.000009 fd=6 *buf=0x1b6efa6 count=13
    0.000190 cpu=2 pid=32759 tgid=32759 read [0] entry fd=6 *buf=0x1b6ceb6 count=8208
    0.000197 cpu=2 pid=32759 tgid=32759 sched_switch syscall=read prio=120 state=SLEEP next_pid=0 next_prio=120 next_tgid=n/a policy=n/a vss=67442 rss=8471 wait_woken+0x44 sk_wait_data+0x128 tcp_recvmsg+0x5e0 inet_recvmsg+0x5d sock_read_iter+0x92 new_sync_read+0x12a vfs_read+0x89 ksys_read+0xa1 do_syscall_64+0x5b entry_SYSCALL_64_after_hwframe+0x44 |  [libpthread-2.26.so]:__read+0x11  [libclntsh.so.21.1]:nttfprd+0x125  [libclntsh.so.21.1]:nsbasic_brc+0x165  [libclntsh.so.21.1]:nioqrc+0x1ef  [libclntsh.so.21.1]:ttcdrv+0x68d4  [libclntsh.so.21.1]:nioqwa+0x3d  [libclntsh.so.21.1]:upirtrc+0x635  [libclntsh.so.21.1]:upirtr+0x9e  [libclntsh.so.21.1]:kputxcmt+0x5c8  [libclntsh.so.21.1]:sqlcomt+0x71  [libclntsh.so.21.1]:sqlnst+0x409  [libclntsh.so.21.1]:sqlcxt+0x249  [posint]:commit_work+0x91  [posint]:main+0x226  [libc-2.26.so]:__libc_start_main+0xea
    0.000296 cpu=22 pid=-1 tgid=-1 sched_wakeup target_pid=32759 prio=120 target_cpu=2 success=1
    0.000297 cpu=2 pid=0 tgid=0 sched_switch syscall=idle prio=n/a state=n/a next_pid=32759 next_prio=120 next_tgid=32759 policy=SCHED_NORMAL vss=0 rss=0
    0.000301 cpu=2 pid=32759 tgid=32759 read [0] ret=0x11 syscallbeg=    0.000110 fd=6 *buf=0x1b6ceb6 count=8208
    0.000303 cpu=2 pid=32759 tgid=32759 times [100] entry *buf=0x7ffe9c541180
    0.000306 cpu=2 pid=32759 tgid=32759 times [100] ret=0x66bebc3b syscallbeg=    0.000002 *buf=0x7ffe9c541180
    0.000308 cpu=2 pid=32759 tgid=32759 write [1] entry fd=1 *buf=0x1a837c0 count=41
    0.000317 cpu=2 pid=32759 tgid=32759 write [1] ret=0x29 syscallbeg=    0.000009 fd=1 *buf=0x1a837c0 count=41 type=REG dev=0x0fe0000a ino=3408599
    0.000318 cpu=2 pid=32759 tgid=32759 times [100] entry *buf=0x7ffe9c541320
    0.000320 cpu=2 pid=32759 tgid=32759 times [100] ret=0x66bebc3b syscallbeg=    0.000002 *buf=0x7ffe9c541320
    0.000321 cpu=2 pid=32759 tgid=32759 write [1] entry fd=1 *buf=0x1a837c0 count=91
    0.000324 cpu=2 pid=32759 tgid=32759 write [1] ret=0x5b syscallbeg=    0.000003 fd=1 *buf=0x1a837c0 count=91 type=REG dev=0x0fe0000a ino=3408599
    0.000325 cpu=2 pid=32759 tgid=32759 times [100] entry *buf=0x7ffe9c5411f0
    0.000327 cpu=2 pid=32759 tgid=32759 times [100] ret=0x66bebc3b syscallbeg=    0.000001 *buf=0x7ffe9c5411f0
    0.000327 cpu=2 pid=32759 tgid=32759 times [100] entry *buf=0x7ffe9c5411f0
    0.000329 cpu=2 pid=32759 tgid=32759 times [100] ret=0x66bebc3b syscallbeg=    0.000001 *buf=0x7ffe9c5411f0
    0.000330 cpu=2 pid=32759 tgid=32759 write [1] entry fd=1 *buf=0x1a837c0 count=91
    0.000333 cpu=2 pid=32759 tgid=32759 write [1] ret=0x5b syscallbeg=    0.000003 fd=1 *buf=0x1a837c0 count=91 type=REG dev=0x0fe0000a ino=3408599
    0.000334 cpu=2 pid=32759 tgid=32759 write [1] entry fd=1 *buf=0x1a837c0 count=75
    0.000337 cpu=2 pid=32759 tgid=32759 write [1] ret=0x4b syscallbeg=    0.000003 fd=1 *buf=0x1a837c0 count=75 type=REG dev=0x0fe0000a ino=3408599
    0.000338 cpu=2 pid=32759 tgid=32759 msgrcv [70] entry msgid=19 *msgp=0x7ffe9c543070 msgsz=3224 msgtyp=0 msgflg=0x0
    0.000341 cpu=2 pid=32759 tgid=32759 msgrcv [70] ret=0x6a syscallbeg=    0.000003 msgid=19 *msgp=0x7ffe9c543070 msgsz=3224 msgtyp=0 msgflg=0x0
    0.000347 cpu=2 pid=32759 tgid=32759 stat [4] entry *path=0x7fa0bf56287f *buf=0x7ffe9c5415a0 filename: /etc/localtime

What am I missing here?

Don't hammer path in profile.d

/etc/profile.d/linuxki.sh should not destroy PATH it should do the following:

PATH=$PATH:/opt/linuxki

Also linuxki rpm should own /etc/profile.d/linuxki.sh and not generate it in %scripts.

Make packages available on the releases page and tag releases

It is great the README.md has links to the latest RPM and APT packages but that requires you to update the file every time you do a release and to add the packages into the source code repository (which is unusual). GitHub projects usually make release files (like RPMs or source tarballs) available on the Releases page and don't check them into the repository. The releases page also allows one to include notes such as what has been added, removed, and modified to highlight changes so users can decide if it makes sense to upgrade to a newer release. It also provides a history of releases and allows users to download older releases if needed.

Information on how to create releases in GitHub is available at:

It is also a common practice to tag releases with a version number. If tags are created, making releases is very simple. Information on creating tags in git is available here.

ZFS on LInux is a good example of a project using tags and releases (take a look at the 0.7.0 release to see how new functionality is documented).

linuxki fails to build on centos 8.2

linuxki 6.1 fails to build on centos 8.2 and it seems it need to have flag "-DRHEL82"

/opt/linuxki/src/liki/likit.c:6864:2: error: implicit declaration of function ‘synchronize_sched’; did you mean ‘synchronize_net’? [-Werror=implicit-function-declaration]
synchronize_sched();
^~~~~~~~~~~~~~~~~
synchronize_net

kiall - segmentation fault - AT_NULL terminator not found, cannot parse auxv structure

getting 'segfault' on SLES15.3 + SLES12.5 during kiall processing

# /opt/linuxki/kiall -r
Processing files in: /.../0827_1223
Merging KI binary files.  Please wait...
kiinfo -likimerge cannot merge ki.bin files.  Suspect default tracing (ftrace) used.
/opt/linuxki/kiinfo -kitrace -ts 0827_1223
/opt/linuxki/kiinfo -kiall csv -html -ts 0827_1223
/opt/linuxki/kiall: line 278: 24861 Segmentation fault      (core dumped) $DIR/kiinfo -kiall csv$oracle$mangle$blkfrq -html -ts $tag
kiall complete

/var/log/messages contains 2 entries - 2 files were also written

kernel: [1233077.597260] kiinfo[24861]: segfault at 7fcad2d94580 ip 0000000000425c30 sp 00007ffcf1299e20 error 4 in kiinfo[400000+e4000]
kernel: [1233077.763743] kiinfo[24854]: segfault at 7f3ad8100580 ip 00000000004a9279 sp 00007ffd3c608c80 error 4 in kiinfo[400000+e4000]
systemd-coredump[24873]: AT_NULL terminator not found, cannot parse auxv structure.
systemd-coredump[24874]: AT_NULL terminator not found, cannot parse auxv structure.

any clue?

error: ‘struct task_struct’ has no member named ‘state’; did you mean ‘__state’?

hostkernel=4.18.0-372.9.1.el8.x86_64
hostdistro=rhel
hostversion=8.6
EXTRA_CFLAGS=-g -O3 -Wall -DRHEL82 -DHARDCLOCK_STACK_SKIP=8 -DOTHER_STACK_SKIP=2
make -C /lib/modules/4.18.0-372.9.1.el8.x86_64/build M=/tmp/tmp.aHlmEJ55Wh modules
make[1]: Entering directory '/usr/src/kernels/4.18.0-372.9.1.el8.x86_64'
hostkernel=4.18.0-372.9.1.el8.x86_64
hostdistro=rhel
hostversion=8.6
EXTRA_CFLAGS=-g -O3 -Wall -DRHEL82 -DHARDCLOCK_STACK_SKIP=8 -DOTHER_STACK_SKIP=2
Building with KERNELRELEASE = 4.18.0-372.9.1.el8.x86_64
CC [M] /tmp/tmp.aHlmEJ55Wh/likit.o
/tmp/tmp.aHlmEJ55Wh/likit.c: In function ‘sched_switch_trace’:
/tmp/tmp.aHlmEJ55Wh/likit.c:2458:21: error: ‘struct task_struct’ has no member named ‘state’; did you mean ‘__state’?
t->prev_state = p->state;
^~~~~
__state
make[2]: *** [scripts/Makefile.build:322: /tmp/tmp.aHlmEJ55Wh/likit.o] Error 1
make[1]: *** [Makefile:1577: module/tmp/tmp.aHlmEJ55Wh] Error 2
make[1]: Leaving directory '/usr/src/kernels/4.18.0-372.9.1.el8.x86_64'
make: *** [Makefile:229: build] Error 2
insmod: ERROR: could not load module likit.ko: No such file or directory

ERROR: Unable to build compatible LiKI module
Please checked to be sure you have the proper kernel headers package installed
and it matches your current kernel version!

kiinfo metric units

Hello.
It would be nice to have information of metrics measurement units in reports. It is there sometimes. But not necessary. F.e. there's info on metrics and their units in kiinfo-kidsk(1). But there is not a word on kiinfo-kipid's AvSz field.

kiall - spell error "budle" --> "bundle"

printf " -t Only process KI budle with matching timestamp\n"

LinuxKI/scripts/kiall

Line 133 in 1d07bd9

printf " -t <timestamp> Only process KI budle with matching timestamp\n"

LinuxKI/experimental/vis/kiall.pk

Line 97 in 1d07bd9

printf " -t <timestamp> Only process KI budle with matching timestamp\n"

likit.ko module says that scsi_dispatch_cmd_start and scsi_dispatch_cmd_done events could not be initialized

When I load linuxki kernel module, it says that it cannot initialize scsi_dispatch_cmd_start and scsi_dispatch_cmd_done events. Is this something I should be worried about?

# dmesg -T | grep -i liki
...
[Wed Apr 24 09:35:32 2019] LiKI: tracing starting up...
[Wed Apr 24 09:35:32 2019] LiKI: event scsi_dispatch_cmd_start could not be initialized
[Wed Apr 24 09:35:32 2019] LiKI: event scsi_dispatch_cmd_done could not be initialized
[Wed Apr 24 09:35:32 2019] LiKI: tracing was set up successfully
[Wed Apr 24 09:35:48 2019] LiKI: tracing shutting down...
[Wed Apr 24 09:35:48 2019] LiKI: shut down successfully

I am using linuxki version 5.9 on HPE ProLiant DL580 Gen9 with SUSE Linux Enterprise Server for SAP Applications 12 SP2 and with SUSE PTF kernel 4.4.120-92.70.1.15108.1.TEST.1086194-default (differences against ordinary 4.4.120-92.70.1 are below).

# rpm -qa linuxki*
linuxki-5.9-1.noarch

# cat /etc/os-release
NAME="SLES_SAP"
VERSION="12-SP2"
VERSION_ID="12.2"
PRETTY_NAME="SUSE Linux Enterprise Server for SAP Applications 12 SP2"
ID="sles_sap"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles_sap:12:sp2"

# uname -r
4.4.120-92.70.1.15108.1.TEST.1086194-default

# dmidecode -s system-product-name
ProLiant DL580 Gen9

# rpm -qi kernel-default-4.4.120-92.70.1.15108.1.TEST.1086194.x86_64 --changelog | less
Name        : kernel-default
Version     : 4.4.120
Release     : 92.70.1.15108.1.TEST.1086194
Architecture: x86_64
Install Date: Mon Mar  4 01:39:08 2019
Group       : System/Kernel
Size        : 168310666
License     : GPL-2.0
Signature   : DSA/SHA1, Wed Mar 28 14:19:37 2018, Key ID 6c74ce73b37b98a9
Source RPM  : kernel-default-4.4.120-92.70.1.15108.1.TEST.1086194.nosrc.rpm
Build Date  : Wed Mar 28 14:01:56 2018
Build Host  : sheep54
Relocations : (not relocatable)
Packager    : http://bugs.opensuse.org
Vendor      : SUSE LINUX Products GmbH, Nuernberg, Germany
URL         : http://www.kernel.org/
Summary     : The Standard Kernel
Description :
The standard kernel for both uniprocessor and multiprocessor systems.


Source Timestamp: 2018-03-28 14:36:53 +0200
GIT Revision: aab93cdae9a62006ad5213c78cd79672f2b99c41
GIT Branch: users/slahl/SLE12-SP2/1085656-3
Distribution: SUSE Linux Enterprise PTF
* Wed Mar 28 2018 [email protected]
- fix for last build.

* Wed Mar 28 2018 [email protected]
- bsc#1085656, bsc#1086194
- commit aab93cd
...

Security Vulnerabilities with LinuxKI

The following security vulnerabilities have been identified in LinuxKI version 6.0-1 and earlier.

CVE-2020-7208 Cody Winkler and Chris Inzinga reported an XSS vulnerability which is resolved in this release.
CVE-2020-7209 Cody Winkler and Chris Inzinga reported remote code execution vulnerabilities which are resolved in this release.

system hang during linuxki

When I try to start runki, system hangs. I have to restart system from IPMI. No output in console.
Founded the problem related with OpenvSwitch. When I stop openvswitch-switch.service, runki works without issues.

# uname -a
Linux ds1-cpu-01.ds1 4.13.0-31-generic #34~16.04.1-Ubuntu SMP Fri Jan 19 17:11:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

# ovs-vsctl -V    
ovs-vsctl (Open vSwitch) 2.5.2
Compiled Oct 17 2017 16:38:57
DB Schema 7.12.1

# dpkg --status linuxki | grep Version
Version: 5.5-1

kiall -f minor issue

Hello.
I was wondering why on my machine `kiall -f' doesn't work with messages:

[sergey@win7 work]$ kiall -f
find: unknown predicate `--header'
kiall complete

It is the line 504 of the script with find command gives such output.
Looking carefully at the files in current directory I found this gem:

sergey@win7 work]$ ll
total 712
-rw-rw-r--. 1 sergey sergey 130547 июн 15 21:39 boot-1.log
[... skip ...]
-rw-rw-r--. 1 sergey sergey    131 июн 12 15:36 --header
[... skip ...]

The fix is simple: just replace '*' to '.' in find parameters. I'm not sure if such trivial thing is worth creating pull requests and all this suff...

Old files in repository

Looks like there are a few files in the repository which don't belong there:

experimental/D3/lib/parcoord/.slickgrid.html.swp
experimental/vis/cluster_timeline.html.orig
experimental/vis/futex_scatter.html.rev2
experimental/vis/kidsk_scatter.html.backup
TaskletIRQs.htm. This hasn't been updated since 5.6 while there is also taskletIRQs.htm which was updated recently. On a case insensitive system like Windows, you may only see one file but the repository has both. I suspect it was renamed but the old file didn't get deleted since the files are very similar. Two files differing only in case is a real problem for syncing solutions like Dropbox.
md_flush.htm. This hasn't been updated since 5.4 while there is also the file 'mdflush.htm'. I suspect it was renamed but the old file didn't get deleted since the files are very similar.

switching to the new unwind code

Are there any plans to move to new unwind code?
https://patchwork.kernel.org/patch/9334149/

Old unwind code is removed since ~4.9 and dump_trace() deprecated. So likit fails to compile with new kernels.

kernel crashed when kiinfo -live exit

Hi Mark,

Greetings.

My KVM guest crashed when I was quitting "kiinfo -live", it looks like a kmem cache problem, the kernel version is centos 3.10.0-514.26.2.el7.x86_64, dmesg is as below:

[ 4257.498687] BUG: unable to handle kernel NULL pointer dereference at           (null)
[ 4257.499665] IP: [<ffffffff811e0081>] kmem_cache_close+0xc1/0x2f0
[ 4257.499665] PGD 31b7ee2067 PUD 31b7ee0067 PMD 0 
[ 4257.499665] Oops: 0000 [#1] SMP 
[ 4257.499665] Modules linked in: binfmt_misc likit(OE-) ebt_ip ebtable_filter e
btables veth xt_nat xt_recent xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_c
omment xt_set ip_set_hash_net ip_set br_netfilter nls_utf8 isofs xt_conntrack br
idge stp llc nf_conntrack_netlink nfnetlink overlay(T) xt_addrtype iptable_nat n
f_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat ip_vs nf_conntrack libcrc32c 
iptable_filter sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw 
gf128mul ppdev glue_helper ablk_helper cryptd parport_pc sg i2c_piix4 parport vi
rtio_balloon pcspkr ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_ac
pi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt
 fb_sys_fops ttm crct10dif_pclmul crct10dif_common ata_piix drm virtio_pci crc32
c_intel libata
[ 4257.499665]  virtio_ring i2c_core serio_raw virtio floppy [last unloaded: likit]
[ 4257.499665] CPU: 14 PID: 4393 Comm: rmmod Tainted: G    B      OE  ------------ T 3.10.0-514.26.2.el7.x86_64 #1
[ 4257.499665] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 4257.499665] task: ffff88199e353ec0 ti: ffff881ba6eb0000 task.ti: ffff881ba6eb0000
[ 4257.499665] RIP: 0010:[<ffffffff811e0081>]  [<ffffffff811e0081>] kmem_cache_close+0xc1/0x2f0
[ 4257.499665] RSP: 0018:ffff881ba6eb3db8  EFLAGS: 00010202
[ 4257.499665] RAX: ffff881baa269001 RBX: ffff8836e0670000 RCX: 0000000000000066
[ 4257.499665] RDX: 0000000000000065 RSI: ffff881baf98f838 RDI: ffff88017fc03e00
[ 4257.499665] RBP: ffff881ba6eb3e08 R08: 0000000000019a20 R09: ffffffff811e01a5
[ 4257.499665] R10: ffff881baf999a20 R11: ffffea006ea89a40 R12: ffffea00db819a00
[ 4257.499665] R13: ffff8836e0668000 R14: ffff8837acf8f700 R15: ffffffffffffffe0
[ 4257.499665] FS:  00007f242a233740(0000) GS:ffff881baf980000(0000) knlGS:00000
00000000000
[ 4257.499665] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4257.499665] CR2: 0000000000000000 CR3: 00000031477d6000 CR4: 00000000000406e0
[ 4257.499665] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4257.499665] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 4257.499665] Stack:
[ 4257.499665]  ffff8837acf8f708 00000001a6eb3e68 ffff883719fa9d00 ffff881baa269000
[ 4257.499665]  ffff883719fa9d10 ffff8837acf8f700 ffff8837acf8f700 ffff88017fc03500
[ 4257.499665]  0000000000000297 0000000001339010 ffff881ba6eb3e28 ffffffff811e02c4
[ 4257.499665] Call Trace:
[ 4257.499665]  [<ffffffff811e02c4>] __kmem_cache_shutdown+0x14/0x80
[ 4257.499665]  [<ffffffff811a5e14>] kmem_cache_destroy+0x44/0xf0
[ 4257.499665]  [<ffffffff811f4619>] kmem_cache_destroy_memcg_children+0x89/0xb0
[ 4257.499665]  [<ffffffff811a5de9>] kmem_cache_destroy+0x19/0xf0
[ 4257.499665]  [<ffffffffa043637a>] shutdown+0x2da/0x4c0 [likit]
[ 4257.499665]  [<ffffffffa043bece>] liki_exit+0xe/0x140 [likit]
[ 4257.499665]  [<ffffffff810fe3db>] SyS_delete_module+0x16b/0x2d0
[ 4257.499665]  [<ffffffff81697a12>] tracesys+0xdd/0xe2
[ 4257.499665] Code: 8d 6a e0 4d 89 ef 75 3f e9 4d 01 00 00 0f 1f 44 00 00 e8 13 3a 15 00 48 8b 45 c0 4c 89 e6 4c 89 f7 48 83 68 08 01 e8 4f b1 ff ff <49> 8b 47 20 49 8d 7f 20 48 83 e8 20 48 3b 7d d0 0f 84 19 01 00 
[ 4257.499665] RIP  [<ffffffff811e0081>] kmem_cache_close+0xc1/0x2f0
[ 4257.499665]  RSP <ffff881ba6eb3db8>
[ 4257.499665] CR2: 0000000000000000

Before the crash messages, there are some repeated messages like this:

[ 4257.423584] =============================================================================
[ 4257.425041] BUG kmalloc-1024(20:0a73932bbc61f72f22a66c679c2c9b5185308d8090420
0dc49b1beb1bccd1deb) (Tainted: G    B      OE  ------------ T): Objects remaining in kmalloc-1024(20:0a73932bbc61f72f22a66c679c2c9b5185308d80904200dc49b1beb1bccd1d
[ 4257.428439] -----------------------------------------------------------------------------

[ 4257.430644] INFO: Slab 0xffffea00db819a00 objects=32 used=31 fp=0xffff8836e066c000 flags=0x6fffff00004080
[ 4257.432323] CPU: 14 PID: 4393 Comm: rmmod Tainted: G    B      OE  ------------ T 3.10.0-514.26.2.el7.x86_64 #1
[ 4257.434146] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 4257.435449]  ffffea00db819a00 00000000ee3e6997 ffff881ba6eb3cd0 ffffffff81687133
[ 4257.437022]  ffff881ba6eb3da8 ffffffff811da714 0000000000000020 ffff881ba6eb3db8
[ 4257.438649]  ffff881ba6eb3d68 656a624f818df958 616d657220737463 6e6920676e696e69
[ 4257.440244] Call Trace:
[ 4257.441183]  [<ffffffff81687133>] dump_stack+0x19/0x1b
[ 4257.442492]  [<ffffffff811da714>] slab_err+0xb4/0xe0
[ 4257.443741]  [<ffffffff811dda13>] ? __kmalloc+0x1f3/0x240
[ 4257.445084]  [<ffffffff811e00eb>] ? kmem_cache_close+0x12b/0x2f0
[ 4257.446447]  [<ffffffff811e010c>] kmem_cache_close+0x14c/0x2f0
[ 4257.447791]  [<ffffffff811e02c4>] __kmem_cache_shutdown+0x14/0x80
[ 4257.449223]  [<ffffffff811a5e14>] kmem_cache_destroy+0x44/0xf0
[ 4257.450616]  [<ffffffff811f4619>] kmem_cache_destroy_memcg_children+0x89/0xb0
[ 4257.452129]  [<ffffffff811a5de9>] kmem_cache_destroy+0x19/0xf0
[ 4257.453475]  [<ffffffffa043637a>] shutdown+0x2da/0x4c0 [likit]
[ 4257.454882]  [<ffffffffa043bece>] liki_exit+0xe/0x140 [likit]
[ 4257.456396]  [<ffffffff810fe3db>] SyS_delete_module+0x16b/0x2d0
[ 4257.457807]  [<ffffffff81697a12>] tracesys+0xdd/0xe2
[ 4257.459103] INFO: Object 0xffff8836e0668000 @offset=0
[ 4257.460432] INFO: Object 0xffff8836e0668400 @offset=1024
[ 4257.461802] INFO: Object 0xffff8836e0668800 @offset=2048
[ 4257.463178] INFO: Object 0xffff8836e0668c00 @offset=3072
[ 4257.464522] INFO: Object 0xffff8836e0669000 @offset=4096
[ 4257.465916] INFO: Object 0xffff8836e0669400 @offset=5120
[ 4257.467292] INFO: Object 0xffff8836e0669800 @offset=6144

I have a vmcore saved, am still looking at it, haven't pinpointed the responsible code yet, can send to you if you are interested.

Best regards,
Wenbin Zeng

LinuxKI_MasterClass.pdf - page 98 sysignore=scall_ignore ; filename = syscall_ignore

$ cat syscall_ignore
gettimeofday
getrusage

$ kiinfo -kitrace tgid=67304,events=all,sysignore=scall_ignore

Feature request: add DKMS support

Since LinuxKI uses a kernel module and it needs to be rebuilt whenever the kernel is updated, it should support DKMS so the module can automatically be rebuilt when a new kernel is installed. Unfortunately, while it is actually relatively easy to create the required configuration file, it isn't well documented.

Here are some useful links:

DKMS source repository
Sample DKMS configuration
Sample RPM specification using DKMS
Dynamic Kernel Module Support documentation for Arch Linux
CentOS HowTos Build Your Own Kernel Modules, Section 2: Building a kernel module using Dynamic Kernel Module Support (DKMS)
Kernel Korner - Exploring Dynamic Kernel Module Support (DKMS)

IRQ name incomplete in kirunq.txt

Hello Mark,

in irqlist I see the below. Please note "60" and "61" for the name...

   IRQ   node         cpu_list  affinity_mask         name
   231      4      353,355,357  00000000,00000000,0000002a,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000  qla2xxx12_qpair60
   232      4      359,361,363  00000000,00000000,00000a80,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000  qla2xxx12_qpair61

But in kirunq I see "6" only

  IRQ Name                                Count      ElpTime    Avg(usec)
  231 qla2xxx12_qpair6                     1331     0.001926     1.446892
  232 qla2xxx12_qpair6                     1476     0.002117     1.434320

The correct mapping can still be done via the IRQ number.

Typo on runki

Hi!

I've notice a typo on runki shell script:

LinuxKI/scripts/runki

Line 1139 in ce33fdf

service trace-cmd status >/dev=null 2&>/dev/null

There is a redirection to "/dev=null" instead on "/dev/null"

Thanks

Linux KI 5.3.1 error in redhat 5.8

rpm --install –nodeps linuxki-5.3-1.noarch.rpm

Or
#rpm –ivh –nodeps –force linuxki-5.3-1.noarch.rpm
But in both the option getting below error:-
Error: unpack aging of Archive failed : cpio: Bad Magic

Generated HTML difficult to read in lynx

After taking a short KI sample, I figured the easiest way to check if the generated file captured the data I wanted would be to pop open a textmode browser directly on the server and take a quick peek at the html output of kiall -r. Unfortunately, most of the result rows throughout the file sort of mushed together. This can imply other accessibility issues: if it doesn't look good in a textmode browser, it probably doesn't sound good to a screenreader!

This could probably be remedied reasonably well by just generating the results as properly-structured HTML tables.

kiall may hang after merging ki.bin files

After adjusting permissions from the defaults, when I run kiall it hang after line like this:

ki.bin files merged by kiinfo -likimerge

In process list I see kiall with one child: rm ki.bin.... ki.bin... ...

IMHO, the problem is that this line may get interactive:
rm ki.bin.*.$tag >/dev/null 2>&1
I think it should be:
rm -f ki.bin.*.$tag >/dev/null 2>&1

ERROR: ftrace dump failed on rhel8

# cat /etc/redhat-release;uname -sr;/opt/linuxki/runki -f
Red Hat Enterprise Linux release 8.7 (Ootpa)
Linux 4.18.0-425.10.1.el8_7.x86_64
=== runki for Linux version 7.5
=== Disabling TTWU_QUEUE scheduling feature ===
=== Starting: Linux KI ftrace trace dump for 20 seconds ===
===
Command line: /opt/linuxki/kiinfo -dump dur=20,nop,nop,debug_dir=/sys/kernel/debug -ts 0310_0825

kiinfo (7.6)

Pausing 10 seconds to sync CPU clocks, please wait...
=== ERROR: ftrace dump failed ===
Restoring Tracing events
===
=== KI trace complete.  Collecting supplemental files.
=== Enabling TTWU_QUEUE scheduling feature ===
=== Running lsof ...

Not working on rhel 9.1

Note: Solution tested and working.
r->rq_disk. It can now be accessed through r->q->disk
#define GENDISK r->q->disk

Error was:

./module_prep

    Unable to find compatible module - attempting to build from source ...

hostkernel=5.14.0-162.6.1.el9_1.x86_64
hostdistro=rhel
hostversion=9.1
EXTRA_CFLAGS=-g -O3 -Wall -DHARDCLOCK_STACK_SKIP=8 -DOTHER_STACK_SKIP=2
make -C /lib/modules/5.14.0-162.6.1.el9_1.x86_64/build M=/tmp/tmp.Nu3mx0EhmZ modules
make[1]: Entering directory '/usr/src/kernels/5.14.0-162.6.1.el9_1.x86_64'
hostkernel=5.14.0-162.6.1.el9_1.x86_64
hostdistro=rhel
hostversion=9.1
EXTRA_CFLAGS=-g -O3 -Wall -DHARDCLOCK_STACK_SKIP=8 -DOTHER_STACK_SKIP=2
Building with KERNELRELEASE = 5.14.0-162.6.1.el9_1.x86_64
CC [M] /tmp/tmp.Nu3mx0EhmZ/likit.o
/tmp/tmp.Nu3mx0EhmZ/likit.c: In function ‘block_rq_insert_trace’:
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2773:31: note: in expansion of macro ‘GENDISK’
2773 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2773:51: note: in expansion of macro ‘GENDISK’
2773 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:18: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2796:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2796 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:38: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2796:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2796 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c: In function ‘block_rq_issue_trace’:
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2839:31: note: in expansion of macro ‘GENDISK’
2839 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2839:51: note: in expansion of macro ‘GENDISK’
2839 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:18: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2857:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2857 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:38: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2857:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2857 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c: In function ‘block_rq_complete_trace’:
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2914:31: note: in expansion of macro ‘GENDISK’
2914 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2914:51: note: in expansion of macro ‘GENDISK’
2914 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:18: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2932:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2932 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:38: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2932:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
2932 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c: In function ‘block_rq_requeue_trace’:
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2986:31: note: in expansion of macro ‘GENDISK’
2986 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2986:51: note: in expansion of macro ‘GENDISK’
2986 | dev_t dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:18: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:3004:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
3004 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2698:18: error: ‘struct request’ has no member named ‘rq_disk’
2698 | #define GENDISK r->rq_disk
| ^~
/tmp/tmp.Nu3mx0EhmZ/likit.c:2714:38: note: in expansion of macro ‘GENDISK’
2714 | T->dev = GENDISK ? disk_devt(GENDISK) : 0;
| ^~~~~~~
/tmp/tmp.Nu3mx0EhmZ/likit.c:3004:9: note: in expansion of macro ‘POPULATE_COMMON_BLOCK_FIELDS’
3004 | POPULATE_COMMON_BLOCK_FIELDS(q, r, t);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [scripts/Makefile.build:295: /tmp/tmp.Nu3mx0EhmZ/likit.o] Error 1
make[1]: *** [Makefile:1915: /tmp/tmp.Nu3mx0EhmZ] Error 2
make[1]: Leaving directory '/usr/src/kernels/5.14.0-162.6.1.el9_1.x86_64'
make: *** [Makefile:262: build] Error 2
insmod: ERROR: could not load module likit.ko: No such file or directory

Any plan to support newest kernel?

Hi @MarkCRay,

Greetings from me!

It seems kallsyms_lookup_name() and kallsyms_on_each_symbol () are not exported any more in the newest kernel (from this commit). So does LinuxKI have plan to support newest kernel? Thanks very much in advance!

Best Regards
Nan Xiao

panic on ppc64le, kernel BUG at ../kernel/irq_work.c:135

Got a panic with LinuxKI 5.9 on PPC64le and wondering if this is something known.
Is it reasonable to try 6.0?

[65540.384286] kernel BUG at ../kernel/irq_work.c:135!
[65540.384390] Oops: Exception in kernel mode, sig: 5 [#1]
.......
[65540.398561] NIP [c000000000298d84] irq_work_run_list+0x14/0x100
[65540.398742] LR [c0000000001bbd30] update_process_times+0x60/0x90
[65540.398895] Call Trace:
[65540.398942] [c000000003d37190] [c0000000001bbd18] update_process_times+0x48/0x90 (unreliable)
[65540.399194] [c000000003d371c0] [c0000000001d3d20] tick_sched_handle.isra.8+0x30/0xb0
[65540.399422] [c000000003d371f0] [c0000000001d3e04] tick_sched_timer+0x64/0xd0
[65540.399631] [c000000003d37230] [c0000000001bc8c4] __hrtimer_run_queues+0x144/0x380
[65540.399836] [c000000003d372b0] [c0000000001bd90c] hrtimer_interrupt+0xfc/0x350
[65540.400112] [c000000003d37380] [c00000000002715c] __timer_interrupt+0x8c/0x260
[65540.400400] [c000000003d373d0] [c000000000027540] timer_interrupt+0x90/0xe0
[65540.400661] [c000000003d37400] [c00000000000de40] restore_check_irq_replay+0x54/0x70
[65540.400931] --- interrupt: 901 at bio_alloc_bioset+0x10/0x2f0

ER: Add Read KB/sec and Write KB/sec to kidsk output

The kidsk.txt output currently looks as follows:

Top 20 Tasks sorted by Multipath I/O

     Cnt      r/s      w/s    KB/sec    Avserv      PID  Process
==============================================================================
   49534      137     2339  728569.2     1.935   223012  abc

Please enhance it so the type of the amount of data transfered by the threads can be identified:

Top 20 Tasks sorted by Multipath I/O

     Cnt       r/s   rAvserv     rKB/s      w/s   wAvserv   wKB/sec    Avserv      PID  Process
====================================================================================================
   49534       137     3.123    8569.0     2339     0.512  720000.2     1.935   223012  abc

Feature request: add man pages

It would be nice if man pages were included in the RPM install package. The tools have a lot of command line options. Being able to look up the documentation for them using the standard man pages would be helpful.

Some useful references on how to create a man page are:

man 7 man -- documents the macros used to format man pages
man man-pages -- documents the convention for writing Linux man pages
man mdoc -- quick reference guide for the -mdoc macro package for groff
man groff -- documents teh front-end for the groff document formatting system
Linux Man Page Howto
Linux Howtos: System -> Creating Your Own MAN Page Version

LiKI module build fails on Fedora 28

When installing the RPM on Fedora 28, the LiKI module build fails:

$ sudo rpm -ivh rpms/linuxki-5.4-1.noarch.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:linuxki-5.4-1                    ################################# [100%]
        Finishing installation of the Linux KI Toolset ...
        Unable to find compatible module - attempting to build from source ...
hostkernel=4.17.2-200.fc28.x86_64
hostdistro=fedora
hostversion=28
EXTRA_CFLAGS=-g -O3 -Wall -DHARDCLOCK_STACK_SKIP=0 -DOTHER_STACK_SKIP=2
make -C /lib/modules/4.17.2-200.fc28.x86_64/build  M=/tmp/tmp.quafAlVvP1   modules  
make[1]: Entering directory '/usr/src/kernels/4.17.2-200.fc28.x86_64'
hostkernel=4.17.2-200.fc28.x86_64
hostdistro=fedora
hostversion=28
EXTRA_CFLAGS=-g -O3 -Wall -DHARDCLOCK_STACK_SKIP=0 -DOTHER_STACK_SKIP=2
Building with KERNELRELEASE = 4.17.2-200.fc28.x86_64
  CC [M]  /tmp/tmp.quafAlVvP1/likit.o
/tmp/tmp.quafAlVvP1/likit.c: In function ‘syscall_exit_trace’:
/tmp/tmp.quafAlVvP1/likit.c:4116:59: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
                                                           ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4116:59: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4116:7: error: too many arguments to function ‘sock->ops->getname’
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
       ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4128:61: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
    if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
                                                             ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4128:61: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4128:8: error: too many arguments to function ‘sock->ops->getname’
    if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
        ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4223:61: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
    if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
                                                             ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4223:61: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4223:8: error: too many arguments to function ‘sock->ops->getname’
    if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
        ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4228:60: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
    if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
                                                            ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4228:60: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4228:8: error: too many arguments to function ‘sock->ops->getname’
    if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
        ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4349:60: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
   if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
                                                            ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4349:60: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4349:7: error: too many arguments to function ‘sock->ops->getname’
   if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
       ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4354:59: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
                                                           ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4354:59: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4354:7: error: too many arguments to function ‘sock->ops->getname’
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
       ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4419:60: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
   if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
                                                            ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4419:60: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4419:7: error: too many arguments to function ‘sock->ops->getname’
   if (sock->ops->getname(sock, (struct sockaddr *)&remote, &remlen, 1)) {
       ^~~~
/tmp/tmp.quafAlVvP1/likit.c:4424:59: warning: passing argument 3 of ‘sock->ops->getname’ makes integer from pointer without a cast [-Wint-conversion]
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
                                                           ^~~~~~~
/tmp/tmp.quafAlVvP1/likit.c:4424:59: note: expected ‘int’ but argument is of type ‘int *’
/tmp/tmp.quafAlVvP1/likit.c:4424:7: error: too many arguments to function ‘sock->ops->getname’
   if (sock->ops->getname(sock, (struct sockaddr *)&local, &loclen, 0)) {
       ^~~~
make[2]: *** [scripts/Makefile.build:319: /tmp/tmp.quafAlVvP1/likit.o] Error 1
make[1]: *** [Makefile:1575: _module_/tmp/tmp.quafAlVvP1] Error 2
make[1]: Leaving directory '/usr/src/kernels/4.17.2-200.fc28.x86_64'
make: *** [Makefile:147: build] Error 2
        Unable to build compatible LiKI module - you may still be able to use ftrace mode

Feature request: add /opt/linuxki to PATH during install

When installing the RPM, it would be useful if a script was put into /etc/profile.d which would add /opt/linuxki to the PATH. This is what most RPMs do so users expect they can run commands provided by the RPM immediately after installation.

If you don't want the RPM to automatically add /opt/linuxki to the PATH, I think it would be better to document the step of updating the PATH in the Installation section of the documentation rather than the Data collection section of the README.md.

Also, updating the PATH isn't listed anywhere in the LinuxKI Quick Reference Guide. I would have expected it to be listed in the Installation section if the RPM didn't automatically add it to the PATH.

Idea: use ebpf to replace liki.ko?

Hi Mark,

I am wondering if it is possible to replace liki ko with ebpf on newer kernels? ebpf is supposed to be more native and robust than kernel modules, isn't it?

Best regards,
Wenbin

Oracle Linux UEK

During the rpm Install the likit.ko gives an error.
insmod: ERROR: could not load module likit.ko: No such file or directory
Having kernel-uek-devel installed.

But if we do the procedure:
cd /opt/linuxki
rm -f /opt/linuxki/modules/likit.ko*
/opt/linuxki/module_prep
The likit.ko get installed

So it seems there is a different procedure on rpm install VS /opt/linuxki/module_prep

Add "how to compile & install from source code" section in document

Hi @MarkCRay and LinuxKI developers,

Greetings from me!

I checked both README and LinuxKI_MasterClass, and it seems that there lacks a section which introduces how to compile & install from source code. Is it possible to add this section in document? Thanks very much in advance!

Best Regards
Nan Xiao

Feature request: add bash completion support

Given the number of command line options supported by the various tools, it would be nice if the RPM install included a bash completion script.

Some useful references on how to create bash completion scripts are:

Non-executables files incorrectly marked as executable

There are many files in the repository which have their executable bit set even though they are not executable files. This really ugly command will provide a list of the files:

fgrep -L '!/bin/bash' $(file $(find . -type f -executable | 
  egrep -v '\.(php|sh)') | egrep -v 'shell|node script' 2>/dev/null | 
  awk '{sub(":", "", $1); print $1}') | sort

./config/config
./config/config.cmu
./config/config.pdsh
./COPYING.kiinfo
./COPYING.liki
./COPYING.linuxki
./experimental/D3/lib/colorbrewer/colorbrewer.css
./experimental/D3/lib/colorbrewer/colorbrewer.js
./experimental/D3/lib/colorbrewer/LICENSE
./experimental/D3/lib/d3/component.json
./experimental/D3/lib/d3/d3.js
./experimental/D3/lib/d3/d3.min.js
./experimental/D3/lib/d3/.gitignore
./experimental/D3/lib/d3/.gitmodules
./experimental/D3/lib/d3/globals.js
./experimental/D3/lib/d3/index-browserify.js
./experimental/D3/lib/d3/index.js
./experimental/D3/lib/d3/LICENSE
./experimental/D3/lib/d3/Makefile
./experimental/D3/lib/d3/.npmignore
./experimental/D3/lib/d3/package.json
./experimental/D3/lib/d3/README.md
./experimental/D3/lib/d3/.travis.yml
./experimental/D3/lib/divgrid.js
./experimental/D3/lib/firebugx.js
./experimental/D3/lib/jit/LICENSE
./experimental/D3/lib/jquery-1.7.min.js
./experimental/D3/lib/jquery.event.drag-2.2.js
./experimental/D3/lib/jquery.event.drop-2.2.js
./experimental/D3/lib/jquery.jsonp-2.4.min.js
./experimental/D3/lib/jquery.simulate.js
./experimental/D3/lib/jquery.sparkline.min.js
./experimental/D3/lib/jquery-ui-1.8.16.custom.min.js
./experimental/D3/lib/mapper.js
./experimental/D3/lib/parcoord/d3.parcoords.css
./experimental/D3/lib/parcoord/d3.parcoords.js
./experimental/D3/lib/parcoord/.slickgrid.html.swp
./experimental/D3/lib/parcoord/style.css
./experimental/D3/lib/penner/LICENSE
./experimental/D3/lib/polymaps/LICENSE
./experimental/D3/lib/protovis/LICENSE
./experimental/D3/lib/qunit.css
./experimental/D3/lib/qunit.js
./experimental/D3/lib/science/LICENSE
./experimental/D3/lib/science/science.js
./experimental/D3/lib/science/science.lin.js
./experimental/D3/lib/science/science.lin.min.js
./experimental/D3/lib/science/science.min.js
./experimental/D3/lib/science/science.stats.js
./experimental/D3/lib/science/science.stats.min.js
./experimental/D3/lib/slickgrid/controls/slick.columnpicker.css
./experimental/D3/lib/slickgrid/controls/slick.columnpicker.js
./experimental/D3/lib/slickgrid/controls/slick.pager.css
./experimental/D3/lib/slickgrid/controls/slick.pager.js
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_flat_75_ffffff_40x100.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_glass_65_ffffff_1x400.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_glass_75_dadada_1x400.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-icons_222222_256x240.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-icons_2e83ff_256x240.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-icons_454545_256x240.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-icons_888888_256x240.png
./experimental/D3/lib/slickgrid/css/smoothness/images/ui-icons_cd0a0a_256x240.png
./experimental/D3/lib/slickgrid/css/smoothness/jquery-ui-1.8.16.custom.css
./experimental/D3/lib/slickgrid/examples/example10-async-post-render.html
./experimental/D3/lib/slickgrid/examples/example11-autoheight.html
./experimental/D3/lib/slickgrid/examples/example12-fillbrowser.html
./experimental/D3/lib/slickgrid/examples/example13-getItem-sorting.html
./experimental/D3/lib/slickgrid/examples/example14-highlighting.html
./experimental/D3/lib/slickgrid/examples/example1-simple.html
./experimental/D3/lib/slickgrid/examples/example2-formatters.html
./experimental/D3/lib/slickgrid/examples/example3a-compound-editors.html
./experimental/D3/lib/slickgrid/examples/example3b-editing-with-undo.html
./experimental/D3/lib/slickgrid/examples/example3-editing.html
./experimental/D3/lib/slickgrid/examples/example4-model.html
./experimental/D3/lib/slickgrid/examples/example5-collapsing.html
./experimental/D3/lib/slickgrid/examples/example6-ajax-loading.html
./experimental/D3/lib/slickgrid/examples/example7-events.html
./experimental/D3/lib/slickgrid/examples/example8-alternative-display.html
./experimental/D3/lib/slickgrid/examples/example9-row-reordering.html
./experimental/D3/lib/slickgrid/examples/example-autotooltips.html
./experimental/D3/lib/slickgrid/examples/example-checkbox-row-select.html
./experimental/D3/lib/slickgrid/examples/example-colspan.html
./experimental/D3/lib/slickgrid/examples/example-composite-editor-item-details.html
./experimental/D3/lib/slickgrid/examples/example-custom-column-value-extractor.html
./experimental/D3/lib/slickgrid/examples/example-explicit-initialization.html
./experimental/D3/lib/slickgrid/examples/example-grouping.html
./experimental/D3/lib/slickgrid/examples/example-header-row.html
./experimental/D3/lib/slickgrid/examples/example-multi-column-sort.html
./experimental/D3/lib/slickgrid/examples/example-optimizing-dataview.html
./experimental/D3/lib/slickgrid/examples/example-plugin-headerbuttons.html
./experimental/D3/lib/slickgrid/examples/example-plugin-headermenu.html
./experimental/D3/lib/slickgrid/examples/examples.css
./experimental/D3/lib/slickgrid/examples/example-spreadsheet.html
./experimental/D3/lib/slickgrid/examples/example-totals-via-data-provider.html
./experimental/D3/lib/slickgrid/examples/index.html
./experimental/D3/lib/slickgrid/examples/slick.compositeeditor.js
./experimental/D3/lib/slickgrid/examples/slick-default-theme.css
./experimental/D3/lib/slickgrid/.gitignore
./experimental/D3/lib/slickgrid/images/actions.gif
./experimental/D3/lib/slickgrid/images/ajax-loader-small.gif
./experimental/D3/lib/slickgrid/images/arrow_redo.png
./experimental/D3/lib/slickgrid/images/arrow_right_peppermint.png
./experimental/D3/lib/slickgrid/images/arrow_right_spearmint.png
./experimental/D3/lib/slickgrid/images/arrow_undo.png
./experimental/D3/lib/slickgrid/images/bullet_blue.png
./experimental/D3/lib/slickgrid/images/bullet_star.png
./experimental/D3/lib/slickgrid/images/bullet_toggle_minus.png
./experimental/D3/lib/slickgrid/images/bullet_toggle_plus.png
./experimental/D3/lib/slickgrid/images/calendar.gif
./experimental/D3/lib/slickgrid/images/collapse.gif
./experimental/D3/lib/slickgrid/images/comment_yellow.gif
./experimental/D3/lib/slickgrid/images/down.gif
./experimental/D3/lib/slickgrid/images/drag-handle.png
./experimental/D3/lib/slickgrid/images/editor-helper-bg.gif
./experimental/D3/lib/slickgrid/images/expand.gif
./experimental/D3/lib/slickgrid/images/header-bg.gif
./experimental/D3/lib/slickgrid/images/header-columns-bg.gif
./experimental/D3/lib/slickgrid/images/header-columns-over-bg.gif
./experimental/D3/lib/slickgrid/images/help.png
./experimental/D3/lib/slickgrid/images/info.gif
./experimental/D3/lib/slickgrid/images/listview.gif
./experimental/D3/lib/slickgrid/images/pencil.gif
./experimental/D3/lib/slickgrid/images/row-over-bg.gif
./experimental/D3/lib/slickgrid/images/sort-asc.gif
./experimental/D3/lib/slickgrid/images/sort-asc.png
./experimental/D3/lib/slickgrid/images/sort-desc.gif
./experimental/D3/lib/slickgrid/images/sort-desc.png
./experimental/D3/lib/slickgrid/images/stripes.png
./experimental/D3/lib/slickgrid/images/tag_red.png
./experimental/D3/lib/slickgrid/images/tick.png
./experimental/D3/lib/slickgrid/images/user_identity.gif
./experimental/D3/lib/slickgrid/images/user_identity_plus.gif
./experimental/D3/lib/slickgrid/lib/firebugx.js
./experimental/D3/lib/slickgrid/lib/jquery-1.7.min.js
./experimental/D3/lib/slickgrid/lib/jquery.event.drag-2.2.js
./experimental/D3/lib/slickgrid/lib/jquery.event.drop-2.2.js
./experimental/D3/lib/slickgrid/lib/jquery.jsonp-2.4.min.js
./experimental/D3/lib/slickgrid/lib/jquery.simulate.js
./experimental/D3/lib/slickgrid/lib/jquery.sparkline.min.js
./experimental/D3/lib/slickgrid/lib/jquery-ui-1.8.16.custom.min.js
./experimental/D3/lib/slickgrid/lib/qunit.css
./experimental/D3/lib/slickgrid/lib/qunit.js
./experimental/D3/lib/slickgrid/MIT-LICENSE.txt
./experimental/D3/lib/slickgrid/plugins/slick.autotooltips.js
./experimental/D3/lib/slickgrid/plugins/slick.cellcopymanager.js
./experimental/D3/lib/slickgrid/plugins/slick.cellrangedecorator.js
./experimental/D3/lib/slickgrid/plugins/slick.cellrangeselector.js
./experimental/D3/lib/slickgrid/plugins/slick.cellselectionmodel.js
./experimental/D3/lib/slickgrid/plugins/slick.checkboxselectcolumn.js
./experimental/D3/lib/slickgrid/plugins/slick.headerbuttons.css
./experimental/D3/lib/slickgrid/plugins/slick.headerbuttons.js
./experimental/D3/lib/slickgrid/plugins/slick.headermenu.css
./experimental/D3/lib/slickgrid/plugins/slick.headermenu.js
./experimental/D3/lib/slickgrid/plugins/slick.rowmovemanager.js
./experimental/D3/lib/slickgrid/plugins/slick.rowselectionmodel.js
./experimental/D3/lib/slickgrid/README.md
./experimental/D3/lib/slickgrid/slick.core.js
./experimental/D3/lib/slickgrid/slick.dataview.js
./experimental/D3/lib/slickgrid/slick.editors.js
./experimental/D3/lib/slickgrid/slick.formatters.js
./experimental/D3/lib/slickgrid/slick.grid.css
./experimental/D3/lib/slickgrid/slick.grid.js
./experimental/D3/lib/slickgrid/slick.groupitemmetadataprovider.js
./experimental/D3/lib/slickgrid/slick.remotemodel.js
./experimental/D3/lib/slickgrid/tests/dataview/dataview.js
./experimental/D3/lib/slickgrid/tests/dataview/index.html
./experimental/D3/lib/slickgrid/tests/grid/grid.js
./experimental/D3/lib/slickgrid/tests/grid/index.html
./experimental/D3/lib/slickgrid/tests/index.html
./experimental/D3/lib/slickgrid/tests/plugins/autotooltips.html
./experimental/D3/lib/slickgrid/tests/plugins/autotooltips.js
./experimental/D3/lib/underscore-min.js
./experimental/exp
./experimental/vis/cluster_network.html
./experimental/vis/cluster_timeline.html
./experimental/vis/cluster_timeline.html.orig
./experimental/vis/futex_scatter.html
./experimental/vis/futex_scatter.html.rev2
./experimental/vis/kidsk.html
./experimental/vis/kidsk_scatter.html
./experimental/vis/kidsk_scatter.html.backup
./experimental/vis/kifile.html
./experimental/vis/kipid_io.html
./experimental/vis/kipid_sched.html
./experimental/vis/kirunq.html
./experimental/vis/kiwait.html
./experimental/vis/network.html
./experimental/vis/network.html.save
./experimental/vis/pid_detail.html
./experimental/vis/pid_switch.html
./experimental/vis/pid_switch_msr.awkpgm
./experimental/vis/pid_switch_steal.awkpgm
./experimental/vis/pid_switch_steal_msr.awkpgm
./experimental/vis/pid_wtree.html
./experimental/vis/sample_network.csv
./experimental/vis/server_network.html
./experimental/vis/server_timeline.html
./experimental/vis/socket.html
./experimental/vis/timeline.html
./src/kiinfo/compile

likit.o: warning: objtool

hostkernel=4.18.0-372.26.1.el8_6.x86_64
hostdistro=rhel
hostversion=8.6
EXTRA_CFLAGS=-g -O3 -Wall -DRHEL86 -DHARDCLOCK_STACK_SKIP=8 -DOTHER_STACK_SKIP=2
Building with KERNELRELEASE = 4.18.0-372.26.1.el8_6.x86_64
CC [M] /tmp/tmp.vXxPcLzG7j/likit.o
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: enable_msr()+0x8f: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: block_rq_issue_trace()+0x21: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: change_installed_traces()+0x16c: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: cpu_buf_read.part.0.isra.0()+0x395: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: syscall_exit_trace()+0x12c9: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: init_trace_state()+0x162: sibling call from callable instruction with modified stack frame
/tmp/tmp.vXxPcLzG7j/.tmp_likit.o: warning: objtool: block_rq_issue_trace.cold()+0xf: sibling call from callable instruction with modified stack frame

LinuxKI_MasterClass.pdf - 3.4 kiinfo – customizing reports

wrong example
kiinfo -kipid pid=27653 -ts 0815_2118 -start 4.0 -end 4.0 -->
kiinfo -kipid pid=27653 -ts 0815_2118 -start 4.0 -end 5.0

URL pointing to wrong section
Be sure to review the kiinfo reference pages in the LinuxKI reports and kiinforeference pages section.

9 LinuxKI Docker container for browser access -->
10 LinuxKI reports and kiinfo reference pages

Not really a BUG: Skipping BTF generation for /tmp/../likit.ko due to unavailability of vmlinux

Linux rhel9 5.14.0-70.26.1.el9_0.x86_64

The BTF stuff is new in RHEL 9.
The kernel config file now has CONFIG_DEBUG_INFO_BTF_MODULES enabled.
With this option set, module build process checks to see if vmlinux is available.
And if not, the "Skipping BTF generation ..." message is displayed.

ched_setscheduler(): Operation not permitted

/opt/linuxki/kiinfo -kidsk

Command line: /opt/linuxki/kiinfo -kidsk

kiinfo (7.3)

sched_setscheduler(): Operation not permitted
failed to make per-CPU threads realtime (possible due to selinux being enabled)
sched_setscheduler(): Operation not permitted
failed to make per-CPU threads realtime (possible due to selinux being enabled)

cat /etc/redhat-release

Red Hat Enterprise Linux release 8.5 (Ootpa)

uname -a

Linux genesis 4.18.0-348.12.2.el8_5.x86_64 #1 SMP Mon Jan 17 07:06:06 EST 2022 x86_64 x86_64 x86_64 GNU/Linux

getenforce

Disabled

Probable false positive/misleading description of detected SCA mitigations

Hi.
There is a message in kp.html that SCA mitigation is active on the host I currently diagnosing:

Side-Channel Attack (Spectre/Meltdown) Mitigations:
/sys/devices/system/cpu/vulnerabilities/gather_data_sampling Vulnerable
/sys/devices/system/cpu/vulnerabilities/itlb_multihit KVM: Mitigation: VMX disabled
/sys/devices/system/cpu/vulnerabilities/l1tf Mitigation: PTE Inversion; VMX: vulnerable
/sys/devices/system/cpu/vulnerabilities/mds Vulnerable; SMT vulnerable
/sys/devices/system/cpu/vulnerabilities/meltdown Vulnerable
/sys/devices/system/cpu/vulnerabilities/mmio_stale_data Vulnerable
/sys/devices/system/cpu/vulnerabilities/retbleed Vulnerable
/sys/devices/system/cpu/vulnerabilities/spec_rstack_overflow Not affected
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1 Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
/sys/devices/system/cpu/vulnerabilities/spectre_v2 Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
/sys/devices/system/cpu/vulnerabilities/srbds Not affected
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort Vulnerable
Warning: Side Channel Attack (Spectre/Meltdown) mitigations present [[INFO]](http://htmlpreview.github.io/?https://github.com/HewlettPackard/LinuxKI/blob/master/documentation/sca_vuln.htm)[[Next]](http://192.168.1.60:8000/kp.0930_1328.html#warncheck1)

But report sample above shows spectre_v1 and spectre_v2 as "Vulnerable" without "Mitigation:" clause. And that system is booted with mitigation=off kernel parameter.

So the question is: are SCA mitigations actually enabled? The CPI column in "Global CPU Usage by CPU" report is around 3.09 - 3.93 if that matters.

"Unable to load likit.ko module" as non-root user

I have set up sudo for non-root user to run kiinfo. But this user is getting this error about kernel module.
I worked around this by adding "/sbin" directory to the user's PATH. But probably it is better to use fully qualified path in likis.c
Or probe for insmod/rmmod location on startup.

Missing files needed to create packages

I forked the repository with the intent to fix issue #8 only to find that the .spec file is not included in the repository. I will generate a patch to scripts/linuxki_install to fix the problem but I believe that anything needed to create the RPM and DEB packages should be in the repository. Can you please add the missing .spec file and the files used to generate the scripts in the RPM to the repository?

insmod: ERROR: could not load module likit.ko: No such file or directory

why? ubuntu,centos and kali all fail

Power Savings report - please include cstate to cstate name mapping and latency in header

currently "Power Savings report" does not include information about the cstate name and latency
this information has to be looked up from cstates.*.txt.
It would be great if the Cstate name along with the cstate is reported. So it can look something like this:

cpu node    Events     Busy%   Cstate0   Cstate1   Cstate2   Cstate3   Cstate4   Cstate5 freq_changes    freq_hi   freq_low
                                           C1-SNB   C1E-SNB    C3-SNB    C6-SNB    C7-SNB
0    [ 0]     11673     1.67%     0.00%     3.30%    20.03%    15.06%     0.17%    59.77%            0          0          0
1    [ 0]      5291     8.11%     0.00%     1.19%    15.50%     5.70%     0.00%    69.49%           29    2600000    1200000
2    [ 0]      3282    15.21%     0.00%     0.49%    14.20%     3.81%     0.00%    66.30%           54    2600000    1200000
3    [ 0]      3829    29.94%     0.00%     0.58%    15.68%     8.42%     0.00%    45.38%           48    2600000    1200000

Current Report

cstate.*.txt

cstate    name        latency
1         C1          2   
2         C1E         10  
3         C6          92  

CPU    cstate0    cstate1    cstate2    cstate3    cstate4    cstate5    cstate6    cstate7    cstate8    cstate9
   0        373      13993     426082   99067486          0          0          0          0          0          0
   1        544      10066     468656  106865050          0          0          0          0          0          0

Thread names limited to 15 char?

seems like thread names are limited to 15char ..

hdbindexserver (FlushResourcesT)
hdbindexserver (SubmitThread-DA)
hdbindexserver (SubmitThread-LO)

Any option to overcome this limitation? I mean is this just a display issue or is it already collected with the 15c limitation?

Thanks

I/Os from nvme devices are not traced?

Hi Mark,

I am running LinuxKI on a kvm guest with passthrough nvme devices, I/Os are very active but I don't see any of them from nvme devices. Is this expected?

Best regards,
Wenbin

Creating visualizations takes an excessive amount of time

When running kiall -V, it takes a long time to generate all of the visualization files when the capture is for a machine which had a lot of threads running on it (23,906 in my case). Running kiall without the -V took ~4 minutes. Running it with -V took ~35 minutes on a machine with 40 cores (E5-2650 @ 2.30 GHz), 64 GB of memory, and all the processing running on an SSD. One core runs at 100% but rest of the cores are essentially idle. Top reported 91.3% idle CPU time and 0 wait time. This looks like a classic case of single thread processing being the bottleneck. The output from strace shows that the process is creating a file in PIDS, then creating the pid_detail.json and pid_wtree.json in VIS/<pid#>, then closing all 3 files and moving onto the next PID. I haven't dug through the code to see what is going on but couldn't the processing be done in multiple threads (i.e., different threads working on generating different different JSON files) to reduce the time needed to generate the visualizations? Is there a fundamental reason why this can't be done (e.g., writing to data structures which aren't multi-thread safe)?

linuxki traces from a core dump?

there was a nice functionality on HP-UX to extract some ki traces from a crash dump. Wondering if we could have something similar, like enable tracing and later dump traces from a core?

hewlettpackard / linuxki Goto Github PK

linuxki's People

Contributors

Stargazers

Watchers

Forkers

linuxki's Issues

rpm --install –nodeps linuxki-5.3-1.noarch.rpm

./module_prep

/opt/linuxki/kiinfo -kidsk

cat /etc/redhat-release

uname -a

getenforce

Recommend Projects

Recommend Topics

Recommend Org