Comments (39)
zendulkaj
commented on May 23, 2024
system log:
... 2021-03-17 12:35:41 charon: 09[CFG] vici terminate IKE_SA 'ipsec3' 2021-03-17 12:35:41 charon: 00[DMN] SIGTERM received, shutting down
process list (top):
... 21333 1 root S 136m 114.3 0.0 /usr/libexec/ipsec/charon --use-syslog 21908 21896 root S 39760 32.5 0.0 /usr/libexec/ipsec/swanctl --terminate --ike ipsec3 --force 21922 21916 root S 38736 31.6 0.0 /usr/libexec/ipsec/swanctl --stats ...
from strongswan.
Thermi
commented on May 23, 2024
Hi,
Do you have a complete stack trace of us with all threads, preferably all the binaries and a core dump?
from strongswan.
zendulkaj
commented on May 23, 2024
I sent signal SIGSEGV to charon when the charon stucks. But not sure if it is enough.
Kernel log:
2021-03-18 10:31:45 potentially unexpected fatal signal 6. 2021-03-18 10:31:45 CPU: 0 PID: 20901 Comm: charon Not tainted 4.14.138 #1 2021-03-18 10:31:45 Hardware name: Microchip SAM9X60 2021-03-18 10:31:45 task: c7992040 task.stack: c7294000 2021-03-18 10:31:45 PC is at 0xb6bb0760 2021-03-18 10:31:45 LR is at 0xffffffff 2021-03-18 10:31:45 pc : [b6bb0760] lr : [ffffffff] psr: 00000010 2021-03-18 10:31:45 sp : beb43158 ip : 00000020 fp : 00000000 2021-03-18 10:31:45 r10: 0083b388 r9 : 00000000 r8 : 00000004 2021-03-18 10:31:45 r7 : 000000af r6 : b6cca038 r5 : beb43158 r4 : 00000000 2021-03-18 10:31:45 r3 : 00000008 r2 : 00000000 r1 : beb43158 r0 : 00000000 2021-03-18 10:31:45 Flags: nzcv IRQs on FIQs on Mode USER_32 ISA ARM Segment user 2021-03-18 10:31:45 Control: 0005317f Table: 2732c000 DAC: 00000055 2021-03-18 10:31:45 CPU: 0 PID: 20901 Comm: charon Not tainted 4.14.138 #1 2021-03-18 10:31:45 Hardware name: Microchip SAM9X60 2021-03-18 10:31:45 Backtrace: 2021-03-18 10:31:45 [c0105114] (dump_backtrace) from [] (show_stack+0x18/0x1c) 2021-03-18 10:31:45 r7:c7295ed0 r6:400004d8 r5:ffffe000 r4:c7295fb0 2021-03-18 10:31:45 [c01053c0] (show_stack) from [c05cf69c] (dump_stack+0x20/0x28) 2021-03-18 10:31:45 [c05cf67c] (dump_stack) from [c0103138] (show_regs+0x14/0x18) 2021-03-18 10:31:45 [c0103124] (show_regs) from [c011a384] (get_signal+0x574/0x698) 2021-03-18 10:31:45 [c0119e10] (get_signal) from [c0104828] (do_signal+0xbc/0x3bc) 2021-03-18 10:31:45 r10:00000000 r9:c7294000 r8:b6bb0760 r7:00000000 r6:c7295ed0 r5:b6bb075c 2021-03-18 10:31:45 r4:c7295fb0
Charon and swanctl binaries:
binaries.zip
from strongswan.
zendulkaj
commented on May 23, 2024
I compiled charon with CFLAGS -O0 -g charon.zip and I attached GDB when issue appears. I got this information from GDB:
3643 root 145m S /usr/libexec/ipsec/charon --use-syslog --debug-dmn 1 --debug-mgr 1 --debug-ike 1 --debug-chd 1 --debug-job 1 --debug-cfg 1 --debug-knl 1 --debug-net 1 --debug-as
3749 root 39768 S /usr/libexec/ipsec/swanctl --stats
3767 root 2576 R ps
...
# gdb --pid 3643
GNU gdb (GDB) 7.5
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux".
For bug reporting instructions, please see:
.
Attaching to process 3643
Reading symbols from /usr/libexec/ipsec/charon...done.
Reading symbols from /usr/lib/ipsec/libstrongswan.so.0...done.
Loaded symbols for /usr/lib/ipsec/libstrongswan.so.0
Reading symbols from /usr/lib/ipsec/libcharon.so.0...done.
Loaded symbols for /usr/lib/ipsec/libcharon.so.0
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.3
Reading symbols from /usr/lib/libcrypto.so.1.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libcrypto.so.1.1
Reading symbols from /usr/lib/ipsec/libtpmtss.so.0...done.
Loaded symbols for /usr/lib/ipsec/libtpmtss.so.0
Reading symbols from /usr/lib/libtss2-sys.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libtss2-sys.so.1
Reading symbols from /usr/lib/libtss2-mu.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libtss2-mu.so.0
Reading symbols from /lib/libnss_dns.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
0xb6d340c8 in pthread_cond_wait () from /lib/libpthread.so.0
(gdb) info stack
#0 0xb6d340c8 in pthread_cond_wait () from /lib/libpthread.so.0
#1 0xb6efb09c in wait_ (this=0x1870778, mutex=0x18706f8) at threading/mutex.c:237
#2 0xb6ee76f0 in cancel (this=0x1870620) at processing/processor.c:501
#3 0xb6e03c60 in destroy (this=0x1871150) at daemon.c:719
#4 0xb6e04b30 in libcharon_deinit () at daemon.c:970
#5 0x000121d8 in main (argc=36, argv=0xbe8edd34) at charon.c:469
(gdb) info frame
Stack level 0, frame at 0xbe8ed8d0:
pc = 0xb6d340c8 in pthread_cond_wait; saved pc 0xb6efb09c
called by frame at 0xbe8ed8f8
Arglist at 0xbe8ed850, args:
Locals at 0xbe8ed850, Previous frame's sp is 0xbe8ed8d0
Saved registers:
r4 at 0xbe8ed8ac, r5 at 0xbe8ed8b0, r6 at 0xbe8ed8b4, r7 at 0xbe8ed8b8, r8 at 0xbe8ed8bc, r9 at 0xbe8ed8c0, r10 at 0xbe8ed8c4, r11 at 0xbe8ed8c8, lr at 0xbe8ed8cc
(gdb) info threads
Id Target Id Frame
* 1 process 3643 "charon" 0xb6d340c8 in pthread_cond_wait () from /lib/libpthread.so.0
(gdb) bt full
#0 0xb6d340c8 in pthread_cond_wait () from /lib/libpthread.so.0
No symbol table info available.
#1 0xb6efb09c in wait_ (this=0x1870778, mutex=0x18706f8) at threading/mutex.c:237
No locals.
#2 0xb6ee76f0 in cancel (this=0x1870620) at processing/processor.c:501
enumerator = 0x1889230
worker = 0x1892a98
job = 0xb6eab000
#3 0xb6e03c60 in destroy (this=0x1871150) at daemon.c:719
No locals.
#4 0xb6e04b30 in libcharon_deinit () at daemon.c:970
this = 0x1871150
#5 0x000121d8 in main (argc=36, argv=0xbe8edd34) at charon.c:469
action = {__sigaction_handler = {sa_handler = 0x1, sa_sigaction = 0x1}, sa_mask = {__val = {16387, 0 }}, sa_flags = 0, sa_restorer = 0x0}
group = 17
status = 0
utsname = {sysname = "Linux", '\000' , nodename = "Router", '\000' , release = "4.14.138", '\000' ,
version = "#1 custom", '\000' , machine = "armv5tejl", '\000' , __domainname = "(none)", '\000' }
levels = {LEVEL_CTRL }
use_syslog = true
(gdb)
Many thanks.
from strongswan.
zendulkaj
commented on May 23, 2024
Any suggestion? Do you think that the problem is in pthread library not in strongswan?
from strongswan.
Thermi
commented on May 23, 2024
Hi,
I didn't forget you. I just didn't come around to take a look at it.
Am March 22, 2021 9:26:35 AM UTC schrieb Jiri Zendulka ***@***.***>:
…Any suggestion? Do you think that the problem is in pthread library not
in strongswan?
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#268 (comment)
--
Sent from mobile
from strongswan.
Thermi
commented on May 23, 2024
Please provide logs.
By default, charon tries to correctly destroy all IKE_SAs and CHILD_SAs it has when it is told to shut down. That is subject to the timeout parameters. The default is the standard, which is quite long. For anything using TCP that isn't a problem, because the kernel can handle terminating the connections. For IKE that's not possible. So the daemon has to keep running until the timeouts are reached. If you don't want that, I got a scrip that you can set to execute when the daemon is told to stop, that terminates all IKE SAs without timeout. It's here: https://github.com/Thermi/strongswan-scripts
The file is connectionCloser.py. Read the README.md, please.
from strongswan.
zendulkaj
commented on May 23, 2024
I think that I do the same when I call swanctl --terminate --ike "ike name" --force. So I think that IKE is terminated succesfully. Charon stucks at at this line processing/processor.c:501 untill device reboots so I guess that is an issue with threads.
Log when issue happens:
2021-02-05 05:36:19 charon: 13[CFG] vici terminate IKE_SA 'ipsec1' 2021-02-05 05:36:19 charon: 15[IKE] destroying IKE_SA in state CONNECTING without notification 2021-02-05 05:36:19 charon: 00[DMN] SIGTERM received, shutting down
from strongswan.
Thermi
commented on May 23, 2024
Double check if there are indeed no connections. Also, logs with debug settings will help.
#196
Expand "Log Config Snippet" for config snippets for usable debug logging settings.
from strongswan.
zendulkaj
commented on May 23, 2024
Not able to connect to charon via swanctl so I cannot check if any connections exists.
Debug log:
Fri, 2021-02-05, 10:13:46 00[DMN] Starting IKE charon daemon (strongSwan 5.9.2, Linux 4.14.138, armv5tejl) Fri, 2021-02-05, 10:13:46 00[PTS] TPM 2.0 - "/dev/tpmrm0" in-kernel resource manager is not present Fri, 2021-02-05, 10:13:46 00[PTS] TPM 2.0 - could not load "libtss2-tcti-tabrmd.so.0" Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'tpm': failed to load - tpm_plugin_create returned NULL Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'nonce': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'pubkey': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'pkcs1': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'pem': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'openssl': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'kernel-netlink': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'socket-default': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'vici': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'updown': loaded successfully Fri, 2021-02-05, 10:13:46 00[LIB] plugin 'xauth-generic': loaded successfully Fri, 2021-02-05, 10:13:46 00[KNL] known interfaces and IP addresses: Fri, 2021-02-05, 10:13:46 00[KNL] lo Fri, 2021-02-05, 10:13:46 00[KNL] 127.0.0.1 Fri, 2021-02-05, 10:13:46 00[KNL] ::1 Fri, 2021-02-05, 10:13:46 00[KNL] eth0 Fri, 2021-02-05, 10:13:46 00[KNL] 10.10.0.1 Fri, 2021-02-05, 10:13:46 00[KNL] eth1 Fri, 2021-02-05, 10:13:46 00[KNL] 192.168.7.243 Fri, 2021-02-05, 10:13:46 00[LIB] feature PUBKEY:DSA in plugin 'pem' has unmet dependency: PUBKEY:DSA Fri, 2021-02-05, 10:13:46 00[LIB] feature PUBKEY:BLISS in plugin 'pem' has unmet dependency: PUBKEY:BLISS Fri, 2021-02-05, 10:13:46 00[LIB] feature PRIVKEY:DSA in plugin 'pem' has unmet dependency: PRIVKEY:DSA Fri, 2021-02-05, 10:13:46 00[LIB] feature PRIVKEY:BLISS in plugin 'pem' has unmet dependency: PRIVKEY:BLISS Fri, 2021-02-05, 10:13:46 00[LIB] feature CERT_DECODE:PGP in plugin 'pem' has unmet dependency: CERT_DECODE:PGP Fri, 2021-02-05, 10:13:46 00[LIB] feature CERT_DECODE:OCSP_REQUEST in plugin 'pem' has unmet dependency: CERT_DECODE:OCSP_REQUEST Fri, 2021-02-05, 10:13:46 00[LIB] feature CERT_DECODE:OCSP_RESPONSE in plugin 'pem' has unmet dependency: CERT_DECODE:OCSP_RESPONSE Fri, 2021-02-05, 10:13:46 00[LIB] feature CERT_DECODE:X509_AC in plugin 'pem' has unmet dependency: CERT_DECODE:X509_AC Fri, 2021-02-05, 10:13:46 00[LIB] feature CERT_DECODE:PKCS10_REQUEST in plugin 'pem' has unmet dependency: CERT_DECODE:PKCS10_REQUEST Fri, 2021-02-05, 10:13:46 00[LIB] loaded plugins: charon nonce pubkey pkcs1 pem openssl kernel-netlink socket-default vici updown xauth-generic Fri, 2021-02-05, 10:13:46 00[LIB] unable to load 9 plugin features (9 due to unmet dependencies) Fri, 2021-02-05, 10:13:46 00[JOB] spawning 16 worker threads Fri, 2021-02-05, 10:13:46 01[LIB] created thread 01 [29003] Fri, 2021-02-05, 10:13:46 02[LIB] created thread 02 [29004] Fri, 2021-02-05, 10:13:46 03[LIB] created thread 03 [28995] Fri, 2021-02-05, 10:13:46 04[LIB] created thread 04 [28996] Fri, 2021-02-05, 10:13:46 05[LIB] created thread 05 [29000] Fri, 2021-02-05, 10:13:46 06[LIB] created thread 06 [28997] Fri, 2021-02-05, 10:13:46 07[LIB] created thread 07 [28998] Fri, 2021-02-05, 10:13:46 08[LIB] created thread 08 [28999] Fri, 2021-02-05, 10:13:46 09[LIB] created thread 09 [29001] Fri, 2021-02-05, 10:13:46 10[LIB] created thread 10 [29002] Fri, 2021-02-05, 10:13:46 11[LIB] created thread 11 [28993] Fri, 2021-02-05, 10:13:46 12[LIB] created thread 12 [28994] Fri, 2021-02-05, 10:13:46 13[LIB] created thread 13 [28992] Fri, 2021-02-05, 10:13:46 14[LIB] created thread 14 [28991] Fri, 2021-02-05, 10:13:46 15[LIB] created thread 15 [28989] Fri, 2021-02-05, 10:13:46 16[LIB] created thread 16 [28990] Fri, 2021-02-05, 10:13:58 05[CFG] vici client 1 connected Fri, 2021-02-05, 10:13:58 07[CFG] vici client 1 requests: get-keys Fri, 2021-02-05, 10:13:58 11[CFG] vici client 1 requests: get-shared Fri, 2021-02-05, 10:13:58 11[CFG] vici client 1 requests: load-shared Fri, 2021-02-05, 10:13:58 11[CFG] loaded IKE shared key with id 'ike-1' for: '243', '120' Fri, 2021-02-05, 10:13:58 11[CFG] vici client 1 requests: get-authorities Fri, 2021-02-05, 10:13:58 13[CFG] vici client 1 requests: get-pools Fri, 2021-02-05, 10:13:58 08[CFG] vici client 1 requests: get-conns Fri, 2021-02-05, 10:13:58 16[CFG] vici client 1 requests: load-conn Fri, 2021-02-05, 10:13:58 16[CFG] conn ipsec1: Fri, 2021-02-05, 10:13:58 16[CFG] child ipsec1: Fri, 2021-02-05, 10:13:58 16[CFG] rekey_time = 3060 Fri, 2021-02-05, 10:13:58 16[CFG] life_time = 3600 Fri, 2021-02-05, 10:13:58 16[CFG] rand_time = 540 Fri, 2021-02-05, 10:13:58 16[CFG] rekey_bytes = 0 Fri, 2021-02-05, 10:13:58 16[CFG] life_bytes = 0 Fri, 2021-02-05, 10:13:58 16[CFG] rand_bytes = 0 Fri, 2021-02-05, 10:13:58 16[CFG] rekey_packets = 0 Fri, 2021-02-05, 10:13:58 16[CFG] life_packets = 0 Fri, 2021-02-05, 10:13:58 16[CFG] rand_packets = 0 Fri, 2021-02-05, 10:13:58 16[CFG] updown = /etc/scripts/updown Fri, 2021-02-05, 10:13:58 16[CFG] hostaccess = 0 Fri, 2021-02-05, 10:13:58 16[CFG] ipcomp = 0 Fri, 2021-02-05, 10:13:58 16[CFG] mode = TUNNEL Fri, 2021-02-05, 10:13:58 16[CFG] policies = 1 Fri, 2021-02-05, 10:13:58 16[CFG] policies_fwd_out = 0 Fri, 2021-02-05, 10:13:58 16[CFG] dpd_action = clear Fri, 2021-02-05, 10:13:58 16[CFG] start_action = restart Fri, 2021-02-05, 10:13:58 16[CFG] close_action = clear Fri, 2021-02-05, 10:13:58 16[CFG] reqid = 0 Fri, 2021-02-05, 10:13:58 16[CFG] tfc = 0 Fri, 2021-02-05, 10:13:58 16[CFG] priority = 0 Fri, 2021-02-05, 10:13:58 16[CFG] interface = (null) Fri, 2021-02-05, 10:13:58 16[CFG] if_id_in = 0 Fri, 2021-02-05, 10:13:58 16[CFG] if_id_out = 0 Fri, 2021-02-05, 10:13:58 16[CFG] mark_in = 0/0 Fri, 2021-02-05, 10:13:58 16[CFG] mark_in_sa = 0 Fri, 2021-02-05, 10:13:58 16[CFG] mark_out = 0/0 Fri, 2021-02-05, 10:13:58 16[CFG] set_mark_in = 0/0 Fri, 2021-02-05, 10:13:58 16[CFG] set_mark_out = 0/0 Fri, 2021-02-05, 10:13:58 16[CFG] inactivity = 0 Fri, 2021-02-05, 10:13:58 16[CFG] proposals = ESP:AES_CBC_128/HMAC_SHA1_96/NO_EXT_SEQ, ESP:3DES_CBC/HMAC_SHA1_96/NO_EXT_SEQ Fri, 2021-02-05, 10:13:58 16[CFG] local_ts = 10.10.0.0/24 Fri, 2021-02-05, 10:13:58 16[CFG] remote_ts = 172.24.0.0/24 Fri, 2021-02-05, 10:13:58 16[CFG] hw_offload = no Fri, 2021-02-05, 10:13:58 16[CFG] sha256_96 = 0 Fri, 2021-02-05, 10:13:58 16[CFG] copy_df = 1 Fri, 2021-02-05, 10:13:58 16[CFG] copy_ecn = 1 Fri, 2021-02-05, 10:13:58 16[CFG] copy_dscp = out Fri, 2021-02-05, 10:13:58 16[CFG] version = 2 Fri, 2021-02-05, 10:13:58 16[CFG] local_addrs = 0.0.0.0 Fri, 2021-02-05, 10:13:58 16[CFG] remote_addrs = coneltest.spdns.org Fri, 2021-02-05, 10:13:58 16[CFG] local_port = 500 Fri, 2021-02-05, 10:13:58 16[CFG] remote_port = 500 Fri, 2021-02-05, 10:13:58 16[CFG] send_certreq = 1 Fri, 2021-02-05, 10:13:58 16[CFG] send_cert = CERT_SEND_IF_ASKED Fri, 2021-02-05, 10:13:58 16[CFG] ppk_id = (null) Fri, 2021-02-05, 10:13:58 16[CFG] ppk_required = 0 Fri, 2021-02-05, 10:13:58 16[CFG] mobike = 1 Fri, 2021-02-05, 10:13:58 16[CFG] aggressive = 0 Fri, 2021-02-05, 10:13:58 16[CFG] dscp = 0x00 Fri, 2021-02-05, 10:13:58 16[CFG] encap = 0 Fri, 2021-02-05, 10:13:58 16[CFG] dpd_delay = 0 Fri, 2021-02-05, 10:13:58 16[CFG] dpd_timeout = 0 Fri, 2021-02-05, 10:13:58 16[CFG] fragmentation = 2 Fri, 2021-02-05, 10:13:58 16[CFG] childless = 0 Fri, 2021-02-05, 10:13:58 16[CFG] unique = UNIQUE_REPLACE Fri, 2021-02-05, 10:13:58 16[CFG] keyingtries = 0 Fri, 2021-02-05, 10:13:58 16[CFG] reauth_time = 3060 Fri, 2021-02-05, 10:13:58 16[CFG] rekey_time = 0 Fri, 2021-02-05, 10:13:58 16[CFG] over_time = 540 Fri, 2021-02-05, 10:13:58 16[CFG] rand_time = 540 Fri, 2021-02-05, 10:13:58 16[CFG] proposals = IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_3072, IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA 1/MODP_2048, IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1536 Fri, 2021-02-05, 10:13:58 16[CFG] if_id_in = 0 Fri, 2021-02-05, 10:13:58 16[CFG] if_id_out = 0 Fri, 2021-02-05, 10:13:58 16[CFG] local: Fri, 2021-02-05, 10:13:58 16[CFG] class = pre-shared key Fri, 2021-02-05, 10:13:58 16[CFG] id = 243 Fri, 2021-02-05, 10:13:58 16[CFG] remote: Fri, 2021-02-05, 10:13:58 16[CFG] class = pre-shared key Fri, 2021-02-05, 10:13:58 16[CFG] id = 120 Fri, 2021-02-05, 10:13:58 16[CFG] added vici connection: ipsec1 Fri, 2021-02-05, 10:13:58 16[CFG] initiating 'ipsec1' Fri, 2021-02-05, 10:13:58 17[LIB] created thread 17 [29082] Fri, 2021-02-05, 10:13:58 16[KNL] using 192.168.7.243 as address to reach 89.24.1.89/32 Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_VENDOR task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_INIT task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_NATD task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_CERT_PRE task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_AUTH task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_CERT_POST task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_CONFIG task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_AUTH_LIFETIME task Fri, 2021-02-05, 10:13:58 16[IKE] queueing IKE_MOBIKE task Fri, 2021-02-05, 10:13:58 16[IKE] queueing CHILD_CREATE task Fri, 2021-02-05, 10:13:58 16[IKE] activating new tasks Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_VENDOR task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_INIT task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_NATD task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_CERT_PRE task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_AUTH task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_CERT_POST task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_CONFIG task Fri, 2021-02-05, 10:13:58 16[IKE] activating CHILD_CREATE task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_AUTH_LIFETIME task Fri, 2021-02-05, 10:13:58 16[IKE] activating IKE_MOBIKE task Fri, 2021-02-05, 10:13:58 16[IKE] sending strongSwan vendor ID Fri, 2021-02-05, 10:13:58 16[IKE] initiating IKE_SA ipsec1[1] to 89.24.1.89 Fri, 2021-02-05, 10:13:58 16[IKE] IKE_SA ipsec1[1] state change: CREATED => CONNECTING Fri, 2021-02-05, 10:14:00 16[LIB] size of DH secret exponent: 384 bits Fri, 2021-02-05, 10:14:00 16[CFG] configured proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_3072, IKE:AES_CBC_128/HMAC_S HA1_96/PRF_HMAC_SHA1/MODP_2048, IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1536 Fri, 2021-02-05, 10:14:00 16[CFG] sending supported signature hash algorithms: sha256 sha384 sha512 identity Fri, 2021-02-05, 10:14:00 16[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) V ] Fri, 2021-02-05, 10:14:00 16[NET] sending packet: from 192.168.7.243[500] to 89.24.1.89[500] (696 bytes) Fri, 2021-02-05, 10:14:00 09[CFG] vici client 2 connected Fri, 2021-02-05, 10:14:00 15[CFG] vici client 2 registered for: control-log Fri, 2021-02-05, 10:14:00 08[CFG] vici client 2 requests: terminate Fri, 2021-02-05, 10:14:00 08[CFG] vici terminate IKE_SA 'ipsec1' Fri, 2021-02-05, 10:14:00 09[IKE] destroying IKE_SA in state CONNECTING without notification Fri, 2021-02-05, 10:14:00 09[IKE] IKE_SA ipsec1[1] state change: CONNECTING => DESTROYING Fri, 2021-02-05, 10:14:00 14[CFG] vici client 2 disconnected Fri, 2021-02-05, 10:14:01 00[DMN] SIGTERM received, shutting down
from strongswan.
Thermi
commented on May 23, 2024
Hi,
Thank you for the log.
What happens then, after you sent sigterm? The behaviour with the timeout as described earlier still applies.
from strongswan.
zendulkaj
commented on May 23, 2024
if I send "kill -s SIGTERM " then nothing happens. Charon process still exists. No messages in debug-log.
from strongswan.
Thermi
commented on May 23, 2024
When you send SIGTERM, is that after you already tried to stop the daemon that way or another way?
from strongswan.
zendulkaj
commented on May 23, 2024
yes, that way. Daemon was stopped by SIGTERM before.
from strongswan.
Thermi
commented on May 23, 2024
Then that's normal. The main thread that sends those messages to the logger waits for all worker threads to join.
from strongswan.
zendulkaj
commented on May 23, 2024
So is it normal that daemon stucks (only reboot helps)? I dont think so becuase normally daemon is stopped correctly. It happens only sometime that the daemon stucks in this way.
from strongswan.
zendulkaj
commented on May 23, 2024
Can I found out somehow which thread blocks the deamon?
from strongswan.
zendulkaj
commented on May 23, 2024
OK, I try runnig our tests with connectionCloser.py script as you recommended.
BTW I think there is a typo (double "kill-tunnels =" ) at https://github.com/Thermi/strongswan-scripts
charon {
stop-scripts {
kill-tunnels = kill-tunnels = /etc/swanctl/connectionCloser.py
}
from strongswan.
zendulkaj
commented on May 23, 2024
Charon got stuck anyway:
Fri, 2021-02-05, 07:48:10 10[IKE] IKE_SA ipsec1[1] state change: CREATED => CONNECTING Fri, 2021-02-05, 07:48:10 10[LIB] size of DH secret exponent: 384 bits Fri, 2021-02-05, 07:48:10 10[CFG] configured proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_3072, IK E:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048, IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1536 Fri, 2021-02-05, 07:48:10 10[CFG] sending supported signature hash algorithms: sha256 sha384 sha512 identity Fri, 2021-02-05, 07:48:10 10[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HAS H_ALG) N(REDIR_SUP) V ] Fri, 2021-02-05, 07:48:10 10[NET] sending packet: from 192.168.7.243[500] to 89.24.0.175[500] (696 bytes) Fri, 2021-02-05, 07:48:10 16[CFG] vici client 2 connected Fri, 2021-02-05, 07:48:10 13[CFG] vici client 2 registered for: control-log Fri, 2021-02-05, 07:48:10 10[CFG] vici client 2 requests: terminate Fri, 2021-02-05, 07:48:10 10[CFG] vici terminate IKE_SA 'ipsec1' Fri, 2021-02-05, 07:48:10 15[IKE] destroying IKE_SA in state CONNECTING without notification Fri, 2021-02-05, 07:48:10 15[IKE] IKE_SA ipsec1[1] state change: CONNECTING => DESTROYING Fri, 2021-02-05, 07:48:10 12[CFG] vici client 2 disconnected Fri, 2021-02-05, 07:48:10 00[DMN] SIGTERM received, shutting down Fri, 2021-02-05, 07:48:10 00[DMN] executing stop script 'kill-tunnels' (python3 /opt/ipsec_scripts/connectionCloser.py) Fri, 2021-02-05, 07:48:11 14[CFG] vici client 3 connected Fri, 2021-02-05, 07:48:11 07[CFG] vici client 3 registered for: control-log Fri, 2021-02-05, 07:48:11 16[CFG] vici client 3 requests: terminate Fri, 2021-02-05, 07:48:11 16[CFG] vici terminate IKE_SA 'ipsec1' Fri, 2021-02-05, 07:48:11 08[CFG] vici client 3 disconnected Fri, 2021-02-05, 07:48:11 13[CFG] vici client 4 connected Fri, 2021-02-05, 07:48:11 14[CFG] vici client 4 requests: get-conns Fri, 2021-02-05, 07:48:11 14[CFG] vici client 4 requests: unload-conn Fri, 2021-02-05, 07:48:11 14[CFG] vici client 4 registered for: list-sa Fri, 2021-02-05, 07:48:11 11[CFG] vici client 4 requests: list-sas Fri, 2021-02-05, 07:48:11 07[CFG] vici client 4 unregistered for: list-sa Fri, 2021-02-05, 07:48:11 00[DMN] kill-tunnels: connectionCloser started Fri, 2021-02-05, 07:48:11 00[DMN] kill-tunnels: connectionCloser finished Fri, 2021-02-05, 07:48:11 12[CFG] vici client 4 disconnected
from strongswan.
Thermi
commented on May 23, 2024
Thank you for testing. Can you make a debug build with -g3, reproduce the problem, attach to the daemon via gdb, make a a crash dump (not just a stack trace) of the whole process, and send that to me with all files related to strongSwan and the libs it loads (visible in /proc/PIDOFCHARONGOESHERE/maps)? That'd be what I need to at the very least try to understand that.
Logs made using the logger config from #196 will be quite helpful, too!
EDIT: Also, if you made any changes to the source code, I need those, too.
from strongswan.
zendulkaj
commented on May 23, 2024
-
libs and bins with - g3
charon_bins_libs.zip -
coredump (gdb gcore)
coredump.zip -
our patches
patches.zip
ipsec1 {
local_addrs = 0.0.0.0
remote_addrs = x.x.x.x
local {
id = 243
auth = psk
}
remote {
id = 120
auth = psk
}
children {
ipsec1 {
local_ts = 10.10.0.0/24
remote_ts = 172.24.0.1/24
mode = tunnel
updown = /etc/scripts/updown
life_time = 3600
rekey_time = 3060
rand_time = 540
esp_proposals = aes128-sha1,3des-sha1
start_action = start
}
}
unique = replace
version = 2
reauth_time = 3060
rekey_time = 0
over_time = 540
rand_time = 540
keyingtries = 0
proposals = aes128-sha256-modp3072,aes128-sha1-modp2048,3des-sha1-modp1536
}
# cat /etc/strongswan.conf
charon {
dh_exponent_ansi_x9_42 = no
i_dont_care_about_security_and_use_aggressive_mode_psk = yes
interfaces_ignore = nat64
retry_initiate_interval = 60
ignore_acquire_ts = yes
send_vendor_id = yes
plugins {
}
stop-scripts {
kill-tunnels = python3 /opt/ipsec_scripts/connectionCloser.py
}
filelog {
charon-debug-log {
path = /var/log/charon_debug.log
time_format = %a, %Y-%m-%d, %H:%M:%S
default = 2
mgr = 0
net = 1
enc = 1
asn = 1
job = 1
ike_name = yes
append = no
flush_line = yes
}
}
}
#
# cat /proc/12857/maps 00010000-00013000 r-xp 00000000 1f:08 782 /usr/libexec/ipsec/charon 00022000-00023000 r-xp 00002000 1f:08 782 /usr/libexec/ipsec/charon 00023000-00024000 rwxp 00003000 1f:08 782 /usr/libexec/ipsec/charon 00ce0000-00d22000 rwxp 00000000 00:00 0 [heap] adcff000-add00000 ---p 00000000 00:00 0 add00000-ae500000 rwxp 00000000 00:00 0 ae500000-ae521000 rwxp 00000000 00:00 0 ae521000-ae600000 ---p 00000000 00:00 0 ae700000-ae721000 rwxp 00000000 00:00 0 ae721000-ae800000 ---p 00000000 00:00 0 ae810000-ae82e000 r-xp 00000000 1f:08 508 /lib/libgcc_s.so.1 ae82e000-ae83e000 ---p 0001e000 1f:08 508 /lib/libgcc_s.so.1 ae83e000-ae83f000 rwxp 0001e000 1f:08 508 /lib/libgcc_s.so.1 ae83f000-ae84f000 r-xp 00000000 1f:08 530 /lib/libresolv-2.30.so ae84f000-ae85f000 ---p 00010000 1f:08 530 /lib/libresolv-2.30.so ae85f000-ae860000 r-xp 00010000 1f:08 530 /lib/libresolv-2.30.so ae860000-ae861000 rwxp 00011000 1f:08 530 /lib/libresolv-2.30.so ae861000-ae863000 rwxp 00000000 00:00 0 ae863000-ae867000 r-xp 00000000 1f:08 515 /lib/libnss_dns-2.30.so ae867000-ae876000 ---p 00004000 1f:08 515 /lib/libnss_dns-2.30.so ae876000-ae877000 r-xp 00003000 1f:08 515 /lib/libnss_dns-2.30.so ae877000-ae878000 rwxp 00004000 1f:08 515 /lib/libnss_dns-2.30.so ae878000-ae879000 ---p 00000000 00:00 0 ae879000-af079000 rwxp 00000000 00:00 0 af079000-af07a000 ---p 00000000 00:00 0 af07a000-af87a000 rwxp 00000000 00:00 0 af87a000-af87b000 ---p 00000000 00:00 0 af87b000-b007b000 rwxp 00000000 00:00 0 b007b000-b007c000 ---p 00000000 00:00 0 b007c000-b087c000 rwxp 00000000 00:00 0 b087c000-b087d000 ---p 00000000 00:00 0 b087d000-b107d000 rwxp 00000000 00:00 0 b107d000-b107e000 ---p 00000000 00:00 0 b107e000-b187e000 rwxp 00000000 00:00 0 b187e000-b187f000 ---p 00000000 00:00 0 b187f000-b207f000 rwxp 00000000 00:00 0 b207f000-b2080000 ---p 00000000 00:00 0 b2080000-b2880000 rwxp 00000000 00:00 0 b2880000-b2881000 ---p 00000000 00:00 0 b2881000-b3081000 rwxp 00000000 00:00 0 b3081000-b3082000 ---p 00000000 00:00 0 b3082000-b3882000 rwxp 00000000 00:00 0 b3882000-b3883000 ---p 00000000 00:00 0 b3883000-b4083000 rwxp 00000000 00:00 0 b4083000-b4084000 ---p 00000000 00:00 0 b4084000-b4884000 rwxp 00000000 00:00 0 b4884000-b4885000 ---p 00000000 00:00 0 b4885000-b5085000 rwxp 00000000 00:00 0 b5085000-b5086000 ---p 00000000 00:00 0 b5086000-b5886000 rwxp 00000000 00:00 0 b5886000-b5887000 ---p 00000000 00:00 0 b5887000-b6087000 rwxp 00000000 00:00 0 b6087000-b6088000 ---p 00000000 00:00 0 b6088000-b6888000 rwxp 00000000 00:00 0 b6888000-b68ca000 r-xp 00000000 1f:08 701 /usr/lib/libtss2-mu.so.0.0.0 b68ca000-b68d9000 ---p 00042000 1f:08 701 /usr/lib/libtss2-mu.so.0.0.0 b68d9000-b68da000 r-xp 00041000 1f:08 701 /usr/lib/libtss2-mu.so.0.0.0 b68da000-b68db000 rwxp 00042000 1f:08 701 /usr/lib/libtss2-mu.so.0.0.0 b68db000-b68f6000 r-xp 00000000 1f:08 707 /usr/lib/libtss2-sys.so.1.0.0 b68f6000-b6905000 ---p 0001b000 1f:08 707 /usr/lib/libtss2-sys.so.1.0.0 b6905000-b6906000 r-xp 0001a000 1f:08 707 /usr/lib/libtss2-sys.so.1.0.0 b6906000-b6907000 rwxp 0001b000 1f:08 707 /usr/lib/libtss2-sys.so.1.0.0 b6907000-b6913000 r-xp 00000000 1f:08 731 /usr/lib/ipsec/libtpmtss.so.0.0.0 b6913000-b6922000 ---p 0000c000 1f:08 731 /usr/lib/ipsec/libtpmtss.so.0.0.0 b6922000-b6923000 r-xp 0000b000 1f:08 731 /usr/lib/ipsec/libtpmtss.so.0.0.0 b6923000-b6924000 rwxp 0000c000 1f:08 731 /usr/lib/ipsec/libtpmtss.so.0.0.0 b6924000-b6b2f000 r-xp 00000000 1f:08 657 /usr/lib/libcrypto.so.1.1 b6b2f000-b6b3e000 ---p 0020b000 1f:08 657 /usr/lib/libcrypto.so.1.1 b6b3e000-b6b53000 r-xp 0020a000 1f:08 657 /usr/lib/libcrypto.so.1.1 b6b53000-b6b55000 rwxp 0021f000 1f:08 657 /usr/lib/libcrypto.so.1.1 b6b55000-b6b57000 rwxp 00000000 00:00 0 b6b57000-b6c8b000 r-xp 00000000 1f:08 501 /lib/libc-2.30.so b6c8b000-b6c9b000 ---p 00134000 1f:08 501 /lib/libc-2.30.so b6c9b000-b6c9d000 r-xp 00134000 1f:08 501 /lib/libc-2.30.so b6c9d000-b6c9e000 rwxp 00136000 1f:08 501 /lib/libc-2.30.so b6c9e000-b6ca1000 rwxp 00000000 00:00 0 b6ca1000-b6ca3000 r-xp 00000000 1f:08 505 /lib/libdl-2.30.so b6ca3000-b6cb2000 ---p 00002000 1f:08 505 /lib/libdl-2.30.so b6cb2000-b6cb3000 r-xp 00001000 1f:08 505 /lib/libdl-2.30.so b6cb3000-b6cb4000 rwxp 00002000 1f:08 505 /lib/libdl-2.30.so b6cb4000-b6ccd000 r-xp 00000000 1f:08 528 /lib/libpthread-2.30.so b6ccd000-b6cdc000 ---p 00019000 1f:08 528 /lib/libpthread-2.30.so b6cdc000-b6cdd000 r-xp 00018000 1f:08 528 /lib/libpthread-2.30.so b6cdd000-b6cde000 rwxp 00019000 1f:08 528 /lib/libpthread-2.30.so b6cde000-b6ce0000 rwxp 00000000 00:00 0 b6ce0000-b6d6d000 r-xp 00000000 1f:08 509 /lib/libm-2.30.so b6d6d000-b6d7c000 ---p 0008d000 1f:08 509 /lib/libm-2.30.so b6d7c000-b6d7d000 r-xp 0008c000 1f:08 509 /lib/libm-2.30.so b6d7d000-b6d7e000 rwxp 0008d000 1f:08 509 /lib/libm-2.30.so b6d7e000-b6e83000 r-xp 00000000 1f:08 725 /usr/lib/ipsec/libcharon.so.0.0.0 b6e83000-b6e92000 ---p 00105000 1f:08 725 /usr/lib/ipsec/libcharon.so.0.0.0 b6e92000-b6e93000 r-xp 00104000 1f:08 725 /usr/lib/ipsec/libcharon.so.0.0.0 b6e93000-b6e99000 rwxp 00105000 1f:08 725 /usr/lib/ipsec/libcharon.so.0.0.0 b6e99000-b6f31000 r-xp 00000000 1f:08 728 /usr/lib/ipsec/libstrongswan.so.0.0.0 b6f31000-b6f40000 ---p 00098000 1f:08 728 /usr/lib/ipsec/libstrongswan.so.0.0.0 b6f40000-b6f44000 r-xp 00097000 1f:08 728 /usr/lib/ipsec/libstrongswan.so.0.0.0 b6f44000-b6f4a000 rwxp 0009b000 1f:08 728 /usr/lib/ipsec/libstrongswan.so.0.0.0 b6f4a000-b6f6a000 r-xp 00000000 1f:08 499 /lib/ld-2.30.so b6f76000-b6f7a000 rwxp 00000000 00:00 0 b6f7a000-b6f7b000 r-xp 00020000 1f:08 499 /lib/ld-2.30.so b6f7b000-b6f7c000 rwxp 00021000 1f:08 499 /lib/ld-2.30.so bec15000-bec36000 rw-p 00000000 00:00 0 [stack] bed4c000-bed4d000 r-xp 00000000 00:00 0 [sigpage] ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors]
cat /var/log/charon_debug.log Fri, 2021-02-05, 11:01:54 08[CFG] added vici connection: ipsec1 Fri, 2021-02-05, 11:01:54 08[CFG] initiating 'ipsec1' Fri, 2021-02-05, 11:01:54 17[LIB] created thread 17 [12919] Fri, 2021-02-05, 11:01:54 08[KNL] using 192.168.7.243 as address to reach 89.24.0.7/32 Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_VENDOR task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_INIT task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_NATD task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_CERT_PRE task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_AUTH task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_CERT_POST task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_CONFIG task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_AUTH_LIFETIME task Fri, 2021-02-05, 11:01:54 08[IKE] queueing IKE_MOBIKE task Fri, 2021-02-05, 11:01:54 08[IKE] queueing CHILD_CREATE task Fri, 2021-02-05, 11:01:54 08[IKE] activating new tasks Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_VENDOR task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_INIT task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_NATD task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_CERT_PRE task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_AUTH task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_CERT_POST task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_CONFIG task Fri, 2021-02-05, 11:01:54 08[IKE] activating CHILD_CREATE task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_AUTH_LIFETIME task Fri, 2021-02-05, 11:01:54 08[IKE] activating IKE_MOBIKE task Fri, 2021-02-05, 11:01:54 08[IKE] sending strongSwan vendor ID Fri, 2021-02-05, 11:01:54 08[IKE] initiating IKE_SA ipsec1[1] to 89.24.0.7 Fri, 2021-02-05, 11:01:54 08[IKE] IKE_SA ipsec1[1] state change: CREATED => CONNECTING Fri, 2021-02-05, 11:01:54 08[LIB] size of DH secret exponent: 384 bits Fri, 2021-02-05, 11:01:54 08[CFG] configured proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_3072, IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MO DP_2048, IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1536 Fri, 2021-02-05, 11:01:54 08[CFG] sending supported signature hash algorithms: sha256 sha384 sha512 identity Fri, 2021-02-05, 11:01:54 08[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) V ] Fri, 2021-02-05, 11:01:54 08[NET] sending packet: from 192.168.7.243[500] to 89.24.0.7[500] (696 bytes) Fri, 2021-02-05, 11:01:54 15[CFG] vici client 2 connected Fri, 2021-02-05, 11:01:54 10[CFG] vici client 2 registered for: control-log Fri, 2021-02-05, 11:01:54 16[CFG] vici client 2 requests: terminate Fri, 2021-02-05, 11:01:54 16[CFG] vici terminate IKE_SA 'ipsec1' Fri, 2021-02-05, 11:01:54 05[IKE] destroying IKE_SA in state CONNECTING without notification Fri, 2021-02-05, 11:01:54 05[IKE] IKE_SA ipsec1[1] state change: CONNECTING => DESTROYING Fri, 2021-02-05, 11:01:54 09[CFG] vici client 2 disconnected Fri, 2021-02-05, 11:01:54 00[DMN] SIGTERM received, shutting down Fri, 2021-02-05, 11:01:54 00[DMN] executing stop script 'kill-tunnels' (python3 /opt/ipsec_scripts/connectionCloser.py) Fri, 2021-02-05, 11:01:55 12[CFG] vici client 3 connected Fri, 2021-02-05, 11:01:55 06[CFG] vici client 3 registered for: control-log Fri, 2021-02-05, 11:01:55 13[CFG] vici client 3 requests: terminate Fri, 2021-02-05, 11:01:55 13[CFG] vici terminate IKE_SA 'ipsec1' Fri, 2021-02-05, 11:01:55 07[CFG] vici client 3 disconnected Fri, 2021-02-05, 11:01:56 10[CFG] vici client 4 connected Fri, 2021-02-05, 11:01:56 11[CFG] vici client 4 requests: get-conns Fri, 2021-02-05, 11:01:56 05[CFG] vici client 4 requests: unload-conn Fri, 2021-02-05, 11:01:56 16[CFG] vici client 4 registered for: list-sa Fri, 2021-02-05, 11:01:56 12[CFG] vici client 4 requests: list-sas Fri, 2021-02-05, 11:01:56 15[CFG] vici client 4 unregistered for: list-sa Fri, 2021-02-05, 11:01:56 00[DMN] kill-tunnels: connectionCloser started Fri, 2021-02-05, 11:01:56 00[DMN] kill-tunnels: connectionCloser finished Fri, 2021-02-05, 11:01:56 09[CFG] vici client 4 disconnected
from strongswan.
zendulkaj
commented on May 23, 2024
If anything is missing let me know. I'll try to get it. Thank you very much.
from strongswan.
Thermi
commented on May 23, 2024
I'm sorry, gdb can't make head or tails of the coredump. It only shows a single thread and fails to unroll the stack frames.
from strongswan.
zendulkaj
commented on May 23, 2024
So the coredump from GDB is not helpfull for you? Do you need coredump from any other tool?
from strongswan.
Thermi
commented on May 23, 2024
The coredump from gdb was supposed to be usable just fine, I personally do not know what exactly is the problem with it. It is supposed to contain the verbatim memory content of the process, but the stack isn't unwound correctly.
Do you have any information from attaching gdb to the process while it hangs? Is the stack unwound correctly and are all threads shown?
from strongswan.
zendulkaj
commented on May 23, 2024
I am not too much familiar with gdb and stack/thread debuging. It would be helpfull for you if I give you remote SSH access to the embedded device when charon stucks? Thanks.
from strongswan.
Thermi
commented on May 23, 2024
Sorry, I can't do that due to liability issues.
from strongswan.
zendulkaj
commented on May 23, 2024
I sent email to Andreas with query for a paid assistance with this issue. We can sign a contract regarding liability issues. But I haven't got reply from Andreas yet.
from strongswan.
Thermi
commented on May 23, 2024
Hi, Andreas is likely not available at all for paid work due to changes in employment with regards to the HSR.
We can go through my employer for paid assistance. I'm listed with my contact information on the strongSwan wiki.
from strongswan.
tpaukrt
commented on May 23, 2024
Hi, it seems that this issue was introduced by e567675 which created a possibility that two threads will wait on the same condvar. The first thread will wait in the function find_entry while the second will wait in the function remove_entry and only one thread is resumed by calling put_entry. If the client is disconnecting and the thread waiting in the function find_entry is resumed then this function returns NULL and the second thread waiting in the function remove_entry will never be resumed, so it will block charon termination. Using broadcast instead of signal solves this issue, but I'm not sure if it does not have any side effects.
diff --exclude CVS --exclude .git -uNr strongswan-5.9.2/src/libcharon/plugins/vici/vici_socket.c strongswan-5.9.2.modified/src/libcharon/plugins/vici/vici_socket.c
--- strongswan-5.9.2/src/libcharon/plugins/vici/vici_socket.c 2020-11-10 20:39:03.000000000 +0100
+++ strongswan-5.9.2.modified/src/libcharon/plugins/vici/vici_socket.c 2021-04-20 15:59:30.411156244 +0200
@@ -270,7 +270,7 @@
{
entry->writers--;
}
- entry->cond->signal(entry->cond);
+ entry->cond->broadcast(entry->cond);
this->mutex->unlock(this->mutex);
}
from strongswan.
tobiasbrunner
commented on May 23, 2024
While testing devices in our test system, we noticed that sometimes Charon is stucking when the IPsec service is stopped. The following commands are executed during service ipsec stop:
Why do you terminate the connection manually with --force before sending a SIGTERM to the daemon? The daemon does exactly the same, i.e. sending a single delete for every established IKE_SA, before it terminates.
@tpaukrt Thanks for your analysis. However, if a client disconnects, the entry can only be removed via a call to remove_entry(), only after that can find_entry() fail and return NULL. So your described scenario isn't entirely correct. It's rather the other way around. That is, if we have one or more threads waiting in find_entry() and one in remove_entry() and the latter is woken first, then the former will be stuck (if a thread in find_entry() is woken first it will call put_entry() and wake another thread). So I think instead of using broadcast() in put_entry(), we should add such a call in remove_entry() to wake all threads that might still be waiting in find_entry(). I pushed a commit to the 268-vici-stuck branch.
from strongswan.
zendulkaj
commented on May 23, 2024
Why do you terminate the connection manually with --force before sending a SIGTERM to the daemon? The daemon does exactly the same, i.e. sending a single delete for every established IKE_SA, before it terminates.
It is done to be sure that there is no active connection before charon is killed. First we thought that killing charon with existing active connections) can be a reason of this issue...
We will test your fix. Thank you very much.
from strongswan.
tpaukrt
commented on May 23, 2024
@tobiasbrunner Calling broadcast() from remove_entry() will not fix this issue, because if a thread in find_entry() is woken first and entry->disconnecting is True then it will never call put_entry().
from strongswan.
tobiasbrunner
commented on May 23, 2024
True, overlooked that flag. So let's call signal() before continue in find_entry(). I've pushed another commit to the branch.
from strongswan.
tobiasbrunner
commented on May 23, 2024
@zendulkaj Any feedback on the changes in the 268-vici-stuck branch?
from strongswan.
zendulkaj
commented on May 23, 2024
@tobiasbrunner We stayed at @tpaukrt patch as we had it properly tested and we needed to release our FW at that time. We will test the 268-vici-stuck branch now. I wil give a feedback in the comming days. Thanks.
from strongswan.
tobiasbrunner
commented on May 23, 2024
Any update? We are preparing the next release and it would be great if we could include this fix.
from strongswan.
zendulkaj
commented on May 23, 2024
Our internal tests have been passing for several days. So it looks that it is definitely fixed. Thanks.
from strongswan.
tobiasbrunner
commented on May 23, 2024
Great, thanks for testing.
from strongswan.
Related Issues (20)
- FORTIFY: pthread_mutex_lock called on a destroyed mutex HOT 1
- charon-systemd Preemptively Exiting on Kernel alg Debugging Messages HOT 1
- segfault when compiled with leak-detective enabled and systemd >=254 HOT 3
- potential automatic mangle rules issues w.r.t. mark_in and mark_out settings HOT 3
- StrongSwan 6 beta 5 | Failed to generate a common proposal even though there is an acceptable choice HOT 3
- StrongSwan 6 Beta 5 integration with liboqs 0.9.1 | Getting error "negotiated key exchange method KYBER_L3 not supported" HOT 2
- Allow comparing connection/child configuration with reported status HOT 1
- Restart router A. The process is successfully started and the configuration parameters are correct. However, the connection to IPsec server B fails. HOT 1
- Incomplete logging of log messages containing newlines when using `swanctl --initiate --loglevel 3/4` HOT 1
- Support systemd socket activation for charon HOT 3
- swanctl listing wrong data HOT 4
- Running suite 'rsa' hangs HOT 1
- Can't connect to StrongSwan VPN with Android 14 native client
- X509v3 Name Constraints incorrectly required on subordinate CAs in chain HOT 6
- StrongSwan Android 2.5.0 Start/Stop Profile Intent profile not found HOT 5
- Restoring EAP-TTLS (and PEAP) support on Android HOT 9
- F-Droid can't build HOT 6
- SecurityException for SCHEDULE_EXACT_ALARM HOT 3
- build project on CentOS failed
- bad memcpy() in dhcp_socket.c, line 253 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from strongswan.