Comments (15)
rfuchs
commented on July 22, 2024
It looks like you have a stripped binary. If this is from a Debian package, please also install the -dbg package and then print the backtrace again.
This call may have had some unusual properties about it, please post the rest of the log related to this call.
from rtpengine.
digipigeon
commented on July 22, 2024
I will work on getting you this now.
from rtpengine.
digipigeon
commented on July 22, 2024
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/rtpengine...Reading symbols from /usr/lib/debug//usr/sbin/rtpengine...done.
done.
[New LWP 30026]
[New LWP 30020]
[New LWP 30021]
[New LWP 30022]
[New LWP 30023]
[New LWP 30024]
[New LWP 30025]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/rtpengine --ip=178.62.XXX.XXX --listen-tcp=25060 --listen-udp=22222 --l'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 __call_stream_address (ps=ps@entry=0x0, variant=variant@entry=0) at call.c:2273
2273 m = ps->call->callmaster;
(gdb) bt full
#0 __call_stream_address (ps=ps@entry=0x0, variant=variant@entry=0) at call.c:2273
m = <optimized out>
sink = 0x0
sink_media = <optimized out>
variants = {0x2, 0x7f470fe966af <g_realloc+15>}
__PRETTY_FUNCTION__ = "__call_stream_address"
#1 0x00000000004118d6 in call_stream_address (o=0x7f47078082a0 "", ps=<optimized out>, format=SAF_UDP, len=0x7f470780829c) at call.c:2303
f = <optimized out>
#2 0x000000000041d8a6 in call_stream_address_gstring (format=SAF_UDP, ps=0x7f470039cca0, o=0x7f46f423ee20) at call_interfaces.c:26
len = 32583
ret = <optimized out>
buf = "\000\000\000\000\000\000\000\000\024\313\352\017G\177\000\000\002\000\000\000\000\000\000\000\240\000\000\000\000\000\000\000 \000\000\000\000\000\000\000\030\000\000\000\000\000\000\000 \000\000\000\000\000\000\000\060\016\071\000\001\000\000"
#3 streams_print (start=<optimized out>, end=1, prefix=<optimized out>, format=format@entry=SAF_UDP, s=0x7f4700390e18) at call_interfaces.c:65
o = 0x7f46f423ee20
i = 1
af = 3787936
port = <optimized out>
l = <optimized out>
media = <optimized out>
ps = 0x7f470039cca0
#4 0x000000000041db34 in call_update_lookup_udp (out=out@entry=0x7f4700381ea0, m=0xe9f800, opmode=opmode@entry=OP_OFFER) at call_interfaces.c:154
c = 0x7f46f038b620
monologue = 0x7f4700390de0
q = {head = 0x0, tail = 0x0, length = 0}
sp = {index = 1, type = {s = 0x0, len = 0}, rtp_endpoint = {ip46 = {__in6_u = {__u6_addr8 = "\000\000\000\000\000\000\000\000\000\000\377\377\324T", <incomplete sequence \334>, __u6_addr16 = {0,
0, 0, 0, 0, 65535, 21716, 56397}, __u6_addr32 = {0, 0, 4294901760, 3696055508}}}, port = 15124}, rtcp_endpoint = {ip46 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>,
__u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, port = 0}, consecutive_ports = 1, protocol = 0x0, crypto = {crypto_suite = 0x0,
master_key = '\000' <repeats 15 times>, master_salt = '\000' <repeats 13 times>, mki = 0x0, mki_len = 0}, sdes_tag = 0, direction = {DIR_UNKNOWN, DIR_UNKNOWN}, desired_family = 0,
fingerprint = {digest = '\000' <repeats 63 times>, hash_func = 0x0}, sp_flags = 0}
ret = <optimized out>
callid = {s = 0x7f4700381f73 "[email protected]", len = 46}
viabranch = {s = <optimized out>, len = 0}
fromtag = {s = 0x7f4700381fb8 "as3ddccc54", len = 10}
totag = {s = 0x0, len = 0}
#5 0x000000000041e057 in call_update_udp (out=out@entry=0x7f4700381ea0, m=<optimized out>) at call_interfaces.c:180
No locals.
#6 0x000000000041259c in control_udp_incoming (obj=0xea3850, buf=<optimized out>, sin=0x7f47078087b0, addr=<optimized out>) at control_udp.c:85
u = 0xea3850
ret = 11
ovec = {0, 96, 0, 10, 11, 12, 12, 16, 17, 63, -1, -1, 64, 77, -1, -1, 78, 83, 84, 94, 95, 96, -1 <repeats 20 times>, 309128, 32583, 15351840, 0, 125863888, 32583, 15344752, 0, 125863952, 32583,
7, 0, 125863864, 32583, 96, 0, 244027229, 32583, 309672, 32583, 40, 48, 125863808, 32583, 125863616, 32583, 0, 0, 244094497, 32583, 125863952, 32583, 10, 0, 50645, 0, 0, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, 95, 84, 78, -1, 64, -1, 17, 12, 11, 0, 0}
out = 0x7f4700381ea0
mh = {msg_name = 0x7f47fbad8001, msg_namelen = 125863696, msg_iov = 0x7f4707808710, msg_iovlen = 139943045269264, msg_control = 0x7f4707808710, msg_controllen = 139943045269278, msg_flags = -1}
iov = {{iov_base = 0xffffffffffffffff, iov_len = 0}, {iov_base = 0x3534363035000000, iov_len = 139943045268848}, {iov_base = 0x7f47078087de, iov_len = 139943045268848}, {
iov_base = 0x7f47078087de, iov_len = 4333964}, {iov_base = 0x7f47078086a8, iov_len = 1}, {iov_base = 0xffffffffffffffff, iov_len = 139943163432964}, {iov_base = 0xfffffffffbad8001,
iov_len = 139943045269470}, {iov_base = 0x7f47078087de, iov_len = 139943045269470}, {iov_base = 0x7f47078087de, iov_len = 139943045269476}, {iov_base = 0xffffffffffffffff,
iov_len = 139943045269470}}
cookie = {s = 0x7f4700381f61 "23925_7184", len = 10}
reply = 0x0
#7 0x0000000000413e21 in udp_listener_incoming (fd=7, p=0xea2470, x=<optimized out>) at udp_listener.c:52
cb = 0xea2470
sin = {sin6_family = 10, sin6_port = 54725, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = "\000\000\000\000\000\000\000\000\000\000\377\377\202\323Nm", __u6_addr16 = {0, 0, 0, 0, 0,
65535, 54146, 27982}, __u6_addr32 = {0, 0, 4294901760, 1833882498}}}, sin6_scope_id = 0}
sin_len = 28
len = <optimized out>
---Type <return> to continue, or q <return> to quit---
buf = "23925_7184 Ud101 [email protected] 212.84.XXX.XXX 15124 as3ddccc54;1\000\061 10.1.XXX.XXX:9999 1844.112024233\000\000\000\064\065\060", '\000' <repeats 52235 times>...
addr = "130.211.XXX.XXX:50645", '\000' <repeats 43 times>
str = {s = 0x7f4707808810 "23925_7184 Ud101 [email protected] 212.84.XXX.XXX 15124 as3ddccc54;1", len = 96}
#8 0x000000000040b54e in poller_poll (p=p@entry=0xe9ad30, timeout=timeout@entry=100) at poller.c:354
ret = 1
i = <optimized out>
it = 0xea1d80
evs = {{events = 1, data = {ptr = 0x7, fd = 7, u32 = 7, u64 = 7}}, {events = 1, data = {ptr = 0x265b, fd = 9819, u32 = 9819, u64 = 9819}}, {events = 1, data = {ptr = 0x26c3, fd = 9923,
u32 = 9923, u64 = 9923}}, {events = 1, data = {ptr = 0x25d7, fd = 9687, u32 = 9687, u64 = 9687}}, {events = 1, data = {ptr = 0x25d7, fd = 9687, u32 = 9687, u64 = 9687}}, {events = 1,
data = {ptr = 0x262f, fd = 9775, u32 = 9775, u64 = 9775}}, {events = 1, data = {ptr = 0x25eb, fd = 9707, u32 = 9707, u64 = 9707}}, {events = 1, data = {ptr = 0x25cb, fd = 9675, u32 = 9675,
u64 = 9675}}, {events = 1, data = {ptr = 0x25bf, fd = 9663, u32 = 9663, u64 = 9663}}, {events = 1, data = {ptr = 0x7, fd = 7, u32 = 7, u64 = 7}}, {events = 1, data = {ptr = 0x256b,
fd = 9579, u32 = 9579, u64 = 9579}}, {events = 1, data = {ptr = 0x2593, fd = 9619, u32 = 9619, u64 = 9619}}, {events = 1, data = {ptr = 0x2537, fd = 9527, u32 = 9527, u64 = 9527}}, {
events = 1, data = {ptr = 0x23b9, fd = 9145, u32 = 9145, u64 = 9145}}, {events = 1, data = {ptr = 0x1d45, fd = 7493, u32 = 7493, u64 = 7493}}, {events = 0, data = {ptr = 0x0, fd = 0,
u32 = 0, u64 = 0}} <repeats 61 times>, {events = 0, data = {ptr = 0x10663dc900000000, fd = 0, u32 = 0, u64 = 1181699885729054720}}, {events = 32583, data = {ptr = 0x0, fd = 0, u32 = 0,
u64 = 0}}, {events = 3045937268, data = {ptr = 0x300000000, fd = 0, u32 = 0, u64 = 12884901888}}, {events = 0, data = {ptr = 0x2e, fd = 46, u32 = 46, u64 = 46}}, {events = 277229944,
data = {ptr = 0x1066481600007f47, fd = 32583, u32 = 32583, u64 = 1181711211557846855}}, {events = 32583, data = {ptr = 0xc, fd = 12, u32 = 12, u64 = 12}}, {events = 125930624, data = {
ptr = 0xfe4b2e000007f47, fd = 32583, u32 = 32583, u64 = 1145236880401399623}}, {events = 32583, data = {ptr = 0x7f4707818d90, fd = 125930896, u32 = 125930896, u64 = 139943045336464}}, {
events = 266688192, data = {ptr = 0x2d6354100007f47, fd = 32583, u32 = 32583, u64 = 204409386381115207}}, {events = 0, data = {ptr = 0x7f4707818d80, fd = 125930880, u32 = 125930880,
u64 = 139943045336448}}, {events = 125930928, data = {ptr = 0x7f47, fd = 32583, u32 = 32583, u64 = 32583}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 277291008,
data = {ptr = 0x4032ef00007f47, fd = 32583, u32 = 32583, u64 = 18070400588087111}}, {events = 0, data = {ptr = 0x7f470fe55f78, fd = 266690424, u32 = 266690424, u64 = 139943186095992}}, {
events = 4199544, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 5, data = {ptr = 0x420, fd = 1056, u32 = 1056, u64 = 1056}}, {events = 4169036481, data = {ptr = 0x43ffff80b8,
fd = -32584, u32 = 4294934712, u64 = 292057743544}}, {events = 32583, data = {ptr = 0x10, fd = 16, u32 = 16, u64 = 16}}, {events = 64, data = {ptr = 0x7818d4000000000, fd = 0, u32 = 0,
u64 = 540868736278593536}}, {events = 32583, data = {ptr = 0x2, fd = 2, u32 = 2, u64 = 2}}, {events = 277340616, data = {ptr = 0x5d, fd = 93, u32 = 93, u64 = 93}}, {events = 0, data = {
ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 110, data = {ptr = 0x77, fd = 119, u32 = 119, u64 = 119}}, {events = 0, data = {ptr = 0x7f4707818d3f, fd = 125930815, u32 = 125930815,
u64 = 139943045336383}}, {events = 1, data = {ptr = 0x2000000000000, fd = 0, u32 = 0, u64 = 562949953421312}}, {events = 0, data = {ptr = 0x1000, fd = 4096, u32 = 4096, u64 = 4096}}, {
events = 125930847, data = {ptr = 0x100007f47, fd = 32583, u32 = 32583, u64 = 4294999879}}, {events = 0, data = {ptr = 0x20000, fd = 131072, u32 = 131072, u64 = 131072}}, {events = 0,
data = {ptr = 0x1087e52000000000, fd = 0, u32 = 0, u64 = 1191172552064499712}}, {events = 32583, data = {ptr = 0x7f4707818d90, fd = 125930896, u32 = 125930896, u64 = 139943045336464}}, {
events = 7168, data = {ptr = 0x1bf000007f47, fd = 32583, u32 = 32583, u64 = 30717606133575}}, {events = 32583, data = {ptr = 0x200, fd = 512, u32 = 512, u64 = 512}}, {events = 32, data = {
ptr = 0x2f00007f47, fd = 32583, u32 = 32583, u64 = 201863495495}}, {events = 0, data = {ptr = 0x33, fd = 51, u32 = 51, u64 = 51}}, {events = 243470927, data = {ptr = 0x20000007f47,
fd = 32583, u32 = 32583, u64 = 2199023288135}}, {events = 0, data = {ptr = 0x7f4700000020, fd = 32, u32 = 32, u64 = 139942919405600}}, {events = 496, data = {ptr = 0x2000000000, fd = 0,
u32 = 0, u64 = 137438953472}}, {events = 0, data = {ptr = 0x2f, fd = 47, u32 = 47, u64 = 47}}, {events = 243477220, data = {ptr = 0x7818e0800007f47, fd = 32583, u32 = 32583,
u64 = 540869595272085319}}, {events = 32583, data = {ptr = 0x8, fd = 8, u32 = 8, u64 = 8}}, {events = 512, data = {ptr = 0xe83473200000000, fd = 0, u32 = 0, u64 = 1045757818554023936}}, {
events = 32583, data = {ptr = 0x20, fd = 32, u32 = 32, u64 = 32}}, {events = 266743998, data = {ptr = 0x8c000007f47, fd = 32583, u32 = 32583, u64 = 9620726775623}}, {events = 32583, data = {
ptr = 0x7f4700001c00, fd = 7168, u32 = 7168, u64 = 139942919412736}}, {events = 2240, data = {ptr = 0x1800007f47, fd = 32583, u32 = 32583, u64 = 103079247687}}, {events = 0, data = {
ptr = 0x7f47000008e0, fd = 2272, u32 = 2272, u64 = 139942919407840}}, {events = 267200961, data = {ptr = 0x2000007f47, fd = 32583, u32 = 32583, u64 = 137438986055}}, {events = 0, data = {
ptr = 0x7f470feac73d <g_slice_alloc+1357>, fd = 267044669, u32 = 267044669, u64 = 139943186450237}}, {events = 2256, data = {ptr = 0xe9f72000007f47, fd = 32583, u32 = 32583,
u64 = 65855386384629575}}, {events = 0, data = {ptr = 0x10, fd = 16, u32 = 16, u64 = 16}}, {events = 7232, data = {ptr = 0x781970000007f47, fd = 32583, u32 = 32583,
u64 = 540879456516996935}}, {events = 32583, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}}
ev = 0x7f4707818870
e = {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}
#9 0x00000000004076ad in poller_loop (d=0xe9ad30) at main.c:553
p = 0xe9ad30
#10 0x000000000040b8d0 in thread_detach_func (d=0xea1a70) at aux.c:160
dt = 0xea1a70
t = 0xe9d4c0
#11 0x00007f470eb7d182 in start_thread (arg=0x7f4707819700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f4707819700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139943045338880, -3679294963828722566, 0, 0, 139943045339584, 139943045338880, 3638499182036841594, 3638514696643998842}, mask_was_saved = 0}},
priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
---Type <return> to continue, or q <return> to quit---
#12 0x00007f470e8aa38d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.
(gdb)
from rtpengine.
rfuchs
commented on July 22, 2024
Could you also please post the rest of the log for this call-id, thanks
from rtpengine.
digipigeon
commented on July 22, 2024
Aug 28 06:05:26 rtp-lon-5 rtpengine[30020]: Got valid command from udp:130.211.XXX.XXX:50645: 23925_7184 Ud101 [email protected] 212.84.XXX.XXX 15124 as3ddccc54;1
Aug 28 06:05:26 rtp-lon-5 rtpengine[30020]: [[email protected]] Creating new call
Aug 28 06:05:26 rtp-lon-5 rtpengine[30020]: [[email protected]] Failed to get 2 consecutive UDP ports for relay
Aug 28 06:05:26 rtp-lon-5 rtpengine[30020]: [[email protected]] Error allocating media ports
from rtpengine.
rfuchs
commented on July 22, 2024
Looks like your proxy ran out of local ports, resulting in a segfault due to this error condition not being handled properly. The above commit should fix that, however you would still have run out of ports, giving you broken calls. It just shouldn't segfault any more.
from rtpengine.
digipigeon
commented on July 22, 2024
Sorry to continue the conversation past fixing the issue, but can you please advise what is required to prevent running out of ports. I have tried setting "ulimit -n 100000" in the init script ,maybe I am placing this in the wrong place?
from rtpengine.
rfuchs
commented on July 22, 2024
Use these options:
-m, --port-min=INT Lowest port to use for RTP
-M, --port-max=INT Highest port to use for RTP
... or through the /etc/defaults/ file if you're using the init.d script.
from rtpengine.
digipigeon
commented on July 22, 2024
I am afraid this did not completely resolve the error, I have set
PORT_MIN=30000
PORT_MAX=50000
in the defaults file, I have also upgraded
git clone and ran dpkg -i on the new packages, checked file modify times on both packages and binaries after install.
Aug 29 12:10:46 rtp-lon-4 rtpengine[5074]: [[email protected]] Error allocating media ports
Aug 29 12:10:46 rtp-lon-4 kernel: [182118.709410] rtpengine[5079]: segfault at 58 ip 000000000040e385 sp 00007f766d5b9240 error 4 in rtpengine[400000+2a000]
I have another CoreDump, however it looks similar to the one before.
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/rtpengine...Reading symbols from /usr/lib/debug//usr/sbin/rtpengine...done.
done.
[New LWP 5079]
[New LWP 5074]
[New LWP 5075]
[New LWP 5076]
[New LWP 5080]
[New LWP 5078]
[New LWP 5077]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/rtpengine --ip=178.62.XXX.XXX --listen-tcp=25060 --listen-udp=22222 --'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 __call_stream_address (ps=ps@entry=0x0, variant=variant@entry=0) at call.c:2273
2273 m = ps->call->callmaster;
(gdb) bt full
#0 __call_stream_address (ps=ps@entry=0x0, variant=variant@entry=0) at call.c:2273
m = <optimized out>
sink = 0x0
sink_media = <optimized out>
variants = {0x2, 0x7f76754466af <g_realloc+15>}
__PRETTY_FUNCTION__ = "__call_stream_address"
#1 0x00000000004118d6 in call_stream_address (o=0x7f766d5b92a0 "", ps=<optimized out>, format=SAF_UDP, len=0x7f766d5b929c) at call.c:2303
f = <optimized out>
#2 0x000000000041d8a6 in call_stream_address_gstring (format=SAF_UDP, ps=0x7f766075a010, o=0x7f765c1ad740) at call_interfaces.c:26
len = 32630
ret = <optimized out>
buf = "\000\000\000\000\000\000\000\000\024\313Euv\177\000\000\002\000\000\000\000\000\000\000\240\000\000\000\000\000\000\000 \000\000\000\000\000\000\000\030\000\000\000\000\000\000\000 \000\000\000\000\000\000\000@\321sX\001\000\000"
#3 streams_print (start=<optimized out>, end=1, prefix=<optimized out>, format=format@entry=SAF_UDP, s=0x7f765873d038) at call_interfaces.c:65
o = 0x7f765c1ad740
i = 1
af = 1618321424
port = <optimized out>
l = <optimized out>
media = <optimized out>
ps = 0x7f766075a010
#4 0x000000000041db34 in call_update_lookup_udp (out=out@entry=0x7f7660732e10, m=0x128c800, opmode=opmode@entry=OP_OFFER) at call_interfaces.c:154
c = 0x7f7658742950
monologue = 0x7f765873d000
q = {head = 0x0, tail = 0x0, length = 0}
sp = {index = 1, type = {s = 0x0, len = 0}, rtp_endpoint = {ip46 = {__in6_u = {
__u6_addr8 = "\000\000\000\000\000\000\000\000\000\000\377\377\324T", <incomplete sequence \331>, __u6_addr16 = {0, 0, 0, 0, 0, 65535, 21716, 55629},
__u6_addr32 = {0, 0, 4294901760, 3645723860}}}, port = 13828}, rtcp_endpoint = {ip46 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0,
0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, port = 0}, consecutive_ports = 1, protocol = 0x0, crypto = {crypto_suite = 0x0,
master_key = '\000' <repeats 15 times>, master_salt = '\000' <repeats 13 times>, mki = 0x0, mki_len = 0}, sdes_tag = 0, direction = {DIR_UNKNOWN, DIR_UNKNOWN},
desired_family = 0, fingerprint = {digest = '\000' <repeats 63 times>, hash_func = 0x0}, sp_flags = 0}
ret = <optimized out>
callid = {s = 0x7f7660732ee3 "[email protected]", len = 46}
viabranch = {s = <optimized out>, len = 0}
fromtag = {s = 0x7f7660732f28 "as30a4176f", len = 10}
totag = {s = 0x0, len = 0}
#5 0x000000000041e057 in call_update_udp (out=out@entry=0x7f7660732e10, m=<optimized out>) at call_interfaces.c:180
No locals.
#6 0x000000000041259c in control_udp_incoming (obj=0x1290850, buf=<optimized out>, sin=0x7f766d5b97b0, addr=<optimized out>) at control_udp.c:85
u = 0x1290850
ret = 11
ovec = {0, 96, 0, 10, 11, 12, 12, 16, 17, 63, -1, -1, 64, 77, -1, -1, 78, 83, 84, 94, 95, 96, -1 <repeats 20 times>, 0, 0, 0, 0, 1834719184, 32630, 19461232, 0,
1834719248, 32630, 7, 0, 1834719160, 32630, 96, 0, 1944489821, 32630, 0, 0, 40, 48, 1834719104, 32630, 1834718912, 32630, 0, 0, 1944557089, 32630, 0, 0, 0, 0,
40288, 0, 892415283, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 95, 84, 78, -1, 64, -1, 17, 12, 11, 0, 0}
out = 0x7f7660732e10
mh = {msg_name = 0x7f76fbad8001, msg_namelen = 1834718992, msg_iov = 0x7f766d5b9710, msg_iovlen = 140146617587472, msg_control = 0x7f766d5b9710,
msg_controllen = 140146617587484, msg_flags = -1}
iov = {{iov_base = 0xffffffffffffffff, iov_len = 0}, {iov_base = 0x3838323034000000, iov_len = 140146617587056}, {iov_base = 0x7f766d5b97dc,
iov_len = 140146617587056}, {iov_base = 0x7f766d5b97dc, iov_len = 4333964}, {iov_base = 0x7f766d5b96a8, iov_len = 1}, {iov_base = 0xffffffffffffffff,
iov_len = 140146727358468}, {iov_base = 0xfffffffffbad8001, iov_len = 140146617587676}, {iov_base = 0x7f766d5b97dc, iov_len = 140146617587676}, {
---Type <return> to continue, or q <return> to quit---
iov_base = 0x7f766d5b97dc, iov_len = 140146617587682}, {iov_base = 0xffffffffffffffff, iov_len = 140146617587676}}
cookie = {s = 0x7f7660732ed1 "9416_41290", len = 10}
reply = 0x0
#7 0x0000000000413e21 in udp_listener_incoming (fd=7, p=0x128f470, x=<optimized out>) at udp_listener.c:52
cb = 0x128f470
sin = {sin6_family = 10, sin6_port = 24733, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = "\000\000\000\000\000\000\000\000\000\000\377\377\\0r\032",
__u6_addr16 = {0, 0, 0, 0, 0, 65535, 12380, 6770}, __u6_addr32 = {0, 0, 4294901760, 443691100}}}, sin6_scope_id = 0}
sin_len = 28
len = <optimized out>
buf = "9416_41290 Ud101 [email protected] 212.84.XXX.XXX 13828 as30a4176f;1\000;1 92.48.XXX.XXX:9999 751.1571470723", '\000' <repeats 52237 times>...
addr = "92.48.XXX.XXX:40288\000\070", '\000' <repeats 43 times>
str = {s = 0x7f766d5b9810 "9416_41290 Ud101 [email protected] 212.84.XXX.XXX 13828 as30a4176f;1", len = 96}
#8 0x000000000040b54e in poller_poll (p=p@entry=0x1287d30, timeout=timeout@entry=100) at poller.c:354
ret = 3
i = <optimized out>
it = 0x128ed80
evs = {{events = 1, data = {ptr = 0x4d5f, fd = 19807, u32 = 19807, u64 = 19807}}, {events = 1, data = {ptr = 0x25e1, fd = 9697, u32 = 9697, u64 = 9697}}, {
events = 1, data = {ptr = 0x7, fd = 7, u32 = 7, u64 = 7}}, {events = 1, data = {ptr = 0x4e11, fd = 19985, u32 = 19985, u64 = 19985}}, {events = 1, data = {
ptr = 0x24d9, fd = 9433, u32 = 9433, u64 = 9433}}, {events = 1, data = {ptr = 0x25d9, fd = 9689, u32 = 9689, u64 = 9689}}, {events = 1, data = {ptr = 0x24de,
fd = 9438, u32 = 9438, u64 = 9438}}, {events = 1, data = {ptr = 0x2619, fd = 9753, u32 = 9753, u64 = 9753}}, {events = 1, data = {ptr = 0x4df3, fd = 19955,
u32 = 19955, u64 = 19955}}, {events = 1, data = {ptr = 0x4cc9, fd = 19657, u32 = 19657, u64 = 19657}}, {events = 1, data = {ptr = 0x4b97, fd = 19351,
u32 = 19351, u64 = 19351}}, {events = 1, data = {ptr = 0x4bd3, fd = 19411, u32 = 19411, u64 = 19411}}, {events = 1, data = {ptr = 0x4e05, fd = 19973,
u32 = 19973, u64 = 19973}}, {events = 1, data = {ptr = 0x4c9d, fd = 19613, u32 = 19613, u64 = 19613}}, {events = 1, data = {ptr = 0x4e01, fd = 19969,
u32 = 19969, u64 = 19969}}, {events = 1, data = {ptr = 0x4d77, fd = 19831, u32 = 19831, u64 = 19831}}, {events = 1, data = {ptr = 0x4bd3, fd = 19411,
u32 = 19411, u64 = 19411}}, {events = 1, data = {ptr = 0x4db3, fd = 19891, u32 = 19891, u64 = 19891}}, {events = 1, data = {ptr = 0x4d77, fd = 19831,
u32 = 19831, u64 = 19831}}, {events = 1, data = {ptr = 0x4e05, fd = 19973, u32 = 19973, u64 = 19973}}, {events = 1, data = {ptr = 0x4db3, fd = 19891,
u32 = 19891, u64 = 19891}}, {events = 1, data = {ptr = 0x4d53, fd = 19795, u32 = 19795, u64 = 19795}}, {events = 1, data = {ptr = 0x4d3f, fd = 19775,
u32 = 19775, u64 = 19775}}, {events = 1, data = {ptr = 0x4b97, fd = 19351, u32 = 19351, u64 = 19351}}, {events = 1, data = {ptr = 0x4d77, fd = 19831,
u32 = 19831, u64 = 19831}}, {events = 1, data = {ptr = 0x4e05, fd = 19973, u32 = 19973, u64 = 19973}}, {events = 1, data = {ptr = 0x4d2f, fd = 19759,
u32 = 19759, u64 = 19759}}, {events = 1, data = {ptr = 0x4d01, fd = 19713, u32 = 19713, u64 = 19713}}, {events = 1, data = {ptr = 0x2511, fd = 9489,
u32 = 9489, u64 = 9489}}, {events = 1, data = {ptr = 0x4d53, fd = 19795, u32 = 19795, u64 = 19795}}, {events = 1, data = {ptr = 0x4dd3, fd = 19923,
u32 = 19923, u64 = 19923}}, {events = 1, data = {ptr = 0x4d93, fd = 19859, u32 = 19859, u64 = 19859}}, {events = 1, data = {ptr = 0x4d77, fd = 19831,
u32 = 19831, u64 = 19831}}, {events = 1, data = {ptr = 0x2535, fd = 9525, u32 = 9525, u64 = 9525}}, {events = 1, data = {ptr = 0x4cf9, fd = 19705,
u32 = 19705, u64 = 19705}}, {events = 1, data = {ptr = 0x4cd1, fd = 19665, u32 = 19665, u64 = 19665}}, {events = 1, data = {ptr = 0x2511, fd = 9489,
u32 = 9489, u64 = 9489}}, {events = 1, data = {ptr = 0x4c61, fd = 19553, u32 = 19553, u64 = 19553}}, {events = 1, data = {ptr = 0x4e09, fd = 19977,
u32 = 19977, u64 = 19977}}, {events = 1, data = {ptr = 0x4d53, fd = 19795, u32 = 19795, u64 = 19795}}, {events = 1, data = {ptr = 0x4e0a, fd = 19978,
u32 = 19978, u64 = 19978}}, {events = 1, data = {ptr = 0x4d3f, fd = 19775, u32 = 19775, u64 = 19775}}, {events = 1, data = {ptr = 0x4d01, fd = 19713,
u32 = 19713, u64 = 19713}}, {events = 1, data = {ptr = 0x4d2f, fd = 19759, u32 = 19759, u64 = 19759}}, {events = 1, data = {ptr = 0x4dd3, fd = 19923,
u32 = 19923, u64 = 19923}}, {events = 1, data = {ptr = 0x4d93, fd = 19859, u32 = 19859, u64 = 19859}}, {events = 1, data = {ptr = 0x4d17, fd = 19735,
u32 = 19735, u64 = 19735}}, {events = 1, data = {ptr = 0x4d77, fd = 19831, u32 = 19831, u64 = 19831}}, {events = 1, data = {ptr = 0x4d97, fd = 19863,
u32 = 19863, u64 = 19863}}, {events = 1, data = {ptr = 0x4e01, fd = 19969, u32 = 19969, u64 = 19969}}, {events = 1, data = {ptr = 0x4db3, fd = 19891,
u32 = 19891, u64 = 19891}}, {events = 1, data = {ptr = 0x4e05, fd = 19973, u32 = 19973, u64 = 19973}}, {events = 1, data = {ptr = 0x2505, fd = 9477,
u32 = 9477, u64 = 9477}}, {events = 1, data = {ptr = 0x2549, fd = 9545, u32 = 9545, u64 = 9545}}, {events = 1, data = {ptr = 0x4d6b, fd = 19819, u32 = 19819,
u64 = 19819}}, {events = 1, data = {ptr = 0x4d07, fd = 19719, u32 = 19719, u64 = 19719}}, {events = 1, data = {ptr = 0x4b07, fd = 19207, u32 = 19207,
u64 = 19207}}, {events = 1, data = {ptr = 0x4b97, fd = 19351, u32 = 19351, u64 = 19351}}, {events = 1, data = {ptr = 0x4df7, fd = 19959, u32 = 19959,
u64 = 19959}}, {events = 1, data = {ptr = 0x4d5b, fd = 19803, u32 = 19803, u64 = 19803}}, {events = 1, data = {ptr = 0x2555, fd = 9557, u32 = 9557,
u64 = 9557}}, {events = 1, data = {ptr = 0x4b43, fd = 19267, u32 = 19267, u64 = 19267}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0,
u64 = 0}} <repeats 26 times>, {events = 1834786000, data = {ptr = 0x92a3633100007f76, fd = 32630, u32 = 32630, u64 = 10566398212845895542}}, {
events = 4294934665, data = {ptr = 0x7f766d5c9ccf, fd = 1834785999, u32 = 1834785999, u64 = 140146617654479}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0,
---Type <return> to continue, or q <return> to quit---
u64 = 0}}, {events = 0, data = {ptr = 0x420, fd = 1056, u32 = 1056, u64 = 1056}}, {events = 2460181185, data = {ptr = 0x43ffff8089, fd = -32631,
u32 = 4294934665, u64 = 292057743497}}, {events = 0, data = {ptr = 0x10, fd = 16, u32 = 16, u64 = 16}}, {events = 64, data = {ptr = 0x6d5c9d4000000000,
fd = 0, u32 = 0, u64 = 7880346346194993152}}, {events = 32630, data = {ptr = 0x2, fd = 2, u32 = 2, u64 = 2}}, {events = 8, data = {ptr = 0x5d, fd = 93,
u32 = 93, u64 = 93}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 110, data = {ptr = 0x77, fd = 119, u32 = 119, u64 = 119}}, {
events = 0, data = {ptr = 0x7f766d5c9d3f, fd = 1834786111, u32 = 1834786111, u64 = 140146617654591}}, {events = 1, data = {ptr = 0x2000000000000, fd = 0,
u32 = 0, u64 = 562949953421312}}, {events = 0, data = {ptr = 0x1000, fd = 4096, u32 = 4096, u64 = 4096}}, {events = 1834786143, data = {ptr = 0x100007f76,
fd = 32630, u32 = 32630, u64 = 4294999926}}, {events = 0, data = {ptr = 0x20000, fd = 131072, u32 = 131072, u64 = 131072}}, {events = 4096, data = {ptr = 0x0,
fd = 0, u32 = 0, u64 = 0}}, {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}, {events = 1610619392, data = {ptr = 0x600019f000007f76, fd = 32630,
u32 = 32630, u64 = 6917557546223959926}}, {events = 32630, data = {ptr = 0x200, fd = 512, u32 = 512, u64 = 512}}, {events = 1610612768, data = {
ptr = 0x2600007f76, fd = 32630, u32 = 32630, u64 = 163208789878}}, {events = 0, data = {ptr = 0x33, fd = 51, u32 = 51, u64 = 51}}, {events = 1943933519,
data = {ptr = 0x20000007f76, fd = 32630, u32 = 32630, u64 = 2199023288182}}, {events = 0, data = {ptr = 0x7f7660000020, fd = 1610612768, u32 = 1610612768,
u64 = 140146393481248}}, {events = 496, data = {ptr = 0x2000000000, fd = 0, u32 = 0, u64 = 137438953472}}, {events = 0, data = {ptr = 0x26, fd = 38, u32 = 38,
u64 = 38}}, {events = 1943939812, data = {ptr = 0x6d5c9e0800007f76, fd = 32630, u32 = 32630, u64 = 7880347205188484982}}, {events = 32630, data = {ptr = 0x8,
fd = 8, u32 = 8, u64 = 8}}, {events = 512, data = {ptr = 0x73de473200000000, fd = 0, u32 = 0, u64 = 8349189039265415168}}, {events = 32630, data = {
ptr = 0x20, fd = 32, u32 = 32, u64 = 32}}, {events = 1967206590, data = {ptr = 0x600008c000007f76, fd = 32630, u32 = 32630, u64 = 6917538648367857526}}, {
events = 32630, data = {ptr = 0x7f7660001a00, fd = 1610619392, u32 = 1610619392, u64 = 140146393487872}}, {events = 1610614976, data = {ptr = 0x1800007f76,
fd = 32630, u32 = 32630, u64 = 103079247734}}, {events = 0, data = {ptr = 0x7f76600008e0, fd = 1610615008, u32 = 1610615008, u64 = 140146393483488}}, {
events = 1967663553, data = {ptr = 0x2000007f76, fd = 32630, u32 = 32630, u64 = 137438986102}}, {events = 0, data = {ptr = 0x7f767545c73d <g_slice_alloc+1357>,
fd = 1967507261, u32 = 1967507261, u64 = 140146750375741}}, {events = 1610614992, data = {ptr = 0x68001c8000007f76, fd = 32630, u32 = 32630,
u64 = 7494021116025929590}}, {events = 32630, data = {ptr = 0x10, fd = 16, u32 = 16, u64 = 16}}, {events = 1610619744, data = {ptr = 0x7f76, fd = 32630,
u32 = 32630, u64 = 32630}}, {events = 0, data = {ptr = 0x128c720, fd = 19449632, u32 = 19449632, u64 = 19449632}}}
ev = 0x7f766d5c9888
e = {events = 0, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}
#9 0x00000000004076ad in poller_loop (d=0x1287d30) at main.c:553
p = 0x1287d30
#10 0x000000000040b8d0 in thread_detach_func (d=0x128ea60) at aux.c:160
dt = 0x128ea60
t = 0x7f76680014b0
#11 0x00007f767412d182 in start_thread (arg=0x7f766d5ca700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f766d5ca700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140146617657088, 4605709186135647645, 0, 0, 140146617657792, 140146617657088, -4537905887093824099,
-4537890921529449059}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#12 0x00007f7673e5a38d in eventfd_write (fd=<optimized out>, value=0) at ../sysdeps/unix/sysv/linux/eventfd_write.c:27
No locals.
#13 0x0000000000000000 in ?? ()
No symbol table info available.
from rtpengine.
rfuchs
commented on July 22, 2024
Can you post the output of rtpengine --version please
from rtpengine.
digipigeon
commented on July 22, 2024
ah ha, my apologies, you are correct in your assumption it appeared to be the original version.
However this time the port range was set correctly, so a port would still not have been assigned even if it didnt crash, any other recommendations on how to ensure port availability?
from rtpengine.
rfuchs
commented on July 22, 2024
There's not much else you can do, ports are a limited resource per machine and are required for RTP forwarding (4 per call in most cases), either increase the port range even more (make sure you don't run into conflicts) or put fewer calls on the proxy. Or run more RTP proxies and balance the calls between them.
from rtpengine.
digipigeon
commented on July 22, 2024
Hi Richard,
I feel that I must continue this thread as I am concerned about the limitations here. Prior to using rtpengine I have rtpproxy deployed, single instance per machine. I have 2 separate clusters running at present, one in the UK one in France each with 5+ servers, each server handles no more than 500 channels of traffic which in general carries the traffic fine.
When moving to RTPEngine, I experience a better and more consistant latency, which is great. However after a few hours of running, now instead of crashing it starts rejecting calls, this pushing back to opensips prevents call setup.
I have previously benchmarked the boxes to over 1000 calls each with rtpproxy. I suspect there may be something that is is not releasing ports, is there any way I can view the amount of ports used or channels that are currently active? so I can cross reference this with my opensips boxes?
Regards Jonathan
from rtpengine.
digipigeon
commented on July 22, 2024
Richard, hopefully this will be helpful:
Below is the tail fo the result for netstat -tulpn
udp6 0 0 :::52371 :::* 27257/rtpengine
udp6 0 0 :::51859 :::* 27257/rtpengine
udp6 0 0 :::51091 :::* 27257/rtpengine
udp6 0 0 :::49299 :::* 27257/rtpengine
udp6 0 0 :::49043 :::* 27257/rtpengine
udp6 0 0 :::48019 :::* 27257/rtpengine
udp6 0 0 :::47763 :::* 27257/rtpengine
udp6 0 0 :::44179 :::* 27257/rtpengine
udp6 0 0 :::43667 :::* 27257/rtpengine
udp6 0 0 :::43411 :::* 27257/rtpengine
udp6 0 0 :::38803 :::* 27257/rtpengine
udp6 0 0 :::38291 :::* 27257/rtpengine
udp6 0 0 :::31635 :::* 27257/rtpengine
udp6 0 0 :::31123 :::* 27257/rtpengine
udp6 0 0 :::29331 :::* 27257/rtpengine
udp6 0 0 :::26771 :::* 27257/rtpengine
udp6 0 0 :::56979 :::* 27257/rtpengine
udp6 0 0 :::56467 :::* 27257/rtpengine
root@rtp-lon-1:~# netstat -tulpn | grep udp | wc -l
32046
This is after I removed all traffic from the server 5 minutes prior, there is 30,000 udp6 ports here. I am not 100% sure of the exact relevance, but it seems a bit strange. I have IP6 addresses on my server, but they are not in use, I am utilizing IPv4 only.
from rtpengine.
rfuchs
commented on July 22, 2024
You really need to debug your signalling. You need to make sure calls are deleted properly at hangup, otherwise the ports will remain open. Rtpengine will handle timeouts (defaults to 60 seconds), but only if no packets are received on any of the pots belonging to a call. Seeing as your proxy sends packets to itself, there's something wrong with the RTP flow as well and the timeouts may never happen.
All this may be related to how the Opensips module sends the messages to rtpengine, perhaps it's doing something wrong there. I can't tell without more detailed information.
As for status reports, you can:
- Run rtpengine with the TCP protocol enabled (--listen-tcp), then issue the
statuscommand to that port (e.g. usingtelnetor perhapsecho status | nc localhost $PORT). The first line gives a summary (how many calls, packet counts), followed by a list of all active call-ids. Make sure you run the latest version (from today) if you do this. - Use
ng-client(from theutilsdirectory) to issue the NG protocollistcommand. Note that the number of call-ids returned by this is limited by the protocol. - External means, as you already did, through
netstat, by looking into the/proc/mediaproxy/$ID/statusand.../listfiles if you use kernel forwarding, and by watching the log file.
from rtpengine.
Related Issues (20)
- compile rtpengine failed on ubuntu 22.04 with error "undefined reference to `av_channel_layout_default'" HOT 4
- SRTP output wanted, but no crypto suite was negotiated HOT 1
- rtpengine mr10.5 crashed in __dtx_send_later HOT 2
- `silent-timeout` not working HOT 1
- for rhel 8.9 mr11.5 there is an compilation issue HOT 1
- rtpengine not support H264 HOT 3
- call.h:921:7: error: unknown type name ‘__auto_type
- kamailio rtpengine re-INVITE HOT 1
- kzalloc() return 0. HOT 6
- DKMS Rebuild modules failed HOT 2
- libg729 crash on rtpengine 9.5.2 HOT 1
- Wrong IPv session stats HOT 3
- ipv4 only media current sessions number wrong after restart HOT 1
- numsessions "current sessions own" and "ipv4 only media" almost always differ HOT 1
- RTP silent-timeou is not working HOT 3
- no log output HOT 1
- How to get linux/btree.h HOT 2
- SRTP with Kamailio Signalling HOT 1
- 如果在某些云主机上部署出现报错绑定网卡相关的问题，可以尝试使用这个仓库中的两个文件替换，属于魔改版
- SRTP Stream switches ports HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rtpengine.