Comments (7)
You should check the free5gc AMF logs, it will certainly indicate why SCTP sockets are getting closed or exhausted. Or eventually check the kernel logs in the machine running the AMF.
Regarding the GIL lock release in pysctp, I guess you don't need to test its removal as you solved your issue: it will only lower your overall performance on the Python side. And yes, You can close this issue.
from pysctp.
Thanks for submitting this issue.
Looking at your implementation, it seems you have 2 concurrent threads working on a single SCTP socket: _ngap_to_ue_thread_function
and _ue_to_ngap_thread_function
are both running in the background. I guess it may happen that you have send() and recv() calls which could overlap. As pysctp releases the GIL when calling sctp_sendmsg()
and sctp_recvmsg()
, this could potentially lead to such an error.
Otherwise, I don't have much ideas from the information provided. Eventually, you can try to recompile pysctp after commenting out the Py_BEGIN_ALLOW_THREADS
and Py_END_ALLOW_THREADS
macros in _sctp.c
.
As a side note, you should take care to manage SCTP stream ID consistently toward the core network: this ID must be 0 for system-level NGAP procedures, while >0 for UE-related procedures.
from pysctp.
Thank you @p1-bmu for the pointer. The concurrent access was likely the error. I added I/O multiplexing with select
and added locking for recv and send. I am now getting a new error ConnectionResetError
, which I am taking as the previous error was resolved by ensuring that the recv and send are only called one at a time. FYI please see the updated code below.
import socket
import logging
import sctp
import threading
import select
logger = logging.getLogger('__SCTPClient__')
# Create a lock to synchronize access to socket
socket_lock = threading.Lock()
class SCTPClient():
""" This class is a base class for all SCTP client contexts. """
def __init__(self, server_config):
try:
self.__socket = self._load_socket()
except:
logger.exception("Failed to create socket")
raise
# TODO: Allow connecting to multiple servers
self.server_config = server_config['amfConfigs'][0]
self.connect()
def connect(self) -> None:
logger.debug("Connecting to 5G Core")
logger.debug("Server config: {}".format(self.server_config))
self.__socket.connect((self.server_config['address'], self.server_config['port']))
def disconnect(self) -> None:
logger.debug("Disconnecting from 5G Core")
self.__socket.close()
def _load_socket(self):
# Create SCTP socket
return sctp.sctpsocket_tcp(socket.AF_INET)
def send(self, data: bytes):
# Acquire the socket lock before sending data
sock = self.__socket.sock()
while True:
readable, writable, exceptional = select.select([], [sock], [sock], 0.1)
if sock in writable:
break
socket_lock.acquire()
try:
self.__socket.sctp_send(data, ppid=socket.htonl(60))
finally:
# Release the socket lock after sending data
socket_lock.release()
def recv(self) -> bytes:
if self.__socket._closed:
return None
sock = self.__socket.sock()
readable, writable, exceptional = select.select([sock], [], [sock], 0.1)
if sock not in readable:
return None
# Acquire the socket lock before receiving data
socket_lock.acquire()
try:
sctp_data = self.__socket.recv(4096)
if not sctp_data:
logger.debug('SCTP disconnection')
return None
return sctp_data
finally:
# Release the socket lock after receiving data
socket_lock.release()
def close(self) -> None:
return super().close()
After resolving the new error below, I will go on to try the suggestion of recompiling the pysctp. I would appreciate any suggestions or pointers on the new error.
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/ubuntu/core-tg/src/NGAPSim.py", line 294, in _ue_to_ngap_thread_function
self.sctp.send(PDU.to_aper())
File "/home/ubuntu/core-tg/src/SCTP.py", line 51, in send
self.__socket.sctp_send(data, ppid=socket.htonl(60))
File "/home/ubuntu/core-tg/.venv/lib/python3.8/site-packages/sctp.py", line 1178, in sctp_send
return _sctp.sctp_send_msg(self._sk.fileno(), msg, to, ppid, flags, stream, timetolive, context)
ConnectionResetError: [Errno 104] Connection reset by peer
Also, thanks for the side note on the stream id, I will update accordingly.
from pysctp.
This certainly indicates that you call select() on the socket, or the socket method recv() or send(), while it has been closed by the remote peer.
from pysctp.
Hi @p1-bmu. Yeah that is correct. I am trying to see why the connection is being closed, and thereafter how to handle it gracefully. I used eBPF to monitor the SCTP connection and it looks like the received window is reduced to 0 bytes. I would like to believe that is why the connection is then reset on the core network side (FYI I am testing with free5gc, also see attached graph of the receive window stats).
Since the purpose of my current study is overloading the control network functions, I assume that's a positive result for now. I just have to be certain that it's the reason the connection is closed. I will feedback my findings here as well. As above I will appreciate any pointers.
Thanks for the insights. I assume I can close this issue, or should recompile pysctp with the suggested and check if that resolves the issue as well before closing the issue?
from pysctp.
Thanks @p1-bmu, I checked the logs on free5gc and the logs showing seem to indicated the connection was closed from the traffic generator side. However, my understating of the ConnectionResetError
as per my search is it indicates the peer closed the connection/ The logs that lead me to thinking free5gc see the connection being dropped on the other side are below
2023-08-10T16:15:54.514674135Z [WARN][AMF][Gmm][amf_ue_ngap_id:RU:207,AU:205(3GPP)][supi:SUPI:] T3560 expires, retransmit Authentication Request (retry: 3)
2023-08-10T16:15:54.514785161Z [INFO][AMF][Ngap][amf_ue_ngap_id:RU:207,AU:205(3GPP)][ran_addr:10.0.0.16:56174] Send Downlink Nas Transport
2023-08-10T16:15:54.515228581Z [ERRO][AMF][Ngap][ran_addr:10.0.0.16:56174] Send error: no such process
I there a way to get a notification with pysctp on a connection that is closing and why it's closing (Outside the socket error message received above)?
from pysctp.
Thanks @p1-bmu, I checked the logs on free5gc and the logs showing seem to indicated the connection was closed from the traffic generator side. However, my understating of the
ConnectionResetError
as per my search is it indicates the peer closed the connection/ The logs that lead me to thinking free5gc see the connection being dropped on the other side are below2023-08-10T16:15:54.514674135Z [WARN][AMF][Gmm][amf_ue_ngap_id:RU:207,AU:205(3GPP)][supi:SUPI:] T3560 expires, retransmit Authentication Request (retry: 3) 2023-08-10T16:15:54.514785161Z [INFO][AMF][Ngap][amf_ue_ngap_id:RU:207,AU:205(3GPP)][ran_addr:10.0.0.16:56174] Send Downlink Nas Transport 2023-08-10T16:15:54.515228581Z [ERRO][AMF][Ngap][ran_addr:10.0.0.16:56174] Send error: no such process
I there a way to get a notification with pysctp on a connection that is closing and why it's closing (Outside the socket error message received above)?
I am seeing that is I use stop_recv instead of recv I will get more information which I assume can be useful. I am going to try that out later to see if it helps. Thanks.
from pysctp.
Related Issues (20)
- sctp ppid value error HOT 4
- undefined symbol: PyString_FromStringAndSize HOT 5
- The sent packet cannot be parsed by wireshark HOT 2
- SCTP payload protocol ID argument issue HOT 2
- [Query] Received data len from sctp_recv API HOT 1
- [Query] PPID for received message? HOT 2
- bindx doesn't work with more than one address HOT 1
- SCTP keeps trying to socket-connect with a dummy IP for hours before it times out. HOT 5
- set_sndbuf doesn't work as expected HOT 2
- pypi version 0.7 is outdated HOT 3
- unable to run the code HOT 4
- Unable to install pysctp HOT 1
- How to configure source port for client HOT 1
- SCTP Server when receiving SCTP request from other subnet do not work? HOT 2
- Source address and add new address HOT 4
- CI/CD publishing to PyPi HOT 5
- Method for specifying the Payload Protocol Identifier for Data Chunk? HOT 1
- Byte-ordering is incorrect when trying to use ppid attribute of sctp_send HOT 2
- ppid field changed from 32 bits to 128 bits HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pysctp.