I'm trying to use simple_rpc interface over an RF link. It is a very slow procedure to load the interface definition (e.g. 2 min to load 29 function definitions). I was wondering if it might be possible to cache the interface definition to disk and re-use it (assuming it doesn't change).

using SerialInterface instead works fine.

pyserial supports telnet & socket connections as well as specifying device properties directly with hwgrep
https://pyserial.readthedocs.io/en/latest/url_handlers.html

can't say what the best approach is, but spy:// is very handy and could cause some confusion

Traceback (most recent call last):
  File "./rpctest.py", line 6, in <module>
    i = Interface("spy:///dev/ttyUSB0", 9600)
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 235, in __new__
    return SocketInterface(device, *args, **kwargs)
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 190, in __init__
    self.open()
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 196, in _auto_open_wrapper
    result = f(self, *args, **kwargs)
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 210, in open
    super().open()
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 105, in open
    self.methods = self._get_methods()
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 82, in _get_methods
    if self._read_byte_string() != _protocol:
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 64, in _read_byte_string
    return read_byte_string(self._connection)
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/io.py", line 90, in read_byte_string
    return _read_bytes_until(stream, _end_of_string)
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/io.py", line 15, in _read_bytes_until
    return b''.join(until(lambda x: x == delimiter, stream.read, 1))
  File "/home/ohsix/.local/lib/python3.8/site-packages/simple_rpc/io.py", line 120, in until
    result = f(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/serial/urlhandler/protocol_spy.py", line 205, in read
    rx = super(Serial, self).read(size)
  File "/usr/lib/python3.8/site-packages/serial/serialposix.py", line 483, in read
    ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())
KeyboardInterrupt

thanks! this library is very useful

Describe the bug
Type hinting for *args and **kwargs in function definitions need to specify the type of the element contained, rather than the type of the *args and **kwargs. See this stackoverflow post: https://stackoverflow.com/questions/37031928/type-annotations-for-args-and-kwargs and the referenced PEP

To Reproduce
Run static type checker (e.g mypy/pylance/etc.) on code when calling e.g, call_method(str, bytes) and notice reported error.

Expected behavior
*args and **kwargs fields should be type hinted with the type of at least one of the contents. In the case of this project, at least for call_mehtod() I presume *args should be type hinted with Any, or maybe Union(bytes, int, float, "rest-of-simpleRPC-types")
Example of change

From:
def call_method(self: object, name: str, *args: list) -> Any:
To:
def call_method(self: object, name: str, *args: Any) -> Any:

Is your feature request related to a problem? Please describe.
I have found myself needing timeouts on RPC calls to safeguard from spotty connections or otherwise be more responsive on the calling side to other tasks that are happening. I have need of it because I am using a Teensy3.2 to manage power (more involved than checking battery levels. There are multiple active systems) and control low-level peripherals. This is connected to a Tx2 serving real-time data. Since the uptime of serving the real time data is paramount I can't afford to be stalled by an RPC call.

This is a quality of life request that I imagine only affects people who have strict timing requirements on the calling side or need to be tolerant to faulty connections.

Describe the solution you'd like
I believe a timeout feature can be "easily" enabled by modifying serial.serial_for_url() to include the keywords timeout and write_timeout which would be passed in as part of the simple_rpc._Interface() constructor. Note that timeout = write_timeout = None is the default and current behavior of write() and read() calls which are blocking.

While both are easily enabled by adding them to the constructor, this is likely meaningless without additional modifications to the rest of the code

I am not 100% sure how that propagates throughout the rest of the code but I believe I have summarized a potential solution below.

A read timeout can produce incomplete data and the only way to know is to compare the received data with the requested (see pyserial/pyserial#18 and pyserial/pyserial#108), i.e:

b = serial.read_until(delimiter)
if b[-1] != terminator and self.timeout is not None:
    do something

OR Alternatively

b = serial.read(num_bytes)
if len(b) != num_bytes and self.timeout is not None:
    do something

I am guessing since simple_rpc has a defined protocol, with known lengths and delimiters, the two approaches above cover most scenarios I see in io.py. It would then be a matter of perhaps throwing some kind of exception (self defined or otherwise) that the user can catch and decide how to handle this. That said the current form of the (io.py > _read_bytes_until() and io.py > until()) function would likely need to be re-thought due to the while True. Is a simple call to serial.read_until(delimiter) sufficient to replace them?

A write timeout is a different but perhaps simpler. In this instance a serial.SerialTimeoutException is raised which can be caught and re-raised as a builtin exception or handled in any other which way.

Note that this does unfortunately mean passing around a timeout flag everywhere, at least for the read() calls. The write() can always be wrapped in a try except block since it has minimal overhead and the timeout behavior is encoded in the pyserial object anyway.

Lastly there is the question of the initial connection after an open(). This can take longer than a defined timeout for any other call due to the exchange of more data. The two options I see here are either ignore the timeout for the open call (self._connection.timeout = None) and restore it once done, or allow a user to pass a startup_timeout value to the constructor.

Describe alternatives you've considered
An alternative that I will be implementing in the short run is a 2-thread solution:

thread-main: run general code and initiate request for RPC to thread-1
thread-1: listen for RPC request, launch thread-2 calling appropriate function and monitor runtime of thread.
                If runtime exceeds timeout, kill thread-2.
thread-2: run simple_rpc.call_method("method", param_list)

Regardless thank you for your time in reading this and hopefully it is useful

The Interface class is currently designed to work with serial ports that remain open for long periods of time. This approach is causing the Arduino ethernet server to get stuck in the communications while loop when waiting for the port to close. It would be nice for ethernet connections to open / close frequently to allow the Arduino to do other things when using ethernet or WiFi.

Describe alternatives you've considered

1. Add an option to keep message definitions instead of deleting them when the port closes. This is the least invasive change but makes the python API somewhat complex:

with Interface('socket://192.168.1.50:10000', wait=0, reuse_methods=True) as interface:
    print(interface.ping(1))

2. Same as option 1 but change Interface defaults (e.g. wait=0 and reuse_methods=True by default). This could break existing functionality but would result in a cleaner python API.

with Interface('socket://192.168.1.50:10000') as interface:
    print(interface.ping(1))

3. Modify Interface class to open / close ports frequently (including serial ports). This would be a heavy modification of the Interface class, would remove the open / close functions and internally use with serial_for_url(device) to open / close the port for each read / write operation. This would cause several breaking changes but would result in the simplest python API.

interface = Interface('socket://192.168.1.50:10000')
print(interface.ping(1))

Describe the solution you'd like
@jfjlaros I haven't selected a solution, maybe will prototype options 1 and 3 and see what you think after that.

Importing the module via import simple_rpc triggers the warning

/home/xyz/.local/lib/python3.6/site-packages/simple_rpc/__init__.py:9: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/xyz/.local/lib/python3.6/site-packages/simple_rpc/setup.cfg' mode='r' encoding='UTF-8'>
  config.read_file(open('{}/setup.cfg'.format(dirname(abspath(__file__)))))

The config file /home/xyz/.local/lib/python3.6/site-packages/simple_rpc/setup.cfg exists and contains:

[metadata]
name = arduino-simple-rpc
version = 2.0.1
description = Arduino simpleRPC API client library and CLI.
long_description = file: README.rst
author = Jeroen F.J. Laros
author_email = [email protected]
url = https://arduino-simple-rpc.readthedocs.io
keywords =
license = MIT
classifiers =
    Programming Language :: Python :: 3
    Programming Language :: Python :: 3.5
copyright = 2018

[options]
packages = find:
install_requires =
    configparser
    pyserial

[options.package_data]
simple_rpc = setup.cfg

[options.entry_points]
console_scripts =
    simple_rpc = simple_rpc.cli:main

I am using arduino-simple-rpc version 2.0.1, Python 3.6 on Ubuntu 18.04.

Describe the bug
Let me preface this by saying that I have very little experience with C/C++/Python, so this is most likely just me not understanding something rather than a bug.

I am attempting to import methods from a class exported with the simpleRPC libary following the instruction on exporting class methods here from a Teensy 3.6, but the presence of a period "." in the name causes Python to throw a syntax error. I suppose this is because Python doesn't want periods in method names, but this is how C++ refers to methods.

To Reproduce
Steps to reproduce the behaviour:

On the Arduino:

Here foo.h is a class header file and bar is a method in foo.

#include <simpleRPC.h>
#include <foo.h>

StreamIO io;
foo foo1;

void setup() {
  Serial.begin(9600);
  io.begin(Serial);
}

void loop() {
  interface(io, pack(&foo1, &foo::bar));
}

Then, on the computer connected to the Arduino via USB Serial, in the python3 shell:

>>> from simple_rpc import Interface
>>> interface = Interface('/dev/ttyACM0')

Traceback
...
File "<string>", line 2
  def foo1.bar():
          ^
SyntaxError: invalid syntax

(the ^ is pointing at the .)

Expected behavior
The function is imported correctly.

Additional context
I don't think it matters, but in my scenario, foo is a templated class and I'm using a Teensy.

I tried just changing the dot to an underscore and ran into a different issue that might be unrelated, but in any case I'm guessing that doing that will make it try to call the wrong method anyways unless I told it to change the underscore back to a dot when the RPC is made.

The actual code I am running is here:
https://github.com/drbeefsupreme/qyron/tree/arduino-rpc

If you have platformio installed you should be able to get it working without too much trouble - just change the platformio.ini file https://github.com/drbeefsupreme/qyron/blob/arduino-rpc/teensy/platformio.ini in the teensy folder to refer to github simpleRPC library rather than the local one (which at time of writing is the same as the github one). But you probably also need to be using a Teensy 3.6 for it to compile.

The actual error message:

>>> interface = Interface('/dev/ttyACM0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/chyron/.local/share/virtualenvs/rpi-cQqRlRR_/lib/python3.7/site-packages/simple_rpc/simple_rpc.py", line 37, in __init__
    self.open()
  File "/home/chyron/.local/share/virtualenvs/rpi-cQqRlRR_/lib/python3.7/site-packages/simple_rpc/simple_rpc.py", line 120, in open
    self, method['name'], MethodType(make_function(method), self))
  File "/home/chyron/.local/share/virtualenvs/rpi-cQqRlRR_/lib/python3.7/site-packages/simple_rpc/extras.py", line 52, in make_function
    context)
  File "<string>", line 2
    def scrollingLayer1.start(self, a, arg1):
                       ^
SyntaxError: invalid syntax

Any pointers would be appreciated. Thanks!

Describe the bug
This "bug" relates mostly to how type hints are handled. It is most problematic for static type checkers like mypy or pyright/pylance. By annotating variables or return values as having type any instead of typing.Any results in the type checker thinking the variable/return value should be of type "(__iterable: Iterable[object]) -> bool" i.e the type of the built-in Python function any

To Reproduce
Run static type checker on code using the simpleRPC library. For example:

value = rpc_interface.call_method("something") / 1000

Should produce a type checking error of
Operator "/" not supported for types "(__iterable: Iterable[object]) -> bool" and "Literal[1000]"
Notice that "(__iterable: Iterable[object]) -> bool" is the type hint for the built-in function any that takes in an iterable and returns a bool

Solution
To resolve change the signature of call_method from:
def call_method(self: object, name: str, *args: list) -> any:
to:
def call_method(self: object, name: str, *args: list) -> typing.Any:

I imagine it is a relatively easy fix for the rest of the variables/return values as well

Thank you for the easy to use library btw!

Describe the bug
When an Arduino function with return type Tuple<T,S> (e.g Tuple< int, const char*>) They Python library fails with ValueError: top level type can not be tuple. If instead an Object<T,S> is returned the system works fine.

This may be purely a documentation issue (or even PEBCAK) rather than a "bug" since I can't seem to find anywhere where it mentions Tuples cant be returned "bare". The only hint I can find is here and here where it is mentioned Objects<> preserve internal structure.

NOTE: This is with Python3.9 on a Windows system. No Linux system handy to test right now but I am 99% certain that it is not an OS level issue.

The Traceback is:

File "test_tuple_rpc.py", line 5, in <module>
    a = ...\simple_rpc.SerialInterface("COM12")
  File "simple_rpc\simple_rpc.py", line 58, in __init__     
    self.open(load)
  File "...\simple_rpc\simple_rpc.py", line 208, in open        
    super().open(handle)
  File "...\simple_rpc\simple_rpc.py", line 149, in open
    self._get_methods()
  File "...\simple_rpc\simple_rpc.py", line 123, in _get_methods
    method = parse_line(index, line)
  File "...\simple_rpc\protocol.py", line 124, in parse_line
    method = _parse_signature(index, signature)
  File "...\simple_rpc\protocol.py", line 69, in _parse_signature
    method['return']['fmt'] = _parse_type(fmt)
  File "...\simple_rpc\protocol.py", line 31, in _parse_type
    raise ValueError('top level type can not be tuple')
ValueError: top level type can not be tuple

To Reproduce
Steps to reproduce the behaviour:

Load test_tuple_rpc.ino on Arduino:

#include "simpleRPC.h"

void setup() {
    Serial.begin(9600);
    while(!Serial) {}
}

//Object<int, const char*> test_func()
//{
//    Object<int, const char*> t ={0, "OFF"};
//    return t;
//}

Tuple<int, char> test_func()
{
    Tuple<int, char> t ={0, "O"};
    return t;
}

void loop() {
    interface(Serial, test_func, "test_func:");
}

Run test_tuple_rpc.py on desktop machine:

a = simple_rpc.SerialInterface("COM12")
b = a.test_func()
print(b)
a.close()

Expected behavior
If it is expected that Tuples can't be top level return types on their own then perhaps that needs to be explicitly mentioned in the docs.

If Tuples are expected to be top level return types then perhaps the python code could "cast" them to Object on the python side? That second option sounds like it would break stuff. Alternatively Tuple and Object could be merged to a single type (if so I personally prefer the Tuple name to match the C++ equivalent of Vector)

WiFi remote procedure calls were running very slowly. The root cause is that the serial_for_url has a 0.3 second sleep in its close function (in pyserial protocol_socket.py file). This ensures each remote procedure call will take longer than 0.3 seconds when using simple_rpc.SocketInterface. Using the simple_rpc.SerialInterface does not have this delay for ethernet or wifi connections but using this interface prevents the microcontroller from doing other things besides communicating in the main loop.

Some ideas for workarounds:

• Request the offending sleep call from the pyserial library be removed
• Use mock.patch to bypass sleep (hack)
• Reimplement the SocketInterface class to use a socket instead of serial.serial_for_url
• Use a SerialInterface for everything and make it the responsibility of the Arduino to periodically exit and do other things

Module hangs on _get_methods()
I have Arduino UNO R3 with HDC1080 sensor, code:

#include <simpleRPC.h>
#include <Wire.h>
#include "ClosedCube_HDC1080.h"


HardwareSerialIO io;
ClosedCube_HDC1080 hdc1080;

double readTemperature(void) {
  return hdc1080.readTemperature();
}

double readHumidity(void) {
  return hdc1080.readHumidity();
}

void setup(void) {
  Serial.begin(9600);
  io.begin(Serial);
  hdc1080.begin(0x40);
}

void loop(void) {
  interface(
    io,
    readTemperature, F("readTemperature: Read temperature from hdc1080 sensor. @return: double."),
    readHumidity, F("readHumidity: Read humidity from hdc1080 sensor. @return: double."));
}

Module hangs when I trying to get instance of interface('/dev/ttyACM0'). When I pressed Ctrl + C:

>>> from simple_rpc import Interface
>>> interface = Interface('/dev/ttyACM0')
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 37, in __init__
    self.open()
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 117, in open
    self.methods = self._get_methods()
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 81, in _get_methods
    if self._read_byte_string() != _protocol:
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/simple_rpc.py", line 62, in _read_byte_string
    return read_byte_string(self._connection)
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/io.py", line 98, in read_byte_string
    return _read_bytes_until(stream, _end_of_string)
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/simple_rpc/io.py", line 18, in _read_bytes_until
    char = stream.read(1)
  File "/home/stark/tmp/arduino/venv/lib/python3.8/site-packages/serial/serialposix.py", line 565, in read
    ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())
KeyboardInterrupt

To Reproduce
Steps to reproduce the behaviour:

1. Open python console
2. Import Interface from simple_rpc
3. Create Interface instance
4. It hangs

Expected behavior
It creates interface instance

Screenshots

Desktop (please complete the following information):

• OS: Arch Linux
• simpleRPC 3.1.0
• Python 3.8.6
• arduino-simple-rpc==2.0.1
• configparser==5.0.1
• pyserial==3.5

Additional context
I get interface instance if I try again after Ctrl + C

Is your feature request related to a problem? Please describe.
While working in C, its easier to work on ioctl than to interface it with external python code talking to serial port.

Describe alternatives you've considered
Currently i am using a bus between my C and python code which talks to serial port.

It would be nice to add ethernet support.