miccoli / pyownet Goto Github PK

Hello, I was trying out pyownet instead of the old ownet module and got this error:

from pyownet import protocol
proxy = protocol.proxy('192.168.1.8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyownet/protocol.py", line 653, in proxy
    owp._init_errcodes()
  File "pyownet/protocol.py", line 408, in _init_errcodes
    m for m in bytes2str(self.read(PTH_ERRCODES)).split(','))
  File "pyownet/protocol.py", line 471, in read
    ret, data = self.sendmess(MSG_READ, str2bytez(path), size=size)
  File "pyownet/protocol.py", line 422, in sendmess
    ret, _, data = conn.req(msgtype, payload, flags, size, offset)
  File "pyownet/protocol.py", line 334, in req
    fromhead, data = self._read_msg()
  File "pyownet/protocol.py", line 372, in _read_msg
    "instead of {1:d}".format(len(payload), header.payload))
pyownet.protocol.ShortRead: short payload, got 1424 bytes instead of 4983

Owserver version is: 2.8p15-1 from debian stable (actually raspbian)

This "fixes" it, but obviously there is something else going on:

@@ -403,10 +405,11 @@ class _Proxy(object):
     def _init_errcodes(self):

         # fetch errcodes array from owserver
+        print ('get errcodes')
         try:
             self.errmess = _errtuple(
                 m for m in bytes2str(self.read(PTH_ERRCODES)).split(','))
-        except OwnetError:
+        except (OwnetError, ShortRead):
             # failed, leave the default empty errcodes
             pass

I added a print (payload) to _read_msg and got this:

proxy = protocol.proxy('192.168.1.8', verbose=True)
('192.168.1.4', 50429) -> ('192.168.1.8', 4304)
-> _ToServerHeader(version=0, payload=0, type=1, flags=0, size=0, offset=0)
.. ''
<- _FromServerHeader(version=0, payload=0, ret=0, flags=0, size=0, offset=0)
('192.168.1.4', 50429) xx ('192.168.1.8', 4304)
('192.168.1.4', 50430) -> ('192.168.1.8', 4304)
-> _ToServerHeader(version=0, payload=32, type=2, flags=0, size=65536, offset=0)
.. '/settings/return_codes/text.ALL\x00'
<- _FromServerHeader(version=0, payload=4983, ret=4983, flags=0, size=4983, offset=0)
Good result,Startup - command line parameters invalid,legacy - No such entity,Startup - a device could not be opened,legacy - Interrupted,legacy - IO error,Startup - network could not be opened,Startup - Avahi library could not be loaded,Startup - Bonjour library could not be loaded,legacy - Bad filesystem,Startup - zeroconf communication problem,legacy - Temporary interruption,legacy - memory exhausted,legacy - Access error,legacy - Communication fault,Startup - memory could not be allocated,legacy - Busy,Program - not initialized,Program - not yet ready,legacy - No such device,legacy - Not a directory,legacy - Is a directory,legacy - Invalid transaction,Program - closing,Program - closed,Program - cannot close,Path - input path too long,Path - bad path syntax,Device - Device name bad CRC8,Device - Unrecognized device name or alias,legacy - Read-only file system,Device - alias name too long,Device - unrecognized device property,Device - device property not an array,legacy - Out of range,Device - device property should be an array,legacy - Name too long,Device - device not a bit field,Device - array index out of range,Device - device property not a subpath,legacy - Loop discovered,Device - device not found,legacy - No message,Device - other device error,Bus - bus short,Bus - no such bus,Bus - bus not appropriate,Bus - bus not responding,Bus - bus being reset,Bus - bus closed,Bus - bus could not be ope

Seems like the response is cut short for some reason.

I'm using a periodic signal handler in my program and bumped into this with another socket connection. Then it occured to me that this could be the cause of the occasional errors I have seen from the pyownet communication.

This short program shows the issue:

from pyownet import protocol
import signal, time

def periodic_signal_handler(signum, frame):
    print 'sig'

signal.signal(signal.SIGALRM, periodic_signal_handler)
signal.setitimer(signal.ITIMER_REAL, 2, 0.01)
p = protocol.proxy('192.168.1.7')

while True:
    p.dir()
    time.sleep(0.2)

typical output:

python errnotest.py 
sig
sig
sig
sig
sig
sig
sig
sig
Traceback (most recent call last):
  File "errnotest.py", line 14, in <module>
    p.dir()
  File "/usr/local/lib/python2.7/dist-packages/pyownet/protocol.py", line 500, in dir
    ret, data = self.sendmess(msg, str2bytez(path), flags)
  File "/usr/local/lib/python2.7/dist-packages/pyownet/protocol.py", line 467, in sendmess
    raise ConnError(*err.args)
pyownet.protocol.ConnError: [Errno 4] Interrupted system call

Some background info I found:
http://stackoverflow.com/questions/16094618/python-socket-recv-and-signals
http://250bpm.com/blog:12

Seems like python does not reastart the syscall on EINTR even though it should, so it has to be handled in the program by looping on the syscall until it ends with something other than EINTR:

while True:
    try:
        data = s.recv(2048)
    except socket.error, e:
        if e.errno != errno.EINTR:
            raise
    else:
        break

(Ubuntu 14.04 with python 2.7.6)

The code for sending messages in protocol._OwnetConnection._send_msg is at present unsatisfactory.

After

sent = self.socket.send(header + payload)

a ShortWrite exception is raised if sent < len(header + payload) but it is not clear to me whether this socket.send should instead be retried in a while sent < len(header + payload): loop. If a retry loop is implemented, which are the conditions to break out of the loop to avoid deadlock?

A minor glitch to correct: the concat string header + payloadis constructed two times, one for sending, and one for measuring its length. Should optimise here.

Any ideas why this is all of a sudden happening on a new RPI3B. I had everything running fine a week ago on an older RPI3B

from pyownet import protocol
owproxy = protocol.proxy(host="localhost", port=4304)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.4/dist-packages/pyownet/protocol.py", line 727, in proxy
raise ConnError(*lasterr)
pyownet.protocol.ConnError: [Errno 111] Connection refused

They dropped support for use_2to3 in setuptools==58, package will fail to build:

       > error in pyownet setup command: use_2to3 is invalid.

version: 0.10.0.post1

In my owfs system I have 7 sensors:

ls /mnt/1wire/
05.4AEC29CDBAAB  10.120169020800  10.1E0B69020800  10.31DE68020800  10.67C6697351FF  10.B50F69020800  10.D80669020800 

but pyownet under Python 3.4 see only 2

Python 3.4.2 (default, Dec  3 2014, 09:03:21)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyownet import protocol
>>> owproxy = protocol.proxy(host="192.168.7.21", port=4304)
>>> owproxy.ping()
>>> owproxy.dir()
['/10.67C6697351FF/', '/05.4AEC29CDBAAB/']

How to debug this issue ?

We want to use ownet from a proprietary Python program, and thus can not accept the GPL license. Would you consider relicensing pyownet? Feel free to close the bug right away if you want to keep it GPL.

I wrote a program to read sensor data from multiple RPis with OWFS and the latest pyownet. The reading repeats itself with the help of an threading timer every 1 to 3 seconds (similar to https://stackoverflow.com/questions/12435211/python-threading-timer-repeat-function-every-n-seconds/24488061#24488061). After an irregular number of iterations the program fails with the error:

Exception in thread Thread-76:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 1180, in run
    self.function(*self.args, **self.kwargs)
  File "/home/elias2/Schreibtisch/test.py", line 42, in _handle_target
    self.target()
  File "/home/elias2/Schreibtisch/test.py", line 98, in pi_to_sql
    sensorTemp = float(owproxy0.read(sensor + 'temperature'))
  File "/usr/local/lib/python3.5/dist-packages/pyownet/protocol.py", line 615, in read
    raise OwnetError(-ret, self.errmess[-ret], path)
pyownet.protocol.OwnetError: [Errno 22] legacy - Invalid transaction: '/28.EE3FE02C1602/temperature'

As I said before, the number of Exception in thread Thread-76: is variable.
I connect to owserver with

owproxy0 = protocol.proxy(host=host0, port=owport, flags=0, persistent=False, verbose=False)
owproxy1 = protocol.proxy(host=host1, port=owport, flags=0, persistent=False, verbose=False)
...

and also tried

owproxy0 = protocol.proxy(host=host0, port=owport)
owproxy1 = protocol.proxy(host=host1, port=owport)
...

which causes the same issue. After that I can restart the code and after some time it will throw the same error.

On the single board computers Raspbian Jessie, kernel version 3.4.112 and OWFS 2.9p8 are used. The program runs on Ubuntu 16.04 LTS.

Thank you in advance for your answer.

protocol._OwnetConnection.__init__ enables socket timeouts, but there is no code to catch socket.timeout exceptions. There are possible scenarios in which pyownet will abort an operation ungracefully with an OSError.

Socket operation timeouts should be handled by pyownet and pyownet.protocol.ConnError raised.

• write test case to confirm bug: not confirmed
• implement appropriate exception handler
• update docs at http://pyownet.readthedocs.org/en/latest/protocol.html#pyownet.protocol.ConnError

In version v0.9.0, OSError exception originating inside _OwnetConnection are not handled and are caught in _Proxy, _PersistentProxy. This is error prone and causes some ambiguity in the error handling.

OSError exception handling should be moved inside _OwnetConnection to keep low level networking issues encapsulated inside this class.