arenaxr / arena-py Goto Github PK

Say we have this series of calls (in a loop, or something):

scene.update_object(box, click_listener=True)
scene.update_object(box, click_listener=True)
scene.update_object(box, click_listener=True)
scene.update_object(box, click_listener=True)
scene.update_object(box, click_listener=True)
etc...

Only the first call should actually do the publishing (make a network call), the rest should not do anything. The library should recognize that the click-listener of the box is already true and not do anything when updating.

The way position is scaled for child objects of the camera has changed.

• All children of the camera, including the ARB panel, and placeholder targets are positioned/scaled too small to use or out of camera frustum.

Get access to 3D text parameters in python. Would like to be able to change parameters like alignment etc

Tried data='{"text":{ "wrapCount":100, "value":"Definitely not working..." } } ' but the text gets replaced with [Object object] and the wrapCount parameter stays at default 40.

The animations of blocks falling/being launched and the avocado's animation do not work in guac.py, while the arm-demo's animation works. Changing the physics attribute to dynamic_body (exchangeable as indicated by the documentation) also doesn't work. After a win/stalemate, the boxes will hold their positions and freeze for 5 seconds after being reset into their default state instead of rendering the physics.

According to https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html
By Oct 3 2022.

Not sure what this would take in ARENA-core yet, but ideally this is how initial objects and models could be placed. Load an object in ARB clipboard and let it land on surfaces util the user is happy and clicks to place it.

Unauthorized Error when rotating screenshare object in scene "meeting" coming from arb.py

Full Traceback:

Traceback (most recent call last):
File "arb.py", line 997, in
arena.handle_events()
File "/Users/agr/Documents/Projects/ARENA/ARENA-py/tools/arb/arena.py", line 196, in handle_events
process_message(messages.pop(0))
File "/Users/agr/Documents/Projects/ARENA/ARENA-py/tools/arb/arena.py", line 115, in process_message
arena_callback(payload)
File "arb.py", line 969, in scene_callback
do_rotate_select(camname, objid)
File "arb.py", line 481, in do_rotate_select
pobjs = arblib.get_network_persisted_obj(objid, BROKER, SCENE)
File "/Users/agr/Documents/Projects/ARENA/ARENA-py/tools/arb/arblib.py", line 561, in get_network_persisted_obj
'https://' + broker + '/persist/' + scene + '/' + object_id).read()
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized

On client: listen to animationstart, animationend events and send those events through mqtt.
On arena-py: add hooks for those events.

https://github.com/conix-center/ARENA-py/blob/5e5150dd8e723915abbaaec68183a16a00cef852/examples/objects/landmarks.py

Goals:

• Central user manifest should have account name, display name, role/position, arbitrary badge names.
• It should allow overwriting of display name from central manifest to "add" role/position.
• It should allow adding arbitrary badges floating above user head.

Support:

• May require flag on account scene permissions to allow overwriting of display name.

Ref: https://pythonspeed.com/articles/faster-json-library/

Since we're only going to see more and more traffic, and arena-py is obligated to digest all of it

• Currently we use the google auth lib's console auth for managing a localhost server to serve as a callback for it's web auth.
• We would like to enable other auth providers or a simple ARENA account in the future through Python.
• To do this, we need to replicate the JWT exchange mechanism Google uses for any auth provider ARENA could provide.

1. New scene subtopics that users subscribe/publish to:

   <realm>/s/<namespace>/<scenename>/u/
   <realm>/s/<namespace>/<scenename>/g/
   <realm>/s/<namespace>/<scenename>/p/

2. Shorter proc: <realm>/proc -> <realm>/p
3. Marker detections go to: <realm>/d/public/m (replace <realm>/g/a)
4. Add userid to network latency topic
5. Event and camera messages refactor
6. Change ‘apriltag’ detection message type to ‘marker’
7. New ‘device_data’ type for device data
8. Change chat topics o=pub; p=priv

Details:
https://docs.google.com/presentation/d/1MRfo1IiIOiseMZ9BSAp-ksWmTouq9SW8RDAf8GKfnag/edit?usp=sharing

To Nuno's suspicion, I see that object.update( clickable=False ) after clickable is true for an object doesn't seem to work in arena.py

• We could try and catch this error, probe the port to provide a better error message.
• Using eduroam VPN on campus is enough to replicate and attempt to connect to a scene. It filters all ports but 80, 443.
• This is not useful:

Traceback (most recent call last):
  File "/Users/mwfarb/git/arena-py/tools/arb/arb.py", line 1394, in <module>
    scene = Scene(
  File "/Users/mwfarb/git/arena-py/arena/scene.py", line 179, in __init__
    self.mqttc.connect(self.host, port=8883)
  File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 937, in connect
    return self.reconnect()
  File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1071, in reconnect
    sock = self._create_socket_connection()
  File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection
    return socket.create_connection(addr, source_address=source, timeout=self._keepalive)
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
socket.timeout: timed out
sys:1: RuntimeWarning: coroutine 'PersistentWorker.run' was never awaited
sys:1: RuntimeWarning: coroutine 'AsyncWorker.run' was never awaited

We need to ad some better error messages when the user sets realm=None or host=None. ATM we are accepting them, using them in topics and urls, and it throws misleading network errors when attempting to connect.

If I leave high-traffic (i.e. head motion test) programs running for a long time, I tend to see this failure state like below. I haven't looked more deeply, but maybe we can add a reconnect retry? I wonder if this is related to the ARTS issue?

`Exception thrown in update()! Terminating update...
Traceback (most recent call last):
File "headbanger.py", line 40, in update
scene.update_object( head, rotation=(0, i, 0),)
File "/home/agr/.local/lib/python3.6/site-packages/arena/scene.py", line 446, in update_object
res = self._publish(obj, "update")
File "/home/agr/.local/lib/python3.6/site-packages/arena/scene.py", line 504, in _publish
self.mqttc.publish(topic, payload, qos=0)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 1274, in publish
local_mid, topic, local_payload, qos, retain, False, info, properties)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2563, in _send_publish
return self._packet_queue(PUBLISH, packet, mid, qos, info)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2911, in _packet_queue
self._sockpairW.send(sockpair_data)
TimeoutError: [Errno 110] Connection timed out

Exception thrown in network_latency_update()! Terminating network_latency_update...
Traceback (most recent call last):
File "/home/agr/.local/lib/python3.6/site-packages/arena/scene.py", line 161, in network_latency_update
self.mqttc.publish(self.latency_topic, "", qos=2)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 1304, in publish
message.dup, message.info, message.properties)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2563, in _send_publish
return self._packet_queue(PUBLISH, packet, mid, qos, info)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2911, in _packet_queue
self._sockpairW.send(sockpair_data)
BrokenPipeError: [Errno 32] Broken pipe

Exception thrown in misc_loop()! Terminating misc_loop...
Traceback (most recent call last):
File "/home/agr/.local/lib/python3.6/site-packages/arena/event_loop/asyncio_mqtt.py", line 44, in misc_loop
while self.client.loop_misc() == mqtt.MQTT_ERR_SUCCESS:
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 1631, in loop_misc
self._message_retry_check()
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2832, in _message_retry_check
self._out_messages, self._out_message_mutex)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2821, in _message_retry_check_actual
properties=m.properties,
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2563, in _send_publish
return self._packet_queue(PUBLISH, packet, mid, qos, info)
File "/home/agr/.local/lib/python3.6/site-packages/paho/mqtt/client.py", line 2911, in _packet_queue
self._sockpairW.send(sockpair_data)
BrokenPipeError: [Errno 32] Broken pipe

Disconnect error! Result code: 1`

debug=True should not remove verification to allow testing against self-signed certs, it is a confusing error and puts the connection in a less than ideal condition for the user.

Persist process and web client now support setting an object's attribute to null in the MQTT message payload. The python lib should be updated to reflect this feature.

When you ssh to a Raspbian terminal, auth does not assume headless mode and uses a local host callback instead of the headless one.

For google auth url address, use prompt=select_account to always show a list of accounts, even if only one is available, like arena-account.

Currently I am using scene.remote_auth_token['token'] to retrieve the token to pass to Jitsi. Now that I am moving to arts-runtime-native, which uses a local .arena_mqtt_auth token file, this does not work. We should maybe have something like a scene.auth_token variable that works no matter the auth method.

(mwfarb)

Causing problems on login

We could replace our current handle_events(): for something that waits for incoming messages (for example, using an event: https://docs.python.org/2/library/threading.html).

Current implementation might be consuming a lot of CPU and causing issues when we try to run several programs.

This is a short list of features to work for the next iteration of init3d:

• add 'found' apps in scene to program controls
• prevent init3d from being 'found'
• react to real-time creates and add to controls
• react to create confirm messages, to wait to enable stop buttons
• ...

Support being able to display multiple pages for a single poster kiosk.

Examples such as arm-demo.py throw error scene argument (scene) is unspecified or None, aborting..., and the parameter cli_args for creating a Scene object is not present in the Scene documents. According to the code in arena-mqtt.py the scene argument can be provided with -s or --scene, but providing it (for example, python arm-demo.py -s playground, using python instead of python3 since it's windows environment) still shows the same error. The argparse library is used to parse commandline arguments, but running python arm-demo.py -h shows the same error instead of showing an options list.

Also, is the discussions panel open for this repository? I feel this kind of questions is better asked in discussion instead of raising an issue. Thanks.

• Attempt to create a green line.
• This creates a white line:

scene.add_object(Line(
    start=Position(1, 1, 1),
    end=Position(2, 2, 2),
    material=Material(color=Color(0, 255, 0))))

• This creates a green line with a deprecated warning:

scene.add_object(Line(
    start=Position(2, 2, 2),
    end=Position(3, 3, 3),
    color=Color(0, 255, 0)))

See title. @EdwardLu2018

It seems remote_auth_token may only be set when a new authentication takes place: https://github.com/conix-center/ARENA-py/blob/3ae730a45d85ed75f0accb345f9b3beca79fb7a0/arena/scene.py#L130-L136

So it is an empty dict unless option 3 is taken:
https://github.com/conix-center/ARENA-py/blob/3ae730a45d85ed75f0accb345f9b3beca79fb7a0/arena/scene.py#L79-L95

I think for object primitives we should set the default color to (128,128,128) if not specified. Helps avoid confusion when people are look at the scene with the random initial colors now.

Handle onefingermove and twofingermove events. Then, ARB can listen for those events and use the data points included to apply rotate and scale for touchscreen cases.

Depends on arenaxr/arena-web-core#30.

Some Apple python env have not certs, and users see the urllib error:
err=URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')), type(err)=<class 'urllib.error.URLError'>
which is pretty confusing for many.

Instead, we can automate the cert check and use first part of this gist: https://gist.github.com/mwfarb/c45958c321f021b3b472248c1e4cfc5d

If it fails instead of this error we can direct users to: https://arena.conix.io/content/troubleshooting.html#python-unable-to-get-local-issuer-certificate

It's happened a few times now that users have set the realm option to something other than realm="realm" and that has caused confusion and authentication errors. Until we actually have another realm setup, we should remove it from our examples and document that it is a reserved parameter.

Library needs to silently fail rather than crash on errors.

We should totally make an office hour queue app that displays a board in the ARENA where TAs can manage office hour queues.

A few key features:

• Force users to write a twitter sized question
• Allow TAs to click on users in their own room to teleport them in
• TAs should be able to click on multiple users to cover conceptual group questions
• Screenshare with laser pointer support

Stretch goals:

• Students should be able to ask to be included if they have the same question (even if it isn't what they posted). Maybe a "same question" checkbox for other student's questions
• VS Code Google Doc like session

Since text entry isn't really a thing in the ARENA, maybe the queue is hosted as a web app somewhere that just has a physical object for seeing status and teleporting people within the ARENA?

It would be nice (certainly not a priority) if there was a way to create an object in arena-py that didn't actually set any parameters on initialization. One use-case would be setting a clickable callback on an object without having to get/maintain its state from the persistence DB. Right now, if you address an object by name it looks like the create function publishes default values in terms of type and location. This would move an existing object and/or change its type.

ATM mosquitto_pub and mosquitto_sub CLI require a full JWT to be passed on the command line which is too cumbersome to resolve for new users.

• We should write the equivalent programs in Python that use the authentication system and the user's JWT issued based on their ACL.
• It should probably accept the same standard arguments, accept for the username/password arguments.