matrix-org / matrix-python-sdk Goto Github PK

We notify only events in response['rooms']['join'], but there are also response['rooms']['invite'] and response['rooms']['leave']. See code at

Hi,

When I try to use your sdk via navi, or when I try to launch the sample:

$ PYTHONPATH=. python samples/SimpleChatClient.py

I provide my host, the username and password, and I get the following error:
Traceback (most recent call last):
  File "samples/SimpleChatClient.py", line 84, in <module>
    main(host, username, password, room_id_alias)
  File "samples/SimpleChatClient.py", line 39, in main
    client.login_with_password(username, password)
  File "/matrix-python-sdk/matrix_client/client.py", line 179, in login_with_password
    token = self.login_with_password_no_sync(username, password)
  File "/navi/navi/navi/matrix-python-sdk/matrix_client/client.py", line 156, in login_with_password_no_sync
    "m.login.password", user=username, password=password
  File "/navi/navi/matrix-python-sdk/matrix_client/api.py", line 128, in login
    return self._send("POST", "/login", content)
  File "/navi/navi/navi/matrix-python-sdk/matrix_client/api.py", line 545, in _send
    verify=self.validate_cert
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 453, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

Is there anything special that needs to be done in my matrix configuration to avoid this ? I have created a normal user to be used has bot. I can login via Riot without issue with this user.

Thanks for your help !

Since pull request #32 was merged, client._sync does not notify listeners registered on the client, but only those registered on the related room.

I personally find this behavior a little counter-intuitive.

There are two solutions:

• remove/deprecate client.add_listener (to make clear that it won't work)
• make client._sync notify listeners on the client for every event

I would be very happy to submit a pull request after we decide what's better. Thanks :)

i have long running bot made of the matrix python client and in past couple of weeks, it would randomly (often overnight) stop receiving incoming traffic with this error message (sorry, i have screenshot only):

The error is too deep in threading for me to understand :/ , sorry. Interestingly enough, this started repeatedly about week ago , this last night it didn't happen....
I am reachable @vanous:matrix.org .
thank you

We currently only use state events to set values in a room. The code should have a listener for all state events so clients can react to it.

Hi!
I'm trying to catch user presence changes, but i don't see this events.
I'm using

client = MatrixClient(config.MATRIX_URL)
client.login_with_password(config.MATRIX_USER, config.MATRIX_PASSWORD)
client.add_listener(all_messages)
client.start_listener_thread()

and

def all_messages(event):
if event['type'] == "m.room.member":
if event['membership'] == "join":
print("{0} joined".format(event['content']['displayname']))
elif event['type'] == "m.room.message":
if event['content']['msgtype'] == "m.text":
if event['sender'] != client.user_id:
# room = client.join_room(event['room_id'])
# room.update_aliases()
# print(repr(matrix_api.get_room_state(room.room_id)))
# print(repr(room.aliases))
evs = matrix_api.get_room_state(event['room_id'])
res = [e.get('content', {}).get('alias') for e in evs if e.get('content', {}).get('alias') is not None]
send_to = parseint(res[0])
facebook.send_message(send_to, event['content']['body'], config.FB_TOKEN)
elif event['type'] == "m.presence":
print('User status changed')

And this catch all events like chat messages, room join/leave but nothing about user status.
One place where i'm able to find user presence -

matrix_api = MatrixHttpApi(config.MATRIX_URL, token)
matrix_api.sync()

Pls advice.

The client is missing a calculateRoomName function:

c++ libmatriclient

or js matrix-js-sdk

MatrixClient's _sync method currently tracks m.room.name,
m.room.topic, and m.room.aliases state events using fields on Room
class. We should probably be tracking other state information in the
same way (e.g. PLs and required PLs).

https://pypi.python.org/pypi/matrix-client/ which is from the readme, does not contain the login_with_password method from client.py. Pls update.

The Room.update_aliases method does not take into account that there can be multiple m.room.aliases states. Because room aliases from separate HSes are in separate m.room.aliases states (with the HS name as the state key), this means the update_aliases function will actually set the aliases list to only contain the aliases from one HS, disregarding the others.

Example (the code should list all aliases for Matrix HQ):

>>> client.rooms['!cURbafjkfsMDVwdRDQ:matrix.org'].update_aliases()
True
>>> client.rooms['!cURbafjkfsMDVwdRDQ:matrix.org'].aliases
['#matrix:utzutzutz.net']

It seems it's not possible to send markdown messages as of today.

Will you consider adding support for it, or is there any workaround (like converting markdown to html automatically then use send_html?

For example, using org.matrix.custom.html format:

{
  "origin_server_ts": 1498105352663,
  "sender": "@XXXXX.com",
  "event_id": "$XXXXX.com",
  "unsigned": {
    "age": 121075294
  },
  "content": {
    "body": "**Message**",
    "msgtype": "m.text",
    "formatted_body": "<strong>Message</strong>",
    "format": "org.matrix.custom.html"
  },
  "type": "m.room.message",
  "room_id": "ZZZZZZ.com"
}

When a client instance logs in / registers, the response contains, among other things, a home_server field. Currently, this information is stored as a .hs member of the instance. However, if the MatrixClient is created with an exiting token, no such registration/login takes plave, hence no response is received, and the instance doesn't set this member -- which means that code can't rely on its being there.

At the moment, this information isn't used anywhere in the code, so probably the best thing to do is to just remove it.

Several methods in MatrixClient are described as deprecated in their docstrings. We should also be calling warnings.warn from within the methods.

As discussed at and around https://matrix.to/#/!XqBunHwQIXUiqCaoxq:matrix.org/$1483970675924545nrAds:matrix.org, it would be helpful to add a method to the Room class for sending HTML-formatted messages.

Putting

def send_html(self, html):
    self.client.api.send_message_event(room_id, "m.room.message", self.client.api.get_html_body(html))

into room.py reportedly works

Unit testing would be nice to ensure nothing gets broken and Jenkins can automate it for us.

When I try to use an global var inside an EventListener it is recognized as the correct type but seems to be unable to call his functions. Is there a way to fix this?

In matrix_client/api.py line 521:

        if headers["Content-Type"] == "application/json":
            content = json.dumps(content)

When using GET APIs such as sync(), content=None is passed to _send, but then content becomes a string 'null', so there will be a 'Content-Length: 4' in the request header. In some HTTP server such as lighttpd, it will return a 400 error when there is a Content-Length header in GET method.

Is there a way to configure the encryption? In the Riot app, messages sent via Python show a unlocked symbol.

Old code in api.py just needs replacing to be up to date with the C->S spec.

Following conversation on #43:

Happy with #44 for keeping old functionality.

Now, onto the other issues:

there are so many different ways to divide the events up into groups that one may want to listen to. For example, currently per-room and per-client listeners are supported (modulo this bug report); the docstring ("Incoming event callbacks") seems to also indicate per-user listeners.

Regrettably, we do not have per user listeners. Users are 'fun' because we can either have profile wide changes or room wide changes to them (via m.room.member events which usually but not always are global changes, so context is everything!) .

I find that behaviour pretty simple and intuitive, but we could be more granular adding arguments to the two add_listener functions (like event_type='m.room.message' or event_family='state')

I like this. It might be the best way to do it because I think specific functions for each type is ridiculous. Potentially you might want to add a few wrappers for things like topic, but I'm not fussed. Event family is probably overkill, add_state_listener and add_listener would be quite fine. I also would add an option for exclusivity if people want it.

You'd end up with something like:
add_state_listener(func, event_type=None, exclusive=False)

This is the issue for looking at implementing https://matrix.org/git/olm in the SDK. It looks like the lib includes bindings for python so it should be a case of staring at the js-sdk's implementation and trying to fit that into our code.

This is likely going to be a long and tedious task so it might be months before this gets ticked off.

Is there a way we can login using a token from previous login instead of using a password every time?

Is there a reason logout is not implemented in this sdk, or is it just not implemented yet?
If login exists, shouldn't we have the opposite logout function? :)

Have you considered adding an appservice abstraction as well? I'd love to experiment with some bridges in Python, but TBH I have a hard time wrapping my head around how the appservices/bridging actually works.

First of all: I get on an valid Letsencrypt Cert an SSL Verify Error when trying to connect.
Second Problem is after disabling the Verify is following:

22:51:22 INFO urllib3.connectionpool: Starting new HTTPS connection (1): matrix.ffslfl.net
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:732: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
22:51:22 INFO urllib3.connectionpool: Starting new HTTPS connection (1): matrix.ffslfl.net
22:51:23 ERROR plugins: EXCEPTION during plugin init: plugins.matrixsync
Traceback (most recent call last):
  File "/home/marcel/hangoutsbot/hangupsbot/plugins/__init__.py", line 359, in load
    the_function(bot)
  File "/home/marcel/hangoutsbot/hangupsbot/plugins/matrixsync/__init__.py", line 43, in _initialise
    matrix_bot.login_with_password(matrixsync_config['username'], matrixsync_config['password'])
  File "/usr/local/lib/python3.4/dist-packages/matrix_client/client.py", line 154, in login_with_password
    self._sync()
  File "/usr/local/lib/python3.4/dist-packages/matrix_client/client.py", line 337, in _sync
    self.sync_token = response["next_batch"]
KeyError: 'next_batch'

matrix_bot is initiliased with: matrix_bot = MatrixClient("https://matrix.ffslfl.net/_matrix/client/api/v1/login?", valid_cert_check=False)
matrixsync_config is just an array with the strings in it.

Since we're supporting both python2 and python3, our async options are very limited. I think we have to either use gevent, use threading, or maybe we can use something from the twisted stack?

Personally, I'd really like to just drop support for python2 (it is the way the ecosystem is moving anyway), and use either the 3.4 syntax of @asyncio.coroutine and yield from or 3.5+ of async def and await. It would give us much more semantic code. Maybe the right answer is just a community-maintained async fork for python 3.4+.

Issue created to separate out discussion from #126.

Hi,

First of all, thanks for the SDK, that helps a lot!

Now onto my issue: I followed MatrixClient's example on registering an user and got back an error about captcha. Could this be a change in Matrix' protocol that wasn't propagated down here?

>>> from matrix_client.client import MatrixClient
>>> client = MatrixClient("https://matrix.org")
>>> token = client.register_with_password(username="foobar1234", password="password")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dryvenn/Downloads/matrix-python-sdk/matrix_client/client.py", line 126, in register_with_password
    "m.login.password", user=username, password=password
  File "/Users/dryvenn/Downloads/matrix-python-sdk/matrix_client/api.py", line 110, in register
    return self._send("POST", "/register", content)
  File "/Users/dryvenn/Downloads/matrix-python-sdk/matrix_client/api.py", line 459, in _send
    code=response.status_code, content=response.text
matrix_client.errors.MatrixRequestError: 400: {"errcode":"M_UNKNOWN","error":"Captcha is required."}

There was a discussion about the refresh token a few weeks (months?) on #matrix-dev. Yet, this SDK doesn’t store it anywhere. I think that should be a good idea (although I don’t know if it is actually handy for any current application).

Per a conversation with Coffee in #matrix-dev:matrix.org, requests is a rather heavyweight dependency when compared to the size of this library. Since we only use requests in the _send api method, using urllib instead of requests shouldn't add much extra burden to maintaining the sdk.

The biggest issue to overcome is rewriting all of the tests that use responses.

See also https://github.com/Matrixcoffee/urllib-requests-adapter

According to the comments:

       """ Return a list of Room objects that the user has joined.

        Returns:
            Room[]: Rooms the user has joined.

        """

In practice, this function returns a dictionary of {room_id: room_object}.

This is needed for the possibility of other protocols in the future, and to be able to test more of our functionality behind a dummy wrapper.

~/Projects/mcat $ cat mcat.py | env/bin/python mcat.py --room "!CJDvntNEPGQmykeIrB:whatthefuck.computer" --stdin -c ~/.mcatrc 
Traceback (most recent call last):
  File "mcat.py", line 55, in <module>
    mcat.f_to_matrix(sys.stdin)
  File "mcat.py", line 24, in f_to_matrix
    self.room.send_text(line)
  File "/home/rrix/Projects/mcat/env/lib/python2.7/site-packages/matrix_client/client.py", line 169, in send_text
    return self.client.api.send_message(self.room_id, text)
  File "/home/rrix/Projects/mcat/env/lib/python2.7/site-packages/matrix_client/api.py", line 194, in send_message
    self.get_text_body(text_content, msgtype)
  File "/home/rrix/Projects/mcat/env/lib/python2.7/site-packages/matrix_client/api.py", line 183, in send_message_event
    return self._send("PUT", path, content)
  File "/home/rrix/Projects/mcat/env/lib/python2.7/site-packages/matrix_client/api.py", line 319, in _send
    code=response.status_code, content=response.text
matrix_client.api.MatrixRequestError: 429: {"errcode":"M_LIMIT_EXCEEDED","error":"Too Many Requests","retry_after_ms":1214}

The javascript SDK implements a fairly complex queuing system, I'm going to strawman something similar to that, shortly.

It would be super convenient to have the documentation hosted somewhere (github webpage or similar).

Still, I tried to build it but it fails with:

Makefile:12: *** recipe commences before first target. Stop.

In line 208:
self.client.rooms.remove(self.room_id)
but self.client.rooms is dictionary and has'nt this method.

I use the last tip of sdk with the patch in #1 .

code says more than thousend words (I got some warning regards to certification problem, I have hide this with [...].

>>> from matrix_client.client import MatrixClient, Room
>>>
>>> username = "@silvio-test:localhost"
>>> password = "XXXXXX"
>>> hs = "https://localhost:8448"
>>>
>>> c = MatrixClient(hs, valid_cert_check=False)
>>>
>>> a = c.login_with_password(username, password)
[...]
>>> a
u'QHNpbHZpby10ZXN0OmxvY2FsaG9zdA...stbZxCrMkBDTEjwFPO'
>>> rs = c.get_rooms()
>>> rs
{u'!GCqRWAbXxwZvTiplaO:localhost': <matrix_client.client.Room object at 0x7f15aa8cf550>}
>>> r = rs['!GCqRWAbXxwZvTiplaO:localhost']
>>> r
<matrix_client.client.Room object at 0x7f15aa8cf550>
>>> r.aliases
[u'#testraum:localhost', u'#kuchenessenraum:localhost']
>>> ra = r.aliases[0]
>>> ra
u'#testraum:localhost'
>>> c.join_room(ra)
[...]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/matrix_client/client.py", line 91, in join_room
    response = self.api.join_room(room_id_or_alias)
  File "/usr/local/lib/python2.7/dist-packages/matrix_client/api.py", line 132, in join_room
    return self._send("POST", path)
  File "/usr/local/lib/python2.7/dist-packages/matrix_client/api.py", line 319, in _send
    code=response.status_code, content=response.text
matrix_client.api.MatrixRequestError: 500: {"error":"Internal server error"}

On server side I get in homeserver.log this messages:

2015-02-23 17:48:11,445 - synapse.http.server - 138 - INFO - POST-279 - Received request: POST /_matrix/client/api/v1/join/
2015-02-23 17:48:11,447 - synapse.http.server - 165 - ERROR - POST-279 - string index out of range
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/synapse/http/server.py", line 143, in _async_render
    *args
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python2.7/dist-packages/synapse/rest/client/v1/room.py", line 232, in on_POST
    identifier = RoomAlias.from_string(room_identifier)
  File "/usr/local/lib/python2.7/dist-packages/synapse/types.py", line 50, in from_string
    if s[0] != cls.SIGIL:
IndexError: string index out of range
2015-02-23 17:48:11,447 - twisted - 455 - INFO - POST-279 - "127.0.0.1" - - [23/Feb/2015:17:48:11 +0000] "POST /_matrix/client/api/v1/join/?access_token=QHNpbHZpby10ZXN0OmxvY2FsaG9zdA...stbZxCrMkBDTEjwFPO HTTP/1.1" 500 33 "-" "python-requests/2.5.1 CPython/2.7.3 Linux/3.18.2-2-ARCH"
2015-02-23 17:48:11,448 - synapse.http.server - 177 - INFO - POST-279 - Processed request: 2ms - POST /_matrix/client/api/v1/join/

If I join with the room_id seems to work:

>>> c.join_room('!GCqRWAbXxwZvTiplaO:localhost')
[...]
<matrix_client.client.Room object at 0x1201e10>

Logfile for this is here:

2015-02-23 17:59:01,038 - synapse.http.server - 138 - INFO - POST-304 - Received request: POST /_matrix/client/api/v1/join/!GCqRWAbXxwZvTiplaO:localhost
2015-02-23 17:59:01,040 - synapse.api.errors - 47 - INFO - POST-304 - SynapseError: 400, Expected RoomAlias string to start with '#'
2015-02-23 17:59:01,057 - twisted - 455 - INFO - GET-303 - "172.17.42.1" - - [23/Feb/2015:17:59:00 +0000] "GET /_matrix/client/api/v1/events?access_token=QHNpbHZpby10ZXN0OmxvY2FsaG9zdA...IBulaUdMGUgQjGKgNr&from=s12_22_10&timeout=30000 HTTP/1.1" 200 549 "https://localhost:8448/_matrix/client/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"
2015-02-23 17:59:01,057 - synapse.http.server - 177 - INFO - GET-303 - Processed request: 2706ms 200 GET /_matrix/client/api/v1/events
2015-02-23 17:59:01,060 - twisted - 455 - INFO -  - "127.0.0.1" - - [23/Feb/2015:17:59:00 +0000] "POST /_matrix/client/api/v1/join/!GCqRWAbXxwZvTiplaO:localhost?access_token=QHNpbHZpby10ZXN0OmxvY2FsaG9zdA...stbZxCrMkBDTEjwFPO HTTP/1.1" 200 43 "-" "python-requests/2.5.1 CPython/2.7.3 Linux/3.18.2-2-ARCH"
2015-02-23 17:59:01,061 - synapse.http.server - 177 - INFO -  - Processed request: 22ms 200 POST /_matrix/client/api/v1/join/!GCqRWAbXxwZvTiplaO:localhost
2015-02-23 17:59:01,081 - synapse.http.server - 138 - INFO - GET-305 - Received request: GET /_matrix/client/api/v1/events
2015-02-23 17:59:01,085 - twisted - 455 - INFO - GET-305 - "172.17.42.1" - - [23/Feb/2015:17:59:00 +0000] "GET /_matrix/client/api/v1/events?access_token=QHNpbHZpby10ZXN0OmxvY2FsaG9zdA...IBulaUdMGUgQjGKgNr&from=s13_22_10&timeout=30000 HTTP/1.1" 200 229 "https://localhost:8448/_matrix/client/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"
2015-02-23 17:59:01,085 - synapse.http.server - 177 - INFO - GET-305 - Processed request: 4ms 200 GET /_matrix/client/api/v1/events
2015-02-23 17:59:01,093 - synapse.http.server - 138 - INFO - GET-306 - Received request: GET /_matrix/client/api/v1/events

ImportError: No module named requests

Some research suggests this is an external dependency which also needs to be installed. Clarify in the readme that this is a dependency, and how to get it.

Hi,

I wonder if somebody could help me.

I have a problem. I made a bot with this sdk and I run it all day.
The system is behind NAT. And after my IP address changes (by ISP) the bot never recognise it and the connection is broken I have to restart the bot. :(

Hi!
Can i get list of existing users from local synapse server ? Something like getUsers() from js sdk ?

https://matrix.to/#/!XqBunHwQIXUiqCaoxq:matrix.org/$1493422976620705zgcwk:matrix.org

When merged, #65 should allow user power levels to be altered, but it doesn't
expose methods at the MatrixClient level of abstraction for changing
required power levels in a room.

Hello - thank you for creating this. I am trying to connect to vector.im, and assuming that is possible, I am not sure what the url path should be. I tried https://vector.im/ and https://vector.im/beta, but neither works. Do I need a port?

https://gist.github.com/RickCogley/36f61898b719738b65a93313c085e125

Could you please advise also:

• is there a way to store the parameters server, username etc in a config file?

As a goal: I would like to run a single non-interactive command on my server, to capture a log of work performed, to a given room.

Thanks in advance for any advice you can provide.
Sincerely,
Rick

My router changes his ip every three days, and then this sdk does not reconnect to the server.
This request method does not return in that case, and so there is also no exception which could be handled.

Solution: add a timeout to the request.
I can do a pull request, but there are some points to discuss:

• The timeout has to be larger than the timeout defined in the query_params. (if available). Should we use 60 seconds by default (if no timeout is available), otherwise 2*query_params['timeout'] (request_timeout = 2 * query_params.get('timeout', 60000)/1000)
• Should we handle the possible timeout exception in listen_forever by default?

Moreover I have to search whether there are more requests in this code, and whether it is useful to add a timeout.

Add a way to search for rooms.

Right now with the API and the client you can only look for rooms the user is subscribed for or one which we already know the address or the alias, but there is no way to search the list of all rooms.

Right now, the only timeout in the sdk is the server-side sync timeout. No client-side timeout is specified. This means that, under some network conditions, requests may hang indefinitely.

In particular, this makes a thread running client.listen_forever freeze for me once every few days on a VPS or sometimes when moving between wifi networks on a laptop (makes using the sdk for a long-running chatbot kinda annoying).

Passing a timeout into requests.request() seems to fix it, throwing an exception as desired.

According Requests doc it accept timeout parameter in seconds.
But in api.py#L78 to this parameter passed huge value 30000 sec = 8.3 hours from api.py#L63
I think line api.py#L78 must be like: "timeout": timeout_ms/1000.0

@gufogio

There's no way to access ephemeral events, like m.receipts.

Currently, we can only send topic and name state events through wrapper functions. The code should expose a generic state sending function.

Incrementing txn_id from 0 is not a great idea, and we should be retrying messages rather than ignoring them. Investigate options around queuing