Git Product home page Git Product logo

pycent's Introduction

Python SDK to communicate with Centrifugo v5 HTTP API. Python >= 3.9 supported.

To install run:

pip install cent

Centrifugo compatibility

  • Cent v5 and higher works only with Centrifugo v5.
  • If you need to work with Centrifugo v3, v4 => use Cent v4
  • If you need to work with Centrifugo v2 => use Cent v3

Usage

First of all, see the description of Centrifugo server API in the documentation. This library also supports API extensions provided by Centrifugo PRO. In general, refer to api.proto Protobuf schema file as a source of truth about all available Centrifugo server APIs. Don't forget that Centrifugo supports both HTTP and GRPC API – so you can switch to GRPC by using api.proto file to generate stubs for communication.

This library contains Client and AsyncClient to work with Centrifugo HTTP server API. Both clients have the same methods to work with Centrifugo API and raise the same top-level exceptions.

Sync HTTP client

from cent import Client

Required init arguments:

  • api_url (str) - Centrifugo HTTP API URL address, for example, http://localhost:8000/api
  • api_key (str) - Centrifugo HTTP API key for auth

Optional arguments:

  • timeout (float) - base timeout for all requests in seconds, default is 10 seconds.
  • session (requests.Session) - custom requests session to use.

Example:

from cent import Client, PublishRequest

api_url = "http://localhost:8000/api"
api_key = "<CENTRIFUGO_API_KEY>"

client = Client(api_url, api_key)
request = PublishRequest(channel="channel", data={"input": "Hello world!"})
result = client.publish(request)
print(result)

Async HTTP client

from cent import AsyncClient

Required init arguments:

  • api_url (str) - Centrifugo HTTP API URL address, for example, http://localhost:8000
  • api_key (str) - Centrifugo HTTP API key for auth

Optional arguments:

  • timeout (float) - base timeout for all requests in seconds, default is 10 seconds.
  • session (aiohttp.ClientSession) - custom aiohttp session to use.

Example:

import asyncio
from cent import AsyncClient, PublishRequest

api_url = "http://localhost:8000/api"
api_key = "<CENTRIFUGO_API_KEY>"

async def main():
    client = AsyncClient(api_url, api_key)
    request = PublishRequest(channel="channel", data={"input": "Hello world!"})
    result = await client.publish(request)
    print(result)

if __name__ == "__main__":
    asyncio.run(main())

Handling errors

This library raises exceptions if sth goes wrong. All exceptions are subclasses of cent.CentError.

  • CentError - base class for all exceptions
  • CentNetworkError - raised in case of network related errors (connection refused)
  • CentTransportError - raised in case of transport related errors (HTTP status code is not 2xx)
  • CentTimeoutError - raised in case of timeout
  • CentUnauthorizedError - raised in case of unauthorized access (signal of invalid API key)
  • CentDecodeError - raised in case of server response decoding error
  • CentApiResponseError - raised in case of API response error (i.e. error returned by Centrifugo itself, you can inspect code and message returned by Centrifugo in this case)

Note, that BroadcastRequest and BatchRequest are quite special – since they contain multiple commands in one request, handling CentApiResponseError is still required, but not enough – you also need to manually iterate over the results to check for individual errors. For example, one publish command can fail while another one can succeed. For example:

from cent import *

c = Client("http://localhost:8000/api", "api_key")
req = BroadcastRequest(channels=["1", "2"], data={})
c.broadcast(req)
# BroadcastResult(
#   responses=[
#       Response[PublishResult](error=None, result=PublishResult(offset=7, epoch='rqKx')),
#       Response[PublishResult](error=None, result=PublishResult(offset=7, epoch='nUrf'))
#   ]
# )
req = BroadcastRequest(channels=["invalid:1", "2"], data={})
c.broadcast(req)
# BroadcastResult(
#   responses=[
#       Response[PublishResult](error=Error(code=102, message='unknown channel'), result=None),
#       Response[PublishResult](error=None, result=PublishResult(offset=8, epoch='nUrf'))
#   ]
# )

I.e. cent library does not raise exceptions for individual errors in BroadcastRequest or BatchRequest, only for top-level response error, for example, sending empty list of channels in broadcast:

req = BroadcastRequest(channels=[], data={})
c.broadcast(req)
Traceback (most recent call last):
    ...
    raise CentApiResponseError(
cent.exceptions.CentApiResponseError: Server API response error #107: bad request

So this all adds some complexity, but that's the trade-off for the performance and efficiency of these two methods. You can always write some convenient wrappers around cent library to handle errors in a way that suits your application.

Using for async consumers

You can use this library to constructs events for Centrifugo async consumers. For example, to get proper method and payload for async publish:

from cent import PublishRequest

request = PublishRequest(channel="channel", data={"input": "Hello world!"})
method = request.api_method
payload = request.api_payload
# use method and payload to construct async consumer event.

Using Broadcast and Batch

To demonstrate the benefits of using BroadcastRequest and BatchRequest let's compare approaches. Let's say at some point in your app you need to publish the same message into 10k different channels. Let's compare sequential publish, batch publish and broadcast publish. Here is the code to do the comparison:

from cent import *
from time import time


def main():
    publish_requests = []
    channels = []
    for i in range(10000):
        channel = f"test_{i}"
        publish_requests.append(PublishRequest(channel=channel, data={"msg": "hello"}))
        channels.append(channel)
    batch_request = BatchRequest(requests=publish_requests)
    broadcast_request = BroadcastRequest(channels=channels, data={"msg": "hello"})

    client = Client("http://localhost:8000/api", "api_key")

    start = time()
    for request in publish_requests:
        client.publish(request)
    print("sequential", time() - start)

    start = time()
    client.batch(batch_request)
    print("batch", time() - start)

    start = time()
    client.broadcast(broadcast_request)
    print("broadcast", time() - start)


if __name__ == "__main__":
    main()

On local machine, the output may look like this:

sequential 5.731332778930664
batch 0.12313580513000488
broadcast 0.06050515174865723

So BatchRequest is much faster than sequential requests in this case, and BroadcastRequest is the fastest - publication to 10k Centrifugo channels took only 60ms. Because all the work is done in one network round-trip. In reality the difference will be even more significant because of network latency.

For contributors

Tests and benchmarks

Prerequisites – start Centrifugo server locally:

CENTRIFUGO_API_KEY=api_key CENTRIFUGO_HISTORY_TTL=300s CENTRIFUGO_HISTORY_SIZE=100 \
CENTRIFUGO_PRESENCE=true CENTRIFUGO_GRPC_API=true ./centrifugo

And install dependencies:

make dev

Then to run tests, run:

make test

To run benchmarks, run:

make bench

Migrate to Cent v5

Cent v5 contains the following notable changes compared to Cent v4:

  • Client constructor slightly changed, refer to the examples above.
  • To call desired API import and construct a request object (inherited from Pydantic BaseModel) and then call corresponding method of client. This should feel very similar to how GRPC is usually structured.
  • Base exception class is now CentError instead of CentException, exceptions SDK raises were refactored.
  • To send multiple commands in one HTTP request SDK provides batch method.

pycent's People

Contributors

alexander-emelin avatar dependabot[bot] avatar fzambia avatar katantdev avatar yurtaev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pycent's Issues

Generate token for client side

Hi,

I am trying to generate a token for the client side user. I went through the documentation on generating token and signatures, and there is a note saying:

Note that our API clients already have functions to generate client token - you don't have to implement this yourself until you want to implement it for another language for which we don't have API client yet

I am using the latest version of centrifugo (v2.1.0) and cent (v3.0.0), but there is no such functions to generate the token. I have looked on the other cent versions (v2.1.0), and the generate_token function and many other functions are available.

Am I missing something on the latest cent version? Could you please help me how to generate token for the client side so that I can connect to the centrifugo server? If it helps, I am using Pythton v3.6.7 and Django v2.1.7.

Thank you

How would I go about reading the API constantly?

I am trying to make a python client that reads the history and gets the latest message in realtime, but only so far have I only been able to do it with the time module. Is there a way I can get the updates in realtime like javascript?

Centrifugo on Production Server

Hello!

I am using Centrifugo in my Django project as a tool for publishing notifications.
All services (also Centrifugo) are in Docker containers on the same server.
I use Nginx as a proxy server for Centrifugo.
Centrifugo is available on endpoint https://my-domain.io/centrifugo
I try to publish a notification with publish method, but get an error:

from cent import Client, CentException
url = "https://my_domain.io/centrifugo"
api_key = "my_api_key"
client = Client(url, api_key=api_key, timeout=1)
channel = 'channel'

try:
    client.publish('channel', {"value": "My message"})
except CentException as e:
    print(e)

HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f088e765130>, 'Connection to my-domain.io timed out. (connect timeout=1)'))

Also I try to publish with python requests package and get almost the same error:

import json
import requests


command = {
    "method": "publish",
    "params": {
        "channel": "channel",
        "data": {
            "value": "My message!"
        }
    }
}


api_key = "my-api-key"
data = json.dumps(command)
headers = {'Content-type': 'application/json', 'Authorization': 'apikey ' + api_key}
resp = requests.post("https://my-domain.io/centrifugo/api", data=data, headers=headers)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out'))

But when I run the same commands from my local server, everything works as expected.
What am I doing wrong? Should I use a separate server (droplet) for Centrifugo?
Thank you in advance!

Poll the server for number of clients

I'm deploying centrifugo in a Django project and I need to manage a process if there are no more clients connected to centrifugo. It doesn't seem to me that cent supports this. Is there a way to find out (from python) how many clients are connected to centrifugo?

Or if you have some other recommendation, let me know. Thanks!

Add license please

Thank you for your work!
I want to copy and modify your module to make it asynchronous, but I can not legally do this until you include license in your repository

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Hi,

from cent import Client, CentException

url = "https://localhost:8999"
api_key = "xxxxx"
client = Client(url, api_key=api_key, timeout=1)

client.channels()

when I ran the above code, got the below error

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Request for cent Package Availability in Anaconda

Hello,

I hope this message finds you well. I am writing this issue today to kindly request you to consider making the cent package available through Anaconda.

There are several compelling reasons for this request:

Ease of Installation: Anaconda provides a simple, one-command installation process for packages. This simplicity would lower the entry barrier for new or less experienced users who might otherwise find it challenging to install and manage the cent package.

Environment Management: Anaconda allows users to create isolated environments. By making cent available in Anaconda, users would be able to use your package without worrying about version conflicts with other packages in their environment.

Reach More Users: Anaconda is widely used in the scientific community, especially among data scientists. Making cent available through Anaconda would allow your package to reach a wider audience and receive more exposure.

I believe that taking the step to make cent available in Anaconda would greatly increase its usability and accessibility.

Thank you for considering this request, and for the fantastic work you've done on the cent package.

Best,
Daniel Amaro

404 on message publish

centrifugo is in docker-compose, using centrifugo/centrifugo:v3.0.1
config.json is:

{
  "token_hmac_secret_key": "CENTRIFUGO_SECRET",
  "api_key": "CENTRIFUGO_API_KEY",
  "admin_password": "Qwerty123",
  "admin_secret": "Qwerty123",
  "port": 9000,
  "namespaces": [
    {
      "name": "seed",
      "history_size": 16,
      "history_ttl": "512s",
      "recover": true
    }
  ],
  "allowed_origins": ["*"]
}

docker-compose.yml is:

centrifugo:
    container_name: centrifugo
    image: centrifugo/centrifugo:v3.0.1
    command: centrifugo -c config.json
    volumes:
      - ./external/centrifugo:/centrifugo

repr is:

self = <cent.core.Client object at 0x7f4f54c947c0>, url = 'http://centrifugo:9000', data = b'{"method": "publish", "params": {"channel": "seed:notifications", "data": {}, "skip_history": false}}'

error is:

cent.core.RequestException: wrong status code: 404

[Question] Broadcast method input parameters

Hi sir, I want to using cent in backend to broadcast a event/message to Centrifugo. And I want to using feature async consumerswith it so I need to use idempotency_key to prevent duplicated event when retry. But I read the code that the broadcast method does not have idempotency_key parameter, right ?
image

publish data with datetime or Decimal

hi, currently client.send() fails if you try to publish data that contains datetime or Decimal objects in https://github.com/centrifugal/cent/blob/master/cent/core.py#L62

It would be great if one could specify a custom JSONEncoder to use somehow (for example the DjangoJSONEncoder if you happen to use django). What I'm trying to do is to publish my serialized django model object into a channel. Or do you think it's a bad idea?

btw thanks for the great work on centrifuge!

Missing Parameters

__init__() missing 1 required positional argument: 'secret' . It seems secret is needed when passing parameters for Client. I added this and i get an wrong status code: 404 when i try to test publish

Python 3.3 Error

When attempting to use cent --config centrc superuser project_list with Python 3.3 I receive the error:

POST data should be bytes or an iterable of bytes. It cannot be of type str.

The offending line is the urlopen on line 87 of core.py. Python 3.3 urlencode wants bytes.

Thanks again for your awesome work on centrifuge/cent!

Documentation

Hi!
Is there any real documentation for this library?

For instance:
generate_api_sign

Thanks!

Latest version does not work with v1

Hey, I've just spent few hours trying to figure out why centrifugo throws out "no sign found in API request". Would be nice to create a branch with 2.1.0 version to avoid confusion for new users.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.