OS: macOS mojave 10.14.6 (18G2022)
Python: Python 3.7.6 (CPython)

stub_action seems to "cache" exceptions given through side_effect, effectively ignoring whatever values come next and re-raising the exception it encountered indefinitely.

STR:

1. pip3 install pytest==5.3.5 pysoa==1.1.0
2. python3
3. Paste this code:

import pytest
from pysoa.client.client import Client
from pysoa.common.transport.errors import MessageReceiveTimeout
from pysoa.test.stub_service import stub_action


def test_it():
    client = Client(config={})
    with stub_action('foo', 'bar', side_effect=[MessageReceiveTimeout('foo'), {}]):
        with pytest.raises(MessageReceiveTimeout):
            client.call_action('foo', 'bar')
        client.call_action('foo', 'bar')


test_it()

Unknown cause at this point.

Traceback (most recent call last):
...
  File "pysoa/client/client.py", line 450, in call_actions
    **kwargs
  File "pysoa/client/client.py", line 245, in result
    self._response = self._get_response(timeout)
  File "pysoa/client/client.py", line 653, in get_response
    self._perform_expansion(response.actions, expansions, **kwargs)
  File "pysoa/client/client.py", line 889, in _perform_expansion
    self._expand_objects(objects_to_expand, **kwargs)
  File "pysoa/client/client.py", line 964, in _expand_objects
    action_response = response.actions[0]
IndexError: list index out of range

This is the tracking issue for removing all support for Python 2.7 and 3.5-3.6. Once closed, PySOA will support only Python 3.7 and above and will use Python Typing syntax instead of Python Typing comments, removing try/except around the existing AsyncIO support, eliminate the use of the six library, and remove backports (enum, typing, etc.) from the list of dependencies. Some or all Attrs classes may possibly switched to dataclasses, as well.

Issue eventbrite/conformity#65 describes migrating the PySOA Settings base to Conformity. This is the counterpart issue in this repo, to drop the Settings base here in favor of Settings from Conformity.

Issues with chunked message reassembly in the Redis transport, such as #240, would be easier to diagnose if we included chunk-length and content-length headers and performed sanity checks on received chunks and re-assembled messages with those headers. However, such a change cannot be included in the fix for #240, because it will require a new minor version and a new Redis protocol version (4).

We have open sourced our previously-internal metrics library as PyMetrics and are adopting that in PySOA. The existing metrics shims in this library can be removed now.

Given this test plan

#258

When you provide a global input and then you create test_cases which, at least, expect over this action_case. Then if you use a new action to test, this action will be overwritten on the global actions to run

Add Python Type comments throughout the library, include a py.typed file in the main package, and verify all typing with MyPy, so that PySOA can be used with MyPy in other projects.

When chunking is enabled, some large, chunked messages are causing a UnicodeDecodeError which causes an InvalidMessageError when the re-assembled chucks are deserialized by Message Pack.

Root cause analysis: We are stripping whitespace from serialized messages when extracting header information. When messages are chunked and the chunk boundary lands on a whitespace character, this causes the re-assembled message to fail to deserialize.

The callable accepted and returned by ServerMiddleware.action accepts an EnrichedActionRequest and returns an ActionResponse. The callable accepted and returned by ServerMiddleware.job returns a JobResponse, but inconsistently accepts a dictionary. It should accept a JobRequest for consistency.

Update the Travis config to also run tests on Python 3.8

Some errors are the result of caller misaction and require caller action to resolve. Other errors are the result of server mis-coding, mis-configuration, or broken dependencies (databases, etc.) and require operational or engineering action to resolve. The nature of these errors are very different and organizations frequently track them differently. For example, organizations that employ "error budgets" for autonomous teams deploying microservices don't typically count user-input validation errors against the error budget, but do count uncaught exceptions and inaccessible resources against the error budget.

This proposal involves add an is_caller_error attribute to Error objects to be able to distinguish the difference between these two sources of errors. JobErrors would have an is_caller_error argument that defaults to False and ActionErrors would have one that defaults to True. Services, middleware, and some PySOA code will override these defaults when necessary to indicate the precise nature of errors.

Add functional tests that verify the behavior of the Redis transport with Redis version 6 and its support for TLS and username-password authentication.

• Your operating system name and version.

ProductName: Mac OS X
ProductVersion: 10.15.5
BuildVersion: 19F101

• Your Python interpreter type and version.

Python 3.7.7

• Any details about your local setup that might be helpful in troubleshooting.
  None
• Detailed steps to reproduce the bug.
  Create a test plan using a global input using a nested list/dict structure, like:

echo.0: global input:                  global_echo_list.0.foo: bar

test name: case_001
test description: testing
# one
echo.0: expect: no errors
# two
echo.1: input emptydict:               local_echo_list.0:
echo.1: input:                         local_echo_list.1.bar: foo
echo.1: input emptydict:               local_echo_list.2:
echo.1: expect: no errors

And this test case:

from __future__ import absolute_import, unicode_literals
import os

from pprint import pprint as pp
from conformity import fields

from pysoa.server.action.base import Action
from pysoa.server.server import Server
from pysoa.test.plan import ServicePlanTestCase

class EchoAction(Action):
    request_schema = fields.SchemalessDictionary()
    response_schema = fields.SchemalessDictionary()
    def run(self, request):
        pp(request.body)
        return request.body

class StubServer(Server):
    service_name = 'stub'
    action_class_map = {'echo': EchoAction}

class TestPySOAPlans(ServicePlanTestCase):
    server_class = StubServer
    server_settings = {}
    fixture_path = os.path.dirname(__file__) + '/plans'

Run that test and verify that local_echo_list only has 1 item in it.

• Expected Behavior:
  Run that test and receive a local_echo_list with 3 items in it.

In the Server method make_client, add a calling_service attribute to the context to automatically provide origin information for outgoing service calls.

In the test assertion helpers, unicode string error codes become sets of single error codes. In Python 2, plain string error codes get turned into sets of individual characters. This is new in 0.69.0 and works correctly in 0.68.0.

The PySOA server might not properly check/recycle old database connections in Django mode, which could cause unexplainable behavior during a failover or transient database connection event. There's no clear evidence to prove this is the case, but after further review, the following code is not quite right and is subject to possible failures:

https://github.com/eventbrite/pysoa/blob/48049d88/pysoa/server/server.py#L767-L793

If the SOA client is on Python 2 while the server is on Python 3, or vice versa, they are not able to communicate due to missing key errors, because the top-level request message keys on the Python 2 side are serialized as non-unicode, while the top-level response message keys on the Python 3 side are expected to be unicode and are actually bytes.

When using raise_job_errors=False, I instead received IndexError: list index out of range. It appears that the client is trying to reference actions[0] in call_action_future, but since there is only one call that had a JobError, there is no action to reference.

Traceback (most recent call last):
  File "/usr/local/www/REDACTED/lib/python2.7/site-packages/service/job.py", line 867, in run
    action.run(self, controller, idx)
  -- STACK TRACE REDACTED
  File "/usr/local/www/REDACTED/lib/python2.7/site-packages/REDACTED/utils.py", line 121, in get_hierarchy_from_coordinates
    raise_job_errors=False,
  File "/usr/local/www/REDACTED/lib/python2.7/site-packages/pysoa/client/client.py", line 390, in call_action
    return self.call_action_future(service_name, action, body, **kwargs).result()
  File "/usr/local/www/REDACTED/lib/python2.7/site-packages/pysoa/client/client.py", line 245, in result
    self._response = self._get_response(timeout)
  File "/usr/local/www/REDACTED/lib/python2.7/site-packages/pysoa/client/client.py", line 594, in <lambda>
    return self.FutureResponse(lambda _timeout: future.result(_timeout).actions[0])
IndexError: list index out of range

Some server gauges exist only in our internal code due to our disparate metrics setup. Now that PySOA uses PyMetrics, move those to PySOA so that they are properly (and more accurately) integrated into the startup and request workflows.

Publish documentation to readthedocs.org

this problem happens in pysoa 0.58.0, and doesn't happen in 0.57.0

it looks like something in 0.58.0 prevents os.kill(os.getpid(), signal.SIGTERM) (https://github.com/eventbrite/pysoa/blob/0c73032ca/pysoa/server/autoreload.py#L221) from shutting down the process

maybe the new AsyncEventLoopThread?

PySOA has a double-import trap to stop standalone services from starting like this:

python /path/to/service_module/standalone.py

This is because starting like that can cause double-imports that break all kinds of things (even overriding system modules).

However, this double-import trap has broken in Python 3.7, due to this change:

When using the -m switch, the initial working directory is now added to sys.path, rather than an empty string (which dynamically denoted the current working directory at the time of each import). Any programs that are checking for the empty string, or otherwise relying on the previous behaviour, will need to be updated accordingly (e.g. by also checking for os.getcwd() or os.path.dirname(__main__.__file__), depending on why the code was checking for the empty string in the first place).

This needs to be fixed.

Sometimes, services legitimately need to return very large responses. The current paradigm of establishing a hard limit on response size discourages that. The reason for this is that multi-megabyte responses can significantly reduce the throughput of Redis. Since it is single-threaded, multiple services and clients can block for an unreasonably long time waiting for Redis to handle the I/O of a single response. We need to make it easier to send very large responses without choking Redis.

Response chunking is the solution. With response chunking, a 10MB response divided into 40 chunks of 250kb each would take longer for that server to send and its client to receive, but it would not impact Redis more than other responses, because the work would be spread out and other requests and responses could be handled while that response is being sent and received.