mindflayer / python-mocket Goto Github PK

View Code? Open in Web Editor NEW

274.0 15.0 41.0 536 KB

a socket mock framework - for all kinds of socket animals, web-clients included

License: BSD 3-Clause "New" or "Revised" License

Python 98.92% Makefile 1.08%

python testing mock socket http framework mocket tdd tdd-utilities mocking

python-mocket's People

Contributors

Stargazers

Watchers

python-mocket's Issues

Some options not implemented in MocketSocket

Describe the bug
When using mocked sockets in some conditions, certain socket properties are not properly implemented.

In AF_INET6 scenarios (IPv6 connection), address can be a tuple of four elements (see https://docs.python.org/3/library/socket.html#socket-families ), but the current implementation only takes tuples with 2 arguments. For example socket.connect((host, port, 0, 0)) should not throw an error.
The socket type argument of getaddrinfo() is called type ( https://docs.python.org/3/library/socket.html#socket.getaddrinfo ) and not socktype as implemented here:

python-mocket/mocket/mocket.py

Line 458 in bce7cde

] = lambda host, port, family=None, socktype=None, proto=None, flags=None: [

test_http_with_xxhash fails on Arch Linux x86_64

The environment is:
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python

The captured stdout is the output of `print(responses['httpbin.org']['80']), which looks like each request was put into two different hashes.

===================================================================================================================================== FAILURES =====================================================================================================================================
___________________________________________________________________________________________________________ HttpEntryTestCase.test_truesendall_with_dump_from_recording ____________________________________________________________________________________________________________

self = <tests.main.test_http_with_xxhash.HttpEntryTestCase testMethod=test_truesendall_with_dump_from_recording>

    @mocketize(truesocket_recording_dir=os.path.dirname(__file__))
    def test_truesendall_with_dump_from_recording(self):
        requests.get('http://httpbin.org/ip', headers={"user-agent": "Fake-User-Agent"})
        requests.get('http://httpbin.org/gzip', headers={"user-agent": "Fake-User-Agent"})

        dump_filename = os.path.join(
            Mocket.get_truesocket_recording_dir(),
            Mocket.get_namespace() + '.json',
        )
        with io.open(dump_filename) as f:
            responses = json.load(f)

        print(responses['httpbin.org']['80'])
>       self.assertEqual(len(responses['httpbin.org']['80'].keys()), 2)
E       AssertionError: 4 != 2

tests/main/test_http_with_xxhash.py:29: AssertionError
------------------------------------------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------------------------------------------
{'27f55d9c': {'request': 'GET /gzip HTTP/1.1\r\nHost: httpbin.org\r\nuser-agent: Fake-User-Agent\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n', 'response': '48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 6C 69 76 65 0D 0A 53 65 72 76 65 72 3A 20 67 75 6E 69 63 6F 72 6E 2F 31 39 2E 39 2E 30 0D 0A 44 61 74 65 3A 20 54 75 65 2C 20 30 31 20 4A 61 6E 20 32 30 31 39 20 31 34 3A 35 32 3A 30 35 20 47 4D 54 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 63 61 74 69 6F 6E 2F 6A 73 6F 6E 0D 0A 43 6F 6E 74 65 6E 74 2D 4C 65 6E 67 74 68 3A 20 31 38 33 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 4F 72 69 67 69 6E 3A 20 2A 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 43 72 65 64 65 6E 74 69 61 6C 73 3A 20 74 72 75 65 0D 0A 56 69 61 3A 20 31 2E 31 20 76 65 67 75 72 0D 0A 0D 0A 1F 8B 08 00 95 7E 2B 5C 02 FF 4D 8F 4D 0E 82 30 10 46 F7 9C A2 E9 92 40 0D 92 68 74 47 0C EA 01 F4 00 D8 8E A5 01 3B 4D 5B 37 12 EE 6E 5B 12 74 39 EF 7B F3 37 65 84 50 F9 51 C6 80 A0 47 E2 ED 1B 0A 12 59 0F 9D 00 EB 02 9B 42 19 40 C3 39 18 1F 6A 9A 6F 72 9A A4 95 96 AD E6 28 94 96 31 8E D3 0A 22 E0 39 76 1E 56 F1 84 5A 03 F7 0A 75 74 F8 88 EE 97 5D D1 A5 C1 BD F7 E6 A1 34 43 2B D7 EC EE C0 96 8D 04 9D 8C 73 37 40 F9 87 82 33 2F F7 BE C0 F7 18 5F A0 97 F6 B6 74 53 B4 4A AA B4 EF 50 B3 6A BF 63 55 5D B1 7A 4B B3 39 FB 02 0D D4 A4 0E F8 00 00 00'}, '31ae71d6074aa0e73fb4feb91f28546a': {'request': 'GET /gzip HTTP/1.1\r\nHost: httpbin.org\r\nuser-agent: Fake-User-Agent\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n', 'response': '48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D 0A 44 61 74 65 3A 20 4D 6F 6E 2C 20 32 37 20 4A 75 6C 20 32 30 32 30 20 31 30 3A 31 38 3A 35 38 20 47 4D 54 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 63 61 74 69 6F 6E 2F 6A 73 6F 6E 0D 0A 43 6F 6E 74 65 6E 74 2D 4C 65 6E 67 74 68 3A 20 32 31 35 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 6C 69 76 65 0D 0A 53 65 72 76 65 72 3A 20 67 75 6E 69 63 6F 72 6E 2F 31 39 2E 39 2E 30 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 4F 72 69 67 69 6E 3A 20 2A 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 43 72 65 64 65 6E 74 69 61 6C 73 3A 20 74 72 75 65 0D 0A 0D 0A 1F 8B 08 00 12 AA 1E 5F 02 FF 4D 8F BB 0E 82 30 14 86 77 9E A2 E9 48 28 06 14 2C 26 0E 0C DE 56 A3 89 6B 6D 0F A5 51 5B 52 EA 82 F1 DD 6D 31 12 C7 FF FB 2F 39 E7 15 21 84 E5 A0 BA 0E 04 5E 21 67 9F 90 A0 C0 5A 60 02 6C EF D9 CB 4B 0F 6A CE A1 73 5E E3 78 16 E3 31 34 51 B2 D1 DC 08 A5 65 B0 C3 5A 82 04 34 77 E6 60 0A EE 4D 3F 96 5B E7 BA AB D2 A9 B1 72 F2 CE 3D 58 52 4B D0 63 62 CB 6E 40 FE D0 2F 75 21 F5 63 D0 E4 64 19 07 72 08 D7 E2 A3 31 6E 9D 91 A2 C9 80 B1 2C 27 85 B7 20 2F CB 9C 57 B4 A1 A2 10 54 5C AB 7C 2E B0 5F 78 7F FF 7A 80 6B CD 58 DE 6D 4E DF 6D 6C AC 92 4A 07 B6 28 52 4A D3 AC 5A A4 E5 12 47 EF E8 03 C3 88 05 C8 1F 01 00 00'}, 'eaa1896a5d6eed6d4dd83a6696efc485': {'request': 'GET /ip HTTP/1.1\r\nHost: httpbin.org\r\nuser-agent: Fake-User-Agent\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n', 'response': '48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D 0A 44 61 74 65 3A 20 4D 6F 6E 2C 20 32 37 20 4A 75 6C 20 32 30 32 30 20 31 30 3A 31 38 3A 35 37 20 47 4D 54 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 63 61 74 69 6F 6E 2F 6A 73 6F 6E 0D 0A 43 6F 6E 74 65 6E 74 2D 4C 65 6E 67 74 68 3A 20 33 31 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 6C 69 76 65 0D 0A 53 65 72 76 65 72 3A 20 67 75 6E 69 63 6F 72 6E 2F 31 39 2E 39 2E 30 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 4F 72 69 67 69 6E 3A 20 2A 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 43 72 65 64 65 6E 74 69 61 6C 73 3A 20 74 72 75 65 0D 0A 0D 0A 7B 0A 20 20 22 6F 72 69 67 69 6E 22 3A 20 22 34 35 2E 38 38 2E 31 39 34 2E 36 37 22 0A 7D 0A'}, 'faef8d42': {'request': 'GET /ip HTTP/1.1\r\nHost: httpbin.org\r\nuser-agent: Fake-User-Agent\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n', 'response': '48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 6C 69 76 65 0D 0A 53 65 72 76 65 72 3A 20 67 75 6E 69 63 6F 72 6E 2F 31 39 2E 39 2E 30 0D 0A 44 61 74 65 3A 20 54 75 65 2C 20 30 31 20 4A 61 6E 20 32 30 31 39 20 31 34 3A 35 32 3A 30 34 20 47 4D 54 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 63 61 74 69 6F 6E 2F 6A 73 6F 6E 0D 0A 43 6F 6E 74 65 6E 74 2D 4C 65 6E 67 74 68 3A 20 33 32 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 4F 72 69 67 69 6E 3A 20 2A 0D 0A 41 63 63 65 73 73 2D 43 6F 6E 74 72 6F 6C 2D 41 6C 6C 6F 77 2D 43 72 65 64 65 6E 74 69 61 6C 73 3A 20 74 72 75 65 0D 0A 56 69 61 3A 20 31 2E 31 20 76 65 67 75 72 0D 0A 0D 0A 7B 0A 20 20 22 6F 72 69 67 69 6E 22 3A 20 22 39 33 2E 31 37 36 2E 31 33 31 2E 33 32 22 0A 7D 0A'}}

Real sendall for HTTPS requests not working

I am using Python 3.6.4 and getting this error when I try to use mocket:

E           http.client.BadStatusLine: 2

/usr/local/Cellar/python3/3.6.4_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py:279: BadStatusLine

Multiple HTTP writes fail

Dear Giorgio,

Introduction

Our background is that we are currently building a test suite based on CPython/Pytest for invoking MicroPython programs, see also micropython/micropython#5786 and micropython/micropython#4955. It works pretty good so far. Thanks again for Mocket, it helped us tremendously already.

Problem

So, after successfully creating a Pytest LoRa socket mock fixture, we are now struggling creating a corresponding thing for the urequests module.

The gist is that urequests will work like that:

sock.connect(address[-1])
sock.write("%s %s HTTP/1.0\r\n" % (method, path))
sock.write("Host: %s\r\n" % host)

which makes Mocket only see the first line:

b'POST /api/data HTTP/1.0\r\n'

so that it will croak with the following exception.

Exception

>       _, self.body = decode_from_bytes(data).split('\r\n\r\n', 1)
E       ValueError: not enough values to unpack (expected 2, got 1)
.venv3/lib/python3.8/site-packages/mocket/mockhttp.py:23: ValueError

Reproduction

I have been able to create a repro using pytest.
https://gist.github.com/amotl/015ef6b336db55128798d7f1a9a67dea

Thoughts

I believe @sjaensch observed this within #66 already and apparently #67 didn't fix it yet.

Thanks already for looking into this!

With kind regards,
Andreas.

Cannot install mocket in docker image

I'm trying to build an image with python 3.5.3-alpine

One of my dependencies is mocket and the build of the images gives me this output

error in mocket setup command: 'extras_require' requirements cannot include environment markers, in 'speedups': 'xxhash; platform_python_implementation == "CPython"'

My docker file is the following

FROM python:3.5.3-alpine

# set envirnoment variables
ENV SRC=project_name
ENV USER_HOME=/home/project_name
ENV CODE_DIR=$USER_HOME/$SRC

# create user `project_name` and add it to group `project_name`
RUN addgroup -S project_name && adduser -S -G project_name project_name

# add bash since alpine does not have it installed
RUN apk add --no-cache bash

# install python requirements
WORKDIR $CODE_DIR
COPY requirements.txt .
RUN apk add --no-cache --virtual .build-deps \
    ca-certificates gcc postgresql-dev linux-headers musl-dev \
    libffi-dev jpeg-dev zlib-dev make \
    && pip install -r requirements.txt \
    && find /usr/local \
        \( -type d -a -name test -o -name tests \) \
        -o \( -type f -a -name '*.pyc' -o -name '*.pyo' \) \
        -exec rm -rf '{}' + \
    && runDeps="$( \
        scanelf --needed --nobanner --recursive /usr/local \
                | awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
                | sort -u \
                | xargs -r apk info --installed \
                | sort -u \
    )" \
    && apk add --virtual .rundeps $runDeps \
    && apk del .build-deps

# copy all the code
COPY . $CODE_DIR

# create logs directory and add logrotate rule
RUN mkdir /var/log/project_name
COPY docker/logrotate/project_name /etc/logrotate.d/

# assign permissions to `project_name`
RUN chown project_name:project_name /var/log/project_name
RUN chown -R project_name:project_name $CODE_DIR

USER project_name

ENTRYPOINT ["python3", "manage.py"]

Is there a way to avoid extras or to install xxhash on an alpine docker image?

Add Documentation

You link to a 37 minute long video. While the video may hold useful information, few will spend time viewing it.

Support for gevent sockets

I don't see any tests involving gevent.
Are gevent sockets supported?

MocketSocket doesn't support SSL connections

It looks like MocketSocket can't be wrapped completely by SSLSocket

>>> import requests
>>> from mocket.mocket import Mocket
>>> Mocket.enable()
>>> requests.get('https://httpbin.org')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/adapters.py", line 291, in send
    timeout=timeout
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 428, in urlopen
    body=body, headers=headers)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 280, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 791, in send
    self.connect()
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 107, in connect
    ssl_version=resolved_ssl_version)
  File "/home/bjmc/Sandbox/vcrpy/.env/local/lib/python2.7/site-packages/requests/packages/urllib3/util.py", line 378, in ssl_wrap_socket
    ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 440, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 168, in __init__
    socket.__init__(self, _sock=sock._sock)
  File "/usr/lib/python2.7/socket.py", line 190, in __init__
    setattr(self, method, getattr(_sock, method))
AttributeError: 'MocketSocket' object has no attribute 'recv'

Mocket works only in test classes

If I try to use it in a single test function:

from unittest import TestCase
import json

from mocket.mocket import mocketize
from mocket.mockhttp import Entry
import requests

@mocketize
def test_json(self):
    url_to_mock = 'http://testme.org/intro'

    response_to_mock = {
        "integer": 1,
        "string": "asd",
        "boolean": False,
    }

    Entry.single_register(
        Entry.GET,
        url_to_mock,
        body=json.dumps(response_to_mock),
        headers={'content-type': 'application/json'}
    )

    response = requests.get(url_to_mock).json()

    self.assertEqual(response, response_to_mock)

it fails with the following error:

========================================================================================== FAILURES ==========================================================================================
_________________________________________________________________________________________ test_json __________________________________________________________________________________________

args = (), kw = {}

    @functools.wraps(test)
    def wrapper(*args, **kw):
>       with Mocketizer(args[0]):
E       IndexError: tuple index out of range

prova/lib/python2.7/site-packages/mocket/mocket.py:235: IndexError
================================================================================== 1 failed in 0.08 seconds ==================================================================================

Binary body

We need to find a way to serve binary files.

  File "/home/drizzt/repos/ingestion/ingestion/items/tests.py", line 387, in test_meta_from_binary
    Entry.single_register(Entry.GET, config.ALFRESCO_BASE_URL + url, body=open(mockfile, 'rb').read())
  File "/home/drizzt/repos/ingestion/local/lib/python2.7/site-packages/mocket/mockhttp.py", line 106, in single_register
    Entry.register(method, uri, Response(body=body, status=status, headers=headers))
  File "/home/drizzt/repos/ingestion/local/lib/python2.7/site-packages/mocket/mockhttp.py", line 37, in __init__
    self.data = self.get_data()
  File "/home/drizzt/repos/ingestion/local/lib/python2.7/site-packages/mocket/mockhttp.py", line 42, in get_data
    return '{0}\r\n{1}\r\n\r\n{2}'.format(status_line, header_lines, decode_utf8(self.body)).encode('utf-8')
  File "/home/drizzt/repos/ingestion/local/lib/python2.7/site-packages/mocket/compat.py", line 24, in decode_utf8
    s = s.decode("utf-8")
  File "/home/drizzt/repos/ingestion/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: invalid start byte

MocketSocket support for context manager

Hi,

MocketSocket doesn't seem to support context manager, so the construct:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((target.host, target.port, 0, 0))
    s.sendall(something)
    data = s.recv(4096)

throws an AttributeError:

Traceback (most recent call last):
  File "/home/scanner/scanner.py", line 32, in scan
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
AttributeError: __enter__

Is it by design? Any way to work around it (besides rewriting the original code that uses with socket.socket )?

Thanks!

Socket recording with multiple connections across several sendall() has issues if there is a common initialization

This is a tricky scenario that happened in a corner case:

Recording socket activity for replay
Recording multiple connections in the same Unit Test
Each connection has multiple sendall() operations
The first sendall() data is identical across all the connections

This doesn't apply with HTTP because connections are mostly stateless, but can happen with other protocols (i.e. HTTP proxy)

Each connection is as follows:
Step #0 sendall(authentication_string): i.e. "AUTHENTICATE user:password"
Step #1 read(authentication_confirmation): i.e. "AUTHENTICATION OK"
Step #2 sendall(some command): i.e. "EXECUTE COMMAND x"

Different connections execute different commands (i.e. connection#1 sends "EXECUTE COMMAND x", connection#2 sends "EXECUTE COMMAND y"), but they require authentication to be performed in order to accept commands.

Problematic scenario pseudocode. At the beginning no data is present in the JSON files.

Mocket.enable('some_log_file','.')

s1 = socket() -> we get a MocketSocket
s1.connect(destination_ip, destination_port)
s1.sendall('AUTHENTICATE user:password') -> this gets sent through a true_socket() as there is no cache for this message
s1.read() -> 'AUTHENTICATION OK' -> data stored in JSON file
s1.write('EXECUTE COMMAND x') -> this gets send through a true_socket() as there is no cache for this message
s1.read() -> 'RESULT OF COMMAND x' -> data stored in JSON file
s1.close()

s2 = socket() -> we get a MocketSocket
s2.connect(destination_ip, destination_port) -> connect to the same IP and port as before
s2.sendall('AUTHENTICATE user:password') -> This is never sent to a true_socket, as the request is identical to the one performed with s1, so the Entry is present in the JSON file and a mocked response is provided
s2.read() -> 'AUTHENTICATION OK' -> from the JSON file
s2.write('EXECUTE COMMAND y') -> this gets send through a true_socket() as there is no cache for this message
s2.read() -> 'ERROR: UNAUTHENTICATED' -> we get an error from the server because the authentication message for this connection has never been sent! 
s2.close()

This is not really a bug: mocket is behaving as expected. However, how can you test this scenario?
In my case I worked around it adding some entropy in the authentication message to make it different every time, but it's not always possible or easy to achieve.

A possible solution could be provide an option to tell mocket that is working in "Record only" mode, so that cached data is ignored.

Thanks!

Simple use of MocketEntry broken

Commit 37e1783 changed MocketEntry.get_response to return response.data instead of str(response), and that doesn't work in the simple case where response_cls is str, like in the example from the presentation:

MocketEntry(('localhost', 8080), ['Show me.\r\n'])

Mocket can't mock writing compressed data by zlib

I tried to use Mocket to mock redis and write the compressed data, it threw this error.

In [1]: from mocket import compat

In [2]: import zlib

In [3]: s = '{[]}'

In [4]: compat.decode_from_bytes(zlib.compress(s))
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-4-0af92b1f9211> in <module>()
----> 1 compat.decode_from_bytes(zlib.compress(s))

/usr/local/lib/python2.7/dist-packages/mocket/compat.pyc in decode_from_bytes(s, charset)
     35 def decode_from_bytes(s, charset=encoding):
     36     if isinstance(s, byte_type):
---> 37         s = s.decode(charset)
     38     return text_type(s)
     39

/usr/lib/python2.7/encodings/utf_8.pyc in decode(input, errors)
     14
     15 def decode(input, errors='strict'):
---> 16     return codecs.utf_8_decode(input, errors, True)
     17
     18 class IncrementalEncoder(codecs.IncrementalEncoder):

UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte

Three SSL test failures on openSUSE Tumbleweed Python 2

test_can_inspect_last_request_with_ssl and test_rotating_responses_with_requests and test_json fail on openSUSE Tumbleweed Python 2 only. They pass on Python 3

[   24s] =================================== FAILURES ===================================
[   24s] ____________________ test_rotating_responses_with_requests _____________________
[   24s] 
[   24s]     @httprettified
[   24s]     def test_rotating_responses_with_requests():
[   24s]         """HTTPretty should support rotating responses with requests"""
[   24s]     
[   24s]         HTTPretty.register_uri(
[   24s]             HTTPretty.GET, "https://api.yahoo.com/test",
[   24s]             responses=[
[   24s]                 HTTPretty.Response(body=b"first response", status=201),
[   24s]                 HTTPretty.Response(body=b'second and last response', status=202),
[   24s]             ])
[   24s]     
[   24s]         response1 = requests.get(
[   24s] >           'https://api.yahoo.com/test')
[   24s] 
[   24s] tests/main/test_httpretty.py:161: 
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   24s]     return request('get', url, params=params, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   24s]     return session.request(method=method, url=url, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   24s]     resp = self.send(prep, **send_kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   24s]     r = adapter.send(request, **kwargs)
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] 
[   24s] self = <requests.adapters.HTTPAdapter object at 0x3ff944e3c50>
[   24s] request = <PreparedRequest [GET]>, stream = False
[   24s] timeout = <urllib3.util.timeout.Timeout object at 0x3ff944e3ba8>, verify = True
[   24s] cert = None, proxies = OrderedDict()
[   24s] 
[   24s]     def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
[   24s]         """Sends PreparedRequest object. Returns Response object.
[   24s]     
[   24s]         :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
[   24s]         :param stream: (optional) Whether to stream the request content.
[   24s]         :param timeout: (optional) How long to wait for the server to send
[   24s]             data before giving up, as a float, or a :ref:`(connect timeout,
[   24s]             read timeout) <timeouts>` tuple.
[   24s]         :type timeout: float or tuple or urllib3 Timeout object
[   24s]         :param verify: (optional) Either a boolean, in which case it controls whether
[   24s]             we verify the server's TLS certificate, or a string, in which case it
[   24s]             must be a path to a CA bundle to use
[   24s]         :param cert: (optional) Any user-provided SSL certificate to be trusted.
[   24s]         :param proxies: (optional) The proxies dictionary to apply to the request.
[   24s]         :rtype: requests.Response
[   24s]         """
[   24s]     
[   24s]         try:
[   24s]             conn = self.get_connection(request.url, proxies)
[   24s]         except LocationValueError as e:
[   24s]             raise InvalidURL(e, request=request)
[   24s]     
[   24s]         self.cert_verify(conn, request.url, verify, cert)
[   24s]         url = self.request_url(request, proxies)
[   24s]         self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
[   24s]     
[   24s]         chunked = not (request.body is None or 'Content-Length' in request.headers)
[   24s]     
[   24s]         if isinstance(timeout, tuple):
[   24s]             try:
[   24s]                 connect, read = timeout
[   24s]                 timeout = TimeoutSauce(connect=connect, read=read)
[   24s]             except ValueError as e:
[   24s]                 # this may raise a string formatting error.
[   24s]                 err = ("Invalid timeout {}. Pass a (connect, read) "
[   24s]                        "timeout tuple, or a single float to set "
[   24s]                        "both timeouts to the same value".format(timeout))
[   24s]                 raise ValueError(err)
[   24s]         elif isinstance(timeout, TimeoutSauce):
[   24s]             pass
[   24s]         else:
[   24s]             timeout = TimeoutSauce(connect=timeout, read=timeout)
[   24s]     
[   24s]         try:
[   24s]             if not chunked:
[   24s]                 resp = conn.urlopen(
[   24s]                     method=request.method,
[   24s]                     url=url,
[   24s]                     body=request.body,
[   24s]                     headers=request.headers,
[   24s]                     redirect=False,
[   24s]                     assert_same_host=False,
[   24s]                     preload_content=False,
[   24s]                     decode_content=False,
[   24s]                     retries=self.max_retries,
[   24s]                     timeout=timeout
[   24s]                 )
[   24s]     
[   24s]             # Send the request.
[   24s]             else:
[   24s]                 if hasattr(conn, 'proxy_pool'):
[   24s]                     conn = conn.proxy_pool
[   24s]     
[   24s]                 low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
[   24s]     
[   24s]                 try:
[   24s]                     low_conn.putrequest(request.method,
[   24s]                                         url,
[   24s]                                         skip_accept_encoding=True)
[   24s]     
[   24s]                     for header, value in request.headers.items():
[   24s]                         low_conn.putheader(header, value)
[   24s]     
[   24s]                     low_conn.endheaders()
[   24s]     
[   24s]                     for i in request.body:
[   24s]                         low_conn.send(hex(len(i))[2:].encode('utf-8'))
[   24s]                         low_conn.send(b'\r\n')
[   24s]                         low_conn.send(i)
[   24s]                         low_conn.send(b'\r\n')
[   24s]                     low_conn.send(b'0\r\n\r\n')
[   24s]     
[   24s]                     # Receive the response from the server
[   24s]                     try:
[   24s]                         # For Python 2.7, use buffering of HTTP responses
[   24s]                         r = low_conn.getresponse(buffering=True)
[   24s]                     except TypeError:
[   24s]                         # For compatibility with Python 3.3+
[   24s]                         r = low_conn.getresponse()
[   24s]     
[   24s]                     resp = HTTPResponse.from_httplib(
[   24s]                         r,
[   24s]                         pool=conn,
[   24s]                         connection=low_conn,
[   24s]                         preload_content=False,
[   24s]                         decode_content=False
[   24s]                     )
[   24s]                 except:
[   24s]                     # If we hit any problems here, clean up the connection.
[   24s]                     # Then, reraise so that we can handle the actual exception.
[   24s]                     low_conn.close()
[   24s]                     raise
[   24s]     
[   24s]         except (ProtocolError, socket.error) as err:
[   24s]             raise ConnectionError(err, request=request)
[   24s]     
[   24s]         except MaxRetryError as e:
[   24s]             if isinstance(e.reason, ConnectTimeoutError):
[   24s]                 # TODO: Remove this in 3.0.0: see #2811
[   24s]                 if not isinstance(e.reason, NewConnectionError):
[   24s]                     raise ConnectTimeout(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, ResponseError):
[   24s]                 raise RetryError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _ProxyError):
[   24s]                 raise ProxyError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _SSLError):
[   24s]                 # This branch is for urllib3 v1.22 and later.
[   24s] >               raise SSLError(e, request=request)
[   24s] E               SSLError: HTTPSConnectionPool(host='api.yahoo.com', port=443): Max retries exceeded with url: /test (Caused by SSLError(CertificateError("partial wildcards in leftmost label are not supported: u'*api.yahoo.com'.",),))
[   24s] 
[   24s] /usr/lib/python2.7/site-packages/requests/adapters.py:514: SSLError
[   24s] ------------------------------ Captured log call -------------------------------
[   24s] ERROR    urllib3.connection:connection.py:404 Certificate did not match expected hostname: api.yahoo.com. Certificate: {u'subject': (((u'organizationName', u'*.api.yahoo.com'),), ((u'organizationalUnitName', u'Domain Control Validated'),), ((u'commonName', u'*.api.yahoo.com'),)), u'subjectAltName': ((u'DNS', u'*api.yahoo.com'), (u'DNS', 'api.yahoo.com'), (u'DNS', u'*')), u'notAfter': 'Sep 12 05:31:12 GMT'}
[   24s] ____________________ test_can_inspect_last_request_with_ssl ____________________
[   24s] 
[   24s]     @httprettified
[   24s]     def test_can_inspect_last_request_with_ssl():
[   24s]         """HTTPretty.last_request is recorded even when mocking 'https' (SSL)"""
[   24s]     
[   24s]         HTTPretty.register_uri(HTTPretty.POST, "https://secure.github.com/",
[   24s]                                body='{"repositories": ["HTTPretty", "lettuce"]}')
[   24s]     
[   24s]         response = requests.post(
[   24s]             'https://secure.github.com',
[   24s]             '{"username": "gabrielfalcao"}',
[   24s]             headers={
[   24s] >               'content-type': 'text/json',
[   24s]             },
[   24s]         )
[   24s] 
[   24s] tests/main/test_httpretty.py:215: 
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:116: in post
[   24s]     return request('post', url, data=data, json=json, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   24s]     return session.request(method=method, url=url, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   24s]     resp = self.send(prep, **send_kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   24s]     r = adapter.send(request, **kwargs)
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] 
[   24s] self = <requests.adapters.HTTPAdapter object at 0x3ff94366cf8>
[   24s] request = <PreparedRequest [POST]>, stream = False
[   24s] timeout = <urllib3.util.timeout.Timeout object at 0x3ff94366dd8>, verify = True
[   24s] cert = None, proxies = OrderedDict()
[   24s] 
[   24s]     def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
[   24s]         """Sends PreparedRequest object. Returns Response object.
[   24s]     
[   24s]         :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
[   24s]         :param stream: (optional) Whether to stream the request content.
[   24s]         :param timeout: (optional) How long to wait for the server to send
[   24s]             data before giving up, as a float, or a :ref:`(connect timeout,
[   24s]             read timeout) <timeouts>` tuple.
[   24s]         :type timeout: float or tuple or urllib3 Timeout object
[   24s]         :param verify: (optional) Either a boolean, in which case it controls whether
[   24s]             we verify the server's TLS certificate, or a string, in which case it
[   24s]             must be a path to a CA bundle to use
[   24s]         :param cert: (optional) Any user-provided SSL certificate to be trusted.
[   24s]         :param proxies: (optional) The proxies dictionary to apply to the request.
[   24s]         :rtype: requests.Response
[   24s]         """
[   24s]     
[   24s]         try:
[   24s]             conn = self.get_connection(request.url, proxies)
[   24s]         except LocationValueError as e:
[   24s]             raise InvalidURL(e, request=request)
[   24s]     
[   24s]         self.cert_verify(conn, request.url, verify, cert)
[   24s]         url = self.request_url(request, proxies)
[   24s]         self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
[   24s]     
[   24s]         chunked = not (request.body is None or 'Content-Length' in request.headers)
[   24s]     
[   24s]         if isinstance(timeout, tuple):
[   24s]             try:
[   24s]                 connect, read = timeout
[   24s]                 timeout = TimeoutSauce(connect=connect, read=read)
[   24s]             except ValueError as e:
[   24s]                 # this may raise a string formatting error.
[   24s]                 err = ("Invalid timeout {}. Pass a (connect, read) "
[   24s]                        "timeout tuple, or a single float to set "
[   24s]                        "both timeouts to the same value".format(timeout))
[   24s]                 raise ValueError(err)
[   24s]         elif isinstance(timeout, TimeoutSauce):
[   24s]             pass
[   24s]         else:
[   24s]             timeout = TimeoutSauce(connect=timeout, read=timeout)
[   24s]     
[   24s]         try:
[   24s]             if not chunked:
[   24s]                 resp = conn.urlopen(
[   24s]                     method=request.method,
[   24s]                     url=url,
[   24s]                     body=request.body,
[   24s]                     headers=request.headers,
[   24s]                     redirect=False,
[   24s]                     assert_same_host=False,
[   24s]                     preload_content=False,
[   24s]                     decode_content=False,
[   24s]                     retries=self.max_retries,
[   24s]                     timeout=timeout
[   24s]                 )
[   24s]     
[   24s]             # Send the request.
[   24s]             else:
[   24s]                 if hasattr(conn, 'proxy_pool'):
[   24s]                     conn = conn.proxy_pool
[   24s]     
[   24s]                 low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
[   24s]     
[   24s]                 try:
[   24s]                     low_conn.putrequest(request.method,
[   24s]                                         url,
[   24s]                                         skip_accept_encoding=True)
[   24s]     
[   24s]                     for header, value in request.headers.items():
[   24s]                         low_conn.putheader(header, value)
[   24s]     
[   24s]                     low_conn.endheaders()
[   24s]     
[   24s]                     for i in request.body:
[   24s]                         low_conn.send(hex(len(i))[2:].encode('utf-8'))
[   24s]                         low_conn.send(b'\r\n')
[   24s]                         low_conn.send(i)
[   24s]                         low_conn.send(b'\r\n')
[   24s]                     low_conn.send(b'0\r\n\r\n')
[   24s]     
[   24s]                     # Receive the response from the server
[   24s]                     try:
[   24s]                         # For Python 2.7, use buffering of HTTP responses
[   24s]                         r = low_conn.getresponse(buffering=True)
[   24s]                     except TypeError:
[   24s]                         # For compatibility with Python 3.3+
[   24s]                         r = low_conn.getresponse()
[   24s]     
[   24s]                     resp = HTTPResponse.from_httplib(
[   24s]                         r,
[   24s]                         pool=conn,
[   24s]                         connection=low_conn,
[   24s]                         preload_content=False,
[   24s]                         decode_content=False
[   24s]                     )
[   24s]                 except:
[   24s]                     # If we hit any problems here, clean up the connection.
[   24s]                     # Then, reraise so that we can handle the actual exception.
[   24s]                     low_conn.close()
[   24s]                     raise
[   24s]     
[   24s]         except (ProtocolError, socket.error) as err:
[   24s]             raise ConnectionError(err, request=request)
[   24s]     
[   24s]         except MaxRetryError as e:
[   24s]             if isinstance(e.reason, ConnectTimeoutError):
[   24s]                 # TODO: Remove this in 3.0.0: see #2811
[   24s]                 if not isinstance(e.reason, NewConnectionError):
[   24s]                     raise ConnectTimeout(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, ResponseError):
[   24s]                 raise RetryError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _ProxyError):
[   24s]                 raise ProxyError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _SSLError):
[   24s]                 # This branch is for urllib3 v1.22 and later.
[   24s] >               raise SSLError(e, request=request)
[   24s] E               SSLError: HTTPSConnectionPool(host='secure.github.com', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("partial wildcards in leftmost label are not supported: u'*secure.github.com'.",),))
[   24s] 
[   24s] /usr/lib/python2.7/site-packages/requests/adapters.py:514: SSLError
[   24s] ------------------------------ Captured log call -------------------------------
[   24s] ERROR    urllib3.connection:connection.py:404 Certificate did not match expected hostname: secure.github.com. Certificate: {u'subject': (((u'organizationName', u'*.secure.github.com'),), ((u'organizationalUnitName', u'Domain Control Validated'),), ((u'commonName', u'*.secure.github.com'),)), u'subjectAltName': ((u'DNS', u'*secure.github.com'), (u'DNS', 'secure.github.com'), (u'DNS', u'*')), u'notAfter': 'Sep 12 05:31:12 GMT'}
[   24s] __________________________________ test_json ___________________________________
[   24s] 
[   24s] response = {'boolean': False, 'integer': 1, 'string': 'asd'}
[   24s] 
[   24s]     @mocketize
[   24s]     def test_json(response):
[   24s]         url_to_mock = 'https://testme.org/json'
[   24s]     
[   24s]         Entry.single_register(
[   24s]             Entry.GET,
[   24s]             url_to_mock,
[   24s]             body=json.dumps(response),
[   24s]             headers={'content-type': 'application/json'})
[   24s]     
[   24s] >       mocked_response = requests.get(url_to_mock).json()
[   24s] 
[   24s] tests/main/test_https.py:33: 
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   24s]     return request('get', url, params=params, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   24s]     return session.request(method=method, url=url, **kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   24s]     resp = self.send(prep, **send_kwargs)
[   24s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   24s]     r = adapter.send(request, **kwargs)
[   24s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   24s] 
[   24s] self = <requests.adapters.HTTPAdapter object at 0x3ff93c7f940>
[   24s] request = <PreparedRequest [GET]>, stream = False
[   24s] timeout = <urllib3.util.timeout.Timeout object at 0x3ff93cc9710>, verify = True
[   24s] cert = None, proxies = OrderedDict()
[   24s] 
[   24s]     def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
[   24s]         """Sends PreparedRequest object. Returns Response object.
[   24s]     
[   24s]         :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
[   24s]         :param stream: (optional) Whether to stream the request content.
[   24s]         :param timeout: (optional) How long to wait for the server to send
[   24s]             data before giving up, as a float, or a :ref:`(connect timeout,
[   24s]             read timeout) <timeouts>` tuple.
[   24s]         :type timeout: float or tuple or urllib3 Timeout object
[   24s]         :param verify: (optional) Either a boolean, in which case it controls whether
[   24s]             we verify the server's TLS certificate, or a string, in which case it
[   24s]             must be a path to a CA bundle to use
[   24s]         :param cert: (optional) Any user-provided SSL certificate to be trusted.
[   24s]         :param proxies: (optional) The proxies dictionary to apply to the request.
[   24s]         :rtype: requests.Response
[   24s]         """
[   24s]     
[   24s]         try:
[   24s]             conn = self.get_connection(request.url, proxies)
[   24s]         except LocationValueError as e:
[   24s]             raise InvalidURL(e, request=request)
[   24s]     
[   24s]         self.cert_verify(conn, request.url, verify, cert)
[   24s]         url = self.request_url(request, proxies)
[   24s]         self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
[   24s]     
[   24s]         chunked = not (request.body is None or 'Content-Length' in request.headers)
[   24s]     
[   24s]         if isinstance(timeout, tuple):
[   24s]             try:
[   24s]                 connect, read = timeout
[   24s]                 timeout = TimeoutSauce(connect=connect, read=read)
[   24s]             except ValueError as e:
[   24s]                 # this may raise a string formatting error.
[   24s]                 err = ("Invalid timeout {}. Pass a (connect, read) "
[   24s]                        "timeout tuple, or a single float to set "
[   24s]                        "both timeouts to the same value".format(timeout))
[   24s]                 raise ValueError(err)
[   24s]         elif isinstance(timeout, TimeoutSauce):
[   24s]             pass
[   24s]         else:
[   24s]             timeout = TimeoutSauce(connect=timeout, read=timeout)
[   24s]     
[   24s]         try:
[   24s]             if not chunked:
[   24s]                 resp = conn.urlopen(
[   24s]                     method=request.method,
[   24s]                     url=url,
[   24s]                     body=request.body,
[   24s]                     headers=request.headers,
[   24s]                     redirect=False,
[   24s]                     assert_same_host=False,
[   24s]                     preload_content=False,
[   24s]                     decode_content=False,
[   24s]                     retries=self.max_retries,
[   24s]                     timeout=timeout
[   24s]                 )
[   24s]     
[   24s]             # Send the request.
[   24s]             else:
[   24s]                 if hasattr(conn, 'proxy_pool'):
[   24s]                     conn = conn.proxy_pool
[   24s]     
[   24s]                 low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
[   24s]     
[   24s]                 try:
[   24s]                     low_conn.putrequest(request.method,
[   24s]                                         url,
[   24s]                                         skip_accept_encoding=True)
[   24s]     
[   24s]                     for header, value in request.headers.items():
[   24s]                         low_conn.putheader(header, value)
[   24s]     
[   24s]                     low_conn.endheaders()
[   24s]     
[   24s]                     for i in request.body:
[   24s]                         low_conn.send(hex(len(i))[2:].encode('utf-8'))
[   24s]                         low_conn.send(b'\r\n')
[   24s]                         low_conn.send(i)
[   24s]                         low_conn.send(b'\r\n')
[   24s]                     low_conn.send(b'0\r\n\r\n')
[   24s]     
[   24s]                     # Receive the response from the server
[   24s]                     try:
[   24s]                         # For Python 2.7, use buffering of HTTP responses
[   24s]                         r = low_conn.getresponse(buffering=True)
[   24s]                     except TypeError:
[   24s]                         # For compatibility with Python 3.3+
[   24s]                         r = low_conn.getresponse()
[   24s]     
[   24s]                     resp = HTTPResponse.from_httplib(
[   24s]                         r,
[   24s]                         pool=conn,
[   24s]                         connection=low_conn,
[   24s]                         preload_content=False,
[   24s]                         decode_content=False
[   24s]                     )
[   24s]                 except:
[   24s]                     # If we hit any problems here, clean up the connection.
[   24s]                     # Then, reraise so that we can handle the actual exception.
[   24s]                     low_conn.close()
[   24s]                     raise
[   24s]     
[   24s]         except (ProtocolError, socket.error) as err:
[   24s]             raise ConnectionError(err, request=request)
[   24s]     
[   24s]         except MaxRetryError as e:
[   24s]             if isinstance(e.reason, ConnectTimeoutError):
[   24s]                 # TODO: Remove this in 3.0.0: see #2811
[   24s]                 if not isinstance(e.reason, NewConnectionError):
[   24s]                     raise ConnectTimeout(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, ResponseError):
[   24s]                 raise RetryError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _ProxyError):
[   24s]                 raise ProxyError(e, request=request)
[   24s]     
[   24s]             if isinstance(e.reason, _SSLError):
[   24s]                 # This branch is for urllib3 v1.22 and later.
[   24s] >               raise SSLError(e, request=request)
[   24s] E               SSLError: HTTPSConnectionPool(host='testme.org', port=443): Max retries exceeded with url: /json (Caused by SSLError(CertificateError("partial wildcards in leftmost label are not supported: u'*testme.org'.",),))
[   24s] 
[   24s] /usr/lib/python2.7/site-packages/requests/adapters.py:514: SSLError

I have enabled all the extras of requests, hoping that might fix it, but it didnt help.

Packaging project at

https://build.opensuse.org/package/show/home:jayvdb:py-new/python-mocket

Mocket fails when there are Redis calls using `hiredis`

Describe the bug
I have a django based software that makes http requests using requests and uses django-redis to store some data of what it has fetched. I have noticed that if the first network call in a TestCase is for redis then mocket mocks redis, otherwise it mocks only requests.

To Reproduce

import requests
from django.core.cache import cache
from django.test.testcases import TestCase
from mocket import Mocket, mocketize
from mocket.mockhttp import Entry

class TestMocketStrangeBehaviour(TestCase):
    def tearDown(self):
        cache.clear()

    @mocketize
    def test_b_random_url(self):
        url = "http://www.example.com"
        Entry.single_register(Entry.GET, url)

        requests.get(url)

        self.assertTrue(Mocket.has_requests())

    @mocketize
    def test_a_random_url(self):
        url = "http://www.example.com"
        Entry.single_register(Entry.GET, url)

        cache.set("key", "value")

        requests.get(url)

        self.assertTrue(Mocket.has_requests())

will throw an exception

Error
Traceback (most recent call last):
  File "<decorator-gen-2>", line 2, in test_a_random_url
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/mocket/mocket.py", line 627, in wrapper
    t(*args, **kw)
  File "/Users/anas/Projects/random_project/random_app/tests.py", line 34, in test_a_random_url
    cache.set("key", "value")
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/django_redis/cache.py", line 27, in _decorator
    return method(self, *args, **kwargs)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/django_redis/cache.py", line 76, in set
    return self.client.set(*args, **kwargs)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/django_redis/client/default.py", line 156, in set
    return bool(client.set(nkey, nvalue, nx=nx, px=timeout, xx=xx))
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/client.py", line 1801, in set
    return self.execute_command('SET', *pieces)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/client.py", line 898, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 1192, in get_connection
    connection.connect()
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 567, in connect
    self.on_connect()
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 664, in on_connect
    if nativestr(self.read_response()) != 'OK':
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 739, in read_response
    response = self._parser.read_response()
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 470, in read_response
    self.read_from_socket()
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/connection.py", line 427, in read_from_socket
    bufflen = recv_into(self._sock, self._buffer)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/redis/_compat.py", line 75, in recv_into
    return sock.recv_into(*args, **kwargs)
  File "/Users/anas/Projects/random_project/.venv/lib/python3.7/site-packages/mocket/mocket.py", line 264, in recv_into
    return buffer.write(self.read(buffersize))
AttributeError: 'bytearray' object has no attribute 'write'

If you swap names of the test functions it works as expected.

Expected behavior
Choose what to mock with some flag.

Additional context
It happens in macOS and ubuntu, with python3.7 and also 3.8

md5 hashing is slow

In https://github.com/mindflayer/python-mocket/blob/master/mocket/mocket.py#L265 we hash the response so we'll have a key. Using md5 to do so is slower than what can be done.
There are more modern hashing algorithms such as xxhash.
Would you accept a PR that replaces MD5 with a non-cryptographic hash function?

redis.exceptions.ConnectionError: Connection not ready with redis==3.2.0

No recv_into ?

Hi,

I recently switched my code to using recv_into, which broke my tests that use mocket. Does mocket support recv_into ? A quick look at the code suggests it doesn't.

Query params should not need to be encoded in url

First of all, thank you for your amazing job building this library.

I translated all my previous tests using httpretty to python mocket because the former seems to not be maintained as it supposed to.

The transition was very easy but I have to point out some small stuff, first of all there's no Mocket.has_request(), but this is minor and I understand you you would not include it.

But I found an issue with queryparams, they should not be in the url while registering the mock,
if you have a lot of params and you're using a library like requests to make requests, you don't know the order of the parameters in the url and this could be a pain in the ass to test with mocket. Is there a way to register only the path of the url without adding parameters? I would like just to check them later.

Thank you and again, I really appreciate your work and like the library.

Can it mock socket errors?

It would be awesome if mocket could also mock all kinds of socket errors on demand.
I'd love to use this library for negative tests as well as regular tests...

Support for async test methods

Async test methods are not fully supported.

From python 3.8 IsolatedAsyncioTestCase is available and with them you can write async def test methods.

Here is a snippet for testing

from unittest import IsolatedAsyncioTestCase

from aiohttp import ClientError, ClientSession
from mocket import Mocket, mocketize
from mocket.mockhttp import Entry


class TestAsyncioMocket(IsolatedAsyncioTestCase):
    @mocketize()
    async def test_example(self):
        url = 'example.com'
        Entry.single_register(Entry.GET, url, status=200)

        async with ClientSession() as session:
            async with session.get(url) as response:
                status = response.status
                text = await response.text('utf-8')

                self.assertEqual(status, 200)
                self.assertEqual(text, '')

        self.assertTrue(Mocket.has_requests())

Gives back this warning

/Users/anas/.envs/mocket-MXOZpDpu/lib/python3.8/site-packages/mocket/mocket.py:613: RuntimeWarning: coroutine 'TestAsyncioMocket.test_example' was never awaited
  t(*args, **kw)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

I have a PR ready for solving this issue, I just have a weird warning that I want to share with you

Problem related to SSL and Python 3.7

I'm attempting to use Mocket in an application that needs to retrieve an auth token using REST call to a remote endpoint. This call is throwing the following error:

super(SSLSocket, self).__init__(**kwargs)
TypeError: super(type, obj): obj must be an instance or subtype of type

Everything works fine without Mocket.enable() being called, as soon as I enable it the program will crash with that error.

Any idea what I need to do to get Mocket working?

`makefile` is broken

Thanks to @KyleKing for reporting it and fixing mocketoy in #128.

Mocking https connections fails when pyOpenSSL is installed

If I pip install pyOpenSSL==17.5.0 and then run the pytest tests/main/test_https.py, I get an exception that looks like this:

if isinstance(e.reason, _SSLError):
                # This branch is for urllib3 v1.22 and later.
>               raise SSLError(e, request=request)
E               requests.exceptions.SSLError: HTTPSConnectionPool(host='testme.org', port=443): Max retries exceeded with url: /json (Caused by SSLError(SSLError("bad handshake: SysCallError(9, 'EBADF')",),))

urllib3 does some things differently when it detects that the OpenSSL module is available. I found prior art in getting mocking of pyOpenSSL in HTTPretty (https://github.com/gabrielfalcao/HTTPretty/pull/215/files) but it looks pretty gnarly.

Is adding support for environments with PyOpenSSL installed anywhere on the roadmap?

Documentation..... is there any?

This looks super helpful, but I cannot find any docs or examples anywhere. Am I missing something?

Updating a request deletes the newest entry from the request list

Unfortunately, #67 doesn't work correctly after all. This small change to the test illustrates the issue:

diff --git a/tests/tests35/test_http_aiohttp.py b/tests/tests35/test_http_aiohttp.py
index 0edc396..2fe1557 100644
--- a/tests/tests35/test_http_aiohttp.py
+++ b/tests/tests35/test_http_aiohttp.py
@@ -29,6 +29,7 @@ class AioHttpEntryTestCase(TestCase):
                     async with session.post(url, data=body * 6) as post_response:
                         assert post_response.status == 201
                         assert await post_response.text() == body * 2
+                        assert Mocket.last_request().method == 'POST'
                         assert Mocket.last_request().body == body * 6

         loop = asyncio.get_event_loop()

The problem is that once the body data gets collected, the call to can_handle() causes the last request to be removed from the list and the body to be appended to the request before the last:
https://github.com/mindflayer/python-mocket/blob/master/mocket/mockhttp.py#L137

The assertion above just illustrates that Mocket.last_request() is actually the first one (the GET request) and not the POST request that was done afterwards.

Properly handle requests that do multiple writes (e.g. by aiohttp 3)

The aiohttp client in version 3 does (at least) two writes if the request has a body: one for the headers, a second for the body. This was done in aio-libs/aiohttp#2126 and is documented in the "Are there changes in behavior for the user?" section of the description. Check out the diff of the write_headers function.

With mocket, this is not handled correctly. The body is lost. Would it be possible to implement proper handling of this scenario?

Example test failures: https://travis-ci.org/sjaensch/aiobravado/jobs/343058521

Test 1 source
Test 2 source

mockhttp.Entry.can_handle doesn't take method into account

if there are two Entries with the same path but one handling e.g. GET and the other POST/PUT, only one of the Entries will ever be returned.

Make magic dependency optional

Installing the library using pip install mocket will lead to an exception if libmagic is not installed:

____________________ ERROR collecting tests/test_actions.py ____________________
ImportError while importing test module '/Users/tmbo/lastmile/bot-ai/rasa_core/tests/test_actions.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_actions.py:5: in <module>
    from mocket.mockhttp import Entry
../../../.virtualenvs/py3/lib/python3.6/site-packages/mocket/mockhttp.py:6: in <module>
    import magic
../../../.virtualenvs/py3/lib/python3.6/site-packages/magic.py:181: in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
E   ImportError: failed to find libmagic.  Check your installation

I checked the code, and although I see that it is nice to unburdon the user from specifying the response content type, but it is also kind of annoying to burdon everyone with installing libmagic.

Can we make that dependency optional? (otherwise I can't add this as a dependency to https://github.com/RasaHQ/rasa_nlu)

requirements.txt missing

Describe the bug
The requirements.txt referenced in setup.py is not part of the Git repo.

python-mocket/setup.py

Lines 11 to 17 in 343f5b9

 install_requires = [ 

 line 

 for line in io.open( 

 os.path.join(os.path.dirname(__file__), "requirements.txt") 

 ).readlines() 

 if not line.startswith("-i") 

 ]

To Reproduce
n/a

Expected behavior
Include the requirements.txt so python setup.py install can work.

Add 'pook' integration

As result of a discussion I had on the pook's GitHub (at h2non/pook#33), I am going to add a pook.engine.MockEngine that will support Mocket.

Question: Can I test websocket things?

Can I test websocket things such as aiohttp websocket client?

Fix simple typo: retur -> return

There is a small typo in tests/main/test_httpretty.py.
Should read return rather than retur.

sdist doesnt include init.py

The current packaging excludes tests from the package source with exclude_packages = ('tests', ) in setup.py , however this oddly only removes tests/__init__.py . To declare that the tests are not part of the source, it needs to be exclude_packages = ('tests', 'tests.*').

And as this is a testing library, IMO it is very important that the test suite is included in the sdist, so tests should be added via MANIFEST.in, so that they are in the sdist, but not in the wheel.

Issue with decorator and patches used together

Describe the bug
In my project when I upgraded mocket I noticed an issue on all the tests that had @mocketize decorator and @patches (from unittest.mock package). I tried different versions and it seems to me that from 3.9.4 there is the issue. I am using django's TestCase I am sure the issue is showing up regardless of the content of the test and without any setUp or tearDown methods. I also tried with python's plain unittest.TestCase with the following code and the issue still happens.

class TestRandom(TestCase):
    @mocketize
    @patch("path.to.function")
    def test_random(
        self,
        method_patch,
    ):
        pass

The error is the following, it seems to me that the function does not get the patches as arguments.

Error
Traceback (most recent call last):
  File "/Users/anas/Projects/project_name/.venv/lib/python3.7/site-packages/decorator.py", line 231, in fun
    args, kw = fix(args, kw, sig)
  File "/Users/anas/Projects/project_name/.venv/lib/python3.7/site-packages/decorator.py", line 203, in fix
    ba = sig.bind(*args, **kwargs)
  File "/Users/anas/.pyenv/versions/3.7.9/lib/python3.7/inspect.py", line 3015, in bind
    return args[0]._bind(args[1:], kwargs)
  File "/Users/anas/.pyenv/versions/3.7.9/lib/python3.7/inspect.py", line 2930, in _bind
    raise TypeError(msg) from None
TypeError: missing a required argument: 'method_patch'

To Reproduce
This is complicated, I am sure the only change that makes the tests fail is to upgrade mocket after 3.9.4 included, with 3.9.3 it works fine.
I am trying to setup a dummy repo but the issue does not show up.

Expected behavior
The patches should be passed as arguments as usual.

I tested only with python 3.7 regardless of the environment, on docker with python3.7-alpine and on my mac using pyenv.

Additional context
I am sorry I did not provide further details but I can't share the code which is having the issue, I will try to create a dummy project with the same dependencies as mine and work my way down removing dependencies. It will take me some time but in the meantime I am opening the issue hoping that maybe you can spot the problem before that.

For the moment my workaround is to use Mocketizer as context manager

How about mocking sockets with trio ?

Using mocket with trio, I get :

TypeError: descriptor 'send' requires a '_socket.socket' object but received a 'MocketSocket'

Is there any known way for this to work ?

Adding record/playback functionality?

I'd like to be able to incorporate Mocket with a project like vcrpy (see also) in order to be able to automatically save stored responses for use in tests.

Is this a feature that you'd like to see added to Mocket itself, or do you think I should just import and use Mocket as part of a separate wrapper library?

JSON body sent as second `sendall` breaks things

This is still an issue; the same 'entry' is reused for requests to the same host/port. mockhttp.Entry tracks _sent_data, and each request appends to _sent_data. but the entire _sent_data is sent to super.collect(), which is where it's passed into the Request class, and everything but the first _sent_data is truncated when the data is parsed.

Originally posted by @ajhodges in #14 (comment)

.shutdown() of the socket is not available

We need to add shutdown method to mocket.py.

Mocket should offer HTTPretty users a simple way to switch to it

mockhttp.Request doesn't like e.g. application/octet-stream contents

When a received request is e.g. a PUT with application/octet-stream with raw non-ascii data, using mockhttp fails on this line:
_, self.body = decode_utf8(data).split('\r\n\r\n', 1)

Test file only failing if run alone

Here's the code snippet:

import json

import aiohttp
import asyncio
import async_timeout
from unittest import TestCase
import unittest

from mocket.plugins.httpretty import HTTPretty, httprettified


class AioHttpEntryTestCase(TestCase):
    @httprettified
    def test_https_session(self):
        url = 'https://httpbin.org/ip'
        HTTPretty.register_uri(
            HTTPretty.GET,
            url,
            body=json.dumps(dict(origin='127.0.0.1')),
        )

        async def main(l):
            async with aiohttp.ClientSession(loop=l) as session:
                with async_timeout.timeout(3):
                    async with session.get(url) as get_response:
                        assert get_response.status == 200
                        assert await get_response.text() == '{"origin": "127.0.0.1"}'

        loop = asyncio.get_event_loop()
        loop.set_debug(True)
        loop.run_until_complete(main(loop))

if __name__ == '__main__':
    unittest.main()

Exception:

E/usr/lib/python3.6/unittest/case.py:633: ResourceWarning: unclosed <socket object, fd=5, family=1, type=1, proto=0>
  outcome.errors.clear()
/usr/lib/python3.6/asyncio/base_events.py:516: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
  source=self)
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=False debug=False>>
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/base_events.py", line 518, in __del__
    self.close()
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 63, in close
    super().close()
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 110, in close
    self._close_self_pipe()
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 120, in _close_self_pipe
    self._remove_reader(self._ssock.fileno())
AttributeError: '_UnixSelectorEventLoop' object has no attribute '_ssock'

======================================================================
ERROR: test_https_session (__main__.AioHttpEntryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "</home/dvnguyen/code/pyvenvs/redesign36/lib/python3.6/site-packages/decorator.py:decorator-gen-1>", line 2, in test_https_session
  File "/home/dvnguyen/code/pyvenvs/redesign36/lib/python3.6/site-packages/mocket/mocket.py", line 572, in wrapper
    t(*args, **kw)
  File "moctests.py", line 29, in test_https_session
    loop = asyncio.get_event_loop()
  File "/usr/lib/python3.6/asyncio/events.py", line 694, in get_event_loop
    return get_event_loop_policy().get_event_loop()
  File "/usr/lib/python3.6/asyncio/events.py", line 599, in get_event_loop
    self.set_event_loop(self.new_event_loop())
  File "/usr/lib/python3.6/asyncio/events.py", line 617, in new_event_loop
    return self._loop_factory()
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 56, in __init__
    super().__init__(selector)
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 67, in __init__
    self._make_self_pipe()
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 129, in _make_self_pipe
    self._ssock, self._csock = self._socketpair()
  File "/usr/lib/python3.6/asyncio/unix_events.py", line 60, in _socketpair
    return socket.socketpair()
  File "/usr/lib/python3.6/socket.py", line 490, in socketpair
    a = socket(family, type, proto, a.detach())
TypeError: __init__() takes from 1 to 4 positional arguments but 5 were given

----------------------------------------------------------------------
Ran 1 test in 0.004s

FAILED (errors=1)

OS: Ubuntu 18.10
Python version: 3.5.6, 3.6.7, 3.7.1
mocket version: 2.7.1

BlockingIOError() when mocking null-byte responses

Dear Giorgio,

first things first: Thanks for conceiving and maintaining this excellent pytest mocking module.

We just found MocketEntry instances pretending to respond with single null bytes like

MocketEntry(location=(None, None), responses=[0x00])

will not yield their responses properly due to the boolean-style test within MocketSocket.recv.

Instead, the BlockingIOError() will be raised.

With kind regards,
Andreas.

#upload_test.py
import json

from mocket import mocketize
from mocket.mockhttp import Entry
import requests
import pytest

@pytest.fixture
def response():
    return {
        "integer": 1,
        "string": "asd",
        "boolean": False,
    }

@mocketize  # use its decorator
def test_json(response):
    url_to_mock = 'https://testme.org/json'
    Entry.single_register(
        Entry.POST,
        url_to_mock,
        body=response,
        headers={'content-type': 'application/json'}
    )

    data = {}
    file_obj = open('./test/data/sdk-skill.zip', 'rb')
    files = { 'content': file_obj }
    resp = requests.post(url_to_mock, files=files, data=data, verify=False)
    mocked_response = resp.json()

    assert response == mocked_response

When I run pytest test/unit/upload_test.py, it gives

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 123: invalid start byte

import asyncio
from unittest import TestCase

from mocket import mocketize
from mocket.mockhttp import Entry
from tornado.curl_httpclient import CurlAsyncHTTPClient


class AioHttpEntryTestCase(TestCase):

  @mocketize
  def test_http_session_2(self):
    url = 'http://httpbin.org/ip'
    body = "asd" * 100
    Entry.single_register(Entry.GET, url, body=body, status=404)
    Entry.single_register(Entry.POST, url, body=body * 2, status=201)

    async def main(l):
      client = CurlAsyncHTTPClient()
      res = await client.fetch(url)
      assert res.body == body
      assert res.code == 401

    loop = asyncio.get_event_loop()
    loop.run_until_complete(main(loop))

is it possible to mocke pycurl?

	install_requires = [
	line
	for line in io.open(
	os.path.join(os.path.dirname(__file__), "requirements.txt")
	).readlines()
	if not line.startswith("-i")
	]