The ipython-restmagic from b3b

Remove trailing newlines from a cell input

IPython adds newline to a cell input, which results in a submission of a wrong data.
For example, "1234\n" body is POSTed instead of "1234":

%%rest -v POST https://httpbin.org/post
    
1234

< POST /post HTTP/1.1
< Host: httpbin.org
< User-Agent: python-requests/2.19.1
< Accept-Encoding: gzip, deflate
< Accept: */*
< Connection: keep-alive
< Content-Length: 5
< 
< 1234

> HTTP/? 200 OK
> Connection: keep-alive
> Access-Control-Allow-Origin: *
> Content-Length: 406
> Server: gunicorn/19.9.0
> Date: Sun, 27 Sep 2020 16:29:55 GMT
> Content-Type: application/json
> Access-Control-Allow-Credentials: true
> 
{
  "args": {}, 
  "data": "1234\n", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "5", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.19.1", 
    "X-Amzn-Trace-Id": "Root=1"
  }, 
  "json": 1234, 
  "origin": "1.1.1.1", 
  "url": "https://httpbin.org/post"
}

minimum working magic

This will be enough for the first version:

UnicodeEncodeError: 'latin-1' codec can't encode characters

Execution of

%%rest -v POST https://httpbin.org/post

поле=Уникод

gives:

Request was not completed.
An exception has occurred, use %tb to see the full traceback.

UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-3: Body ('поле') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

🧚🤖 Pixeebot Activity Dashboard

👋 This dashboard summarizes my activity on the repository, including available improvement opportunities.

Recommendations

Last analysis: Apr 29 | Next scheduled analysis: Apr 30

Open

✅ Nice work, you're all caught up!

Available

✅ Nothing yet, but I'm continuing to monitor your PRs.

Metrics

What would you like to see here? Let us know!

Resources

📚 Quick links
Pixee Docs | Codemodder by Pixee

🧰 Tools I work with
Sonar, CodeQL, Semgrep

🚀 Pixee CLI
The power of my codemods in your local development environment. Learn more

💬 Reach out
Feedback | Support

❤️ Follow, share, and engage with Pixee: GitHub | LinkedIn | Slack

Add connection timeout option

There should be the way to specify connection timeout,
default timeout should be set to reasonably short value.

Add the ability to display only parts of API responses

It is common, when you need to display only a small part of the bulk API response.
Support for some kind of a query language, like XPath, could be very handy.

Add notebook with functional tests

Notebooks could be used for functional tests:

save executed notebook and log of HTTP responses
execute notebook during the tests, using recorded responses instead of real HTTP calls
check that notebook is executed successfully, and all cells output is identical to original.

Do not display chained exceptions in traceback message

It is hard to spot a cause of an error because of chained exceptions tracebacks ("During handling of the above exception, another exception occurred").

Plus, current IPython version adds additional exception ipython/ipython#12104 .

For example, for an obvious connection error traceback is:
%rest someunknowndomain

Request was not completed.
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
  File ".../lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File ".../lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.8/socket.py", line 914, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File ".../lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File ".../lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File ".../lib/python3.8/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File ".../lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File ".../lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File ".../lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='someunknowndomain', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../lib/python3.8/site-packages/restmagic/magic.py", line 178, in rest
    response = sender.send(
  File ".../lib/python3.8/site-packages/restmagic/sender.py", line 50, in send
    self.response = session.send(
  File ".../lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File ".../lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='someunknowndomain', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../lib/python3.8/site-packages/IPython/core/ultratb.py", line 1171, in get_records
    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)
  File ".../lib/python3.8/site-packages/IPython/core/ultratb.py", line 316, in wrapped
    return f(*args, **kwargs)
  File ".../lib/python3.8/site-packages/IPython/core/ultratb.py", line 352, in _fixed_getinnerframes
    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
  File "/usr/lib/python3.8/inspect.py", line 1503, in getinnerframes
    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)
AttributeError: 'tuple' object has no attribute 'tb_frame'
---------------------------------------------------------------------------
gaierror                                  Traceback (most recent call last)
~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connection.py in _new_conn(self)
    158         try:
--> 159             conn = connection.create_connection(
    160                 (self._dns_host, self.port), self.timeout, **extra_kw

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     60 
---> 61     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
     62         af, socktype, proto, canonname, sa = res

/usr/lib/python3.8/socket.py in getaddrinfo(host, port, family, type, proto, flags)
    913     addrlist = []
--> 914     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
    915         af, socktype, proto, canonname, sa = res

gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    669             # Make the request on the httplib connection object.
--> 670             httplib_response = self._make_request(
    671                 conn,

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    975         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 976             conn.connect()
    977 

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connection.py in connect(self)
    307         # Add certificate verification
--> 308         conn = self._new_conn()
    309         hostname = self.host

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connection.py in _new_conn(self)
    170         except SocketError as e:
--> 171             raise NewConnectionError(
    172                 self, "Failed to establish a new connection: %s" % e

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~/.virtualenvs/j38/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    438             if not chunked:
--> 439                 resp = conn.urlopen(
    440                     method=request.method,

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    723 
--> 724             retries = retries.increment(
    725                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

~/.virtualenvs/j38/lib/python3.8/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    438         if new_retry.is_exhausted():
--> 439             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    440 

MaxRetryError: HTTPSConnectionPool(host='someunknowndomain', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
~/.virtualenvs/j38/lib/python3.8/site-packages/restmagic/magic.py in rest(self, line, cell)
    177         try:
--> 178             response = sender.send(
    179                 RESTRequest('GET', 'https://') + root + rest_request,

~/.virtualenvs/j38/lib/python3.8/site-packages/restmagic/sender.py in send(self, rest_request, verify, proxy, max_redirects, timeout)
     49             warnings.filterwarnings("ignore", category=InsecureRequestWarning)
---> 50             self.response = session.send(
     51                 prepared_request,

~/.virtualenvs/j38/lib/python3.8/site-packages/requests/sessions.py in send(self, request, **kwargs)
    642         # Send the request
--> 643         r = adapter.send(request, **kwargs)
    644 

~/.virtualenvs/j38/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    515 
--> 516             raise ConnectionError(e, request=request)
    517 

ConnectionError: HTTPSConnectionPool(host='someunknowndomain', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8d28484af0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-3-465dfb032bab> in <module>
----> 1 get_ipython().run_line_magic('rest', 'someunknowndomain')

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2324                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2325             with self.builtin_trap:
-> 2326                 result = fn(*args, **kwargs)
   2327             return result
   2328 

<decorator-gen-130> in rest(self, line, cell)

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

<decorator-gen-129> in rest(self, line, cell)

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

~/.virtualenvs/j38/lib/python3.8/site-packages/restmagic/magic.py in rest(self, line, cell)
    190         except Exception:
    191             print("Request was not completed.", file=sys.stderr)
--> 192             self.shell.showtraceback(exception_only=True)
    193             return None
    194 

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/interactiveshell.py in showtraceback(self, exc_tuple, filename, tb_offset, exception_only, running_compiled_code)
   2035                     stb = ['An exception has occurred, use %tb to see '
   2036                            'the full traceback.\n']
-> 2037                     stb.extend(self.InteractiveTB.get_exception_only(etype,
   2038                                                                      value))
   2039                 else:

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in get_exception_only(self, etype, value)
    823         value : exception value
    824         """
--> 825         return ListTB.structured_traceback(self, etype, value)
    826 
    827     def show_exception_only(self, etype, evalue):

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in structured_traceback(self, etype, evalue, etb, tb_offset, context)
    698             chained_exceptions_tb_offset = 0
    699             out_list = (
--> 700                 self.structured_traceback(
    701                     etype, evalue, (etb, chained_exc_ids),
    702                     chained_exceptions_tb_offset, context)

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in structured_traceback(self, etype, value, tb, tb_offset, number_of_lines_of_context)
   1435         else:
   1436             self.tb = tb
-> 1437         return FormattedTB.structured_traceback(
   1438             self, etype, value, tb, tb_offset, number_of_lines_of_context)
   1439 

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in structured_traceback(self, etype, value, tb, tb_offset, number_of_lines_of_context)
   1335         if mode in self.verbose_modes:
   1336             # Verbose modes need a full traceback
-> 1337             return VerboseTB.structured_traceback(
   1338                 self, etype, value, tb, tb_offset, number_of_lines_of_context
   1339             )

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in structured_traceback(self, etype, evalue, etb, tb_offset, number_of_lines_of_context)
   1192         """Return a nice text document describing the traceback."""
   1193 
-> 1194         formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
   1195                                                                tb_offset)
   1196 

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in format_exception_as_a_whole(self, etype, evalue, etb, number_of_lines_of_context, tb_offset)
   1150 
   1151 
-> 1152         last_unique, recursion_repeat = find_recursion(orig_etype, evalue, records)
   1153 
   1154         frames = self.format_records(records, last_unique, recursion_repeat)

~/.virtualenvs/j38/lib/python3.8/site-packages/IPython/core/ultratb.py in find_recursion(etype, value, records)
    451     # first frame (from in to out) that looks different.
    452     if not is_recursion_error(etype, value, records):
--> 453         return len(records), 0
    454 
    455     # Select filename, lineno, func_name to track frames with

TypeError: object of type 'NoneType' has no len()

Add response processor option

There should be the way to set callable on response, that could possibly modify the response for a pretty display.

b3b / ipython-restmagic Goto Github PK

ipython-restmagic's People

Contributors

Stargazers

Watchers

Forkers

ipython-restmagic's Issues

Recommendations

Open

Available

Metrics

Resources

Recommend Projects

Recommend Topics

Recommend Org