I have a using Athena + SQLAlchemy to run a query and have the results of that

Handle InvalidRequestException errors raised about pyathena HOT 3 CLOSED

laughingman7743 commented on May 23, 2024

Handle InvalidRequestException errors raised

from pyathena.

Comments (3)

koshy1123 commented on May 23, 2024

My bad - the actual error I get is a sqlalchemy.exc.DatabaseError and I can except that and my script runs fine, however in my run, I still continue to see this

Failed to execute query.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pyathena/common.py", line 165, in _execute
    **request)
  File "/usr/local/lib/python3.6/site-packages/pyathena/util.py", line 44, in retry_api_call
    return retry(func, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 330, in call
    start_time=start_time)
  File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 279, in iter
    return fut.result()
  File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 333, in call
    result = fn(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: Your query has exceeded the maximum query length of 262144 bytes. Please reduce the length of your query and try again.  If you continue to see this issue after reducing your query length,  contact customer support for further assistance.

from pyathena.

laughingman7743 commented on May 23, 2024

All exceptions raised in this library are wrapped in the exception class described in error.py.
https://github.com/laughingman7743/PyAthena/blob/master/pyathena/error.py
https://www.python.org/dev/peps/pep-0249/#exceptions

If you want to handle InvalidRequestException, it is a good idea to do the following.

from pyathena import connect, DatabaseError

cursor = connect(s3_staging_dir='s3://YOUR_BUCKET/path/to/',
                 region_name='us-west-2').cursor()
try:
    cursor.execute("SELECT * FROM one_row orderby hoge")
except DatabaseError as e:
    print(type(e))  # pyathena.error.DatabaseError
    print(type(e.__cause__))  # botocore.errorfactory.InvalidRequestException
    # https://github.com/boto/botocore/pull/1113
    if isinstance(e.__cause__,
                  cursor._connection.exceptions.InvalidRequestException):
        print('catch InvalidRequestException')

from pyathena.

koshy1123 commented on May 23, 2024

Gotcha - I ended up implementing a version of this^, thanks! Only concerned that I was using a private variable _connection in my script, otherwise works!

Closing this issue.

from pyathena.

Recommend Projects