google / protorpc Goto Github PK

View Code? Open in Web Editor NEW

67.0 14.0 37.0 3 MB

License: Apache License 2.0

Python 80.27% HTML 0.42% Shell 0.07% JavaScript 18.42% GAP 0.83%

protorpc's Introduction

ProtoRPC

protorpc's People

Contributors

Stargazers

Watchers

protorpc's Issues

0.11.0 release breaks apitools 0.4.11

Source of googleapis/google-cloud-python#1133

  File "../gcloud-python/gcloud/storage/blob.py", line 28, in <module>
    from apitools.base.py import http_wrapper
  File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/__init__.py", line 10, in <module>
    from apitools.base.py.extra_types import *
  File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/extra_types.py", line 38, in <module>
    class DateField(messages.Field):
  File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/extra_types.py", line 50, in DateField
    class __metaclass__(messages.Field.__metaclass__):
AttributeError: type object 'Field' has no attribute '__metaclass__'

protorpc source distributions (.tar.gz) do not include subdirectories, e.g. "wsgi"

This results in errors like the following when using the App Engine "endpoints" library:

ERROR    2018-05-21 14:00:19,811 wsgi.py:263]
Traceback (most recent call last):
  File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/opt/root/src/appengine-cloudsql-benchmark/main.py", line 27, in <module>
    import endpoints
  File "/opt/root/src/appengine-cloudsql-benchmark/lib/endpoints/__init__.py", line 27, in <module>
    from .apiserving import *
  File "/opt/root/src/appengine-cloudsql-benchmark/lib/endpoints/apiserving.py", line 81, in <module>
    from protorpc.wsgi import service as wsgi_service
ImportError: No module named wsgi

The solution is to do make the following change to setup.py:

diff --git a/setup.py b/setup.py
index 2c83901..0ff14c0 100644
--- a/setup.py
+++ b/setup.py
@@ -18,7 +18,7 @@

 import platform

-from setuptools import setup
+from setuptools import find_packages, setup

 # Configure the required packages and scripts to install, depending on
 # Python version and OS.
@@ -34,9 +34,6 @@ if py_version < '2.6':
   REQUIRED_PACKAGES.append('simplejson')

 _PROTORPC_VERSION = '0.12.0'
-packages = [
-    'protorpc',
-]

 setup(
     name='protorpc',
@@ -46,7 +43,7 @@ setup(
     author='Google Inc.',
     author_email='[email protected]',
     # Contained modules and scripts.
-    packages=packages,
+    packages=find_packages(),
     entry_points={
         'console_scripts': CONSOLE_SCRIPTS,
         },

NameError: global name 'Enum' is not defined

I pip install gsutil on a Mac OS 10.10 laptop and receive this error. I was under the impression that enum was a python 3.4 feature. The pypi package may need updating to list this as a dependency?

six 1.11.0 causes TypeError: Error when calling the metaclass bases

This is the same error seen in apitools here: google/apitools#175

How you import ProtocolBuffer?

Hi, i'm new in this. I tried to run the project but give me an error in

google.net.proto import ProtocolBuffer

note: win 8 with python 2.7.7

Python 3.7+ support

Starting from Python 3.7 async is a reserved keyword and cannot be used, as it causes SyntaxError:

lib/python3.7/site-packages/protorpc/remote.py", line 581
    return async_method(self.async, *args, **kwargs).response
                                 ^
SyntaxError: invalid syntax

Bring `protorpc` Python3 support.

...and Travis support.

See @craigcitro comment.

/cc @tseaver

I'm using Google NDB, to do a post put hook to generate a search Index. However, I am receiving the error below when it attempts to generate the document. I'm not sure why I am getting this as all my types. All the types are already in Unicode when checking the type. Unsure if this should be here or elsewhere...

GAE Standard, Py27

My Model:

class DirectionModel(ndb.Model):
    created_at = ndb.DateTimeProperty(auto_now_add=True)
    deviceId = ndb.StringProperty()
    deviceName = ndb.StringProperty()
    updated_at = ndb.DateTimeProperty(auto_now=True)
    settings = ndb.StringProperty(indexed=False)
    keywords = ndb.StringProperty(repeated=True)
    testId = ndb.StringProperty()

    def _post_put_hook(self, future):
        doc = search.Document(doc_id=self.key.id(), fields=[
            search.TextField(name='deviceId', value=self.deviceId),
            search.TextField(name='deviceName', value=self.deviceName),
            search.TextField(name='settings', value=self.settings),
            search.TextField(name='testId', value=self.testId)
        ])
        search.Index('cookingDirections').put(doc)

Encountered unexpected error from ProtoRPC method implementation: TypeError (coercing to Unicode: need string or buffer, long found) (/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/wsgi/service.py:191)
Traceback (most recent call last):
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/wsgi/service.py", line 181, in protorpc_service_app
    response = method(instance, request)
  File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/lib/endpoints/api_config.py", line 1272, in invoke_remote
    return remote_method(service_instance, request)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/remote.py", line 414, in invoke_remote_method
    response = method(service_instance, request)
  File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/main.py", line 135, in create
    keywords=keywords.split(',')).put()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3458, in _put
    return self._put_async(**ctx_options).get_result()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 378, in check_success
    self.wait()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 362, in wait
    if not ev.run1():
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 268, in run1
    delay = self.run0()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 230, in run0
    callback(*args, **kwds)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 533, in _on_future_completion
    self._help_tasklet_along(ns, ds_conn, gen, val)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 444, in _help_tasklet_along
    self.set_result(result)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 327, in set_result
    callback(*args, **kwds)
  File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/main.py", line 59, in _post_put_hook
    search.TextField(name='testId', value=unicode(self.testId))
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/api/search/search.py", line 1926, in __init__
    doc_id = _ConvertToUnicode(doc_id)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/api/search/search.py", line 247, in _ConvertToUnicode
    return unicode(some_string, 'utf-8')
TypeError: coercing to Unicode: need string or buffer, long found

Does the Webapp service handler code support Webapp2?

There's much confusion as to ProtoRPC and webapp2 support. Can you clarify if this does indeed support webapp2?

webapp2 framework used to provide support for ProtoRPC but that code was removed - supposedly it only worked with Python 2.5 and the origional webapp framework. What I need to do is host my remote.Service methods from an existing webapp2 application.

Storing protos in Google Cloud Datastore?

Does protorpc convert .proto files into Message instances that can be stored in Cloud Datastore? I don't understand why the Datastore client doesn't accept the standard Python codegen produced by the protoc compiler.

According to this StackOverflow answer, there is no current facility for storing arbitrary protocol buffers in Datastore, but it looks to me that this library may be capable of doing that.

Non-Unicode strings are silently ignored in StringFields

from protorpc import messages

class LogMessage(messages.Message):
    is_error = messages.BooleanField(1)
    # message value is silently ignored if it's a bytestring
    # make sure it is a unicode string!
    message = messages.StringField(2, required=True)

>>> LogMessage(is_error=False, message='fnord')
<LogMessage
 is_error: False>
>>> LogMessage(is_error=False, message=u'fnord')
<LogMessage
 is_error: False
 message: u'fnord'>
>>> LogMessage(is_error=False, message='fnord').check_initialized()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".tox/py27/lib/python2.7/site-packages/protorpc/messages.py", line 802, in check_initialized
    (type(self).__name__, name))
protorpc.messages.ValidationError: Message LogMessage is missing required field message
>>> LogMessage(is_error=False, message=u'fnord').check_initialized()
>>> LogMessage(is_error=False, message='fnord').is_initialized()
False
>>> LogMessage(is_error=False, message=u'fnord').is_initialized()
True

I have no problem with StringFields being required to be unicode strings. I do have a problem with non-unicode strings being silently ignored.

This seems to be a regression since 0.11.1; I do not experience this problem on 0.11.1 with #25 applied manually; I do experience it on master with #25 applied. (Needless to say, one cannot use this library without #25 applied anymore.)

Empty messages skip validation in protojson.decode_message

In protojson.decode_message, if encoded_message is an empty string, the validation will be skipped (and no ValidationError will be raised even if the message has some fields marked as required).

In https://github.com/google/protorpc/blob/master/protorpc/protojson.py#L201 , the following

if not encoded_message.strip():
  return message_type()

should be changed to

if not encoded_message.strip():
  message = message_type()
  message.check_initialized()
  return message

Bazel support?

I'm setting up a project using Google's Bazel as my build system. Many Google projects already support Bazel. It would be helpful if this repository (protorpc) exposed its tools in a way that can easily be consumed by other Bazel targets.

More context: I'd like a build rule that takes the .proto files as srcs and produces the resulting generated Python Message classes, so that I can use those in Google Cloud Datastore (the generated proto code produced by py_proto_library does not seem usable in Datastore models, since it does not subclass the Message class.)

Provide a way to explicitly set fields to null

Context: gsutil uses apitools to create messages and send them in HTTP requests to our JSON API. In the case of PATCH requests used to change object metadata (see [1]), one must set a field's value to null to clear it (as mentioned in the patch semantics at [2]). However, this isn't possible to do for predefined message fields. For example, an Object message, which describes a GCS object's metadata, has the field cacheControl:

class Object(_messages.Message):
    ...
    cacheControl = _messages.StringField(3)
    ...

but if we attempt to set that field to None, it's treated as if the field was unset:

(Pdb) obj_metadata = apitools_messages.Object()
(Pdb) obj_metadata
<Object
 acl: []>
(Pdb) obj_metadata.cacheControl = 'public'
(Pdb) obj_metadata
<Object
 acl: []
 cacheControl: 'public'>
(Pdb) obj_metadata.cacheControl = None
(Pdb) obj_metadata
<Object
 acl: []>

... and it won't be included in the JSONified representation of the message that's sent as part of the HTTP request to our API. In order to clear these fields' values via PATCH requests, we need a way to set them to some value such that they'll be represented as null when translated to JSON.

Note: This isn't the first time we've run into this limitation, and others have run into it as well, e.g. GoogleCloudPlatform/endpoints-proto-datastore#65 (comment).

[1] https://cloud.google.com/storage/docs/json_api/v1/objects/patch
[2] https://cloud.google.com/storage/docs/json_api/v1/how-tos/performance#patch

Please add per-file licenses

The Chromium project (www.chromium.org) pulls in protorpc indirectly through the catapult (https://github.com/catapult-project/catapult/) and gsutil (https://github.com/GoogleCloudPlatform/gsutil) source repositories. In order for Chromium to be pulled into various Linux source distributions there's a requirement that all of the third party files pass the Linux licensecheck utility. Currently there are many files in the protorpc repository missing per-file licenses. From a current run of licensecheck, they are:

$ licensecheck -r . | grep "No copyright"
./protorpc/google_imports.py: *No copyright* UNKNOWN
./protorpc/protorpc_test_pb2.py: *No copyright* GENERATED FILE
./protorpc/non_sdk_imports.py: *No copyright* UNKNOWN
./protorpc/webapp/google_imports.py: *No copyright* UNKNOWN
./ez_setup.py: *No copyright* UNKNOWN
./experimental/javascript/build.sh: *No copyright* UNKNOWN
./demos/quotas/backend/quotas/__init__.py: *No copyright* UNKNOWN
./demos/experimental/wsgi/services.py: *No copyright* UNKNOWN
./demos/tunes_db/server/appengine_config.py: *No copyright* UNKNOWN

We'd like to ask that per-file licenses be added to these files to make it easier to integrate protorpc not only into Chromium, but also Linux distributions in general. Thanks.

UnicodeEncodeError in logging statements

Hello,

I am using appengine and I noticed that some of my requests crashed without outputing any logs.

After investigating I found some logs that were not forwarded to the output :

File "/home/vmagent/python_vm_runtime/lib/protorpc-1.0/protorpc/wsgi/service.py", line 191, in protorpc_service_app
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 3604468: ordinal not in range(128)

It's happening here : https://github.com/google/protorpc/blob/master/protorpc/wsgi/service.py#L190

When the error raised is a unicode instead of an str it's crashing. You can reproduce the issue easily with this piece of code :

In  [1]: try:                                                      
   ....:         raise ValueError(u'sample unicode : \xa0')
   ....: except Exception as err:
   ....:         logging.exception('%s %s' % ('text', err))
   ....:     
---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-22-6518b912bd3d> in <module>()
      2         raise ValueError(u'sample unicode : \xa0')
      3 except Exception as err:
----> 4         logging.exception('%s %s' % ('text', err))
      5 

UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 17: ordinal not in range(128)

In order to fix this you just have to add a u before the string in the logging statement :

In  [2]: try:                                                      
        raise ValueError(u'sample unicode : \xa0')
except Exception as err:
        logging.exception(u'%s %s' % ('text', err))
   ....:     
ERROR:root:text sample unicode :  
Traceback (most recent call last):
  File "<ipython-input-23-fded89cbdcf2>", line 2, in <module>
    raise ValueError(u'sample unicode : \xa0')
ValueError: sample unicode : \xa0

Is that something that could be fixed ? Would you accept a PR with this fix ?

README.md is empty

Even a short description of what this is would be handy, I'm guessing RPC over protocol buffers ?

google / protorpc Goto Github PK

protorpc's Introduction

ProtoRPC

protorpc's People

Contributors

Stargazers

Watchers

Forkers

protorpc's Issues

Recommend Projects

Recommend Topics

Recommend Org