Comments (15)
Trying to understand the issue.
- You seem to be generating the descriptor in two ways: "by the protocol buffer compiler", and through "Scala plugin". Can you specify exactly how the descriptors are being generated in each of the two options?
-
currently I'm using AnnotationsProto.http.get(methDescriptor) from ScalaBP generated class for com.google.api.annotations.proto, which has a lens that access extension from unknownFields, and doesn't work for javadsl(bcs it's using pbc generated java class).
Like you indiciate, the ScalaPB extensions API works on ScalaPB descriptors/extensions. Is the expectation that it would work with Java descriptors?
Can you somehow make the problem statement more clear?
from scalapb.
Trying to understand the issue.
- You seem to be generating the descriptor in two ways: "by the protocol buffer compiler", and through "Scala plugin". Can you specify exactly how the descriptors are being generated in each of the two options?
currently I'm using AnnotationsProto.http.get(methDescriptor) from ScalaBP generated class for com.google.api.annotations.proto, which has a lens that access extension from unknownFields, and doesn't work for javadsl(bcs it's using pbc generated java class).
Like you indiciate, the ScalaPB extensions API works on ScalaPB descriptors/extensions. Is the expectation that it would work with Java descriptors?
Can you somehow make the problem statement more clear?
sry for being confusing.
for 1, to be precisely, "by the protocol buffer compiler" in which I mean sbtprotoc.ProtocPlugin.gens.java
that from sbt plugin protocbridge
, and "Scala plugin" in which I mean scalapb.gen.SandboxedGenerator
that using ScalaPbCodeGenerator
.
for 2, yes, I was expect ScalaPB generated descriptors' lens work with Java descriptors, it feels weird to me that ScalaPB generated descriptors' implementation for proto extensions differ from the Java one's.
from scalapb.
I am sorry I still don't follow. Can you provide a minimal project that demonstrate the problem and the expectation that is not met by ScalaPB? In any case, the implementation details do not have to match between ScalaPB and Java - the respective APIs for each should give the same results.
from scalapb.
I guess I can, but would you like to elaborate about which point confused you? I mean if it is like why setup a project like this anyway or what specific code structure I was made to make this situation?
Also you actually answered my problem, and I would like to argue a little bit more about it.
To be fair, I'm far from protobuf expert by any means, but from a user perspective, I see Descriptor as a data that represent the programmable structure of a proto, also the API javaDescriptor
returns a class that lays in com.google.protobuf
which makes it looks like the "common part" of all different protobuf implementations), it just feels wrong to have the same data class/type but works actually different.
And from implementation perspective, it looks reasonable to put extension into extensions
field instead of unknownFields
, also relatively easy to do it I guess?
from scalapb.
sry I was not make my point very clear, I've update the Problem section.
from scalapb.
I am still confused. The two screenshots you share show the Java protobufs, none of them is a ScalaPB object, but then you mention you try to access them through the ScalaPB extensions API which isn't supposed to work with the Java objects.
from scalapb.
I am still confused. The two screenshots you share show the Java protobufs, none of them is a ScalaPB object, but then you mention you try to access them through the ScalaPB extensions API which isn't supposed to work with the Java objects.
uh, I knew what I was missed now, I mean I did manually transform Java protobufs(com.google.protobuf.MethodOptions
) to SPB ones(com.google.protobuf.descriptor.MethodOptions
) by copying each fields, and then access the extension through generated lens, sry for cause confusion.
so just for sure, which I was misunderstand and you pointed is, ScalaPB extensions API(generated lens) isn't supposed to work with neither pbc generated Java classes or ScalaPB's javaDescriptor
. thanks for clearing that.
if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor
, if it is, as I pointed, I don't think it works bcs one just simply differ from other.
from scalapb.
And for clarify what I want to achieve, akka-grpc support both javadsl and scaladsl, and for some reason, javadsl and scaladsl use different code generator, when you try to implement functionality like HTTP transcoding which requires you to parse HttpRule from Descriptors, naturally you would to choose the common part of both generated classes and use Scala API for accessing, and it's javaDescriptor
from ScalaPB and descriptor
from Java ones.
Even they're the same type, but the implementation is actually different as I pointed, so after the same transformation procedure(to ScalaPB class), one works and one doesn't. that's why I think it is an issue.
from scalapb.
Also to be fair, I do heard that unknownSets
is the widely used approach, it maybe protobuf-java suddenly changes it's implementation at some point.
from scalapb.
if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.
The Java API's getExtension
should work with javaDescriptor
.
...one works and one doesn't. that's why I think it is an issue.
Can you provide a minimal project that demonstrates what doesn't work?
from scalapb.
if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.
The Java API's
getExtension
should work withjavaDescriptor
.
Bcs some weird issues of akka-grpc I couldn't get the Java API works(google common proto library's Java classes missing from the target), so I haven't test it yet, still working on that.
And if that's the case, sry for misunderstanding the API, I will try it.
from scalapb.
...one works and one doesn't. that's why I think it is an issue.
Can you provide a minimal project that demonstrates what doesn't work?
Also I just realize that since I'm only copy unknownSets
and there is no place for extensions
in ScalaPB's class, so use ScalaPB's lens is not gonna work anyway.
from scalapb.
if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.
The Java API's
getExtension
should work withjavaDescriptor
....one works and one doesn't. that's why I think it is an issue.
Can you provide a minimal project that demonstrates what doesn't work?
I just update the reproduce project to show it seems getExtension
doesn't work with javaDescriptor
from scalapb.
Thanks for providing the reproduction. It helps understanding the issue you are seeing better.
When ScalaPB generates the code for javaDescriptor
it has a limitation: it can't assume that there are Java classes for anything that gets imported by the proto file. In this case, the generator has no way of knowing that AnnotatationsProto
exists, so it can't reliably emit code like registry.add(com.google.api.AnnotationsProto.http);
There are two solutions you can have:
- Enable
javaConversions=true
generator option. This makesjavaDescriptor
to be a reference to the actual java descriptors. - You could use
javaDescriptor.toProto.toByteString
and re-parse it using an extension registry you manually build in your code.
The code in Options.scala
seems unnecessarily complex. What is it actually trying to do? Why doesn't it use the ScalaPB extensions API directly?
from scalapb.
Thanks for your patience and comprehensive answer.
I can see the limitation here now, since akka-grpc doesn't include java classes for scaladsl and vice verse, I guess reconstruct descriptor is the only option.
The code in Options.scala seems unnecessarily complex. What is it actually trying to do? Why doesn't it use the ScalaPB extensions API directly?
I was thinking that use javaDescriptor
would work, also wanna using Scala API for convenience, and parse from bytes seems expensive, so i've decide why not just manually copy every fields, after all it's just field mapping...
yeah, I guess it is a bad idea.
Anyway, I have no further question, pls feel free to close this issue.
from scalapb.
Related Issues (20)
- JSON serialization of Any message fails when the message has a special JSON representation HOT 2
- scalapb grpc runtime & InProcessTransport HOT 4
- Unrecognized enum serialized as int HOT 2
- Cannot name a 'oneof' field 'option' if an optional field exists HOT 1
- --jvm_0_out: protoc-gen-jvm_0: Plugin failed with status code 1. WIth Java version 11, change to 17 is fixed HOT 21
- Type Mismatch Error with `asRecognized` in versions 0.11.14 and 0.11.15 HOT 3
- sealed_oneof_companion_extends doesn't work for optional sealed oneof
- Sealed oneof "extend" for Empty case HOT 1
- java.lang.IllegalAccessError: tried to access method 'com.google.protobuf.LazyStringArrayList' HOT 10
- google/protobuf/empty.proto: File not found. HOT 1
- Define trait for GeneratedEnumCompanion’s fromJavaValue/toJavaValue HOT 1
- Support Scala 3.4 type wildcards HOT 2
- `_typemapper` are defined as package private, causing issues when deriving schemas HOT 4
- Protobuf with a field named using generates invalid Scala 3 code HOT 1
- ArrayIndexOutOfBoundsException on getFieldByNumber HOT 5
- UTF-8 Strings are unparseable? HOT 2
- JsonFormat.toJsonString omits authorisation string in the output JSON HOT 2
- Dependency on `scalapb.options.ScalapbProto` in generated code. HOT 3
- New release HOT 1
- Add an option to generate oneof fields as a sealed abstract class
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scalapb.