infinispan / protostream Goto Github PK

View Code? Open in Web Editor NEW

35.0 18.0 29.0 2.78 MB

ProtoStream is a serialization library based on Protocol Buffers

License: Apache License 2.0

Java 100.00%

protobuf java serialization-library infinispan

protostream's Issues

Use ByteBuddy instead of Javassist

https://issues.redhat.com/projects/IPROTO/issues/IPROTO-259

Standalone usage

Is it possible to use protostream standalone without an Infinispan cache? Just for serialization and deserialization from/to a byte array or ByteBuffer?

ObjectArrayAdapter generating marshaller in java.lang

I'm trying to create an adaptor for a dto which contains an object field, that's either an object of a registered marshaller, or an object array of objects of various registered marshallers [1].. I tried adding the ObjectArrayAdapter to my project but protostream-processor is generating an Object marshaller in java.lang inside the generated sources folder, am I using it incorrectly / bug?

[1]

public class Message
{
    private int messageType;
    private int messageId;

    private NodeAddress fromNode;
    private NodeAddress toNode;

    private Map<String, Object> headers;

    // target reference fields
    private int interfaceId;
    private Object objectId;
    private NodeAddress referenceAddress;

    // method fields
    private int methodId;
    private Object payload; // problematic field

    private long localRequestTime;

    private long remoteInvocationCreationTime;
    private long remoteInvocationCompletionTime;

Protobuf encoding marshling is not working when in a Java collection is used in the value part of the map

With the following cache definition e.g: Cache<String, ArrayList<Sensor>> , with Protobuf, the values are not being marshalled.

To recreated this issue, you can create a new project:

mvn archetype:generate -DarchetypeGroupId=org.infinispan.archetypes  -DarchetypeArtifactId=embedded -DarchetypeVersion=14.0.1.Final

added the latest lib in the pom.xml:

  ...
        <dependency>
            <groupId>org.infinispan.protostream</groupId>
            <artifactId>protostream-processor</artifactId>
            <version>14.0.0.CR2</version>
        </dependency>
  ...

Edited the upd xml file:

<infinispan
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:xi="http://www.w3.org/2001/XInclude"
        xsi:schemaLocation="urn:infinispan:config:14.0 http://infinispan.org/schemas/infinispan-config-14.0.xsd"
        xmlns="urn:infinispan:config:14.0">

    <cache-container default-cache="default" statistics="true">

        <serialization marshaller="org.infinispan.commons.marshall.ProtoStreamMarshaller">
            <context-initializer class="test.com.LibraryInitializerImpl"/>
            <allow-list>
                <class>test.com.Sensor</class>
            </allow-list>
        </serialization>

        <transport stack="udp" cluster="mycluster" />

        <distributed-cache name="stockTickers" mode="SYNC">
            <encoding>
                <key media-type="application/x-protostream"/>
                <value media-type="application/x-protostream"/>
            </encoding>
            <locking isolation="REPEATABLE_READ" striping="false"/>
        </distributed-cache>

        <replicated-cache name="default"/>

    </cache-container>
</infinispan>

A very simple POJO

public class Sensor {

	@ProtoField(value = 1, defaultValue = "0")
	public int anInt;

	public Sensor() {
	}

	public Sensor(int anInt) {
		this.anInt = anInt;
	}
}

This code:

   public void basicUse() {
      System.out.println("\n\n1.  Demonstrating basic usage of Infinispan.  This cache stores arbitrary Strings.");
      Cache<String, ArrayList<Sensor>> cache = cacheManager.getCache("stockTickers", true);

      cache.put("test", new ArrayList<>(List.of(new Sensor(1), new Sensor(2), new Sensor(3), new Sensor(4))));

      // this fails
      List<Sensor> test = cache.get("test");

   }

yields in: Exception in thread "main" org.infinispan.commons.dataconversion.EncodingException: ISPN000495: ProtostreamTranscoder encountered error transcoding content because in this code, since in the example the array contains 4elements (4tags and 4 bytes arrays), the field count gives 8, throwing the error. If you only add one into the list the code works:

   //this works
   cache.put("test", new ArrayList<>(List.of(new Sensor(1))));

I've tried simple types, like a list of integers Cache<String, ArrayList<Integer>> and it works because of the break in the switch

ProtoField for abstract class

I am trying to create a @ProtoAdapter for a third party class and I couldn't find examples for usage with inheritance.
Lets say there is a class called CachedAnimal that contains an Animal and CacheDuration.

@ProtoAdapter(CachedAnimal.class)
public class CachedAnimalAdapter {
  @ProtoFactory 
  CachedAnimal create(Animal animal, String cacheDuration) {
    return new CachedAnimal(animal, cacheDuration);
  }

  @ProtoField
  Animal getAnimal(CachedAnimal cachedAnimal) {
     cachedAnimal.getAnimal();
  }
  
  @ProtoField
  String getCacheDuration(CachedAnimal cachedAnimal) {
     cachedAnimal.getDuration();
  }
}

However, this fails to compile with @AutoProtoSchemaBuilder saying the @ProtoField type "should not be abstract".

Please provide details on how to deal with abstract types and add some examples to the documentation.

Support for generic marshallers

I would like a way to use generic marshallers without relying on deprecated methods, right now org.infinispan.protostream.impl.SerializationContextImpl looks up marshallers using a hashmap with the class as a key which doesn't find a compatible marshaller. I am currently working around this by creating a MarshallerProvider.. would you consider keeping MarshallerProvider in order to maintain this functionality?

 serializationContext.registerMarshallerProvider(new MarshallerProvider() {
            @Override
            public BaseMarshaller<?> getMarshaller(String typeName) {
                return null;
            }

            @Override
            public BaseMarshaller<?> getMarshaller(Class<?> javaClass) {
                if (AbstractActor.class.isAssignableFrom(javaClass)) {
                    return getContext().getMarshaller(AbstractActor.class);
                }
                if (ActorObserver.class.isAssignableFrom(javaClass)) {
                    return getContext().getMarshaller(ActorObserver.class);
                }
                if (RemoteReference.class.isAssignableFrom(javaClass)) {
                    return getContext().getMarshaller(RemoteReference.class);
                }
                return null;
            }
        });

Caused by: org.infinispan.protostream.AnnotationParserException: Error: 1,43: illegal escape character: u

Not able to use @protofield with kotlin.

Got this:

        at org.infinispan.protostream.impl.parser.AnnotationLexer.scanLiteralChar(AnnotationLexer.java:167)
        at org.infinispan.protostream.impl.parser.AnnotationLexer.nextToken(AnnotationLexer.java:388)
        at org.infinispan.protostream.impl.parser.AnnotationParser.expect(AnnotationParser.java:55)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseArray(AnnotationParser.java:247)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseValue(AnnotationParser.java:184)

Gradle kotlin dsl is

kapt("org.infinispan.protostream:protostream-processor:4.6.2.Final")
	annotationProcessor("org.infinispan.protostream:protostream-processor:4.6.2.Final")
	implementation("org.infinispan.protostream:protostream-processor:4.6.2.Final")

Context is

@AutoProtoSchemaBuilder(

    includeClasses = [ UserData2::class],
)
interface LibraryInit : SerializationContextInitializer

UserData2 class is

open class UserData2 {

    @get:ProtoField(number = 1)
    @Max(501)
    var description: String? = null
}

if I move UserData2 to pure java module and include into first module - everything works fine.

full stack trace

api/build/tmp/kapt3/stubs/main/com/proto/LibraryInit.java:5: error: org.infinispan.protostream.DescriptorParserException: org.infinispan.protostream.AnnotationParserException: Error: 1,43: illegal escape character: u
public abstract interface LibraryInit extends org.infinispan.protostream.SerializationContextInitializer {
                ^
        at org.infinispan.protostream.descriptors.FileDescriptor.resolveDependencies(FileDescriptor.java:266)
        at org.infinispan.protostream.descriptors.FileDescriptor.resolveDependencies(FileDescriptor.java:210)
        at org.infinispan.protostream.descriptors.ResolutionContext.resolve(ResolutionContext.java:57)
        at org.infinispan.protostream.impl.SerializationContextImpl.registerProtoFiles(SerializationContextImpl.java:118)
        at org.infinispan.protostream.annotations.impl.BaseProtoSchemaGenerator.generateAndRegister(BaseProtoSchemaGenerator.java:171)
        at org.infinispan.protostream.annotations.impl.processor.AutoProtoSchemaBuilderAnnotationProcessor.processClass(AutoProtoSchemaBuilderAnnotationProcessor.java:336)
        at org.infinispan.protostream.annotations.impl.processor.AutoProtoSchemaBuilderAnnotationProcessor.processElement(AutoProtoSchemaBuilderAnnotationProcessor.java:266)
        at org.infinispan.protostream.annotations.impl.processor.AutoProtoSchemaBuilderAnnotationProcessor.process(AutoProtoSchemaBuilderAnnotationProcessor.java:189)
        at org.jetbrains.kotlin.kapt3.base.incremental.IncrementalProcessor.process(incrementalProcessors.kt:90)
        at org.jetbrains.kotlin.kapt3.base.ProcessorWrapper.process(annotationProcessing.kt:197)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:1018)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:934)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1262)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1377)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1248)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at org.jetbrains.kotlin.kapt3.base.AnnotationProcessingKt.doAnnotationProcessing(annotationProcessing.kt:90)
        at org.jetbrains.kotlin.kapt3.base.AnnotationProcessingKt.doAnnotationProcessing$default(annotationProcessing.kt:31)
        at org.jetbrains.kotlin.kapt3.base.Kapt.kapt(Kapt.kt:47)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at org.jetbrains.kotlin.gradle.internal.KaptExecution.run(KaptWithoutKotlincTask.kt:311)
        at org.jetbrains.kotlin.gradle.internal.KaptWithoutKotlincTask$KaptExecutionWorkAction.execute(KaptWithoutKotlincTask.kt:257)
        at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:63)
        at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1.create(NoIsolationWorkerFactory.java:66)
        at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1.create(NoIsolationWorkerFactory.java:62)
        at org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:100)
        at org.gradle.workers.internal.NoIsolationWorkerFactory$1.lambda$execute$0(NoIsolationWorkerFactory.java:62)
        at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:44)
        at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:41)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
        at org.gradle.workers.internal.AbstractWorker.executeWrappedInBuildOperation(AbstractWorker.java:41)
        at org.gradle.workers.internal.NoIsolationWorkerFactory$1.execute(NoIsolationWorkerFactory.java:59)
        at org.gradle.workers.internal.DefaultWorkerExecutor.lambda$submitWork$2(DefaultWorkerExecutor.java:212)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runExecution(DefaultConditionalExecutionQueue.java:187)
        at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.access$700(DefaultConditionalExecutionQueue.java:120)
        at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner$1.run(DefaultConditionalExecutionQueue.java:162)
        at org.gradle.internal.Factories$1.create(Factories.java:31)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:249)
        at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:109)
        at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:114)
        at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runBatch(DefaultConditionalExecutionQueue.java:157)
        at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.run(DefaultConditionalExecutionQueue.java:126)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1623)
  Caused by: org.infinispan.protostream.AnnotationParserException: Error: 1,43: illegal escape character: u
        at org.infinispan.protostream.impl.parser.AnnotationLexer.scanLiteralChar(AnnotationLexer.java:167)
        at org.infinispan.protostream.impl.parser.AnnotationLexer.nextToken(AnnotationLexer.java:388)
        at org.infinispan.protostream.impl.parser.AnnotationParser.expect(AnnotationParser.java:55)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseArray(AnnotationParser.java:247)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseValue(AnnotationParser.java:184)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseAttribute(AnnotationParser.java:173)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseAttributes(AnnotationParser.java:152)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parseAnnotation(AnnotationParser.java:89)
        at org.infinispan.protostream.impl.parser.AnnotationParser.parse(AnnotationParser.java:43)
        at org.infinispan.protostream.impl.AnnotatedDescriptorImpl.processAnnotations(AnnotatedDescriptorImpl.java:83)
        at org.infinispan.protostream.impl.AnnotatedDescriptorImpl.getProcessedAnnotation(AnnotatedDescriptorImpl.java:309)
        at org.infinispan.protostream.descriptors.Descriptor.setFileDescriptor(Descriptor.java:124)
        at org.infinispan.protostream.descriptors.FileDescriptor.collectDescriptors(FileDescriptor.java:311)
        at org.infinispan.protostream.descriptors.FileDescriptor.resolveDependencies(FileDescriptor.java:245)
        ... 58 more

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':api:kaptKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.gradle.internal.KaptWithoutKotlincTask$KaptExecutionWorkAction
   > java.lang.reflect.InvocationTargetException (no error message)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 12s

Generated java class is:

package com.sashka.proto;

@kotlin.Metadata(mv = {1, 9, 0}, k = 1, xi = 48, d1 = {"\u0000\n\n\u0002\u0018\u0002\n\u0002\u0018\u0002\n\u0000\bg\u0018\u00002\u00020\u0001\u00a8\u0006\u0002"}, d2 = {"Lcom/sashka/proto/LibraryInit;", "Lorg/infinispan/protostream/SerializationContextInitializer;", "api"})
@org.infinispan.protostream.annotations.AutoProtoSchemaBuilder(includeClasses = {com.sashka.model.UserData2.class})
public abstract interface LibraryInit extends org.infinispan.protostream.SerializationContextInitializer {
}

ProtoReserved statements always commented in proto file

Annotating a class with @ProtoReserved(numbers = 3) results in the following:

message Foo {
   //reserved 3;
}

However the proto format for reserved statements is as follows:

message Foo {
  reserved 3;
}

java.lang.NoClassDefFoundError: ProtoSchema - v5.0.1.Final

Using the new 5.0.1.Final dependency creates the following error:
Execution failed for task ':compileJava'.

java.lang.NoClassDefFoundError: org/infinispan/protostream/annotations/ProtoSchema

This happens as soon as the annotation processor is added to the project. This also happens with all the other v5 versions. Any feedback on this ? Thank you

Compilation error "@ProtoAdapter can only be applied to classes" on enum

Hi, I wrote a ProtoAdapter for an Enum and get a compilation error:

[ERROR] /home/user/.../caching/src/main/java/com/example/ColorEnumAdapter.java:[7,8] @ProtoAdapter can only be applied to classes.

But documentation says

@ProtoAdapter is a marshalling adapter for a class or enum that you cannot annotate directly.

I also found an Unit Test in source code doing this. My example files are adopted from there:

ColorEnum.java

package com.example;

public enum ColorEnum {
    RED, GREEN, BLUE
}

ColorEnumAdapter.java

package com.example;

import org.infinispan.protostream.annotations.ProtoAdapter;
import org.infinispan.protostream.annotations.ProtoEnumValue;

@ProtoAdapter(ColorEnum.class)
public enum ColorEnumAdapter {

    @ProtoEnumValue(number = 0, name = "red")
    RED,

    @ProtoEnumValue(number = 1, name = "green")
    GREEN,

    @ProtoEnumValue(number = 2, name = "blue")
    BLUE
}

Can you help me to find out what I'm doing wrong?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.