Comments (8)
dstelljes
commented on July 23, 2024
For the first issue, could you provide the trace as well as the TestClassRoot class if you can?
For the second, it should work to create custom cases for T. See https://github.com/ch-robinson/dotnet-avro/blob/main/examples/Chr.Avro.UnionTypeExample/Program.cs for an example. If you want to use int as the dictionary key, you may also have to implement a custom case for int since Avro map keys are strings and Chr.Avro won’t convert int to string implicitly.
from dotnet-avro.
koncq
commented on July 23, 2024
First issue - No coercion operator is defined between types 'System.String' and 'System.Char[]'.
Trace:
Chr.Avro.UnsupportedTypeException: The RequestInfoContext member on DataContract.EngineRequest could not be mapped to the RequestInfoContext field on DataContract.EngineRequest.
---> Chr.Avro.UnsupportedTypeException: The Parameter member on DataContract.Data.InfoContext could not be mapped to the Parameter field on DataContract.Data.InfoContext.
---> Chr.Avro.UnsupportedTypeException: Failed to map StringSchema to System.Char[].
---> System.InvalidOperationException: No coercion operator is defined between types 'System.String' and 'System.Char[]'.
at System.Linq.Expressions.Expression.GetUserDefinedCoercionOrThrow(ExpressionType coercionType, Expression expression, Type convertToType)
at Chr.Avro.Serialization.ExpressionBuilder.BuildStaticConversion(Expression value, Type target)
at Chr.Avro.Serialization.StringDeserializerBuilderCase.BuildStaticConversion(Expression value, Type target)
at Chr.Avro.Serialization.ExpressionBuilder.BuildConversion(Expression value, Type target)
at Chr.Avro.Serialization.BinaryStringDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
--- End of inner exception stack trace ---
at Chr.Avro.Serialization.BinaryStringDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_1.<BuildExpression>b__0(RecordField field)
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_1.<BuildExpression>b__0(RecordField field)
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_3.<BuildExpression>b__7(RecordField field)
--- End of inner exception stack trace ---
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_3.<BuildExpression>b__7(RecordField field)
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
at System.Linq.Enumerable.ConcatNIterator`1.LazyToArray()
at System.Dynamic.Utils.CollectionExtensions.ToReadOnly[T](IEnumerable`1 enumerable)
at System.Linq.Expressions.Expression.Block(IEnumerable`1 variables, IEnumerable`1 expressions)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_3.<BuildExpression>b__7(RecordField field)
--- End of inner exception stack trace ---
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.<>c__DisplayClass7_3.<BuildExpression>b__7(RecordField field)
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
at System.Linq.Enumerable.ConcatNIterator`1.LazyToArray()
at System.Dynamic.Utils.CollectionExtensions.ToReadOnly[T](IEnumerable`1 enumerable)
at System.Linq.Expressions.Expression.Block(IEnumerable`1 variables, IEnumerable`1 expressions)
at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildDelegateExpression[T](Schema schema, BinaryDeserializerBuilderContext context)
at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildDelegate[T](Schema schema, BinaryDeserializerBuilderContext context)
at KafkaMessageConsumer.Deserialize(Byte[] data, Schema schema) in C:\Repos\AvroGen\KafkaMessageConsumer.cs:line 29
at KafkaMessageConsumer.Consume(ConsumeContext`1 context) in C:\Repos\AvroGen\KafkaMessageConsumer.cs:line 20
Simplified class, without multiple, additional properties, stripped of ProtoMemberAttributes:
[DataContract]
public sealed class Request
{
[DataMember] public bool Debug { get; set; }
[DataMember] public int NetworkId { get; set; } = -1;
[DataMember] public int SiteId { get; set; } = -1;
[DataMember] public int PageId { get; set; }
[DataMember] public IEnumerable<Size> RequestedSizes { get; set; }
[DataMember] public string RequestUrl { get; set; }
[DataMember] public string PageUrl { get; set; }
[DataMember] public string PageDomain { get; set; }
[DataMember] public string Keywords { get; set; }
[DataMember] public InfoContext RequestInfoContext { get; set; }
[DataMember] public bool HasIfa { get; set; }
[DataMember] public bool HasIfv { get; set; }
[DataMember] public string ImpressionUrl { get; set; }
[DataMember] public string MraidOnePxImpressionUrl { get; set; }
[DataMember] public string InventoryPartnerDomain { get; set; }
[DataMember] public string UniqueId { get; set; }
}
[DataContract]
public class InfoContext
{
[DataMember] public TestClass Parameter { get; set; }
[DataMember] public int MaxDuration { get; set; }
[DataMember] public int Size { get; set; }
}
[DataContract]
public class TestClass
{
[DataMember] public string Platform { get; private set; }
[DataMember] public int CurrentSpot { get; private set; }
[DataMember] public string AdvertisingId { get; private set; }
[DataMember] public string IdType { get; private set; }
[DataMember] public bool IsLat { get; private set; }
[DataMember] public string TransactionId { get; private set; }
[DataMember] public TvDaiFeature DaiFeature { get; private set; }
}
Eveery other property gets mapped, except for that IdType in TestClass.
Second issue
I would like to use int as the key, I did not write it clear enough, my bad. Value Types are easy, it is basically handled automatically by SchemaBuilder. So what would I have to do to have int keys?
from dotnet-avro.
dstelljes
commented on July 23, 2024
It looks like private set on those properties might be the issue here. To get around that, try adding BindingFlags.NonPublic to memberVisibility.
Custom serde cases would be the way to handle value type conversions as well: implement a case that matches the "int" schema and the string type.
from dotnet-avro.
koncq
commented on July 23, 2024
Well, cannot really go with BindingFlags.NonPublic, but when I set a public setter, it did not really help...
from dotnet-avro.
dstelljes
commented on July 23, 2024
Hmm, I'm unable to reproduce with the following code on .NET 7:
var schemaBuilder = new SchemaBuilder(
memberVisibility: BindingFlags.Public | BindingFlags.Instance,
enumBehavior: EnumBehavior.Symbolic,
nullableReferenceTypeBehavior: NullableReferenceTypeBehavior.Annotated,
temporalBehavior: TemporalBehavior.Iso8601);
var schemaBuilderContext = new SchemaBuilderContext();
var schema = schemaBuilder.BuildSchema<Request>(schemaBuilderContext);
var deserializer = new BinaryDeserializerBuilder(
BinaryDeserializerBuilder
.CreateDefaultCaseBuilders()
)
.BuildDelegate<Request>(schema);Which version of .NET and which version of Chr.Avro are you using?
from dotnet-avro.
koncq
commented on July 23, 2024
.net 7 and 9.4.1 - latest. i am unable to provide the whole project with repro today. I assume it won't be so easy to reproduce it without the whole context...
from dotnet-avro.
koncq
commented on July 23, 2024
I have sent you link to the repo with reproduced issue.
EDIT: Could you explain what did you mean by?
Custom serde cases would be the way to handle value type conversions as well: implement a case that matches the "int" schema and the string type
from dotnet-avro.
dstelljes
commented on July 23, 2024
When I run the reproduction locally, this is the trace I see:
Creating random SSPEngineRequest
Serializing request...
Deserializing request...
Unhandled exception. Chr.Avro.UnsupportedTypeException: The RequestContext member on AvroRepro.Request could not be mapped to the RequestContext field on AvroRepro.Request.
---> Chr.Avro.UnsupportedTypeException: The Parameter member on AvroRepro.Context could not be mapped to the Parameter field on AvroRepro.Context.
---> System.ArgumentException: Expression of type 'System.Object' cannot be used for constructor parameter of type 'System.String' (Parameter 'arguments[14]')
at System.Dynamic.Utils.ExpressionUtils.ValidateOneArgument(MethodBase method, ExpressionType nodeKind, Expression arguments, ParameterInfo pi, String methodParamName, String argumentParamName, Int32 index)
If I remove the spotId parameter on the Parameter constructor, the program completes successfully. There’s no corresponding SpotId property on the Parameter class.
Custom serde cases would be the way to handle value type conversions as well: implement a case that matches the "int" schema and the string type
I’ll try to be a little more detailed. The easiest path would probably be to create a class that extends BinaryStringSerializerBuilderCase and override BuildStaticConversion to support conversion from int.
from dotnet-avro.
Related Issues (20)
- Unable to generate a schema with default values HOT 2
- Default values support in codegen HOT 5
- `[EnumMember]` value should be used when serializing enum types to `"string"`
- Add support for NodaTime HOT 1
- Could not load file or assembly System.ServiceModel.Primitives.dll HOT 1
- Method not found exception HOT 1
- Union with null and multiple selected types throws exception when serializing null. HOT 1
- Exception serializing an anonymous object HOT 3
- Exception when building a serializer for type with a missing field from the schema HOT 2
- Support C# records in codegen HOT 1
- Support DateOnly and TimeOnly types
- JSON test failures under NetCore 3.1
- `ArraySegment<T>` deserialization fails on .NET Framework
- Support for Nullable Uri HOT 1
- Support `IgnoreDataMember` attribute
- Support unsigned number as logical types in codegen from Avro HOT 9
- Change to ConstructValueSubjectName HOT 1
- System.Datetime is not handled consistently HOT 1
- Deserialize string to DateOnly fails on net6 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dotnet-avro.