NonSerialized doesn't work with getter setter variable so It would be nice to have a attribute for message pack to do the same job.

This is an extremely strange issue however I've tested it several times and it seems to be consistent at least.

Take the following classes:

public class MyClass {
    public int Id { get; set; }
    public string Name { get; set; }
    public Sub MySub { get; set; }

    public MyClass()
    {
    }
}

public class Sub
{
    public string Id { get; set; }
    public string DId { get; set; }
    public string Something { get; set; }

    public Sub(){}
}

The following will fail with an exception (can be different exceptions)

var myClass = new MyClass() {
    Id= 100,
    MySub=new Sub() { Id= "SubId" },
    Name="Testclass"
};

var serializer = MsgPack.Serialization.MessagePackSerializer.Create<MyClass>();
var data = serializer.PackSingleObject(myClass);
var result = serializer.UnpackSingleObject(data);

However comment out the property "DId" from class Sub or simply rename it to for example "SId" and it will work, "FId" will for example not work either. I'm running 0.3.2 for aspnet 4.0.

I have yet to find a way to serialize anything other than trivial object graphs from msgpack-cli and have them successfully deserialize using msgpack-java. I have tried all combinations of msgpack-cli 0.4.1 and 0.5.5 and msgpack-java 0.6.8 and 0.6.11. Obviously, it doesn't work in the other direction either, from java to csharp.

Here are two classes that successfully interop (note that the Submission.queueName field is commented out:

public class SubmitterId
{
    public String machineName;
    public String submitterName;
    public String submitterVersion;
}

public class Submission
{
    public SubmitterId submitterId;
    //public String queueName;
}

Here are the equivalent classes in java:

import org.msgpack.annotation.Message;

@Message
public class SubmitterId
{
    public String machineName;
    public String submitterName;
    public String submitterVersion;
}

@Message
public class Submission
{
    public SubmitterId  submitterId;
    //public String       queueName;
}

Here is the csharp serialization code I used to serialize:

        var msgPack = MessagePackSerializer.Create<Submission>();
        using ( var stream = new FileStream(@"c:\submission-csharp.dat", FileMode.Create) )
        {
            var sid = new SubmitterId();
            sid.machineName = System.Environment.MachineName;
            sid.submitterName = "SubmitterTests";
            sid.submitterVersion = "1.0";

            var s = new Submission();
            s.submitterId = sid;
            //s.queueName = "test-queue";

            msgPack.Pack( stream, submission );
        }

Here is the java code to deserialize:

    MessagePack mp = new MessagePack();

    InputStream file = new FileInputStream("c:\\submission-csharp.dat");
    Submission s = mp.read( file, Submission.class );
    assertEquals("MACHINE_NAME", s.submitterId.machineName);
    assertEquals("SubmitterTests", s.submitterId.submitterName);
    assertEquals("1.0", s.submitterId.submitterVersion);
    //assertEquals("test-queue", s.queueName);

Whit the above version of these classes, the serialization and deserialization complete without errors, and all the expected fields have all the expected values.

However, the deserialization in java fails as soon as I remove the comments from the "queueName" field in the classes and tests. As soon as I do that, I get the following exception when deserializing using msgpack-java:

org.msgpack.MessageTypeException: Expected array, but got raw value
    at org.msgpack.unpacker.Accept.acceptRaw(Accept.java:73)
    at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStack(MessagePackUnpacker.java:112)
    at org.msgpack.unpacker.MessagePackUnpacker.readOne(MessagePackUnpacker.java:73)
    at org.msgpack.unpacker.MessagePackUnpacker.readArrayBegin(MessagePackUnpacker.java:478)
    at com.messages.msgpack.SubmitterId_$$_Template_23910357_0.read(SubmitterId_$$_Template_23910357_0.java)
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
    at com.messages.msgpack.Submission_$$_Template_23910357_1.read(Submission_$$_Template_23910357_1.java)
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
    at org.msgpack.MessagePack.read(MessagePack.java:527)
    at org.msgpack.MessagePack.read(MessagePack.java:511)
    at com.msgpack.test.SubmissionTests.submissionDeserializesFromCSharpFile(SubmissionTests.java:26)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

For what it's worth, here is the error I get when trying to deserialize this same object graph in msgpack-cli after serialization from java:

Result Message: 
System.Runtime.Serialization.SerializationException : Cannot deserialize member 'queueName' of type 'Messages.Msgpack.Submission'.
  ----> MsgPack.MessageTypeException : Cannot convert 'System.String' type value from type 'FixedArray'(0x93).
Result StackTrace:  
at MsgPack.Serialization.EmittingSerializers.Generated.Messages_Msgpack_SubmissionSerializer0.UnpackFromCore(Unpacker )
at MsgPack.Serialization.MessagePackSerializer`1.UnpackFrom(Unpacker unpacker)
at MsgPack.Serialization.AutoMessagePackSerializer`1.UnpackFromCore(Unpacker unpacker)
at MsgPack.Serialization.MessagePackSerializer`1.UnpackFrom(Unpacker unpacker)
at MsgPack.Serialization.MessagePackSerializer`1.Unpack(Stream stream)
at JPTest.SubmitterTests.readSerializedSubmissionFromFile() in c:\MessagePackCli\Test\SubmitterTests.cs:line 98
--MessageTypeException
at MsgPack.ItemsUnpacker.ReadString(String& result)
at MsgPack.Serialization.EmittingSerializers.Generated.Messages_Msgpack_SubmissionSerializer0.UnpackFromCore(Unpacker )

Are these language-specific implementations of MessagePack tested against each other for interoperability? Should this simple case be working? Am I using the libraries correctly?

I am developing game for IOS, so I used mpu.exe to generate pre-serializer.
Usually, I used mpu like this(on mac):

mono mpu.exe -s ClassA.cs

For example, to ClassA like below:

public class ClassA {
        public int a = 0;

        public class ClassB{
                //...ryaku
        }
        //...
}

it will generate only one result file. (ClassASerializer.cs).
Sometimes I wanna generate ClassBSerializer.cs too(For inner class).
I type

mono mpu.exe -?

and it seems there is not related options.

Is there any option for mpu.exe to generate ClassBSerializer.cs (classB is inner class in classA) ?

Any help would be appreciated.
Thanks
shuidong

It would be great it there was an opportunity to (de)serialize enumerations as integers, not as strings. Currently it is not possible (unless I am missing something) and makes some problems.

Hi, Thanks for the great work on msgpack-cli!

I'm having an issue in the following scenario: I want to implement a custom serializer for a type that I don't own. (it belongs to a dll that I can't modify)

The type implements the IEnumerable interface:

public class Person : IEnumerable<Person>
{
    public string Name { get; set; }

    private IEnumerable<Person> Children { get; set; }

    public IEnumerator<Person> GetEnumerator()
    {
        return Children.GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

I've implemented a custom serializer and registered it:

SerializationContext serializationContext = 
    new SerializationContext();

serializationContext.Serializers.Register(new PersonSerializer());

When I try serialize an instance of the Person class, it works.

But if I try to serialize an array of objects which one of them is an instance of the Person class, I get a NullReferenceException:

object[] array = new object[] {new Person {Name = "Joe"}, 3};

MessagePackSerializer<object[]> context =
    serializationContext.GetSerializer<object[]>();

byte[] packed = context.PackSingleObject(array); // throws NullReferenceException

Thanks for your help,
Elad

I am using MessagePack for a library that sends messages from a client to a server via a message queue. The objects I am sending are a Dictionnary<string, List>. The speed was really lacking, though.

Doing some profiling, I found out that the execution profile on the client is dominated (> 80%) by the PackerUnpackerExtensions' packObjectCore method call. Actually, what's dominating the profile is the call to compile() (which is done each time).

Is it possible to speed things up there? Any reason why the right methods are not called directly (instead of compiling code dynamically that does the call)?

Hi @yfakariya

I'm newbie to msgpack-cli :)

I just tested the lastest version, 0.5.3 build on iOS.
My test class includes a field of enum.
MsgPack can't pack the class, but it can without the enum property.

Here is my test code.

public enum TEST_ENUM {
   NONE = 0
 }

public class Test {
  public int value = 0;
  public TEST_ENUM test_num = TEST_ENUM.NONE;
}

//pack
Test test = new Test();
byte[] byteArray;
using (MemoryStream ms = new MemoryStream ()) {
     var se = SerializationContext.Default.GetSerializer<Test> ();
     se.Pack (ms, test);
     byteArray = ms.ToArray ();
}

And I got an exception below.

ExecutionEngineException: Attempting to JIT compile method 
'MsgPack.Serialization.MessagePackSerializer:CreateReflectionInternal<TEST_ENUM> 
(MsgPack.Serialization.SerializationContext)'
while running with --aot-only.

on Android and on windows mono, it can pack well.

I think this is only iOS bug.
Thanks.

I really love your work, and I love messagepack. I'm coding C# plugins in NinjaTrader 7, which only supports .NET 3.5.

How difficult would it be to build a version of messagepack I can use? There's really no other comparable library.

Thanks.

I'm trying to use this and have no clue what is the good practice. Especially I'm wondering about PackToCore and UnpackFromCore pair. Should I use Read* functions inside UnpackFromCore? It is too painful to mine knowledge from unittests.

Thanks for this really nice implementation of MessagePack!

We use a lot of immutable objects in our code, and these are currently not very compatible with this implementation. It would be nice if the serializer could detect the possibility of deserializing an object via the constructor if there are no settable properties. Current, this implementation throws the following exception in this scenario:

Cannot serialize type 'ConstructorInjectionDemo' because it does not have any serializable fields nor properties

I've written a sample test that demonstrates the feature I'm proposing here:

using System.IO;
using MsgPack.Serialization;
using NUnit.Framework;

public class ConstructorInjectionDemo
{
    public string Value { get; private set; }

    public ConstructorInjectionDemo(string value)
    {
        Value = value;
    }
}

[TestFixture]
public class MessagePackConstructorInjectionTests
{
    [Test]
    public void Given_a_class_without_public_setters_When_MessagePack_deserializes_the_object_Then_the_constructor_should_be_used()
    {
        var serializer = MessagePackSerializer.Get<ConstructorInjectionDemo>();
        using(var stream = new MemoryStream())
        {
            serializer.Pack(stream, new ConstructorInjectionDemo("Hello!"));
            stream.Position = 0;
            var deserialisedObject = serializer.Unpack(stream);
            Assert.That(deserialisedObject.Value, Is.EqualTo("Hello!"));
        }
    }
}

Is there any appetite to implement this feature?

Thanks,
Bart

Hi @yfakariya
From my understanding, MsgPack 5 supports distinction between string values to binary values.

How can I determinate what have I read from the stream?

I tried this code

MemoryStream stream = new MemoryStream();
Packer packer = Packer.Create(stream);
packer.PackBinary(new byte[16]);
stream.Position = 0;

Unpacker unpacker = Unpacker.Create(stream);
unpacker.Read();
Type type = unpacker.LastReadData.UnderlyingType; // string?

But it tells me a string was written.
Thanks

I packed a dictionary, which contains another dictionary inside of it.
When I unpacked that again, instead of the inner dictionary, I found an integer. (count of the keys of the inner dictionary).

Following code will produce the problem.

I used the latest code of master. With VS 2010, and .Net Framework 3.5.

Dictionary<string, object> data = new Dictionary<string,object>();

data["name"] = "test";
data["age"] = 20;

Dictionary<string, object> innerDict = new Dictionary<string, object>();
innerDict["x"] = 13.5;

data["inner-dict"] = innerDict;

SerializationContext context = new SerializationContext();
context.CompatibilityOptions.PackerCompatibilityOptions = MsgPack.PackerCompatibilityOptions.None;
MessagePackSerializer<object> serializer = MessagePackSerializer.Get<object>(context);

Stream packed = new MemoryStream();
serializer.Pack(packed, data);

packed.Seek(0, SeekOrigin.Begin);

object unpacked = serializer.Unpack(packed);
// unpacked: { "name" : "test", "age" : 20, "inner-dict" : 1 }

There is a bug where DateTime max value in UTC does not roundtrip through .Pack and .Unpack

DateTime epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
DateTime date = new DateTime(3155378975999999999L, DateTimeKind.Utc);

var serializer = MessagePackSerializer.Get<DateTime>();

using (MemoryStream ms = new MemoryStream())
{
    serializer.Pack(ms, date);

    var test = ms.ToArray();

    ms.Position = 0;

    long diff = (long)date.Subtract(epoch).TotalMilliseconds;
    //=253402300800000 which cannot be added back to epoch successfully 
    //*ERROR*: epoch.AddMilliseconds(diff);

    //*ERROR*: The unpacked value is not expected type. The added or subtracted value results in an un-representable DateTime.
    DateTime result = serializer.Unpack(ms);
}

I have encountered an issue passing a set of integers between C++ and C#.

To recreate, create two projects, one C++ and one C#, which transfer a set of integers (vector to list). Example code is below.

When running this, the List() in C# will contain the wrong values (mine were [10, 13, 10, 13, ... ]); if any other members are present MessagePack will throw an exception as it fails to deserialise them. This, along with seeing the 'wrong' data in the value fields (for other members) when looking at objects unpacked manually on the managed side with Unpacker.ReadItem() suggests that MessagePack loses its place.

This only occurs with vectors of integers, not vectors of floats or other arbitrary/custom data types, which are serialised/deserialised correctly.

I do not know enough about the code to say what is causing the issue, but two places I thought were possibilities stepping through with the debugger, is the serialisation stage, since it appeared when the array was being updated each integer was being passed with a length of 1.
It may also be in the deserialiser as when looking at the object using Unpacker manually, each entry in the created array was of type Byte, not Int.

//C++
class myClass
{
public:
    vector<int> myInts;
    MSGPACK_DEFINE(myInts);//,r,r2,cls,myString);
};

int _tmain(int argc, _TCHAR* argv[])
{
    myClass c;

    for(int i = 0; i < 10; i++)
    {
        c.myInts.push_back(10);
    }

    msgpack::sbuffer sbuf;
    msgpack::pack(sbuf, c);
    ofstream myFile;
    myFile.open("E:\\test.test");
    myFile.write(sbuf.data(), sbuf.size());
    myFile.close();

    return 0;   
}

//C#
namespace MessagePackTest_Managed
{
    public class myClass
    {
        [MessagePackMember(0)]
        public List<float> myInts;
    }

    class Program
    {
        static void Main(string[] args)
        {
            FileStream fs = new FileStream("E:\\test.test", FileMode.Open, FileAccess.Read);
            MessagePackSerializer<myClass> c = MessagePackSerializer.Create<myClass>();
            myClass v = c.Unpack(fs);
        }
    }
}

Awesome project. But I met some problems in IOS.

Version:

• MsgPack.dll=0.5.6
• OS=MAC 10.10
• Unity=4.5.3

Here is my data struct class.

public class MyData
{
    public string channel;
    public int version;
    public List<int> a = new List<int>();
        //ryaku
}

I used mpu.exe generate pre-serializer by this command:

mono mpu.exe -s ${path_of_MyData_cs} 

then use it to pack my data, below is the code:

MemoryStream dest = new MemoryStream(1024);

var context = new SerializationContext();
context.Serializers.RegisterOverride(new MsgPack.Serialization.GeneratedSerializers.MyDataSerializer(context));

var serializer = MessagePackSerializer.Get<MyData>(context);
serializer.Pack(dest, MyDataProvider.getMyData());//MyDataProvider.getMyData() returns a MyData instance

And when I start my Unity3D game in IOS, it throws exception like this:

App Start Exception: Exception has been thrown by the target of an invocation.   at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
  at System.Reflection.MonoCMethod.Invoke (BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
  at System.Reflection.ConstructorInfo.Invoke (System.Object[] parameters) [0x00000] in <filename unknown>:0 
  at System.Activator.CreateInstance (System.Type type, Boolean nonPublic) [0x00000] in <filename unknown>:0 
  at System.Activator.CreateInstance (System.Type type) [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.DefaultSerializers.GenericSerializer.CreateListSerializer (MsgPack.Serialization.SerializationContext context, System.Type itemType) [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.DefaultSerializers.GenericSerializer.Create (MsgPack.Serialization.SerializationContext context, System.Type targetType) [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.DefaultSerializers.GenericSerializer.Create[List`1] (MsgPack.Serialization.SerializationContext context) [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.SerializationContext.GetSerializer[List`1] (System.Object providerParameter) [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.SerializationContext.GetSerializer[List`1] () [0x00000] in <filename unknown>:0 
  at MsgPack.Serialization.GeneratedSerializers.PBCommonRequestSerializer..ctor (MsgPack.Serialization.SerializationContext context) [0x00000] in <filename unknown>:0 
  at HttpMgr.SendPost (System.Collections.Generic.List`1 sendQ) [0x00000] in <filename unknown>:0 

I have no idea why it happened, it seems that List type doesnt support on IOS?
When I comment out this line, and it works fine.

public List<int> a = new List<int>();

Any help would be appreciated.
Thank you.
shuidong

Hi, I am completely new on this. I am trying to deserialize an array of bytes that contain a class composed of two constructors an integer (size) and a List (results).

public class KnnQueryResponse
{
int size;
List < ImageResult > resutls;
}
public class ImageResult
{
string productURL;
float price;
string description;
byte[] thumbnail;
}

I am programming in c# under visual studio 2013 for a window 8 mobile app.
I have already added the MsgPack and MsgPack.Serialization packages into my project.
I just don't know and don't really understand how to use the msgpack api ;(
I came across a code that allow me to deserialise a simple integer.
Here is my method:

public int deserializeInteger(byte[] bytes)
{
var serializer = MessagePackSerializer.Create< int >();
using (var byteStream = new MemoryStream(bytes))
{
return serializer.UnpackSingleObject(bytes);
}
}

My question is: can someone tell me how to deserialise the class KNNQueryResponse?
If this is not clear enough or just need more information about what I am trying to do, then just let me know :).

Thanks in advance for help.

Trying to serialise/deserialise this class, messagePack fails because an exception is thrown because no setter is available.

I've noticed that while parsing members, it will take all properties or variables but doesn't filter out properties that are readonly or writeonly.

Class TestClass
Public item As String = ""
Public ReadOnly Property readonlyProperty As String
Get
Return item
End Get
End Property

Private _getSetproperty As String = ""
Public Property getSetPproperty() As String
    Get
        Return _getSetproperty
    End Get
    Set(ByVal value As String)
        _getSetproperty = value
    End Set
End Property

End Class

In a fresh universal app targeting windows phone 8,1, I get the following error:

Installing 'MsgPack.Cli 0.5.1'.
Successfully installed 'MsgPack.Cli 0.5.1'.
Adding 'MsgPack.Cli 0.5.1' to <my project>
Uninstalling 'MsgPack.Cli 0.5.1'.
Successfully uninstalled 'MsgPack.Cli 0.5.1'.
Install failed. Rolling back...
Could not install package 'MsgPack.Cli 0.5.1'. You are trying to install this package into a project that targets 'WindowsPhoneApp,Version=v8.1', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.

hello , great library!
i'll use an example to describe my problem: i'm trying to unpack object Foo to object Bar. with the Map option of the SerializerContext - SerializationMethod but an exception is thrown:

Foo {
prop string X,Y,
prop int A
}

Bar { prop X,Y,Z }

is there a way to get X and Y values in the new object and leave Z null or default (and ignore A completly)?

thank you

actual: The following code fails w/ MsgPack.Cli-0.4.4:

public class Data
{
    public int X { get; set; }
    public int X2 { get { return this.X * 2; } }
}

static void Test()
{
    var serializer = MessagePackSerializer.Create<Data>();
    // ^ SerializationException: Cannot set value to 'Foo.X2' property
    var dat = serializer.PackSingleObject(new Data() { X = 123, });
    serializer.UnpackSingleObject(dat);
}

expect: except properties without public setter (X2 in the above code) from the serialization target.

According to the document, the Data class should be serialized that has only one member for the property X.

all read/write PUBLIC members (that is, fields which are not initonly and properties which have both of getter and etter) are serialized.

            using (var fs = File.OpenRead(path))
            {
                var serializer = MessagePackSerializer.Create<T>();
                result = serializer.Unpack(fs);//MessagePack serializer for the type 'System.Collections.Generic.Dictionary`2[System.String,IISLog.LogEntity]' is not constructed yet.
            }

1. Single Thread Invoke is ok,then Parallel Invoke is ok.
2. but when Parallel Invoke first, then ex:

MessagePack serializer for the type 'System.Collections.Generic.Dictionary`2[System.String,IISLog.LogEntity]' is not constructed yet.

Hey all! Wanna use MsgPack in my AppProject. I have to handle very differtent packages, so i wanted to code a deserializer, that unpacks the msgPack to a c# Dictionary. If i have the dictionary i will check out to which function of my Server the package is adressed to (e.g. Login, general communication, ...).

My actual code looks like this:

public void Deserialize(byte[] rawMessage) {

        Dictionary<MessagePackObject, MessagePackObject> mpoDict = MessagePackSerializer.Create<Dictionary<MessagePackObject, MessagePackObject>>().Unpack(new MemoryStream(rawMessage));
        Dictionary<String, MessagePackObject> dict = mpoDict.ToDictionary(kv => (String)kv.Key, kv => kv.Value);
        MessagePackObject theValue;

        if (dict.TryGetValue("error", out theValue))
        {
            var error = (String)theValue;
        }
        else if (dict.TryGetValue("engine", out theValue))
        {
            var engine = (String)theValue;
            if(engine.Equals("@LOGIN")){Server.Client.Handler.LoginHandler(dict)}
            else if(engine.Equals("@REGISTER")){Server.Client.Handler.LoginHandler(dict)}
            else if(engine.Equals("@MESSAGE")){Server.Client.Handler.LoginHandler(dict)}
            else if(engine.Equals("@LOGOUT")){Server.Client.Handler.LoginHandler(dict)}
            else{Server.Clinet.Handler.ErrorHandler.throwError("Unrecognized Engine")}
        }
    }

some part of this seems to be not working, i get always errorstuff. Can someone help?

Hello,

I am transmitting a dict over MessagePack from Python to C#, which either has a key with name "results", pointing to a list of doubles, or a key "error", pointing to a string. I thought the "dynamic" API of the C# MessagePack library would allow me to programmatically unpack this, but after a fair bit of time trying different things and reading some of the source code of the library, I gave up.

I got things working by creating a helper class with the right member variables and passing that to Unpack(). But I think the documentation in https://github.com/msgpack/msgpack-cli/wiki/Messagepackobject could be improved, as I really wasn't able to figure out the "dynamic" way of doing what I wanted from said documentation. For one, adding a few examples of unpacking dynamic, multi-level data using the MessagePackObject, etc. API would be really helpful. Thanks!

I am trying to get the new enum serialization to interop with a java enum class where all of the fields of the same name are mapped to the same values in both classes.

Here are the csharp classes:

[MessagePackEnum(SerializationMethod=EnumSerializationMethod.ByUnderlyingValue)]
public enum MessageId
{
    FirstMessage = 0,
    SecondMessage = 1
};

public class MessageDescriptor
{
    public byte version = 1;
    public MessageId messageId;

    public MessageDescriptor( MessageId theId )
    {
        messageId = theId;
    }

    public MessageDescriptor() { }
}

...and here are the corresponding java classes:

@MessagePackOrdinalEnum
public enum MessageId
{
     FirstMessage(0),
     SecondMessage(1);

     public final int Value;

     private MessageId(int value)
     {
          Value = value;
     }

     private static final Map<Integer, MessageId> _map = new HashMap<Integer, MessageId>();
    static
    {
        for (MessageId id : MessageId.values())
            _map.put(id.Value, id);
    }

    public static MessageId from(int value)
    {
        if ( !_map.containsKey( value ) ) {
            throw new IllegalStateException("The value <" + value + "> is not a valid message id");
        }

        return _map.get(value);
    }
}

@Message
public class MessageDescriptor
{
    public byte version = 1;
    public MessageId messageId;

    public MessageDescriptor(MessageId theId)
    {
        messageId = theId;
    }

    public MessageDescriptor() {}
}

...should I be expecting the serialized csharp classes to deserialize correctly in java?

I am finding that deserialization on the java side completes without any error, but the value of the deserialized enum is always "SecondMessage", regardless of the value when serialization occurs.

If I should not expect this to be working, what changes can I make to the types to get them to work?

Hey,

I was trying to compile your library through the .NET Native toolchain and realised that it has a dependency on libc.dll (UnsafeNativeMethods.cs:52) for memcmp. (As a result, it doesn't run and probably fails during runtime.) I believe MSFT has deprecated libc.dll (http://msdn.microsoft.com/en-us/library/abx4dbyh.aspx) and only allows you to use libcmt or mscvrt. (It's in that tiny Note section.)

Is it possible to update this method?

Thanks!
Connie

Hi,
One of the issues we had was that deserialization requires knowledge of the type you are deserializing to. For a generic component this is not always possible. I may have missed something but we ended up writing a thin wrapper around Pack and Unpack to embed type information into the stream and adding at least the IFormatter interface so we could use MsgPack where an IFormatter is required.

Does the library have a higher-level formatter that imbed type information and can deserialize from it without prior type knowledge?

I'm using the library to write a automation module in PowerShell for Metasploit. The command for getting a list of sessions when received gets parsed in Ruby as:

>> rpc.call("session.list")
=> {1=>{"type"=>"meterpreter", "tunnel_local"=>"192.168.1.104:8080", "tunnel_peer"=>"192.168.1.183:1028", "via_exploit"=>"exploit/windows/smb/ms08_067_netapi", "via_payload"=>"payload/windows/meterpreter/reverse_tcp", "desc"=>"Meterpreter", "info"=>"NT AUTHORITY\\SYSTEM @ WIN2K3VMTEST", "workspace"=>"default", "session_host"=>"192.168.1.183", "session_port"=>445, "target_host"=>"192.168.1.183", "username"=>"carlos", "uuid"=>"zkzemzq5", "exploit_uuid"=>"mjqmtx7b", "routes"=>"", "platform"=>"x86/win32"}}

Now when I use msgpack.dll in both C# and PowerShell I get the following exception:

"Do not convert System.Byte (binary:0x1) MessagePackObject to MsgPack.MessagePackString."

Any clue what the error could be? I even tried it in python and it seemed to work fine with the python library. I compiled msgpack.dll using VS 2012

I would use msgpack on Netduino using its .net microframework.
Might work?
Alternatively, you know a library for microfw?

Hi @yfakariya
Is it possible to add an option to name deserialized/serialized properties by a custom method rather than adding a MessagePackMemberAttribute or a DataMemberAttribute?

Newtonsoft.Json has a IContractResolver interface which has a default implementation named DefaultContractResolver which has a CreateProperty method that can be overriden. Is it possible to add something like this to msgpack-cli?

Thanks

I am download the msgpack on the release and using the pre-generation to Generate a serializer file.
My Information : version : 0.5.6 os:mac 10.10

for example:
mono mpu.exe -s xxx.cs

but it will show this log:
Source 'xxx.cs'
Line:8, Column:13
CS0246
The type or namespace name `xxx' could not be found. Are you missing an assembly reference?
I find it not reference dll and used : mono mpu.exe -s xxx.cs --references=a.dll,b.dll,c.dll(all the dll is in the project)

show log : System.IO.FileNotFoundException: Could not load file or assembly 'Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies.

My xxx.dll file is copy from the project and the xxx.dll file is Special Generated ?

what can i do to resolve the problem?

Hi! I'm working on Unity3D and I tried write a code which receive data and unpack it.
The data received has been packed with MsgPack on Python, but I don't know how it was made. Now, I tried unpack the data on Unity3D/C# and I have some of problem about this because I don't know how I must do it.
I know that I must obtain from the packed data a structure with a key-values pair.
The code, which I have write like as:

    // "package" is the data received (as string)
    // "GetBytes()" take the string and return their bytes.
    var stream = new MemoryStream(GetBytes(package));
    string unpackage = string.Empty;
    for (int i = 0; i<=package.Length; i++) {
        var rawObject = Unpacking.UnpackObject (stream);
        unpackage += rawObject.ToString();
    }
    Debug.Log (unpackage);

So, the raw data, before the code:
??elevation?@@?33333?name?GnssMessage?timestamp?A??'1????longitude?@b?�?)S�?source_timestamp?
And the data after the code:
630630101010801010118097011601050111011006306406406305105105105105106301100970109010106307101100115011507701010115011509701030101063011601050109010101150116

Can any help me?

Thanks!

When trying to serialize an object that has an IList<> property, an exception is thrown:

using System.Collections.Generic;
using MsgPack.Serialization;

namespace Test
{
    public class DataObj
    {
        public IList<string> Data { get; set; }
    }

    public static class Program
    {
        public static void Main()
        {
            var dataObj = new DataObj
            {
                Data = new string[] { "Hello", "World" }
            };

            var serz = MessagePackSerializer.Create<DataObj>();
            var bytes = serz.PackSingleObject(dataObj);

            serz.UnpackSingleObject(bytes); // --> System.NotSupportedException: This operation is not supported because
                                            //     'System.Collections.Generic.IList`1[System.String]' cannot be instanciated.
        }
    }
}

I think the serializer should serialize the concrete type.

Or if that is too problematic, then maybe just the basic generic collections should be supported? IList<> can be serialized as an array and deserialized back to either an array or List<>.

I see you have pushed a NuGet package over a year ago and the current library is much more advanced then a year ago. Would love to have the latest stable on NuGet (you can also add them as -Pre packages)

Currently built-in object serializer uses fixed serialization context(dedicated instance before issue #25, and default instance as of issue #25), but it should accept custom serialization context instance.
I consider adding context as constructor parameter, and its value should be the context passed when the serializer is created.

I'll plan to add this improvement in 0.5 branch.

We cannot re-register serializers to override default (built-in) serializer like more-precised DateTime serializer.

Hi,

I was wondering if there is a way to replace the default object serialization method? I'd like to try to use MsgPack for an environment where I don't know anything about the underlying types. I'd like to have support for several data types: string, integers, list, maps, and (lets call them) "GenericObjects".
So everything which is not of type string, integer, list, or map should get a GenericObject in the end, so I'd implement my own serializer/deserializer for it.
My question is: Can I take advantage of the currently existing serialization methods for maps/lists while my custom serializer is used for any "unknown" object?

Currently, either lists or maps are generated for an unknown object I didn't register a serializer for. But this doesn't allow me to distinguish between "real" list/dicts and unknown objects after deserialization.
I tried to register my custom serializer at the SerializationContext with type "object", but this doesn't seem to work.

Is there an existing approach for this kind of "setting the fallback serializer"?

Thanks and keep up the great work!

In MsgPack.Cli 0.4.3, I can not unpack the IDictionary<string, string>. It's works in the previous 0.4.2.

By pack such a class, then unpack

public class SomeOne
{
    public string Target;
    public IDictionary<string, string> Data;
}

It is an error such as:

ハンドルされていない例外: System.NullReferenceException: オブジェクト参照がオブジェクト インスタンスに設定されていません。
   場所 MsgPack.Serialization.DefaultSerializers.System_Collections_Generic_KeyValuePair_2MessagePackSerializer`2.UnpackFromCore(Unpacker unpacker)
   場所 MsgPack.Serialization.MessagePackSerializer`1.UnpackFrom(Unpacker unpacker)
   場所 MsgPack.Serialization.UnpackHelpers.UnpackCollectionTo[T](Unpacker unpacker, MessagePackSerializer`1 serializer, IEnumerable`1 collection, Act
ion`1 addition)
   場所 MsgPack.Serialization.EmittingSerializers.Generated.System_Collections_Generic_IDictionary_2_System_String_System_String_Serializer0.UnpackFro
mCore(Unpacker )
   場所 MsgPack.Serialization.UnpackHelpers.InvokeUnpackFrom[T](MessagePackSerializer`1 serializer, Unpacker unpacker)
   場所 MsgPack.Serialization.EmittingSerializers.Generated.TestMsgpackCli001_TestMsgpackCli001_SomeOneSerializer0.UnpackFromCore(Unpacker )
   場所 MsgPack.Serialization.MessagePackSerializer`1.UnpackFrom(Unpacker unpacker)
   場所 MsgPack.Serialization.AutoMessagePackSerializer`1.UnpackFromCore(Unpacker unpacker)
   場所 MsgPack.Serialization.MessagePackSerializer`1.UnpackFrom(Unpacker unpacker)
   場所 MsgPack.Serialization.MessagePackSerializer`1.Unpack(Stream stream)
   場所 TestMsgpackCli001.TestMsgpackCli001.Extentions.UnPack[T](Byte[] data) 場所 c:\GitHub\sandbox\csharp\TestMsgpackCli001\TestMsgpackCli001\Progra
m.cs:行 51
   場所 TestMsgpackCli001.TestMsgpackCli001.Program.Main(String[] args) 場所 c:\GitHub\sandbox\csharp\TestMsgpackCli001\TestMsgpackCli001\Program.cs:
行 31

Reproduce code
https://gist.github.com/takekazuomi/10332165

Threading.Interlocked.CompareExchange doesn't work on Unity 3D iOS build because they runs with AOT-only mode so JIT is cannot be used.

SerializationContext.Default calls CompareExchange and SerializerRepository.GetDefault accesses to SerializationContext.Default. We have no way to set a custom SerializerRepository using public constructors of SerializationContext.
#16 was fixed but this problem still exists.

First at all awesome library,! scarely fast ;)

I noticed that the API doesn't have byte array methods , would love to have them. I use MessagePack as serializer/deserializer and creating a memory stream to read/write the byte array creates a lot of overhead.

Hello,
I jiust started to use your lib and faced with exception:
when I try to do this:
var serializer = SerializationContext.Default.GetSerializer<EntityDataContainer>();
I receive ArgumentException that says that element with the same key was already added.

Please, tell me what I am doing wrong here?

I tried to get my codebase mono compatible, but the messagepackobject keeps getting me a headache.

I'm on mono 3.0.6 and MessagePackObject.FromObject("test") breaks with a System.TypeInitializationException: An exception was thrown by the type initializer for MsgPack.MessagePackString ---> System.NotImplementedException: The requested feature is not implemented.

My code works perfectly on .Net tho.
I'm using the 0.4.0 nuget.

Edit:
Works fine with your source compiled into my project.

Question: Will the msgpack-cli nuget become mono-compatible or do i have to tinker around with different project files?
I would prefer a nuget which works on all platforms, even if it is a bit slower.

I tried to unpack { 0x82, 0x01, 0x81, 0x0a, 0x14, 0x02, 0x93, 0x14, 0x1e, 0x28 }. It can be represented as {1: {10: 20}, 2: [20, 30, 40]} in Python. It has nested dynamic collections as values.

using (var buffer = new MemoryStream(new byte[] { 0x82, 0x01, 0x81, 0x0a, 0x14, 0x02, 0x93, 0x14, 0x1e, 0x28 }))
using (var unpacker = Unpacker.Create(buffer))
{
    unpacker.Read();
    var serializer = MessagePackSerializer.Get<MessagePackObjectDictionary>();
    var val = serializer.UnpackFrom(unpacker);
}

MsgPack-CLI has a bug to unpack nested dynamic collection. It unpacks the above input as {1: 1, 10: 20}.

Hi I am trying to use MsgPack on a client project, it contains types where DataMember decorates private properties and not fields. When MsgPack tries to serialize those it fails because it doesn't look for DataMembers on private properties.

Could you maybe help me find out the best way to add support for that in order to be fuly compatible with DataContract specs.

I'm working on the 0.5 beta 1 version / branch

Thank you
Brgrds,

Serialize an object with a date set to a value such that the last two digits of ToBinary() are set.

Deserialize that object.

Compare the original date's ToBinary() with the deserialized date's ToBinary()

The last two digits get lost.

This is a really Great Project and Thank you for your superb work.
I would like to know if it's possible to use with Unity3D.

Cheers.

We get following error while unpacking packed message to an object.
This error is not noticed on local development environment (windows 7) but occurs in the server environment. (server 2008).

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Runtime.Serialization.SerializationException: The unpacked value is not expected type. Do not convert System.Boolean (binary:0x0) MessagePackObject to System.Int64. ---> System.InvalidOperationException: Do not convert System.Boolean (binary:0x0) MessagePackObject to System.Int64.
at MsgPack.MessagePackObject.ThrowInvalidTypeAs[T](MessagePackObject instance)
at MsgPack.MessagePackObject.AsInt64()
at MsgPack.Serialization.DefaultSerializers.System_DateTimeMessagePackSerializer.UnpackFromCore(Unpacker unpacker)
--- End of inner exception stack trace ---
at MsgPack.Serialization.DefaultSerializers.System_DateTimeMessagePackSerializer.UnpackFromCore(Unpacker unpacker)
at MsgPack.Serialization.DefaultSerializers.NullableMessagePackSerializer1.UnpackFromCore(Unpacker unpacker) at MsgPack.Serialization.MessagePackSerializer1.UnpackFrom(Unpacker unpacker)

Hi,

I'm unpacking List<List<MessagePackObject>> from byte array, here is my code

byte[] buffer = ...;

 var serializer = MessagePackSerializer.Create<List<List<MessagePackObject>>>();

var readerResults = serializer.UnpackSingleObject(buffer);

It is okay with small count of data, but with large number of data, it throws

"Unpacker is not in the array header. The stream may not be array."

Could it be memory problem?

Thanks!

Create or Pack with a class that has a reference to System.Exception will cause the following:

Unhandled Exception: System.Runtime.Serialization.SerializationException: Member
'HResult' is read only and its elementType ('System.Int32') is not an appendabl
e collection
at MsgPack.Serialization.AbstractSerializers.SerializerBuilder3.EmitSetMembe rValueStatement(TContext context, TConstruct instance, MemberInfo member, TConst ruct value) at MsgPack.Serialization.AbstractSerializers.SerializerBuilder3.<>c__Display
Class69.b__65(TConstruct unpackedItem)
at MsgPack.Serialization.AbstractSerializers.SerializerBuilder3.<EmitUnpackI temValueExpressionCore>d__1e.MoveNext() at System.Linq.Buffer1..ctor(IEnumerable1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable1 source)
at MsgPack.Serialization.EmittingSerializers.ILEmittingSerializerBuilder2.Em itSequentialStatements(TContext context, Type contextType, IEnumerable1 stateme
nts)
at MsgPack.Serialization.AbstractSerializers.SerializerBuilder3.<EmitObjectU npackFromArrayCore>d__6b.MoveNext() at System.Linq.Buffer1..ctor(IEnumerable1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable1 source)
at MsgPack.Serialization.EmittingSerializers.ILEmittingSerializerBuilder2.Em itSequentialStatements(TContext context, Type contextType, IEnumerable1 stateme
nts)
...