Git Product home page Git Product logo

msgpack-arduino's Introduction

msgpack-arduino

Introduction

An implementation of the msgpack protocol for Arduino framework (also works with Platform IO).

COBS

COBS tokenisation is supported out of the box using the COBSRWStream class, e.g.:

#include <msgpack.hpp>

COBSRWStream stream(Serial);

void write() {
	msgpack::writeMapSize(stream, 4);
	{
		msgpack::writeString(stream, "first");
		{
			msgpack::writeInt(stream, 1);
		}

		msgpack::writeString(stream, "second");
		{
			msgpack::writeString(stream, "hello");
		}

		msgpack::writeString(stream, "third");
		{
			msgpack::writeFloat(stream, 3.14f);
		}

		msgpack::writeString(stream, "fourth");
		{
			msgpack::writeInt(stream, 4);
		}
	}

	// Call this to declare the end of a 'packet' so that the other side can decode it well
	stream.flush();
}

Any function that can be performed on the Serial object directly can be performed on the COBSRWStream.

Streams

All data in and out is streamed. This means that you don't have to hold an entire message in memory before it can be sent or received, e.g. if you want to receive a 10MB image so that you will send out to Neopixels, then you wouldn't need to store the entire image in memory at any point.

Serializer

There are some utility functions for writing using the Serializer class, e.g.:

void write() {
	msgpack::Serializer serializer(stream);
	
	// sugar method
	serializer.writeMap
	(
		"state", (uint8_t) this->getState()
		, "status", (uint8_t) this->getStatus()

		, "temperatureHeatsink", this->getTemp(TEMP_HEATSINK)
		, "temperatureDiode", this->getTemp(TEMP_DIODE)
		, "output", this->getOut()
		, "sensorError", this->getSensorError()
	);

	// stream operator method
	serializer.beginMap((uint8_t) 3);
	{
		serializer << this->projection.getTypeName();
		this->projection.reportStatus(serializer);

		serializer << this->device.getTypeName();
		this->device.reportStatus(serializer);

		serializer << this->tcm.getTypeName();
		this->tcm.reportStatus(serializer);
	}	
}

Usage patterns

There are different usage patterns. Pick and choose which ones you want to use. Generally all the important code is housed in the explicit functions. If you need to keep your code size small (e.g. if you're really fighting with data size limits) then consider only using one pattern type in your application.

  1. Explicit typed functions e.g. msgpack::readIntU8
  2. Implicit typed functions e.g. msgpack::readInt(Stream&, uint8_t &)
  3. Serializer class e.g. serializer << "my key" << myValue;

Optimisation

When writing this library, I've opted to write things specifically and safely. Based on my personal opinion, the design is well optimised when working with a good compiler. The use of streams instead of buffers will give significant performance advantages in many scenarios.

Known 'non-optimal' bits:

  • In the general functions, e.g. readInt, readFloat, the header will be read twice
  • Occasional stream.read() to skip header bytes when we already have the header

These are essentially 'comparing a one-byte value once more than necessary', so i doubt it matters. Also a compiler with whole program optimisation can likely notice that and ignore.

Also i believe that the getNextDataType() == XX will also be optimised away to be equivalent to getNextDataTypeIsXX().

Testing

We are doing some unit tests (so far especially for COBS). These are designed to work in Platform IO and are currently running on Windows with MinGW installed.

Background

I wrote this library after using https://github.com/HEADS-project/arduino_msgpack, so some of the ideas are taken from them (e.g. operating directly on the stream).

I'd suggesting reading through the msgpack spec at https://github.com/msgpack/msgpack/blob/master/spec.md so that you can understand the limitations.

License

MIT license

msgpack-arduino's People

Contributors

elliotwoods avatar scls19fr avatar seonghoonban avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

msgpack-arduino's Issues

String length and allocation

When calling the readString functions, I have to magically know how big the string buffer needs to be. There appears to be no way to check this ahead of time.

I propose a variant of readString, thusly:

char* readNewString(Stream& stream, bool safely = true) {
 DataType dataFormat;
 getNextDataType(stream, dataFormat, safely);
 size_t outputSize;
 char* value = nullptr;
 switch(dataFormat) {
  case DataType::String5: {
   MSGPACK_SAFETY_FORMAT_CHECK(DataType::String8);
   stream.read();
   MSGPACK_SAFELY_RUN(readRaw(stream, outputSize, safely));
   value = new char[outputSize + 1];
   MSGPACK_SAFELY_RUN(readRaw(stream, value, outputSize, safely));
   value[outputSize] = '\0';
   return value;
  }
  /* ... */
 }
}

What use is this?

If I don't want to pack data into a stream - e.g. Serial - then this library is pretty much useless. How the hell do I pack into a string? Every class constructor takes a Stream& - where am I getting this stream from?

It would be easier to write my own implementation than try to force yours to work.

Add msgpack-arduino to Arduino Library Manager

Hello,

I noticed that msgpack-arduino is available using PlatformIO as it have a library.json and is available at https://platformio.org/lib/show/5646/msgpack-arduino/

but it will be nice if msgpack-arduino could be added to Arduino library manager

According https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification
a library.properties file is required - and msgpack-arduino have one.
But it's not currently in http://downloads.arduino.cc/libraries/library_index.json

Maybe you should register create a new issue using https://github.com/arduino/Arduino/issues?q=is%3Aissue+is%3Aopen+label%3A%22Component%3A+Board%2FLib+Manager%22

Kind regards

Implicit functions question

Hi, and thanks for making this library available.

I notice that writeMapSize(Stream&, const uint8_t) checks if the value exceeds 1 << 4, and if not, encodes it as a map4 value.

However, I note that the other writeMapSize overloads (uint16_t and uint32_t) don't perform this check. Why is this? Surely if the value can fit in a map4, then it should be encoded as such? Wouldn't that be more space-efficient?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.