tobozo / yamlduino Goto Github PK

YAML <=> JSON converter for ESP32, ESP8266, RP2040 and possibly other devices

License: Other

C++ 13.59% C 86.41%

arduino arduinojson cjson esp32 esp8266 libyaml yaml2json yamltojson json yaml

yamlduino's Issues

load external yaml file

Would be great to have an example that shows how to load a yaml file that is embedded on the device, e.g. a config.yaml.

would also like to know if it's possible to load a yaml file; modify structure (with a simple physical button to toggle a boolean value for example) and save back as a yaml file?

Undefined Reference to `deserializeYml`

I have the following code:

...

#include <SD.h>
#include <SPI.h>
#include <FS.h>
#include <ArduinoJson.h>
#include <YAMLDuino.h>

...

void SdHelpers::readInTagInfo(){
	File file = SD.open(tagInfoFileName);
	
	if( !file ) {
		LifeHelpers::unrecoverableError("Can't open config file for reading.");
	}

	DynamicJsonDocument jsonDoc(2048);
	auto err = deserializeYml(jsonDoc, file); // convert yaml to json
	file.close();
	
	if( err ) {
		LifeHelpers::unrecoverableError(String("Unable to deserialize YAML to JsonDocument: ") + String(err.c_str()));
	}
	
	JsonObject configJson = jsonDoc.as<JsonObject>();

	TagInfo newInfo(configJson);
	
	SdHelpers::tagInfoObj = newInfo;
}

Which seems to get past the compiler's syntax check, but when fails during the linking process:

~/.arduino15/packages/esp32/tools/xtensa-esp32s3-elf-gcc/gcc8_4_0-esp-2021r2-patch5/bin/../lib/gcc/xtensa-esp32s3-elf/8.4.0/../../../../xtensa-esp32s3-elf/bin/ld: sketch/SdHelpers.cpp.o:(.literal._ZN9SdHelpers13readInTagInfoEv+0x20): undefined reference to `YAML::libyaml_arduinojson::deserializeYml(ArduinoJson6200_F1::JsonDocument&, Stream&)'
~/.arduino15/packages/esp32/tools/xtensa-esp32s3-elf-gcc/gcc8_4_0-esp-2021r2-patch5/bin/../lib/gcc/xtensa-esp32s3-elf/8.4.0/../../../../xtensa-esp32s3-elf/bin/ld: sketch/SdHelpers.cpp.o: in function `SdHelpers::readInTagInfo()':
~/gitRepos/NameTag/arduinoSketch/NameTag/SdHelpers.cpp:14: undefined reference to `YAML::libyaml_arduinojson::deserializeYml(ArduinoJson6200_F1::JsonDocument&, Stream&)'
collect2: error: ld returned 1 exit status
exit status 1
Error compiling for board UM FeatherS3.

Any ideas?

Excessive memory use loading large yaml stream?

We are developing a very large project base on ESP32 and currently in the process of switching from json based config file to YAML format because of readability. We are using a YAML file from around 9000 characters.

The YAML file is read from SPIFFS and provided as a Stream, The code that parses the YAML file, allocates a DynamicJsonDocument of 20000 characters and then calls deserializeYml(jsonDocument, stream). We also have ArduinoJson in our project.
Right before calling deserializeYml() the ESP.getMaxAllocHeap() returns around 77.000 and still deserializing results in a "Not enough memory" error.

I've tried enabling debug or verbose level logging, but that doesn't give me any more logging then the "Not enough memory" error message

Why does it take more than 77kB to parse a 10.000 byte stream? I would think that te purpose of a stream is to not have all the data in memory at once

If I comment out the "currentcontoller" map at the bottom of the yaml file, the parsing does work ...

sensors:
  Main_Vref:
    id: 11
    nodes:
      main:
        type: ADCVoltage
        terminalId: 17
        pollingInterval: 10000
        r1: 36000
        r2: 10000

  OutsideTemp:
    id: 13
    nodes:
      iomodule:
        type: ADCThermistor
        terminalId: 44
        pollingInterval: 30000
        vRefNodeId: 11
      main:
        type: RemoteIntegerTimes10
        communicatorId: 4
        readOnly: true

  CabinetTemp:
    id: 25
    nodes:
      iomodule:
        type: ADCThermistor
        terminalId: 18
        pollingInterval: 30000
        vRefNodeId: 11
      main:
        type: RemoteIntegerTimes10
        communicatorId: 4
        readOnly: true

  LightsLargeBedroom:
    id: 33
    nodes:
      iomodule:
        type: PMW
        terminalId: 51
        storage: pwm
        key: large
      main:
        type: LightZoneRemoteSensor
        communicatorId: 4

  LightsBathroom:
    id: 30
    nodes:
      main:
        type: PWMLightZone
        terminalId: 24
        storage: pwm
        key: bathroom

  LightsSmallroom:
    id: 31
    nodes:
      main:
        type: PWMLightZone
        terminalId: 22
        storage: pwm
        key: smallroom

  LightsLivingroomLedstrip:
    id: 34
    nodes:
      iomodule:
        type: PMW
        terminalId: 49
        storage: pwm
        key: livingmood
      main:
        type: LightZoneRemoteSensor
        communicatorId: 4

  LightLivingroomTable:
    id: 35
    nodes:
      iomodule:
        type: PMW
        terminalId: 47
        storage: pwm
        key: living
      main:
        type: LightZoneRemoteSensor
        communicatorId: 4

  LightsOutside:
    id: 36
    nodes:
      iomodule:
        type: PMW
        terminalId: 45
        storage: pwm
        key: outside
      main:
        type: LightZoneRemoteSensor
        communicatorId: 4

  Outlets:
    id: 37
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 43
        stateNodeId: 107
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  OutletState:
    id: 107
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 10
        interruptEnabled: true
        pollingInterval: 10300

  Boiler:
    id: 41
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 42
        stateNodeId: 111
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  BoilerState:
    id: 111
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 11
        interruptEnabled: true
        pollingInterval: 10700

  HighpowerOutlet:
    id: 38
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 41
        stateNodeId: 108
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  HigpowerOutletState:
    id: 108
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 12
        interruptEnabled: true
        pollingInterval: 10500

  HeaterLiving:
    id: 39
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 40
        stateNodeId: 109
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  HeaterLivingState:
    id: 109
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 13
        interruptEnabled: true
        pollingInterval: 10600

  HeaterBathroom:
    id: 40
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 39
        stateNodeId: 110
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  HeaterBathroomState:
    id: 110
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 14
        interruptEnabled: true
        pollingInterval: 10400

  Stove:
    id: 42
    nodes:
      iomodule:
        type: MxPulseRelay
        terminalId: 38
        stateNodeId: 112
      main:
        type: ControlledRemoteBoolean
        currentControllerId: 180
        communicatorId: 4

  StoveState:
    id: 112
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 15
        interruptEnabled: true
        pollingInterval: 10800

  # Not present on testboard
  # Current:
  #   id: 43
  #   nodes:
  #     iomodule:
  #       type: ADCCurrent
  #       terminalId: 1
  #       pinMode: INPUT
  #       pollingInterval: 500
  #     main:
  #       type: RemoteIntegerTimes10
  #       readOnly: true
  #       communicatorId: 4

  # Not present on testboard
  # KwhCounter:
  #   id: 45
  #   nodes:
  #     iomodule:
  #       type: MxInterruptCount
  #       terminalId: 26
  #       pollingInterval: 10200
  #       divider: 100
  #       interruptMode: FALLING
  #       storage: intcount
  #       key: kwh
  #     main:
  #       type: StoredSettings<InterruptCounterType>
  #       storage: haaksnano
  #       key: powerusage

  WaterflowCounter:
    id: 50
    nodes:
      iomodule:
        type: InterruptCount
        terminalId: 3
        divider: 1
        debounceMs: 1000
        pollingInterval: 10900
        storage: intcount
        key: water
      main:
        type: StoredSettingsInterruptCounterType
        storage: haaksnano
        key: waterusage

  FlushRelais:
    id: 83
    nodes:
      iomodule:
        type: MxDigital
        terminalId: 37
        pinMode: OUTPUT
      main:
        type: RemoteBoolean
        communicatorId: 4

  SunUpdown:
    id: 21
    nodes:
      main:
        type: SunUpDown
        lightzoneIds:
          - 36
        pollingInterval: 60000

currentcontroller:
  id: 180
  main:
    maxCurrent: 25
    currentNodeId: 43
    currentNodeMode: PositiveLoad
    mainSwitchNode: 129
    mainSwitchReverseLogic: true
    pollingInterval: 10000
    nodes:
      - nodeId: 42
        prio: 100
      - nodeId: 37
        prio: 90
      - nodeId: 38
        prio: 80
      - nodeId: 182
        prio: 70
      - nodeId: 41
        prio: 60
      - nodeId: 181
        prio: 50

ArduinoJson binding issue with lists of objects

There seems to be a problem with the ArduinoJson binding specifically affecting lists of objects. Here is a minimal example that takes a simple YAML list of objects, deserializes it, then reserializes it to Serial. It does this twice, in two different ways:

The manual way: use YAMLDuino to convert the YAML to JSON, then deserialize that JSON using ArduinoJSON, then reserialize using ArduinoJSON.
Use YAMLDuino + ArduinoJson bindings to deserialize the YAML directly to an ArduinoJson document, then serialize to JSON.

(1) works, (2) loses the content of the objects.

Code:

// #1
YAMLNode yamlnode;
JsonDocument doc;
String buffer;
deserializeYml(yamlnode,"[{\"A\":\"1\"},{\"B\":\"1\"}]");
serializeYml(yamlnode.getDocument(),buffer,OUTPUT_JSON_PRETTY);
deserializeJson(doc,buffer);
Serial.println("YAML -> JSON -> deserializeJson -> serializeJson");
serializeJson(doc,Serial);
Serial.println("\n");

// #2
deserializeYml(doc,"[{\"A\":\"1\"},{\"B\":\"1\"}]");
Serial.println("YAML -> deserializeYml -> serializeJson");
serializeJson(doc,Serial);
Serial.println();

Output:

YAML -> JSON -> deserializeJson -> serializeJson
[{"A":1},{"B":1}]

YAML -> deserializeYml -> serializeJson
{"":[{},{}]}

I also tried #1 with OUTPUT_JSON instead of OUTPUT_JSON_PRETTY and it still worked fine, so the issue seems to be somewhere in the bindings themselves.

Request : wish the library support for ArduinoUno, Arduino MEGA.

It's a great library.
By the ways, if the library support for Arduino Uno/MEGA,
it's convenient for communication between MCUs by YAML.

Thank now.

serialize comments to yaml

Hello,

The reason why I prefer to use YAML is the ability to use comments. So my question is :
Since JSON doesn't support comments, how can I serializeYml() with comments?

I know you can deserialize a YAML with comments to a JsonObject, but I guess you loose the comments.

I want to :

read a YAML file (with comments in it)
change the content
write back the YAML with the changes and the comments

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.