256dpi / arduino-mqtt Goto Github PK

• Make the MAX_PACKET_SIZE configurable and add a note about the memory problem.

WDT reset occurs with ESP8266 sketch after connection lost from MQTT broker.. I this the normal behaviour??

Hello, I keep getting this when I basically copied the code and tried it on my Uno/Wifi101 Combination.
checking wifi...
connecting.............
Seems like that it could not connect to the broker.shiftr.io
Does someone have the same issue?

Thank you!

Is it feasible?
Thanks!

Ok, this is really weird but I'm able to reproduce this....

I'm porting code from using the PubSubClient library to this library. As you can imagine I had a few incompatibility issues. Nonetheless, I pressed on and but I was foxed with this one issue and couldn't figure out what I was doing wrong. So eventually, I use one of the Example sketches in order to provide a minimum reproducible case.

// This example uses an Adafruit Huzzah ESP8266
// to connect to shiftr.io.
//
// You can check on your device after a successful
// connection here: https://shiftr.io/try.
//
// by Joël Gähwiler
// https://github.com/256dpi/arduino-mqtt

#include 
#include 

const char *ssid = "ssis";
const char *pass = "pass";

WiFiClientSecure net;
 MQTTClient client;

unsigned long lastMillis = 0;

void connect(); // <- predefine connect() for setup()

void setup() {
 Serial.begin(9600);
 WiFi.begin(ssid, pass);
 client.begin("iot.eclipse.org", 1883, net); // MQTT brokers usually use port 8883 for secure connections

connect();
 }

void connect() {
 Serial.print("checking wifi...");
 while (WiFi.status() != WL_CONNECTED) {
 Serial.print(".");
 delay(500);
 }

Serial.print("\nconnecting...");
 while (!client.connect("arduino")) {
 Serial.print(".");
 }

Serial.println("\nconnected!");

client.subscribe("/example");
 // client.unsubscribe("/example");
 }

void loop() {
 client.loop();
 delay(10); // <- fixes some issues with WiFi stability

if(!client.connected()) {
 connect();
 }

// publish a message roughly every second.
 if(millis() - lastMillis > 1000) {
 lastMillis = millis();
 client.publish("/hello", "world");
 }
 }

void messageReceived(String topic, String payload, char * bytes, unsigned int length) {
 Serial.print("incoming: ");
 Serial.print(topic);
 Serial.print(" - ");
 Serial.print(payload);
 Serial.println();
 }

The changes I've made are:

1. I'm not using SSL, so I've changed the port to 1883
2. I'm using iot.eclispe.org as the MQTT broker
3. I'm using the connect method's overload that takes just the clientId parameter.

Now I have other cases (different code) where when I use a clientId of "NestEyeESP8266", I get the same soft WDT reset. However, if I change the clientId to "NestEyeESP826" or "NestEyeESP82A66" everything works as expected. I don't mind sharing this code privately if needed.

As proposed by @sandeepmistry:

[..], however the Wire and new I2S Arduino libraries have a function to called onReceive to set a function handler for received data (https://www.arduino.cc/en/Reference/WireOnReceive). It would be nice to have this, instead of forcing sketched to define void messageReceived(...).

Hi, any plans on including TLS or showing a TLS support example?

Ideally one would be able to determine why a connection request failed. By simply returning false this makes it difficult to establish a recovery procedure. Consider the following scenario: I use the library WiFiManager to acquire a SSID, password and authorization information for a MQTT server. In the event that the WiFi connection fails, I throw the device back into AP mode giving the user an opportunity to retype the SSID and password. However, if the MQTT connection fails I have no idea if it was because the server was offline, not accessible from my location or the authorization information was incorrect. In the event of the latter, I want to throw it back into AP mode so the user can reenter their MQTT authorization whereas in the former two scenarios I simply want to retry the connection. The CONNACK packet clearly states the reason for connection failure but it is not exposed to the application. I'm aware that simply returning this value instead of the BOOL would be a breaking change for existing code. Perhaps you could make it available as a class instance getter such as?

MQTTClient client;
client.begin(...);

if (!client.connect(id, name, password)
{
   if (client.getReturnCode() == MQTT_BADUSERNAME) // 0x04
   {
      // Get a new userid and password
    }
}

Thanks.

I know in Readme there is this pharagraph: "The first release of the library only supports QoS0 and the basic features to get going. In the next releases more of the features will be available. Please create an issue if you need a specific functionality." So, well...I'm creating the issue, then.

Thanks

PS: This alternative (https://github.com/adafruit/Adafruit_MQTT_Library) supports QoS1 publishing. This other (https://github.com/knolleary/pubsubclient) supports QoS1 subscribing

Hi
ESP8266_SSL get blocked with this library when receives more than 4-5 messages in the same time. Messages are greater then 1000 bytes.

Ex:
60,1730,294,866,50,44,52,140,52,44,56,38,58,38,56,38,56,40,56,40,56,38,56,136,56,38,58,40,54,138,54,138,54,40,54,138,50,142,52,140,58,134,56,136,56,38,58,38,56,38,58,38,56,38,58,40,54,42,48,46,50,46,48,46,50,46,48,48,50,44,52,44,50,44,52,44,52,42,52,44,52,42,52,44,50,46,50,44,52,46,48,46,50,46,50,44,50,46,48,48,48,46,50,46,50,44,52,44,50,44,52,44,52,140,50,140,58,282,290,866,58,136,54,40,52,46,48,46,50,46,48,46,54,42,50,44,52,44,50,142,50,44,52,44,52,140,50,44,52,140,52,140,52,140,50,142,50,144,54,138,54,40,54,42,50,44,52,44,52,42,52,44,52,42,52,44,52,42,52,44,52,44,52,44,50,46,48,46,50,46,52,44,50,44,50,46,54,40,54,42,54,42,52,42,54,42,54,40,54,42,54,40,54,42,54,40,56,40,54,44,50,44,52,44,50,44,52,44,52,42,50,46,54,282,296,864,54,138,54,40,56,40,54,40,56,40,54,40,54,42,54,40,56,40,54,140,52,44,52,42,52,44,52,140,52,140,52,138,54,42,54,138,54,138,54,136,54,42,54,140,52,44,50,140,52,140,52,44,54,40,54,42,54,138,54,138,54,136,56,40,54,40,56,40,54,42,54,40,56,42,52,140,52,140,52,140,52,140,54,136,54,42,54,138,54,40,54,42,54,138,54,40,54,42,54,44,52,42,52,44,52,42,52,44,52,140,54,136,56,0

First, thanks for the Arduino library. It works quite painlessly on the ESP8266.

However, in src/lib/MQTTClient.h there is a construct in PacketId::getNext() which, while it seems to work OK, is actually undefined by the C spec because you're preincrementing, doing a comparison on, and updating the same variable, "next":

---snip---
class PacketId
{
...
public:
int getNext()
{
return next = (next == MAX_PACKET_ID) ? 1 : ++next;
}
};
---snip---

GCC also complains about this line, so I think there really is something iffy here:
In file included from /home/earle/Arduino/libraries/MQTT/src/MQTTClient.h:14:0,
from /home/earle/Arduino/psychoplug/psychoplug.ino:26:
/home/earle/Arduino/libraries/MQTT/src/lib/MQTTClient.h: In member function 'int MQTT::PacketId::getNext()':
/home/earle/Arduino/libraries/MQTT/src/lib/MQTTClient.h:73:55: warning: operation on '((MQTT::PacketId*)this)->MQTT::PacketId::next' may be undefined [-Wsequence-point]
return next = (next == MAX_PACKET_ID) ? 1 : ++next;

I've unfortunately not traced the code to understand what exactly you're trying to do here, but it feels like splitting this into a couple lines would pay dividends in clarifying things and making sure that no GCC update can bonk it.

Thanks,
-EFP3

Hi,

it would be really great if you could implement support for the Particle Photon platform.

Thanks a lot!

As the library supports more and more boards and platforms like ESP8266 the specific YunMQTTClient implementation makes not that much sense anymore. The proposal is to remove the implementation and focus on lowering the memory footprint for the c++ implementation.

I am programing with esp8266 and want to public a message with retain, could you help me add this feature?

Download Paho python client class without installing the whole package.

Since that variant is not constrained, we could enable the full spectrum of MQTT features (QoS2).

The documentation only shows you how to connect to whatever.io, how would I set this up to connect to a local broker running on my PC? Commented code or actual documentation would help.

There no way to set will message retain parameter.

I think it its the best to remove the special yun client for v2 and focus on improving the main library and the backing paho library.

i see your library and it's declair support for qos1 on publish and subscribe, but i can't see any exaple how can i use this features.

If I turn off the Mosquitto broker and start it again, the Mega2560-client will not reconnect. The ESP8266 reconnects without a problem.

To be able to reconnect I had to add a 'disconnect' into the connection loop. Now my code looks like this:

void mqConnect() {
  uint8_t loops = 0;
  while (!mq.connect(MQTT_CLIENTID, MQTT_USERNAME, MQTT_KEY)) {
    Serial.print('.');
    if (++loops >= 2)
      return;
    delay(200);
    mq.disconnect();
  }
  Serial.print("\nConnected to mqtt-broker\n");
...

On the ESP8266, if I run 'disconnect' without being connected, it reboots.

I have all work-arounds I need now, but it would of course be nice to other beginners if this could be handled by the library.

I use esp8266/Arduino, and do same the example, network is well, but reconnect every few minutes，logs:

Scan wifi ... 2 networks found
Connecting to 03405
........
WiFi connected
IP address: 192.168.0.110
network time: 6:27:13
MQTT connecting...connected
MQTT connecting...connected
MQTT connecting...connected
MQTT connecting...connected

See #60.

Running SSL works fine on ESP8266. If you have time I would like the possibility to verify the servers SHA1 fingerprint. If I connect to a server manually it looks like this:

const char* fingerprint = "A8:D6:B6:0F:8B:E2:F6:5B:EE:94:16:88:BA:9B:4D:BD:58:CC:C0:AC"; // https://www.grc.com/fingerprints.htm
// ...
WiFiClientSecure  c;

if (!c.connect(host, httpsPort)) {
  Serial.println("Secure connection to api.pushjet.io failed");
}

if (c.verify(fingerprint, host)) {
  Serial.println("certificate matches");
} else {
  Serial.println("certificate doesn't match");
}
// ...

Thanks!

I am getting an error message - 'MQTTClient' does not name a type - unable to proceed

I'm using a uMQTT Broker that runs on anESP8266
https://github.com/martin-ger/esp_wifi_repeater/tree/uMQTT_broker

For the moment the Broker is compatible only with MQTT 3.1
How could I force arduino-mqtt library to use MQTT 3.1 and not 3.1.1?

Thanks

Please add support for last testament message.

If I exceed the max length of the MQTT packet, I expect to get some sort of failure. Instead, on ESP8266 at least, I get a system crash. I can't see any way of doing this without changing the API unless you simply ignore overly long messages.

This is possibly a bug in paho embedded of course, I'm having a bit of a hard time finding who's entirely responsible.

Is not better for the user's api to have only a single constructor for both mqttclient and yunmqttclient?
Considering the current code status could be quite easy to enable this feature at compile time with some #if defined(__AVR_ATmega32U4__)

If it is considered a good improvement I can make a PR.

As the underlying library is using templates to manage buffers we cannot just use any size dynamically.
To allow some freedom, we can integrate specific sizes the user can choose from:

client.use64Bytes();
client.use128Bytes(); // default
client.use256Bytes();
client.use512Bytes();

Hi Jöel, thanks for the processing fix about bytes in the other thread.

While with processing test.mosquitto.org works perfectly, I cannot make it work with Arduino.
I have verified that broker.hivemq.com works both in Arduino and Processing.

my example with Arduino is this:
https://github.com/marcobrianza/wemakecolors/blob/master/Arduino/test_WiFiMQTT/test_WiFiMQTT.ino

Marco

Getting status codes from publish/subscribe/unsubscribe would be very useful. Now they return void, so we don't know if the call has succeeded of failed. Getting status codes is important for the main program/sketches and would affect getting right decision in our programs.
Base MQTT libraries return status codes. You just need to "unhide" them.

You know, I've been worked with the PubSubClient and now I find your library.

Is your library better? In what aspects?

There will be an error if data contains a :.

I'm seeing references to WiFi101.h although it's not included in the branch. Where could I find that?

Thanks,

Adam

I found that it is possible to change the keep-alive time in lwmqtt.h where it defaults to 60 seconds.
I would appreciate an easier way to do it.

Hi Jöel is still me testing hard your framework :)

I have noticed that client.connected() returns true after stopping the MQTT server.

on the ESP8266 sending a message with the client disconnected crashes the board.

You call strlen on the payload internally. There should be a way to publish a binary payload with potential embedded nulls and provide the payload length explicitly.

Hi,

I am writing a library on top of arduino-mqtt for internal use. However I am getting weird error like Process::~Process() has undefined reference. The error messages are as below:

Arduino: 1.6.9 (Windows 7), Board: "Arduino Yún"

libraries\FalconMQTT_Yun\FalconMQTT_Yun.cpp.o: In function `~YunMQTTClient':

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.h:12: undefined reference to `Process::~Process()'

libraries\MQTT\YunMQTTClient.cpp.o: In function `Process':

C:\Program Files (x86)\Arduino\libraries\Bridge\src/Process.h:28: undefined reference to `vtable for Process'

C:\Program Files (x86)\Arduino\libraries\Bridge\src/Process.h:28: undefined reference to `vtable for Process'

C:\Program Files (x86)\Arduino\libraries\Bridge\src/Process.h:28: undefined reference to `vtable for Process'

C:\Program Files (x86)\Arduino\libraries\Bridge\src/Process.h:28: undefined reference to `vtable for Process'

libraries\MQTT\YunMQTTClient.cpp.o: In function `YunMQTTClient::updateBridge()':

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:22: undefined reference to `Process::runShellCommand(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:23: undefined reference to `Process::runShellCommand(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:24: undefined reference to `Process::runShellCommand(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:26: undefined reference to `Process::~Process()'

libraries\MQTT\YunMQTTClient.cpp.o: In function `YunMQTTClient::connect(char const_, char const_, char const*)':

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:43: undefined reference to `Process::begin(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:44: undefined reference to `Process::addParameter(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:45: undefined reference to `Process::addParameter(String const&)'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:46: undefined reference to `Process::runAsynchronously()'

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:83: undefined reference to `Process::close()'

libraries\MQTT\YunMQTTClient.cpp.o: In function `YunMQTTClient::publish(char const_, char_, unsigned int)':

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:115: undefined reference to `Process::write(unsigned char)'

libraries\MQTT\YunMQTTClient.cpp.o: In function `YunMQTTClient::connected()':

C:\Users\deric.dominic\Documents\Arduino\libraries\MQTT\src/YunMQTTClient.cpp:172: undefined reference to `Process::running()'

collect2.exe: error: ld returned 1 exit status

exit status 1
Error compiling for board Arduino Yún.

This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

Any idea on this issue?

Hi, can you help me saying what is wrong with my code? I want to send the status of one GPIO and recived one message and turn on an other GPIO with this comand but I actually send the status but I never recived the message. This is the code:

#include <ESP8266WiFi.h>
#include <MQTTClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>
#include <ESP8266HTTPUpdateServer.h>

/* ---------- DO NOT EDIT ANYTHING ABOVE THIS LINE ---------- */

//Only edit the settings in this section

/* WIFI Settings */
// Name of wifi network
const char* ssid = "Casa 2.4";

// Password to wifi network
const char* password = "%Cvap1992";

/* Web Updater Settings */
// Host Name of Device
const char* host = "Door1";

// Path to access firmware update page (Not Neccessary to change)
const char* update_path = "/firmware";

// Username to access the web update page
const char* update_username = "[email protected]";

// Password to access the web update page
const char* update_password = "$Olrvp3527";

/* MQTT Settings */
// Topic which listens for commands
char* outTopic = "Garage";

//MQTT Server IP Address
const char* server = "192.168.15.208";

//Unique device ID
const char* mqttDeviceID = "Device1";


/* ---------- DO NOT EDIT ANYTHING BELOW THIS LINE ---------- */

//the time when the sensor outputs a low impulse
long unsigned int lowIn;

//the amount of milliseconds the sensor has to be low
//before we assume all detection has stopped
long unsigned int pause = 100;

//sensor variables
boolean lockLow = true;
boolean takeLowTime;

//the digital pin connected to the door sensor's output
int sensorPin = 12;

//webserver
ESP8266WebServer httpServer(80);
ESP8266HTTPUpdateServer httpUpdater;

//MQTT
WiFiClient net;
MQTTClient client;

//Time Variable
unsigned long lastMillis = 0;
int channel1 = 13;
int channel3 = 16;
//Connect to WiFI and MQTT
void connect();

//Setup pins, wifi, webserver and MQTT
void setup()
{
  pinMode(sensorPin, INPUT);
  digitalWrite(sensorPin, LOW);
  Serial.begin(115200);
  WiFi.mode(WIFI_STA);

  WiFi.begin(ssid, password);
  client.begin(server, net);
  client.onMessage(messageReceived);

  connect();

  MDNS.begin(host);

  httpUpdater.setup(&httpServer, update_path, update_username, update_password);
  httpServer.begin();

  MDNS.addService("http", "tcp", 80);
}

//Connect to wifi and MQTT
void connect()
{
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(1000);
  }

  while (!client.connect(mqttDeviceID))
  {
    delay(1000);
  }
}

void loop()
{
  // MQTT Loop
  client.loop();
  delay(10);

  // Make sure device is connected
  if (!client.connected())
  {
    connect();
  }

  httpServer.handleClient();

  //Sensor Detection

  if (digitalRead(sensorPin) == HIGH)
  {
    if (lockLow)
    {
      //makes sure we wait for a transition to LOW before any further output is made:
      lockLow = false;
      client.publish(outTopic, "OPEN");
      delay(50);
    }
    takeLowTime = true;
  }

  if (digitalRead(sensorPin) == LOW)
  {
    if (takeLowTime)
    {
      lowIn = millis();          //save the time of the transition from high to LOW
      takeLowTime = false;       //make sure this is only done at the start of a LOW phase
    }
    //if the sensor is low for more than the given pause,
    //we assume that no more detection is going to happen
    if (!lockLow && millis() - lowIn > pause)
    {
      //makes sure this block of code is only executed again after
      //a new detection sequence has been detected
      lockLow = true;
      client.publish(outTopic, "CLOSED");
      delay(50);
    }
  }
}
void messageReceived(String &topic, String &payload)
{
  //String payload = payload;
  Serial.println("incoming: " + topic + " - " + payload);
  if (payload == "Z1ON")
  {
    digitalWrite(channel1, 1);
    delay(250);
  }
  else if (payload == "Z1OFF")
  {
    digitalWrite(channel1, 0);
    delay(250);
  }
  else if (payload == "Z2ON")
  {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(250);
  }
  else if (payload == "Z2OFF")
  {
    digitalWrite(LED_BUILTIN, LOW);
    delay(250);
  }
  else if (payload == "Z3ON")
  {
    digitalWrite(channel3, 1);
    delay(250);
  }
  else if (payload == "Z3OFF")
  {
    digitalWrite(channel3, 0);
    delay(250);
  } //This sensor does not recieve anything from MQTT Server so this is blank
}

Raised by @gibix in #33:

For expert users we could provide a subscribe(topic, *(void)(MQTT:Message)) method that allows attaching a message handler to a subscribe topic.

@gibix @sandeepmistry and all others.

I want to collect some ideas and thoughts about a possible V2 of this library. Do you have anything to add to what is already tagged with the v2 milestone?

AFAIK Arduino does not support this ATM.

Hello Joël
I changed today from imroy's mqtt library to yours, since I have stuck connections where I suspect his library to be the culprit.
Unfortunately, I have an error with your library too, but another one.

Here is an extract from the mqtt broker log (mosquitto on RaspberryPi):
1502914705: New connection from 192.168.11.80 on port 8883.
1502914705: New client connected from 192.168.11.80 as u (c1, k60).
It seems that the clientID is 'u'.
Since I use the same code for many devices, I fetch the hostname from a JSON formatted string stored in SPIFFS on the device. This hostname is definitely not 'u', since it is used at other places in the code and where it works. But since I'm not a programmer, but merely Frankensteining together foreign code, the error might be fully on my side and not in your library. For your reference, I attach the full code below.
Many thanks for any insight.
Kind regards,
Urs.
Sonoff_Generic_v1k.txt

The basic API of the library currently does not expose the return code of a connack packet when the connection has been denied. The following function could assist in the case a broker is rejecting the connection:

Serial.println("\nconnection attempt...");
while (!client.connect("arduino-mqtt", "try", "try")) {
  Serial.println(client.explainStatus());
}

I have the feeling the current signature is a little complicated for amateurs.

So rather than:

void messageReceived(String topic, String payload, char * bytes, unsigned int length) {
  Serial.print("incoming: ");
  Serial.print(topic);
  Serial.print(" - ");
  Serial.print(payload);
  Serial.println();
}

We could have something like this:

void messageReceived(MQTTMessage msg) {
  Serial.print("incoming: ");
  Serial.print(msg.topic());
  Serial.print(" - ");
  Serial.print(msg.payload());
  Serial.println();
}

Then we would also have the ability to add more convenience functions like:

if (msg.topicBeginsWith("/namespace")) {
  // do something
}

Hello,

Long story short, I'm trying to automize my program to make multiple connections to a Mosquitto brocker, without having to change the code every time I have to make a new connection. So far I've added 3 more connections from a single NodeMCU-ESP8266 with adding a PubSubClient client(espClient) and a reconnect() with just changing the name (client). My question is this: Is it possible to add some sort of counter (client counter), so when an event occurs, it to make a new connection?

I think it would make a lot of sense to support QoS 1 in this client - from my understanding of the protocol, it would require:

• allocating storage space for tracking packets which are yet to be acknowledged (capacity should be configurable)
• retransmitting packets when no confirmation has been received

I think a good starting point would be retaining 10 last packet ids by default. Once a packet is confirmed, it is removed from the set.

Further refinements could add an exponential delay mechanism for retransmissions, etc.

Did I miss anything?

Sorry, I know that this is not really an issue with your code and more with my stupidity or lack of knowledge. What I cannot figure out is how do I publish a retained message.

I know how to publish a normal message, as:

client.publish(stateSpeedTopic + String(invId), (fanSpeedStatus));

Both stateSpeedTopic and fanSpeedStatus are of type string.

How do I alter this to send a retained message instead?

Thank for so much for any help, I have been scratching my head for hours and not getting anywhere:(

P.S. Thank you so much for the code, it has been working excellent and is easier for me to folllow than the other alternative for esp8266 mqtt client available. I just need to get the state of my fan to be retained in my MQTT server. Otherwise, my fan decides to turn on randomly...lol.