We are seeing a lot of disconnects due to CONNECTION_LOST in AWS, where each of them correlates to an Error awaiting for last ping response.

3.295891 ERROR Error awaiting for last ping response
└─ mqttrust::state::{{impl}}::handle_outgoing_ping @ /home/mathias/.cargo/git/checkouts/mqttrust-76b1690cafd0c799/b51eee9/src/state.rs:309
3.295891 DEBUG Disconnecting from an event error.
└─ mqttrust::eventloop::{{impl}}::yield_event::{{closure}} @ /home/mathias/.cargo/git/checkouts/mqttrust-76b1690cafd0c799/b51eee9/src/eventloop.rs:128
3.296187 ERROR Abort error
└─ factbird_rs::idle @ factbird-rs/src/main.rs:788
3.298387 DEBUG Network connected!
└─ mqttrust::eventloop::{{impl}}::network_connect @ /home/mathias/.cargo/git/checkouts/mqttrust-76b1690cafd0c799/b51eee9/src/eventloop.rs:243
3.298387 INFO  MQTT connecting..
└─ mqttrust::eventloop::{{impl}}::mqtt_connect @ /home/mathias/.cargo/git/checkouts/mqttrust-76b1690cafd0c799/b51eee9/src/eventloop.rs:269
3.299087 INFO  Signal power: -109 dBm, qual: 2
└─ factbird_rs::idle @ factbird-rs/src/main.rs:605
3.299287 DEBUG MQTT connected!

Hi :)
I am currently trying to access the AWS IoT SDK in Rust from an XTensa ESP32 which uses the ESP-IDF.

So I stumbled across this neat project, rustot, which seems to be able to do exactly what I am trying to do.
There are some other similar libraries, like the official one from AWS. But they all seem to use Tokio, which is - as far as I know - not working with xtensa yet.

So rustot is the only promising library I could find which might work for my usecase.
But mqttrust - which is used by the project - does not compile for me with the ESP Rust toolchain and its xtensa-esp32-espidf target.

My test project is just a simple instance of the template, generated like described in the Rust on ESP Book with the mqttrust_core 0.4.1 dependency.

This also occurs outside of my project, when I checkout a fresh version of mqttrust and try to compile it standalone with the esp toolchain, so this is not a problem in my project setup.

Using the default nightly linux toolchain everything is working fine.
I have no idea why this error is occurring. I'd appreciate any input on this topic.

❯ cargo build

   Compiling mqttrust_core v0.4.1 (/run/media/jan/e061bc91-5c45-46ec-a182-e8145a1c0faa/projects/mqttrust/mqttrust_core)
error[E0432]: unresolved import `state::BOXED_PUBLISH`
  --> mqttrust_core/src/lib.rs:21:25
   |
21 | use state::{StateError, BOXED_PUBLISH};
   |                         ^^^^^^^^^^^^^ no `BOXED_PUBLISH` in `state`

error: cannot find macro `pool` in this scope
  --> mqttrust_core/src/state.rs:43:1
   |
43 | pool!(BOXED_PUBLISH: PublishNotification);
   | ^^^^
   |
note: `pool` is imported here, but it is a module, not a macro
  --> mqttrust_core/src/state.rs:8:16
   |
8  | use heapless::{pool, pool::singleton::Pool};
   |                ^^^^

error[E0433]: failed to resolve: use of undeclared type `BOXED_PUBLISH`
  --> mqttrust_core/src/state.rs:87:9
   |
87 |         BOXED_PUBLISH::grow(unsafe { &mut PUBLISH_MEM });
   |         ^^^^^^^^^^^^^ use of undeclared type `BOXED_PUBLISH`

error[E0433]: failed to resolve: use of undeclared type `BOXED_PUBLISH`
   --> mqttrust_core/src/state.rs:256:29
    |
256 |         let boxed_publish = BOXED_PUBLISH::alloc().unwrap();
    |                             ^^^^^^^^^^^^^ use of undeclared type `BOXED_PUBLISH`

warning: unused import: `pool::singleton::Pool`
 --> mqttrust_core/src/state.rs:8:22
  |
8 | use heapless::{pool, pool::singleton::Pool};
  |                      ^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
warning: `mqttrust_core` (lib) generated 1 warning
error: could not compile `mqttrust_core` due to 4 previous errors; 1 warning emitted

Currently we rely on a timer implementing embedded_hal::Countdown for keeping track of keep-alive, but this means that we cannot use the same timer for retry handling, as there is no way of obtaining the current timer value, rather than just checking if it has reached the timeout value.

Replacing the Countdown requirement with a Clock from https://github.com/FluenTech/embedded-time/, allows us to use a single timer for multiple things, including keep-alive, packet timeout, retry and delays.

This will make it much easier to approach #7, #3 and #6

Currently populate_pending is commented out, as it is untested.. Might work, might not, but until #1 is fixed, i wont handle it correctly

Add retry logic on outgoing packets.

If an error arises while connecting the socket to the remote IP, the socket is not closed before being dropped.

Example:

161502 DEBUG [TCP] Connect socket
161521 DEBUG Sending command with too long payload (81 bytes) to log! #(Connect msg)
161541 ERROR Received error response InvalidResponse
161541 ERROR InvalidResponse: b"\xaa\x00M\x00DAT+UDCP=\"tcp://54.171.23.225:8883/?ca=root_ca&cert=cert&privKey=priv_key\"\r\nU"
161561 DEBUG Sending command with too long payload (67 bytes) to log!  #(DNS)
161581 DEBUG Received response: "b"\xaa\x00\x08\x00E\r\nOK\r\nU""
161641 DEBUG Received response: "b"\xaa\x00X\x00A+UUPING:1,32,\"a3f8k0ccx04zas-ats.iot.eu-west-1.amazonaws.com\",\"54.171.23.225\",231,45\r\nU""
161642 TRACE [URC] PingResponse
161642 DEBUG [TCP] Opening socket
161643 DEBUG Adding socket! 0 Tcp
161643 DEBUG [Socket Set] Adding to: [(Handle(0), Tcp(State::Created)), (Handle(2), Tcp(State::Connected)), (Handle(4), Tcp(State::Connected))]
161644 ERROR [TCP] Opening socket Error: DuplicateSocket

This call is at eventloop.rs:341, where the socket is dropped without calling close first.

Solution

Add a close call to the call in eventloop.rs:341

Currently max inflight packets defaults to one, as it seems like it is not correctly handled at the moment.

I would like to see this crate split into a workspace of two crates:

• The mqtt abstraction (the trait)
• The actual implementation

This would allow us to do changes to the implementation, without higher level crates like rustot having to bump it's dependency version, unless the actual trait signature has changed.

This would furthermore allow people to easier use higher level mqtt crates with other mqtt clients, as long as they implement the shared abstraction.

Any suggestions on naming of said abstraction crate?

VecDeque elimination

As part of the strides towards an alloc-free implementation, the first first step would be to remove the VecDeque, in favor of something like heapless::LinearMap.
Fixed

Alloc-free mqttrs

An issue on the dependant mqttrs has been opened to push for alloc-free mqtt encoders and decoders. 00imvj00/mqttrs#27
Fixed

Remove String and Vec

Once the above mqttrs has been fixed, the String and Vec from alloc can be replaced with equivalent heapless versions. 00imvj00/mqttrs#29

Currently a few things are blocking a release to crates.io

• FactbirdHQ/ublox-cellular-rs#17 / FactbirdHQ/ublox-cellular-rs#19
• Release of defmt (https://github.com/knurling-rs/defmt/)
• 00imvj00/mqttrs#29
• Better documentation
• #2

Currently the TLS options in MqttOptions are unused:

https://github.com/BlackbirdHQ/mqttrust/blob/55cdb17605647aa18849028f13ab729072a329a5/src/options.rs#L19-L23

These should be used, but are currently missing an embedded-nal trait to be supported. Related issue: rust-embedded-community/embedded-nal#7

Currently SubAck packets are silently ignored, as the responseCode array needs to be handled correctly.

See https://github.com/BlackbirdHQ/mqttrust/blob/1600266795bcc5d3f33e566d5574a3ee4d5fc638/mqttrust/src/encoding/v4/subscribe.rs#L286-L308

And

https://github.com/BlackbirdHQ/mqttrust/blob/1600266795bcc5d3f33e566d5574a3ee4d5fc638/mqttrust_core/src/state.rs#L191-L199

The current PublishPayload trait is not really worth anything, and could be removed to make things much more lean!

Currently connection timeouts are handled through keep alive Ping-Pong's, but timeouts on SubAck, PubAck etc. are not handled.

Current implementation has a very high stack memory usage. Figure out how to reduce it.

Motivation

We need a way to allow users to persist outgoing MQTT packets to a non-volatile storage space, in case of network loss for longer periods. Ideally this should be a full opt-in feature.

Basic idea would be to make it two steps.

Persistent buffering to external flash

Work in progress i am thinking something along the lines of a key-value store, highly inspired by the persistence layer in Paho:

use embedded_storage::ReadWriteStorage;

pub enum PersistenceError {

}

pub struct Key();

pub trait MqttPerisistence: ReadWriteStorage {
    type Packet: MqttPayload;

    /// Initialize the persistent store.
    ///
    /// Either open the existing persistent store for this client ID or create a new
    /// one if one doesn't exist. If the persistent store is already open, return 
    /// without taking any action.
    ///
    /// An application can use the same client identifier to connect to many
    /// different servers. The clientid in conjunction with the 
    /// server_uri uniquely identifies the persistence store required.
    fn open(&mut self) -> Result<(), PersistenceError>;

    /// Close the persistent store referred to by the handle.
    fn close(&mut self) -> Result<(), PersistenceError>;

    /// Put the specified data into the persistent store.
    fn put(&mut self, key: Key, data: Self::Packet) -> Result<(), PersistenceError>;

    /// Retrieve the specified data from the persistent store.
    fn get(&mut self, key: Key) -> Result<Self::Packet, PersistenceError>;

    /// Remove the data for the specified key from the store
    fn remove(&mut self, key: Key) -> Result<(), PersistenceError>;

    /// Returns the keys in this persistent data store.
    fn keys(&mut self) -> impl Iter<Key>;

    /// Clears the persistence store, so that it no longer contains any persisted data.
    fn clear(&mut self) -> Result<(), PersistenceError>;

    /// Returns whether any data has been persisted using the specified key.
    fn contains(&mut self, key: Key) -> bool;


    /// A callback which is invoked just before a write to persistence.  This can be
    /// used to transform the data, for instance to encrypt it.
    fn before_write(&mut self, key) -> Result<(), PersistenceError> {};

    /// A callback which is invoked just after a read from persistence.  This can be
    /// used to transform the data, for instance to decrypt it.
    fn after_read(&mut self) -> Result<(), PersistenceError> {};
}

https://github.com/BlackbirdHQ/mqttrust/blob/55cdb17605647aa18849028f13ab729072a329a5/src/lib.rs#L205-L207

Strategies include jitter & exponential backoff on reconnection attempts.

This should also be tested by the Device Advisor test

Relevant implementation: https://github.com/FreeRTOS/backoffAlgorithm/blob/main/source/backoff_algorithm.c