min-protocol / min Goto Github PK

I think that to
func valid_frame_received should add fix:
case ACK:
...
if(num_acked <= num_in_window && self->transport_fifo.n_frames) { // There is one frame at least
...

Also to func min_poll one fix:
...
else {
// Sender cannot send new frames so resend old ones (if there's anyone there)
if((window_size > 0) && remote_connected && self->transport_fifo.n_frames) { //There is at least one frame

Better to be safe than sorry...

Hello ,

I am trying to port this library.

// Must be regularly called, with the received bytes since the last call.
// NB: if the transport protocol is being used then even if there are no bytes
// this call must still be made in order to drive the state machine for retransmits.
void min_poll(struct min_context *self, uint8_t const *buf, uint32_t buf_len);

#define TRANSPORT_FRAME_RETRANSMIT_TIMEOUT_MS       (50U) // Should be long enough for a whole window to be transmitted plus an ACK / NACK to get back
#define TRANSPORT_IDLE_TIMEOUT_MS                   (1000U)

>> In my application scenario, min_poll() will be blocked until the serial port has data. But I think this will destroy the internal state machine. If it must be called periodically, what is the best delay time?

Hello,
Have someone succeeded in running the protocol smoothly on TI MCUs?
it's working for me only on STM32F303VE
but it's not working on TI RM41L232 and TI CC3200MOD

1. ON STM ITS WORKS like this:
void HAL_UART_RxCpltCallback(UART_HandleTypeDef huart)
{
if (huart->Instance == UART4)//STM2DebugPort
{
/ Receive one byte in interrupt mode */
HAL_UART_Receive_IT(&huart4, &STM2DebugRxByte, sizeof(uint8_t));
min_poll(&STM2DebugMin_ctx, (uint8_t *)&STM2DebugRxByte, sizeof(uint8_t));
}
}

2. in CC3200MOD its can work slowly, but only if I pull the whole data from the UART interrupt handler Rx buffer at one shot. When I get an 0x55 byte (EOF) I'm pulling the buffer (SOF to EOF) at one shot from a different thread (Task) - SEE BELOW CODE

void Uart0IntHandler(void)
{
/* Receive interrupt /
if (MAP_UARTIntStatus(UARTA0_BASE, false));
{
MAP_UARTIntClear(UARTA0_BASE, UART_INT_RX);
/ Clear Interrupt /
while (MAP_UARTCharsAvail(UARTA0_BASE))
{
/ get the byte from the usart and push to rx buffer */
lRcvByte = (uint8_t)MAP_UARTCharGetNonBlocking(UARTA0_BASE);
if (lRcvByte != -1)
{
STMTxWifiRxBuff[idx] = lRcvByte;
tmp_byte = lRcvByte;
Report("%x ",tmp_byte);
/check if we have reached to EOF then poll on other thread/
if (tmp_byte == 85)
dataReady = true;
else
idx++;
}
}
}
}

the polling thread is:
void Uart0Poller(void)
{
if (dataReady)
{
dataReady = false;
min_poll(&STM2WiFiMin_ctx, (uint8_t*)&STMTxWifiRxBuff[0], ++idx);
// Report("Data polled -------> ");
idx = 0;
}
}

3. ON RM41L232 (Safety MCU - with no OS) it doesn't work and I'm losing Rx Frames, I tried it in both 2 ways like the above.
(it just that I cannot multithread like with the CC3200)

Please help I'm stuck on it a long time now

Apparently MIN needs python >= 3.5 to work (due to "type hints", at least).
Is this intentional?
I know we are in end-of-life for python2, but it is still widely used (and installed by default in many distributions), Installing python3 just to have MIN working seems a bit excessive, unless there are other motivations.

Hi Ken, I have been trying to remember C coding from long ago, and came across this stuff on StackExchange:

http://stackoverflow.com/questions/252780/why-should-we-typedef-a-struct-so-often-in-c

I note that some commenters there are strongly against using "typedef struct" and consider it an abuse of the C programming language LOL. Even: "If I have to maintain your code, I will remove your typedef'd structs."

Not sure what you think about this. I just thought it might be useful to think about.

I think I have the Python part of this working on Ubuntu Linux with Python 3. Should I send instructions, or do a pull request, or just post you the details?

Hi Ken, I think this callback function documentation needs some more info. This is called by your code but I'm not sure what it is passing...?

min_time_ms()
This is called to obtain current time in milliseconds. This is used by the MIN transport protocol to drive
timeouts and retransmits.

Hi Ken, it has been a frustrating day. Arduino is C++ so it took quite a bit of effort to get the C code to compile. I got there eventually using extern "C" to wrap your header file. There is now some serial activity but it is not working. I am really having problems with the steps I need to go thru to get things up and running.

The Python code also seems to be different to the previous version. I installed Continuum's Anaconda with Python 3.6 (I realise you specify 2.7, but figure it would likely work with a little tweaking). Not so far however -- the code just runs and exits:

C:\Users\resonanz\Documents\GitHub\min\host>python min.py

C:\Users\resonanz\Documents\GitHub\min\host>

I'm sure there is more to come :-)

Hi Ken,

First, thanks for what looks to be a great little protocol. However, during compilation I ran into following warning. I don't know how critical it is in itself, but it do prevent you code from doing a conditional decision as you intended.

min.c:497:42: warning: comparison is always true due to limited range of data type [-Wtype-limits]
                 if(self->rx_frame_length <= MAX_PAYLOAD) {

I run gcc compiler options -Wall -Wextra

Cheers,
Mogens

It seems that min_poll will go to handle TRANSPORT_PROTOCOL stuff even rx_byte() is still waiting more chars.

Would it be better just add to min_poll():
#ifdef TRANSPORT_PROTOCOL
uint8_t window_size;
if(self->rx_frame_state == SEARCHING_FOR_SOF)
{
now = min_time_ms();

I hit this when using code w linux.
I also use one-wire, so devices always sees (receives) own transmission.
usart hw has collision detection trigger also.
I also had to add src and tgt fields to the frame.

There seems to be a bug if you use multiple MIN ports at the same time with the TRANSPORT_PROTOCOL option enabled.

There is only one ring buffer defined in min.c:

// Where the payload data of the frame FIFO is stored
uint8_t payloads_ring_buffer[TRANSPORT_FIFO_MAX_FRAME_DATA];

While all other port related data - including access indices for this buffer - are available for each port in

struct min_context

The effect is that each port overwrites buffer data of the other ports.

I suggest that
uint8_t payloads_ring_buffer[TRANSPORT_FIFO_MAX_FRAME_DATA];
is moved to
struct min_context
so that each port has its own ring buffer.

The required changes are:

1. min.h - Add ring buffer in
   

2. min.c - Remove old ring buffer
   

3. min.c - make sure the two usages of the old buffer access the new buffer instead
   
   

Using min.c for multiple contexts (multiple calls to min_init_context with different port numbers and multiple calls to min_poll), I get cross-port data corruption due to the shared payload buffer "payloads_ring_buffer". If I tie payloads_ring_buffer to the min context struct, these issues are solved.
Am I missing something or is this a bug ?

As I understand it, SOF & EOF flags, coupled with a payload length field, help to align with incoming bytes for better frame synchronization. Another approach is using a defined time gap between frames.

See Section 5.3.1 of this paper or the way Modbus-RTU does SOFs & EOFs. I am wondering:

1. Has MIN tried this before and perhaps found it to not be effective for the use cases?
2. Are there any forks in existence that you know of that perhaps do this very thing?
3. If not, would you advise against me trying to fundamentally change the protocol like this?

Some USB serial handlers (e.g. REPL on MicroPython) interpret 0x03 as a CTRL-C character, so escape the 0x03 character to prevent it appearing in the MIN stream.

I'm trying to send some frames, about 60bytes in length. My mcu was not sending, because the AvailableForWrite() non-blocking limit on the Teensy 3.2 is 39 bytes.

What is the correct way to overcome this? I just overrode the limit and returned 1000UL available for write.

Thanks for the help and the great algorithm!
Tim

I think it would be helpful if the layers that are referenced in the WIKI and elsewhere were listed and defined. For example, I see there is a Layer 0 -- but it is not clear whether this is this part of the 7 layer OSI model? The Bit layer is presumably the Physical layer in the OSI model.

Thanks!

References

• https://support.microsoft.com/en-us/help/103884/the-osi-model-s-seven-layers-defined-and-functions-explained
• https://en.wikipedia.org/wiki/OSI_model

This ticket is a question, not a bug or feature request.

I'm interested in whether I can use a data-streaming pattern (as opposed to a request-response one) with this library. Hoping to have a sensor just continuously send over measurements, without the MCU having to explicitly ask for it each time.

Searching around, I noticed the "open loop" design objective which might suggest that this is supported. The same with this example code. Just looking for confirmation to be 100% sure.

Thanks!

Hi Ken,

I'm planning to use the min protocol on a device that runs a USB CDC, as I believe you did on the Arduino platform.

I know from experience that interface changes are a no-no. But, I hope it would be possible to perhaps add a compile options for a second version of the min_tx_byte where which allows for more bytes to be send on one call. Perhaps something like:

min_tx_byte(uint8_t port, uint8_t* data, uint8_t bytes_to_send, uint8_t* bytes_send)

You get the idea.

Apart from using a device with USB many devices also have DMA support making it possible to send a chunk of data with an UART.

HI Ken, are you still maintaining this code? Wondering also if you have used it on an Arduino? Thanks.

Hi @kentindell
Will you make a demo for C# in the future? I think that C# is closer to C than Python so it's easier for newbies to understand each part by learning the host/target demo.
I hope you would consider that soon.

Anyways, your work is awesome, thank you. You're my inspiration!

I was just going through the Python code for the host. Found this line:

self._sn_max - self._sn_max will always be zero and the if condition will always evaluate to False.

Hi,

Is there any experience with MIN protocol outside Arduino? I'm having trouble adapting it to the middleware supplied by STM32CubeIDE. Shouldn't be too difficult; unfortunately, I'm a newbie.

Sorry, I did some Google research, didn't find anything.

In min.c, the function transport_fifo_reset uses the now variable, but does not update it first. I suggest to insert a call to min_time_ms().

static void transport_fifo_reset(struct min_context *self)
{
    // Clear down the transmission FIFO queue
    self->transport_fifo.n_frames = 0;
    self->transport_fifo.head_idx = 0;
    self->transport_fifo.tail_idx = 0;
    self->transport_fifo.n_ring_buffer_bytes = 0;
    self->transport_fifo.ring_buffer_tail_offset = 0;
    self->transport_fifo.sn_max = 0;
    self->transport_fifo.sn_min = 0;
    self->transport_fifo.rn = 0;

    // Reset the timers
    now = min_time_ms();  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I suggest to insert this line
    self->transport_fifo.last_received_anything_ms = now;
    self->transport_fifo.last_sent_ack_time_ms = now;
    self->transport_fifo.last_received_frame_ms = 0;
}

Hey!
Would you be interested in a PR that would transform "host" part of MIN (Python bindings) into a proper (installable) package?

Background you might find interesting: A few years ago - before I discovered MIN - I've written SDP, a project very similar to MIN.
It served its purpose back then, but I wouldn't use it for any new projects. Now, I've got some old hardware that I want to reuse, but I need to take care of the communication - I've been already doing some changes to python code, but, since MIN looks way more mature than SDP, maybe investing some time in MIN is a better option for all of us. ;)

I am working with STM32 discovery board and I am testing this framework with a serial monitor (Docklight) since I don't have the host code setup yet. Since I know the frame structure I can construct a valid frame and send it via the serial monitor to the MCU.

Sending data from serial monitor to MCU works fine. When I send data from MCU to the serial monitor, I notice these things:

1. When sending via min_queue_frame, I receive the frame in the serial monitor multiple times. I am guessing that since there is no host code sending ACKs to the MCU, the target code keeps resending the frame. Am I correct?
2. When sending via min_send_frame I notice that one another frame is sent repeatedly to the serial monitor - AA AA AA FF 00 01 00 E6 5A E8 AC 55. This frame is sent 9 times after the original frame. What is this frame?

On a more general level, since these things are not mentioned in the wiki:

1. What is the main difference between the transport protocol and sending via min_send_frame? I read that the transport protocol guarantees delivery, but how is that achieved? Is it via sending ACKs from host to MCU?
2. How can I construct the ACK frame? I want to send the ACK when testing with a serial monitor.
3. If I use min_send_frame will frames be dropped on a noisy line?

Hi @kentindell,

I was implementing the HDLC protocol and while looking for more information about sliding window information I finally came across your library. *You did a really nice work, I like it. Congratulations.

I need something similar and I think that re-implementing HDLC from scratch would not be productive since I discover min.

But I have a couple of questions to which I can not find clear explanations.

1. What is the purpose of min_id parameters ? I dig into the exemple and source code but I'm not sure I understand the purpose/reason. Is it a kind of abstraction around the sequence number of the protocole ? Or it's more like an ID to recognize the min context associate with the operation ?

2. I'm saw a some magic number, 0x3fU, 0x80U, 0x55u from the source code. Would it be possible to to declare those in a define with the suffix of _MASK ? Ex. #define IS_TRANSPORT_MASK = 0x.... Also could you give me more details about these 3 masks ? Not about the arithmetic, but about the purpose of them ?

3. What is the purpose of adding Payload Length ? Don't you already can deduce it with the (EOF - SOF - bytes stuffing - header length - crc) = payload length ?

4. Why using 3 times the SOF (0xaa) instead of single one ? Since you are stuffing bytes (like HLDC framing), you are already preventing flag sequence to occur within the content of the frame ?

5. It would be nice to get exemple that compile on Linux (gcc) or Windows (VSxxxx). Are you going to add one? If I've come to better understand having received your answers, maybe I could do that 🍻

Sorry to create an issue, I would have liked to contact you by email, but I did not find it anywhere.

Best regards,
Martin

Hi,
In function valid_frame_received(), transport_fifo_send() is called without checking if the Tx has space or not.
Probably if(ON_WIRE_SIZE(frame->payload_len) <= min_tx_space(self->port)) should also be called before calling the line transport_fifo_send during nacked message tranmission. see the below code snipshot.

            // Now retransmit the number of frames that were requested
            for(i = 0; i < num_nacked; i++) {
                struct transport_frame *retransmit_frame = &self->transport_fifo.frames[idx];
                transport_fifo_send(self, retransmit_frame);
                idx++;
                idx &= TRANSPORT_FIFO_SIZE_FRAMES_MASK;
            }

Scenario for this problem:

1. Host (using min.py) sends some frames to client (which is Arduino with min.c)
2. Arduino listens for incoming frames and sends "keep alive" frames to notify host that everything is OK
3. At that point Arduino client is restarted (hard-reset)
4. Host keeps sending ACK frames for some old seq, client prints "Received spurious ACK" and basically stops working
5. Due to lack of "keep alive" frames, Host decides to perform transport_reset() - that resets client, but unfortunately Host keeps sending "spurious ACK"
   To fix that I've added self._rn = 0 to _transport_fifo_reset, it seems to work OK, but need to be confirmed by someone with expertise.

def _transport_fifo_reset(self):
        self._transport_fifo = []
        self._last_received_anything_ms = self._now_ms()
        self._last_sent_ack_time_ms = self._now_ms()
        self._last_sent_frame_ms = 0
        self._last_received_frame_ms = 0
        self._sn_min = 0
        self._sn_max = 0
        self._rn = 0

Should also stretch stuff byte for headers to 0xaa 0xaa 0xaa 0xaa so that a stuff byte is worst-case at most every three bytes, not two. This will reduce the size of the frame buffer that needs to be allocated.

Configure the embedded code to allow for a smaller buffer such that larger MIN frames are rejected. This will support smaller devices that cannot afford to allocate a large buffer and where the application doesn't need large frames.

Hi @kentindell

I'm currently having a project, where I want to use your nice min protocol!
I have an PC which is communicating over UART with an uC and that uC is communicating with another uC over RS485. I'm using a windows PC and STM32 m4 and m7 controllers.

The example from this repo is already working between STM32 and the PC.

My question:
Would it be possible for you to upload an example of the implementation on the Arduino & python side with the signal layer in the json format? Like the example you posted in your blog: https://kentindell.wordpress.com/2015/02/18/micrcontroller-interconnect-network-min-version-1-0/.

It would be nice to get an working example of how to implement the generated .c and .h files from the python code generator you wrote.

Best regards and thank you for the work you have done on this open source project,
Daniel

Hi Ken, I have tried to compile the min.c file inside an Arduino environment. To do this I have simply #included the min.c file (which itself #includes min.h). The compilation is perfect except for the following:

`In file included from C:\Users\resonanz\Documents\Arduino\MIN_testing\MinTest\MinTest.ino:25:0:

C:\Users\resonanz\Documents\GitHub\min\target\min.c: In function 'void rx_byte(min_context_t*, uint8_t)':

C:\Users\resonanz\Documents\GitHub\min\target\min.c:443:47: warning: left shift count >= width of type

         self->rx_frame_checksum = byte << 24;

                                           ^

C:\Users\resonanz\Documents\GitHub\min\target\min.c:447:48: warning: left shift count >= width of type

         self->rx_frame_checksum |= byte << 16;

                                            ^ `

byte is an uint8_t and so I guess the compiler is having a hissy about the shifting more that 8 bits. To test, I replaced the 24 and 16 above with 8 and everything compiles fine.

I'm trying to integrate MIN to my project, which targets ESP32, based on Arduino framework and developped in PlatformIO.

I tried to implement the min_application_handler callback in my pc_comm.cpp file, (the feature/include-min branch), but got linking error:

$ pio run

Linking .pio/build/esp-wrover-kit/firmware.elf
/home/quan/.platformio/packages/[email protected]+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp-wrover-kit/libb33/libmin-proto.a(min.c.o):(.literal.min_poll+0x0): undefined reference to `min_application_handler'
/home/quan/.platformio/packages/[email protected]+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/esp-wrover-kit/libb33/libmin-proto.a(min.c.o): in function `valid_frame_received':
/home/quan/Works/AgriConnect/Embedded/ebee-esp32/lib/min-proto/min.c:431: undefined reference to `min_application_handler'
collect2: error: ld returned 1 exit status
*** [.pio/build/esp-wrover-kit/firmware.elf] Error 1

Could you tell me what is the proper way to include, or how to fix it?

Note that, the main branch contains a working code, where I put

#include "min.c"

at the end of pc_comm.cpp, but I feel it is ugly. I try to include *.h file only.

Hi,

I'm trying to use the MIN protocol for a board that talks to 4 other boards via RS485 using one UART (with 8 RS485 transceivers, 4 on the control board and one on each sub board). Unfortunately, the control board sequence numbers increment 4 times faster than the sub-boards and so any message after the first ACK from the control board will not be handled by any board except the first. Is there some reason you used '==' for the sequence number check instead of >=? From what I remember, TCP does it that way (i.e. it doesn't care if the sequence number is equal but only if it's >=). I'd prefer not to have to use 4 min_contexts for my comms.

Thanks!
-m

Hi,

On Visual Studio 2015, this does not compile

#ifdef MIN_DEBUG_PRINTING
// Debug print
void min_debug_print(const char *msg, ...);
#else
#define min_debug_print(args...)         <-------- THIS
#endif

Give the error

Error	C2010	'.': unexpected in macro formal parameter list

To fix it replace by.

#define min_debug_print(...)

Suggestion 1

Sometime we only want to reset locale and not remote. Can you add a boolean to the function min_transport_reset(bool sendToOtherSide) ?

• When set to true you locally reset and you send the reset to the other side.
• When set to false, you only reset locally.

Other alternative would be to expose function transport_fifo_reset to min.h.

Suggestion 2

For some reason I would like to handle heartbeat on application side. Can you add a define to deactivate idle (periodic) ACK ?

#ifdef TRANSPORT_ACK_RETRANSMIT_ENABLED
    // Periodically transmit the ACK with the rn value, unless the line has gone idle
    if(now - self->transport_fifo.last_sent_ack_time_ms > TRANSPORT_ACK_RETRANSMIT_TIMEOUT_MS) {
        if(remote_active) {
            send_ack(self);
        }
    }
#endif

Hello, everyone.
I had written a rust library for MIN: https://github.com/qianchenzhumeng/min-rs.

The two MCUs I am working with - and most modern processors today, I believe - have built-in CRC computation units / modules / engines. As such, I'm curious if there is the possibility to feed in my own CRC, rather that MIN computing on its end?

I understand that the design intent is to be HW-agnostic. Perhaps some hook defined where the application can, on its end, invoke the CRC driver and return accordingly?

To be clear, more than requesting this be added to the library, I'm inquiring if it is possible in the current state and, if not, how easy it would be to do this in a fork of my own?

The host implementation appears to be only provided in Python, and the target implementation is provided in C.

It appears based on some comments in the Python implementation that is an asymmetrical (different) implementation to the C implementation.

We are not interested in running Python on the host, but we want to run C. To some degree it seems to work with the C implementation on the host and target. What features or aspects of the protocol are we missing out on by running the target implementation on the host?

Is it a common enough use case to want C for the host implementation that the C implementation will fully support the host as well as target?