Allow to use Hex-Mode instead of binary mode for USOWR about ubxlib HOT 11 CLOSED

Great, thanks for letting me know! I'll be happy to test your preview branch, I still have my test setup on my desk.

Power consumption is not a priority for us, as we are not in a battery driven application. It is indeed purely about the time savings. We have small data packets, but they are sent as bursts in regular intervals (e.g. about 50 packets every 10 seconds). With the current delays in ubxlib, these bursts could actually end up taking longer than our interval timing, so that's why it's an issue for us.
Buffering these bursts is not really an option for us unfortunately, since we are using a Posix compatible socket API and a standard library on top of that, so we'd have to modify that library to allow buffering packets for some time. That's why I want to get the individual packets out as quickly as possible.

from ubxlib.

Hi again: that's fascinating data, we would be very interested in seeing your logic analyzer traces. A few questions for clarification:

• which cellular module is this?
• you mentioned an ST processor in your previous issue: which one do you happen to be using here?
• I guess the RTOS tick is 1 ms (it is by default in our ST configuration)?
• what are the "to" and "from" markers for your measurement: it looks like the "from" marker is having completely sent the USOWR AT command and the "to" marker is having received the OK\r\n response back from the USOWR AT command but please confirm.

The 50 ms delay that's in there after the @ prompt is somewhat of a fossil: it was in the original code I took as a basis many years ago, that code originally written for the SARA-G350 module, which of course ubxlib doesn't support, so it may well be that it could be removed.

Intuitively, having the MCU convert a message from binary to hex, sending it, and then the module converting it back from hex to binary again would seem like it should take longer but your measurement suggests that the process of the module sending the @ prompt, the MCU code detecting the @ and then reacting to the @, which will involve a few task-switches, is significant.

from ubxlib.

Hi,

We are testing this on a SARA-R422, which is normally hooked up to a STM32F7.
The test with ubxlib was running on a STM32F4 Nucleo board.
In both cases, we have UART with RTS/CTS flow control.
Our custom driver is also using DTR and DSR, but I don't think this should be an issue here.

The RTOS tick is indeed 1ms in our custom system, not sure about your demo.
The timing markers were placed on the A byte of the USOWR command

Regarding the 50ms timing with binary mode: This requirement is still part of AT manual, so I guess it's best to leave it there - we previously observed some issues when some timings were not respected in the custom driver, so we're careful to adhere to these timings for now. Imho, this additional delay should be necessary as a hardcoded thing at all, the flow control lines should be used to indicate modem readiness, but it seems like this is not done at all.

I've uploaded the traces as an attachment here:
logic_captures.zip
The filenames should be self-explanatory.
You can also observe a difference in the timing after the last data byte and the first AT result byte, I guess this could be down to server response timing - these tests were done on two different servers for simplicities sake.
You'll also notice that the data bytes are different, that should not be important either in my opinion.

Regarding your intuition about the hex-conversion: I had the same thoughts when we initially started working on our driver, but ended up abandoning binary mode due to the timing requirements. The 50ms delay would always result in this slowdown, even if there were no additional delays like the initial waiting time for the @ prompt.

This delay is actually pretty significant for us, since our system will mostly use periodic small data packets and infrequently send anything bigger than 50 bytes.

from ubxlib.

Thanks for the clarifications, we'll take a detailed look at this tomorrow.

My initial inclination is to keep it simple and add an API something like uCellSockSetHexModeOn() / uCellSockSetHexModeOff() / uCellSockHexModeIsOn() allowing the application to tell ubxlib to switch modes and leaving any decision about when to do so also in the hands of the application, rather than trying to be too clever "downstairs".

from ubxlib.

Sounds fine to me, I think you could easily add the automation for selecting the mode as a wrapper function if you really wanted it. Let me know when you decide on how to proceed or when there is anything that I can test!

from ubxlib.

We took a look at your graphs and it seems pretty clear: as you say, there is (a) the 50 ms to be concerned about and (b) there is delay, either on the module side or in the MCU/task-switching side, in waiting for the @ and in getting the OK\r\n back after the module has received the data, so even if we were to reduce (a) somewhat, we'd still have the other delays, which are almost as large when put together, to worry about.

We will implement a hex mode configuration item, something like what I suggested above. It might take a week as we have to test it across all module types but as soon as it is available I will post a preview branch here for you to test out, give us your opinion on.

Can I ask: is your drive for efficiency ultimately about time or about power? i.e. are you just as interested in management of power saving on the module side (which is something we are working on now)?

from ubxlib.

Hi again: we have implemented the hex mode option and pushed a preview branch here; if you get the chance please give it a try, the most recent commit message describes what's been added. Once we have reviewed the commit internally and received your feedback we will merge the change and push it here, at which point we will delete the preview branch.

from ubxlib.

I missed one file change off the preview branch, now updated.

from ubxlib.

Hey, thanks for integrating this so quickly!
I evaluated it (Seems like the missing change was not an issue) and compared the echo test with and without hex mode.
I've attached the logic analyzer traces again, the results are as expected:

• Sending takes 16.5ms in hex mode and 86.7ms in binary mode (same server this time, so there's probably no big impact from the network side here).
• Receiving gets slowed down a bit (15.9ms in hex mode and 12.7ms in binary mode)

All in all, I think the change works great and it seems like a great way to save a lot of time on writing. The performance loss on receiving could be reduced by adding a wrapper function in the application code that would dynamically toggle back to binary mode if the data amount exceeds some threshold.
logic_captures_ubxlib_hex_vs_bin.zip

from ubxlib.

Very good, thanks for doing that. @philwareublox: if you're happy with that and can complete the review I will merge the change, push it here and let @johannesmay know.

from ubxlib.

The stuff that is present on the preview_cell_sock_hex branch has now been pushed to master in commit b9b61a7, hence I will now delete the preview branch and close this issue.

Any problems, please either re-open it or create another.

from ubxlib.