xhaskx / luamqtt Goto Github PK
View Code? Open in Web Editor NEWluamqtt - Pure-lua MQTT v3.1.1 and v5.0 client
Home Page: https://xhaskx.github.io/luamqtt/
License: MIT License
luamqtt - Pure-lua MQTT v3.1.1 and v5.0 client
Home Page: https://xhaskx.github.io/luamqtt/
License: MIT License
the luasocket connector uses shutdown
;
Line 22 in eacac48
the luasec based luasocket_ssl connector however uses close
;
luamqtt/mqtt/luasocket_ssl.lua
Line 45 in eacac48
is there a reason behind these asymmetrical calls?
It looks like openwrt/Makefile is missing a line for installing protocol5.lua.
Maybe there is only one line missing in Makefile
When large messages are received (I'm testing with a payload of 2625000 bytes), the library starts to misbehave.
Sometimes it hangs forever waiting for the actual message data to come in. Other times it'll think that the payload is part of the topic name. Sometimes it will not read the entire message, and think that a part of the payload is the header for the next message.
I noticed that increasing the socket timeout seems to largely resolve/workaround the issue.
It's not support PINGRESP packet_type received!
But When you invoke receive_iteration, you maybe wanna check keepalive.
label:Question
Hi,
I tried the simple client with TLS enable with correct credential information but it does not work.
I'm new to lua so not sure how to troubleshoot. I have the nodejs example working but the lua script does not connect. The original lua script is working.
Result:
# lua simple.lua
created MQTT client mqtt.client{id="lua-simple-client"}
running ioloop for it
done, ioloop is stopped
Code change:
uri = "abcdef.123.iot.us-east-1.amazonaws.com",
id = "lua-simple-client",
secure = {
mode = "client",
protocol = "tlsv1_2",
verify = "peer",
options = "all",
cafile="/lua/root-CA.pem",
certificate="/lua/Device.cert.pem",
key="/lua/Device.private.key"
},
version= mqtt.v311,
Compliments for this nice project!
I'm currently experimenting with it, but my lua environment is version 5.2 and cannot use LuaBitOp (or any other dynamic binding). Looking at the code, I suspect 32 bits support may be sufficient and I ported all of it to my framework simply using the bit32 functions; all unittests are succeeding.
Am I right about my suspicion or am I missing something?
There's a placeholder argument for ioloop
-- @tparam[opt] function args.sleep_function custom sleep function to call after each iteration
but arguments are not used anywhere and there's no direct access to this setting
My case is to connect 2 streams of operations for back and fourth messaging (between KNX bus and MQTT), so I need simultaneously read messages from both local bus and MQTT. My first assumption was that sleep function might be best place to put my bus loop.
Here's an example of bus read loop:
function groupcallback(event)
if event.dst == '1/1/1' then
local value = knxdatatype.decode(event.datahex, dt.uint16)
submit_to_mqtt(value)
end
end
lb = require('localbus').new(0.5) -- timeout is 0.5 seconds
lb:sethandler('groupwrite', groupcallback)
while true do
lb:step()
do_some_other_stuff()
end
What is the best way to implement this type of messaging loop?
Hello
I test with RabbitMQ, If I use from file: examples/quick-receive.lua, No SSL, port 1883 everything is Ok,
But when I use SSL port, self certs, has problem as below
Please correct the 'ssl_params' syntax in below config
Thank you so much
Use SSL with self certs
client = mqtt.client{
debug = client_debug,
uri = "x.x.x.x:8883",
auth = {username = "mqtt_user", password = "123456" },
clean = true,
ssl = true,
ssl_params = { cafile = "/certs/device/cacert.pem",
certificate = "/certs/device/clientcert.pem",
key = "/certs/device/clientkey.pem" },
}
Error examples/quick-receive2.lua
lua: /usr/lib/lua/mqtt/luasocket_ssl.lua:30: attempt to concatenate local 'err' (a nil value)
stack traceback:
/usr/lib/lua/mqtt/luasocket_ssl.lua:30: in function 'connect'
/usr/lib/lua/mqtt/init.lua:411: in function '_open_connection'
/usr/lib/lua/mqtt/init.lua:305: in function 'connect'
Hello
I was using with success the release v1.4 with OpenResty, by providing my own connector which uses ngx.socket.tcp
. But now, with the latest version (v3.0), I can't make it work due to the new ioloop
implementation, which apparently depends on luasocket
.
Is it possible to run this latest version on OpenResty? Maybe by providing the necessary settings (sleep function, maybe) to ioloop
or to use a different implementation for ioloop
(which is suited for OpenResty)?
Thank you.
Hi, using the current master openwrt Makefile, the build fails with the following verbose output:
mkdir -p /home/ubuntu/openwrt/dl
SHELL= flock /home/ubuntu/openwrt/tmp/.v3.1.tar.gz.flock -c ' /home/ubuntu/openwrt/scripts/download.pl "/home/ubuntu/openwrt/dl" "v3.1.tar.gz" "x" "" "https://github.com/xHasKx/luamqtt/archive/" '
Cannot find appropriate hash command, ensure the provided hash is either a MD5 or SHA256 checksum.
Makefile:49: recipe for target '/home/ubuntu/openwrt/dl/v3.1.tar.gz' failed
make[2]: *** [/home/ubuntu/openwrt/dl/v3.1.tar.gz] Error 255
make[2]: Leaving directory '/home/ubuntu/openwrt/feeds/jng_lua_openwrt/luamqtt'
time: package/feeds/jng_lua_openwrt/luamqtt/download#0.17#0.10#0.27
package/Makefile:111: recipe for target 'package/feeds/jng_lua_openwrt/luamqtt/download' failed
make[1]: *** [package/feeds/jng_lua_openwrt/luamqtt/download] Error 2
make[1]: Leaving directory '/home/ubuntu/openwrt'
make[1]: Entering directory '/home/ubuntu/openwrt'
make[2]: Entering directory '/home/ubuntu/openwrt/target/linux'
make[3]: Entering directory '/home/ubuntu/openwrt/target/linux/ar71xx'
make[3]: Nothing to be done for 'download'.
make[3]: Leaving directory '/home/ubuntu/openwrt/target/linux/ar71xx'
make[2]: Leaving directory '/home/ubuntu/openwrt/target/linux'
time: target/linux/download#0.03#0.03#0.07
make[1]: Leaving directory '/home/ubuntu/openwrt'
I am working in an environment that doesn't have access to Lua rocks and doesnt have access to luasockets.
The Library I have access to is a TcpSocket, found here.
https://q-syshelp.qsc.com/#Control_Scripting/Using_Lua_in_Q-Sys/TcpSocket.htm
The library is not blocking, and the environment does not allow me to block either. I have access to a Timer library found here that utilizes callbacks.
https://q-syshelp.qsc.com/#Control_Scripting/Using_Lua_in_Q-Sys/Timer.htm?TocPath=Design%257CControl%2520Scripting%257CQ-SYS%2520Extensions%2520to%2520Lua%257C_____34
Can you see a way to create a connector with this? Thank you,
Hi.
It looks like luasocket-copas.lua is missing from rockspec and it is not getting installed.
Our broker was recently DDoSed, which caused a disconnect on our luamqtt end, and a reconnect attempt where the connecting routine took over a minute after which our service was automatically killed for being frozen by our watchdog.
The stacktrace showed us that in fact the ioloop is not used for connecting in mqtt:
Line 730 in 929209a
Is this an oversight or by design? Would a rearchitecturing be out of the question so that connect would be asynchronous also?
(Connect will always block if there is DNS resolving to be done so this may also not be the perfect solution for everyone).
Another option is we could establish a connection ourselves and just provide the connected socket to luamqtt.
luamqtt fails on Lua 5.1 and 5.3
/usr/local/bin/lua: /usr/local/share/lua/5.1/mqtt/client.lua:47: module 'mqtt.protocol5' not found:
/usr/local/bin/lua: /usr/local/share/lua/5.3/mqtt/client.lua:47: module 'mqtt.protocol5' not found:
I've used LuaRocks to install luamqtt.
For Lua 5.1 and 5.3, the installation is the same; for 5.2, it's different.
See the attached file.
One of the odd things is that the installation for Lua 5.2 is also missing mqtt.protocol5
, but it never complains ... and works. Stranger even is that mqtt.protocol5
appears in here in GitHub.
these lines; https://github.com/xHasKx/luamqtt/blob/eacac48e31aa1a58974ef05d76687d8554a33207/mqtt/client.lua#L991:L994
send a "connect" event, when the connection actually fails. Is this intentional or a bug?
hello, salam, is it possible to make a mqtt connection between browser and lua over websocket
Hi!
How do I specify own args
(esp. timeout
) to default autocreated ioloop?
Can't see how to pass them to https://github.com/xHasKx/luamqtt/blob/master/mqtt/ioloop.lua#L168
TIA
Hi @xHasKx
Thx for this nice library, I've been eyeballing MQTT for a while now, and thought to give it a go actually.
The code looks nice and clean, but what I'm missing is a streaming interface. Looking at the code, currently client.lua
mixers client functionality with io-loop functionality. Which isn't the right separation of concerns imho.
So would you be interested in a streaming interface? I've seen another issue about blocking IO (#23 ). So what I would propose to separate the client from the runloop is along the following lines;
client:consume_bytes(byte_string)
. This means the client is no longer responsible for reading data, only for processing it. Any runloop can read data in whatever way it sees fit, and then feed that into this method.send_bytes(byte_string)
. Whenever the client needs to send data, it calls this callback with the data to be transmitted.This way the client and the runloop are cleanly decoupled.
wdyt?
as a start I sent a PR (#31 ) to expose the keepalive functionality to external runloops.
Hello,
I am currently working on a project involving Teltonika devices, specifically with smart meters. The objective of this project is to retrieve meter readings and transmit the data to the Azure portal. When I am trying to publish a message to Azure portal with the help of mqtt in Lua language but I am getting the "Connection Refused, not authorized" . I am referring this document https://github.com/xHasKx/luamqtt. I have used the same credentials for sending the data to Azure in python language and the python script is working. I am using the Lua 5.4 version and below is my lua script
local mqtt = require("mqtt")
local json = require("lunajson")
local luasec = require("ssl")
print("using luasec runtime version: ", luasec._VERSION)
local path_to_root_cert = "<root directory>/src/cert/digicert.crt"
local device_id = "Teltonika02";
local sas_token = "SharedAccessSignature sr=<hostname>.azure-devices.net&sig=xeJT3F1XsYB5uIcbLjW3L9do4NXlUIOC1kBfNZ5zHuQ%3D&skn=iothubowner&se=2045963831";
local iot_hub_name = "<hostname>.azure-devices.net/<device id>/?api-version=2021-04-12";
local client = mqtt.client({
uri = "<hostname>.azure-devices.net",
username = iot_hub_name,
secure = {
mode = "client",
protocol = "tlsv1_2",
cafile = path_to_root_cert,
certificate = nil,
key = nil
},
password = sas_token,
version = mqtt.v50,
clean = true
});
print(client, ": client")
local topic = "devices/" .. device_id .. "/messages/events/$.ct=application%2Fjson%3Bcharset%3Dutf-8"
local payload = [[{'id':123}]]
client:on{
connect = function(connack)
print("connack....", connack)
if connack.rc ~= 0 then
print("connection to broker failed:", connack:reason_string(), connack)
return
end
print("Now go for subscribe")
-- connection established, now subscribe to test topic and publish a message after
assert(client:subscribe{ topic=topic, qos=1, callback=function()
assert(client:publish{ topic = topic, payload = payload })
end})
end,
message = function(msg)
print(msg, ": messgae")
assert(client:acknowledge(msg))
-- receive one message and disconnect
print("received message", msg)
client:disconnect()
end,
}
-- run ioloop for client
mqtt.run_ioloop(client)
-- Check if the MQTT client is connected
if client.connected then
print("Connection is established")
else
print("Connection is not established : ", client.connected)
end
Hello:
Im try to using in Corona SDK and show a a error
loop or previous error loading module 'mqtt.bit'
12:55:47.955 stack traceback:
12:55:47.955 [C]: in function 'require'
12:55:47.955 ?: in function <?:820>
12:55:47.955 (tail call): ?
12:55:47.955 [C]: in function 'require'
12:55:47.955 ?: in function <?:820>
12:55:47.955 (tail call): ?
12:55:47.955 [C]: in function 'require'
12:55:47.955 ?: in function 'require'
do you know why it?
Hi,
luamqtt does not install under luarocks for lua 5.4.
the culprit appears to be the line "lua >= 5.1, < 5.4" in the rockspec file.
Changing the '5.4' to '5.5' allows building/installation of luamqtt under lua 5.4/luarocks.
Hi,
I am embedding Lua in my Qt application. However, it seems to me that running luamqtt would require me to run it in a separate thread, because it uses its own eventloop. Is it possible to run luamqtt via a custom eventloop? The workflow would be:
Is it possible to implement something like this? I can see that there is an iteration
function, that would maybe work for this usecase? I'm not how many iterations would one operation take and also, I don't want the Lua code to block if the connection is disrupted (when an operation gets a timeout). Or maybe I can use a custom connector? From what I can read, the connector doesn't allow me to use a custom eventloop.
Thanks
Hi again ;)
Thanks for quick fix for #40 it makes working with docker much easier.
I bumped into another problem while using copas connector, but I'm not sure if it's stricly related to it.
I'm getting closed connection to broken after publishing ~150 messages in batch. Adding "copas.sleep(0)" after each publish solved issue completly. Doing it is good enough for me, but i wanted to let you know about problem.
daniel@memory:~/mqtt/luamqtt/examples (master)*$ lua simple.lua
lua: /usr/local/share/lua/5.1/mqtt/init.lua:31: module 'mqtt.protocol4' not found:
no field package.preload['mqtt.protocol4']
no file './mqtt/protocol4.lua'
no file '/usr/local/share/lua/5.1/mqtt/protocol4.lua'
no file '/usr/local/share/lua/5.1/mqtt/protocol4/init.lua'
no file '/usr/local/lib/lua/5.1/mqtt/protocol4.lua'
no file '/usr/local/lib/lua/5.1/mqtt/protocol4/init.lua'
no file '/usr/share/lua/5.1/mqtt/protocol4.lua'
no file '/usr/share/lua/5.1/mqtt/protocol4/init.lua'
no file './mqtt/protocol4.so'
no file '/usr/local/lib/lua/5.1/mqtt/protocol4.so'
no file '/usr/lib/x86_64-linux-gnu/lua/5.1/mqtt/protocol4.so'
no file '/usr/lib/lua/5.1/mqtt/protocol4.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file './mqtt.so'
no file '/usr/local/lib/lua/5.1/mqtt.so'
no file '/usr/lib/x86_64-linux-gnu/lua/5.1/mqtt.so'
no file '/usr/lib/lua/5.1/mqtt.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'require'
/usr/local/share/lua/5.1/mqtt/init.lua:31: in main chunk
[C]: in function 'require'
simple.lua:2: in main chunk
[C]: ?
error info:
./lua: ./mqtt/clent.lua:148: unexpected key in client args: ssl = table: 0x38e418
stack traceback:
[C]: in function 'error'
./mqtt/client.lua:148: in method '__init'
./mqtt/client.lua:1165: in function 'mqtt.client.create'
(...tail calls...)
test_mqtt.lua:5: in main chunk
[C]: in ?
How can I resovle it. Thanks
I wrap luamqtt like this:
local require = require
local mqtt = require("mqtt")
local ioloop = require("mqtt.ioloop")
local log = require("core.log")
local setmetatable = setmetatable
local timer_at = ngx.timer.at
local str_format = string.format
local function on_timer(premature, ctx)
if premature then
log.warn("exit mqtt timer!")
return
end
log.info("starting mqtt client...")
local loop = ioloop.create{
timeout = ctx.cli.args.keep_alive,
sleep_function = ngx.sleep,
}
loop:add(ctx.cli)
ctx.cli:start_connecting()
loop:run_until_clients()
timer_at(1, on_timer, ctx)
end
local _M = {}
local mt = { __index = _M }
function _M:new(conf)
local cli = mqtt.client{
id = str_format("server/center/%d", ngx.worker.id()),
uri = conf.broker,
username = conf.username,
password = conf.password,
clean = true,
connector = require("mqtt.ngxsocket")
}
cli:on{
connect = function(connack)
if connack.rc ~= 0 then
log.error("connect mqtt broker failed!")
return
end
log.info("mqtt client connected!")
cli:publish{
topic = "luamqtt/simpletest",
payload = "hello",
qos = 0
}
end,
message = function(msg)
cli:acknowledge(msg)
log.info("receive mqtt message! ", msg)
end,
error = function(err)
log.error("mqtt client error: ", err)
end,
close = function(conn)
log.warn("mqtt conn closed: ", conn.close_reason)
end
}
return setmetatable({
conf = conf,
cli = cli
}, mt)
end
function _M:start()
return timer_at(1, on_timer, self)
end
function _M:publish()
timer_at(0, function ()
self.cli:publish{
topic = "luamqtt/simpletest",
payload = "hello",
qos = 0
}
end)
end
return _M
When mqtt connected, in connect callback function, the publish is worked, but i want publish a message when i received a http request, so i call the wrap function publish
, but it failed:
2020/04/24 16:54:17 [error] 2002#2002: *3424 lua entry thread aborted: runtime error: /mnt/d/Workspace/Demo/Openresty/test//lib/mqtt/client.lua:1147: bad request
stack traceback:
coroutine 0:
[C]: in function 'send'
/mnt/d/Workspace/Demo/Openresty/test//lib/mqtt/client.lua:1147: in function '_send_packet'
/mnt/d/Workspace/Demo/Openresty/test//lib/mqtt/client.lua:455: in function 'publish'
/mnt/d/Workspace/Demo/Openresty/test//lua/core/mqtt.lua:82: in function </mnt/d/Workspace/Demo/Openresty/test//lua/core/mqtt.lua:81>, context: ngx.timer, client: 127.0.0.1, server: 0.0.0.0:7500
2020/04/24 16:54:46 [info] 2004#2004: *3423 client closed connection while waiting for request, client: 127.0.0.1, server: 0.0.0.0:7500
It seems that broker is dropping connection (by design) because ping is sent only when connection is idle (if i read source correctly), but if i'm in receive-only mode and constantly receiving something, then ping may be never sent out. Have you faced this issue?
Steps to reproduce – with default settings connect to broker that sends something about every minute, and in ~1.5 minutes you'll be disconnected.
gcc——toolchain-mipsel_24kec+dsp_gcc-4.8-linaro_uClibc-0.9.33.2
lua:5.1.5
skynet:1.5.0
mqtt:3.4.2
My code is as follows:
local io = require("ioloop")
local ioloop = require("mqtt.ioloop")
local mq_loop = ioloop.create {
sleep = 1,
timeout = 1,
sleep_function = skynet.sleep
}
local client = mqtt.client(conn_params)
client:on{
connect = function(connack)
assert(not mqtt_client)
if not (connack and connack.rc == 0) then
logger:error("connection to mqtt fail: %s %s", connack and connack:reason_string() or "", connack)
return
end
logger:info("MQTT client connected server success!!!") -- successful connection
state_table.connect_ok = true
local response_count = 0
for _, item in ipairs(topics) do
local module, topic = item.module, item.topic
client:subscribe{
topic = topic,
qos = 0,
callback = function(suback, args)
response_count = response_count + 1
if suback.rc[1] ~= 0 then
client:disconnect(0)
logger:error("subscribe [%s %s] fail %s", module, topic, suback.rc[1])
return
end
logger:info("subscribe from cloud [%s, %s] ok", module, topic)
if response_count >= #topics then
mqtt_client = client
logger:info("All topic subscribes OK, MQTT client is ready!!!")
--Not support, upload software version with 5mins timer.
--[[
local mqtt_dispatcher = skynet.queryservice("mqtt_dispatcher")
skynet.send(mqtt_dispatcher, "lua", "post")
]]
end
end
}
end
end,
message = function(msg)
local err = client:acknowledge(msg)
if not err then
logger:error("received a error msg!")
return
end
table.insert(work_queue, {
msg.topic,
msg.payload
})
logger:info("Recv from cloud, topic: %s, payload: %s", msg.topic, msg.payload)
skynet.wakeup(work_loop_co)
skynet.yield()
end,
error = function(err, args, connack)
logger:error("mqtt client err: %s, args: %s, connack: %s", err, args, connack)
if not connack then
logger:error("MQTT client error maybe unable to connect to network")
return
end
if connack.rc == 4 then
state_table.connect_ok = false
client.args.reconnect = client.args.reconnect + 5
end
end,
close = function()
state_table.connect_ok = false
mqtt_client = nil -- set to nil mqtt_client = nil -- set to nil
client = nil
logger:error("mqtt client is close, set to not ready!!!")
end
}
mq_loop:add(client)
client:start_connecting()
while true do
if not client then
break
end
mq_loop:iteration() --------when run here is stuck
skynet.yield()
end
The mqTT_client is ready when the client is connected to the broker and three topics are subscribed. The mqTT_client is then stuck at mq_loop: Iteration ()
What is the cause of the jam? Appreciate receiving
I tried to install luamqtt with luarocks on Ubuntu 16.04, but I failed with the following logs:
daniel@memory:~/mqtt $ sudo luarocks install luamqtt
Installing https://rocks.moonscript.org/luamqtt-1.4.3-1.rockspec...
Using https://rocks.moonscript.org/luamqtt-1.4.3-1.rockspec... switching to 'build' mode
Cloning into 'luamqtt'...
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 29 (delta 2), reused 18 (delta 1), pack-reused 0
Receiving objects: 100% (29/29), 25.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (2/2), done.
Checking connectivity... done.
Note: checking out 'e2e16e029ff06f622d4750463ba64c18674d8d1f'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
Updating manifest for /usr/local/lib/luarocks/rocks
luamqtt 1.4.3-1 is now built and installed in /usr/local (license: MIT)
No code is installed:
daniel@memory:~/mqtt $ ls /usr/local/lib/luarocks/rocks/
index.html luabitop luafilesystem luamqtt luasec luasocket manifest manifest-5.1 manifest-5.2 manifest-5.3 penlight
daniel@memory:~/mqtt $ ls /usr/local/lib/luarocks/rocks/luamqtt/
1.4.3-1
daniel@memory:~/mqtt $ ls /usr/local/lib/luarocks/rocks/luamqtt/1.4.3-1/
doc luamqtt-1.4.3-1.rockspec rock_manifest
daniel@memory:~/mqtt $
https://github.com/xHasKx/luamqtt/blob/master/mqtt/protocol5.lua#L729 -- here len
could get false
breaking the comparison with number below and dumping the program.
Please revisit this chunk of code.
TIA
openwrt/Makefile compile error
last version
Cloud you please provide an example of LWT (Last Will & Testament) with luamqtt? Thanks very much.
Or provides some clues, I can make a PR with a LWT example.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.