Comments (22)
jeffwilliams
commented on August 13, 2024
Hey Mike,
Hmm, that's odd. So what the program tries to do is start a firefox, and
then connect to it to control it. That connection refused is it trying to
connect to firefox and for some reason either firefox is not running, or
not accepting the connection from nest-scape.
Can you try running nest-scrape with the -s option? Maybe something like:
nest-scrape -v2 -s
That -s option should make firefox appear visibly and so you can see if
there is something going wrong when it starts. It could be that it is
expecting some initial setup; in my tests it displayed some sort of profile
entry dialog for a few seconds, but then nest-scrape was able to navigate
to the nest page anyway. Perhaps that's not working on your system.
…-v 2 will set the max verbosity for debugging.
Let me know what you see. If it's not working I can add try and add some
extra debugging to help narrow down what's going wrong.
On Sat, Jan 26, 2019 at 1:06 PM Michael Ingraham ***@***.***> wrote:
Jeff,
I KNEW someone would have devised this workaround to Nest's
unresponsiveness in providing API access to the temperature sensor. Thanks!
I executed the command and it just sat there staring at me. So I ran it
with verbosity:
nest-scrape -v 1
browser: running browser using cmd '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile'
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
...
The connection refused message is repeated until I interrupt it with
Ctrl-C. Can you provide some guidance as to what the issue might be?
Thank you.
Mike
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA_n1tn7kZ8K6SAXIJNtAXpUY68so8Yaks5vHJkxgaJpZM4aUTA_>
.
from nest-scrape.
meingraham
commented on August 13, 2024
OK, I gave that a try. Same result. It seems to fail immediately without having been able to invoke Firefox at all.
What Linux distribution did you develop nest-scrape under? I am using the latest Raspbian Stretch. Also, the information I found said that the only Firefox version available for Raspbian is Firefox-ESR. This is the description given on that version's page:
Firefox Extended Support Release (ESR) is an official version of Firefox developed for large organizations like universities and businesses that need to set up and maintain Firefox on a large scale. Firefox ESR does not come with the latest features but it has the latest security and stability fixes.
I do not know what underlying facilities your code makes use of. Could it be that it's not finding something due the fact that ESR "does not come with the latest features"? Are there additional debugging levels that I can set to see what the call is where it's failing to connect successfully?
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
I developed this on Debian Jessie.
What happens if you run the firefox command manually with those options?
I.e.:
/usr/bin/firefox --marionette --headless --profile
/home/pi/.nest-scrape/firefox-profile
or alternately:
/usr/bin/firefox --marionette --profile
/home/pi/.nest-scrape/firefox-profile
Does it print any output? After running that last command, what is the
value of the $? environment variable? I.e.:
/usr/bin/firefox --marionette --profile
/home/pi/.nest-scrape/firefox-profile
echo $?
Also what is the output of /usr/bin/firefox --version?
It's odd that firefox is failing with no output to stderr, and yet the
Go exec call is not returning an error on first launch (i.e. it was
able to find the binary and execute it).
…On Sat, Jan 26, 2019 at 3:02 PM Michael Ingraham ***@***.***> wrote:
OK, I gave that a try. Same result. It seems to fail immediately without
having been able to invoke Firefox at all.
What Linux distribution did you develop nest-scrape under? I am using the
latest Raspbian Stretch. Also, the information I found said that the only
Firefox version available for Raspbian is Firefox-ESR
<https://support.mozilla.org/en-US/kb/switch-to-firefox-extended-support-release-esr>.
This is the description given on that version's page:
Firefox Extended Support Release (ESR) is an official version of Firefox
developed for large organizations like universities and businesses that
need to set up and maintain Firefox on a large scale. Firefox ESR does not
come with the latest features but it has the latest security and stability
fixes.
I do not know what underlying facilities your code makes use of. Could it
be that it's not finding something due the fact that ESR "does not come
with the latest features"? Is there additional debugging levels that I can
set to see what the call is where it's failing to connect successfully?
Mike
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA_n1ulvRwtGBtw6tD6MZr-wAZsVeT4Mks5vHLRMgaJpZM4aUTA_>
.
from nest-scrape.
meingraham
commented on August 13, 2024
Jeff,
It does appear to run successfully when run interactively. Here are the results.
$ /usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile
1548592442824 Marionette INFO Listening on port 2828
$ echo $?
130
$ /usr/bin/firefox --version
Mozilla Firefox 52.9.0
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Thanks Mike.
When you run firefox interactively like that, did it ask you to fill out any settings or anything, or could you immediately browse? If it did ask, does the script work after that?
Also, could you try running it interactively as you did, and in another terminal try connecting to that marionette port? I.e. in another terminal try:
telnet 127.0.0.1 2828
You should be able to connect and see something like the following:
$ telnet 127.0.0.1 2828
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
50:{"applicationType":"gecko","marionetteProtocol":3}
Then you can disconnect by typing CTRL-] (or whatever the escape character is in your telnet) and then typing close.
One thing that is odd is that the connection refused seems to be happening on the getsockopt call in the Go net library:
dial tcp 127.0.0.1:2828: getsockopt: connection refused
where on mine it fails in the connect call (until firefox is ready for connections):
dial tcp 127.0.0.1:2828: connect: connection refused
Let me see if there is any debugging I can add around there.
from nest-scrape.
jeffwilliams
commented on August 13, 2024
One other thing, can you try running firefox interactively as previously:
/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile
Then once you see:
1548592442824 Marionette INFO Listening on port 2828
From another terminal can you try seeing what address the OS thinks the firefox marionette socket is listening on? This command should do it:
netstat -an -A inet,inet6
I am wondering if 127.0.0.1 is not the configured local loopback address for some reason, or firefox is listening on IPv6 (address ::1).
from nest-scrape.
meingraham
commented on August 13, 2024
Jeff,
So... I had been running nest-scrape from MobaXterm (an ssh client) - i.e., not from the Raspbian GUI Desktop. Your note last night where you asked
When you run firefox interactively like that, did it ask you to fill out any settings or anything, or could you immediately browse?
clued me in that Firefox, even though invoked as headless, appears that it wants to render a browser window. Sure enough, invoking Firefox from the command line does indeed start a browser window. So, I then ran nest-scrape from a Terminal... and progress... but not success... yet.
If I start with a fresh session (no Firefox running) and just try running nest-scrape, I get the same repeating output:
$ nest-scrape -v2
browser: running browser using cmd '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile'
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
If I first invoke
$ /usr/bin/firefox --marionette --headless --profile ./.nest-scrape/firefox-profile
this opens a Firefox window (it does not request any more information of me - the window opens with the default Start tab)
If then, in another Terminal, I invoke
$ nest-scrape -v2
The browser opens another tab with a directory listing of file:///home/pi/.nest-scrape/firefox-profile/
and in the Terminal, the following output:
browser: running browser using cmd '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile'
[0,1,"WebDriver:NewSession",{"capabilities":null,"sessionId":""}]
2019/01/27 16:32:00 65:[0,1,"WebDriver:NewSession",{"capabilities":null,"sessionId":""}]
2019/01/27 16:32:00 [1,1,{"error":"unknown command","message":"WebDriver:NewSession","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\n - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,2,"newSession",{"capabilities":null,"sessionId":""}]
2019/01/27 16:32:05 55:[0,2,"newSession",{"capabilities":null,"sessionId":""}]
2019/01/27 16:32:05 [1,2,null,{"sessionId":"f26c1564-7aea-48dc-8eb8-d00466a59fda","capabilities":{"browserName":"firefox","browserVersion":"52.9.0","platformName":"linux","platformVersion":"4.14.79+","pageLoadStrategy":"normal","acceptInsecureCerts":false,"timeouts":{"implicit":0,"page load":300000,"script":30000},"rotatable":false,"specificationLevel":0,"moz:processID":4901,"moz:profile":"/home/pi/.nest-scrape/firefox-profile","moz:accessibilityChecks":false}}]
Navigating to site
[0,3,"WebDriver:Navigate",{"url":"https://home.nest.com"}]
2019/01/27 16:32:10 58:[0,3,"WebDriver:Navigate",{"url":"https://home.nest.com"}]
2019/01/27 16:32:10 [1,3,{"error":"unknown command","message":"WebDriver:Navigate","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\nDi - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
Checking if login is required
Waiting for thermostat location to load (element css selector .puck-item > a)
[0,4,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:12 84:[0,4,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:12 [1,4,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,5,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:16 84:[0,5,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:16 [1,5,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,6,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:18 84:[0,6,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:18 [1,6,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
Finding thermostat location (element with id .puck-item > a)
[0,7,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:20 84:[0,7,"WebDriver:FindElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/27 16:32:20 [1,7,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
Waiting for login field to load (element id email)
[0,8,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:20 60:[0,8,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:20 [1,8,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,9,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:21 60:[0,9,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:21 [1,9,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\ - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,10,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:22 61:[0,10,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:22 [1,10,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,11,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:24 61:[0,11,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:24 [1,11,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,12,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:25 61:[0,12,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:25 [1,12,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,13,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:26 61:[0,13,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:26 [1,13,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
[0,14,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:27 61:[0,14,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:27 [1,14,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
Finding login field (element with id email)
[0,15,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:28 61:[0,15,"WebDriver:FindElement",{"using":"id","value":"email"}]
2019/01/27 16:32:28 [1,15,{"error":"unknown command","message":"WebDriver:FindElement","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12 - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
Error: When finding login field by id email, got an error: WebDriver:FindElement
Login failed: When finding login field by id email, got an error: WebDriver:FindElement
[0,16,"Marionette:Quit",{"flags":["eForceQuit"]}]
2019/01/27 16:32:29 49:[0,16,"Marionette:Quit",{"flags":["eForceQuit"]}]
2019/01/27 16:32:29 [1,16,{"error":"unknown command","message":"Marionette:Quit","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nUnknownCommandError@chrome://marionette/content/error.js:444:5\nDispatcher.prototype.execute/req<@chrome://marionette/content/dispatcher.js:121:13\nTaskImpl_run@resource://gre/modules/Task.jsm:319:42\nTaskImpl@resource://gre/modules/Task.jsm:277:3\ncreateAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:252:14\nTask_spawn@resource://gre/modules/Task.jsm:166:12\nDisp - END - spatcher.prototype.onPacket@chrome://marionette/content/dispatcher.js:89:5\nDebuggerTransport.prototype._onJSONObjectReady/<@chrome://marionette/content/server.js -> resource://devtools/shared/transport/transport.js:483:11\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\nexports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:101:14\n"},null]
$
So, nest-scrape still can't successfully start up Firefox, but if it's already started, then it can use that session.
Perhaps there is something in there that gives you a clue as to what we can look at next.
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
It seems like we have two problems with nest-scrape now:
-
It can't start firefox properly.
-
It can't communicate properly with firefox when it's running.
So for problem #1, I believe nest-scrape should be able to start firefox from ssh (without a GUI) as long as -s is not specified, but in your original post it failed. Just to confirm I understand the situation, can you confirm the following?
a) Over ssh (i.e. from MobaXterm) if you run '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile' manually, does it start and remain running? That is, does it display "*** You are running in headless mode." and then retain control of the terminal (you don't get a prompt back) until you hit CTRL-C?
b) Over ssh, if you run 'nest-scrape -v 1' (without -s) is it able to connect after some time (maybe 1 minute)?
c) From the Raspbian GUI if you run 'nest-scrape -v 1' is it able to connect after some time (maybe 1 minute)?
d) From the Raspbian GUI if you run 'nest-scrape -v 1 -s' is it able to connect after some time (maybe 1 minute)?
Now for problem #2, I have a hunch that the firefox version from Raspbian (which seems to be 52.9.0) might have a different Marionette API than the version I used (64.0.2). The reason I think this is that the Go library I'm using tries to call the WebDriver:NewSession function in Firefox which fails with "unknown command", but then the library falls back to using an older call newSession which succeeds. Then later WebDriver:Navigate fails similarly... I suspect a number of functions got renamed in later versions.
I have a feeling that that version might be too old to support the Go library nest-scrape uses to talk Marionette with Firefox. The library seems to only test as early as firefox 56.0 (see the 'matix' line in this file):
https://github.com/njasm/marionette_client/blob/master/.travis.yml
Let me think about some possible options.
from nest-scrape.
meingraham
commented on August 13, 2024
a. MobaXterm
$ /usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile
1548645820820 Marionette INFO Listening on port 2828
No other output ever came after over 5 minutes. I hit Ctrl-C to stop it. Before doing that, I looked at running processes (in another terminal session):
$ ps -ax | grep fox
5439 pts/0 Sl+ 2:20 firefox-esr --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile
b. MobaXterm
$ nest-scrape -v 1
browser: running browser using cmd '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile'
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
1548646307702 Marionette INFO Listening on port 2828
Navigating to site
Checking if login is required
Waiting for thermostat location to load (element css selector .puck-item > a)
Finding thermostat location (element with id .puck-item > a)
Waiting for login field to load (element id email)
Finding login field (element with id email)
Error: When finding login field by id email, got an error: WebDriver:FindElement
Login failed: When finding login field by id email, got an error: WebDriver:FindElement
While it was starting up
$ ps -ax | grep fox
5507 pts/0 Rl+ 0:04 firefox-esr --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile
5509 pts/0 Z+ 0:03 [firefox-esr] <defunct>
The failure left behind the Firefox process. I killed it manually before going to the next test.
Obviously I did not give the startup enough time in the initial invocations yesterday. During my testing & debugging, I'm running this on a Pi ZeroW so it is slow... which explains why it takes a long time to start up. Good call on
connect after some time (maybe 1 minute)
Once I get this to work, I'll put nest-scrape on my RPi 3. I just wanted this testing on a separate system until I was ready to put it into "production".
c. & d. Raspian GUI Terminal
In both cases, a Firefox window opened. Both generated the same output as 'b' test above.
Thanks.
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Hi Mike,
Thanks for testing it out.
Regarding the firefox process that was left behind, I think there might be a bug where nest-scrape is not cleaning up resources on those failures. I'll look into that.
As for the Marionette API version problem (the WebDriver:FindElement error) I think I might be able to solve it by getting the source code for an older version of the Go Marionette library and compiling against that for Raspbian. When I look in the git history for the library it seems like at one point it supported Firefox version 52. I'll try that when I get an opportunity, and provide you a version of the source code you can try.
from nest-scrape.
meingraham
commented on August 13, 2024
Much appreciated!
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Hi Mike,
I've made a new branch of the code to support Firefox version 52. Could you update your clone of the source code to that branch and try it out? I'm assuming since you're running this on Raspbian you compiled this yourself using Go. So to get this new version you should just have to:
- cd into your go workspace directory (i.e. your $GOPATH)
- From there
cd src/github.com/jeffwilliams/nest-scrape - Do a
git pull - Checkout the branch:
git checkout firefox-52 - Get the modified version of the Go marionette library:
go get github.com/jeffwilliams/marionette_client - Build/Install the binary like before:
go install .
Let me know how it goes.
Also, I see what you mean about firefox still trying to open a window even with --headless specified. For some reason in version 52 it does that sometimes; in later versions it doesn't seem to do that.
from nest-scrape.
meingraham
commented on August 13, 2024
Jeff,
nest-scrape seems do be toiling a bit more... before it fails
The results look pretty much the same from both MobaxTerm as well as the Desktop Terminal. Of course, from the Terminal, a Firefox window does pop up.
One thing from watching the Firefox window is that it did start to load the home.nest.com page... but it did seems as though it did not completely load when nest-scrape exited. I could see messages in the browser status and the the page link in the URL bar... but the page had not rendered. Do you think that the fact that it's running on a very slow Pi ZeroW, and that the page takes a long time to load, that the nest-scrape Firefox (marionette?) calls are timing out? If so, is there a way to extend the timeout for those calls, at least for testing purposes. It could be a command line option so if one has a slow system, you tell it to give it more time?
Also, in every case, nest-scrape did clean up after itself nicely in that the Firefox process was gone a few seconds after nest-scrape exited (when the "Failed downloading XML" message was output.
$ nest-scrape -v 2
browser: running browser using cmd '/usr/bin/firefox --marionette --headless --profile /home/pi/.nest-scrape/firefox-profile'
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
...
dial tcp 127.0.0.1:2828: getsockopt: connection refused
browser: retrying connection in 1 second
1548729152910 Marionette INFO Listening on port 2828
[0,1,"newSession",{"capabilities":null,"sessionId":""}]
2019/01/28 21:32:59 55:[0,1,"newSession",{"capabilities":null,"sessionId":""}]
2019/01/28 21:32:59 [1,1,null,{"sessionId":"97966f14-1a8a-4ad9-b53f-28501ed0bbc6","capabilities":{"browserName":"firefox","browserVersion":"52.9.0","platformName":"linux","platformVersion":"4.14.79+","pageLoadStrategy":"normal","acceptInsecureCerts":false,"timeouts":{"implicit":0,"page load":300000,"script":30000},"rotatable":false,"specificationLevel":0,"moz:processID":1067,"moz:profile":"/home/pi/.nest-scrape/firefox-profile","moz:accessibilityChecks":false}}]
Navigating to site
[0,2,"get",{"url":"https://home.nest.com"}]
2019/01/28 21:33:10 43:[0,2,"get",{"url":"https://home.nest.com"}]
2019/01/28 21:33:10 [1,2,null,{}]
Checking if login is required
Waiting for thermostat location to load (element css selector .puck-item > a)
[0,3,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:13 74:[0,3,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:13 [1,3,{"error":"no such element","message":"Unable to locate element: .puck-item > a","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
[0,4,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:17 74:[0,4,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:17 [1,4,{"error":"no such element","message":"Unable to locate element: .puck-item > a","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
[0,5,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:21 74:[0,5,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:21 [1,5,{"error":"no such element","message":"Unable to locate element: .puck-item > a","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
Finding thermostat location (element with id .puck-item > a)
[0,6,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:24 74:[0,6,"findElement",{"using":"css selector","value":".puck-item \u003e a"}]
2019/01/28 21:33:24 [1,6,{"error":"no such element","message":"Unable to locate element: .puck-item > a","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
Waiting for login field to load (element id email)
[0,7,"findElement",{"using":"id","value":"email"}]
2019/01/28 21:33:38 50:[0,7,"findElement",{"using":"id","value":"email"}]
2019/01/28 21:33:38 [1,7,{"error":"no such element","message":"Unable to locate element: email","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
Finding login field (element with id email)
[0,8,"findElement",{"using":"id","value":"email"}]
2019/01/28 21:33:41 50:[0,8,"findElement",{"using":"id","value":"email"}]
2019/01/28 21:33:41 [1,8,{"error":"no such element","message":"Unable to locate element: email","stacktrace":"WebDriverError@chrome://marionette/content/error.js:200:5\nNoSuchElementError@chrome://marionette/content/error.js:388:5\nelement.find/</<@chrome://marionette/content/element.js:272:16\n"},null]
Error: When finding login field by id email, got an error: Unable to locate element: email
Login failed: When finding login field by id email, got an error: Unable to locate element: email
[0,9,"quitApplication",{"flags":"eForceQuit"}]
1548729221630 Marionette INFO New connections will no longer be accepted
2019/01/28 21:33:42 46:[0,9,"quitApplication",{"flags":"eForceQuit"}]
2019/01/28 21:33:42 [1,9,null,{}]
$
1548729101820 addons.productaddons WARN Failed downloading XML, status: 0, reason: timeout
Here is the log without the verbose output:
$ nest-scrape
Error: When finding login field by id email, got an error: Unable to locate element: email
Login failed: When finding login field by id email, got an error: Unable to locate element: email
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Hi Mike,
Yes, there is a setting for making the timeout higher:
-t, --timeout int
Number of seconds to wait until each page element loads (default 8)
Maybe try something high like 60? I.e:
nest-scrape -v 2 -t 60
You can see all the options by running nest-scrape --help.
from nest-scrape.
meingraham
commented on August 13, 2024
Duh! I'd seen that but I misread/misinterpreted it.
Well, timeouts are definitely causing issues. After a bit more testing, it's obvious that a Pi ZeroW is not suitable for this.
I went ahead and powered down one of my Pi 3B. I can do without it for a couple of days. Much better!
As before, the browser window definitely wants to render... even when I'm using MobaXterm. I inadvertently had my VNC client up and invoked nest-scrape from MobaXterm... and to my surprise, the Firefox window popped up. Anyhow...
The process has gotten through different stages on different test runs.
The Nest thermostat page renders
The credentials dialog loads
The credentials are filled in
The credentials are authenticated
The next page begins to load
After any of those, add "sometimes". And there does not seem to be a pattern as to when it will and when it won't. In fact, sometimes when the credentials authentication fails, I can type the password in manually and the process continues. Sometimes it tells me that the service is unavailable (I know I'm entering the right username/password). If I try again, it goes through fine. Sometimes when the "service is unavailable", I'll go to my PC and bring up Opera or IE and the nest thermostat page loads and lets me log in without issue.
Well, damn... it might have really been that the Nest service was unavailable - https://www.reddit.com/r/Nest/comments/al48qf/nest_down_500_and_504_error/?utm_source=reddit-android
I want to install this on one of my Windows machines just as a test. Is that possible? I installed Go and Firefox. nest-scrape --generate and nest-scrape --help do what is intended. I edited nest.yaml and specified the parameters with Windows in mind (e.g., double quotes around file paths that contain spaces).
When I try to execute nest-scrape -v 2, I guess it doesn't like the permissions on my nest.yaml file
Loading config failed: The permissions on the config file must not allow group or other any access
Is trying to run it on Windows a bridge too far?
Mike
from nest-scrape.
meingraham
commented on August 13, 2024
I tried just now (19:15 EST) and I can report that it was a success! Thanks! It looks like Nest is back online!
It still taxes an RPi 3 at times - CPU pegged at 100% for portions of the scrape.
Running it from an SSH session (i.e., MobaXterm) doesn't work. Even if my VNC client session is open and visible and the browser window can render. VNC is needed to allow the browser window to render. it's moot anyway. I've seen it error out, complete but not retrieve any actual measurements, just the JSON labels.
If I need VNC up and running, at that point. it kind of defeats the purpose of using MobaXterm since requiring VNC means I can just run nest-scrape from a Desktop Terminal. This non-headless thing with version 52.9.0 is unfortunate. I did see that Raspberry was working on getting Firefox 60 as the supported version. But no real timeline on that!
Seems to take about 3 minutes start to finish. I thought that if I told it to not kill the browser session (-s option) that it might save it some time by not having to open another session and window. It didn't affect it because running nest-scrape again just opened a new browser window (not an additional tab in the existing browser window).
This much I can probably already say - it's going to require a dedicated platform to scrape the data. I now need to figure out how to couple this with a mechanism to publish it over MQTT so my home automation system can capture the information and make use of it in a more flexible way than what Nest allows!
Curious - what are you doing with the data?
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Hi Mike,
I'm glad it's working somewhat better. Yes, that non-headless behavior is indeed a problem.
Are you still interested in a Windows version? I haven't tried it on Windows, but I think with a few tweaks it could probably run on it. That permissions error is just a flimsy attempt to make sure if you're running it on a multi-user system other users aren't able to read your password from that file.
Hmm, I'll see there's some way I can get it to push MQTT. If it helps, there is an option to change the output format to JSON.
For myself, I'm just running this on a Debian server from a cron job, and appending the output to a text file. We had an issue resolved under our home warranty related to a cold room and so I've been monitoring that room and a few other points to see how much of a temperature difference there is on cold days. Really nothing too fancy. I just ssh in and check the data to see how it's looking.
from nest-scrape.
meingraham
commented on August 13, 2024
Yeah, I'm using the JSON option.
Yes, I'd like to try it on Windows to see if that runs fast enough where I can run it in the scheduler and have it pump the JSON over MQTT to my automation hub. If you have a means to integrate MQTT publish, then problem solved. Just a matter of adding some MQTT configuration information (broker, credentials, topic).
Mike
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Ok, I've just pushed a commit that adds an option that disables nest-scrape from checking the permissions on the config file. So you can now add the option -p or --no-check-perms and it should work in Windows.
from nest-scrape.
meingraham
commented on August 13, 2024
Thanks. I'll give it a try... after I figure out why Firefox on my laptop crashes every time 😞 Nothing to do with nest-scrape - just won't run!
from nest-scrape.
meingraham
commented on August 13, 2024
I still have not been able to resolve the Firefox installation issues so I can't try nest-scrape on my Windows PC. In the interim, I purchased some Wi-Fi connected temperature sensors. I am now using those as my remote temperature inputs to the home automation routines I built. I wanted to re-use the Nest sensors as "unattached" sensors by scraping their readings. Unfortunately, Firefox on the Raspberry Pi is too resource "hungry" (as is Chrome, BTW). Without being able to successfully deploy Firefox on my PC and run nest-scrape there, I probably will just scrap my Nest sensors altogether now that I have an alternative. I guess I'll be putting my Nest 3-pack on eBay ;-)
Mike
P.S. Even if I had gotten it going on my PC, I would have still had to get MQTT communications going. My new sensors are MQTT enabled out of the box. They are roughly $30 each and monitor both temperature and humidity.
from nest-scrape.
jeffwilliams
commented on August 13, 2024
Hi Mike,
Sorry it didn't work out. Ideally there would be a simpler way to get these readings from the Nest temperature sensors. I hope the new sensors work out well.
from nest-scrape.
Related Issues (3)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nest-scrape.