Push notifications stop after ~24 hours about ring HOT 107 CLOSED

I've released 11.7.7 with some attempted fixes.

Thanks for all the insight @tsightler! My guess at this point is that there is something going on with the session, and that the bug with restarts has to do with the fact that a session is now being created (even if one already exists) immediately when the rest-client starts up. I'm also creating a new session every 12 hours after the pervious update, but that doesn't seem to have the same negative effect for some reason 😕.

As an attempted work around, I'm updating the camera class to watch for when new sessions are created and automatically re-subscribe to ding/motion regardless of the current subscriptions. I'm hoping this will fix the restart issues in the current version, but I can't reliably reproduce the issue to confirm this fixes anything 😅.

Regarding the double device issue I was able to get to the bottom of that! The problem stems from differing systemIds at the time of generating the initial refresh token, and the systemId used by homebridge at runtime. For homebridge-ui in particular, this meant ensuring that we use the same logic to generate a consistent systemId for both the initial setup via the homebridge-ui and when homebridge actually runs. Note, users creating their refresh token using ring-client-api auth cli or via HOOBS UI will still end up seeing two devices. I've released a fix for homebridge-ui users only. Authors for other libraries will need to look at how their systemId is being set both for initial refresh token and their main runtime to make sure it's consistent.

On a personal note, I have family in town this weekend and would very much like to focus on spending time with them. I understand this is a frustrating issue and want to get it resolved quickly, but I have to balance with real life 😄. If the latest release does not fix the issue, then I suggest users turn on notifications from the official Ring app in the mean time. I understand this doesn't help with automations via homebridge/mqtt, but it's the best I can recommend at this point. Ring broke something and it will take time for us to find a perfect fix. Thank you for you patience ❤️.

from ring.

Fully understand @dgreif. I'm personally spending some time with family after long weeks of travel for work, so not a ton of time myself, just kind of poking at it here and there.

from ring.

Ok, here we go 🤞. 11.8.0-beta.0 is released with a number of changes that should hopefully tackle all the notifications issues and the double device issue across all platforms.

For anyone updating, please follow the Notification Troubleshooting steps after updating - namely removing existing devices and creating a new Refresh Token

Per my previous comment, I updated the "refreshToken" to actually be a wrapper around an object, which includes the original refresh token needed for auth, the hardware id so that it stays consistent across refreshes (fix for double devices), and the entire Credentials object generated/used by the push notifications client. That last bit should be the ticket for preventing notification breakage after a restart.

I ran into some fun stuff with this change, in particular the fact that Ring will re-send recent push notifications when you reconnect if you don't set persistentIds in the push notification config. Rather than storing those ids, I'm just ignoring push notifications for the first two seconds after connecting. In local testing, I saw those come through on the scale of 15ms after connecting, so 2s should be plenty.

I kept this backward compatible with existing refresh tokens, so you don't technically have to create a new one, but it's a good idea to do so to avoid any push notification token overlap with the existing device session. Note, I did not make the backward compatible for downgrading, so if you try this beta and want to revert to the previous version, you will need to create another new refresh token after you downgrade. If all goes well, I'll probably release a breaking change in the near future which forces users to create a new refresh token to update.

Let me know how it goes!

from ring.

I spent some time comparing the behavior to traces of ring-client-api with the official Ring app taken with mitmproxy and I just can't figure out why it's impossible to get push notifications working again after a restart. It feels like Ring has perhaps changed the behavior of the API call to register the FCM token to not accept updated tokens for a given hardware device once it has been set initially (or perhaps doesn't allow it frequently).

I've noticed that both my real hardware device, and the emulated android device use the same FCM token every time they startup.

On the other hand, ring-client-api does not provide any method of storing and reusing existing GCM/FCM registration and effectively generalates a new device registration, keys and FCM token on every startup and tries to push this to Ring. I hacked up my local copy of ring-client-api to use the same saved GCM credentials/keys/FCM token, and indeed that works for surviving a restart. Note that push-receiver actually suggest that GCM credentials should be stored/preserved for re-use.

This also seems to be why removing the authorized device works, because that also removes the FCM token. When the new authorized device is created the property "has_push_notification_token" is set to false. Calling the client_devices API with the hardware ID and the updated push_notification_token changes this to "true", but, after that, it doesn't seem possible to call this API to set a new token.

Really strange, and I could be somehow wrong, but the code is pretty straightforward, just a single call to an API to update the FCM token. I see the call happen, but, after the first time the client is registered, it just seems like this call is silently ignored.

from ring.

Thanks for the responses everyone!

However, its clear that other users see this work only for about 24 hours. For whatever reason, I've noticed that a lot of people with the issue do not have a Ring subscription, but not everyone. Perhaps there's just a higher percentage of Ring users that don't have a subscription than I realized. I'm also not sure that anyone that has reported the issue to this point is a US resident, so I wonder if the problem is happening in other regions vs the US.

@tsightler you may have nailed the problem in this paragraph. I think the symptoms we are seeing likely stem from new Data Residency laws, which require data to only be stored within the region of the user. In some of the new laws, there is an acceptable period for data to be stored in another region, but only temporarily (something on the scale of 24-48 hours). If Ring is storing the session and push notification data in the US, it would make sense that they would only be able to store that same info for 24-48 hours for users in Spain/Germany.

Assuming this theory is correct, I'm guessing the apps are now configured to recreate the user session and push notification registration at least once a day. We should be able to do something similar from ring-client-api, re-posting these on a periodic basis. I haven't had a chance to try it out yet, but I think this should be pretty simple to change. I'll post back here as soon as I have some progress. I have a pretty relaxed weekend so I might be able to get to it tomorrow 🤞

from ring.

For users of ring-mqtt that may be monitoring this issue, I've updated the dev branch to include 11.7.6 of ring-client-api to make this easier to test. Simply change the "branch" config option from "addon" to "dev" and restart the addon/docker container and it will pull down the latest dev branch for testing purposes. If this works I'll push out a full update later this week.

from ring.

That's a fair question @McJohnPi. I'll track that down separately

from ring.

@dgreif I pushed out an update to ring-mqtt that uses the beta ring-client-api and, so far, things seem pretty good. Of course, it's not even been a full 24 hours yet, but at least I'm not flooded with issues from upgrade problems so far.

from ring.

Given the positive results so far, I feel confident we have a working solution. I've released 11.8.0 with the changes. I'll keep this issue open for a few more days for additional feedback in case users have continued notifications issues after updating.

from ring.

Great question @smockle! The hard part with mitmproxy is that the Ring App uses certificate pinning, which makes any MITM setup not work by default. There are ways around this, but they require a rooted device and a few pieces of software to get everything working. It's not a route I recommend others try to go down 😅.

Honestly I think this issue might be one that we can debug without necessarily seeing the traffic from the App. Based on the assessment from @tsightler in the other issue, I have a suspicion that the subscribed values are getting set to false after some period of time (24-48 hrs), so theoretically we should be able to take a token and hardware id belonging to a session from a few days ago and see what those values are. It might be as simple as watching those values and re-subscribing if they go false.

Another suspicion is that users experiencing this issue might be on a setup where they are not getting a consistent hardware id. This would present in the form of many "devices" showing up in the Control Center during that first 24 hour period. @salemsayed I'm curious, after you cleared your devices and waited a day, did you have just a few devices in Control Center, or was the list very long again?

from ring.

It might be as simple as watching those values and re-subscribing if they go false.

Unfortunately, the code in ring-mqtt already does this, at least it attempts to by calling subscribeToDingEvents()/subscribeToMotionEvents() when it detects this by checking the value on every poll event (20 seconds). The thing is, subscribing to the events doesn't appear to fail, but the observed behavior in the logs is that, once these values start showing up as false, they will revert back to unsubscribed every few minutes even after calling the subscribe function.

@smockle If you do decide to go down the path of mitmproxy, below is a post from a ring-mqtt user that did just that when they wanted me to implement a function that we didn't know how the API worked. It might help if you decide to go down that route, it didn't seem too hard.

tsightler/ring-mqtt#528 (reply in thread)

I actually want to set this up myself, but time just hasn't been available for me lately.

from ring.

Hi,

I use the ring-client-api in an app for the Homey Smart Home Platform (https://homey.app/) and there are quite some users of the app that are reporting the same exact problems with the notifications. The strange thing is that not all users seem to be affected by the bug, the app has ~5000 users but only a few compared to that amount have reported it.
For some the removal of the devices at Ring has been a solution for a couple of days now, for others it only lasted a day.
In my app the api connection is registered using the hardware ID of the Homey device, this is obviously unique per Homey and does not change. This results in 2 registrations at Ring, one with the name Homey (hardcoded in my app) and one with the name ring-client-api.
When the app is restarted and thus a new connection is made and notifications are registered again there are no new devices created at Ring. Restarting the app does not necessarily cause the notifications to stop working, for some it’s a problem for others it isn’t.
There are also users who lost notification that did not restart the app during the period it worked.

That’s all information I can add for now, I’m currently on vacation and have currently no way to do any troubleshooting myself. When I get more relevant information from my users I will post it here.

Edit: For anyone interrested, this is the code that sets up the connection with the Ring api:
https://github.com/daneedk/com.amazon.ring/blob/22b0d0596e1114a374b9ef57757d88ac971cbb0e/lib/Api.js#L97

from ring.

This problem occurs on both the old and new Homey. The old Homey uses nodejs v12.22.11 and the new Homey, which is really new and it’s firmware is still in Release Candidate phase, uses v18.16.0.

The old Homey has not been available in the US so no US based users for that one, the new Homey will be available in the US, but I’m not sure if any are shipped yet (I’ll try to get that confirmed)

The first report about this particular problem was posted on the Homey Community forum on the 10th of may. My app has a dedicated support topic on that forum, I will ask users that are experiencing the problem to answer the questions above.

from ring.

Sorry - yes that did it. Will report back if it stops again. Thank you!!

from ring.

I was wondering why the Ring plugin causes 2 new devices in the Ring Authorized device environment?

from ring.

@JeffGoldner Not an expert on Scrypted, but I'm pretty sure it uses a forked version of ring-client-api that re-implements polling. Also, there was a new version of the Scrypted Ring plugin pushed just two days ago that attempts to fix the polling interface for the Ring plugin in Scrypted. Are you using that version?

I'd be quite surprised if polling doesn't work, but I've seen other reports of this with the Home Assistant Ring addon as well, so I'm starting to be suspicious.

from ring.

@dgreif, thank you for the recent patch.

This has removed the double-ups I was seeing in the Ring web portal and I can confirm my Ring notifications are working again through Homebridge.

I will post an update after a few days too to report on what appeared to be a 24 hour bug.

Thanks again for making this plug-in a reality and I hope that you were able to have a great weekend.

from ring.

Thanks for the updates @tsightler. I've been thinking through how we can do this globally and I think I have a potential path forward. Basically, we need to ensure we use the same value across restarts for:

• hardware id
• gcm/fcm token

Given the variety of systems that consume ring-client-api, I'd prefer to have a solution that does not require storage to a specific file/location. Instead, I'm thinking about extending the information stored in the refreshToken string given consumers of ring-client-api already need to have a mechanism for subscribing to refresh token updates and storing that string somewhere. Basically, I'm thinking about changing the refresh token to be a string with structured data, allowing it to store the actual auth refresh token, the hardware id, and the push notification token. We should be able to do this in a backward compatible way so that we can transparently make this change without impacting existing usage patterns. Storing the hardware id in here would have the added bonus of fixing the double-device issue for all consumers as well, regardless of how the initial refresh token is created.

Not positive on the exact format yet, but it will probably be something like

{
  "rt": "e*** auth refresh token",
  "hid": "123***",
  "pnt": "gcm/fcm token"
}

All of that can be stringified and probably base64 encoded just to make it less likely to be tinkered with by users. WDYT @tsightler?

from ring.

Thank you so much @dgreif!
Just installed 11.8.0-beta.0.
Strange behavior so far: Homebridge shows functional Ring devices, button presses open the door (intercom), but HomeKit only shows inactive Ring devices. Will uninstall and install again. Maybe this is just a random thing.
All other Homebridge devices show up as active.

from ring.

The beta is rock solid so far.

from ring.

Working great on this end @dgreif thank you tons! Been over 24-hours and restarted server multiple times with no issues

from ring.

@tsightler I can confirm your latest ring-mqtt update has fixed my notifications issue and I have yet to see any other issue arise yet.

from ring.

For me it's working flawless, notifications (ding and motion) survived multiple homebridge reboots during the last days.

from ring.

from ring.

Correct, notifications worked for 1 or 2 days max then stopped also for my Ring Intercom

from ring.

We should investigate this further…

If someone was interested in conducting this investigation, what steps would they need to take? Setting up mitmproxy and watching app traffic for a day or so?

from ring.

Great question @smockle! The hard part with mitmproxy is that the Ring App uses certificate pinning, which makes any MITM setup not work by default. There are ways around this, but they require a rooted device and a few pieces of software to get everything @salemsayed I'm curious, after you cleared your devices and waited a day, did you have just a few devices in Control Center, or was the list very long again?

Great question @smockle! The hard part with mitmproxy is that the Ring App uses certificate pinning, which makes any MITM setup not work by default. There are ways around this, but they require a rooted device and a few pieces of software to get everything working. It's not a route I recommend others try to go down 😅.

Honestly I think this issue might be one that we can debug without necessarily seeing the traffic from the App. Based on the assessment from @tsightler in the other issue, I have a suspicion that the subscribed values are getting set to false after some period of time (24-48 hrs), so theoretically we should be able to take a token and hardware id belonging to a session from a few days ago and see what those values are. It might be as simple as watching those values and re-subscribing if they go false.

Another suspicion is that users experiencing this issue might be on a setup where they are not getting a consistent hardware id. This would present in the form of many "devices" showing up in the Control Center during that first 24 hour period. @salemsayed I'm curious, after you cleared your devices and waited a day, did you have just a few devices in Control Center, or was the list very long again?

Just two devices right now, an unnamed device and homebridge-ring device.

from ring.

Just two devices right now, an unnamed device and homebridge-ring device.

Same here. Have been deleting and re-authenticating the devices for days now. Each time it's just the two: unnamed and Homebridge ring.

from ring.

Thanks for the info @daneedk! That sounds in-line with the other experiences described so far.

I personally use the standard Ring App notifications so I wasn't sure if this was affecting me. Enabled notifications this morning and mine are working just fine, so I'm not able to repro/debug 😞.

It sounds like there is a lot of variance in who is experiencing this. For those having issues, what version of Node.js are you on? I'm on 18.16.0 without issues.

from ring.

@daneedk That sounds exactly the same as what I'm seeing from ring-mqtt users as well. I estimate at least 6000 users of ring-mqtt based on various analytics data (it's probably higher, but so far only a few sporadic reports. Almost all started around May 4th and it's the first time it ever impacted my setup. However, since resetting all authorized users, my setup works perfectly for two weeks now.

However, its clear that other users see this work only for about 24 hours. For whatever reason, I've noticed that a lot of people with the issue do not have a Ring subscription, but not everyone. Perhaps there's just a higher percentage of Ring users that don't have a subscription than I realized. I'm also not sure that anyone that has reported the issue to this point is a US resident, so I wonder if the problem is happening in other regions vs the US.

However, I think it would be really good if we could start collecting this basic information from users experiencing the 24 hour issue to see if there's anything common. My suggestions for questions:

1. When did you notice the problem?
2. Did you remove all device and re-authenticate, if so, did this work and for how long?
3. What devices do you have?
4. What region/country are you located in?
5. Do you have a ring subscription?

However, I'm also open to other suggestions. Not being able to reproduce the issue makes it particularly difficult to find any solution. One thing, at least one user has reported that the native Home Assistant Ring integration has the same problem, but it still uses the old polling method vs push notifications (as far as I know anyway).

from ring.

1. When did you notice the problem?
   First week of May
2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Six times so far. Between 16 and 36 hours.
3. What devices do you have?
   Ring Intercom, Ring Chime
4. What region/country are you located in?
   Germany
5. Do you have a ring subscription?
   No

from ring.

1. When did you notice the problem?
   First week of May
2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes. Between 16 and 24 hours with ding events
3. What devices do you have?
   Ring Doorbells (2).
4. What region/country are you located in?
   Europe/Spain
5. Do you have a ring subscription?
   No

from ring.

1. When did you notice the problem?
   First week of May
2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes, worked for around 48 hours, maybe a coincidence but it stopped after I had to restart Homebridge
3. What devices do you have?
   Ring intercom
4. What region/country are you located in?
   Germany
5. Do you have a ring subscription?
   No

from ring.

Hi, I’m having the same problem here in Canada.

When did you notice the problem?
A few weeks ago, can’t tell exactly.

Did you remove all device and re-authenticate, if so, did this work and for how long?
yes, 1-2 days

What devices do you have?
Doorbell pro and stickup cam

What region/country are you located in?
Canada

Do you have a ring subscription?
Yes

from ring.

@dgreif I think the theory of changes regarding data residency may make sense. Around the same time reports started of notification issues there was also another issue reported with ring-mqtt that I was pretty sure was indirectly related to the infrastructure changes. The ring-mqtt code polls the device health API every 5 minutes to update various attribute data. Around the same time notifications started having issues some users also reported an issue where this data was generating an error in ring-mqtt due to null data.

Investigating this issue, it appeared to only impact EU users. Querying the API still returned a result, but the JSON response mostly contained null values, especially for personally identifiable data like wifi network name. After about 24-48 hours everyone that reported the problem indicated that the behavior returned to normal. I had thought that maybe Ring was making changes to keep personally identifiable data, like wifi names, in-region.

Assuming this theory is correct, I'm guessing the apps are now configured to recreate the user session and push notification registration at least once a day.

One thing though, does this not happen on restart? So far users have reported restarting doesn't help.

from ring.

@dgreif @tsightler the theory about it impacting mostly EU users sounds plausible, most users of my app are in the European region. There are also some in Australia and New Zealand, I can contact a user from Australia and check with him.
Btw, here’s my info, I’m collecting the rest of the information from other users and will post that later.

1. When did you notice the problem?
   Around 12 may
2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes, on three different Homeys, it worked between 24-48 hours
3. What devices do you have?
   Ring Video Doorbell Pro 2, Ring Video Doorbell Pro, Stickup Cam, Indoor Cam
4. What region/country are you located in?
   EU, the Netherlands
5. Do you have a ring subscription?
   Yes

from ring.

Having the issue here as well. No dings and no motion events coming through to Homebridge. Ring App works. Node.js version is 18.16.0

1. When did you notice the problem?
   Beginning of May

2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes, worked for max 24 hours.

3. What devices do you have?
   Ring Pro 2 wired and Ring 2 battery doorbells

4. What region/country are you located in?
   EU, Germany

5. Do you have a ring subscription?
   Yes

from ring.

I've released v11.7.6 which recreates the user session every 12 hours, and attaches the most recent push notification token whenever the session is recreated. I'd appreciate help testing this since I can't verify it within the US! Anyone who updates, please let me know if notifications stop working around the 24 hour mark still, otherwise hopefully we will know this is working in 2-3 days 🤞

One thing though, does this not happen on restart? So far users have reported restarting doesn't help.

Great question! The original setup actually didn't create a session (even on restart) unless we got a 404 back on a standard data request. Now I'm creating a session before any data requests go out, as well as on a 12 hour interval.

from ring.

Well, I'm in the U.S., Michigan, and I have a Ring subscription. I've changed nothing since the Motion Detection on the cameras stopped getting to Homebridge.

Except for now I've updated the plugin.
There is still no change. Cameras still no longer pass Motion Detection to Homebridge. ☹️

None of these cameras are passing motion:

from ring.

@MoTechnicalities Did you follow the instructions of deleting all authorized clients and re-authenticating.

from ring.

The issue I was bringing attention to here: #1219 got closed as a duplicate of this post.

Just letting you know that the issue 1219 is not resolved after upgrading to the latest version 11.7.6, the button on the ringdoorbell is still not pushing any notifications to homekit

Thought you might like to know

from ring.

@benjackson1111 Same question, have you followed the steps to remove all devices and re-authenticate? Does this fix it for some period of time?

from ring.

👍 I deleted the Authenticated Devices and the Motion Sensors are back on line.😀

Will keep you posted. Thanks.

from ring.

Thanks for the new version @dgreif. Unfortunately I can’t test it yet as I’m still on vacation and won’t be home until saturday.

from ring.

I'm not in the US, updated to latest version and will report back after 24H

from ring.

I am in the UK, updated to the latest version, and will report back in 24H.

from ring.

@dgreif Thank you so much for your nice and fast work!

We have had the same problem here in the Netherlands.
Just did the update to v11.7.6.
But still the same results; no ring/push and motion notifications.
Thought that we had deleted all the existing devices but that was in the Ring master account...
Then we deleted all the devices from the separate homebridge ring (Shared access) account, made a new token and voila!
There was Homekit sound and motion again!
(The strange thing was that the Homekit live view was working well all the time).
We look forward to tomorrow to hear if it still works...
Thanks again for all your work, family is happy again! 👍

from ring.

I am in the US, have updated to the newest version of the plugin, and removed and re-added my Ring doorbell and I'm still not getting motion or doorbell ring alerts at all.

I'm also now getting this in my Homebridge logs:

[5/22/2023, 12:12:41 PM] [homebridge-ring] Failed to connect push notification receiver
[5/22/2023, 12:12:41 PM] [homebridge-ring] Error: GCM register has failed
at postRegister (/usr/local/lib/node_modules/homebridge-ring/node_modules/@eneris/push-receiver/src/gcm.ts:87:19)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at doRegister (/usr/local/lib/node_modules/homebridge-ring/node_modules/@eneris/push-receiver/src/gcm.ts:54:22)
at exports.default (/usr/local/lib/node_modules/homebridge-ring/node_modules/@eneris/push-receiver/src/gcm.ts:15:25)
at PushReceiver.register (/usr/local/lib/node_modules/homebridge-ring/node_modules/@eneris/push-receiver/src/client.ts:169:30)
at PushReceiver.connect (/usr/local/lib/node_modules/homebridge-ring/node_modules/@eneris/push-receiver/src/client.ts:122:36)
at RingApi.registerPushReceiver (/usr/local/lib/node_modules/homebridge-ring/node_modules/ring-client-api/lib/api.js:179:13)

from ring.

Hmm, this only works for me the first time I restart after deleting the authorized device. If I recycle my client again then it stops receiving push notifications again. Deleting the client device fixes the problem again.

from ring.

@djm181 you are right. Unfortunately same problem here; after a restart of Homebridge it stops receiving push notifications again...

from ring.

Sorry, unfortunately the problem persits with ring-mqtt / HA in docker with branch option to dev
I think it has been running for 18 hours, until the system was rebooted. Thereafter, neither ding nor motion events

from ring.

can confirm that v11.7.6. notifications survive 24 hours, but are killed by home bridge restart.

from ring.

I live in the US and have the same issue reported by other users, so the reference to the EU may be a red herring:

1. When did you notice the problem?
   Earlier in May

2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes, a number of times, it works for a period of between 24-48 hours.Also setup a a dedicated account to see if that helps, but same issue occurs.

3. What devices do you have?
   Ring Video Doorbell Pro 2, Ring Video Doorbell Pro, Ring Floodlight Cam Wired Pros x 2

4. What region/country are you located in?
   USA

5. Do you have a ring subscription?
   Yes

While this may not be relevant, I wanted to make sure that everyone was aware of this notice from Ring that is an indication of some changes taking place in their backend.

https://support.ring.com/hc/en-us/articles/360059615351-Ring-Desktop-App-will-be-Discontinued-in-2023

from ring.

Tonight I Restarted my entire Raspberry PI Server just to test things out. Here is the order:
I do not run the Ring plugin in a Child Bridge. It's been running since my last post up above.

1. It worked great until this [Restart Server] but returned to NO Motion Detections after the restart.
2. I deleted the Authenticated Unit (Android-Homebridge) from the iPhone Ring App and Refreshed the Homebridge Token. Still no detections!
3. I then did a [Quick Restart] (NOT a full [Restart Server]) and Motion Detection came back.

For other testers: I did not repeat the process a second time, but since it came back after the [Quick Restart] that followed the Token Refresh, It may be only goofing up after a full [Restart Server]. I'm not sure yet and will wait.

from ring.

i think i may have identified the issue

the motion sensors tied to the ring cams get stuck in an Updating status within homekit, which then stops notifications from coming through

any possible fix for this, if it is in fact the same issue

When did you notice the problem?
Earlier in May

Did you remove all device and re-authenticate, if so, did this work and for how long?
Yes, a number of times, it works for a period of time of a few hours

What devices do you have?
Ring Video Doorbell Pro, Ring Floodlight Cam Wired Pro

What region/country are you located in?
USA

Do you have a ring subscription?
Yes

from ring.

Firstly, @dgreif thank you for your continued efforts.

The refresh steps work for restoring notifications but only for around 24 hours.

As noted by others, I also see 2 authorised devices via Ring's control panel after reregistering Homebridge.

After that 24 hour period I start to receive a series of errors in the Homebridge log similar to this:
[5/25/2023, 9:11:25 PM] [Ring] Failed to reach Ring server at https://app.ring.com/api/v1/mode/location/. Timeout awaiting 'request' for 20000ms. Trying again in 5 seconds...

Whatever causes this timeout also seems to result in the network being blocked from connecting to anything ring.com related as all connections (apps and hardware) report being offline. To reset the IP block I need to disable the Ring plugin in Homebridge and wait between 30-60 minutes.

To answer the various questions:

1. When did you notice the problem?
   Earlier in May

2. Did you remove all device and re-authenticate, if so, did this work and for how long?
   Yes, around 24 hours each time. Have been testing over the last few days and seeing this consistently.

3. What devices do you have?
   Ring Video Doorbell (2nd edition), Ring chime

4. What region/country are you located in?
   Australia

5. Do you have a ring subscription?
   No

from ring.

The strange thing is that in our case the push notifications are working perfect after more than 48 hours. But we are punished horribly if we (accidentally) restart HomeBridge..
So, that's now forbidden.

from ring.

In my case, I automatically have HB restart every 24 hours for stability. So yes, I can confirm as well that it worked perfectly before a restart.

from ring.

When did you notice the problem?
Not sure

Did you remove all device and re-authenticate, if so, did this work and for how long?
Yes, but it doesn't really work. The first notification works, then it stops.

What devices do you have?
Ring Video Doorbell (2nd edition), Ring chime

What region/country are you located in?
US

Do you have a ring subscription?
No

from ring.

I have noticed that when I get a notification through scrypted, the Ring app shows that as "Answered Motion". Following that, subsequent motions show up as "Motion" in the event log, not "Answered Motion" - and I don't get Ring app notifications for these nor does anything get through to the Scrypted plug-in. And this seems to be the pattern each day: First motion of the day, I get "Answered Motion", Scrypted catches it, throws it to Homekit, life is as expected. Then nothing for the rest of the day as all subsequent motion events are showing "Motion". I believe this was also the case a few days ago when I was just using the homebridge-ring plug-in.

I have cleared all the crud from previous apps, just two authorized now, so they aren't piling up unless I do a new login, then I get two more (Unnamed app, ring-api).

from ring.

And it could very well be the case that restarting Scrypted allows the first notification, but that doesn't explain Ring app behavior.

from ring.

Already discussing with Koush. That update did one thing - it terminated motion after 30 seconds. That part does work, but otherwise it's subject to the same thing. He's watching this as well I believe.

from ring.

OK, but my point is that, unlike code built directly on ring-client-api here, that code uses both push notifications and polling as it subscribes to both onMotionDetected and onMotionDetectedPolling, the latter being included only in the Scrypted fork of ring-client-api. This implies that the problem might not be directly related to push notifications, but more broadly even with polling for dings, which is a little bit surprising for me.

from ring.

@dgreif I don't know if this will help you in any way, but I thought I would share it.

I am basically also seeing the behavior reported above, i.e. if I remove the account and re-authenticate, then notifications will work for the initial connection, and seem to survive mulitple days (worked from Sunday of last week until Saturday). However, if I restart ring-mqtt which uses the existing saved token, then push notifications will not work (well, they do seem to randomly work maybe one time out of 10, but mostly they do not work).

However, what I've discovered is that I don't have to completely re-authenticate to get them to work again. As noted above there are two "Authorized Client Devices" that show up with ring-client-api, one that has "Device name not found" and the other with whatever name the client uses ("ring-mqtt" or "ring-mqtt-addon" in the case of my project). This has of course been the case since the start of the project, and I've never really concerned myself with it, but I thought it was interesting.

What I've found is that, if I delete the authorized device with the registered name, but do not delete the one with "Device name not found", then I can restart ring-mqtt and authentication will still work with the token, and the second authorized client will be registered again with the name, but notifications will start working again for that session.

from ring.

Confirmed. 2 Authorized Devices for the job of one.

from ring.

Hello - since i had the same issue (#1229) here is what i experienced:
after updating to version homebridge-ring v11.7.7 and then following the instructions stated in #1218 (re-login / re-auth)
i can report now that push-notifications are working for me again! THANK YOU!

When did you notice the problem? ->can't remember
Did you remove all device and re-authenticate, if so, did this work and for how long? ->yes and now it works thank you
What devices do you have? ->intercom
What region/country are you located in? i'm european (austria)
Do you have a ring subscription? ->no

from ring.

@dgreif, many thanks for your great work!

Here are my findings in relation to using the ring-client-api in the Ring Doorbell and Camera app for Homey:

I've collected information from several users, this is the overall result:

When did you notice the problem?
1st to 2nd week of May
Did you remove all device and re-authenticate, if so, did this work and for how long?
Yes, between 24 - 36 hours (I think it's 24 as most users don't immediately notice it not working)
What devices do you have?
Miscellaneous Ring Doorbell types, Miscellaneous Camera types
What region/country are you located in?
UK, Belgium, Netherlands, Norway, Sweden, Germany, France, Spain, Italy, Poland, Australia
Do you have a ring subscription?
Several have, others don't. (I can't see a connection having a subscription or not)

Here's some results from test I've done with my Ring Doorbell and Camera app for Homey using the ring-client-api

I've been running v11.7.6 since Friday evening on a 2 Homeys, one with node v12.22.11 and another with V18.16.0, they both behave the same, notifications are still working since then (almost 48 hours right now)

I've also installed v11.7.6 on another Homey with node v12.22.11. On that one I stopped and started the app after 24 hours, this resulted in notifications no longer working.
Next I deleted the Homey entry from the Authorised Client Devices in the official Ring app and restarted the app on my Homey. This re-created the Homey entry plus the notifications started to work again, so that confirms @tsightler observation.
After that I updated my app to ring-client-api v11.7.7 and performed the same test (though not waiting 24 hours before restarting the app) and experienced the same, notifications stop but get working again after deleting Homey from the Authorised Client Devices list and restarting the app.

I've also done some other testing, I've removed both the Homey and Device name not found entries and temporarily disabled the actual connecting to the Ring API after authenticating (The app has a builtin UI for authenticating at Ring)

When I authenticated in the app, the Device name not found entry was created, the Homey entry was not. Next I enabled the code that actually connects tot the Ring API and when running that the Homey entry was immediately created and operation was as desired.
Next I deleted the Device name not found entry and restarted the Homey app, this resulted in a 401 and required me to authenticate again.
As far as I can see the Device name not found device is created at both grant_type: "password" and grant_type: "refresh_token" at https://oauth.ring.com/oauth/token. In my code I'm using the same hardware_id for both the authentication requests as the systemId I use when setting up the RingApi connection.
What I don't do in the authentication request is using (the equivalent of) controlCenterDisplayName which I use when setting up the RingApi (as I don't know how to integrate that in the authentication request) So I think that's connected to the fact 2 entries are created and not just one.

from ring.

I may be a little late with this but I am not receiving any notification no matter what I do. Last 2 updates since this issue started have made no difference.

When did you notice the problem?
2023-05-18
Did you remove all device and re-authenticate, if so, did this work and for how long?
Yes.
No notification at all since 2023-05-17.
Events are showing in Ring iOS App
What devices do you have?
Video Doorbell Pro 2
What region/country are you located in?
Australia
Do you have a ring subscription?
No

Homebridge.log from last notification:
[17/05/2023, 22:50:00] [Ring] Front Door Detected Motion. Loading snapshot before sending event to HomeKit
[18/05/2023, 06:31:17] [Ring] Request to https://app.ring.com/api/v1/mode/location/**** failed with status 500.
Response body:
[18/05/2023, 06:31:50] Registering platform 'homebridge-ring.Ring’
[18/05/2023, 06:31:59] [Ring] Initializing Ring platform…
[18/05/2023, 06:32:01] [Ring] Configuring cached accessory **** Front Door
[18/05/2023, 06:32:06] [Ring] Found the following locations:
[18/05/2023, 06:32:06] [Ring] locationId: ****
[18/05/2023, 06:32:07] [Ring] Request to https://app.ring.com/api/v1/mode/location/**** failed with status 503.
Response body: {"errors":[4007]}
[18/05/2023, 06:32:07] [Ring] Error connecting to API
[18/05/2023, 06:32:07] [Ring] HTTPError: Response code 503 (Service Unavailable)
[18/05/2023, 06:32:07] [Ring] Request to https://app.ring.com/api/v1/mode/location/**** failed with status 503.
Response body: {"errors":[4007]}
[18/05/2023, 06:32:07] [Ring] HTTPError: Response code 503 (Service Unavailable)
[18/05/2023, 06:32:33] [Ring] Request to https://app.ring.com/api/v1/mode/location/**** failed with status 500.
Response body:
[18/05/2023, 06:33:05] Registering platform 'homebridge-ring.Ring'
[18/05/2023, 06:33:15] [Ring] Initializing Ring platform...
[18/05/2023, 06:33:16] [Ring] Configuring cached accessory **** Front Door
[18/05/2023, 06:33:21] [Ring] Found the following locations:
[18/05/2023, 06:33:21] [Ring] locationId: ****
[18/05/2023, 06:33:22] [Ring] Configuring 1 cameras and 1 devices for location "****" - locationId: ****

Times are Australian central time (UTC +9:30).
From this point no further motion detections or door bell notification are in the log.

from ring.

@dgreif, just confirming the update is working for me.

It has removed the duplicate entry in the Ring web portal and notifications have been working through homebridge for the last few days without any issues. There is no lag either as I get a Ring motion alert from both the Ring app and Apple devices at the same time.

The API errors connecting to Ring servers via Homebridge have also cleared up.

Thanks again for your efforts on this.

from ring.

My Steps:

1. Installed homebridge-ring v11.7.7
2. In the Ring App, delete the Authenticated Device associated with Homebridge.
3. Refresh Token in the Plugin.
4. Restart Server
   Motion Detect is all good for a while. But then:
5. Restart Server
   No more Motion Detect again.😕
   

from ring.

My Steps:

1. Installed homebridge-ring v11.7.7
2. In the Ring App, delete the Authenticated Device associated with Homebridge.
3. Refresh Token in the Plugin.
4. Restart Server
   Motion Detect is all good for a while. But then:
5. Restart Server
   No more Motion Detect again.😕
   

Same on my end

from ring.

Hello,

Hello - since i had the same issue (#1229) here is what i experienced: after updating to version homebridge-ring v11.7.7 and then following the instructions stated in #1218 (re-login / re-auth) i can report now that push-notifications are working for me again! THANK YOU!

When did you notice the problem? ->can't remember Did you remove all device and re-authenticate, if so, did this work and for how long? ->yes and now it works thank you What devices do you have? ->intercom What region/country are you located in? i'm european (austria) Do you have a ring subscription? ->no

Hello,

unfortunatelly i have to tell you that push notifications (and automations for exposed doorbell switch device) have stopped working again.

Two days ago i updated to version homebridge-ring v11.7.7 and re-authenticated + removed 2 devices in ring app then it worked.

Now it isn't working again.

If you need any further information or logs or anything please dont hesitate to ask, i will try to provide as good as possible.

thanks for your efforts in advance.

from ring.

Same here. I removed and re-tokened the plugin again this morning and it worked for a couple of hours and then stopped. No motion triggers or doorbell events are being reported.

from ring.

I just started playing with this library a couple days ago and I haven't been able to get notifications except immediately after getting a new token. It doesn't seem to work for me at all after a token is refreshed. Everything else seems to work, even push notifications for settings updates, but motion or doorbell notifications don't work. I'm assuming my problem is the same as everyone else?

from ring.

Now I'm not getting motion or doorbell notifications even with a new token, so maybe I'm just doing something wrong. onData works fine, but nothing else ever gets called.

from ring.

We have installed version 11.7.7 at May 28 in the (European) morning.
Without a restart it's running flawless for 4 days now.
All the push notifications are working perfect.

[5/31/2023, 3:37:09 PM] [Ring] Front Door Detected Motion. Loading snapshot before sending event to HomeKit
[5/31/2023, 3:38:14 PM] [Ring] Front Door Detected Motion. Loading snapshot before sending event to HomeKit
[5/31/2023, 3:54:31 PM] [Ring] Front Door Detected Motion. Loading snapshot before sending event to HomeKit

from ring.

I think it's abundantly clear at this point that a restart will commonly break push notifications. For whatever reason, the Ring API seems to not update the FCM push token even though the API appears to be called correctly, it only works the first time. I have some ideas why this might be the case, but need to continue to experiment to attempt to prove them.

However, for now, the only way I've been able to get push notifications to survive a restart is to save the FCM token generated immediately after initial authentication. For ring-mqtt I've hacked up ring-client-api to expose the GCM/FCM authentication information and store in the state file and also to accept re-use of previous authentication during restart instead of generating an entirely new FCM token. This at least allows ring-mqtt to survive restarts.

For ring-mqtt users, if you want to test this code, you can use the dev branch right now (set "branch" option to "dev" on the configuration settings). Note that you will almost certainly need to remove the device from authorized client devices and re-authenticate one more time to generate a fresh FCM token but, after that, ring-mqtt will just re-use this token.

Note that I'm not saying this is a final fix, actually I'm quite sure it is not, but it should at least buy more time between needing to re-authenticate vs having to constantly do so for every restart.

from ring.

Thank you so much tsightler, just installed your addon using the dev branch and it immediately started working for me again. Experienced the same issues as all the others through homebridge plugin; and for some reason haven't been able to successfully get it back running even with all the steps explained above.
But with ring-mqtt addon at least I have my dings back now. Will report when there's any news.

from ring.

@dgrief I think that approach could potentially work. I've already coded up fixes for the duplicate device issue using your examples, so that's not a huge issue for ring-mqtt, but I'm open to however you'd like to handle it. One thing to note, I'm pretty sure you will need to store the entire push-receiver credentials object, not just the FCM token.

I still have lots of question marks around the entire GCM/FCM token issue. I don't understand if Ring is simply limiting the frequency at which it can be updated, or if they are doing something like immediately pushing a message and invalidating the token if the get back an UNREGISTERED error.

I'm assuming the GCM credentials and the FCM push token are not valid forever, so what will happen when they need to somehow be updated? What will happen if I pass GCM credentials that don't work? Right now push-receiver appears to just fail in that case, so I guess ring-client-api would need to detect this and retry with empty credentials again. There doesn't appear to be any method in push-receiver to get an updated FCM token after the initial GCM/FCM registration process as the only place I can find the onCredentialsChange is called is during initial login when no credentials initially provided. Maybe they don't change?

from ring.

All good questions! I think it's worth experimenting on and I'll dig through the code for push receiver some more to understand it better. Hopefully I can get a working prototype by this weekend 🤞

from ring.

I've also been digging a little bit with the Ring web based client since it basically just does the same thing. However, it appears to use an entirely different method to get an FCM token, registering with a Firebase application. The push-receiver code is using a very old (and seemingly largely undocumented) method that is still dependent on the long since deprecated C2DM registration process. However, I spend a little bit of time digging at the Chromium source code, and it's GCM support, and it still uses the same method as push-receiver, which is what it appears to have been reversed engineered from.

Maybe these token/credentials really don't expire. Looking at some old documentation, it seems to indicate that the tokens can change, but only for things like removing/reinstalling the app, deleting app data, or restoring a backup on a new device. There doesn't seem to be an expire, so maybe just reusing these forever is OK.

For now I've just hacked up ring-mqtt/ring-client-api so that people can easily test some code that re-uses the existing GCM/FCM credentials and, so far so good, but we'll see how it goes for a few days.

from ring.

Please let me know if this is the wrong place to ask this, but as a brand new user of ring-client-api, I can't tell if I'm doing something wrong or if I'm having the same problem as described by others here.

If I delete the client device and re-authenticate, my app (a simple nodejs application using ring-client-api) gets motion notifications just fine until I restart the app (and create a new RingApi). It doesn't matter if I pass the same refreshToken to the constructor that I got from the original auth request, or if I subscribe to onRefreshTokenUpdated and use the latest token returned.

Am I doing something wrong, or am I having the same issue as the others?

from ring.

@saltyzoo Yes, this is the same issue as reported here.

from ring.

Just reporting back that homebridge-ring v11.7.7 has been working fine for the past 4 days (since installation) with no visible troubles, except that (and I have to, out of spite, mention it) each Ring Light Bulb still to this day and age adds a fake Motion Sensor in HomeKit.
I have not Restarted the Server, yet, trying to see what comes first, the need for a Restart and Token Refresh or a new homebridge-ring version to try out.
Good thing homebridge-lutron-caseta-leap v2.4.5-beta1 is working great now or I probably would have been forced to Restart by now!

Also in the news today: Amazon Fined $30.8 Million Over Alexa and Ring Data Privacy Issues.

from ring.

Thanks @dgreif

I've just tested the new version in my Ring app for Homey but something is going wrong.
Everything is Ok at the first start of the app (after deleting the devices and logging on at Ring) but when I restart the Homey app I get the following error in my method to get a new refresh-token:

[log] 2023-06-03 12:12:51 [Api] Error: invalid_refresh 401 {"error":"invalid_grant","error_description":"The provided authorization grant (e.g., authorization code, resource owner credentials) or refresh token is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."}
    at IncomingMessage.<anonymous> (/lib/Api.js:408:29)
    at IncomingMessage.emit (node:events:538:35)
    at endReadableNT (node:internal/streams/readable:1345:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

This error originates from a method in my app, not the ring-client-api, I will look into that further and report back.

I don't think the error I'm getting is exemplary for the behaviour of the ring-client-api on other platforms though.

from ring.

First reboot after generating new token and deleting devices on Ring.com brings up the following log entries:

[6/3/2023, 1:10:51 PM] [homebridge-ring] Found the following locations:
[6/3/2023, 1:10:51 PM] [homebridge-ring]   locationId: [edited] - Rheinberg Zuhause
[6/3/2023, 1:10:52 PM] [homebridge-ring] 404 from endpoint https://api.ring.com/clients_api/doorbots/57832924/motions_subscribe
[6/3/2023, 1:10:52 PM] [homebridge-ring] Session hardware_id not found.  Creating a new session and trying again.
[6/3/2023, 1:10:52 PM] [homebridge-ring] 404 from endpoint https://api.ring.com/clients_api/doorbots/151854300/motions_subscribe
[6/3/2023, 1:10:52 PM] [homebridge-ring] Session hardware_id not found.  Creating a new session and trying again.
[6/3/2023, 1:10:52 PM] [homebridge-ring] 404 from endpoint https://api.ring.com/clients_api/doorbots/151854300/subscribe
[6/3/2023, 1:10:52 PM] [homebridge-ring] Session hardware_id not found.  Creating a new session and trying again.
[6/3/2023, 1:10:52 PM] [homebridge-ring] 404 from endpoint https://api.ring.com/clients_api/doorbots/57832924/subscribe
[6/3/2023, 1:10:52 PM] [homebridge-ring] Session hardware_id not found.  Creating a new session and trying again.
[6/3/2023, 1:10:52 PM] [homebridge-ring] Configuring 2 cameras and 3 devices for location "Rheinberg Zuhause" - locationId: [edited]

Maybe it is intended, I am not sure? Consecutive reboots of my Homebridge do not produce these entries. Will report back if I have collected information about the functionality of dings and motions sensors.

from ring.

Thanks for reporting in!

@jchristianj thanks for pointing it out, those 404 logs are normal after you create a new refresh token. Nothing to worry about there.

@daneek that error definitely seems related to the refresh token as that's what you would get if you had an invalid refresh token. I'm not able to reproduce any issues with it, so hopefully you can find something while debugging on your end 🤞.

@tammotheus that's an odd one, but likely some issue with communication between homebridge and homekit. I'd try rebooting all the things 😄

Side note, I'll be AFK much of the weekend so unfortunately I can't promise a quick turnaround on fixing anything with this beta. A few negative reports so far, but hopefully others are able to upgrade with less issues and try out the changes in regard to double devices and notifications after 24 hours/restarts.

from ring.

@dgreif many thanks for this fast solution. :-)
after updating to beta and reauth the intercom push is working again for me!!
i will keep an eye on it in the next days and report back if anything happens.

regards and have a good weekend

from ring.

It appears to be working for me as well on initial testing. Thanks so much. Great job!

from ring.

Is there anything else I can try? I have updated to the latest beta, removed the camera in the home app, re-added it, refreshed the token in the homebridge config....and its worse than before. At least I'd get it working for 24 hours, now the button on the doorbell doesn't trigger anything, even after following the instructions as above.

What could I be missing when everyone else is having success?! Sorry to the bearer of bad news!

from ring.

@benjackson1111 I would suggest simply following the staps again very carefully. As you state, others have shown that this method works, so my guess is that you likely did something out-of-order without realizing it. I'd suggest making sure that no ring-homebridge connection is made and don't try to re-authenticate until you are sure there are no devices listed in the Ring Control Center Authorized devices list. Some users have had to remove all devices to get it working, not just the ones associated with ring-homebridge/ring-client-api/ring-mqtt/etc. It's very critical that the authorized device created is 100% a new device or the push notifications will not be enabled.

from ring.

from ring.

@benjackson1111
I couldn’t get it to work at first so here’s the steps I took

1. Removed Ring child bridge from HomeKit
2. Deinstalled the plug-in in Homebridge and did a full restart of my raspberry pi
3. Removed every device in the ring control center, even the associated iOS devices so list was completely empty
4. Installed beta plugin in Homebridge, linked it to my ring account and set it as child bridge
5. Restart Homebridge via config interface
6. Added child bridge and its devices to HomeKit again
7. working now
8. logging into ring app on my iPhone and iPad again

from ring.

from ring.

@dgreif I've been able to make some changes to my Ring Doorbell and Cameras app for Homey to make it work with the 11.8.0-beta.0 version and it's behaving as it should 👍

I've just got one minor issue left, the authorisation (which is part of my app) still results in a device with the Device name not found name but it doesn't influence the workings of the app so I'll look into that later.

For now I've published a test version of my app to the Homey App Store and instructed my users in the Ring Doorbell and Camera topic on the Homey Community Forum on how to proceed.
I expect to see no problems as I thoroughly tested the new API over some days and multiple Homeys, if anything should arise I'll post it here.

Thanks again for your great work!

from ring.

Running 11.7.7

When did you notice the problem?
1st week of may
Did you remove all device and re-authenticate, if so, did this work and for how long?
Yes, also removed the child bridge and connect directly to each camera; stopped working after about 48 hours.
What devices do you have?
Floodlight Cam and Doorbell
What region/country are you located in?
USA
Do you have a ring subscription?
Yes

from ring.

Latest beta looks to be working fine here, passed the 24H with no issues, will keep testing and report back.

from ring.

@dgreif Thanks again! 11.8.0-beta.0 works fine so far after resetting/reinstalling the child bridge. Notifications come through over the past 3 days after daily reboots.

from ring.

Same, flawless since 11.8.0 beta - no notification issues for 5 days including multiple restarts. Thank you so much.

from ring.

Same, I’ve published a stable version of the Ring Doorbell and Camera app for Homey that uses 11.8.0-beta.0 so all users could benefit from it and no problems have been reported. Just reports how happy users are it’s working again.

Thanks for your effort!

from ring.