[FR] Add Zone support from Frigate about double-take HOT 24 CLOSED

Just wanted to let you know that configuration with zones is working fine.
As it's putting less load on Frigate only looking at zones I am finding Frigate doesn't crash so much.

Not done anything with MQTT yet, but the topics look ok and make sense.

from double-take.

Hey! Definitely seems possible, I could add another environment variable you could pass in to specify which cameras and zones you want to watch. Maybe something like FRIGATE_ZONES: camera-name:zone-name, drive:drive_zone_0?

from double-take.

Yea that sounds an easy enough environment to specify.

That way when Frigate detects a person entering the zone, the image gets sent to Deepstack.
Also by this point the person would be facing straight into the camera, so would result in a better image quality / detection. As it wouldn't be passing a side on image, which can happen if it responds just before the person turns into the drive.

from double-take.

Hey! Definitely seems possible, I could add another environment variable you could pass in to specify which cameras and zones you want to watch. Maybe something like FRIGATE_ZONES: camera-name:zone-name, drive:drive_zone_0?

exactly, Id like to set up a notification/TTS for when a match is found for my frigate zone 'front_porch'

from double-take.

@wills106 @deputyd0ng, I pushed a new beta build to include the changes we talked about. The README shows the new options that can be passed to the container.

Match results are also now published to MQTT in this format: double-take/cameras/:camera-name. Did you have any opinions on how you want the values stored for that topic? Right now it's an array of objects for each person found in the image.

Let me know what you think of these changes and if there's anything you'd like to see changed.

from double-take.

@wills106 @deputyd0ng, I pushed a new beta build to include the changes we talked about. The README shows the new options that can be passed to the container.

Match results are also now published to MQTT in this format: double-take/cameras/:camera-name. Did you have any opinions on how you want the values stored for that topic? Right now it's an array of objects for each person found in the image.

Let me know what you think of these changes and if there's anything you'd like to see changed.

Working on this now. Im trying to make a node-red entity, similar to the example that was created, I have the node just called "front door face"

I am using essentially the same node from the example, but ti doesnt seem to be working, what would be the best way to parse out any match from double-take/cameras/front_door topic? I thought just using the parsed JSON object would work, but it aint.

And thanks again! If I was more competent with this stuff Im sure It would be even more useful for me.

from double-take.

@deputyd0ng does your node look something like this? The output of this should be an array of objects, each one represents a matched person. Can you post a screenshot of what your entire Node-Red flow looks like? I can try to help debug.

from double-take.

Do I need to set my QOS to 0 (no idea what that is)

This is my second node, basically the same as the example:

from double-take.

@deputyd0ng Yes, try to set the QOS to 0. I'd also put a debug node after the topic to make sure you're getting the data. The data for the camera topic is an array, the entity node you have above I believe will only work if you subscribe to just the notifications for a specific person. If you only care about the first face of the results you can probably just replace payload with payload[0] (not 100% sure if this will work, but I don't see why not).

I dropped payload.room from the current beta build since it was just a manipulated string from the camera string. Since this is a front door HA entity, you'll probably want to set the state of this node to be payload[0].match.name so HA will show the most recent face it found.

from double-take.

@deputyd0ng I added better documentation and some samples for HA + Node-Red

from double-take.

Hi @jakowenko, i am using the beta docker build and a config.yml file trying to configure multiple zones per frigate camera....the docs show the following format;

  zones:
    - camera: driveway
      name: zone-1

This for me results in -> zones: [ [Object] ]

Adding this way;

  zones:
    - homeparkingfull0:zone_drive

Results in -> zones: [ 'homeparkingfull0:zone_drive' ] in the log :)

Adding;

  zones:
    - camera1:zone1
    - camera1:zone2
    - camera1:zone3
    - camera1:zone4
    - camera2:zone1
    - camera2:zone2

Appears to populate the zones correctly in the log, but goes against the docs.

I wondered what the correct way (if supported) is of stipulating multiples zones per camera.

Thanks

from double-take.

Hey @mannp, the docs should be correct even though the logs show Object. I'll work on correcting this so it's readable. Maybe I'll just move it to a page on the UI. The issue right now is that it's too nested for the logs to show the values. But if you follow the docs it should work.

Let me know if you are still running into issues and I'll let you know when I correct the output to show the value.

from double-take.

Thanks for that @jakowenko :)

So multiple zones per camera is multiple name: zone.x lines under the camera definition?

from double-take.

Multiple zones for the same camera should look like this. I'm on mobile, sorry if formatting is off.

zones:
  - camera: driveway
    name: zone-1
  - camera: driveway
    name: zone-2

from double-take.

Hi @jakowenko I have that setup now, although nothing showing in doubletake as yet.

It seems frigate crashes alot with doubletake for some reason.

Maybe I'll just move it to a page on the UI.

Level of work permitting this would be great, for a noob, it is hard to tell if doubletake has what it needs to work and all connections are okay etc. There are no errors in the log, but nothing showing up in doubletakes gui right now.

Had a few frigate events but nothing showing in the doubletake log that the event is being processed....will give it some more time to see if things imrpove.

from double-take.

@mannp can you post your config and I can see if everything looks right.

There does seem to be some issues around Frigate snapshot images. There were some open issues on Frigate's repo around connections not being closed. You should try setting frigate.attempts.snapshot to 0 to see if that resolves your issue. I've been running my current setup like that for 2 weeks now without any Frigate crashes.

You can also hit the API directly to force an image to show up in the GUI to make sure everything seems like it's working. I have it documented here.

from double-take.

@jakowenko this is my current config, but getting errors in the log around the zone definition, see below.

Thanks for any insight as to my issues 👍🏻

See last log where doubletake is picking up camera definitions as zones?

Edit: Ignore the yi-1 duplicate in the first log, as that was my error and should have been hv-1

- camera: yi-1
      name: zone_keepclear

frigate:
  url: http://192.168.30.60:5000
  zones:
    - camera: homeparkingfull0
      name: zone_drive
    - camera: yi-1
      name: zone_keepclear
    - camera: yi-1
      name: zone_roaddoor
    - camera: yi-1
      name: zone_keepclear
    - camera: hv-1
      name: zone_drive
    - camera: hv-2
      name: zone_ourpath  

Initial log with deepstack enabled and my error in the zone definition

deepstack process error: Cannot read property 'map' of undefined
latest attempt 2
1621161143.562615-gh3zbl - homeparkingfull0 zone not on approved list
deepstack process error: Cannot read property 'map' of undefined
1621161168.762443-pak4xe - homeparkingfull0 zone not on approved list
1621161170.911035-l1bq1g - yi-1 zone not on approved list
1621161185.484552-ctckc4 - yi-1 zone not on approved list
latest attempt 3
deepstack process error: Cannot read property 'map' of undefined
1621161178.197109-rmzndh - homeparkingfull0 zone not on approved list
latest attempt 4
deepstack process error: Cannot read property 'map' of undefined
1621161170.887676-5pfwgv - still processing previous request
1621161156.707334-6eitm0 - still processing previous request
latest attempt 5
deepstack process error: Cannot read property 'map' of undefined
1621161171.34417-ptct30 - homeparkingfull0 zone not on approved list
latest attempt 6
deepstack process error: Cannot read property 'map' of undefined
1621161171.34417-ptct30 - homeparkingfull0 zone not on approved list
latest attempt 7
deepstack process error: Cannot read property 'map' of undefined
1621161143.562615-gh3zbl - homeparkingfull0 zone not on approved list
latest attempt 8
deepstack process error: Cannot read property 'map' of undefined
1621161178.197109-rmzndh - homeparkingfull0 zone not on approved list
1621161185.484552-ctckc4 - yi-1 zone not on approved list
latest attempt 9
deepstack process error: Cannot read property 'map' of undefined
latest attempt 10
deepstack process error: Cannot read property 'map' of undefined

:
{
id: '1621161170.887676-5pfwgv',
duration: 8.79,
timestamp: '5/16/2021, 11:33:12 AM',
attempts: 10,
camera: 'hv-1',
zones: [ 'zone_drive' ],
matches: []
}
5/16/2021, 11:33:12 AM
done processing hv-1: 1621161170.887676-5pfwgv in 8.79 sec
1621161185.484552-ctckc4 - yi-1 zone not on approved list
1621161170.911035-l1bq1g - yi-1 zone not on approved list
1621161206.354517-r26dt5 - homeparkingfull0 zone not on approved list
1621161206.354517-r26dt5 - homeparkingfull0 zone not on approved list
1621161206.354517-r26dt5 - homeparkingfull0 zone not on approved list

Second log with deepstack disabled and the zone definition corrected...I am still seeing zone errors though, will check them again...

done processing hv-1: 1621161821.151975-1kwf5m in 7.93 sec
1621161815.965575-0ip2p7 - homeparkingfull0 zone not on approved list
1621161821.151975-1kwf5m - hv-1 zone not on approved list
1621161823.716671-vp36dh - skip processing on end events
1621162003.636007-mmy11r - label not a person, car found
1621162004.008706-672sla - label not a person, car found
1621162004.110347-og5zjv - yi-1 zone not on approved list
1621162005.962307-01vxsw - homeparkingfull0 zone not on approved list
1621162006.346672-mccnyv - homeparkingfull0 zone not on approved list
1621162004.008706-672sla - label not a person, car found
1621162004.110347-og5zjv - yi-1 zone not on approved list
1621162005.962307-01vxsw - homeparkingfull0 zone not on approved list
1621162005.962307-01vxsw - homeparkingfull0 zone not on approved list
1621162004.110347-og5zjv - yi-1 zone not on approved list
1621162004.008706-672sla - skip processing on end events
1621162006.346672-mccnyv - homeparkingfull0 zone not on approved list
1621162003.636007-mmy11r - skip processing on end events
1621162006.346672-mccnyv - homeparkingfull0 zone not on approved list
1621162071.531892-ae0hy4 - yi-1 zone not on approved list
1621162071.531892-ae0hy4 - yi-1 zone not on approved list
1621162084.350896-uo6gi9 - homeparkingfull0 zone not on approved list
1621162084.350896-uo6gi9 - homeparkingfull0 zone not on approved list

from double-take.

@mannp the config looks ok. I did just push a new beta build that includes a config route you can hit via the API until I add it to the UI. I also fixed the logs to show the full config object.

In terms of the logs you posted, it appears DeepStack is having issues, or is not sending back the expected response. It's related to this error, deepstack process error: Cannot read property 'map' of undefined. Try restarting DeepStack and see if that fixes it.

Try hitting the API manually to see if you can get an image to show up on the UI.

curl -X GET "http://localhost:3000/api/recognize?url=https://jakowenko.com/img/david.92f395c6.jpg" \
-H "Content-type: application/json"

I'm also curious what you think about the config.yml setup for the cameras / zones. I wonder if it can be improved and be more straightforward with something like this. In the below example, only Frigate events would be processed if they came from the backyard in zone deck or yard and any events from the driveway camera. Having this in two objects right now may be a bit more confusing and I like the idea of trying to combine them.

frigate:
  url: http://192.168.1.2:4000
  cameras:
    - name: backyard
      zones:
        - deck
        - yard
    - name: driveway

Same idea, slightly different format.

frigate:
  url: http://192.168.1.2:4000
  cameras:
    backyard:
      zones:
        - deck
        - yard
      driveway:

I think either of these are better than what I currently have. What do you think?

from double-take.

@jakowenko the new beta worked better for me, with the same config I got pictures and attempts at matches in the doubletake gui.... some car hub caps recognised as faces, but at least some progress thanks :)

Edit: After a full day of activity in the log I have no more activity in the gui....seems hit and miss for me, not sure why or how to fix it... :-/

My log now shows the following;

zones: [
{ camera: 'homeparkingfull0', name: 'zone_drive' },
{ camera: 'yi-1', name: 'zone_keepclear' },
{ camera: 'yi-1', name: 'zone_roaddoor' },
{ camera: 'hv-1', name: 'zone_keepclear' },
{ camera: 'hv-1', name: 'zone_drive' },
{ camera: 'hv-2', name: 'zone_ourpath' }

Personally the log would make more sense if 'name' was replaced by 'zone' and so I am leaning more towards the latter config format, just keeping things simple (obviously 'driveway' needs to be indented to the left one, just a github formatting think i guess).

Happy to create another issue, rather than post here, but re deepstack, its log is as follows;

[GIN] 2021/05/17 - 10:29:51 | 403 | 29.025µs | 192.168.30.69 | POST /v1/vision/face/recognize
[GIN] 2021/05/17 - 10:29:53 | 403 | 33.722µs | 192.168.30.69 | POST /v1/vision/face/recognize

The IP is correct for an api call from doubletake, so no issues there, but perhaps newer versions of deepstack return slightly different responses or the api call changed? Isn't there supposed to be a file url in the POST call..

DeepStack: Version 2021.02.1

Also the new config api call also works for me :)

from double-take.

Further testing and using a previous face image from frigate, the logs seem better but still nothing further in the gui.

curl -X GET "http://192.168.30.69:3000/api/recognize?url=http://192.168.30.60:5000/clips/homeparkingfull0-1621245633.860043-hwr6z9.jpg&results=all"

MQTT: connected
MQTT: subscribed to frigate/events
5/17/2021, 5:05:28 PM
processing double-take: 51c9f669-3aa8-4c4b-bdff-68f3a49cf19e
deepstack process error: Cannot read property 'map' of undefined
5/17/2021, 5:05:28 PM
processing double-take: 51c9f669-3aa8-4c4b-bdff-68f3a49cf19e
deepstack process error: Cannot read property 'map' of undefined

:
{
id: '51c9f669-3aa8-4c4b-bdff-68f3a49cf19e',
duration: 2.26,
timestamp: '5/17/2021, 5:05:30 PM',
attempts: 1,
camera: 'double-take',
zones: [],
matches: [],
results: [
{
duration: 2.26,
type: 'manual',
attempts: 1,
results: [
detector: 'deepstack',
duration: 0,
attempt: 1,
results: [],
filename: 'babc2ebd-b3e8-4261-8bc8-c16517975d76.jpg'
},
detector: 'facebox',
duration: 2.25,
attempt: 1,
results: [],
filename: 'babc2ebd-b3e8-4261-8bc8-c16517975d76.jpg'
]
}
]
}
5/17/2021, 5:05:30 PM
done processing double-take: 51c9f669-3aa8-4c4b-bdff-68f3a49cf19e in 2.26 sec
{
id: '51c9f669-3aa8-4c4b-bdff-68f3a49cf19e',
duration: 2.26,
timestamp: '5/17/2021, 5:05:30 PM',
attempts: 1,
camera: 'double-take',
zones: [],
matches: [],
results: [
{
duration: 2.26,
type: 'manual',
attempts: 1,
results: [
detector: 'deepstack',
duration: 0,
attempt: 1,
results: [],
filename: 'babc2ebd-b3e8-4261-8bc8-c16517975d76.jpg'
},
detector: 'facebox',
duration: 2.25,
attempt: 1,
results: [],
filename: 'babc2ebd-b3e8-4261-8bc8-c16517975d76.jpg'

from double-take.

@mannp you do have vision face and detection setup for your DeepStack container, right? It should look something like this.

  deepstack:
    container_name: deepstack
    image: deepquestai/deepstack
    restart: unless-stopped
    environment:
      VISION-FACE: 'True'
      VISION-DETECTION: 'True'

from double-take.

@jakowenko I am not using any env variables, since I have only starting setting up with the config.yml file using the beta.

Oops for deepstack, let me check...

Edit: So that explains the deepstack issue, as I now have this is the deepstack log

Edit: Facepalm .. apologies for that noise...

DeepStack: Version 2021.02.1
/v1/vision/face
/v1/vision/face/recognize
/v1/vision/face/register
/v1/vision/face/match
/v1/vision/face/list
/v1/vision/face/delete
/v1/vision/detection
v1/restore
[GIN] 2021/05/17 - 17:55:47 | 500 | 1m0s | 192.168.30.69 | POST /v1/vision/face/recognize

The manual recognize doesn't get added to the gui though.

Not sure what has changed since the flurry of images in the gui first thing today...

from double-take.

Did the deepstack process error: Cannot read property 'map' of undefined go away once you rebuild the DeepStack container with the env vars? We should be getting close now that DeepStack is setup correctly.

Feel free to message me on Discord if that's easier, my name should be Jako#4153.

from double-take.

Did the deepstack process error: Cannot read property 'map' of undefined go away once you rebuild the DeepStack container with the env vars? We should be getting close now that DeepStack is setup correctly.

No the error hasn't gone away unfortunately :) but I haven't had it enabled for 99% of the testing above, so its really a nice to have for me.

I have been using facebox most of the time alone.

Feel free to message me on Discord if that's easier, my name should be Jako#4153.

Done and thanks :)

from double-take.