zoneminder / mlapi Goto Github PK

View Code? Open in Web Editor NEW

56.0 10.0 34.0 1.63 MB

An easy to use/extend object recognition API you can locally install. Python+Flask. Also works with ZMES!

License: Other

Python 85.52% Shell 14.48%

machine-learning python3 flask api-gateway object-detection face-detection api api-server zoneminder cctv

mlapi's Introduction

ZoneMinder

All documentation for ZoneMinder is now online at https://zoneminder.readthedocs.org

Overview

ZoneMinder is an integrated set of applications which provide a complete surveillance solution allowing capture, analysis, recording and monitoring of any CCTV or security cameras attached to a Linux based machine. It is designed to run on distributions which support the Video For Linux (V4L) interface and has been tested with video cameras attached to BTTV cards, various USB cameras and also supports most IP network cameras.

Contacting the Development Team

Before creating an issue in our github forum, please read our posting rules: https://github.com/ZoneMinder/ZoneMinder/wiki/Github-Posting-Rules

Our Dockerfile has moved

Please file issues against the ZoneMinder Dockerfile here: https://github.com/ZoneMinder/zmdockerfiles

Installation Methods

Install from a Package Repository

This is the recommended method to install ZoneMinder onto your system. ZoneMinder packages are maintained for the following distros:

Ubuntu via Isaac Connor's PPA
Debian from their default repository
RHEL/CentOS and clones via RPM Fusion
Fedora via RPM Fusion
OpenSuse via third party repository
Mageia from their default repository
Arch via the AUR
Gentoo via Portage Overlays

If a repository that hosts ZoneMinder packages is not available for your distro, then you are encouraged to build your own package, rather than build from source. While each distro is different in ways that set it apart from all the others, they are often similar enough to allow you to adapt another distro's package building instructions to your own.

Building from Source is Discouraged

Historically, installing ZoneMinder onto your system required building from source code by issuing the traditional configure, make, make install commands. To get ZoneMinder to build, all of its dependencies had to be determined and installed beforehand. Init and logrotate scripts had to be manually copied into place following the build. Optional packages such as jscalendar and Cambozola had to be manually installed. Uninstalls could leave stale files around, which could cause problems during an upgrade. Speaking of upgrades, when it comes time to upgrade all these manual steps must be repeated again.

Better methods exist today that do much of this for you. The current development team, along with other volunteers, have taken great strides in providing the resources necessary to avoid building from source.

Building a ZoneMinder Package

Building ZoneMinder into a package is not any harder than building from source. As a matter of fact, if you have successfully built ZoneMinder from source in the past, then you may find these steps to be easier.

When building a package, it is best to do this work in a separate environment, dedicated to development purposes. This could be as simple as creating a virtual machine, using Docker, or using mock. All it takes is one “Oops” to regret doing this work on your production server.

Lastly, if you desire to build a development snapshot from the master branch, it is recommended you first build your package using an official release of ZoneMinder. This will help identify whether any problems you may encounter are caused by the build process or is a new issue in the master branch.

Please visit our ReadtheDocs site for distro specific instructions.

Package Maintainers

Many of the ZoneMinder configuration variable default values are not configurable at build time through autotools or cmake. A new tool called zmeditconfigdata.sh has been added to allow package maintainers to manipulate any variable stored in ConfigData.pm without patching the source.

For example, let's say I have created a new ZoneMinder package that contains the cambozola javascript file. However, by default cambozola support is turned off. To fix that, add this to the packaging script:

./utils/zmeditconfigdata.sh ZM_OPT_CAMBOZOLA yes

Note that zmeditconfigdata.sh is intended to be called, from the root build folder, prior to running cmake or configure.

Docker

Docker is a system to run applications inside isolated containers. ZoneMinder, and the ZM webserver, will run using the Dockerfile contained in this repository. However, there is still work needed to ensure that the main ZM features work properly and are documented.

Contribution Model and Development

Source hosted at GitHub
Report issues at GitHub Issues
Questions/feature requests in Slack or forums

Pull requests are very welcome! If you would like to contribute, please follow the following steps. While step 3 is optional, it is preferred.

Fork the repo
Open an issue at our GitHub Issues Tracker. Follow the issue template to describe the bug or security issue you found. Please note feature requests or questions should be posted in our user forum or Slack channel.
Create your feature branch (git checkout -b 456-my-new-feature)
Commit your changes (git commit -am 'Added some feature') It is preferred that you 'commit early and often' instead of bunching all changes into a single commit.
Push your branch to your fork on github (git push origin 456-my-new-feature)
Create new Pull Request
The team will then review, discuss and hopefully merge your changes.

mlapi's People

Contributors

Stargazers

Watchers

mlapi's Issues

mlapi does not behave as ES does

I have different behaviour from mlapi than from the ES server directly.
Running a test on an event reported by mlapi I have the following :
[s] detected:person,car--SPLIT--{"labels": ["person", "car"], "boxes": [[156, 251, 288, 515], [203, 3, 441, 93]], "frame_id": "snapshot", "confidences": [0.9882749915122986, 0.8627339005470276], "image_dimensions": {"original": [480, 640], "resized": [600, 800]}}

and log from zmesdetect shows :
zmesdetect_m10[65922] INF zm_detect.py:327 [Connecting with ZM APIs]
02/26/21 18:24:14 zmesdetect_m10[65922] INF zm_detect.py:48 [Detecting using remote API Gateway http://192.168.1.178:5002/api/v1]
02/26/21 18:24:17 zmesdetect_m10[65922] INF zm_detect.py:495 [Prediction string:[s] detected:person,car]

That matches what mlapi is sending :
[DBG 3] full_image intersects object:person[[(22, 390), (240, 390), (240, 584), (22, 584)]] Feb 26 2021 18:27:16.582004 [DBG 3] breaking out of same model loop, as matches found and strategy is "first" Feb 26 2021 18:27:16.582369 [DBG 2] pyzm_uid1000_cpu_lock portalock already released Feb 26 2021 18:27:16.582394 [DBG 1] perf: TOTAL detection sequence (with image loads) took: 2074.66 ms to process 622971 Feb 26 2021 18:27:16.582806 [DBG 1] Returning {'matched_data': {'boxes': [[156, 251, 288, 515], [203, 3, 441, 93]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9882749915122986, 0.8627339005470276], 'frame_id': 'snapshot', 'image_dimensions': {'original': (480, 640), 'resized': (600, 800)}, 'image': None}, 'all_matches': [{'frame_id': 'snapshot', 'boxes': [[156, 251, 288, 515], [203, 3, 441, 93]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9882749915122986, 0.8627339005470276], 'models': ['object']}, {'frame_id': 'alarm', 'boxes': [[203, 3, 453, 93], [22, 390, 240, 584]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7241158485412598, 0.6331931948661804], 'models': ['object']}], 'polygons': []}

However, this is not the desired behaviour, nothing (car or person) should be reported. And when run directly on the ES server (not using mlapi) the output is :
zmesdetect_m10[66169] INF zm_detect.py:279 [---------| app:6.1.12, pyzm:0.3.25, ES:6.1.12 , OpenCV:4.5.1|------------] 02/26/21 18:29:59 zmesdetect_m10[66169] INF zm_detect.py:304 [Importing local classes for Object/Face] 02/26/21 18:29:59 zmesdetect_m10[66169] INF zm_detect.py:327 [Connecting with ZM APIs] 02/26/21 18:30:01 zmesdetect_m10[66169] INF detect_sequence.py:282 [object:car at POLYGON ((162 3, 364 3, 364 75, 162 75, 162 3)) does not fall into any polygons, removing...]
which is the correct beahviour. 'car' is found but not inside the defined polygon, so nothing reported.

It seems it is not 'correctly' passed on to mlapi.
I tried to set the exact same monitor definition in mlapiconfig.ini on the mlapi server, but it throws an error analyzing the same event :
Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__ return self.wsgi_app(environ, start_response) File "/usr/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app response = self.handle_exception(e) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router return original_handler(e) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router return original_handler(e) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 468, in wrapper resp = resource(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask/views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 583, in dispatch_request resp = meth(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper return fn(*args, **kwargs) File "/home/x/.mlapi/mlapi.py", line 229, in post matched_data,all_matches = m.detect_stream(stream=stream, options=stream_options, ml_overrides=ml_overrides) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 466, in detect_stream _b,_l,_c, _e = self._filter_patterns(seq,_b,_l,_c, polygons) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 246, in _filter_patterns polygons[:] = self._rescale_polygons(polygons, neww / oldw, newh / oldh) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 199, in _rescale_polygons for x, y in p['value']: KeyError: 'value'
I've re-installed mlapi server again, no better.
I don't understand why, when leaving mlapiconfig.ini with no monitor definition it 'works' in the sense that it does not crash (albeit reporting an event when it should not) and when setting the monitor there, it crashes..
Can you help?

Face detection issues/turn on debug

I have ES configured with MLAPI to run face,object on the YOLOV4 model. I have faces trained and I see it looping through in the output but nothing is recognized or unrecognized.

How do I turned on DEBUG for this

installing python3-edgetpu breaks the coral detection

Hi, I've previously installed mlapi+coral in vm, for one reason or another I've ended up not using it, now I'm planning to use it again and installing it on a physical computer it appears some thing changed in regards to python3-edgetpu as installing it breaks the coral detection test.

I've installed all the prerequisites and got the coral detection working doing the parrot test:

$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.4ms
3.8ms
3.8ms
3.9ms
3.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.77734

then when installing python3-edgetpu I've noticed it removed libedgetpu1-std and replaced it and spat out a warning:

$ sudo apt-get install python3-edgetpu
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libedgetpu1-legacy-std
The following packages will be REMOVED:
  libedgetpu1-std
The following NEW packages will be installed:
  libedgetpu1-legacy-std python3-edgetpu
0 upgraded, 2 newly installed, 1 to remove and 0 not upgraded.
Need to get 0 B/3,374 kB of archives.
After this operation, 11.2 MB of additional disk space will be used.
Do you want to continue? [Y/n]
(Reading database ... 126795 files and directories currently installed.)
Removing libedgetpu1-std:amd64 (15.0) ...
Selecting previously unselected package libedgetpu1-legacy-std:amd64.
(Reading database ... 126789 files and directories currently installed.)
Preparing to unpack .../libedgetpu1-legacy-std_15.0_amd64.deb ...
Unpacking libedgetpu1-legacy-std:amd64 (15.0) ...
Selecting previously unselected package python3-edgetpu.
Preparing to unpack .../python3-edgetpu_15.0_amd64.deb ...
Unpacking python3-edgetpu (15.0) ...
Setting up libedgetpu1-legacy-std:amd64 (15.0) ...
Setting up python3-edgetpu (15.0) ...

      =========WARNING========
      python3-edgetpu package is deprecated. In order to maintain compatibility
      with libedgetpu, we've created a legacy version that will not receive any
      future updates; it exists only to support python3-edgetpu. At this moment,
      python3-edgetpu depends on libedgetpu1-legacy-std, which runs the
      Edge TPU at its reduced operating frequency; if you want to use
      the Edge TPU at its max operating frequency, you can do:
      sudo apt install libedgetpu1-legacy-max
      For more information, see g.co/coral/python3-edgetpu.
      =========================
Processing triggers for libc-bin (2.31-0ubuntu9.1) ...

after which the parrot test breaks:

$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
Traceback (most recent call last):
  File "classify_image.py", line 122, in <module>
    main()
  File "classify_image.py", line 100, in main
    interpreter.allocate_tensors()
  File "/home/sammael/.local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 259, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Internal: Unsupported data type in custom op handler: 45494944Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

If I install libedgetpu1-std the parrot test works again, but doing so removes python3-edgetpu

I'm unsure how to proceed now. Do I ignore the broken parrot test and proceed with compiling opencv and installing mlapi?

TypeError: 'float' object is not subscriptable

I put object_min_confidence=0.38 into one of the monitor sections in mlapiconfig.ini, and mlapi.py crashed with a TypeError on line 132, where it formats the debugging information.

g.logger.Debug(2, 'Overriding global {} with {}...'.format(key, g.monitor_config[mid][key][:30]))

Unable to run mlapi as a service. Something to do with @jwt_required()

I was running mlapi manually (not a system service) and having a problem with errors reported in zoneminder log. After some research I found someone with a similar problem found that there was some change to flask-jwt-extended version 4.0.0+? There was a recommendation to change @jwt_required to @jwt_required() vimalloc/flask-jwt-extended#175

So I edited mlapi.py and changed
class Detect(Resource):
@jwt_required

to
class Detect(Resource):
@jwt_required()

at that point object detection started working great and I could see from the logs output to screen by mlapi that it was processing as expected. Happy days, I thought that I fixed the problem.

So I went ahead and tried to install mlapi as a service and unfortunately the service will not start with the @jwt_required()

So I next changed mlapi.py back to the original, without (). With the original mlapi.py it will start and run as a system service.
However, unfortunately this puts me back in the starting condition where I get strange errors in zoneminder log such as shown below. Changing back to @jwt_required() and running manually, not as a system service works. Sorry I am not much of a python programmer so I don't really know what is going on or how to fix this. Any help getting to this work as a system service would be appreciated. The system service runs as www-data, when I run it with the extra () I use sudo and run it as root. So I don't know if maybe that has anything to do with it at all.

2021-04-11 15:21:18	zmesdetect_m10	115518	ERR	Error during remote post: 422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	158
2021-04-11 15:21:18	zmesdetect_m10	115518	ERR	Error with remote mlapi:422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	390
2021-04-11 15:21:18	zmesdetect_m10	115518	FAT	Unrecoverable error:'NoneType' object is not subscriptable Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 547, in main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 430, in main_handler 'labels': matched_data['labels'],TypeError: 'NoneType' object is not subscriptable	zm_detect.py	552
2021-04-11 15:20:32	zmesdetect_m11	115483	FAT	Unrecoverable error:'NoneType' object is not subscriptable Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 547, in main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 430, in main_handler 'labels': matched_data['labels'],TypeError: 'NoneType' object is not subscriptable	zm_detect.py	552
2021-04-11 15:20:32	zmesdetect_m11	115483	ERR	Error with remote mlapi:422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	390
2021-04-11 15:20:32	zmesdetect_m11	115483	ERR	Error during remote post: 422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	158
2021-04-11 15:20:20	zmesdetect_m10	115463	ERR	Error during remote post: 422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	158
2021-04-11 15:20:20	zmesdetect_m10	115463	ERR	Error with remote mlapi:422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	390
2021-04-11 15:20:20	zmesdetect_m10	115463	FAT	Unrecoverable error:'NoneType' object is not subscriptable Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 547, in main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 430, in main_handler 'labels': matched_data['labels'],TypeError: 'NoneType' object is not subscriptable	zm_detect.py	552
2021-04-11 15:19:19	zmesdetect_m10	115430	FAT	Unrecoverable error:'NoneType' object is not subscriptable Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 547, in main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 430, in main_handler 'labels': matched_data['labels'],TypeError: 'NoneType' object is not subscriptable	zm_detect.py	552
2021-04-11 15:19:19	zmesdetect_m10	115430	ERR	Error with remote mlapi:422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	390
2021-04-11 15:19:19	zmesdetect_m10	115430	ERR	Error during remote post: 422 Client Error: UNPROCESSABLE ENTITY for url: http://192.168.2.4:5000/api/v1/detect/object?type=object&delete=True&response_format=zm_detect	zm_detect.py	158

Support ML and stream sequences

Allow chaining of ML models, arbitrary frame analysis

ValueError: malformed node or string: <_ast.Name object at 0x7f0e4cec2390>

Updated the eventserver to 6.1.7 and it seems to run fine, tried in manuel mode.

Updated mlapi and hitting this:

[root@ryzen mlapi]# python3 ./mlapi.py -c mlapiconfig.ini
Jan 12 2021 19:23:15.548093 [DBG 1] secret filename: ./secrets.ini
Jan 12 2021 19:23:15.548496 [DBG 1] Secret token found in config: !MLAPI_SECRET_KEY
Jan 12 2021 19:23:15.548767 [DBG 1] Secret token found in config: !PLATEREC_ALPR_KEY
Jan 12 2021 19:23:15.548854 [DBG 4] Finally, doing parameter substitution
Jan 12 2021 19:23:15.549052 [DBG 4] substitution key: tpu_object_weights_mobiledet not found
Jan 12 2021 19:23:15.549072 [DBG 4] substitution key: tpu_object_labels not found
Jan 12 2021 19:23:15.549085 [DBG 4] substitution key: tpu_min_confidence not found
Jan 12 2021 19:23:15.549097 [DBG 4] substitution key: tpu_object_framework not found
Jan 12 2021 19:23:15.549241 [DBG 4] substitution key: tpu_object_weights_mobiledet not found
Jan 12 2021 19:23:15.549256 [DBG 4] substitution key: tpu_object_labels not found
Jan 12 2021 19:23:15.549267 [DBG 4] substitution key: tpu_min_confidence not found
Jan 12 2021 19:23:15.549278 [DBG 4] substitution key: tpu_object_framework not found
Jan 12 2021 19:23:15.549302 [DBG 4] substitution key: tpu_object_weights_mobiledet not found
Jan 12 2021 19:23:15.549314 [DBG 4] substitution key: tpu_object_labels not found
Jan 12 2021 19:23:15.549324 [DBG 4] substitution key: tpu_min_confidence not found
Jan 12 2021 19:23:15.549335 [DBG 4] substitution key: tpu_object_framework not found
Jan 12 2021 19:23:15.551693 [DBG 1] Opening DB at ./db/db.json
Jan 12 2021 19:23:15.551876 [DBG 1] DB engine ready
Jan 12 2021 19:23:15.554448 [DBG 2] API SSL certificate check has been disbled
Jan 12 2021 19:23:15.554499 [DBG 1] using username/password for login
Jan 12 2021 19:23:15.668480 [DBG 2] Using new token API
Jan 12 2021 19:23:15.668547 [DBG 4] make_request called with url=https://zoneminder.localdomain/zm/api/host/gettimezone.json payload={} type=get query={'token': None}
Jan 12 2021 19:23:15.752241 [DBG 2] using ml_sequence
Traceback (most recent call last):
  File "./mlapi.py", line 303, in <module>
    ml_options = ast.literal_eval(ml_options)
  File "/usr/lib64/python3.6/ast.py", line 85, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib64/python3.6/ast.py", line 66, in _convert
    in zip(node.keys, node.values))
  File "/usr/lib64/python3.6/ast.py", line 65, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/lib64/python3.6/ast.py", line 66, in _convert
    in zip(node.keys, node.values))
  File "/usr/lib64/python3.6/ast.py", line 65, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/lib64/python3.6/ast.py", line 61, in _convert
    return list(map(_convert, node.elts))
  File "/usr/lib64/python3.6/ast.py", line 66, in _convert
    in zip(node.keys, node.values))
  File "/usr/lib64/python3.6/ast.py", line 65, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/lib64/python3.6/ast.py", line 63, in _convert
    return set(map(_convert, node.elts))
  File "/usr/lib64/python3.6/ast.py", line 63, in _convert
    return set(map(_convert, node.elts))
  File "/usr/lib64/python3.6/ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x7f0e4cec2390>
[root@ryzen mlapi]#

Any suggestions? Did I miss something obvious?

WORKER TIMEOUT, Booting worker

I am trying to use an mlapi server for face detection. Objection detection works fine, but face detection fails.

$ gunicorn mlapi:app 
[2020-06-13 17:52:02 +0000] [1048] [INFO] Starting gunicorn 20.0.4
[2020-06-13 17:52:02 +0000] [1048] [INFO] Listening at: http://0.0.0.0:5001 (1048)
[2020-06-13 17:52:02 +0000] [1048] [INFO] Using worker: sync
[2020-06-13 17:52:02 +0000] [1050] [INFO] Booting worker with pid: 1050
[2020-06-13 17:52:02 +0000] [1051] [INFO] Booting worker with pid: 1051
Initializing log
Initializing log
DEBUG: secret filename: ./secrets.ini
DEBUG: Secret token found in config: !MLAPI_SECRET_KEY
INFO: --------| mlapi version: 1.0.7 |--------
DEBUG: Opening DB at ./db/db.json
DEBUG: DB engine ready
DEBUG: secret filename: ./secrets.ini
DEBUG: Secret token found in config: !MLAPI_SECRET_KEY
INFO: --------| mlapi version: 1.0.7 |--------
DEBUG: Opening DB at ./db/db.json
DEBUG: DB engine ready
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0
DEBUG: trained file not found, reading from images and doing training...
DEBUG: trained file not found, reading from images and doing training...
ERROR: No known faces found to train, encoding file not created
ERROR: No known faces found to train, encoding file not created
ERROR: Error loading face recognition file: [Errno 2] No such file or directory: './known_faces/faces.dat'
ERROR: Error loading face recognition file: [Errno 2] No such file or directory: './known_faces/faces.dat'
DEBUG: Object Recognition requested
DEBUG: get_file returned: ./images/659d7566-c10f-44df-88e1-aa8ff83ffde0.jpg
DEBUG: Initializing Yolo
DEBUG: config:./models/yolov3/yolov3.cfg, weights:./models/yolov3/yolov3.weights
DEBUG: YOLO initialization (loading model from disk) took: 537.807 milliseconds
DEBUG: Setting CUDA backend for OpenCV. If you did not set your CUDA_ARCH_BIN correctly during OpenCV compilation, you will get errors during detection related to invalid device/make_policy
DEBUG: YOLO detection took: 969.331 milliseconds
DEBUG: YOLO NMS filtering took: 0.979 milliseconds
INFO: object:person at [744, 265, 934, 601] has a acceptable confidence:0.9830182194709778 compared to min confidence of: 0.4, adding
INFO: object:car at [954, 46, 1182, 138] has a acceptable confidence:0.9492545127868652 compared to min confidence of: 0.4, adding
INFO: object:bicycle at [948, 227, 1048, 323] has a acceptable confidence:0.8091424107551575 compared to min confidence of: 0.4, adding
INFO: object:pottedplant at [9, 139, 49, 209] has a acceptable confidence:0.5524479746818542 compared to min confidence of: 0.4, adding
INFO: object:pottedplant at [34, 128, 98, 222] has a acceptable confidence:0.5505369901657104 compared to min confidence of: 0.4, adding
INFO: object:pottedplant at [79, 182, 121, 246] has a acceptable confidence:0.5200017690658569 compared to min confidence of: 0.4, adding
INFO: rejecting object:pottedplant at [2, 99, 24, 227] because its confidence is :0.3217852711677551 compared to min confidence of: 0.4
INFO: rejecting object:car at [38, 557, 562, 671] because its confidence is :0.31937599182128906 compared to min confidence of: 0.4
INFO: rejecting object:pottedplant at [74, 136, 132, 242] because its confidence is :0.27401870489120483 compared to min confidence of: 0.4
INFO: rejecting object:pottedplant at [318, 116, 372, 224] because its confidence is :0.22889778017997742 compared to min confidence of: 0.4
DEBUG: Face Recognition requested
DEBUG: get_file returned: ./images/564d23eb-b85d-4188-8c29-95f9c22e9b00.jpg
DEBUG: |---------- Face recognition (input image: 1200w*675h) ----------|
[2020-06-13 17:52:47 +0000] [1048] [CRITICAL] WORKER TIMEOUT (pid:1051)
[2020-06-13 17:52:49 +0000] [1238] [INFO] Booting worker with pid: 1238
Initializing log
DEBUG: secret filename: ./secrets.ini
DEBUG: Secret token found in config: !MLAPI_SECRET_KEY
INFO: --------| mlapi version: 1.0.7 |--------
DEBUG: Opening DB at ./db/db.json
DEBUG: DB engine ready
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0
DEBUG: trained file not found, reading from images and doing training...
ERROR: No known faces found to train, encoding file not created
ERROR: Error loading face recognition file: [Errno 2] No such file or directory: './known_faces/faces.dat'

Issue with "Get an Access Token"

I've followed your instructions up until the "Get an Access Token", and where I'm stuck is the whole example of "locahost:5000"

What do you mean by that? What should I put there? My ZoneMinder https url? or something else?

Sorry for the confusion.

ModuleNotFoundError: No module named 'cv2.cv2'

Recent system (Arch) upgrade (I guess mostly the python 3.8 -> 3.9 part) and mlapi is broken :
Traceback (most recent call last):
File "/home/x/.mlapi/mlapi.py", line 16, in <module>
import cv2
File "/usr/lib/python3.9/site-packages/cv2/__init__.py", line 3, in <module>
from .cv2 import *
ModuleNotFoundError: No module named 'cv2.cv2'

Tried to reinstall most of the python modules but cant figure out this one. I tried to install opencv2 but build fails.
Also tried to re-install mlapi from git clone, same error.
Any idea?

Error running model: invalid index to scalar variable.

Running mlapi 2.2.23 with opencv 4.5.4.
Following error on object detection:

`Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Setting up signal handler for logs]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Switching global logger to ZMLog]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Switched to ZM logs]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Reading config from: /var/lib/zmeventnotification/mlapiconfig.ini]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Reading secrets from: /etc/zm/secrets.ini]

Nov 3 13:25:41 UNRAID web_php[813]: 11/03/21 13:25:41.182655 web_php[813].INF [172.10.0.1] [Login successful for user "juan"] at /usr/share/zoneminder/www/api/app/Controller/AppController.php line 86

Nov 3 13:25:41 UNRAID web_php[813]: 11/03/21 13:25:41.185551 web_php[813].INF [172.10.0.1] [Creating token for "juan"] at /usr/share/zoneminder/www/api/app/Controller/HostController.php line 157

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [--------| mlapi version:2.2.23, pyzm version:0.3.55 |--------]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Using bjoern as WSGI server]

Nov 3 13:26:41 UNRAID /mlapi.py[1219]: ERR [zm_mlapi] [Error running model: invalid index to scalar variable.]

Nov 3 13:26:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Removing matches to past detections for monitor:6]`

If YoloV4 is set to 'enabled': 'no' in mlapiconfig.ini error disapears.

This error is not present with opencv 4.5.3

Sending multiple requests in parallel messes up detection

I use a single instance of face and object recognition which has a single instance of the network inside it. When multiple detect() functions are called together, the network variable will change messing up all the layers, resulting in bogus detections. The reason I use a single variable is to avoid re-loading the network on every request (very slow operation).

I need to look at threads or queues to handle this

Training known faces

There seems to be no way to train known faces. The code 'modules/face_train.py' seems to exist, but an example on how to invoke this will be helpful.

dont getting gpu support running on jetson xavier agx

Hi I installed everything as discribed and followed this guide: https://www.pyimagesearch.com/2020/02/03/how-to-use-opencvs-dnn-module-with-nvidia-gpus-cuda-and-cudnn/ to install opencv.
All his exaples work with gpu support after this.
Even older yolo exaples could I get running with gpu support after changing the code as discribed.
The part:...

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

print(cv2.getBuildInformation())

General configuration for OpenCV 4.4.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /xavier_ssd/opencv_contrib/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2020-08-28T16:39:08Z
    Host:                        Linux 4.9.140-tegra aarch64
    CMake:                       3.10.2
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    NEON FP16
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 7.5.0)
    C++ flags (Release):         -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static -lpthread dl rt nppc nppial nppicc nppicom nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda-10.2/lib64 -L/usr/lib/aarch64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cvv hdf java js julia matlab ovis python2 sfm viz
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         YES

  GUI:
    GTK+:                        YES (ver 3.22.30)
      GThread :                  YES (ver 2.56.4)
      GtkGlExt:                  NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
    JPEG 2000:                   build Jasper (ver 1.900.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      YES (2.2.5)
    FFMPEG:                      YES
      avcodec:                   YES (57.107.100)
      avformat:                  YES (57.83.100)
      avutil:                    YES (55.78.100)
      swscale:                   YES (4.8.100)
      avresample:                NO
    GStreamer:                   YES (1.14.5)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.4)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 10.2, CUFFT CUBLAS FAST_MATH)
    NVIDIA GPU arch:             53 62 72
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.0.0)

  OpenCL:                        YES (no extra features)
    Include path:                /xavier_ssd/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.6.9)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python3.6/dist-packages/cv2/python-3.6

  Python (for build):            /usr/bin/python3

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

however mlapi always start just with cpu support:

Aug 28 2020 20:29:44.544842 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.545016 [DBG 1] Initializing face recognition with model:hog upsample:1, jitters:0
Aug 28 2020 20:29:44.545139 [DBG 1] trained file not found, reading from images and doing training...
Aug 28 2020 20:29:44.545189 [DBG 1] If you are using a GPU and run out of memory, do the training using zm_train_faces.py. In this case, other models like yolo may already take up a lot of GPU memory
Aug 28 2020 20:29:44.545260 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.545339 [DBG 1] Face Recognition library load time took: 0.003 milliseconds
Aug 28 2020 20:29:44.545485 [ERR] No known faces found to train, encoding file not created
Aug 28 2020 20:29:44.545549 [DBG 1] Face Recognition training took: 0.151 milliseconds
Aug 28 2020 20:29:44.545633 [ERR] Error loading KNN model: [Errno 2] No such file or directory: './known_faces/faces.dat'
Aug 28 2020 20:29:44.545722 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.546516 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.546611 [DBG 1] Using CPU for detection
Aug 28 2020 20:29:44.546660 [DBG 1] Initializing Yolo
Aug 28 2020 20:29:44.546704 [DBG 2] config:./models/yolov3/yolov3.cfg, weights:./models/yolov3/yolov3.weights
Aug 28 2020 20:29:44.547058 [DBG 2] Semaphore: max:1, name:pyzm_cpu_lock, timeout:120
Aug 28 2020 20:29:44.547121 [DBG 1] Waiting for cpu lock...
Aug 28 2020 20:29:44.547501 [DBG 1] Got cpu lock for initialization...
Aug 28 2020 20:29:44.721449 [DBG 1] init lock released
Aug 28 2020 20:29:44.722410 [DBG 1] YOLO initialization (loading model from disk) took: 174.803 milliseconds
Aug 28 2020 20:29:44.722548 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.722660 [INF] Using simple log output (default)
Aug 28 2020 20:29:44.722730 [DBG 1] PlateRecognizer ALPR initialized with url: https://api.platerecognizer.com/v1
INFO: --------| mlapi version:2.0.0 |--------
INFO: Starting server with max:1 processes
 * Serving Flask app "mlapi" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

Any suggestions what I could to to enable gpu support?

Secrets file is required, not optional

The provided example mlapiconfig.ini says that the secrets file is optional:

[general]
# This is an optional file
# If specified, you can specify tokens with secret values in that file
# and onlt refer to the tokens in your main config file

#secrets=./secrets.ini
secrets=/etc/zm/secrets.ini

However, if I comment out both secrets= lines, mlapi dies with an unhandled error:

mlapi-1  | Dec 18 2023 06:13:34.796375 [ERR] Error parsing config:/etc/mlapiconfig.ini
mlapi-1  | Dec 18 2023 06:13:34.796390 [ERR] Error was:'NoneType' object is not iterable
mlapi-1  | Dec 18 2023 06:13:34.797252 [FAT] error: Traceback:Traceback (most recent call last):
mlapi-1  |   File "/mlapi/modules/utils.py", line 403, in process_config
mlapi-1  |     secrets = pyzmutils.read_config(g.config['secrets'])
mlapi-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mlapi-1  |   File "/usr/local/lib/python3.11/dist-packages/pyzm/helpers/utils.py", line 46, in read_config
mlapi-1  |     config_file.read(file)
mlapi-1  |   File "/usr/lib/python3.11/configparser.py", line 710, in read
mlapi-1  |     for filename in filenames:
mlapi-1  | TypeError: 'NoneType' object is not iterable

And if I set them to secrets= (empty string), mlapi dies with a different unhandled error:

mlapi-1  | Dec 18 2023 06:18:47.444560 [INF] Reading config from: /etc/mlapiconfig.ini
mlapi-1  | Dec 18 2023 06:18:47.444588 [INF] Reading secrets from: 
mlapi-1  | Dec 18 2023 06:18:47.444666 [ERR] Error parsing config:/etc/mlapiconfig.ini
mlapi-1  | Dec 18 2023 06:18:47.444680 [ERR] Error was:[Errno 2] No such file or directory: ''
mlapi-1  | Dec 18 2023 06:18:47.445022 [FAT] error: Traceback:Traceback (most recent call last):
mlapi-1  |   File "/mlapi/modules/utils.py", line 262, in process_config
mlapi-1  |     with open(secrets_filename) as f:
mlapi-1  |          ^^^^^^^^^^^^^^^^^^^^^^
mlapi-1  | FileNotFoundError: [Errno 2] No such file or directory: ''

It seems like either the example config needs to be updated to clarify that a secrets file is required or (preferred for my use case) the code needs to actually handle not having a secrets file, as the example config indicates.

I should hopefully have a PR coming to fix this, once I can actually get mlapi up and running.

No timeout or no answer from coral Google TPU

i use coral Tpu with mlapi and sometimes

i have no answer from tpu

DEBUG: get_file returned: ./images/52f889a3-9fba-43a0-8b30-17e91aa74cf4.jpg Dec 13 2020 17:43:39.463413 [DBG 1] |---------- TPU (input image: 600w*337h) ----------| Dec 13 2020 17:43:39.463456 [DBG 1] Waiting for TPU lock... Dec 13 2020 17:43:39.463608 [DBG 1] Got TPU lock for detection... Dec 13 2020 17:43:39.478238 [DBG 1] Released TPU detection lock Dec 13 2020 17:43:39.478296 [DBG 1] Coral TPU detection took: 14.678 milliseconds Dec 13 2020 17:43:39.478356 [DBG 2] core model detection over, got 1 objects. Now filtering Dec 13 2020 17:43:39.478384 [DBG 3] Max object size found to be: 70% Dec 13 2020 17:43:39.478427 [DBG 2] Converted 70% to 141540.0 Dec 13 2020 17:43:39.478457 [DBG 2] Returning filtered list of 1 objects. DEBUG: Object Recognition requested DEBUG: get_file returned: ./images/fd0ff2f5-3154-4ce5-965f-6b7b360d8583.jpg Dec 13 2020 17:43:39.527889 [DBG 1] |---------- TPU (input image: 600w*337h) ----------| Dec 13 2020 17:43:39.527935 [DBG 1] Waiting for TPU lock... Dec 13 2020 17:43:39.528090 [DBG 1] Got TPU lock for detection... Dec 13 2020 17:43:39.543669 [DBG 1] Released TPU detection lock Dec 13 2020 17:43:39.543741 [DBG 1] Coral TPU detection took: 15.64 milliseconds Dec 13 2020 17:43:39.543770 [DBG 2] core model detection over, got 0 objects. Now filtering Dec 13 2020 17:43:39.543807 [DBG 3] Max object size found to be: 70% Dec 13 2020 17:43:39.543855 [DBG 2] Converted 70% to 141540.0 Dec 13 2020 17:43:39.543881 [DBG 2] Returning filtered list of 0 objects. DEBUG: Object Recognition requested DEBUG: get_file returned: ./images/b7aeeafe-2de1-46d6-9848-35e4834e8f0d.jpg Dec 13 2020 17:45:57.693730 [DBG 1] |---------- TPU (input image: 600w*337h) ----------| Dec 13 2020 17:45:57.693800 [DBG 1] Waiting for TPU lock... Dec 13 2020 17:45:57.694104 [DBG 1] Got TPU lock for detection...

and zm_event_start continue to send analyse

1507288 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718145 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718145" 1507289 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718145 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718145 1507290 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718145 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718145 1508483 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718187 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718187" 1508484 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718187 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718187 1508485 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718187 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718187 1508897 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718202 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718202" 1508898 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718202 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718202 1508899 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718202 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718202 1512622 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718335 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718335" 1512623 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718335 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718335 1512624 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718335 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718335 1512802 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718353 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718353" 1512803 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718353 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718353 1512804 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718353 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718353 1513016 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718374 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718374" 1513017 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718374 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718374 1513018 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718374 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718374 1514023 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718445 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718445" 1514024 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718445 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718445 1514025 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718445 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718445 1514116 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718450 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718450" 1514117 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718450 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718450 1514118 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718450 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718450 1514542 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718478 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718478" 1514543 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718478 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718478 1514544 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718478 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718478 1514669 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718489 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718489" 1514670 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718489 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718489 1514671 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718489 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718489 1514889 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718507 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718507" 1514890 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718507 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718507 1514891 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718507 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718507 1515015 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718517 5 "jardin" "Motion All" "/var/cache/zoneminder/events/5/2020-12-13/4718517" 1515016 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718517 5 jardin Motion All /var/cache/zoneminder/events/5/2020-12-13/4718517 1515017 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718517 5 jardin Motion All /var/cache/zoneminder/events/5/2020-12-13/4718517 1515085 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718522 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718522" 1515086 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718522 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718522 1515087 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718522 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718522 1515439 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718549 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718549" 1515440 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718549 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718549 1515441 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718549 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718549 1515713 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718571 5 "jardin" " All" "/var/cache/zoneminder/events/5/2020-12-13/4718571" 1515714 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718571 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718571 1515715 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718571 5 jardin All /var/cache/zoneminder/events/5/2020-12-13/4718571 1516031 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718603 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718603" 1516032 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718603 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718603 1516033 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718603 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718603 1518568 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718741 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718741" 1518569 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718741 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718741 1518570 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718741 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718741 1523472 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718941 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718941" 1523473 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718941 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718941 1523474 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718941 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718941 1523704 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718954 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718954" 1523705 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718954 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718954 1523706 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718954 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718954 1525097 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4718994 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4718994" 1525098 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718994 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718994 1525099 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4718994 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4718994 1530244 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4719355 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4719355" 1530245 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4719355 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4719355 1530246 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4719355 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4719355 1530350 ? S 0:00 sh -c '/var/lib/zmeventnotification/bin/zm_event_start.sh' 4719364 12 "portier" " All" "/var/cache/zoneminder/events/12/2020-12-13/4719364" 1530351 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4719364 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4719364 1530352 ? S 0:00 /bin/bash /var/lib/zmeventnotification/bin/zm_event_start.sh 4719364 12 portier All /var/cache/zoneminder/events/12/2020-12-13/4719364

I juste restart mlapi and notification start

I thinks bug arrived when i have a very fast alarm and lot of request to TPU

Update utils.py to pull down yolo4 models

The eventserver now supports yolo4, so update the code to pull the models down if necessary.

Return Image with boxes?

I got this running after fighting with python, keras and tensorflow versions and forcing downgrades on those and it appears to work well now but how do I get the image with the box drawn as a return from the api?

The api server seems to make a new random hash like name of any remote or local images in the /image directory but that new random name is not specified in the returned json. So if I am serving a lot of requests to it I won't know which image name goes with my request or if there is a way to ask for it from the api. Thanks. Sorry for the novice questions.

face detection saves unknown face then bugs out

Hi,
I got ZM stable in docker up and detecting persons via coral on mlapi with yolov4 local fallback + working zmeventnotifications+zmninja. Trying to add face recognition to the mix I'm getting an error, although the recognition itself works correctly as the face gets saved into unknown_faces folder, but after that I receive following error in mlapi_error.log:

 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
172.17.0.4 - - [29/Nov/2020 19:10:22] "POST /api/v1/detect/object?type=object&delete=True HTTP/1.1" 200 -
172.17.0.4 - - [29/Nov/2020 19:10:50] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
172.17.0.4 - - [29/Nov/2020 19:10:50] "POST /api/v1/detect/object?type=face_names HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/werkzeug/serving.py", line 323, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.8/dist-packages/werkzeug/serving.py", line 312, in execute
    application_iter = app(environ, start_response)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/dist-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/dist-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/home/sammael/mlapi/mlapi.py", line 106, in post
    'names': face_obj.get_classes().tolist()
AttributeError: 'list' object has no attribute 'tolist'

and in zoneminder log:

zoneminder    | Nov 29 19:10:50 b23435ec1b9d /zm_detect.py[33997]: FAT [zmesdetect_m2] [Unrecoverable error:Expecting value: line 1 column 1 (char 0) Traceback:Traceback (most recent call last):
zoneminder    |   File "/var/lib/zmeventnotification/bin/zm_detect.py", line 852, in <module>
zoneminder    |     main_handler()
zoneminder    |   File "/var/lib/zmeventnotification/bin/zm_detect.py", line 457, in main_handler
zoneminder    |     data = r.json()
zoneminder    |   File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 900, in json
zoneminder    |     return complexjson.loads(self.text, **kwargs)
zoneminder    |   File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
zoneminder    |     return _default_decoder.decode(s)
zoneminder    |   File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
zoneminder    |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
zoneminder    |   File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
zoneminder    |     raise JSONDecodeError("Expecting value", s, err.value) from None
zoneminder    | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
zoneminder    | ]

but mlapi.log makes it look as if everything went ok so I'm not really sure which part of the whole setup is failing:

INFO: --------| mlapi version:2.0.0 |--------
INFO: Starting server with max:3 processes
 * Serving Flask app "mlapi" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
DEBUG: Object Recognition requested
DEBUG: get_file returned: ./images/c008d257-65ec-47bc-8fda-b5757d351386.jpg
Nov 29 2020 19:10:20.213197 [DBG 1] Waiting for TPU lock...
Nov 29 2020 19:10:20.215382 [DBG 1] Got TPU Lock
Nov 29 2020 19:10:20.215412 [DBG 1] Loading TPU model from disk
Nov 29 2020 19:10:22.924630 [DBG 1] TPU initialization (loading model from disk) took: 709.172 milliseconds
Nov 29 2020 19:10:22.924706 [DBG 1] |---------- TPU (input image: 1200w*675h) ----------|
Nov 29 2020 19:10:22.952415 [DBG 1] Released TPU lock
Nov 29 2020 19:10:22.952499 [DBG 1] Coral TPU detection took: 27.611 milliseconds
Nov 29 2020 19:10:22.952605 [DBG 2] core model detection over, got 3 objects. Now filtering
Nov 29 2020 19:10:22.952631 [DBG 3] Max object size found to be: 100%
Nov 29 2020 19:10:22.952944 [DBG 2] Converted 100% to 810000.0
Nov 29 2020 19:10:22.952990 [DBG 1] Ignoring person [901, 208, 1105, 597] as conf. level 0.2109375 is lower than 0.3
Nov 29 2020 19:10:22.953018 [DBG 1] Ignoring clock [206, 8, 356, 278] as conf. level 0.2109375 is lower than 0.3
Nov 29 2020 19:10:22.953038 [DBG 2] Returning filtered list of 1 objects.
DEBUG: Face Recognition requested
DEBUG: get_file returned: ./images/a5e46389-e539-403f-a0e6-8b75afe22dc6.jpg
Nov 29 2020 19:10:23.001230 [DBG 1] |---------- Face recognition (input image: 1200w*675h) ----------|
Nov 29 2020 19:10:23.001324 [DBG 1] Waiting for cpu lock...
Nov 29 2020 19:10:23.001782 [DBG 1] Got cpu lock for initialization...
Nov 29 2020 19:10:23.001823 [DBG 1] init lock released
Nov 29 2020 19:10:50.018758 [DBG 1] Finding faces took 11.795 milliseconds
Nov 29 2020 19:10:50.215418 [DBG 1] pyzm_cpu_lock Lock already released
Nov 29 2020 19:10:50.215505 [DBG 1] Computing face recognition distances took 183.94 milliseconds
Nov 29 2020 19:10:50.215533 [DBG 1] No faces to match, so creating empty set
Nov 29 2020 19:10:50.215556 [DBG 1] Matching recognized faces to known faces took 0.022 milliseconds
Nov 29 2020 19:10:50.216098 [INF] Saving cropped unknown face at [1040,79,1197,236 - includes leeway of 50px] to ./unknown_faces/Nov29-19h10m50s-7d409ab3-4521-426d-b8bb-c77728559a6e.jpg
Nov 29 2020 19:10:50.229169 [DBG 1] FACE:Returning: [[1090, 129, 1147, 186]], ['unknown face'], [1]
DEBUG: List of face names requested
[]

however the unknown face pictures end up named such as Nov29-19h43m49s-57a4b9ab-dd4a-4870-80fc-5a9bc27553d9.jpg regardless of value of unknown_face_name in mlapiconfig.ini

Yolo v5 support

Does this work with Yolo v5?

ImportError: cannot import name 'soft_unicode' from 'markupsafe'

When setting up a fresh install of mlapi, the error below occurs. It has to do with this breaking change: https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-0

root@32f4ffe5fa9a:/app# python3 mlapi.py -c /etc/mlapi/mlapiconfig.ini 
Traceback (most recent call last):
  File "mlapi.py", line 3, in <module>
    from flask import Flask, send_file, request, jsonify, render_template
  File "/usr/local/lib/python3.8/dist-packages/flask/__init__.py", line 14, in <module>
    from jinja2 import escape
  File "/usr/local/lib/python3.8/dist-packages/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/usr/local/lib/python3.8/dist-packages/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/usr/local/lib/python3.8/dist-packages/jinja2/defaults.py", line 3, in <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/usr/local/lib/python3.8/dist-packages/jinja2/filters.py", line 13, in <module>
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)

missing yolov3_classes.txt

Hi, just updated the mlapi server and for some reason had this file yolov3_classes.txt missing in the yolov3 folder. Found it somewhere googling it and added it. now works perfect.
Not sure if I missed a step on my side (did install requirements.txt and getmodels.sh), reporting it just in case.

fix crappy ml_overrides approach from zm_detect to mlapi

https://zmeventnotification.readthedocs.io/en/latest/guides/hooks.html#mlapi-overrides explains what is going on

Edge TPU module

Hi,

just to let you know i;ve started to implement an EdgeTPU module for MLapi.
Its running on a Raspberry Pi 4b and it works properly. I added a few configuration settings and the new module file. It's still in a alpha state as i would like to have it coexist with the current OpenCV implementation. If you would like me to cut the changes into small parts and send some pull requests, just let me know. My fork is at https://github.com/Rick-Jongbloed/mlapi

Log file:

DEBUG: Object Recognition requested
DEBUG: get_file returned: ./images/f178ffe7-c457-4da3-9599-6cb36c75edc7.jpg
DEBUG: Initializing MobileNet SSD v2 TFlife EdgeTPU model
DEBUG: edgetpu_ssd_mobilenet_models:./models/mobilenet-ssd-edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
DEBUG: MobileNet SSD v2 EdgeTPU initialization (loading model from disk) took: 834.326 milliseconds
DEBUG: MobileNet SSD v2 EdgeTPU detection took: 35.33070373535156 milliseconds
INFO: object:person at [836, 137, 918, 405] has a acceptable confidence:0.953125 compared to min confidence of: 0.4, adding
INFO: object:car at [303, 120, 611, 364] has a acceptable confidence:0.91015625 compared to min confidence of: 0.4, adding
INFO: object:person at [227, 146, 351, 373] has a acceptable confidence:0.58203125 compared to min confidence of: 0.4, adding`

127.0.0.1 - - [03/Apr/2020 23:28:12] "POST /api/v1/detect/object HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: ./images/42b52941-8eae-4dbc-9f05-d5cf41a598e1.jpg
DEBUG: MobileNet SSD v2 EdgeTPU detection took: 15.930500984191895 milliseconds
INFO: object:person at [836, 137, 918, 405] has a acceptable confidence:0.953125 compared to min confidence of: 0.4, adding
INFO: object:car at [303, 120, 611, 364] has a acceptable confidence:0.91015625 compared to min confidence of: 0.4, adding
INFO: object:person at [227, 146, 351, 373] has a acceptable confidence:0.58203125 compared to min confidence of: 0.4, adding

remote ML requirements question

Hello -

I have been using dlandon's zoneminder container which up until recently included ES/ML options. I had these working well but he's decided to drop hooks / ml from his container but is keeping ES. Therefore I need to stand up a remote ML server to work along side his container so I can continue to do object detection.

I have a fresh Ubuntu 20.04 server stood up but what's not 100% clear to me is what specifically is required to standup a remote machine learning host. I have gone through your es/ms installation documentation as well as your mlapi documentation.

I believe I do require the mlapi project you have here (to handle the remote side of this) but looking through it, like for example the adding user part, it's looking for a db which doesn't natively exist with just mlapi. Therefore I assume that I need to do some basic installation of es/ml from your zmeventnotification project.

I have also looked at the zmeventnotification project, specifically around ES/ML installation and know the install script there will prompt me if I want things like ES/hooks/ml.

So, could you let me know a basic outline of what is required to stand up a remote ML server based on your two projects? Would love to get this going again. It was working great in his container but now I need to stand up a remote ml server to continue my object detection. The ml_sequence stuff was working great!

RFE: crop and save (unknown) faces

I miss a feature to crop and save faces which can then be fed into the known faces structure.

Mvh.

Torkil

mlapi crash on

mlapi crashes with shapely/speedups/_speedups.pyx
when running an event (not all of them do crash) I have:

This model iteration inside object found: labels: ['person'],conf:[0.9985199570655823]
Mar 03 2021 18:02:23.172784 [DBG 3] intersection: comparing object:person,POLYGON ((389 30, 513 30, 513 238, 389 238, 389 30)) to polygon:smaller_m7,POLYGON ((0 476, 54 455, 623 240, 604 453, 0 476))
Traceback (most recent call last):
  File "shapely/speedups/_speedups.pyx", line 247, in shapely.speedups._speedups.geos_linearring_from_py
AttributeError: 'list' object has no attribute '__array_interface__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/home/x/.mlapi/mlapi.py", line 228, in post
    matched_data,all_matches = m.detect_stream(stream=stream, options=stream_options, ml_overrides=ml_overrides)
  File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 466, in detect_stream
    _b,_l,_c, _e = self._filter_patterns(seq,_b,_l,_c, polygons)
  File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 263, in _filter_patterns
    poly = Polygon(p['value'])
  File "/usr/lib/python3.9/site-packages/shapely/geometry/polygon.py", line 243, in __init__
    ret = geos_polygon_from_py(shell, holes)
  File "/usr/lib/python3.9/site-packages/shapely/geometry/polygon.py", line 509, in geos_polygon_from_py
    ret = geos_linearring_from_py(shell)
  File "shapely/speedups/_speedups.pyx", line 341, in shapely.speedups._speedups.geos_linearring_from_py
ValueError: A LinearRing must have at least 3 coordinate tuples

when running the same event on ES server directly I have :

zmesdetect_m7[56308] DBG4 detect_sequence.py:437 [This model iteration inside object found: labels: ['person'],conf:[0.9997007250785828]]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG3 detect_sequence.py:240 [intersection: comparing object:person,POLYGON ((114 142, 258 142, 258 344, 114 344, 114 142)) to polygon:all,POLYGON ((0 476, 1 216, 636 23, 604 453, 0 476))]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG3 detect_sequence.py:244 [all polygon/zone has its own pattern of (bicycle|motorbike), using that]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG3 detect_sequence.py:273 [all intersects object:person[[(114, 142), (258, 142), (258, 344), (114, 344)]] but does NOT match your detect pattern filter]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG2 detect_sequence.py:482 [We did not find any object matches in frame: snapshot]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG2 yolo.py:63 [pyzm_uid33_cpu_lock portalock already released]
03/03/21 17:59:15 zmesdetect_m7[56308] DBG1 detect_sequence.py:526 [perf: TOTAL detection sequence (with image loads) took: 0:00:03.229984  to process 644885]

not sure where to start..

EDIT : looking closer, it seems mlapi does not take the right zone into account. ES server checks (correctly) the detected object vs 'all' zone, whereas mlapi checks vs 'smaller_m7' zone.

the monitor is defined as below:

[monitor-7]
resize=no
match_past_detections=no
detection_sequence=object
smaller_m7=0,476 54,455 623,240 604,453
smaller_m7_zone_detection_pattern=(person|bear|dog|cat)
all_zone_detection_pattern=(bicycle|motorbike)
wait=2

Error executing remote API: Extra data: line 1 column 329 (char 328)

Hi, not sure where to start here. I don't know when those errors have started to show up, as it fallback to server when the mlapi server is down, I didn't notice up until now..
I've reinstalled mlapi but no luck.

Dockerised version

FYI, I've put together a docker version with python3.8, opencv a dlib builds at https://github.com/rossbuggins/mlapi-docker/ https://hub.docker.com/r/rossbuggins/mlapi

Early return in Detect() prevents image deletion

In the Detect() function in mlapi.py, there is a return before the code that deletes the image file. This means that images will not be deleted even if the delete argument is specified in the URL.

show_percent=no not taken into account by mlapi

Hi, it seems objectconfig.ini show_percent=no is not taken into account by the mlapi server. Images are shown with detected object and % when analyzed by mlapi.

platerecognizer chosen image quality

Hi,

When uploading images to platerecognizer the image quality appears poor, and therefore plates are unreadable.

What the stills capture:

What platerecognizer.com receives:

I've run a test by posting the alarm image from this recording to platerecognizer and it picked it up (albeit mistaking 1 character) but the quality on there was much better, and readable.

Hopefully i have the right location o post this - wondering what i can do to force it to post the best image to use?

RuntimeError: Internal: Unsupported data type in custom op handler

Hello

I've installed the lasted version of mlapi, and the egretpu library.

When I call
curl -H "Content-Type:application/json" -H "Authorization: Bearer ${ACCESS_TOKEN}" -XPOST -d "{\"url\":\"https://upload.wikimedia.org/wikipedia/commons/c/c4/Anna%27s_hummingbird.jpg\"}" http://192.168.31.11:5000/api/v1/detect/object

I've got this message:

Mar 06 2021 11:28:57.617030 [DBG 3] --------- Frame:0 Running variation: #1 -------------
Mar 06 2021 11:28:57.629720 [DBG 2] Waiting for pyzm_uid0_tpu_lock portalock...
Mar 06 2021 11:28:57.630728 [DBG 2] Got pyzm_uid0_tpu_lock portalock
Mar 06 2021 11:28:57.630857 [DBG 1] |--------- Loading TPU model from disk -------------|
Mar 06 2021 11:29:00.371399 [DBG 2] Released portalock pyzm_uid0_tpu_lock
Mar 06 2021 11:29:00.371586 [ERR] Error running model: Internal: Unsupported data type in custom op handler: 653409188Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare.

Mar 06 2021 11:29:00.374299 [DBG 2] Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/pyzm/ml/detect_sequence.py", line 458, in detect_stream
    _b,_l,_c = m.detect(image=frame)
  File "/usr/local/lib/python3.7/dist-packages/pyzm/ml/object.py", line 54, in detect
    b,l,c = self.model.detect(image)
  File "/usr/local/lib/python3.7/dist-packages/pyzm/ml/coral_edgetpu.py", line 157, in detect
    self.load_model()
  File "/usr/local/lib/python3.7/dist-packages/pyzm/ml/coral_edgetpu.py", line 93, in load_model
    self.model.allocate_tensors()
  File "/usr/local/lib/python3.7/dist-packages/tflite_runtime/interpreter.py", line 242, in allocate_tensors
    return self._interpreter.AllocateTensors()
  File "/usr/local/lib/python3.7/dist-packages/tflite_runtime/interpreter_wrapper.py", line 115, in AllocateTensors
    return _interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
RuntimeError: Internal: Unsupported data type in custom op handler: 653409188Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare.


Mar 06 2021 11:29:00.374897 [DBG 3] --------- Frame:0 Running variation: #2 -------------
Mar 06 2021 11:29:00.375074 [DBG 2] Waiting for pyzm_uid0_gpu_lock portalock...
Mar 06 2021 11:29:00.375630 [DBG 2] Got pyzm_uid0_gpu_lock portalock

pyzm is in the latest version
root@eole:/home/jerry/mlapi# sudo -H pip3 install pyzm Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Requirement already satisfied: pyzm in /usr/local/lib/python3.7/dist-packages (0.3.36)

What do I wrong ?

thank by advance,

RFE: More than one face detection model pr image

Detecting and recognizing faces seems to be difficult since I get very few faces detected but persons are detected just about every time.

I would like an option to do multiple face detection passes with different models to improve the odds of finding faces, like HOG, CNN, Caffe and whatnot instead of just picking one.

For saving unknown faces you could just loop over all selected models and save the crops but some more logic might be needed for returning coordinates to avoid intersections. Perhaps just something simple like the first hit, or some math to ensure no intersections/duplicates.

Thanks!

Disabling vibrate on push has no effect.

Samsung Galaxy S7. Have force quit the app to no effect, but not rebooted phone.

Mvh.

Torkil

Add OpenCV 4.1.2 DNN support

https://github.com/pliablepixels/zmeventnotification/issues/214

Enhance to use a proper WSGI server

Enhance mlapi to utilize a proper WSGI server.

No images written, neither in zoneminder dir nor bbox debug

With processes = 1 it seems to work, but I get no images.

`[root@ryzen mlapi]# python3 ./api.py -c config.ini
Initializing log
DEBUG: DB engine ready
DEBUG: No secrets file configured
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0
DEBUG: pre-trained faces found, using that. If you want to add new images, remove: ./known_faces/faces.dat
INFO: --------| mlapi version:1.0.1 |--------
INFO: Starting server with max:1 processes

Serving Flask app "api" (lazy loading)
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: off
Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/2bf2b853-265a-4c05-a6da-35656dd76527.jpg
DEBUG: Initializing Yolo
DEBUG: config:./models/yolov3/yolov3.cfg, weights:./models/yolov3/yolov3.weights
192.168.40.138 - - [20/Jan/2020 07:10:11] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/7b7ac66c-9eb4-4f34-b4cf-b26273cea75d.jpg
INFO: object:train at [-4, 40, 512, 442] has a acceptable confidence:0.7992180585861206 compared to min confidence of: 0.4, adding
192.168.40.138 - - [20/Jan/2020 07:10:13] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/5164491b-f26d-4b75-89e0-2411f7fca166.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [20/Jan/2020 07:10:14] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/be67ae34-90ad-4bd1-86c7-c8364c28d8cb.jpg
192.168.40.138 - - [20/Jan/2020 07:10:16] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/e85484eb-3d29-493b-91b6-e2864d7aba80.jpg
192.168.40.138 - - [20/Jan/2020 07:10:18] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/a7e82e7b-4473-43f3-8d96-382040b480a7.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [20/Jan/2020 07:10:18] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/3ed7ddf3-2900-4886-9052-f32651f28403.jpg
192.168.40.138 - - [20/Jan/2020 07:10:31] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/7e94b3b8-c3ed-4bfd-9423-817494ed6a1f.jpg
192.168.40.138 - - [20/Jan/2020 07:10:32] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/a95a802b-20f1-4851-92d7-12c0cff18e26.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [20/Jan/2020 07:10:32] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/e1e952ad-234f-427c-a4f3-ba639ac5c2b3.jpg
INFO: object:train at [-4, 74, 514, 478] has a acceptable confidence:0.7119221091270447 compared to min confidence of: 0.4, adding
192.168.40.138 - - [20/Jan/2020 07:10:36] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/08f73b58-1892-4624-b692-2cfe49044df0.jpg
INFO: object:train at [-4, 74, 514, 478] has a acceptable confidence:0.7119221091270447 compared to min confidence of: 0.4, adding
192.168.40.138 - - [20/Jan/2020 07:10:38] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/d6695aac-711d-49eb-80ad-7fec5de0c696.jpg
DEBUG: Face recognition: no faces found
[snip]
`

With processes = 2 there's an error:

DEBUG: Object Recognition requested DEBUG: get_file returned: images/52ac9970-d23f-4158-990d-ac65fee2a52e.jpg DEBUG: Initializing Yolo DEBUG: config:./models/yolov3/yolov3.cfg, weights:./models/yolov3/yolov3.weights INFO: object:person at [462, 318, 600, 654] has a acceptable confidence:0.9877113103866577 compared to min confidence of: 0.4, adding 192.168.40.138 - - [20/Jan/2020 07:07:21] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 - DEBUG: Face Recognition requested DEBUG: get_file returned: images/bfe44a18-5ce6-4d08-a27b-6845c24e73ec.jpg 192.168.40.138 - - [20/Jan/2020 07:07:22] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 500 - Error on request: Traceback (most recent call last): File "/usr/local/lib64/python3.6/site-packages/werkzeug/serving.py", line 304, in run_wsgi execute(self.server.app) File "/usr/local/lib64/python3.6/site-packages/werkzeug/serving.py", line 292, in execute application_iter = app(environ, start_response) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2463, in __call__ return self.wsgi_app(environ, start_response) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app response = self.handle_exception(e) File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router return original_handler(e) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router return original_handler(e) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper resp = resource(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request resp = meth(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper return fn(*args, **kwargs) File "/root/mlapi/api.py", line 125, in post detections = m.detect(image) File "/root/mlapi/modules/face_recognition.py", line 68, in detect face_locations = face_recognition.face_locations(rgb_image, model=self.model, number_of_times_to_upsample=self.upsample_times) File "/usr/local/lib/python3.6/site-packages/face_recognition/api.py", line 116, in face_locations return [_trim_css_to_bounds(_rect_to_css(face.rect), img.shape) for face in _raw_face_locations(img, number_of_times_to_upsample, "cnn")] File "/usr/local/lib/python3.6/site-packages/face_recognition/api.py", line 100, in _raw_face_locations return cnn_face_detector(img, number_of_times_to_upsample) RuntimeError: Error while calling cudaGetDevice(&the_device_id) in file /tmp/pip-install-28f7zm6b/dlib/dlib/cuda/gpu_data.cpp:201. code: 3, reason: initialization error
Works with local detection. I'll be happy to provide whatever debug is neccesary.

Mvh.

Torkil

zoneminder / mlapi Goto Github PK

mlapi's Introduction

ZoneMinder

Overview

Contacting the Development Team

Our Dockerfile has moved

Installation Methods

Install from a Package Repository

Building from Source is Discouraged

Building a ZoneMinder Package

Package Maintainers

Docker

Contribution Model and Development

mlapi's People

Contributors

Stargazers

Watchers

Forkers

mlapi's Issues

Recommend Projects

Recommend Topics

Recommend Org