Performing updates on the server (or fresh installs) with commits past commit 19942c1 results in not being able to connect to the server locally and unable to configure.

I'm trying without success to test the detection system. I turned the sensitivity all the up and I'm trying to place a ball of spaghetti on the build plate mid print. It does get detected and sometimes rises to the red meter level but I never get a notification. I have tried moving the ball around as well as removing it and placing it back after a minute.

Is there a way to trigger a notice for purposes of testing the smtp? Or a recommended way to produce an alert to confirm the system is working?

Because of #4 , one ml_api server can only serve 1 request at a time, which makes it prone to cascading failure when GCP is slow at returning images. Therefore we had to implement a strict timeout to mitigate this problem: https://github.com/TheSpaghettiDetective/TheSpaghettiDetective/blob/255855272e2b3e0e12f0e9ae396d569b1fa00de1/ml_api/server.py#L34

When @4 is resolved, we should remove this timeout param.

I don't know if is the same in every country, but in France, with the mobile operator "Free" we have an API for sending SMS on our number.

Its juste a GET request like
https://smsapi.free-mobile.fr/sendmsg?user=28XXXX47&pass=PWj20HXxXxXxsG&msg=Hello%20World.

It would be very nice if we can use this service for failure notification.

On a ubuntu system how do we set it to run at boot?

Hi, I just stumbled upon your project, it looks really cool.
In the readme you mentioned the long build process. I just skimmed over your docker configuration and have a few suggestions:

• ml_api: Use nvidia/cuda:9.0-runtime-ubuntu16.04 instead of nvidia/cuda:9.0-devel-ubuntu16.04, it should be sufficient and uses 1GB less space. -> Saves download time
• web: Use python:3.6-alpine instead of python:3.6 it uses 850MB less space, vim and ffmpeg are available in alpine, vim can probably be removed. -> Saves download time
• web: Make better use of docker caching: First add the requirements file, then pip install, then add the rest of the project. Using this the build process is much faster if no package requirements have changed. -> Faster development cycle
• redis (minor): Use redis:alpine, uses 50 MB less space
• all docker images: build them and upload them to Dockerhub. Most of the time the build process downloads and installs apt and pip packages. By prebuilding and uploading to Dockerhub, this can be consolidated into one docker pull for each image. If you still provide the Dockerfiles, anybody can build the images themselves when they want to debug something. -> Saves build time.

By using all suggestions above, you could get rid of all package installations (using the prebuilt image) which saves a lot of time and also save 1.9GB of Downloads.

Hi,

Disclaimer: the below suggestions are in a effort to reduce this model size to where I can run it on my RPI3. My ML experience is solely in emotion detection.

I am looking over this code, because it's awesome, and looking at your model I believe it is too complex for the task at hand.

• Too many convolutions: I see 23 convolutions and a respective number of pooling layers. This is too many, you are wasting too much space on useless data. Check the activation of your neurons as a percentage of the model size, you might be surprised how few are actually used.
• Too many filters on convolutions: Many of the model convolutions have 512-1024 filters. This is way too many in my opinion -- past 32 or 64 filters is more or less useless at extracting meaningful data. I mean just look this Example Sigma = 6 convolution
• Too large of an input picture: This is a harder problem to solve, but a more efficient model will arise when your input dimensions are smaller. I have not looked thoroughly through the code, however, you may want to explore recognizing the nozzle and carriage and take a fixed image bounding around the estimated nozzle location. This would be done through classical computer vision as opposed to machine learning. Or just follow the GCODE around and estimate the printer head location / dimensions.

If I have some free time this week I will look at the code and see if I can 'minify' the model and get it running on the pi.. That would be awesome and could be something you charge extra for as part of the service.

As a viewer of spaghetti gallery, I'd like to see the tags associated with each timelapse, and upvote existing tags or add new tags, such as "false alarm", "missed failure", "back light".

The current version of model.so that is provided in The Spaghetti Detective only runs on a single CPU core when not using GPU's. This is not very efficient.

I cloned https://github.com/AlexeyAB/darknet

modified Makefile Lines 6 and 7

OPENMP=1
LIBSO=1

Built libdarknet.so

Copied it to TheSpaghettiDetective/ml_api/bin and replaced the existing model.so with the new file.

Now TSD runs multi threaded and is much quicker on a system with lots of CPU cores but not Nvidia GPU.

Test the idea of using p-increase, rather than p itself. Maybe 5-rolling avg over 50 rolling avg?

Example: https://github.com/getsentry/onpremise

503 may happen either on GCP api calls, or an http GET https://storage.googleapis.com/prod-tsd-pics/raw/124/1552445901.jpg?GoogleAccessIdxxxx

Currently a docker-compose up --build results in an explicit deletion of the timelapses while rebuilding the web - container:

Deleting 'media/tsd-timelapses/private/1_p.json'
Deleting 'media/tsd-timelapses/private/2.mp4'
Deleting 'media/tsd-timelapses/private/1_tagged.mp4'
Deleting 'media/tsd-timelapses/private/1.mp4'
Deleting 'media/tsd-timelapses/private/1_poster.jpg'
Deleting 'media/tsd-timelapses/private/2_poster.jpg'
Deleting 'media/tsd-timelapses/private/2_p.json'
Deleting 'media/tsd-timelapses/private/2_tagged.mp4'

It didn't seem to matter if this directory was mounted as an external volume or not. This does seem counter-productive if you want to keep the timelapses for future training.

I'm attempting to run my own server (not through docker-compose, but rather a custom K8s chart).

I am seeing POSTs to /octo/pic and I am able to see the image of my printer on the web UI, however, the web UI never shows the filename or estimated time left. The web container logs don't ever indicate any POSTs to /octo/status.

Any ideas on what to check next?

$ kubectl -n tsd logs the-spaghetti-detective-568cb69cc6-5b295 -c the-spaghetti-detective-web | grep -i 'POST /api/octo/pic' | wc -l`
16
$ kubectl -n tsd logs the-spaghetti-detective-568cb69cc6-5b295 -c the-spaghetti-detective-web | grep -i 'POST /api/octo/status' | wc -l
0

Current ml_api forces workers=1 because it calls out to native code and there are a lot of static bridging variables exist between Python and C. Need to have a way to enable concurrency > 1 when running prediction. Ideally it should be done in a way to maximize GPU utilization.

For the container "web", the image is created from web/Dockerfile. This Dockerfile has EXPOSE 3333, but docker-compose.yml starts the service on port 3334 and defines ports "3334:3334". Wouldn't it make sense to get these differences in line and define just one port?

Not sure if it's my configuration that's to blame but when I receive the email saying my print might be failing (I have it set to just notify) the snapshot in the email is not showing

Current files accumulate in storage indefinitely. Need to have a periodical task that does the clean up.

Steps to reproduce:

1. Sign up using Google account [email protected]
2. Log out
3. Try to sign up using Facebook with same email [email protected]
4. Error!

I have 16 printers that will soon move to a backyard office I'm hoping to build or possibly even a small rented office space and I would like to eventually have them all monitored by TSD. I'll likely have to build a new pc for the job so I'm trying to figure out how much computing power I'll need.

What specs would you recommend for such a setup? Is it even possible to build one machine to watch 16 printers? Or maybe it's less demanding than I'm picturing?

How about the specs for most other users who will monitor just one printer? How important is the gpu? I see watching one printer a lot of cpu usage but not so much on my nvidia gtx1060.

I have successfully installed the octoprint module and was able to connect and test using the beta servers. Then I wanted to try it locally so, I setup TSD server on my local computer and added the printer. Octoprint verifies the token correctly but it never connects.

I get false alarms due to the bed clips holding my glass plate on. Is it possible maybe to train the model to ignore them? I have no idea how ML modeling works so forgive me if that's a ridiculous suggestion.

When running docker-compose up -d, here's what I get

ERROR: Service 'ml_api' failed to build: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:297: copying bootstrap data to pipe caused \"write init-p: broken pipe\"": unknown

Docker host is ubuntu 16.04, running Docker CE 18.06.3, Compose version 1.23.2

First off kudos for a great looking project 🎉

It would be nice to have some indication of the hardware needed to run this server side bit. I believe CUDA only runs on specific hardware, and seeing the nvidia docker image, I guess you need an NVIDIA card.

Am a bit of a newbie on the subject, but would this project run on a low-end Nvidia Jetson Nano Developer Kit (99$) https://www.sparkfun.com/products/15297

I think it might be a great project for tinkerers (into 3D printing) that want to look into machine learning and image processing. Thanks for sharing 👐 👏

But they behave as expected on "Spaghetti Gallery" page.

I was able to start the build under windows in powershell by cd to the TSD directory and then the normal docker-compose up -d, I'm sure there is a syntax that would make it one line but I'm not that versed.

The problem I get is:

ERROR: Service 'ml_api' failed to build: The command '/bin/sh -c wget --quiet -O model/model.weights $(cat model/model.weights.url)' returned a non-zero code: 8

Any ideas? I checked the url in the file and was able to download it manually in a browser so it's reachable from my pc

hi,

I was having issues connecting the plugging and this get fixed here
#26

Now, it connects and the image appear when the printer is not printing. When you start pringint we have no video, and also no detection.

Let me know if I can attach some logs, or need more information.

(The idea is amazing!)
Thanks!

For local implementations of TSD, it would be really nice to add Pushbullet integration for web-only notification. Documentation: https://docs.pushbullet.com/ This would allow home users to still get notifications outside of just email

When user observes false positives or otherwise don't want the print to be paused on possible failures, and they don't want to change the settings since they are afraid they will forget to change it back. This toggle that's effective only for current print is a convenient way to temporarily disable pause-on-print.

The plugin connects to 'app.thespaghettidetective.com' once every 10 seconds to send only one picture from the webcam for the server to run algorithm on. In a 12 hour period the octopi connected to 'app.thespaghettidetective.com' approximately 11K times, while the printer was idle.

The plug in should only function when octopi senses a print.

TItle says it all, Self hosting the server.
It used to fail whenever it saw spaghetti, but now when there's nothing but spaghetti, it doesn't fail. Even though the fail gauge is in the red the whole time.

I was curious if you could share some details about the actual nn running. How was it generated and is it possible to rebuild the nn ourself?