modischfabrications / cutsolver Goto Github PK

Pipenv is nice, but poetry should be even nicer. Check the progress, seems like pipenv awoke from a long hiatus, might get some new updates soon.

Deploying to pypi is the biggest factor as of right now:
https://stackoverflow.com/questions/58218592/feature-comparison-between-npm-pip-pipenv-and-poetry-package-managers

Hello,

Is it possible to extend CutSolver to be able to accept multiple number/length tuples for stock and a default length, so I can use this solution to optimize the use of available stock?

At the moment when I have a stock length of lets say 1500, an item of 1500 leads to an error because of the cut width.

I think it should be easy to check if an item is equal to the stock length (or rather stock length - cut_width) and allow that special case.

https://realpython.com/pytest-python-testing/

Create a sample deployment using the frontend, traefik and more.

https://www.smarthomebeginner.com/traefik-docker-security-best-practices/

https://microbadger.com/images/traefik

all comparisons use amd64. Check using docker image ls

old size (pre-multistage): 344 MB

new size (py3.7 builder, py3.7slim): 239 MB

new size (3.8): 287 MB

no idea what's wrong, might be because of complete copy of pip?
-slim base image has <100MB

length: quantity

would make compressing (#11) easier and contains the same information in a structured way.

Track versions only in git, depend everything else on it.

1. Git hook (pre commit?): If tag update version in file (simple >> ?)
2. Python code loads variable from file
3. Running instances can curl version file to warn with outdated answers.

look for things that have a common format and are useful to have.

1. author
2. repo
3. license?
4. version?

Hello, I am testing CutSolver and when trying a job I don't get an optimal solution:
Is there a way to select the solver type?
Thanks for your efforts!
{
"job": {
"max_length": 6500,
"cut_width": 20,
"target_sizes": {
"1258": 2,
"3958": 2
}
},
"solver_type": "bruteforce",
"time_us": 85,
"lengths": [
[
3958
],
[
3958,
1258
],
[
1258
]
]
}

There are often solutions with identical trimmings. Current code just picks one at random, but there could be a setting to prioritize based on secondary metrics.

1. Most cuts of the same length in a row
2. largest trimmings at once / largest distribution of trimmings
3. ..?

Oh boy, what a mess.

https://app.codacy.com/project/ModischFabrications/CutSolver/dashboard

Bootstrap, Angular, Vue?

As easy and slim as possible, this should just have nicer graphics and visualisations than /docs.

https://codecov.io/gh/ModischFabrications/CutSolver

Token was registered already and pushing was added to .travis.yml, but nothing is showing yet. Compare to codacy coverage statistics.

Look at CutSolverFront for the new values

Hi,

I've done a simple test, see screenshot:

Using "cutting width = 0", shouldn't this yield 10 parts per rod?

cheers,
Thorsten

https://github.com/ModischFabrications/CutSolverFrontend

Some pip-packages are installed multiple times, find a way to prevent that without risking missed packages

The different solvers return different results

Bruteforce:

FFD:

The FFD result is orrect in my opinion. 2x 495 = 990 + 6 = 996 => Two pieces should fit into one stock of 1000

I would propose to ditch the Bruteforce algorithm altogether as it doesn't matter if the FFD is sub milliseconds slower when used for < 10 items. Above that threshold the FFD comes into action anyway and that way we do not have to maintain two different algorithms.

The same goes for the gapfill algorithm.

"CutSolver" is not really a unique name, think about a nicer name with similar precision.

Ideas:

• Cutsy
• Partitioner
• Cutify
• CutIt
• ...

Look into what gunicorn and others are doing.

Limits should probably be passed as a block, see constants.py

    Implemented some things with GH Actions

• coverage report
• docker build + push
• docker test + healthcheck

Originally posted by @ModischFabrications in #44 (comment)

https://black.readthedocs.io/en/stable/

See TODO

this also kind of helps with #6 .

make Jobs more secure by including a watchdog that kills a calculation if it is running for longer than X.

proposed X: 10s (1min with auth)

pytest is the coolest kid on the block, evaluate that.

https://realpython.com/python-testing/#pytest

Domain is easy, but a server takes money.

Might want to go serverless: https://www.deadbear.io/simple-serverless-fastapi-with-aws-lambda/

see ModischFabrications/CutSolverFrontend#6

https://shields.io/category/dependencies -> pipenv versions of fastapi and more

Check which license FastAPI (and other submodules) uses and if that imposes any limits for this project.

Target is something permissive that encourages sharing extensions to this project but allows unmodified usage for everything. (probably LGPL?)

1. Many Jobs
2. Invalid Jobs
3. Huge Jobs
4. Weird random stuff I know nothing about (Schmiddi?)

Curl as addon adds another 15 MB to the image, which is way too much. Try using wget instead, see if that reduces the size increase to <<5MB

Having workers and a queue with pending jobs was considered but seemed useless,
as ideally all requests have their own thread and a (by comparison) short calculation time.
This makes a queue useless. The same argumentation also holds true for a result-buffer.

brute-forcing could be improved significantly by ignoring duplicates (like 4x200mm)

users might be interested to see if their solution is the guaranteed optimum or if it is just an approximation.

{
...
"solver": "bruteforce (perfect)"
}

test docker image health? -> waiting for 5m is bad, check it manually with curl after 20s or something.

might want to put the test stuff from the magic file back or split it up completely.

CI will skip (unnecessary) build repetition on tag pushes, but docker-meta won't handle tags otherwise as it seems.

https://github.com/ModischFabrications/CutSolver/actions

Ideally latest would follow the newest git tag and each git tag get's it's own build. I could go and allow tag pushes, but that feels crazy slow and inefficient

Add interface for consoles.

First idea for this interface:
python main.py -i job.json -o result.json

might be extendable to work with pipes or stdin/stdout, depending on what is actually requested.

Deploying this requires building it manually. While it is not that hard it should be a quick fix to upload an image that resembles the master-branch.

https://docs.docker.com/docker-hub/builds/

User/Frontend can't see how good of a solver he gets, allow /constants to see parameters:

bruteforce: <=9
FFD: <=999
...?

People that are trying this service out are likely to queue the same result multiple times.
Caching these values will really save some execution time.

This is somehow relevant to #24

Random variations in solvers can result in test working only some of the time. Problem was found during rework of #66 , where unsorted sets can result in different results each time

Travis based multiarchitecture build is a bad hack, try the new builder:

https://www.docker.com/blog/docker-v2-github-action-is-now-ga/

suggest solver while sending a Job

1. allow requesting worse but faster requests
2. allow really expensive Jobs (with auth for security!)

The algorithm is very CPU intense, might be a good idea to do that close to the metal.

SIMD or numpy? There are probably a few ways to do that, everything else can stay Python.

Docker hub builds until the timeout hits while arm64 is done after 2min (according to Logs). Builds can't be cancelled manually.

No errors are shown, seems like other architectures are being ignored even though qemu seems to register successful.

Try installing qemu before building the image instead. Post to docker/hub-feedback#1261 if no other solution was found.

$ sudo apt update
$ sudo install qemu qemu-user-static qemu-user binfmt-support

Building in Docker Cloud's infrastructure...
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '192.30.253.112' to the list of known hosts.
Reset branch 'master'
Your branch is up-to-date with 'origin/master'.
Executing post_checkout hook...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 619 0 619 0 0 605 0 --:--:-- 0:00:01 --:--:-- 605
0 1678k 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0qemu-3.0.0+resin-arm/
qemu-3.0.0+resin-arm/qemu-arm-static
100 1678k 100 1678k 0 0 1081k 0 0:00:01 0:00:01 --:--:-- 7770k
Pulling cache layers for index.docker.io/modischfabrications/cutsolver:latest...
Done!
Executing pre_build hook...
Unable to find image 'multiarch/qemu-user-static:register' locally
register: Pulling from multiarch/qemu-user-static
fc1a6b909f82: Pulling fs layer
ed0d08c6c573: Pulling fs layer
96aa3d7b9b30: Pulling fs layer
8d669bf48302: Pulling fs layer
8d669bf48302: Waiting
ed0d08c6c573: Verifying Checksum
ed0d08c6c573: Download complete
96aa3d7b9b30: Verifying Checksum
96aa3d7b9b30: Download complete
8d669bf48302: Verifying Checksum
8d669bf48302: Download complete
fc1a6b909f82: Verifying Checksum
fc1a6b909f82: Download complete
fc1a6b909f82: Pull complete
ed0d08c6c573: Pull complete
96aa3d7b9b30: Pull complete
8d669bf48302: Pull complete
Digest: sha256:e972d7cd2aa56ed083dc74d7f0cb708e0a9d041aa64e50462785338b29bceeca
Status: Downloaded newer image for multiarch/qemu-user-static:register
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
KernelVersion: 4.4.0-1060-aws
Components: [{u'Version': u'18.03.1-ee-3', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.4.0-1060-aws', u'Os': u'linux', u'BuildTime': u'2018-08-30T18:42:30.000000000+00:00', u'ApiVersion': u'1.37', u'MinAPIVersion': u'1.12', u'GitCommit': u'b9a5c95', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.2'}}]
Arch: amd64
BuildTime: 2018-08-30T18:42:30.000000000+00:00
ApiVersion: 1.37
Platform: {u'Name': u''}
Version: 18.03.1-ee-3
MinAPIVersion: 1.12
GitCommit: b9a5c95
Os: linux
GoVersion: go1.10.2
Starting build of index.docker.io/modischfabrications/cutsolver:latest...
Step 1/7 : FROM python:3.7
---> a4cc999cf2aa
Step 2/7 : EXPOSE 80
---> Using cache
---> 62a54b98454e
Step 3/7 : COPY ./requirements.txt requirements.txt
---> Using cache
---> 82eaf342eb1e
Step 4/7 : RUN pip install -r requirements.txt
---> Using cache
---> bae301531cc6
Step 5/7 : COPY ./app /app
---> Using cache
---> b23abeae1990
Step 6/7 : WORKDIR /app
---> Using cache
---> 60309b3c52ab
Step 7/7 : CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]
---> Using cache
---> 21fdc721f83e
Successfully built 21fdc721f83e
Successfully tagged modischfabrications/cutsolver:latest
Pushing index.docker.io/modischfabrications/cutsolver:latest...
Done!
Build finished``