soroco / pyce Goto Github PK
View Code? Open in Web Editor NEWEncrypted Python Execution
License: Apache License 2.0
Encrypted Python Execution
License: Apache License 2.0
encrypt_path returns a tuple of file location and key whereas the PYCEPathFinder.KEYS accepts a dictionary.
>>> encrypt_path('.')
[('./hello.pyce', '69b9f28ba960bb3a83abef44950c99baf427f671bc33cd112cf87e22259ab835'), ('./mod.pyce', 'b6bea595bea6b95098d9b9a4f98e17cb399c99d3cdd99daeab5078757ec65f3c')]
PYCEPathFinder.KEYS = {'hello.pyce':'69b9f28ba960bb3a83abef44950c99baf427f671bc33cd112cf87e22259ab835','mod.pyce':'b6bea595bea6b95098d9b9a4f98e17cb399c99d3cdd99daeab5078757ec65f3c'}
Hi,
thank you for releasing your work as open source!
I noticed that the dictionary key used to lookup the encryption key for a source file changes depending on which filesystem path I execute the code from.
When I execute the demo code everything works fine:
$PYTHON -c "from pyce import PYCEPathFinder; \
import sys; \
PYCEPathFinder.KEYS=dict(${KEYS}); \
sys.meta_path.insert(0, PYCEPathFinder); \
from pyce import hello; \
hello.hello()"
Hello World!
However, when I navigate one directory down and execute the same code I get a KeyError:
cd .. # change execution directory
$PYTHON -c "from pyce import PYCEPathFinder; \
import sys; \
PYCEPathFinder.KEYS=dict(${KEYS}); \
sys.meta_path.insert(0, PYCEPathFinder); \
from pyce import hello; \
hello.hello()"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 724, in exec_module
File "/pyce/pyce/_imports.py", line 73, in get_code
data = decrypt(data, PYCEPathFinder.KEYS[normcase(relpath(path))])
KeyError: 'pyce/pyce/hello.pyce'
As the relative path changes, it can no longer be looked up in the keys dictionary. You should be able to reproduce by inserting the cd ..
line into demo.sh as seen above.
How would you handle this? A naive solution would be to use absolute instead of relative paths but then the code would have to be deployed in the exact same location as where it was built.
So another idea I had was to use SHA-256 hashes of the contents of each encrypted source file as the lookup key instead of the filesystem path. This should make the key lookup location-independent, but would result in a slight increase in startup time as every file would need to be hashed once upon key lookup.
Instead of using filesystem paths in the key list as before...
[('pyce/hello.pyce', '43908f4464e86bfabaacbd1a6b5f0948f43e69ee1c050b2e131087733cd98707')]
... the keys would look something like this when using hashing:
[('ed968e840d10d2d313a870bc131a4e2c311d7ad09bdf32b3418147221f51a6e2', '43908f4464e86bfabaacbd1a6b5f0948f43e69ee1c050b2e131087733cd98707')]
... where ed968e840d10d2d313a870bc131a4e2c311d7ad09bdf32b3418147221f51a6e2
would be the SHA-256 hexdigest()
of the contents of pyce/hello.pyce
.
Let me know what you think, I am happy to try and make a contribution.
Thanks for @pradyunsg for bringing this up: f9fb6ad
The hard dependency on cryptography 2.3.1 is a serious problem. I can install pyce successfully by ignoring dependencies.
the requirement on < 3.7 seems overly cautious. Or does python regularly break stuff from version to version?
# pip install --ignore-requires-python --no-deps pyce
but will it work? there are no included test cases.
Otherwise, many problems ensue, which I try to document below:
python: 3.9.7, 3.8.10
pip: 20.3.4
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/pyce/ HTTP/1.1" 304 0
Link requires a different Python (3.9.7 not in: '>=3.6, <=3.7'): https://files.pythonhosted.org/packages/0a/b8/4730349f86114d4ad6410e2d363855fefa42e37218886c833154cbae824d/pyce-1.0.0-py3-none-any.whl#sha256=33d77ebe84414bb7ae23229bea9fe3ec6a492d618e8f836a1867f5aae69712a6 (from https://pypi.org/simple/pyce/) (requires-python:>=3.6, <=3.7)
Link requires a different Python (3.9.7 not in: '>=3.6, <=3.7'): https://files.pythonhosted.org/packages/40/cf/9e4f0a388e6bea327af5f5fcba0002c9c64bc77d6d08ba02812669008111/pyce-1.0.0.tar.gz#sha256=0eeea00d70385740d59ecbb1482101973fee2ac876203155f00f6a0e6f34426e (from https://pypi.org/simple/pyce/) (requires-python:>=3.6, <=3.7)
Link requires a different Python (3.9.7 not in: '>=3.7, <3.8'): https://files.pythonhosted.org/packages/39/79/cba14351f49d92d9ece1a5d25700808304a13ba7c57c275397ff7945b894/pyce-2.0.0-py3-none-any.whl#sha256=3a679de025d444dc66250ed73a52f61ef2930ff595b69ba471d6ba438253f877 (from https://pypi.org/simple/pyce/) (requires-python:>=3.7, <3.8)
Link requires a different Python (3.9.7 not in: '>=3.7, <3.8'): https://files.pythonhosted.org/packages/4e/cd/9d8d9299e8ad81a0fa0352d242f269a2789228c3976f622c9364ef7e2723/pyce-2.0.0.tar.gz#sha256=edb5d4d44ebec0453f76d7930c85c92af2c036b6ecd5d1ad83f6c3213c7be758 (from https://pypi.org/simple/pyce/) (requires-python:>=3.7, <3.8)
Given no hashes to check 0 links for project 'pyce': discarding no candidates
ERROR: Could not find a version that satisfies the requirement pyce
ERROR: No matching distribution found for pyce
--ignore-requires-python
gets me passed the above error.
But now I have cffi problems..
Collecting cffi!=1.11.3,>=1.7
Downloading cffi-1.15.0.tar.gz (484 kB)
|██████████▏ | 153 kB 3.8 MB/s eta 0:00:01ERROR: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
crit = self.state.criteria[name]
KeyError: 'cffi'
Cffi is required by 'cryptography'. But I can install this with alpine:
apk add py3-cffi
apk add py3-cryptography
(success)
Now, the pip install tries to replace the cryptography module (3.x) with 2.3.1, and of course it wants to compile.
pip install --ignore-requires-python pyce -v
...
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fno-semantic-interposition -g -fno-semantic-interposition -g -fno-semantic-interposition -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/python3.8 -c build/temp.linux-aarch64-3.8/_openssl.c -o build/temp.linux-aarch64-3.8/build/temp.linux-aarch64-3.8/_openssl.o -Wconversion -Wno-error=sign-conversion
build/temp.linux-aarch64-3.8/_openssl.c:57:10: fatal error: Python.h: No such file or directory
57 | #include <Python.h>
| ^~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
Running setup.py install for cryptography ... error
...
Replacing /usr/lib/python3.8/site-packages/cryptography from /usr/lib/python3.8/site-packages/~ryptography
Replacing /usr/lib/python3.8/site-packages/cryptography-3.3.2-py3.8.egg-info from /usr/lib/python3.8/site-packages/~ryptography-3.3.2-py3.8.egg-info
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-udngud2t/cryptography_04c4441d2e864f5d807f3ad1be83458c/setup.py'"'"'; __file__='"'"'/tmp/pip-install-udngud2t/cryptography_04c4441d2e864f5d807f3ad1be83458c/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-jo2yh9iw/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.8/cryptography Check the logs for full command output.
Exception information:
Traceback (most recent call last
(tried many versions, usually failing with)
/usr/lib/gcc/aarch64-alpine-linux-musl/10.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: cannot find -lssp_nonshared
Also, I'm not keen on creating an entire development distribution for a docker container.
ie you can simply
#!/usr/bin/python3
from pyce._crypto import decryptf
with open('output.pyc', 'wb') as output:
output.write(decryptf('./__init__.pyce', '3e8a6f52eb5b41e0883c594dc026c70aa04258790ca71259d9d1c1bf47cfc0aa'))
Then use uncompyle6 output.pyc
to get full source..
if you have to use 3rd party to encrypt the source, what is the point in this module?
Running pipenv's security check shows that the used version has a vulnerability in cryptography==2.1.4.
$ pipenv check
Checking PEP 508 requirements�
Passed!
Checking installed package safety�
36351: cryptography >=1.9.0,<2.3 resolved (2.1.4 installed)!
python-cryptography versions >=1.9.0 and <2.3 did not enforce a minimum tag length for finalize_with_tag API. If a user did not validate the input length prior to passing it to finalize_with_tag an attacker could craft an invalid payload with a shortened tag (e.g. 1 byte) such that they would have a 1 in 256 chance of passing the MAC check. GCM tag forgeries can cause key leakage.
I have a module that requires the ' __ file __ ' variable for loading a bunch of files. When I run my program without any encryption this works flawlessly. However, after encrypting, the __ file __ attribute is not defined. After digging through 'dir()' a bit, I found that two attribute - ' __ file __ ' and ' __ cached __ ' get lost somewhere after encryption. Is anybody else facing such a problem? Is there a workaround for this?
pip3/pip install pyce
doesn't work on Debian:root@milani:~/Desktop# pip install pyce
ERROR: Could not find a version that satisfies the requirement pyce (from versions: none)
ERROR: No matching distribution found for pyce
easy_install-3.6 pyce
is using python 3.8.5 for installation:root@milani:~/Desktop# easy_install-3.6 pyce
WARNING: The easy_install command is deprecated and will be removed in a future version.
Searching for pyce
Reading https://pypi.org/simple/pyce/
Downloading https://files.pythonhosted.org/packages/39/79/cba14351f49d92d9ece1a5d25700808304a13ba7c57c275397ff7945b894/pyce-2.0.0-py3-none-any.whl#sha256=3a679de025d444dc66250ed73a52f61ef2930ff595b69ba471d6ba438253f877
Best match: pyce 2.0.0
Processing pyce-2.0.0-py3-none-any.whl
Installing pyce-2.0.0-py3-none-any.whl to /usr/local/lib/python3.8/dist-packages
Adding pyce 2.0.0 to easy-install.pth file
Installed /usr/local/lib/python3.8/dist-packages/pyce-2.0.0-py3.8.egg
Processing dependencies for pyce
Searching for cryptography==2.3.1
Reading https://pypi.org/simple/cryptography/
Downloading https://files.pythonhosted.org/packages/59/32/92cade62c645756a83598edf56289e9b19aae5370642a7ce690cd06bc72f/cryptography-2.3.1-cp34-abi3-manylinux1_x86_64.whl#sha256=8229ceb79a1792823d87779959184a1bf95768e9248c93ae9f97c7a2f60376a1
Best match: cryptography 2.3.1
Processing cryptography-2.3.1-cp34-abi3-manylinux1_x86_64.whl
Installing cryptography-2.3.1-cp34-abi3-manylinux1_x86_64.whl to /usr/local/lib/python3.8/dist-packages
Adding cryptography 2.3.1 to easy-install.pth file
Installed /usr/local/lib/python3.8/dist-packages/cryptography-2.3.1-py3.8-linux-x86_64.egg
Finished processing dependencies for pyce
easy_install pyce
is using Python 2.7.15 for installation:root@milani:~/Desktop# easy_install pyce
WARNING: The easy_install command is deprecated and will be removed in a future version.
Searching for pyce
Reading https://pypi.org/simple/pyce/
Downloading https://files.pythonhosted.org/packages/4e/cd/9d8d9299e8ad81a0fa0352d242f269a2789228c3976f622c9364ef7e2723/pyce-2.0.0.tar.gz#sha256=edb5d4d44ebec0453f76d7930c85c92af2c036b6ecd5d1ad83f6c3213c7be758
Best match: pyce 2.0.0
Processing pyce-2.0.0.tar.gz
Writing /tmp/easy_install-uV_mN_/pyce-2.0.0/setup.cfg
Running pyce-2.0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-uV_mN_/pyce-2.0.0/egg-dist-tmp-IUDJVP
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 2321, in main
**kw
File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 424, in run
self.easy_install(spec, not self.no_deps)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 685, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 711, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 896, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1164, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1150, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-uV_mN_/pyce-2.0.0/setup.py", line 32, in <module>
File "/tmp/easy_install-uV_mN_/pyce-2.0.0/pyce/__init__.py", line 36, in <module>
File "/tmp/easy_install-uV_mN_/pyce-2.0.0/pyce/_crypto.py", line 52
def __init__(self, message: str) -> None:
^
SyntaxError: invalid syntax
Hi guys,
A customer is sometimes (rarely) getting the error ImportError: bad magic number
, just wondering if anyone else has experienced this... and come up with a solution?
Example conflict when installing pyce 1.0.0 and paramiko through pipenv:
Could not find a version that matches cryptography==2.1.4,>=2.2.
Could we change the strict version to >=2.1.4, <3.0.0
?
Within pyce/_imports.py...
from importlib._bootstrap_external import (_compile_bytecode, _validate_bytecode_header)
fails to execute because the library importlib packaged with Python 3.7.0 lacks the function '_validate_bytecode_header'.
For reference, here is the latest code for importlib: https://github.com/python/cpython/blob/3.7/Lib/importlib/_bootstrap_external.py
When the code changed: python/cpython@42aa93b#diff-42218ca77825a5fef3c3d62a9f6b1304
On some systems, and during development, we may need to use a custom Python runtime. For example, Ubuntu 18.04 does not ship Python 3.7, but this project is now pinned to that version of Python.
Thanks to @pradyunsg for submitting this enhancement: https://github.com/soroco/pyce/pull/15/files
I'm entertaining the idea of encrypting an entire package. Preferably I would like to have it under a single key, but it does not appear to be the intended use case.
What I do have at the moment is encryption per file and I am able to import each module with its individual key. However, I am only able to do this if I set zip_safe=False
in my setup.py
. Ideally I would like to have it zip_safe
. All my attempts throws an ImportError
Goals:
sdist
with only library files (no need for test / demo files)src
folder and test
or demo
folderThanks to @pradyunsg for jumpstarting this: https://github.com/soroco/pyce/pull/16/files
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.