Comments (9)
Which version (pydeps --version
) are you using?
I don't really understand the difference between --max-bacon and --noise-level.
--max-bacon
determines how many "hops" from the root that should be included in the graph (max bacon of 6 will likely include the entire universe.. -- the name comes from the game "6 degrees of Kevin Bacon"), while --noise-level=n
excludes nodes that have more than n
incoming/outgoing arrows. --max-bacon
increases the size of the graph, --noise-level
decreases the size of the graph.
If I remember correctly, then graphviz/dot has a limitation of 1000 nodes (or maybe it was 10 000), and if you hit that limit there will be no output.
I'm wondering if I can attribute a color for a folder level.
The color assignment is done here https://github.com/thebjorn/pydeps/blob/master/pydeps/colors.py#L47 (it uses the part before the first dot in the module name and adjusts saturation depending on how many arrows go in/out of the node). Early versions of pydeps tried to use different colors for different modules, but my implementation(s) didn't work that well. Maybe you'll have better luck..?
can I import and use pydeps in a python script ?
Yes, check e.g. https://github.com/thebjorn/pydeps/blob/master/tests/test_cli.py#L25
I run pydeps on my file with this following command line (with different parameters) :
pydeps JobScriptTemplate.py --max-bacon 6 --noise-level 2 --pylib
I would suggest starting without any flags, i.e. pydeps JobScriptTemplate.py
, although the preferred method is to call it on the root of the module: pydeps commons
. Then you can increase --max-bacon
if you don't reach all the modules you want to find. Changing --noise-level
is almost never needed (setting it to 2 will remove most nodes - the default is 200).
PS: Python is not designed to support running scripts from sub-folders (mostly because Guido doesn't like it), so you're fighting against the system when you try to run a script that is two levels deep. In particular, you need to use absolute imports (like you're doing: from commons.core.sql.TableJobAdaptator import TableJobAdaptator
) instead of relative imports (from ..sql.TableAdapter import ..
). The implications for pydeps is that from a.b.c import d
loads a.__init__
, b.__init__
, c.__init__
, and d
. If you try to import from sub-packages you'll easily get circular imports.
The solution is to create a myproj/setup.py
file with entry points to the file:function you want to use as a script (like pydeps does: https://github.com/thebjorn/pydeps/blob/master/setup.py#L47). After you run pip install -e myproj
the command is available directly from the shell, and you're free to use relative imports again.
from pydeps.
Thank you for your quick reply.
I'm using pydeps v1.7.3. I will try your suggestions and I come back to you if I encounter any issue.
from pydeps.
I tried without any flag and pydeps finds nothing for JobScriptTemplate.py
. Then I tried with --max-bacon (0 to 30) and it's the same, I obtain an empty graph.
from pydeps.
Does pydeps --show-dot JobScriptTemplate.py
give any output? How about pydeps --show-raw-deps JobScriptTemplate.py
?
from pydeps.
With pydeps --show-dot
. I obtain an empty graph and this :
digraph G {
concentrate = true;
rankdir = TB;
node [style=filled,fillcolor="#ffffff",fontcolor="#000000",fontname=Helvetica,fontsize=10];
}
With pydeps --show-raw-deps
. I obtain an empty graph too and this :
{
"__main__": {
"bacon": 0,
"imports": [
"commons",
"commons.core",
"commons.core.launcher"
],
"name": "__main__",
"path": null
},
"commons": {
"bacon": 1,
"imported_by": [
"__main__"
],
"name": "commons",
"path": "/usr/local/REPET_linux-x64-2.5/commons/__init__.py"
},
"commons.core": {
"bacon": 1,
"imported_by": [
"__main__"
],
"name": "commons.core",
"path": "/usr/local/REPET_linux-x64-2.5/commons/core/__init__.py"
},
"commons.core.launcher": {
"bacon": 1,
"imported_by": [
"__main__"
],
"name": "commons.core.launcher",
"path": "/usr/local/REPET_linux-x64-2.5/commons/core/launcher/__init__.py"
}
}
from pydeps.
Hmm.. I'm afraid I can't reproduce what you're seeing. I've created the following directory tree:
(dev) go|c:\srv\tmp\pydeps28> tree commons
commons
|-- __init__.py
|-- core
| |-- __init__.py
| |-- checker
| | |-- RepetException.py
| | `-- __init__.py
| |-- launcher
| | |-- JobScriptTemplate.py
| | `-- __init__.py
| `-- sql
| |-- DbFactory.py
| |-- Job.py
| |-- TableJobAdaptator.py
| `-- __init__.py
|-- launcher
| `-- __init__.py
`-- tools
`-- __init__.py
6 directories, 12 files
with the following contents:
(dev) go|c:\srv\tmp\pydeps28> cat pydeps28.yaml
commons:
__init__.py: ''
core:
__init__.py: ''
checker:
__init__.py: ''
RepetException.py: |
class RepetException: pass
launcher:
__init__.py: ''
JobScriptTemplate.py: |
import os
import sys
import time
import shutil
from commons.core.checker.RepetException import RepetException
from commons.core.sql.TableJobAdaptator import TableJobAdaptator
from commons.core.sql.DbFactory import DbFactory
from commons.core.sql.Job import Job
sql:
__init__.py: ''
DbFactory.py: |
class DbFactory: pass
Job.py: |
class Job: pass
TableJobAdaptator.py: |
class TableJobAdaptator: pass
launcher:
__init__.py: ''
tools:
__init__.py: ''
If you grab yamldirs==1.1.7
(published a few seconds ago), you can save the yaml in a file (e.g. myfile.yaml
) and reconstitute the directory with yamldirs myfile.yaml
.
With the above, and the following command:
(dev) go|c:\srv\tmp\pydeps28> pydeps commons
or commands (I set the PYTHONPATH
to the directory containing commons
so Python will find the imports):
(dev) go|c:\srv\tmp\pydeps28> set PYTHONPATH=c:\srv\tmp\pydeps28
(dev) go|c:\srv\tmp\pydeps28> cd commons\core\launcher
(dev) go|c:\srv\tmp\pydeps28\commons\core\launcher> pydeps JobScriptTemplate.py
I can look at it further if you can change the code above to demonstrate your problem, and then attach the output of yamldirs commons
.
from pydeps.
I tried your example and I obtained the same result.
For my case, I created a minimal example that reproduce the issue REPET_linux-x64-2.5.zip.
You'll notice that the script is working for commons/core/launcher/Launcher.py
but not for commons/core/launcher/JobScriptTemplate.py
.
from pydeps.
I'm guessing that is because you have syntax errors in that file:
(dev) go|c:\srv\tmp\REPET_linux-x64-2.5\REPET_linux-x64-2.5> python -c "import commons.core.launcher.JobScriptTemplate"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "commons\core\launcher\JobScriptTemplate.py", line 43
@@cmdStart@@
^
SyntaxError: invalid syntax
from pydeps.
You're absolutely right, it's @@cmdStart@@
and @@cmdFinish@@
that causes the error, I assume they exist because this script is used to generate python script that is submitted as a job (for a scheduler).Pydeps worked after removing them.
Thank you for your time.
from pydeps.
Related Issues (20)
- The cycles display mode shows useless information and does not work properly for larger projects. HOT 8
- [Feature Request] Specify module name instead of filename HOT 4
- Resolving relative dependencies? HOT 7
- Support for namespace packages HOT 1
- use of `--externals` results in TypeError: 'Config' object does not support item assignment HOT 1
- --dot-output option is not effective HOT 1
- Support output markdown mermaid HOT 1
- empty result with WARNING: SKIPPING ILLEGAL MODULE_NAME: D:.test.a HOT 2
- Support for specifying more than one `fname`
- 1.12.19: pitest fails in `tests/test_package_names.py::test_find_package_names` unit HOT 2
- `--exclude` option not working HOT 1
- Show only top-level imports HOT 1
- Exclude/include dependencies by filepath HOT 1
- Add type hints HOT 1
- Use of pydeps in CI to reject introduction of new import cycles? HOT 3
- Constrain output geometry / size HOT 2
- pydeps randomly generates compiled pyc files when examine python source file HOT 4
- Support for editable packages / direct_url.json HOT 10
- Different types of imports get disconnected. HOT 3
- Feature Request: Pydeps for pipelines HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pydeps.