Comments (7)
I added restrictions on master
so that pull requests are required to commit there, and tests are required to pass before merging (that includes @bhatele). This should keep master
clean so we do not have to fix other peoples' bugs if the tests would've caught them.
from hatchet.
I'm not sure where the tests started failing but I get 2 callgrind failures:
_______________________________________________________________________________ test_graphframe _______________________________________________________________________________
self = Index([], dtype='object'), key = 'node', method = None, tolerance = None
@Appender(_index_shared_docs['get_loc'])
def get_loc(self, key, method=None, tolerance=None):
if method is None:
if tolerance is not None:
raise ValueError('tolerance argument only valid if using pad, '
'backfill or nearest lookups')
try:
> return self._engine.get_loc(key)
/usr/local/lib/python3.7/site-packages/pandas/core/indexes/base.py:2657:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
pandas/_libs/index.pyx:108:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
pandas/_libs/index.pyx:132:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
pandas/_libs/hashtable_class_helper.pxi:1601:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E KeyError: 'node'
pandas/_libs/hashtable_class_helper.pxi:1608: KeyError
During handling of the above exception, another exception occurred:
calc_pi_callgrind_dot = '/private/var/folders/0s/q_y0zhfj6xdd5n7rn780zz6h001qr7/T/pytest-of-gamblin2/pytest-10/test_graphframe1/callgrind.dot.64042.0.1'
def test_graphframe(calc_pi_callgrind_dot):
"""Sanity test a GraphFrame object with known data."""
gf = GraphFrame()
> gf.from_gprof_dot(str(calc_pi_callgrind_dot))
tests/callgrind.py:28:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hatchet/graphframe.py:56: in from_gprof_dot
self.inc_metrics) = reader.create_graphframe()
hatchet/gprof_dot_reader.py:121: in create_graphframe
dataframe.set_index(index, drop=False, inplace=True)
/usr/local/lib/python3.7/site-packages/pandas/core/frame.py:4178: in set_index
level = frame[col]._values
/usr/local/lib/python3.7/site-packages/pandas/core/frame.py:2927: in __getitem__
indexer = self.columns.get_loc(key)
/usr/local/lib/python3.7/site-packages/pandas/core/indexes/base.py:2659: in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
pandas/_libs/index.pyx:108: in pandas._libs.index.IndexEngine.get_loc
???
pandas/_libs/index.pyx:132: in pandas._libs.index.IndexEngine.get_loc
???
pandas/_libs/hashtable_class_helper.pxi:1601: in pandas._libs.hashtable.PyObjectHashTable.get_item
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E KeyError: 'node'
pandas/_libs/hashtable_class_helper.pxi:1608: KeyError
_________________________________________________________________________ test_read_calc_pi_database __________________________________________________________________________
calc_pi_callgrind_dot = '/private/var/folders/0s/q_y0zhfj6xdd5n7rn780zz6h001qr7/T/pytest-of-gamblin2/pytest-10/test_read_calc_pi_database1/callgrind.dot.64042.0.1'
def test_read_calc_pi_database(calc_pi_callgrind_dot):
"""Sanity check the GprofDot reader by examining a known input."""
reader = GprofDotReader(str(calc_pi_callgrind_dot))
list_roots = reader.create_graph()
root_names = []
for root in list_roots:
root_names.append(root.callpath[-1])
> assert all(rt in root_names for rt in roots)
E assert False
E + where False = all(<generator object test_read_calc_pi_database.<locals>.<genexpr> at 0x1194aab10>)
tests/callgrind.py:44: AssertionError
Some of the pandas errors are a bit hard to interpret because it's written partially in cython
, and I guess there's no context for the cython
frames.
I'm not sure when the python 3 tests last worked, though -- @bhatele?
from hatchet.
Thanks @tgamblin!
I started working on a fix for this yesterday and just pushed up changes to my branch here: https://github.com/slabasan/hatchet/tree/fix-travis-ci. Using regular expressions containing hexadecimals (e.g., \xc3\x97) in Python2 and Python3 have different behaviors. I could not find an elegant way of handling both cases with a single call. The branch I have checks for the Python version and handles them accordingly -- that is, using byte representation if Python3. Open to ideas on a more elegant solution.
from hatchet.
Hey @slabasan: that's a nice catch. I see what you're talking about now with the hex literals. I think we should figure out a way to simplify that code, though. Are you around tomorrow?
As it stands, the file looks pretty unmaintainable. In particular, I think the real issue here is that we're parsing dot lines with a really complicated regex -- there are simpler (and more correct) ways to do this.
I'm curious why @bhatele isn't parsing using pydot
or another library before reading the data in.
from hatchet.
I am switching the gprof reader to use pydot
here: https://github.com/slabasan/hatchet/tree/pydot-for-dot-parsing. Two challenges: (1) pydot
imports the beginning keyword lines as nodes, and is included in the list of roots. The keywords are not rows in the dataframe, so I get an error KeyError: 'the label [graph] is not in the [index]'; (2) this code works for Python3, pydot v1.4.1, pyparsing v2.3.0, and setuptools v40.6.3. Using python2 has the following error
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 305: ordinal not in range(128). I'm not sure trying to specify different version dependencies for
pydot,
pyparsing, and
setuptools` depending on the Python version a good path forward.
from hatchet.
This should be fixed with #20
from hatchet.
Fixed by #25
from hatchet.
Related Issues (20)
- Graph Literal Validator
- Elide collapsed subtrees with triangle icons [Roundtrip] HOT 1
- Pan/Zoom on Trees [Roundrip] HOT 1
- Draw Trees to Utilize more Vertical Space [Roundtrip] HOT 2
- Collapse Trees by Default [Roundtrip]
- Add "inplace" to hatchet operations HOT 1
- Add bivariate color map to roundtrip HOT 1
- Make Roundtrip Javascript Loading More Dynamic / Change Javascript Development Workflow HOT 1
- Q: How to read in traces generated by Caliper? HOT 4
- Add a Reader for ROCProfiler HOT 2
- Hatchet multi-process problem when handling large graphs HOT 8
- build error for Python 3.9 HOT 1
- feature request: subtree aggregate, graphframe subtree HOT 4
- import multiprocess as mp HOT 8
- bug in cprofile reader
- Support for Perfetto Trace Format HOT 1
- Filter/squash error on HPCToolkit database: `AttributeError: 'int' object has no attribute '_hatchet_nid'` HOT 2
- build failure on python 3.11, longintrepr.h not found HOT 2
- Error Reading GProf Profiles HOT 3
- Sphinx Api docs section missing doc for submodules
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hatchet.