Git Product home page Git Product logo

Comments (7)

tgamblin avatar tgamblin commented on May 26, 2024

I added restrictions on master so that pull requests are required to commit there, and tests are required to pass before merging (that includes @bhatele). This should keep master clean so we do not have to fix other peoples' bugs if the tests would've caught them.

from hatchet.

tgamblin avatar tgamblin commented on May 26, 2024

I'm not sure where the tests started failing but I get 2 callgrind failures:

_______________________________________________________________________________ test_graphframe _______________________________________________________________________________

self = Index([], dtype='object'), key = 'node', method = None, tolerance = None

    @Appender(_index_shared_docs['get_loc'])
    def get_loc(self, key, method=None, tolerance=None):
        if method is None:
            if tolerance is not None:
                raise ValueError('tolerance argument only valid if using pad, '
                                 'backfill or nearest lookups')
            try:
>               return self._engine.get_loc(key)

/usr/local/lib/python3.7/site-packages/pandas/core/indexes/base.py:2657: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/index.pyx:108: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/index.pyx:132: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/hashtable_class_helper.pxi:1601: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   KeyError: 'node'

pandas/_libs/hashtable_class_helper.pxi:1608: KeyError

During handling of the above exception, another exception occurred:

calc_pi_callgrind_dot = '/private/var/folders/0s/q_y0zhfj6xdd5n7rn780zz6h001qr7/T/pytest-of-gamblin2/pytest-10/test_graphframe1/callgrind.dot.64042.0.1'

    def test_graphframe(calc_pi_callgrind_dot):
        """Sanity test a GraphFrame object with known data."""
    
        gf = GraphFrame()
>       gf.from_gprof_dot(str(calc_pi_callgrind_dot))

tests/callgrind.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hatchet/graphframe.py:56: in from_gprof_dot
    self.inc_metrics) = reader.create_graphframe()
hatchet/gprof_dot_reader.py:121: in create_graphframe
    dataframe.set_index(index, drop=False, inplace=True)
/usr/local/lib/python3.7/site-packages/pandas/core/frame.py:4178: in set_index
    level = frame[col]._values
/usr/local/lib/python3.7/site-packages/pandas/core/frame.py:2927: in __getitem__
    indexer = self.columns.get_loc(key)
/usr/local/lib/python3.7/site-packages/pandas/core/indexes/base.py:2659: in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
pandas/_libs/index.pyx:108: in pandas._libs.index.IndexEngine.get_loc
    ???
pandas/_libs/index.pyx:132: in pandas._libs.index.IndexEngine.get_loc
    ???
pandas/_libs/hashtable_class_helper.pxi:1601: in pandas._libs.hashtable.PyObjectHashTable.get_item
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   KeyError: 'node'

pandas/_libs/hashtable_class_helper.pxi:1608: KeyError
_________________________________________________________________________ test_read_calc_pi_database __________________________________________________________________________

calc_pi_callgrind_dot = '/private/var/folders/0s/q_y0zhfj6xdd5n7rn780zz6h001qr7/T/pytest-of-gamblin2/pytest-10/test_read_calc_pi_database1/callgrind.dot.64042.0.1'

    def test_read_calc_pi_database(calc_pi_callgrind_dot):
        """Sanity check the GprofDot reader by examining a known input."""
        reader = GprofDotReader(str(calc_pi_callgrind_dot))
    
        list_roots = reader.create_graph()
        root_names = []
        for root in list_roots:
            root_names.append(root.callpath[-1])
    
>       assert all(rt in root_names for rt in roots)
E       assert False
E        +  where False = all(<generator object test_read_calc_pi_database.<locals>.<genexpr> at 0x1194aab10>)

tests/callgrind.py:44: AssertionError

Some of the pandas errors are a bit hard to interpret because it's written partially in cython, and I guess there's no context for the cython frames.

I'm not sure when the python 3 tests last worked, though -- @bhatele?

from hatchet.

slabasan avatar slabasan commented on May 26, 2024

Thanks @tgamblin!

I started working on a fix for this yesterday and just pushed up changes to my branch here: https://github.com/slabasan/hatchet/tree/fix-travis-ci. Using regular expressions containing hexadecimals (e.g., \xc3\x97) in Python2 and Python3 have different behaviors. I could not find an elegant way of handling both cases with a single call. The branch I have checks for the Python version and handles them accordingly -- that is, using byte representation if Python3. Open to ideas on a more elegant solution.

from hatchet.

tgamblin avatar tgamblin commented on May 26, 2024

Hey @slabasan: that's a nice catch. I see what you're talking about now with the hex literals. I think we should figure out a way to simplify that code, though. Are you around tomorrow?

As it stands, the file looks pretty unmaintainable. In particular, I think the real issue here is that we're parsing dot lines with a really complicated regex -- there are simpler (and more correct) ways to do this.

I'm curious why @bhatele isn't parsing using pydot or another library before reading the data in.

from hatchet.

slabasan avatar slabasan commented on May 26, 2024

I am switching the gprof reader to use pydot here: https://github.com/slabasan/hatchet/tree/pydot-for-dot-parsing. Two challenges: (1) pydot imports the beginning keyword lines as nodes, and is included in the list of roots. The keywords are not rows in the dataframe, so I get an error KeyError: 'the label [graph] is not in the [index]'; (2) this code works for Python3, pydot v1.4.1, pyparsing v2.3.0, and setuptools v40.6.3. Using python2 has the following error UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 305: ordinal not in range(128). I'm not sure trying to specify different version dependencies for pydot, pyparsing, and setuptools` depending on the Python version a good path forward.

from hatchet.

slabasan avatar slabasan commented on May 26, 2024

This should be fixed with #20

from hatchet.

bhatele avatar bhatele commented on May 26, 2024

Fixed by #25

from hatchet.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.