0.2.3 performance issue about freezegun HOT 20 CLOSED

ping. The problem is urgent. I have about 0.5 sec per api.py:223(start) call. It is called 127 times, so total time is ~1 minute only for freezegun start.

from freezegun.

I wrote my own simple tool https://github.com/pokidovea/immobilus It works on another principle. it is much faster.

from freezegun.

PRs welcome

from freezegun.

"Me too" -- I am using 0.3.9 and am observing something like 30ms per call to the context manager, making my test suite multiples slower. I am hoping the previous PR will fix this.

from freezegun.

Thanks for reporting.

I'm having trouble reproducing this. When I save the above code as tester.py and run python -m cProfile tester.py, I get the following:

0.2.3: ~0.063 seconds
0.2.2: ~0.065 seconds

It definitely varies a bit, but I'm not seeing the 100x change you are seeing. Is the code you have listed the actual code you are running?

from freezegun.

Hi, using cProfile gives me the same results as you got, however the tests were done on ipython with line_profiler like this:

%load_ext line_profiler
%prun test()

Also all my project tests that call freeze_time got much slower on 0.2.3 (and 0.2.5).
I managed to duplicate this on 2 computers with Ubuntu 14.04 OS.

from freezegun.

Okay, so the slowdown is a result of #64. We need to iterate through all the attributes of a module to see if there are any aliased datetimes, dates, or times. I can't currently think of a faster way to do this.

I just pushed a fix that will make stop a lot faster though. We now store all of the changes we make and then just revert those. Can you give HEAD for a spin?

from freezegun.

Thanks for quick help, I'll give it a try on Friday.
As for making start faster - I have no idea yet, but I'll give it a though.

from freezegun.

I ran into this too. For anyone who can't wait for a fix, I wrote a drop-in alternative that's much faster (2 orders of magnitude on my machine) but only supports a subset of what freezegun does: https://github.com/simon-weber/python-libfaketime.

Some ideas on making freezegun faster:

• make the new behavior optional:
  • easiest fix I can think of
  • doesn't fix the problem for people who need aliased module support, though having folks find-and-replace is probably easier than adding freezegun support 😁
• abuse the garbage collector to make the search for aliased modules faster:
  • potentially a quick fix
  • I'm not sure if a) it'd work, or b) it'd even be much faster (since python probably doesn't keep a reverse index for all objects?)
  • pretty ridiculous
• change the replacement logic to install patches once, then leave them in place and dynamically turn them on and off:
  • probably the most robust solution, but might take some time to implement
  • introduces some overhead on non-mocked date/time calls. https://pypi.python.org/pypi/wrapt could alleviate some of that if needed
  • this is what https://github.com/venmo/btnamespace does (code)

Hope that helps!

from freezegun.

@simon-weber thanks for the suggestions! I am probably leaning toward the first one.

from freezegun.

@pokidovea did you develop a solution yet?

from freezegun.

I know "me too" comments aren't helpful, but I want to emphasize how significant of an issue this is. I saw Freezegun suggested for the "control time in tests" case in multiple places, so I installed it today and modified 3 of my tests to utilize it. Those 3 tests now run slower than all of my other tests combined, which includes over 20 tests involving extremely slow operations like generating and verifying Argon2 hashes.

This level of performance makes Freezegun completely unusable in any setup that runs tests often (which relies on making sure the entire set of tests finishes very quickly). It looks like an excellent library and I really wish I could use it, but the effect on my tests' running time is far too large.

from freezegun.

I think it's possible to keep a warm cache of modules in freezegun and avoid iterating over all modules over and over for each test. I am working on a patch (branch speed-up-module-search) and it sped up freezegun's own tests by almost 1/3.

My changeset is not finished yet, but can you give it a try and see if it improves your tests speed, @matino and @Deimos? Please run your tests against the branch speed-up-module-search.

from freezegun.

@hltbra Hmm, that branch just completely crashes for me at the moment. On Python 3.6:

>           result[mod_name] = (id(module), hashlib.md5(','.join(dir(module))).hexdigest(), attrs)
E           TypeError: Unicode-objects must be encoded before hashing

from freezegun.

I added some encoding to both places that do the hashing to fix that error so I could test it, and it's definitely considerably faster.

Previously, the first test using Freezegun took about 0.45s, with all other ones using it taking about 0.15s each.

After applying that patch, the first test takes about 0.25s, and then there doesn't seem to be much significant effect at all on any of the others using it.

So now it seems to just be a fairly small one-time cost, which is definitely way better. Thanks, I think that's definitely a much better approach if it's possible to implement it similar to that method.

from freezegun.

Sorry, @Deimos, I just pushed changes to that branch (refactoring and python3-compatibility). I am opening a PR and will tag you. Please fetch the latest changes and let me know if it works as is.

from freezegun.

In 0.3.9 I'm seeing about 0.15 seconds per freezegun decorator. For example, this takes 0.15 secs on the box I ran the tests on:

class TestFoo(TestCase):
    @freeze_time("2018-01-22 01:00:00")
    def test_bar(self):
        pass

That's some pretty significant overhead.

from freezegun.

I found that we were spending almost 30% of total test time in Freezegun 0.3.8.

As a quick hack workaround if you can't revert to 0.2.2 (which broke some tests for us), you can supply ignore=list(sys.modules.items()) to the freeze_time constructor (or write a wrapper that does this automatically). It's still quite slow because it iterates over sys.modules and ignores each one individually, but it's about 20x faster than actually inspecting every module every time :\

from freezegun.

Closed with #175

from freezegun.

I believe this ticket was prematurely closed. See my PR that addresses the performance issues here: #260

I have a test case where we freeze over 2000 tests and my changes made this test suite go from 8 minutes to 23 seconds.

from freezegun.