Git Product home page Git Product logo

Comments (8)

thecoolreaver avatar thecoolreaver commented on July 27, 2024 1
~# python3
Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getrecursionlimit()
1000
>>>

from falocalrepo.

thecoolreaver avatar thecoolreaver commented on July 27, 2024

fa.log

from falocalrepo.

MatteoCampinoti94 avatar MatteoCampinoti94 commented on July 27, 2024

Wow, never thought there would be a submission that broke the recursion limit o.o

Could you please run these two lines in a Python prompt?

import sys
sys.getrecursionlimit()

I can solve the issue directly in the FAAPI library, but I cannot remove the recursion limit. There is a sys command to do it, but there is no way to predict whether it will cause issues :(

from falocalrepo.

MatteoCampinoti94 avatar MatteoCampinoti94 commented on July 27, 2024

The fox will take longer than I thought. Because of how Python calculates its stack, the recursion fails after less than 500 comments in a chain, that specific function (_remove_recursion) cannot be avoided or the dict method will keep going in circles because comments contain an id link to the parent submission/journal.

Never encountered a comments chain that long before so it never popped up.

For the moment the solution is to edit the library manually to remove the problematic line.

Change line 220 in comment.py to:

        yield "comments", []  # [dict(_remove_recursion(c)) for c in self.comments]

It's only a temporary fix, but it will stop the recursion error for now.

from falocalrepo.

thecoolreaver avatar thecoolreaver commented on July 27, 2024

I applied that patch but I still get an error.

# falocalrepo download submissions 8601571
1/1 0008601571 [#######][#]
Trace written to /mnt/gd-data/FALocalRepo/falocalrepo/FA.log

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
8 <module> /usr/local/bin/falocalrepo
> sys.exit(main())

59 main /usr/local/lib/python3.10/dist-packages/falocalrepo/__main__.py
> exit(app.main(standalone_mode=False) or 0)

1055 main /usr/local/lib/python3.10/dist-packages/click/core.py
> rv = self.invoke(ctx)

1657 invoke /usr/local/lib/python3.10/dist-packages/click/core.py
> return _process_result(sub_ctx.command.invoke(sub_ctx))

1657 invoke /usr/local/lib/python3.10/dist-packages/click/core.py
> return _process_result(sub_ctx.command.invoke(sub_ctx))

1404 invoke /usr/local/lib/python3.10/dist-packages/click/core.py
> return ctx.invoke(self.callback, **ctx.params)

760 invoke /usr/local/lib/python3.10/dist-packages/click/core.py
> return __callback(*args, **kwargs)

26 new_func /usr/local/lib/python3.10/dist-packages/click/decorators.py
> return f(get_current_context(), *args, **kwargs)

316 download_submissions /usr/local/lib/python3.10/dist-packages/falocalrepo/console/download.py
> downloader.download_submissions(list(submission_id))

783 download_submissions /usr/local/lib/python3.10/dist-packages/falocalrepo/downloader.py
> self.download_submission(submission_id,

406 download_submission /usr/local/lib/python3.10/dist-packages/falocalrepo/downloader.py
> save_comments(self.db, submissions_table, submission.id, submission.comments,

126 save_comments /usr/local/lib/python3.10/dist-packages/falocalrepo/downloader.py
> for comment in filter(lambda c: not c.hidden, flatten_comments(comments)):

161 flatten_comments /usr/local/lib/python3.10/dist-packages/faapi/comment.py
> return sorted({c for c in [r for c in comments for r in [c, *flatten_comments(c.replies)]]})

161 <listcomp> /usr/local/lib/python3.10/dist-packages/faapi/comment.py
> return sorted({c for c in [r for c in comments for r in [c, *flatten_comments(c.replies)]]})

.... repeats for awhile ....

161 flatten_comments /usr/local/lib/python3.10/dist-packages/faapi/comment.py
> return sorted({c for c in [r for c in comments for r in [c, *flatten_comments(c.replies)]]})

161 <listcomp> /usr/local/lib/python3.10/dist-packages/faapi/comment.py
  Flattens a list of comments. Replies are not modified.

  :param comments: A list of Comment objects (flat or tree-structured)
  :return: A flat date-sorted (ascending) list of comments
  """
> return sorted({c for c in [r for c in comments for r in [c, *flatten_comments(c.replies)]]})



c: 62102234 THE-DALEK-SUPREME
OP

RecursionError:
maximum recursion depth exceeded

from falocalrepo.

MatteoCampinoti94 avatar MatteoCampinoti94 commented on July 27, 2024

You also have to use the --no-comments option. The FAAPI patch makes it so the comments tree is not traversed when casting the Submission object to a dict. The --no-comments options makes FALocalRepo ignore comments. this way neither _remove_recursion nor flatten_comments are called and the recursion limit isn't reached.

from falocalrepo.

MatteoCampinoti94 avatar MatteoCampinoti94 commented on July 27, 2024

I fixed the issue FurryCoders/FAAPI@97944f1!

Had to completely rewrite the tree-parser in a procedural style, which I really do not like, but alas that's the only solution :(

I will push a new FAAPI release soon and then a patch for FALocalRepo.

from falocalrepo.

MatteoCampinoti94 avatar MatteoCampinoti94 commented on July 27, 2024

Issue is completely fixed and I have also solved another problem caused by the new OP (Original Poster) tags that FA now adds to comments from the author of the submission/journal.

You can download release 4.4.6 to get the fixes :)

from falocalrepo.

Related Issues (12)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.