mjishnu / pypdl Goto Github PK

I'm getting errors when trying to download some CSV files, even though there are no problems in the URL (tested via browser).

Is this something I'm doing wrong? Any tips on how to debug this?

Code:

from pypdl import Downloader
dl = Downloader()
dl.start('https://www.gov.br/anp/pt-br/centrais-de-conteudo/dados-abertos/arquivos/shpc/dsas/ca/ca-2004-01.csv')

Result:
ERROR:root:(ConnectionError) [Server Returned: Forbidden(403), Invalid URL]

• I encounter an issue like the image below

• being blocked indefinitely like this and unable to continue

• i using default:
  dl.start( url='http://example.com/file.txt', file_path='file.txt', segments=10, display=True, multithread=True, block=True, retries=0, mirror_func=None, etag=True )

Why init.py in repo ?
blank file ?

Can this repo be used to serve files to users over my fastapi or flask app? Or is it only for end users, not for serving whatsoever. Thanks for answering in advance my super dumb question.

Hi,

I was reviewing this project's code and came across what looks to be some distasteful code in create_segment_table. In

        if segment != (segments - 1):
            end -= 1  # [0-100, 100-200] -> [0-99, 100-200]
        # No segment_size+=1 for last setgment since final byte is end byte

end -= 1 should also be applied to the last segment; if the size is 200, the last byte is at index 199 (0-index, inclusive).

There is also careless use of floating points -- int(size / segments * segments) doesn't always equal size, e.g. int(109 / 11 * 11) returns 108. This problem is accidentally mitigated by the above problem. However, cases where the calculation is 2 (or greater) less can also occur, though you'll need very large files: int(9999999999999966 / 17 * 17) returns 9999999999999964.

Setting the last segment's end as size - 1 is acceptable, though the sane thing to do is to also have partition_size be an int.

There library looks promising.

As the title says, is it possible to download multiple files (option to limit N threads) simultaneously with progress bar?

Would be happy to see some example in this.

Are there any plans to add unit tests to this project to ensure correct functionality?

(UnicodeEncodeError, 'charmap' codec can't encode characters in position 1-31: character maps to )

getting this error while downloading. btw I am on windows.

hi, sorry for my bad English,
i have this problem : the program that i run is stuck "Speed: 0.00 MB/s, ETA: 99:59:59 " for hours,
then i tried this :
"timeout: (number or tuple, Optional) A number, or a tuple, indicating how many seconds to wait for the client to make a connection and/or send a response. Default is 20 seconds."
but somehow this do not work in python script, only work in API
i don't understand how to use this API,
can you give me tutorial how to use "timeout" feature?

I am getting a ModuleNotFoundError: No module named 'utls' while trying to from pypdl import Downloader

Looking at the source code and commit history, I think this is caused by removing the dot in main.py

from .utls import Multidown, Singledown, timestring

Can you guys help take a look? From my experience, it should work without dot, so I am not sure what the root cause is ...

Thanks,

I'm trying to use this package to download a files from a Gamevault server. This is the API endpoint that I am using. The auth seems to work and it will download the file segments but once it hits 99% it halts and just sits there.

from pypdl import *
from aiohttp import BasicAuth
import logging

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger()

def main():
    # Using Basic Authentication
    username = "test"
    password = "asdfghjkl"
    headers = {"User-Agent":"My cool Program/0.1"}
    auth = BasicAuth(username, password)
    print(f"Auth object: {auth}")
    dl = Pypdl(auth=auth, headers=headers)
    # Start the download
    dl.start(url='http://192.168.0.200:5006/api/games/1/download', multisegment=True, block=True)

if __name__ == '__main__':
    main()

would you happen to have any insight on how I can get this to work?

Steps to reproduce:

1. Run the example usage:

from pypdl import Pypdl

dl = Pypdl()
dl.start('http://example.com/file.txt')

Expected result:

Download starts and completes.

Actual result:

SyntaxError: f-string: expecting '}'

in pypdl/factory.py", line 219,

    download_stats = f"Downloading...{" " * 95}\n"

The new f-string usage is only available in Python 3.12.

Environment:

• Python 3.9.18
• pypdl 1.4.0