Git Product home page Git Product logo

packtpub-downloader's Introduction

PacktPub Downloader

Script to download all your PacktPub books inspired by https://github.com/ozzieperez/packtpub-library-downloader

Since PacktPub restructured their website packtpub-library-downloader became obsolete because the downloader used webscraping. So I figured out that now PacktPub uses a REST API. Then I found which endpoint to use for downloading books and made a simple script. Feel free to fork and PR to improve. Packtpub's API isn't documented :'(

Usage:

pip install -r requirements.txt
python main.py -e <email> -p <password> [-d <directory> -b <book file types> -s -v -q]
Example: Download books in PDF format
python main.py -e [email protected] -p p@ssw0rd -d ~/Desktop/packt -b pdf,epub,mobi,code

Docker integration

You must put your data in the .env file.

mv data.env-sample data.env

and replace the sample data with your login credentials.

docker-compose up

After the execution, you can see the content in the book directory.

Commandline Options

  • -e, --email = Your login email
  • -p, --password = Your login password
  • -d, --directory = Directory to download into. Default is "media/" in the current directory
  • -b, --books = Assets to download. Options are: pdf,mobi,epub,code
  • -s, --separate = Create a separate directory for each book
  • -v, --verbose = Show more detailed information
  • -q, --quiet = Don't show information or progress bars

Book File Types

  • pdf: PDF format
  • mobi: MOBI format
  • epub: EPUB format
  • code: Accompanying source code, saved as .zip files

I'm working on Python 3.6.0

packtpub-downloader's People

Contributors

burnhamg avatar keek avatar lmbringas avatar progressify avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

packtpub-downloader's Issues

random error durring download

ERROR (please copy and paste in the issue)
{'message': 'jwt expired', 'errorCode': 1000100, 'errorId': 'b7ff1925-fc4a-40f3-8cfd-b967144ced9d'}
401
Starting to download /home/sam_tunder/git/packtpub-downloader/packtpub/Go_Cookbook/Go_Cookbook.code
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 218, in main
download_book(filename, url)
File "main.py", line 104, in download_book
r = requests.get(url, stream=True)
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/sessions.py", line 519, in request
prep = self.prepare_request(req)
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/sessions.py", line 452, in prepare_request
p.prepare(
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/models.py", line 313, in prepare
self.prepare_url(url, params)
File "/home/sam_tunder/.local/lib/python3.8/site-packages/requests/models.py", line 387, in prepare_url
raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?

Broken login

Looks like PaktPub changed their login code. The script now fails on login. I noticed the website also doesn't have a "claim now" button, just an "Access Now" button.

You have 0 books

I added book to a playlist... But that was not enough.
How do i add a book to my library?

Invalid URL

The script works for the most part, but fails randomly. It will download many PDFs and ZIPs then suddenly fail with the error below. When I go to my media directory I will see a file 0 byte file size, delete it, run script with exact same parameters and it downloads just fine....until it randomly fails again with same error. If it means anything I am using Python 3.7 and I see you are working on 3.6.

File "C:\Python\Python37\lib\site-packages\requests\models.py", line 387, in prepare_url
raise MissingSchema(error)
equests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?

Error 'data'

Hello

I got the following error message. Anybody could help please?

Regards.

You are in !
You have 272 books
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=10&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=20&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=30&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=40&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=50&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=60&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=70&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=80&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=90&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=100&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=110&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=120&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=130&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=140&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=150&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=160&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=170&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=180&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=190&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=200&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=210&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=220&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=230&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=240&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=250&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=260&limit=10
https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=270&limit=10
**Error 'data'
You are in !
Traceback (most recent call last):
File "main.py", line 48, in get_url_book
return r.json()['data']
KeyError: 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 130, in
main(sys.argv[1:])
File "main.py", line 125, in main
url = get_url_book(user, book['productId'], book_assets)
File "main.py", line 52, in get_url_book
return r.json()['data']
KeyError: 'data'**

KeyError: 'count'

packtpub-downloader | You are in!
packtpub-downloader | Traceback (most recent call last):
packtpub-downloader | File "/app/main.py", line 226, in
packtpub-downloader | main(sys.argv[1:])
packtpub-downloader | File "/app/main.py", line 198, in main
packtpub-downloader | books = get_books(user, is_verbose=verbose, is_quiet=quiet)
packtpub-downloader | File "/app/main.py", line 41, in get_books
packtpub-downloader | print(f'You have {str(r.json()["count"])} books')
packtpub-downloader | KeyError: 'count'

books can't be downloaded if the filename contains symbol <?>

Starting to download G:\packt/AWS_Certified_Developer_???Associate_Guide-Second_Edition.epub
Traceback (most recent call last):
File "G:\packtpub-downloader-master\main.py", line 226, in
main(sys.argv[1:])
File "G:\packtpub-downloader-master\main.py", line 218, in main
download_book(filename, url)
File "G:\packtpub-downloader-master\main.py", line 103, in download_book
with open(filename, 'wb') as f:
OSError: [Errno 22] Invalid argument: 'G:\packt/AWS_Certified_Developer
???Associate_Guide-_Second_Edition.epub'

packtpub-downloader for python3 not working

HI I have run the following code using python3:
python3 main.py -e <myemail> -p <mypassword> -d ~/Desktop/packt -b pdf

getting this issue, but my email and password was correct. It have $ symbol for the password but it should not be the issue right ? anyone face this issue before ?


Error login,  check user and password
Traceback (most recent call last):
  File "main.py", line 226, in <module>
    main(sys.argv[1:])
  File "main.py", line 195, in main
    user = User(email, password)
  File "/home/ranjen/softwares/packtpub-downloader-master/user.py", line 24, in __init__
    self.header["Authorization"] = self.get_token()
  File "/home/ranjen/softwares/packtpub-downloader-master/user.py", line 39, in get_token
    print("Error {}".format(e))
NameError: name 'e' is not defined

bug, code not work

hi
i have just today error , before work well !?

Error login, check user and password
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 195, in main
user = User(email, password)
File "E:\packt\user.py", line 24, in init
self.header["Authorization"] = self.get_token()
File "E:\packt\user.py", line 39, in get_token
print("Error {}".format(e))
NameError: name 'e' is not defined

please fix bug
thank you

Download stop

If I run it:
You are in!
You have 1505 books
Getting list of books...
100%|████████████████████████| 150/150 [00:36<00:00, 4.19Pages/s]
Downloading books...
0%|
Starting to download /home/pr/Desktop/packt/Raspberry_Pi_for_Secret_Agents_-_Third_Edition.pdf

...and it stops and do nothing.

Some days ago it worked. :o

program crashes when filename contains /

whenever i try to download a file called media/Oracle_10g/11g_Data_and_Database_Management_Utilities.code
the program terminates with "no such file or directory", which kind of makes sense because filenames under linux can't contain a /, and the filename is set equal to the book name.

user token bad request

when user.py tries to execute get_token, requests.post with a correct username and password returns status code 400. The username and password however are correct.

'errorCode': 1000100

ERROR (please copy and paste in the issue)
{'message': 'jwt expired', 'errorCode': 1000100, 'errorId': 'e41ebd90-bdc2-4263-
aa2f-cdd7c9394156'}
401
Starting to download media/Building_an_RPG_with_Unity_5_x.code
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 218, in main
download_book(filename, url)
File "main.py", line 104, in download_book
r = requests.get(url, stream=True)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\sessions.py", line 519, in request
prep = self.prepare_request(req)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\sessions.py", line 462, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\models.py", line 313, in prepare
self.prepare_url(url, params)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\site-p
ackages\requests\models.py", line 387, in prepare_url
raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps y
ou meant http://?

NameError: name 'e' is not defined

This was working yesterday...

It would time out after downloading about 100 books but I could rerun it and it would start where it left off. Now I get the following error.

Is it possible the Packt just happened to change their API last night?

I tried using the "--email" option but it makes no difference.

Thank for all of your work in putting this together!

The error:

(base) packtpub-downloader-master>python main.py -e [email protected] -p XXXXXXX -d ~/Desktop/packt -b pdf,epub,mobi,code
Error login, check user and password
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 195, in main
user = User(email, password)
File "C:\Users\Burke\Documents\Python Scripts\packtpub-downloader-master\user.py", line 24, in init
self.header["Authorization"] = self.get_token()
File "C:\Users\Burke\Documents\Python Scripts\packtpub-downloader-master\user.py", line 39, in get_token
print("Error {}".format(e))
NameError: name 'e' is not defined

JWT token expiry/refresh will still prematurely end downloads

get_url_book(user, book_id, format) # call recursive

Whenever the token refresh is hit and the function returns, it will continue on to the error which is probably why issues #19 and #9 were also created.

Adding a 'return' at the beginning of this line will make it so that it returns the value of the recursively called function and exit the initially called function.
This will stop it from flowing onto the error message and returning the blank string, indicating a failure to retrieve the book URL.

message': 'No record Found', 'errorCode': 1001054 - Downloaded About 208 Books

Done in a docker container using the docker config files in the project.

packtpub-downloader | Requirement already satisfied: urllib3==1.24.1 in /usr/local/lib/python3.6/site-packages (from -r /app/requirements.txt (line 13))
packtpub-downloader | Requirement already satisfied: yarl==1.3.0 in /usr/local/lib/python3.6/site-packages (from -r /app/requirements.txt (line 14))
packtpub-downloader | You are using pip version 9.0.1, however version 20.3.1 is available.
packtpub-downloader | You should consider upgrading via the 'pip install --upgrade pip' command.
100%|██████████| 23/23 [00:05<00:00, 4.10Pages/s]

100%|██████████| 232/232 [07:01<00:00, 2.34s/Book]
packtpub-downloader | You are in!
packtpub-downloader | You have 232 books
packtpub-downloader | Getting list of books...
packtpub-downloader | Downloading books...
packtpub-downloader | ERROR (please copy and paste in the issue)
packtpub-downloader | {'message': 'No record Found', 'errorCode': 1001054, 'errorId': 'f5dd3fb4-77d8-4432-a425-6f62d2b781da'}
packtpub-downloader | 404
packtpub-downloader exited with code 0

Add a permissive LICENSE

Please add a LICENSE to your code, this will add the possibility to improve it.
Without a License we can only "read" your code and no one can make his modification... I suppose this is not your idea for this code.

To add a license follow :
https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository

Why to add a License :
https://opensource.stackexchange.com/questions/1720/what-can-i-assume-if-a-publicly-published-project-has-no-license

Choose a License : https://choosealicense.com/

P.S. generally I use the MIT license.

refugee from packtpub-library-downloader need feature compatability

First up thanks for taking the cause up.
second: packtpub-library-downloader used to create directories and store things inside it hence everything will have a seperate directory. any reason that appraoch was abandoned in this project. I have a huge library downloaded via packtpub-library-downloader but if i run this tool in same directory it will simply redownload everything and i endup getting a large folder with all files in single place.

Thanks for taking time to read the rant and thanks in advance if you end up doing the changes.

Error 'jwt expired'

I got an error

You are in!
ERROR (please copy and paste in the issue)
{'message': 'jwt expired', 'errorCode': 1000100, 'errorId': '61a76feb-f05c-4e4e-83bb-8e739d534730'}
401
Starting to download D:\Workspace\packtpub-downloader\ebook/Raspberry_Pi_for_Secret_Agents_-_Third_Edition.epub
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 218, in main
download_book(filename, url)
File "main.py", line 104, in download_book
r = requests.get(url, stream=True)
File "C:\Python\3\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Python\3\lib\site-packages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python\3\lib\site-packages\requests\sessions.py", line 519, in request
prep = self.prepare_request(req)
File "C:\Python\3\lib\site-packages\requests\sessions.py", line 462, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Python\3\lib\site-packages\requests\models.py", line 313, in prepare
self.prepare_url(url, params)
File "C:\Python\3\lib\site-packages\requests\models.py", line 387, in prepare_url
raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?

problem dowloading

I get the following error.
Traceback (most recent call last):
File "main.py", line 227, in
main(sys.argv[1:])
File "main.py", line 199, in main
books = get_books(user, is_verbose=verbose, is_quiet=quiet)
File "main.py", line 41, in get_books
print(f'You have {str(r.json()["count"])} books')
KeyError: 'count'
Can anyone please tell me how exactly it should be done. i.e.
python main.py -e email -p password -d ~/Desktop/packt -b pdf
What should be after -b, is it file name or just pdf.

Get-Token function is broken

Thank you for the great work, the script worked like a charm but since few days the website has been upgraded and seems now to use ReCaptcha to secure the access.

Dumping the request JSON response in get-token function in user.py :

**{'message': 'The recaptcha solution is missing.', 'errorCode': 7000035, 'errorId': 'b494baa7-255c-4c50-8864-a45ef32d3b0e'}
<Response [400]>**
Error login,  check user and password
Traceback (most recent call last):
  File "main.py", line 226, in <module>
    main(sys.argv[1:])
  File "main.py", line 195, in main
    user = User(email, password)
  File "E:\Utilisateurs\letouane\Downloads\packtpub-downloader-master\user.py", line 27, in __init__
    self.header["Authorization"] = self.get_token()
  File "E:\Utilisateurs\letouane\Downloads\packtpub-downloader-master\user.py", line 44, in get_token
    print("Error {}".format(e))
NameError: name 'e' is not defined

I'm trying to adapt the script but I'm not sure I'll succeed.

could not download when name contains ?

Traceback (most recent call last):
File "G:\packtpub-downloader-master\main.py", line 226, in
main(sys.argv[1:])
File "G:\packtpub-downloader-master\main.py", line 218, in main
download_book(filename, url)
File "G:\packtpub-downloader-master\main.py", line 103, in download_book
with open(filename, 'wb') as f:
OSError: [Errno 22] Invalid argument: 'G:\packt/AWS_Certified_Developer_???Associate_Guide-_Second_Edition.epub'

Connection is being broken for long downloading sessions

Hi,
I wanted to download my rather big collection from packtpub (~430 books) and after some long time this errors end execution of script.
Python 3.9, Fedora 33.
`
Traceback (most recent call last):###############################################1 | 292k/423k [01:39<00:43, 2.99kKB/s]
File "/usr/local/lib/python3.9/site-packages/urllib3/contrib/pyopenssl.py", line 294, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/OpenSSL/SSL.py", line 1840, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/lib/python3.9/site-packages/OpenSSL/SSL.py", line 1663, in _raise_ssl_error
raise SysCallError(errno, errorcode.get(errno))
OpenSSL.SSL.SysCallError: (104, 'ECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 360, in _error_catcher
yield
File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 442, in read
data = self._fp.read(amt)
File "/usr/lib64/python3.9/http/client.py", line 458, in read
n = self.readinto(b)
File "/usr/lib64/python3.9/http/client.py", line 502, in readinto
n = self.fp.readinto(b)
File "/usr/lib64/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.9/site-packages/urllib3/contrib/pyopenssl.py", line 299, in recv_into
raise SocketError(str(e))
OSError: (104, 'ECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 750, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 494, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 459, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "/usr/lib64/python3.9/contextlib.py", line 135, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 378, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: OSError("(104, 'ECONNRESET')")', OSError("(104, 'ECONNRESET')"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/khazakar/downloadedwithgit/packtpub-downloader/main.py", line 226, in
main(sys.argv[1:])
File "/home/khazakar/downloadedwithgit/packtpub-downloader/main.py", line 218, in main
download_book(filename, url)
File "/home/khazakar/downloadedwithgit/packtpub-downloader/main.py", line 111, in download_book
for chunk in tqdm(r.iter_content(chunk_size=1024), total=math.ceil(total//1024), unit='KB', unit_scale=True):
File "/usr/local/lib/python3.9/site-packages/tqdm/_tqdm.py", line 1000, in iter
for obj in iterable:
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 753, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: OSError("(104, 'ECONNRESET')")', OSError("(104, 'ECONNRESET')"))
`

Video downloads

Hi there,
at first, thank you for your work. It's nice to see how the opensource community works.
"It's outdated? No problem, I will do this"

Is there a possibility to set a parameter, which downloads videos too?
I tried to set -b video and something happens! There is a .zip file with content like code-samples and single video files.
But I don't find the word "video" anywhere in the code. So how can this work?
This is YOUR moment to say "Bug? No no, it's a feature" ;)

Issue while getting requirements.txt

 Could not find a version that satisfies the requirement aiohttp==3.5.4 (from -r requirements.txt (line 2)) (from versions: 0.1, 0.2, 0.3, 0.4, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.5.0, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.14.4, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.16.5, 0.16.6, 0.17.0, 0.17.1, 0.17.2, 0.17.3, 0.17.4, 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.18.4, 0.19.0, 0.20.0, 0.20.1, 0.20.2, 0.21.0, 0.21.1, 0.21.2, 0.21.4, 0.21.5, 0.21.6, 0.22.0a0, 0.22.0b0, 0.22.0b1, 0.22.0b2, 0.22.0b3, 0.22.0b4, 0.22.0b5, 0.22.0b6, 0.22.0, 0.22.1, 0.22.2, 0.22.3, 0.22.4, 0.22.5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.2.0, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6.post1, 2.0.7, 2.1.0, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.3.0a4, 2.3.0, 2.3.1, 2.3.2b2)
No matching distribution found for aiohttp==3.5.4 (from -r requirements.txt (line 2))

Any suggestions?

looping through books several times

I fired off the main script and it says I have 374 books. It then downloads the books (37 pages at 10 per page). When it starts downloading books, the progress shows that it is trying to download 6818 books. After downloading just over 350 books, it starts listing the same books again and skipping the downloads because they already exist. It is like the database of books was loaded 19 or 20 times over.

Here is the output with Verbose turned on, cutting out some repetitive sections and showing a random error in between:

chris@chris-VB-LinuxMint:~/Linux_Shared/packtpub-downloader-master$ python3 main.py -e user@domain -p PaSsWoRd -d /packtpub_books -b pdf,epub,mobi,code -v -s
You are in!
You have 374 books
Getting list of books...
0%| | 0/37 [00:00<?, ?Pages/s]https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=10&limit=10
3%|████▌ | 1/37 [00:00<00:27, 1.31Pages/s]https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=20&limit=10
5%|█████████▏ | 2/37 [00:01<00:29, 1.18Pages/s]https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=30&limit=10

......
97%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 36/37 [00:23<00:00, 1.78Pages/s]https://services.packtpub.com/entitlements-v1/users/me/products?sort=createdAt:DESC&offset=370&limit=10
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 37/37 [00:24<00:00, 1.98Pages/s]
Downloading books...
/packtpub_books/Digital_Forensics_and_Incident_Response/Digital_Forensics_and_Incident_Response.epub already exists, skipping.
/packtpub_books/Digital_Forensics_and_Incident_Response/Digital_Forensics_and_Incident_Response.mobi already exists, skipping.
/packtpub_books/Digital_Forensics_and_Incident_Response/Digital_Forensics_and_Incident_Response.pdf already exists, skipping.
/packtpub_books/Beginning_Data_Science_with_Python_and_Jupyter/Beginning_Data_Science_with_Python_and_Jupyter.code already exists, skipping.
/packtpub_books/Beginning_Data_Science_with_Python_and_Jupyter/Beginning_Data_Science_with_Python_and_Jupyter.epub already exists, skipping.
/packtpub_books/Beginning_Data_Science_with_Python_and_Jupyter/Beginning_Data_Science_with_Python_and_Jupyter.mobi already exists, skipping.
/packtpub_books/Beginning_Data_Science_with_Python_and_Jupyter/Beginning_Data_Science_with_Python_and_Jupyter.pdf already exists, skipping.
/packtpub_books/Extreme_C/Extreme_C.code already exists, skipping.
/packtpub_books/Extreme_C/Extreme_C.epub already exists, skipping.
/packtpub_books/Extreme_C/Extreme_C.mobi already exists, skipping.
/packtpub_books/Extreme_C/Extreme_C.pdf already exists, skipping.
/packtpub_books/Wireframing_Essentials/Wireframing_Essentials.epub already exists, skipping.
/packtpub_books/Wireframing_Essentials/Wireframing_Essentials.mobi already exists, skipping.
/packtpub_books/Wireframing_Essentials/Wireframing_Essentials.pdf already exists, skipping.
/packtpub_books/TensorFlow_Reinforcement_Learning_Quick_Start_Guide/TensorFlow_Reinforcement_Learning_Quick_Start_Guide.code already exists, skipping.
/packtpub_books/TensorFlow_Reinforcement_Learning_Quick_Start_Guide/TensorFlow_Reinforcement_Learning_Quick_Start_Guide.epub already exists, skipping.
/packtpub_books/TensorFlow_Reinforcement_Learning_Quick_Start_Guide/TensorFlow_Reinforcement_Learning_Quick_Start_Guide.mobi already exists, skipping.
/packtpub_books/TensorFlow_Reinforcement_Learning_Quick_Start_Guide/TensorFlow_Reinforcement_Learning_Quick_Start_Guide.pdf already exists, skipping.
0%| | 5/6818 [00:12<4:28:38, 2.37s/Book]Starting to download /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.code
Finished /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.code
Starting to download /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.epub
Finished /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.epub
Starting to download /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.mobi
Finished /packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.mobi
/packtpub_books/Splunk_7_x_Quick_Start_Guide/Splunk_7_x_Quick_Start_Guide.pdf already exists, skipping.
0%|▏ | 6/6818 [00:17<6:20:58, 3.36s/Book]Starting to download /packtpub_books/Salesforce_Platform_Developer_I_Certification_Guide/Salesforce_Platform_Developer_I_Certification_Guide.code
Finished /packtpub_books/Salesforce_Platform_Developer_I_Certification_Guide/Salesforce_Platform_Developer_I_Certification_Guide.code
Starting to download /packtpub_books/Salesforce_Platform_Developer_I_Certification_Guide/Salesforce_Platform_Developer_I_Certification_Guide.epub
Finished /packtpub_books/Salesforce_Platform_Developer_I_Certification_Guide/Salesforce_Platform_Developer_I_Certification_Guide.epub
......

Eventually I get the same list of books being attemted over and over (skipping) as the counter keeps going up into the thousands.

Starting to download /packtpub_books/Learning_Drupal_8/Learning_Drupal_8.pdf
Finished /packtpub_books/Learning_Drupal_8/Learning_Drupal_8.pdf
5%|████████▊ | 373/6818 [1:02:11<7:54:10, 4.41s/Book]Starting to download /packtpub_books/Functional_C3/Functional_C3.code
Finished /packtpub_books/Functional_C3/Functional_C3.code
Starting to download /packtpub_books/Functional_C3/Functional_C3.epub
Finished /packtpub_books/Functional_C3/Functional_C3.epub
Starting to download /packtpub_books/Functional_C3/Functional_C3.mobi
Finished /packtpub_books/Functional_C3/Functional_C3.mobi
Starting to download /packtpub_books/Functional_C3/Functional_C3.pdf
Finished /packtpub_books/Functional_C3/Functional_C3.pdf
/packtpub_books/Hands-On_Dashboard_Development_with_Shiny/Hands-On_Dashboard_Development_with_Shiny.code already exists, skipping.
/packtpub_books/Hands-On_Dashboard_Development_with_Shiny/Hands-On_Dashboard_Development_with_Shiny.epub already exists, skipping.
/packtpub_books/Hands-On_Dashboard_Development_with_Shiny/Hands-On_Dashboard_Development_with_Shiny.mobi already exists, skipping.
/packtpub_books/Hands-On_Dashboard_Development_with_Shiny/Hands-On_Dashboard_Development_with_Shiny.pdf already exists, skipping.
/packtpub_books/Enterprise_Augmented_Reality_Projects/Enterprise_Augmented_Reality_Projects.code already exists, skipping.
/packtpub_books/Enterprise_Augmented_Reality_Projects/Enterprise_Augmented_Reality_Projects.epub already exists, skipping.
/packtpub_books/Enterprise_Augmented_Reality_Projects/Enterprise_Augmented_Reality_Projects.mobi already exists, skipping.
/packtpub_books/Enterprise_Augmented_Reality_Projects/Enterprise_Augmented_Reality_Projects.pdf already exists, skipping.
/packtpub_books/Continuous_Delivery_with_Docker_and_Jenkins/Continuous_Delivery_with_Docker_and_Jenkins.code already exists, skipping.
/packtpub_books/Continuous_Delivery_with_Docker_and_Jenkins/Continuous_Delivery_with_Docker_and_Jenkins.epub already exists, skipping.
/packtpub_books/Continuous_Delivery_with_Docker_and_Jenkins/Continuous_Delivery_with_Docker_and_Jenkins.mobi already exists, skipping.
/packtpub_books/Continuous_Delivery_with_Docker_and_Jenkins/Continuous_Delivery_with_Docker_and_Jenkins.pdf already exists, skipping.

......

When I ctl-c killed it, I saw:
/packtpub_books/Azure_for_Architects_-Second_Edition/Azure_for_Architects-Second_Edition.code already exists, skipping.
/packtpub_books/Azure_for_Architects
-Second_Edition/Azure_for_Architects-Second_Edition.epub already exists, skipping.
/packtpub_books/Azure_for_Architects
-Second_Edition/Azure_for_Architects-Second_Edition.mobi already exists, skipping.
/packtpub_books/Azure_for_Architects
-Second_Edition/Azure_for_Architects-_Second_Edition.pdf already exists, skipping.
30%|████████████████████████████████████████████████▋ | 2064/6818 [2:12:51<3:21:20, 2.54s/Book]^CTraceback (most recent call last):
File "/home/chris/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 377, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

Yes, up to book # 2064 of 6818 books (and all of the skips in between).

Is there a way to de-dupe the books list? We probably want to keep the order so that it downloads most recent adds first, so you can kill it when you get into the repeated skipping for all of the books you have already downloaded.

I am not familiar with python, but I used to code in C. Is there a bit of code I could add to dump the books list to a file before it starts so that I can see what it is about to loop through? Or could we add an option to dump the data to a file and leave it after it has completed (same writable location as target directory, remove file if it already exists, create new database dump file, dump the books data into the file, close the file, if verbose then echo the file name and location (dumped the books database to xxx) - something like that for an additional option - maybe -x or --extrainfo)?

Question mark in book name causes python in Windows to crash

There is a book exists with title "AWS Certified Developer ??? Associate Guide - Second Edition" in my account, so if I run the python script on Windows (or any other OSes), the script always terminates with the same error [WinError 123] FIle name, directory name or sth syntax error.: '\path\to\Packt/AWS_Certified_Developer_???Associate_Guide-_Second_Edition', which already broke my account. I have no choice but to create a new Packt account until the issue with this script is fixed.

not work

python main.py -e [email protected] -p nopassword -d media -b pdf,epub,mobi,code
You are in!
Traceback (most recent call last):
File "main.py", line 226, in
main(sys.argv[1:])
File "main.py", line 198, in main
books = get_books(user, is_verbose=verbose, is_quiet=quiet)
File "main.py", line 41, in get_books
print(f'You have {str(r.json()["count"])} books')
KeyError: 'count'

How to fix it?

Having this issues - Does this still work?

You are in!
Traceback (most recent call last):
File "/Users/xx/Downloads/packtpub-downloader-master/main.py", line 226, in
main(sys.argv[1:])
File "/Users/xx/Downloads/packtpub-downloader-master/main.py", line 198, in main
books = get_books(user, is_verbose=verbose, is_quiet=quiet)
File "/Users/xx/Downloads/packtpub-downloader-master/main.py", line 41, in get_books
print(f'You have {str(r.json()["count"])} books')
KeyError: 'count'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.