allejok96 / jw-scripts Goto Github PK

Hi!

First of all, thank you for this great scripts!
I installed on my rPI the jwb rpi scripts (based on this), and I would like to use this to download all videos from tv.jw.org.
Download works ok, but after that, I realized, that omxplayer always running.
The "now_playing" file updateing in every minutes.
When I kill omxplayer process, it's restarted immediately.
It's normal? I would like to use this script to download videos into my plex directory.
Thank you for your help.
David

I tried to use your programm as it is exactly what I need. Thank you for your work.
Problem is that I dont know much about python so cant help much myself atmo.

When I try to load the jwb-index command as given in your example I get the follwoing error message.
TypeError: the JSON object must be str, not 'bytes'

When I try to run the programm on my windows laptop over an ide it works fine but not through the terminal on ubuntu.

Hallo Thanks for the awesome work.

i wanted to ask if it is possible to get the friendly names for the videos as the file name

Thanks for he help
kind regards

Julian

jwb-index downloads seem to be flat (no subdirectories) by default. I am running the script with flags [--quality 999 --download "/directory/to/download/"]. Am I doing something incorrectly?

This only seems to be a recent error.
Running on Ubuntu 18.04
Python 3.6.6
curl 7.58.0

Here is the command:
jwb-index --mode filesystem --download

And the stack trace:

Traceback (most recent call last):
  File "/usr/local/bin/jwb-index", line 4, in <module>
    __import__('pkg_resources').run_script('jw-scripts==1.0', 'jwb-index')
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 658, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1445, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/local/lib/python3.6/dist-packages/jw_scripts-1.0-py3.6.egg/EGG-INFO/scripts/jwb-index", line 102, in <module>
  File "/usr/local/lib/python3.6/dist-packages/jw_scripts-1.0-py3.6.egg/jwlib/output.py", line 162, in output_filesystem
FileNotFoundError: [Errno 2] No such file or directory: '../bhat_E_r720P.mp4' -> './jwb-E/VODMinistryExpandMinistry/Be Honest in All Things (Bethel/Construction Application).mp4'

The dir ./jwb-E/VODMinistryExpandMinistry/ is empty

Dear brother,

This is not an issue....more of a feature request ^_^
Perhaps there's a way we can download all audio recordings from experiences section ?

Thank you

I would like to run this on my NAS drive which runs Plex but it only supports Python 2.7.10r10
Thank you in advance! 😄

You did good job with this app.
But i got a bug with video.
I use Xiaomi Mi Box S with Android TV 9.
First this bug i solved with factory reset.
But this is not a solution if i must factory reset even couple days.
Audio files work fine.

Plz. Fix this bug.

In version 1.5.0 jwb-index script is very slow synchronizing with jw.org. These are steps I presume your script is taking (I didn't look into code, just observing its behaviour):

1. Downloading json files from all categories. This is fast.
2. Checking it against files on disk. This is the problem. Before update it was really fast, maybe not instant, but now it takes 10-15 minutes or more.
3. Downloading new or updated files. There's no lag here.
4. Creating links. It's OK.

How can I help track down this issue? Is there a log file?

I managed to register on GitHub and write this comment before it moved to 3rd phase.

Ok I'm sure I missed something but how do I define the path/datadir?

I tried this string
jw-scripts$ sudo jwb-index --lang=E --quality=240 --download --latest

and the result is this:
/usr/local/bin/jwb-index: line 256: realpath: command not found LatestVideos ./shared-functions: line 318: realpath: command not found Error in write_to_playlist

Any ideas of what is happening?
Thanks

Hello! I love what you've done with this project.

I would love to help create a dedicated Plex mode, to enable better listing and ease of use.

Please e-mail at [email protected]

hi is it possible to download the songs in mp3 format? maybe it could be added under quality so you could run it like this: jwb-index --mode filesystem --download --quality=mp3 --category jwb-D/VODOriginalSongs/

Thank you @allejok96 for these scripts and their continual maintenance.
An example below for your consideration - that can be added to wiki for (windows) users.

Download all Monthly broadcasts (Subcategory of VODStudio) in Swahili language from 2019 January to present

python jwb-index -c StudioMonthlyPrograms --lang=SW -Q 360 --since 2019-01-01 --download C:\Users\User\Videos\JWB

Dear Brother

Thank you for your script.
Perhaps you could help with error I experienced:

Thank you.

Hello,

Thanks for writing this script! So after running the command to download everything from July 1 of this year, I now have a directory of videos in the jwb-E folder, and a symlink to a directory named Video on Demand, with symlinks to other folders within it. Looks good. However, when I later run the below command using the latest flag, it deletes all the symlinks? Should it not just delete ones that are no longer pointing to valid video files?

1rst run - jwb-index --download --since 2017-07-01 --mode filesystem --free 275000 /Users/leejk/jw_broadcasting

2nd run - jwb-index --latest --download --mode filesystem --free 275000 /Users/leejk/jw_broadcasting

My goal is to set this up as an Automator job after getting the initial download. As a feature request, would it be possible to enable logging for the script to capture all of the output when used in an automated process?

Hello,

If this obvious I apologize, but I'd like to exclude a few categories from download, but not sure what the category list is. Is the list the same as the directory structure created on the filesystem, or something else? Perhaps if the --exclude argument is passed with no parameter, it could return a valid category list?

Hi, I installed Python3.6.2 and ran the setup.py with no errors. However when trying to run job-index I get these errors in terminal:

errors.txt

Any thoughts?

hello, thanks for all you comment in previous issues:

if i run python "jwb-stream" --lang F --category VODOurOrganization,VODStudio,VODOurActivities,VODIntExp,VODFamily,VODChildren,VODFamily "C:\Program Files\VideoLAN\VLC\vlc.exe" --fullscreen --video-on-top --sub-autodetect-file
the vlc need to be close the same number of category... can we change to close only one time VLC?

and also is possible to have an argument in jw-stream --quiet to not see the print of indexing!

thanks

Hi,

Thank you for sharing this scripts.
I would like to share how you can get them to work on a windows machine. I tried on Windows 10 but I am sure this can be adapted for other versions of windows.

Briefly- without many details, these are the steps, I am sure there could be better ways - but this is what worked for me.

1. Create a folder in C drive e.g. C:\JWBroadcast\
2. Install git ( optionally you can install inside the folder in 1. above posh-git )
3. Install python for windows - make sure to add environment variable to python path.
4. Install simplejson inside the folder in 1. above
5. Install jw-scripts inside the folder in 1. above

• You will need to set the Execution policy to allow scripts to be run by admin account
• git clone https://github.com/allejok96/jw-scripts.git
• cd jw-scripts
• run from command prompt / powershell python setup.py install
  you can change the working directory in jwb-index "c:\\JWBroadcast\\"
  Addition: use two back slashes

6. Install curl - download curl and copy and unzip the 7/zip file to the folder in 1. above. Rename it to curl
   Addition: install urllib3
7. Add environment variable path to curl\bin folder in 6 above.
8. open Windows PowerShell or command prompt with admin account (not sure why it didn't work with normal account)
9. run: jwb-index --download --category VODChildren
10. run jwb-index --mode html --download --category VODChildren

I may have missed some steps (especially during the installation process - however these are the general steps I took)
I hope this helps someone out there!
Thank you allejok96 for these scripts! 🥇 👍

Thank you SOOO much for making this. I was getting ready to try to tackle this, but I got stuck at fetching the URLs. I only spent 5 minutes before deciding to search GitHub and there you are! And it's written in Python. Love it! Thank you again. Can we get a download speed indicator? If I could figure out how to do pull requests, I'd look into figuring it out

In KODI addon JW Broadcasting there are function to return watching previously stopped playback.

This is very usefull function when watching long programs like monthly program...

hello, my brother,

congratulations on your script, I want to ask for authorization to use JW-stream in our GitHub that we develop, and I search a way to have the monthly broadcasting.

I will be happy if you want to contact us for a synergy job.

Brother love.

I would like to thank you for this script. It will help greatly with family worship. We like to watch the latest videos but often experience buffering. With this I can automate the downloads.

I did run across one issue with my installation:

Curl reports that it "failed to verify the legitimacy of the server".

I'm sure its a problem with my certs but couldn't figure out how to resolve it on Windows 10 using curl 7.58.0 (x86_64-w64-mingw32) Release-Date: 2018-01-24

My solution was to add -k to the call for curl in parse.py as follows:

if curl_path:
    proc = [curl_path, url, '-ko', file]

Using the insecure method may not best. Perhaps you could suggest a better/more elegant solution.

Thanks

Is it possible to download all media for week meetings?

This is not a bug but just a suggestion, if possible offcourse.

I would love to have the ability to download the videos in language A and have the subs in language B. But when did it manually with the given commands i endup with different names (same videos but in 2 languages) and plex doesn't see the subs because the filename is not the same. It would be need to download the subs with the same names as language A (video language) but have the subs in language B. (prefabably with the language code in the name).

Thanks for you're hard work!!

First of all: thank you so much for making it so easy to have an offline copy of all the videos available and easily accessible. :)

Unfortunately, I have a problem with -m filesystem: only the two top-level symlinks (for Latest and VoD) are set correctly. Any other symlink is missing "../" in the target, no matter whether it's a video or a folder.

As you can see, the symlinks always expect the target to be in the same directory when, in reality, they should be pointing to the parent directory.

This is the command that I used to download the video files and generate the directory structure.

# python3 ./jwb-index -d --since 2020-01-01 -l X -m filesystem --limit-rate 0 --free 2048 /media/jwb-videos

Nothing to see here

Sup all ye watchers! 🐶

I've been doing a few changes which will break compatibility 🤦 (don't worry, it's just some flags). So I figured: why not go all in and make a 2.0? Now before I go and do something stupid, I'd like to hear your stories! Don't be shy. Right now I'm just looking for some reasonable defaults for the program.

How do you use jw-scripts?

📺 Tell me your use-case, operating system, example commands...

😕 Was something hard to setup or figure out?

🔨 Did you need to do any tweaks to Plex etc?

🤸 Do you do some extra "post-processing"?

💡 Other suggestions?

Even if you do nothing special, just drop a comment with a example, just so I get an idea.

Dear Brothers:

Sorry to be a pain but I am trying to get this working for an elderly brother in my hall. My plan is to download all the content and he can watch it on a loop in Kodi.

I have tried the instructions on Windows installation and have tried the sample text to download all Children videos but get the following errors:

downloading: pksjj_E_054_r720P.mp4 (Song 054 "This Is the Way")
Traceback (most recent call last):
File "jwb-index", line 71, in
jwb.download_all(os.path.join(wd, subdir))
File "C:\JWtv\jw-scripts\jwlib\parse.py", line 322, in download_all
media.file = self.download_media(media, wd)
File "C:\JWtv\jw-scripts\jwlib\parse.py", line 284, in download_media
_curl(media.url, file + '.part', rate_limit=self.rate_limit)
File "C:\JWtv\jw-scripts\jwlib\parse.py", line 422, in _curl
subprocess.call(proc, stderr=stderr)
File "C:\Users\luke\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 247, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\Users\luke\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 676, in init
restore_signals, start_new_session)
File "C:\Users\luke\AppData\Local\Programs\Python\Python35\lib\subprocess.py", line 957, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

Any idea what I have done wrong?

It would be great if it has multi-language support. Obviously if you had time to do it.

running
 jwb-index --mode filesystem --download --category VODMoviesModernDay --percent --quality=720
returns
usage: jwb-index [options] [DIR]
jwb-index: error: unrecognized arguments: --percent

Hello,

After running the script and having it download a few videos, I am getting this error, which I assume is at the end of the script's completion?

macmini:jw-scripts-master leejk$ ./jwb-index --latest --download --mode filesystem --free 50000 /Users/leejk/jw_broadcasting
free space: 324526 MiB, minimum limit: 50000 MiB
LatestVideos (Latest Videos)
downloading: jwb_E_201708_11_r720P.mp4 (2016 Puerto Rico Special Convention)
size mismatch, deleting: jwb_E_201708_11_r720P.mp4.part
Traceback (most recent call last):
File "./jwb-index", line 90, in
jo.output_filesystem(r, wd, subdir)
File "/Users/leejk/Downloads/jw-scripts-master/jwlib/output.py", line 130, in output_filesystem
os.symlink(source, link, dir_fd=dir_fd)
NotImplementedError: dir_fd unavailable on this platform

Hey. I've installed your scripts successfully on Windows 10, but links aren't working. Specifically, the ones that are inside localized folders. If I right-click them, I see they point to e.g. "C:\JWB\Dzieci\pkon_P_017_r720.mp4", while they should point to "C:\JWB\jwb-P\pkon_P_017_r720.mp4". Links that are inside "C:\JWB\jwb-P\BJF" are working perfectly (although they got no thumbnail). I understand that they are hard links, right? Maybe if your script generate symbolic links, they would be fine?...

Error code:

downloading: William Turner: Wees niet partijdig (Joh. 4:31).mp4 (William Turner: Wees niet partijdig (Joh. 4:31))
Traceback (most recent call last):
File "/usr/local/bin/jwb-index", line 33, in
sys.exit(load_entry_point('jw-scripts==1.10.1', 'console_scripts', 'jwb-index')())
File "/usr/local/lib/python3.10/site-packages/jw_scripts-1.10.1-py3.10.egg/jwlib/main.py", line 193, in main
File "/usr/local/lib/python3.10/site-packages/jw_scripts-1.10.1-py3.10.egg/jwlib/download.py", line 70, in download_all
File "/usr/local/lib/python3.10/site-packages/jw_scripts-1.10.1-py3.10.egg/jwlib/download.py", line 165, in download_media
File "/usr/local/lib/python3.10/site-packages/jw_scripts-1.10.1-py3.10.egg/jwlib/download.py", line 255, in download_file
File "/usr/local/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
response = meth(req, response)
File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
response = self.parent.error(
File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Command that i run:

jwb-index -d -m filesystem -l O --friendly --limit-rate 5 --checksum --fix-broken /downloads --clean-symlinks

Good day,

I am happy with jw-index to find audio files.

Some categories are not language sensitive, but vocally they are.

So I try to find the Dutch version with:

python jwb-index --mode stdout --lang nl --category Audio> C: \ Users \ berga \ Documents \ JWB \ Output \ test

I get an error:

python jwb-index --mode stdout --lang nl --category Audio> C: \ Users \ berga \ Documents \ JWB \ Output \ test

Traceback (most recent call last):
File "jwb-index", line 34, in
settings = parser.parse_args ()
File "C: \ Users \ berga \ Documents \ JWB \ jw-scripts-master \ jwlib \ arguments.py", line 107, in parse_args
super (). parse_args (* args, namespace = settings, ** kwargs)
File "C: \ Users \ berga \ AppData \ Local \ Programs \ Python \ Python38 \ lib \ argparse.py", line 1768, in parse_args
args, argv = self.parse_known_args (args, namespace)
File "C: \ Users \ berga \ AppData \ Local \ Programs \ Python \ Python38 \ lib \ argparse.py", line 1800, in parse_known_args
namespace, args = self._parse_known_args (args, namespace)
File "C: \ Users \ berga \ AppData \ Local \ Programs \ Python \ Python38 \ lib \ argparse.py", line 2006, in _parse_known_args
start_index = consume_optional (start_index)
File "C: \ Users \ berga \ AppData \ Local \ Programs \ Python \ Python38 \ lib \ argparse.py", line 1946, in consume_optional
take_action (action, args, option_string)
File "C: \ Users \ berga \ AppData \ Local \ Programs \ Python \ Python38 \ lib \ argparse.py", line 1874, in take_action
action (self, namespace, argument_values, option_string)
File "C: \ Users \ berga \ Documents \ JWB \ jw-scripts-master \ jwlib \ arguments.py", line 14, in call
result = function (values)
File "C: \ Users \ berga \ Documents \ JWB \ jw-scripts-master \ jwlib \ arguments.py", line 32, in verify_language
raise ValueError (code + ': invalid language code')
ValueError: en: invalid language code
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

ValueError: en: invalid language code

--lang nl is therefore not good.

But if I search the list of languages ​​with:
python jwb-index --mode stdout --lang nl --category Audio >C:\Users\berga\Documents\JWB\Output\test

Then I will get output, but not what I expect!

VideoOnDemand (Video Categories)
VODStudio (JW Broadcasting)
VODChildren (Children)
VODTeenagers (Teenagers)
VODFamily (Family)
VODProgramsEvents (Programs and Events)
VODOurActivities (Our Activities)
VODMinistry (Our Meetings and Ministry)
VODOurOrganization (Our Organization)
VODBible (The Bible)
VODMovies (Movies)
VODMusicVideos (Music)
VODIntExp (Interviews and Experiences)
StudioFeatured (Featured Videos)
..............
.............
............
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

What am I doing wrong?

I hope to receive an answer soon.

Thank you very much.

Arie van den Berg.

Hi Alex i want to download all original songs videos at one go with the following options:

• language
• subtitles (and select the language , when more than one is available)
  i would also need to the program to find all resources coming with the song(lead sheet,lyrics,and video) and be able to download them in the specified language

hello,

i comment random.shuffle(links) in jw-stream and the movie continue to be shuffle !

i can not understand why?

First of all I want to thank you for making this tool.

I've followed all the installation steps on Windows 10 but when I run some command like:
jwb-index --mode filesystem --ntfs --download --lang=T

It starts indexing all categories and then throws the following error:

Traceback (most recent call last):
File "C:\JWB\jw-scripts-master\jwb-index", line 53, in
data = parse_broadcasting(settings)
File "C:\JWB\jw-scripts-master\jwlib\parse.py", line 186, in parse_broadcasting
j_media_file = get_best_video(j_media['files'], quality=s.quality, subtitles=s.hard_subtitles)
File "C:\JWB\jw-scripts-master\jwlib\parse.py", line 116, in get_best_video
rankings.sort()
TypeError: '<' not supported between instances of 'dict' and 'dict'

Idk if it is something related to the language (Portuguese) and its special characters.

Edit: I am able to download on other languages (like TPO instead of T).