HTML Log Request about hama.bundle HOT 45 CLOSED

Already done. Check the readme forthe list of all

from hama.bundle.

Is it supposed to show up with the other HTML logs (\Plug-in Support\Data\com.plexapp.agents.hama\DataItems)?

I added an episode, that does not have a description anywhere, as a test -- and I didn't see any log generated.

from hama.bundle.

@ZeroQI I think I had seen this in previous workings on Hama. The missing summaries only logs when there is no tvdb mapping available but NOT when the mapping is there but empty/blank for the summary.

from hama.bundle.

I did a "Refresh All" and there are a ton of Naruto Shippuuden episodes without description (TVDB locks it and adds them later), but no HTML logs appear as @EndOfLine369 has indicated.

from hama.bundle.

The mechanics are there but logs not generated. Need to fix that it used to
work. There was a file for tvdb summary missing
On 17 May 2016 22:23, "sven-7" [email protected] wrote:

I did a "Refresh All" and there are a ton of Naruto Shippuuden episodes
without description (TVDB locks it and adds them later), but no HTML logs
appear as @EndOfLine369 https://github.com/EndOfLine369 has indicated.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#43 (comment)

from hama.bundle.

@sven-7: "TVDB summaries missing" is the right log, was in hte list of log files, but is not called anywhere in the code like it should, i never coded it even though i was 100% sure i did. My apologies sven-7, thanks EndOfLine369 for backing him up there and precising

We can do better maybe by putting a line per episode with the link to the episode but that could flood the log... One log per series with the list of episodes seem nice and simple, have added the change to github, please test

@EndOfLine369 : spotted something weird in code: if not episode.xpath('absolute_number')[0].text or episode.xpath('absolute_number')[0].text == ''
seem like if not empty or empty... something

Explanation on html error logs:

error_log is an array of potential created html log files:
'anime-list anidbid missing'
'anime-list tvdbid missing'
'anime-list studio logos':
'Missing episodes'
'Plex themes missing'
'AniDB summaries missing'
'AniDB posters missing'
'TVDB summaries missing'
'TVDB posters missing'
'TVDB season posters missing'

Usage: error_log['TVDB summaries missing'].append("string with links to allow easy update of metadata source")
The 'xxxxx'.html file log is created if at least an entry needs to be written and it loads the file and only add the new line if it is not already in the file, hence the string in logs is unique, no duplicate (was very proud of that)

from hama.bundle.

@ZeroQI : I was testing the 'TVDB summaries missing' log and noticed the 'Missing episodes' log wasn't getting any updates for tvdb shows. Put a fix together but I'm not super familiar with the code yet so if you or @EndOfLine369 have any suggestions I'd be happy to make changes.

from hama.bundle.

I think @MagellanSails is right. Coincidentally, I was looking at my Naruto Shippuuden files and noticed I episode two was missing (ep 1 was a double episode and I had it mislabeled), but that was not appearing in my missing episode file (I have this marked as TVDB3). The only shows I have appearing in that log are ones that I do not dictate with an .id file.

from hama.bundle.

looked quickly but seem good to me
i made a patch reduction but got pushed to prod by mistake, hope nothing breaks

line 306-307, hope this replace well the group of tests to assign value to numbering variable:

if defaulttvdbseason=="a": numbering = currentAbsNum
else: numbering = "s" + currentSeasonNum + "e" + (currentEpNum if currentSeasonNum == '0' or not metadata.id.startswith("tvdb3-") else currentAbsNum)

from hama.bundle.

Tested this out.

Missing episodes from shows mapped to TVDB are now populating 'missing episodes' as expected, however, the formatting is different from the AniDB ones. I think this (and perhaps the missing summaries one) could get all formatted the same?

I think the ideal is how the AniDB ones look, with the link feature of the TVDB ones for the ID. I think it's nice to have the link to TVDB to identify them, but I can see how that could be made difficult as it's the AniDB ID and not the TVDB it's listing. Thoughts?

anidbid: 11523, Title: 'Bungo Stray Dogs', Missing Episodes: [' s1e8', ' s1e7', ' s1e12', ' s1e11', ' s1e10', ' s1e9']

from hama.bundle.

Here are how my logs populate:

HTML files.zip

Side note: I see in there, Rurouni Kenshin (TVDB: 70863) says I'm missing basically every episode, but they're clearly in Plex...so that's odd.

from hama.bundle.

@sven-7 Good point, it would be nice if the logs were consistent. It should also be pretty easy to make them all say:

databaseId: #####, Title: 'Series Title', Missing Whatever: ['...', '...']

Where databaseId is anidbid or tvdbid, Whatever is Episodes or Summaries, and the #### is a link. I'll be able to make that change in about an hour unless someone disagrees or beats me to it.

from hama.bundle.

Yeah, I think that would be a really good format for both of them.

You'll notice a lot of what is missing are Season 0 things...which is painful to look at, as I typically ignore/don't collect those...but alas...this is super helpful.

from hama.bundle.

I don't have any missing episodes from Anidb mapped shows, so if someone could check to make sure that the links are working for those in my last change that would be helpful.

from hama.bundle.

@ZeroQI nevermind, just saw you already responded to the Pull Request

from hama.bundle.

I had to rewrite though, as i try to keep the length and simplicity.
Feel free to edit the code, and have constructive feedback like you did.
Possible improvement:
. no report for missing eps in season 0
. additional log for tvdb season posters missing

from hama.bundle.

Would definitely love to see both of those improvements.

I'll test this in a few hours and see how it goes.

from hama.bundle.

Something seems to be stalling. I added new shows, but it just kept on spinning and never matched the shows. I reverted back to the Hama agent before and it went through.

from hama.bundle.

Checking Plex Medis Server / Logs / PMS Plugins Logs / com.plexapp.agents.hama.log
SyntaxError: Line 133: "nsre" is an invalid variable name because it starts with ""
uploaded new version, now works, suggestion above implemented
Please test and feedback, and if you feel a log is missing or not populated please let me know

��

from hama.bundle.

Looks like one of the hama crashes had to do with '\Plug-in Support\Data\com.plexapp.agents.hama\DataItems\TVDB\blank' not existing.

from hama.bundle.

It looks like, with the latest release, the formatting has reverted.

It's now:

TVDB: databaseId: #####, Missing Whatever: ['...', '...'] -- linked
AniDB: databaseId: #####, Title: 'Series Title', Missing Whatever: ['...', '...'] -- not linked

from hama.bundle.

After a full refresh, I can say it's definitely heading in the right direction. I've found that if AniDB has a show as two series: (Example: A Certain Scientific Railgun / A Certain Scientific Railgun S) while TVDB has it as one (A Certain Scientific Railgun (Season 1/Season 2)....before, because I had it as AniDB no missing episodes popped up (I did not have season 2/Railgun S at all).

Now when it runs, the new logs let me know I had missing episodes:

114921 missing episodes: [' s0e1', ' s0e2', ' s0e3', ' s0e4', ' s0e5', ' s0e6', ' s0e7', ' s0e8', ' s0e9', ' s2e1', ' s2e2', ' s2e3', ' s2e4', ' s2e5', ' s2e6', ' s2e7', ' s2e8', ' s2e9', ' s2e10', ' s2e11', ' s2e12', ' s2e13', ' s2e14', ' s2e15', ' s2e16', ' s2e17', ' s2e18', ' s2e19', ' s2e20', ' s2e21', ' s2e22', ' s2e23', ' s2e24']

I downloaded the second series, made the entire series a TVDB2.id based show.

Other thoughts: the logs are definitely overrun by Season 0 files still -- nice to have, but somehow separated might be a "nice to have" feature.

Would also be easier to read if the series names were in the log, in addition to the ID and links.

from hama.bundle.

looks like there is an inconsistency in logging of 'Missing episodes':

303730 missing episodes: [' s2e1']
anidbid: 11203, Title: 'Assassination Classroom: Second Season', Missing Episodes: [' s1e20', ' s1e24', ' s1e23', ' s1e22', ' s1e21']

Should probably have both as:
(tvdbid|anidbid) : <id_num>, Title '<series_title>', Missing Episodes: ['<ep>', ...]
And have the 'id_num' linked to 'http://thetvdb.com/?tab=series&id=%s' or 'http://anidb.net/a%s'
?
Or probably best to stick with the more common (as in other files):
<id_num>, missing episodes: ['<ep>', ...]
but would we have issues in loading if anidbid and tvdbids were the same?

from hama.bundle.

I'd prefer having the title listed in the log as well. If you only have one or two shows missing something then it isn't a big deal to click on each one, but if you have 50 or 100+ with something missing it's hard to deal them without titles.

from hama.bundle.

Will add title and sort the list.
Need to use anidb missing eps on anidb series and tvdb missing on series with tvdb ids otherwise full anidb series will be showing missing eps in tvdb logs.

from hama.bundle.

@ZeroQI I think we could make everyone happy and close this ticket (originally it was only asking for missing summaries) with a couple small changes. It sounds like the first fomat @EndOfLine369 posted should work for everyone:
(tvdbid|anidbid) : <id_num>, Title '<series_title>', Missing Episodes: ['<ep>', ...]

And then we could address the Season 0 issues by renaming some of the logs:

• Missing episodes: spilts into Missing Episodes and Missing Specials
• TVDB summaries missing: splits Missing Episode Summaries and Missing Special Summaries

Diving the logs this way would fix @sven-7 's issue with missing specials overwhelming his logs and, in my opinion, make the log file names very clear.

From my perspective, getting these files nailed down would be super helpful, and then if someone is actually having issues with other logs (e.g. posters or anidb summaries) we can start looking at those in a new issue.

Thoughts?

from hama.bundle.

I think that's an ideal scenario. For many people, the specials aren't always something you can actually collect -- they might never get uploaded, become available, etc.... so you'll always have them missing. I think separating them out is nice, while still allowing those that want them to continue having the logs for it.

from hama.bundle.

Excellent update MagellanSails! Thanks

from hama.bundle.

It's looking good so far. I did notice that is says I'm missing Rurouni Kenshin, but it's definitely in my Plex. The scanner picks it up, but something seems to happen in Hama where it thinks I'm missing it, yet...have all the metadata still. Comic Party S2 seems to be the same way.

Logs.zip

from hama.bundle.

@sven-7, The problem is probably because the htm log listings are not as maintained as you think they are. The files are basically only checked to make sure what is being entered does not already exactly exist. So if you have a running series or a bad scan, the initial entry is not updated but only entered again with the new info.

EX: Where my 'Ushio and Tora (2016)' was scanned, I added in the new ep9, rescanned 'Ushio and Tora (2016)'. Now there are two entries for the same series.

from hama.bundle.

Yep - I totally get that it's recording unique lines and will make an entry that has you missing Tora 9-31, then an entry with Tora 10-13 missing.

The thing that is bewildering me lies purely around Rurouni Kenshin right now. I know I'm missing one episode, #95, which it registers correctly. Then comes the weird part -- It says I'm missing basically everything, expect all of the files exist. That's what is confusing. I've deleted the htm logs several times, done several 'full refreshes', I've pulled it out of my library and put it back in, still to no avail. Hama still 'thinks' it's missing.

from hama.bundle.

Got it.

Looks like like 'Rurouni Kenshin New Kyoto Arc' as these are both the same tvdbid but different anidbids. So this is causing the second entry where only ep1 & ep2 are not listed.

2016-05-13 06:04:21 Scanner call - root: 'B:\Anime Series', path: 'Rurouni Kenshin New Kyoto Arc', dirs: '0', files: '3'
2016-05-13 06:04:21 =============================================================================================================================================================
2016-05-13 06:04:21 file: 'B:\Anime Series\Rurouni Kenshin New Kyoto Arc\[CBM]_Rurouni_Kenshin_New_Kyoto_Arc_[720p].sfv', ext: 'sfv' not in video_ext
2016-05-13 06:04:21 "Rurouni Kenshin New Kyoto Arc" s0001e001 "Rurouni Kenshin New Kyoto Arc - 01 - Cage of Flames.mkv" "Cage of Flames" 
2016-05-13 06:04:21 "Rurouni Kenshin New Kyoto Arc" s0001e002 "Rurouni Kenshin New Kyoto Arc - 02 - Warble of Light.mkv" "Warble of Light" 
2016-05-13 06:04:21 -------------------------------------------------------------------------------------------------------------------------------------------------------------

2016-05-24 13:48:40,760 (1734) :  DEBUG (logkit:13) - Update - metadata source: 'anidb', id: '8595', Title: 'Rurouni Kenshin: New Kyoto Arc',([...], [...], True)
2016-05-24 13:48:40,788 (1734) :  DEBUG (logkit:13) - anidbTvdbMapping() - anidb: '8595', tvbdid: '70863', tmdbid: '', imbdid: '', studio: '', defaulttvdbseason: '0', name: 'Rurouni Kenshin: Meiji Kenkaku Romantan - Shin Kyoto Hen'

from hama.bundle.

@MagellanSails / @ZeroQI, Looks like there is an inconsistency on both the 'Missing%20Episodes.htm' & 'Missing%20Specials.htm' where it is not using the correct one of either anidbid or tvdbid. It should be defaulting to anidbid and using tvdbid when it is specified by id in file or folder name (tvdb/tvdb2/tvdb3).

from hama.bundle.

"Need to use anidb missing eps on anidb series and tvdb missing on series with tvdb ids otherwise full anidb series will be showing missing eps in tvdb logs."
That is what i meant. Series with anidb guid get mapped to tvdbid. We need to only fill anidb log then.

Series with seasons or tvdb guid can use thetvdb ep missing logs.

Logs wise need to check if every log line is contained in every entry in the lines to add and replace them if so to keep them unique in logs, that will solve new lines with couple more eps in it...

from hama.bundle.

Seems something in general is going screwy with the missing eps listings. The exact same anime series refreshed twice in a row put two different entries instead of even just sticking to one. As well as the tvdbid entry not taking into account the default season from scudlee mapping.

anidbid: 11203, Title: 'Assassination Classroom: Second Season', Missing Episodes: ['s1e24', 's1e23', 's1e22', 's1e21']
tvdbid: 283947, Title: 'Assassination Classroom', Missing Episodes: ['s1e21', 's1e22', 's2e1', 's2e2', 's2e3', 's2e4', 's2e5', 's2e6', 's2e7', 's2e8', 's2e9', 's2e10', 's2e11', 's2e12', 's2e13', 's2e14', 's2e15', 's2e16', 's2e17', 's2e18', 's2e19', 's2e20', 's2e21', 's2e22', 's2e23', 's2e24', 's2e25', 's2e26']

from hama.bundle.

The "### Check for Missing Episodes ###" section probably needs the additional test of metadata.id.startswith("tvdb") as it should just stick to tvdb ep checks and let the pre-existring anidbid entry deal with the others.

FROM:
if currentSeasonNum and not (currentSeasonNum...
TO:
if metadata.id.startswith("tvdb") and currentSeasonNum and not (currentSeasonNum...

added 36287f4

from hama.bundle.

@ZeroQI, Looks like if i'm reading this right, there is also an issue in how the htm file data is saved. As if there is ever more than one entry for a log that needs to be added in, only the last entry processed for that series will be added as 'string2' is not being updated at save. Look right?

FROM:
        for entry in error_log[log]:
          if entry not in string2:  Data.Save(log+".htm", string2 + entry + "<br />\r\n")
TO:
        for entry in error_log[log]:
          if entry not in string2:  string2 = string2 + entry + "<br />\r\n"; Data.Save(log+".htm", string2)

Explains the strange 'Assassination Classroom' logging behavior above as the agent had both an 'anidbid' & 'tvdbid' entry for the log but only one showed up per refresh.

UPDATE:
Testing confirmed so fix has been committed 25e109d

from hama.bundle.

Got an update in testing where all of these log files will be fully maintained.

from hama.bundle.

Series can be with anidb guid and seasons if tvdbid not forced so need to check anidb gui and seasons<2 for anidb missing html log
For tvdb missing logs need tvdb guid or seasons>1

For logs 'messages to add' array your code change nothing. I meant that if we add a string already there, it is not added BUT if we add a string with one more missing episode it will keep both entries, and so on...
We therefore need to load the log file as an array and not a stringand check if an entry start with a string already in one of the lines of the logs and if so replace it...
Unless we can loop per line in the log string which make code simpler as we don't have to rewrite an array into file... will think about that one.

from hama.bundle.

Created enhancement pull #48 to make the error logs fully maintained.
Also placed here for you to more easily get the file for peer testing https://gist.github.com/EndOfLine369/8ee08c812a5f3943e6cc

from hama.bundle.

I also have an update so where the Movie entries in AniDB do not show incorrect entries in the "Missing Episodes" log. Like if you have the "Complete Movie"(ep1) but it still shows your missing ep2 & ep3 as they are separate entries for "Part 1 of 2"/"Part 2 of 2" or "Part X of 3" etc.

Will put in for review once this prev update has been peer reviewed/tested.

from hama.bundle.

Sorry I dropped out of the conversation; was pretty much offline for the weekend. @EndOfLine369 : were you looking for people to test what's currently in Master, or in your new Pull Request?

from hama.bundle.

I do a lot of testing myself so fairly confident on the changes but always good to have someone else confirm worked as expected. If you want to test the pull code as well, feel free too. 😀

from hama.bundle.

I set a full refresh to run this morning -- I'll let you know how it goes.

from hama.bundle.

I'm closing out this issue thread now as I think everything that was raise here has been handled. 😄

from hama.bundle.