nntmux / newznab-tmux Goto Github PK
View Code? Open in Web Editor NEWLaravel based usenet indexer
License: GNU General Public License v3.0
Laravel based usenet indexer
License: GNU General Public License v3.0
error: Undefinded index ID RequestIDLocal.php on line 281
master and dev
line 281: $this->_newTitle['ID'],
sickbearch search throws error :
SEARCHQUEUE-MANUAL-SEARCH :: Error trying to load site XML data
nZEDb added release files to the Sphinx search some time ago which allowed searching thru file names, seems nn-tmux has some files modified for it (rt optimize script mentions it) however the populate_rt_indexes.php script only allows the releases tables at the moment.
Are there any plans to fork this over from nZEDb?
Just done a clean install from dev and its not saving anything into the covers area.
mediainfo and other things being generated okay so assuming the deep inspection all okay. Looked at post processing and "j" and "s" are popping up all okay.
Have the covers alias in the apache file and folder resources/* been set to 777.
Any ideas?
Cheers
getting Warning: PDO::quote() expects parameter 1 to be string, object given in **/lib/framework/db.php on line 374
*dev
when trying to download nzb
plus it just shows nzb in raw format:
Other > Misc FRNS15DDdiv0012BSF alt.binaries.cores alt.binaries.nl alt.binaries.x [email protected] alt.binaries.cores alt.binaries.nl alt.binaries.x [email protected] alt.binaries.cores alt.binaries.nl alt.binaries.x [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
Since the recent update having an issue with database slowing down. Found the issue but not sure where this is as cant find it anywhere.
SELECT p.ID AS prehashID, p.title, p.source, p.searched FROM prehash p WHERE LENGTH(title) >= 15 AND title NOT REGEXP "["<> ]" AND searched = 0 AND DATEDIFF(NOW(), predate) > 1 ORDER BY predate ASC LIMIT 500 OFFSET 500
This repeats up to OFFSET 5000, so it uses 10 threads.
When these 10 threads run, the database comes to a standstill and hangs all other processes as it max's out all cpu resources.
Done a comparison and doesn't happen on nzedb scripts.
Any help appreciated.
Few issues with a new install. I do a fresh install before I upgrade the tmux on my main server just to route out any issues.
Hopefully you can help. I am working through each tmux pane to resolve each issues.
On all tmux panes I get :
PHP Notice: Constant DB_TYPE already defined in /var/www/newznab/www/config.php on line 6....
When I running update binaries I get :
sprintf(): Too few arguments in /var/www/newznab/www/lib/binaries.php on line 598
I have upgraded to TPG.
Any ideas?
Many Thanks.
Had a look and this is not being done on the update_releases part of the scripts. Neither the local or web parts.
I thought it was working but it was the post processing was doing a search of the prehash table and then changing the names.
Many Thanks
Before the recent update, these files where being renamed correctly.
143890-0[001/102] - "143890-1.0" yEnc (1/100)
Now nothing happening with these files.
Any ideas?
Cheers
error: a non well formed numeric value entered in /lib/binaries.php line 665
error viewable in tmux in update binaries window
line 665 : $this->_colorCLI->alternateOver(number_format($timeLoop) . 's') .
error comes and goes..
*dev
A while ago I was saying had many incompletes and thought it was highwind providers
After a few clean installs and testing various providers and settings, it only happens when using update_binaries and when its set to complete threaded in tmux.
When set to simple threaded so only one group per thread it all woks fine. So guessing its something to do with the way complete_threaded handles tpg and headers but that is as far as my knowledge will go.
Love the new themes by the way.
Cheers
I get the following error hundreds of times when I run php NNTmux/misc/update/decrypt_hashes.php full
:
PHP Notice: Undefined index: releases_id in /var/www/NNTmux/nntmux/NameFixer.php on line 1133
Any ideas what could be causing this?
During the Update_releases process all releases are put in the releases table and visible with Sphinx searches right away, the remove by min/max group/category size is only ran every x number of runs so those will remain visible for a certain period depending on how fast update_releases runs.
So during that there are fake releases visible on the index which get retrieved by API / RSS / Search requests which I noticed with Couch and RSS readers.
Is it possible to let update_releases run the remove by min/max group/category size more often or maybe prevent those from being inserted into the Sphinx index?
For now I've modified the RSS / API / Search pages to a minimal 20MB release size which works well enough but of course not a good fix.
My parts table keeps growing and growing. Not sure what I have done wrong, but it keeps growing and then I need to truncate it.
Also number of binaries waiting to be converted is keep increasing also. WHat can I do to fix this? Many thanks.
Completed adding 13 releases in 12.92 second(s). 646,287 binaries waiting to be converted
Hey Dar, thought I'd give your fork a try.
OS: Ubuntu 16.04, fresh install, all updates applied.
Branch: git branch -v
The http://server/install script seems broken when using a fresh 16.04 install following the guide on the front page. I deviated a little to use MariaDB 10.1 and nginx. When you input the usenet server login/pw/port using the install steps and click the button to go to the next step, a little red "-1" appears at the bottom and the script doesn't advance to the next step. I manually skipped to the next step to setup admin user, but when I got done, there was no server listed in config.php. I manually editted config.php but I can't fetch any binaries if I use encryption. If I switch to not use encryption, it works, but if you use encryption, all you get is a bunch of "PHP Notice: Only variables should be assigned by reference in /usr/share/php/PEAR.php on line 601" and finally
"Error: Unable to connect to usenet" when running update_binaries.php. No more specific php_errors.log was generated.
Whatever I try to enter as NZBGet server/username/password I get this error:
The NZBGet URL you entered is invalid!
Whenever I enter some stuff manually in the DB and then load the page it doesnt populate the input fields.
The populate_rt_indexes.php script is missing a required reference to the SphinxSearch class, below is the modified file which has the reference at the top which allowed me to populate the indexes (failed before):
<www_root>\misc\sphinx\ populate_rt_indexes.php
https://gist.github.com/RickDB/f608a46b3c7984f0da29
Hi,
With nn+ you had the option to get an movie release overview by using its IMDB ID like this:
<site>/movies/?imdb=4685096
However with NN-TMUX this will result in an SQL error:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'GROUP BY m.imdbid ORDER BY MAX(r.postdate) desc LIMIT 20 OFFSET 0' at line 23
This was a handy feature as just like with TV shows you get all the releases for a particular movie grouped together and filter / sort afterwards.
Would be great if this could be fixed.
Hi there,
I found the time to make a fresh install of the latest dev.
Great work by the way.
First thing I noticed:
PHP Fatal error: Call to undefined method Nfo::deleteReleaseNfo() in /var/www/html/newznab/newznab/controllers/Releases.php on line 2220
PHP Fatal error: Call to undefined method TvRage::refreshRageInfo() in /var/www/html/newznab/www/admin/rage-edit.php on line 15
Just did some testing with a few users and even though the option "Remember me" option in the login window was ticked they are logged out eventually ( < 120 minutes estimate), database / web server never restarted so can't figure out why that is happening.
With NN+ this worked as I could stay logged in for weeks so don't think it's an php config issue at least, it must something like the "remember me" function not working or it's clearing the sessions.
Also noticed that the redirect URL you get when you have to re-log (redirectUrl=*) isn't redirecting to that url afterwards.
PHP message: PHP Fatal error: require_once(): Failed opening required '/home/nnbeta/www//lib/users.php' (include_path='/home/nnbeta/www/:.:/usr/share/php:/usr/share/pear') in /home/nnbeta/newznab/controllers/NZBVortex.php on line 2" while reading response header from upstream, client: REMOVEDIPADDR, server: REMOVED, request: "GET /nzbvortex?addQueue=c2025e37a716c0c0488d7685ff455881&isAjax&_=1432155911426 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "REMOVED", referrer: "http://REMOVED/details/c2025e37a716c0c0488d7685ff455881"
The path to lame won't save, probably because the lamepath entry is missing in de database after a fresh install.
The nfo is being processed but the releasenfoID is not being updated correctly.
Had a quick look but couldn't find anything obvious.
Many Thanks
Ammendment
Looks like its just on the page view that its not showing the link as a button.
When you view the release in more detail, that part is working okay.
Have a weird issue happening now.
I have the new dev version running now for 3 days, but seeing lots of releases missing.
But when I reset the group last record number by 2,000,000 headers and let it do the pull of headers again, the update releases does find the missing releases.
Done this for all groups and all the missing releases are being found.
Have an old tmux running before the TPG stuff and that is working fine with the same news provider.
Any ideas?
Many Thanks
You may want to change these files in dev branch.
Going through regexless as never used before and want to give it a try.
newznab/NameFixer.php
829 - "INNER JOIN releasesearch rs ON rs.releaseid = r.id
829 + "INNER JOIN release_search_data rs ON rs.releaseid = r.id
1008 - FROM predb p INNER JOIN predbhash h ON h.pre_id = p.id
1008 + FROM predb p INNER JOIN predb_hashes h ON h.pre_id = p.id
newznab/ReleaseSearch.php
43 - $this->fullTextJoinString = 'INNER JOIN releasesearch rs on rs.releaseid = r.id';
43 + $this->fullTextJoinString = 'INNER JOIN release_search_data rs on rs.releaseid = r.id';
www/pages/preinfo.php (predbhash to predb_hashes)
198 - sprintf('SELECT * FROM predb p INNER JOIN predbhash ph ON ph.pre_id = p.id WHERE MATCH(hashes) AGAINST (%s) %s %s %s LIMIT %d OFFSET %d',
198 + sprintf('SELECT * FROM predb p INNER JOIN predb_hashes ph ON ph.pre_id = p.id WHERE MATCH(hashes) AGAINST (%s) %s %s %s LIMIT %d OFFSET %d',
215 - sprintf('SELECT * FROM predb p INNER JOIN predbhash ph ON ph.pre_id = p.id WHERE MATCH(hashes) AGAINST (%s) %s %s %s LIMIT %d OFFSET %d',
215 + sprintf('SELECT * FROM predb p INNER JOIN predb_hashes ph ON ph.pre_id = p.id WHERE MATCH(hashes) AGAINST (%s) %s %s %s LIMIT %d OFFSET %d',
misc/sphinxsearch/toggle_search_type.php has lots of reference to releasesearch to release_search_data
Many Thanks for updating and keeping nntmux fresh.
Line 2257 in newznab/conreollers/releases.php needs to be changed as throwing up errors.
Should be
LEFT OUTER JOIN release_comments rc ON rc.releaseid = r.id
Cheers
Just done a new install with dev and the new schema file.
No releases are being created with reqid in their names.
Downloaded 1,000,000 headers. They are in the binaries table okay.
But nothing in releases.
Any ideas?
Cheers
Thanks for sorting out TV stuff.
When deleting 1 or more items from cart:
2015/05/20 22:13:07 [error] 12691#0: *63298888 FastCGI sent in stderr: "PHP message: PHP Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error' in /home/nnbeta/newznab/db/DB.php:688
Stack trace:
#0 /home/nnbeta/newznab/db/DB.php(688): PDOStatement->fetchAll()
#1 /home/nnbeta/newznab/db/DB.php(665): newznab\db\DB->queryArray('delete from use...')
#2 /home/nnbeta/newznab/controllers/Users.php(697): newznab\db\DB->query('delete from use...')
#3 /home/nnbeta/www/pages/cart.php(30): Users->delCartByGuid(Array, '1')
#4 /home/nnbeta/www/index.php(20): include('/home/nnbeta/ww...')
#5 {main}
thrown in /home/nnbeta/newznab/db/DB.php on line 688" while reading response header from upstream, client: 77.175.170.57, server:REMOVED, request: "POST /cart?delete HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "REMOVED", referrer: "http://REMOVED/cart"
The deletion does work but it throws this error in my nginx error log.
From what it looks it's trying to fetch results from a delete statement which wont work. Just like update/insert.
I been using the dev for a while.
But weird issue today, I cannot login at all.
Was getting SQL errors so changed line 1087 to $querySuffix = "username = $user"; to resolve the sql error.
But still cannot login at all. Any ideas?
Wasn't showing the prehash info in the browse page.
Needed to add case 'ajax_prehashinfo'; into the index.php page.
Cheers
Just upgraded to the new version of the git.
I think that the rarinfo is being used only no matter what you set the in the site setting to not use but to use the rar/7zip.
Can you just check for me
Done the updates up to "Fix wrong path to postprocess...." in dev
On each browse page, I am missing the first page. So page 1 is actually page 2 or results.
Gone through all themes and same issue.
I think its to do with changes to releases.php, but cannot be 100% sure.
Any ideas?
Many Thanks
seems forgot password link fails to get function sendEmail in util.php ?
After upgrading to the latest DEV branch RemoveCrapReleases is crashing on every enabled category when running it thru the tmux session (standalone works fine) because it isn't parsing the range correctly I think:
Running removeCrapReleases for size
Error: Error, time must be a number or full
Also noticing an ZSH error on every pane now and again which varies from zsh:1 to zsh:4 but doesn't seem to affect anything as far as I can tell:
zsh:3: command not found: ^M
zsh:4: command not found: ^M
ZSH is the default shell but used to work fine in nn-tmux before upgrading to latest dev branch today, found a similar issue which suggests the files have Windows style line endings which can't be used:
http://stackoverflow.com/questions/24213094/terminal-errors-on-load-zsh-command-not-found-m
// Update
Also amazon post processing seems to start in the correct tmux pane (post processing) however sometimes moves to the one where monitor.php runs in.
forgot password link not working in new omicron
not giving an option to enter a captcha
The files are being found but no processing is being done, mainly file contents which has a knock on effect later down the line.
New Version of Tmux
229190-7[01/58] - "229190-2.0" yEnc (1/100) YES
228983-11[001/117] - "228983-3.0" yEnc (1/50) NO
147004-0[001/114] - "147004-2.0" yEnc (1/50) YES
Older Version of Tmux
229190-7[01/58] - "229190-2.0" yEnc (1/100) YES
228983-11[001/117] - "228983-3.0" yEnc (1/50) YES
147004-0[001/114] - "147004-2.0" yEnc (1/50) YES
Same with nZEDb so could be linked.
They are mainly series files that have the same initial numbers but different e.g -11 in above example.
Many Thanks
Tried running this script and was working okay about 2 weeks ago. (I think about that time scale)
Now when I do a import it is adding LOTS of groups in group table with date as the group name, and also the rows are all over the place in the prehash table.
Example
Title : 13.36.30'
Source : pred.complete.on.at.least.1.site_i.imgur.com.92Z6J
Files list : N 'Moto3.2014.America.720p.HDTV.x264-W4F'
Weird one this.
The php file is up to date, and tried the one before you made the change, but same output.
Any Ideas?
sabqueuedata.php missing from /pages
seems to be and error:
maybe just mine..
when i am viewing viewseries example: url/series/38657
backchat with jack whitehall and his dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad / Backchat with Jack Whitehall and his Dad
is tvrage etc writing it every time it finds an episode
When new movies (i.e. no imdb/trakt/tmdb info in database) are being processed after binary retrieval it looks them up and they get shown in the covers wall however for newer ones I've seen so far is that they have a linked cover in the database but they don't exist on the filesystem (failed to download?) so you get an error and no cover shown.
Few movies I noticed it with today:
tt1368153
tt0271196
In Admin-Edit site under 3rd Party API Keys I've added all the necessary api keys except those for giant bomb and anidb, since it happens only with new movies this one is a bit hard to recreate as I have to wait for something new to get post processed first.
Checked write permissions in case it was that but they are all setup per the docs (777 for resources etc..), all other things like previews / samples / xxx covers / nzbs are getting created but only for new movies covers it doesn't.
Any ideas where this could be coming from?
// EDIT
Looked in the resources\covers\movies folder and seems some are getting created with an tt prefix, for example:
tt1368153-cover.jpg
So they exist just not with the default naming scheme for movie covers.
Maybe a feature where admin can give mods permission to edit nzb titles and be ranked like the old newzbin mods.
Would need to have a page where all nzbs that are edited can be logged so admin can browse through to be checked etc.
When using an Invite you get the error
"Sorry, the invite code is old or has been used."
Also the text when sending an invite vanishes to quickly:
"Invite sent. Alternatively paste them following link to register"
would be nice feature like the old newzbin where as you put mouse over cmt's also mouse over nfo in browse pages comments are displayed, just saves clicking on nzb to go and read comments
THINK I might have resolved the issue with files being stuck when they hit -2 and thus not showing up on the website
It was due to the "haspreview" being set to 0 and not -1
In postprocessAdditional.php around line 1507
I Changed it from
$iSQL = ', haspreview = 0';
To
$iSQL = ', haspreview = -1';
Wasn't sure if other people have this issue, might want to check before you make changes as I am sure you know hell of a lot more than me how all this works.
Cheers
When I run this file, its finding lots that aren't being picked up in general processing.
Anyway this can be incorporated into the post processing?
Many Thanks
every then a fresh install.
latest dev
No import from:
resources/db/schema/nntmux_fi_schema.sql
Bug:
PARTITION books VALUES LESS THAN (8000);
);
must be:
PARTITION books VALUES LESS THAN (8000)
);
a few bugs from the log
Newznab\db\DB.queryDirect] [SQLSTATE[42S02]: Base table or view not found: 1146 Table 'newznab.binaries_29' doesn't exist]
This one is a little nasty...
newznab\db\DB.queryInsert] [SQLSTATE[42000]: Syntax error or access violation: 1139 Got error 'unmatched parentheses at offset 21' from regexp]
[newznab\db\DB.queryInsert] [INSERT INTO releases (name, searchname, totalpart, groupid, adddate, guid, categoryid, regexid, postdate, fromname, size,
reqid, passwordstatus, completion, haspreview, nfostatus, nzbstatus, isrenamed, iscategorized, reqidstatus, preid) VALUES ('b5fda91a7a932-M2M3NmViOWI3OWQwNzc2YjlmNDhjZDYx', 'b5fda91a7a932-M2M3NmViOWI3OWQwNzc2YjlmNDhjZDYx', 12, 19, now()
, 'e3961c46d3f3b20535c97e9fb30f3dc8782812c5', 10, 1832, '2016-02-18 14:25:17', 'JBinUp.com [email protected]', 0, null , -1, 100,-1, -1, 0, 0, 1, 0, 0)]
looks like a problem with the trigger.
I think its is not a good to use double space´s in the trigger.
newznab\db\DB.queryDirect] [SQLSTATE[42S22]: Column not found: 1054 Unknown column 'r.prehashid' in 'where clause']
[newznab\db\DB.queryDirect] [SELECT r.id AS releaseid, r.guid, r.groupid FROM releases r WHERE r.guid LIKE 'a%' AND r.nzbstatus = 1 AND r.proc_par2 = 0 AND r.prehashid = 0 ORDER BY r.postdate ASC LIMIT 2]
maybe i made something wrong but.
resources/db/schema/nntmux_fi_schema.sql
had no prehashid
maybe interesting for you.
/lib/copy_this/www/lib/Musik.php Line 168
Works suboptimal because there is no settings table.
/lib/DB/patches/0052~gamesinfo.sql Line 9
works better with genre_id
Tried 3 different providers and this is getting failed parts all the time and partrepair is growing.
Compared to both my original installation and nzedb and they are getting none.
I always seem to be bearer of bad news.
Many Thanks
calendar - nothing being listed.
When Sphinx is enabled and RemoveCrapReleases is run it will no longer remove files matching the "executable" option (not sure if affects more) from categories like movies / tv (returns 0 results)
However when I revert to the default search engine (NN_RELEASE_SEARCH_TYPE' = 0) it will return plenty of matches but so much more that I'm not sure if that is correct either (6000 in last 12hrs seems excessive).
This is with releases that have 1 file which have an .exe extension and are posted in a.b.multimedia with category Movies -> SD.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.