jonnyjd / musicbrainz-isrcsubmit Goto Github PK

Right now the script will calculate the discId without connecting to MusicBrainz at all. However, the script will ask for the password even if there is nothing new to submit.

It should be possible to use an anonymous connection for query purposes.

When we find duplicates on the server we always have one track we attached this ISRC to. For that track we know the track number because we gathered it for the submission.

The other track attached to the ISRC on the server might also be on our release, so we can try to find out the track number (on our release) and display it. This might help with the edit message.

We should have a fairly recent version of libdiscid when building isrcsubmit packages.

Having outdated libdiscid versions published isn't going to help adoption of newer libdiscid features.

Maybe I can even get libdiscid 0.4.0 out before that, but having libdiscid 0.3.0 for Mac would help already.

http://www.python.org/dev/peps/pep-0008/ is the way to go.

Isrcsubmit1 is somewhat grown and I don't want to change everything, but maybe it helps just changing enough that isrcsubmit2 can have a better style AND not too much of a difference to isrcsubmit1.

There are some changes in the cleanup branch:
f6a100e (variableName -> variable_names)
and 1f5c9f4 (other PEP 8 changes).

It's probably more difficult to rebase these commits than to re-apply them on current code.

The cleanup branch also moves everything that runs "globally" to main():
That might be a better change for isrcsubmit2-only.

There is a libdiscid branch that reads ISRCs on Mac:
http://bugs.musicbrainz.org/browser/libdiscid/branches/isrc/src/disc_darwin.c
http://bugs.musicbrainz.org/browser/libdiscid/branches/isrc/examples/discisrc.c

This is supposed to be much faster than drutil.

EDIT:
starting with libdiscid 0.3.2 this backend is also for Linux available (and since 0.3.0 for Mac and Windows)

If multiple tracks on a CD have an empty ISRC, isrcsubmit will register them as having the same ISRC (which, I guess, they technically do), and will also try to submit these empty ISRCs to MusicBrainz.

See this log: https://pastee.org/6g6wt

When multiple backends are implemented and available we have some kind of priority.

Whatever the priority is, there needs to be an option to choose a specific backend manually.

It might be wise to switch to argparse/optparse, like mineo already did:
https://github.com/mineo/mbstuff/blob/master/isrcsubmit-cdrdao.py

However, I would like the option parsing to not be as strict.
isrcsubmit <username>
should still be possible.

When a discid is not found this Error occurs:

ERROR: Couldn't fetch release: caused by: HTTP Error 404: Not Found

What should happen is that isrcsubmit asks the user to submit the disc ID to MusicBrainz.

Could it be possible to start the release number from 1 instead of from 0 in please select your release
And could we have the 0 for quitting program (no other way than CTRL+C now, and this is not good result when using a dos.bat file which then asks for quit confirm).

We have a couple of problems with multi-disc-releases with python-musicbrainz2 and a version that actually works with the webservice 2 somehow would have benefits.

There are two possibilities:

1. Use the webservice directly.
   This might not be the most efficient or sustainable, but there is already a script doing exactly this: isrcsubmit.py (kuno) This is maybe enough or at least a starting point.
2. Use python-musicbrainz3
   We might have to wait until this is usable and actually available for the users. At the mentioned repository I don't see commits since 2010, but there is also a github repo: alastair/python-musicbrainz-ngs

As far as I see, Alastair's library is the most useful, but python-musicbrainz3 is still the official one.
See also:
http://comments.gmane.org/gmane.comp.audio.musicbrainz.devel/4510
http://musicbrainz.1054305.n4.nabble.com/python-musicbrainz3-or-python-wrapper-for-libmusicbrainz-4-td3708264.html

The problem I see with Kuno's version is that it is less robust. As far as I see, you only attach ISRCs to recording-MBIDs. This is not wrong, but you don't see what you actually do. If I would have taken that approach with my script when I started, I would have added many ISRCs in the wrong place without knowing or had to check the IDs on the web.

Right now the cdrdao backend always issues an ERROR telling that only the default device can be used.

This should be a warning, rather than an error and maybe it only needs to be displayed when cdrdao scanbus actually detects more than one drive and the device is not the defaultDevice.

The next step would be to include an cdrdao-drive option (only for windows) and mention that in the warning (if it is not set).

The reason why that problem exists at all is because on windows cdrdao doesn't work with drive letters, but with devices numbers like 1,0,0.

If I have a DVD or Blu-ray or other release that isn't a CD, which has a CD as an additional medium in its release (e.g., https://musicbrainz.org/release/0600cd78-a7b8-4d1b-8922-b0f1679f8e74), running isrcsubmit on the CD will make it think disc 2 is the bonus disc and apply the ISRCs to the first (DVD/Blu-ray) medium instead of the CD.

The check that seems to be at fault:

    elif discIdCount == 1 and disc.trackCount < releaseTrackCount:
        # bonus DVD (without disc ID) given in MB?
        # better handling in version 2 api
        trackOffset = 0
        print("This release probably has a bonus DVD without a discID.")
        print("Guessing track offset as %d" % trackOffset)

Perhaps there's a way to tell straight from the WS response which medium it should be applied to, but I don't have time to look into it more right now. :/

Nobody would download these and nobody is going to download this script as tar.gz. However, having these here might be nice.

README can be README.md, since it stays at github anyways.

There is already a windows tool to submit ISRCs: http://forums.musicbrainz.org/viewtopic.php?id=1838
and I don't use windows.

However, there are request of getting ISRCs with mediatools because the algorithm will always (?) get correct ISRCs.

Additionally the windows tool still has the same problems with multi-disc releases we had with #16

the help specified 'The default is /dev/rdisk2 (and '1' for mac)' for device, but it errors out for me. It works if I manually specify '-d 1'.

Starting with python-musicbrainz2 0.7.4 (Nov2011) we can add our own user agent string.

The main reason:
http://musicbrainz.org/doc/XML_Web_Service%2FRate_Limiting

See also the forked commit:
mineo/mbstuff@f3a9210#isrcsubmit-cdrdao.py (using "isrcsubmit-cdrao")

isrcsubmit-icedax 0.2.5 as agent string?

I'm trying to submit ISRCs for CD 1 of http://musicbrainz.org/release/70dfcd0a-29ed-4a5f-ab72-e2be62eadd85

$ isrcsubmit.py foolip
isrcsubmit 0.4 by JonnyJD for MusicBrainz

using python-musicbrainz2 0.7.4
using Cdrdao 1.2.3

Please input your Musicbrainz password
Password:

DiscID: PC5dKsHXfLSTtLUNUgecT68qGBI-
Tracks on Disc: 11
Artist: 陳奕迅
Release: 上五樓的快活
Tracks in Release: 15
CRITICAL: track count mismatch!
CRITICAL: There are 11 tracks on the disc,
CRITICAL: but 15 tracks on a SINGLE-disc-release.
CRITICAL: This is not supposed to happen.

Looks like an old API that doesn't understand mediums is being used?

You only need a browser submitting new discIDs and showing information about duplicates. If the browser cannot be found, the link is given in the output.

This should work similar to the backend options, meaning: We have an internal list and check which browser to use automatically (if available) and have an option to set it explicitely.

In contrast: there is no real reason to restrict the option to a certain list. Every browser should allow browserprog [url].

Please comment if you think this is an important feature.

According to http://www.ifpi.org/content/section_resources/isrc.html drutils is suggested for extracting ISRCs on mac.

This should also work on linux, but I couldn't find a package in Arch. I might create one and see if it works.

With Windows we now have the second platform with users that are not used to using the terminal. Having a isrcsubmit.bat that you can just "double click" might help.

We should just ask for the username when we ask for the password (if the username is not provided)

On Linux we work with "which" to find out if a backend exists without actually starting the backend.

On Windows we normally don't have "which", but if GNU which is installed, it somehow doesn't detect if a backend exists.
Possibly because which mediatools.exe works, but "which mediatools" doesnt.

Thanks @jesus2099 for reporting.

Clean up after implementing #14

submit should be renamed to verified in most places or similar. It is all about verification, not about whether we want to submit at all. The user decides if he wants to submit in the end.

It would be handy if isrcsubmit could read/submit ISRCs from .cue files which include them.

1f7fa42 fixed the first regression (some single disc releases had unnecessary offset questions)

So having master as completely stable branch did not work?
Maybe I should work on dev branches between releases?

Especially because https://github.com/JonnyJD/musicbrainz-isrcsubmit/raw/master/isrcsubmit.py would be a nice "latest stable" download link.
Another option is to create a stable branch, but since master is the default, this might not be optimal.

Note, that also, that https://raw.github.com/JonnyJD/musicbrainz-isrcsubmit/v0.2.4/isrcsubmit.py is a valid download for 0.2.4.

The github integrated download (for tags) is not really usefull, by the way.

Would it be possible to fallback backend when needed.

I still need isrcsubmit.exe because of when the ISRC are in the CD-TEXT and not in the Q-Channel (it just happened again today with one of my library’s WIGCD162 ☜ linking in case you have it).

In these case, isrcsubmit.exe has correct values (I think) as it takes all the ISRC at once from some kind of TOC instead of bad reading the Q-Channels of each tracks.

reported by @kaikasurinen:
https://gist.github.com/kaikasurinen/3f2904a7d807ec3571ae

DiscID:     8FsDc5De9N.Fu_VnUcb9l07xc1A-
...
3: Wolfgang Amadeus Mozart - The Classic Composers, Volume 3: Musical Masterpieces (Official)
Traceback (most recent call last):
  File "./isrcsubmit.py", line 879, in <module>
    releaseId = disc.release.getId()        # implicitly fetches release
  File "./isrcsubmit.py", line 491, in release
    self._release = self.getRelease(self._verified)
  File "./isrcsubmit.py", line 518, in getRelease
    release_type = release.getTypes()[1].rpartition('#')[2]
IndexError: list index out of range

This is in windows version

isrcsubmit 0.5 by JonnyJD for MusicBrainz

using python-musicbrainz2 0.7.4
using mediatools
e:
\\.\e:

DiscID:         TK5efmSk3QXYTIqtVZuCGisoJDg-
Tracks on Disc: 16
This Disc ID is not in the Database.
recalculating to re-check..
This Disc ID is not in the Database.
Would you like to open the browser to submit the disc? [y/N] y
Traceback (most recent call last):
  File "isrcsubmit.py", line 726, in <module>
    releaseId = disc.release.getId()        # implicitly fetches release
  File "isrcsubmit.py", line 392, in release
    self._release = self.getRelease(self._submit)
  File "isrcsubmit.py", line 465, in getRelease
    self.getRelease(submit=True)
  File "isrcsubmit.py", line 456, in getRelease
    % (options.browser, str(e)))
TypeError: not all arguments converted during string formatting

To keep backporting easy, we should try to make obvious fixes to the 1.0 branch to work mostly in python3 before we fork 1.0 to work on 2.0.

That includes:

• print "as a function" (in python2 with only 1 argument = no actual tuple/arglist, be aware of newlines!)
• except Error AS err
• raw_input -> input (try input = raw_input except: pass)

note: python-musicbrainz2 only works with Python 2 (though it imports fine), so this won't help isrcsubmit 1.0 to work with Python 2. We should test that anyways.

We want to keep the users from submitting new discIDs just because there drive "wasn't ready".

If you insert a disc in the drive and start isrcsubmit rightaway, the discID might be different than with letting the drive settle.

The release name contains \u2019 apostrophe :

isrcsubmit 0.5 by JonnyJD for MusicBrainz

using python-musicbrainz2 0.7.4
using Cdrdao 1.2.3
e:
\\.\e:

DiscID:         0wD2_XE_WlugbdbCFq1wM95MW38-
Tracks on Disc: 26
This Disc ID is ambiguous:
0: Miles Davis - Ascenseur pour l'échafaud (Official)
        JP      2004-03-09
1: Miles Davis - Ascenseur pour l'échafaud (Official)
        DE      1989-03-20       042283630529
2: Miles Davis - Traceback (most recent call last):
  File "isrcsubmit.py", line 726, in <module>
    releaseId = disc.release.getId()        # implicitly fetches release
  File "isrcsubmit.py", line 392, in release
    self._release = self.getRelease(self._submit)
  File "isrcsubmit.py", line 416, in getRelease
    print "-", release.getTitle(),
  File "D:\Tristan\PRG\i\Python26\lib\encodings\cp850.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019' in position 16: character maps to <undefined>

Getting TypeError with DiscID "Rl7E4c2SbxEwLh9gwYX57V6pR6I-":

freso@kotake> isrcsubmit.py --debug Freso
isrcsubmit 0.4.2 by JonnyJD for MusicBrainz

using python-musicbrainz2 0.7.4
using Cdrdao 1.2.3

DiscID:     Rl7E4c2SbxEwLh9gwYX57V6pR6I-
Tracks on Disc: 20
Traceback (most recent call last):
  File "/usr/bin/isrcsubmit.py", line 616, in <module>
    release = query.getReleaseById(releaseId, include=include)
  File "/usr/bin/isrcsubmit.py", line 314, in getReleaseById
    return self._query.getReleaseById(releaseId, include=include)
  File "/usr/lib/python2.7/site-packages/musicbrainz2/webservice.py", line 1044, in getReleaseById
    result = self._getFromWebService('release', uuid, include)
  File "/usr/lib/python2.7/site-packages/musicbrainz2/webservice.py", line 1186, in _getFromWebService
    stream = self._ws.get(entity, id_, includeParams, filterParams)
  File "/usr/lib/python2.7/site-packages/musicbrainz2/webservice.py", line 274, in get
    url = self._makeUrl(entity, id_, include, filter, version)
  File "/usr/lib/python2.7/site-packages/musicbrainz2/webservice.py", line 244, in _makeUrl
    path = '/'.join((self._pathPrefix, version, entity, id_))
TypeError: sequence item 3: expected string, NoneType found

lines 897 and 898:
ws2.submit_isrcs(tracks2isrcs)
print("Successfully submitted %d ISRCS." % len(tracks2isrcs))

There's no check for failure, leading to messages like:

ERROR: Couldn't send ISRCs: caused by: HTTP Error 400: Bad Request
Successfully submitted 19 ISRCS.

Catalogue numbers are easy and very quick spot for me.
Could you display them, additionally to bar codes ?
(no need for labels though)

If you look up a DiscID which matches multiple releases and press enter (or presumably any non-numeric character), you will get this nice little exception:

Traceback (most recent call last):
  File "/usr/bin/isrcsubmit.py", line 620, in <module>
    releaseId = disc.release.getId()        # implicitly fetches release
  File "/usr/bin/isrcsubmit.py", line 348, in release
    self._release = self.getRelease(self._submit)
  File "/usr/bin/isrcsubmit.py", line 382, in getRelease
    self._release = results[int(num)].getRelease()
ValueError: invalid literal for int() with base 10: ''

Right now we can distinguish between differences in:
Title, Type (Official/Bootleg)

There are examples where this is not enough. Maybe because there is one release per release-event in NGS.

Example:
http://musicbrainz.org/release-group/2fc8f9fc-50a8-339f-87b1-243314f168f1

Found this while testing multi-disc (the example has 3 discs)

libcdio support was requested by sbontrager on the MB forums.

cd-info (part of libcdio) does read ISRCs just fine (not only in CD-TEXT) and gives something like this:

...
Media Catalog Number (MCN): 0000000000000
TRACK  1 ISRC: USXH21200030
TRACK  2 ISRC: USXH21200031
TRACK  3 ISRC: USXH21200032
...

There is also a python library pycdio, but I couldn't find anything of much help there. We should probably just parse cd-info output.

reported by @jesus2099 in #33 (comment)

I have a global name 'saneWhich' is not defined error when I try to use cdrdao.

hasBackend() is used when checking for a backend given as option before saneWhich is defined.

icedax (in cdrkit) is a fork of cdda2wav (in cdrtools) and is not well maintained and might also miss some features.

Icedax is however the standard in major distributions.

The icedax packages mostly include a cdda2wav symlink.

If icedax is not detected, cdda2wav should get tried. The output of both is the same in terms of what is interesting for us.

1. Could it be possible to display full URL of Disc ID instead of just Disc ID (http://musicbrainz.org/cdtoc/vBGR9DI85nmXBvcZvAzrnbY2eLA-) ?
2. Could it be possible to display the device name on one (better) line instead of two ?
   using devide e: instead of e: + \\.\e:

Here is an extract from a cdrdao TOC (of http://musicbrainz.org/release/f1c048a3-69cc-4906-9ce1-3bfbfc76f951):

// Track 1
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
ISRC "DED811023010"
CD_TEXT {
  LANGUAGE 0 {
    TITLE "Get The Party Started"
    PERFORMER "VIVA VOCE"
    ISRC ""
  }
}
FILE "data.wav" 0 02:46:58


// Track 2
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
ISRC "DED811023020"
CD_TEXT {
  LANGUAGE 0 {
    TITLE "Die Welt, ein Dorf"
    PERFORMER "VIVA VOCE"
    ISRC "DED811023020"
  }
}
FILE "data.wav" 02:46:58 04:00:59

As you’ll see, the CD has CD-Text which also includes ISRCs for each track (though, for some reason, it is not set on track 1). The parser doesn’t expect two lines containing “ISRC” and fails:

found new ISRC for track 1: DED811023010
Traceback (most recent call last):
  File "./isrcsubmit.py", line 658, in <module>
    track = tracks[trackNumber + trackOffset - 1]
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Probably the simplest fix would be to ignore all lines starting with a space.

Since I can't upload files to github anymore and the download section disappeared, I have to make some changes in the documentation.

I also might change every link to the github project to point to jonnyjd.github.com/musicbrainz-isrcsubmit, since there is no download link at the repo anymore (except the zipped repo, which only works well for linux).

I should also update the install information to cater for the mac and windows users.

The original idea was to drop backends completely and use libdiscid ISRC support.

However, I think libdiscid 0.3.2 won't be in openSUSE 12.3 and also not in Ubunut 13.04.
This means, that many Linux users probably won't have a libdiscid version that supports ISRC reads (without using backports).

So we might drop only backends that only work in Windows and Mac OS X, since for these packages we ship libdiscid anyways.
Might keep supporting mediatools until libdiscid fixes metabrainz/libdiscid#6

Not sure if that makes sense anyways, since python-discid also won't be in the repos and we WILL use python-discid because python-musicbrainz2 doesn't work with Python 3.

So we might drop backends anyways.

(Feedback welcome)

EDIT:
It is already planned to maintain isrcsubmit1 for a while. At least until the dependencies of isrcsubmit2 are somewhat available.
We might also provide a backport PPA for libdiscid on some ubuntu releases.

example:
http://musicbrainz.org/edit/19736151

When our discId is the last discId found for a release in the database, we guess this is the last disc for the release.
However, there can be another disc after that if that disc has no discId. This is always the case for (bonus) DVDs.

The frequent case CD+DVD is only affected if there are multiple discIds given for the first disc AND our discId is the last discId in the list.

see also: mineo/mbstuff@c1e90aa

same problem as: http://tickets.musicbrainz.org/browse/LMB-3

This is already fixed in a forked version of this script:
https://github.com/mineo/bin/blob/c381519c7a8080c3bbbdd7f1521b28c14d6d4e94/isrcsubmit-cdrdao.py
(first version on github already has this fix)

There the solution is to add an offset on the commandline, because the tracks from both discs are listed.

Possibly there is an automated solution like on the above mentioned ticket for the C++ isrcsubmit (not on github)

The Windows code for libdiscid has:

 strcpy(filename, "\\\\.\\");
        len = strlen(device);
        if (colon = strchr(device, ':')) {
                len = colon - device + 1;
        }
        strncat(filename, device, len > 120 ? 120 : len);
        printf("%s\n", device);
        printf("%s\n", filename);

Which on our end leads to this output:

D:
\\.\D:

after using mediatools.

There is no output like this on other platforms.

Since we don't print these lines, we can't just "not print" these. We can try to disable stdout just before we use libdiscid: http://stackoverflow.com/questions/4178614/suppressing-output-of-module-calling-outside-library

When no ISRCs are on the disc, discisrc still prints Track #: for every Track.
This then leads to:

can't find ISRC in: Track 1  : 

can't find ISRC in: Track 2  : 

can't find ISRC in: Track 3  : 

...

Other backends just keep quiet and tell "no new ISRCs found" a the end.

0.7.4 is needed for the user agent string (otherwise we have sporadic server errors)

0.7.3 is needed AUTH DIGEST, which is necessary for NGS?
See: http://bugs.musicbrainz.org/ticket/5905

Mineo did also something similar:
mineo/mbstuff@f3f1fa9#isrcsubmit-cdrdao.py

According to http://forums.musicbrainz.org/viewtopic.php?id=2400 icedax sometimes has issues with extracting invalid ISRCs.
cdrdao seems to fix this.

It is unknown if using cdda2wav istead of the fork icedax also solves problems like these.

The mineo version of isrcsubmit uses cdrdao: https://github.com/mineo/mbstuff/blob/master/isrcsubmit-cdrdao.py