This is a performance optimization. Codecs that cant handle that can implement their own one-at-a-time loops

1. Define initial subcode buffer [96] state (based on a standard assortment of subcode)
2. Initial subcode state should include encoded MSF in subq
3. determine which channels need copying to adjust them from the initial buffer state and copy/encode whichever way only those channels
4. Consider how the P subchannel is all 0x00 or 0xFF and this is a good compression opportunity.

hm i should also allow for a variable (specified in header) padding before the raw sector data begins, and possibly also a concept of host media block size, so that we can align disc sectors such that the number of host media block that need to be read can be bounded. for instance suppose the host reads 1MB at a time from its media; we would hate for any sector to straddle that boundary, else it would require two reads from host media.

i want the header to have a metadata format version and/or registry version.

This would include such things as for instance referencing a redump database, and then "identifying" a disc by checking it's hash against the database. But... I guess the existing auditioning tools could just use libjad, huh? Maybe instead we should make an example/reference 'audition' tool which does that to show the way, but leave the bulk of the work to others.

Parse a filesystem and determine what types files are; track those back to sectors armed with knowledge of what type of file it is and choose the best compression filters.
i told you....this...is...exotic.

calculate filters used and compression savings of each (how could you even isolate the effects of individual filters without rerunning the encoding on each sector with different sets of filters?) and print a report, so we can find out which filters are useful and which filters arent. Genererally, at least. Presumably someone would find at least one good test case before committing a filter...

This may be difficult due to the requirements of the flac precoder, but other than that it should be doable. I think whether libjad can be ported to other languages might be a condition for calling it a success. Therefore we might have to let this inform exactly how ruthlessly we strip down libflac when absorbing it (libflac being one of the larger slabs of external code that we'd incorporate)

Once done, implement in bizhawk

Actually, I'm not completely sure whether these belong in jadtool or libjad.

Calculating a hash would only be useful in libjad in case an emulator wanted to load a jad and check the hash itself. Since we're intending for hash memos/tags to be used instead for any purpose an emulator would want, overall I think the actual hashing methods belong in jadtool.

Hashes should include our custom PSX quick-hash, as well as some other suitable hash all over the disc contents, possibly based on what redump would do (although that may not be the best choice). Note that the idealized PSX quick-hash may depend on parsing the filesystem, but we haven't gone that far yet.

Depends on #2, naturally.

it's conceivable that this library could work multi-threaded. each sector can encode in parallel as long as the results are queued to one thread for writing to the output file. even though it will complexify the encoding main loop, it might be worth doing this early on for speedier testing, and it might free us to try wilder ideas for selecting better sector encodings by brute forcing more possibilities.

See also #18

Add support for libmirage and libavcodec and libavformat to the makefile building. This is starting to get into the realm of bulkier dependencies but hopefully it won't hurt the makefile too badly. But at this point, we need a linux developer on-board.

Depends on #12

For better compression, we could cluster several sectors together. This would have to be optional. The size of clusters would be put in the header. Any given user of libjad would have to reject clusters larger than it's prepared to deal with (embedded systems may not have huge sector buffers)

Howevever, due to the concept for how the sector encoding works, this might be really difficult. Basically, all the filters are designed to operate on one sector at a time. Maybe we'd need a special encode stream command which said to extend the range of operation across several sectors worth. For instance a command like "just do basic decompression of 2340 bytes (that is, excluding sync)" would change to "just do basic decompression of 2340 bytes, twice (and skip both sync fields)"

Add support for mednadisc to makefile-based ports. (optional, of course).
This sounds like less work than #13 but may be harder due to mednafen codebase's surliness. Consider re-making mednadisc to be a patch on top of mednafen and then just add mednafen as a dependency, since mednafen's build surliness is already largely tamed.
It should be done earlier in windows simply because I already have it building in windows.
Depends on #12

Should be done after #3 (since #3 is more value, letting us ingest a wider array of weird discs for testing on) (and also since it's no use until cue loading is somewhat respectable)

Another idea would be to add support for ffmpeg (maybe simpler to get rolling in windows than figuring out how to build the whole libav* stuff)

Note, a ZEROES filter would be redundant, since the coder pipeline is defined to start with a zero-filled sector buffer anyway. And at any rate an RLE over a zero-filled sector would reduce to practically the same thing, just a few bytes.

allow format to specify whether index fields are 4byte or 8byte.. 8bytes would only be needed for dvd which im not supporting yet anyway

Depends on #7. Depends on #19

Remember, a disc can have multiple hashes.

This is for spurring development of the basic architecture of the tool and making sure it can scale up to several commands. Also it will exercise the basic libjad apis.

Reminder: jadtool won't be able to print info about non-jad files. For instance jadtool info somefile.cue would not be allowed. All jadtool can do with cues is import (and later maybe export) them. Basically every jad command other than import and export operate on jads solely.

See also #19 (implementing metadata system)

This shouldn't be done for a long time, but it needs to be done eventually

Since cue is a lossy format, occasionally assumptions must be made about how to reconstruct the original disc. I'm not sure where the appropriate point in the code is for these assumptions to be realized, but jadtool will need to receive them, and they need to be stored in metadata

Some thoughts.

1. We could have it only support SBI when importing cue/ccd
2. We could have a jadtool option to apply an SBI
   In both cases, metadata should be added to indicate SBI (with a given hash ID perhaps) has been added

Proof of concept makefile-based compiling of jadtool (and necessarily libjadvac and libjad).
This should not be hard since these are lightweight components. No autotools or cmake or anything is necessary. We'll update the makefile sources by hand (if we dont glob for inputs) when adding sources

png has several precoders (most complex of which is 'paeth') which could be inspirational for compressing some sectors containing bitmaps. although come to think of it that paeth precoder needs some sense of bitmap pitch and that might be utterly impractical here. But we should be able to get mileage out of trying a few different bitplane configurations and seeing how well they compress.

But if you wanted to be really slick you could try various pitches, looking for the one that minimizes entropy from paeth's point of view, judging that to be the correct pitch. this would be a big job, but given that it's being applied a sector at a time, only a small range of pitches would need to be assessed (for instance, pitches of 4000 would not even fit in a sector; pitches of 2000 would not benefit much from the 2d precoding since it would really only be 2 lines. Therefore pitches between 2 and 1000 or so would be reasonable. This would be equivalent to trying to trial-compressing about 1000 45x45 pngs, or in other words, a single 450x450 png--not too bad, right?)

As mentioned in other tickets (#23), having a filesystem parser could help here with selecting the right precoder parameters to use by tracking sectors back to known media types.

implement metadata system
see also #1

primary consideration of metadata system is that it should be updatable without rewriting the jadfile. This means it's in a table at the end of the file, like MP3 tags, you know. However, it's a little cleaner than that, since the jad header can point to the metadata table.

warning: old metadata concepts in docs are probably archaic

MDS or NRG would be good candidates, but this is mostly for the joy of it and the sake of completion--these aren't exactly extremely useful

also a good dumb ISO loader would be a good introductory task for someone (it would be largely based on the libmirage loader, which is not yet as graceful as it could be)

CHD might not make sense, but it's worth investigating. The suspected awkwardness of loading CHD was one of the motivating factors for libjad in the first place; but it would be nice if that awkwardness was resolved for other people once and for all within this pile of code, instead of just avoided (such that they could pluck the chd-reader out of libjadvac if all they wanted was a chd reader)

clarify this concept. internal verification may be privileged--it may be stored in the header and hash the entire contents using a hash guaranteed to be available in libjad. that is, it could hash the sectors, the TOC, the metadata separately, and every single byte of the file. this is independent of identification hashes which libjad doesn't know anything about in detail (but can read out as comments)