Git Product home page Git Product logo

dawfilekit's Introduction

About Me

My name is Steffan Andrews and I hail from beautiful Vancouver, Canada.

I'm a professional composer for film and TV with over 16 years of experience specializing in animated series.

I'm also a life-long coder since 1995. I have helped build numerous commercial Mac applications including Audio Design Desk and Dipper. In my spare time I build and maintain open-source packages for Swift and SwiftUI.

Featured Swift Packages

Modern multi-platform Swift CoreMIDI wrapper with MIDI 2.0 support.
Open Sound Control (OSC) library written in Swift.
A robust and precise Swift library for working with SMPTE timecode.
Translate integers to/from radix strings (binary, hex, etc.) using convenient syntax.

Featured SwiftUI Packages

SwiftUI menu builder DSL & controls that mimic macOS Control Center.
Show/hide SwiftUI MenuBarExtra menu using Bindings.
Open your macOS app's Settings scene programmatically without needing SettingsLink.

dawfilekit's People

Contributors

orchetect avatar rex4539 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

dawfilekit's Issues

MIDI File Support

  • Extract markers from MIDI File, calculating timecodes based on tempo track
    • Could reuse/adapt the Cubase XML import code calculations based on a multi-event tempo track
  • Export markers to MIDI File
    • Test fractional tempos that are not common multiples of 60, such as 152.5
    • Add "Lossy convert extended characters" function that can convert special unicode chars in Text events to MIDI file ASCII-compatible equivalents. (There's native string API in Foundation/AppKit that can help with this.)

Final Cut Pro clipboard data read/write

Related to #22, out of curiosity I poked around at the clipboard content Final Cut Pro uses when copying markers from the application.

It may be possible to add methods to DAWFileKit to read/write FCP clipboard contents.

FCP uses the following UTI type:

extension NSPasteboard.PasteboardType {
    /// Final Cut Pro's pasteboard UTI type. The content is a binary plist.
    static let finalCutPro = Self("com.apple.flexo.proFFPasteboardUTI")
}

The content of the data is a binary plist. Converted to XML, this is what it looks like:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>ffpasteboardcopiedtypes</key>
    <dict>
        <key>pb_anchoredObject</key>
        <dict>
            <key>count</key>
            <integer>1</integer>
        </dict>
    </dict>
    <key>ffpasteboardobject</key>
    <data>
        <!-- base64-encoded data -->
    </data>
    <key>kffmodelobjectIDs</key>
    <array/>
</dict>

The ffpasteboardobject data blob contains a binary plist payload encoded by NSKeyedArchiver.

ffpasteboardobject

The root object is NSDictionary so it should be possible to deserialize the archive into a dictionary. It's not immediately apparent if the dictionary structure is similar to FCP XML.

ffpasteboardobject-root-obj

ffpasteboardobject-root-obj2

It definitely contains the selected markers from Final Cut Pro marker list and their details (start, duration, name).

ffpasteboardobject-marker

However, attempting to directly unarchive the data will produce errors because proprietary FCP classes are included in the archived data.

try NSKeyedUnarchiver.unarchivedObject(
    ofClasses: [NSDictionary.self, NSMutableArray.self],
    from: data
)
Error Domain=NSCocoaErrorDomain Code=4864 "*** -[NSKeyedUnarchiver decodeObjectForKey:]: cannot decode object of class (FFAnchoredCollection) for key (NS.objects) because no class named "FFAnchoredCollection" was found; the class needs to be defined in source code or linked in from a library (ensure the class is part of the correct target).

fcp-classes

I'm not sure if Apple's ProExtension framework (which contains some FCP objects) has these classes. Worth checking out.

This means it may be necessary to manually parse the archived data, or at least selectively unarchive its children instead of attempting to unarchive the entire root dictionary. (It does not seem practical to try to model all known proprietary class types for the unarchiver to use.)

MusicXML Support

Proposal

Add support for reading and authoring MusicXML.

Support for markers will be the main priority.

Considerations

A bit of testing is needed to determine if precise timing information can be derived.

  • There is a possibility that it may be a fairly lossy format.
  • It's also not totally clear whether MusicXML supports a novel 'marker' element type. It appears be treated as generic text which means that various DAWs may encode or interpret it differently.
  • In practicality, using MusicXML may not be as worthwhile as other potential formats.

Preliminary Observations

For example, a MusicXML xml file exported from Digital Performer 10 is summarized below.

  • It contains two markers: one at 1:00:00:00, and another at 1:00:01:00
  • DP exports markers as Direction markings. The <direction> element with inner element <words> are used.
  • Similar to a MIDI file, it contains tempo, time signature, and PPQ information.
  • However, no frame rate or origin timecode information is included and would need to be supplied separately by the user.
<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE score-partwise PUBLIC "-//Recordare//DTD MusicXML 3.0 Partwise//EN" "http://www.musicxml.org/dtds/partwise.dtd">

<score-partwise>
	<identification>
		<creator type="composer">Steffan Andrews</creator>
		<rights>© Steffan Andrews</rights>
		<encoding>
			<software>Digital Performer 10.13 86175</software>
			<encoding-date>2021-02-06</encoding-date>
			<supports attribute="new-system" element="print" type="no" value="yes"/>
			<supports attribute="new-page" element="print" type="no" value="yes"/>
			<supports element="accidental" type="yes"/>
			<supports element="beam" type="yes"/>
			<supports element="stem" type="no"/>
		</encoding>
	</identification>
	<part-list>
		<part-group number="2" type="start">
			<group-symbol>bracket</group-symbol>
			<group-barline>yes</group-barline>
		</part-group>
		<score-part id="P0">
			<part-name>MIDI-1</part-name>
			<part-abbreviation>MI.1</part-abbreviation>
		</score-part>
		<part-group number="2" type="stop"/>
	</part-list>
	<part id="P0">
		<measure number="1">
			<attributes>
				<divisions>480</divisions>
				<key>
					<fifths>0</fifths>
					<mode>major</mode>
				</key>
				<time>
					<beats>4</beats>
					<beat-type>4</beat-type>
				</time>
				<staves>1</staves>
				<clef number="1">
					<sign>G</sign>
					<line>2</line>
				</clef>
				<transpose>
					<diatonic>0</diatonic>
					<chromatic>0</chromatic>
					<octave-change>0</octave-change>
				</transpose>
			</attributes>
			<note>
				<rest measure="yes"/>
				<duration>1920</duration>
				<voice>1</voice>
				<staff>1</staff>
			</note>
			<direction placement="above">
				<direction-type>
					<metronome>
						<beat-unit>quarter</beat-unit>
						<per-minute>120.00</per-minute>
					</metronome>
				</direction-type>
				<offset>-1920</offset>
				<voice>1</voice>
				<staff>1</staff>
				<sound tempo="120.00"/>
			</direction>
		</measure>
		<measure number="2">
			<note>
				<rest measure="yes"/>
				<duration>1920</duration>
				<voice>1</voice>
				<staff>1</staff>
			</note>
		</measure>

                <!-- ... (snipped: empty measures 3 through 1800 which are identical to measure 2) ... -->

		<measure number="1801">
			<note>
				<rest measure="yes"/>
				<duration>1920</duration>
				<voice>1</voice>
				<staff>1</staff>
			</note>
			<direction placement="above">
				<direction-type>
					<words>Unlocked Marker 1_00_00_00</words>
				</direction-type>
				<offset>-1920</offset>
				<voice>1</voice>
				<staff>1</staff>
			</direction>
			<direction placement="above">
				<direction-type>
					<words>Locked Marker 1_00_01_00</words>
				</direction-type>
				<offset>-960</offset>
				<voice>1</voice>
				<staff>1</staff>
			</direction>
		</measure>
		<measure number="1802">
			<note>
				<rest measure="yes"/>
				<duration>1920</duration>
				<voice>1</voice>
				<staff>1</staff>
			</note>
			<barline location="right">
				<bar-style>light-heavy</bar-style>
			</barline>
		</measure>
	</part>
</score-partwise>

References

https://github.com/w3c/musicxml/blob/799e2defb2ece0ae7bafe08dcbcac25b2c631d53/schema/direction.mod#L83-L97

https://github.com/w3c/musicxml/blob/799e2defb2ece0ae7bafe08dcbcac25b2c631d53/schema/direction.mod#L117-L127

Final Cut Pro FCPXML Support

Baseline

  • Support for parsing most common FCPXML elements into model objects, primarily with a view to extracting annotations such as markers, keywords, captions, etc.
  • Reasoning on the model:
    • Extracting elements (clips, markers) and calculating their absolute start timecode on the main timeline
    • Filtering and contextual data gathering for extracted events

Parsing and Element Extraction

  • sync-clip roles
  • mc-clip roles
  • analysis-marker
  • Method to parse top-level story elements that are immediate children of fcpxml
  • Detection for elements that are out of their containing clip's timecode bounds
    • write FCPXML utility method to detect if a clip or attribute (marker, caption, etc.) is out of bounds of the clip it's within
    • but also have a method that can recursively check up all the parent breadcrumbs to see if it's occluded from the main timeline
    • add property to ExtractionSettings to allow filtering out elements that are either partially or fully out-of-bounds, and therefore not visible on the main timeline
  • Change absoluteStart to inTime. Add outTime property to context, calculated from inTime + duration.
  • Add logic to convert sub-role names to their base role. ie: convert "Dialogue.Dialog-1" to "Dialogue"
  • Add progress reporting to element extraction (useful for very large FCPXML documents)

Frame Rate Scaling

Frame rate scaling for clips using conform-rate child element.

As support for timeline/media frame rate pairs are added, this table will be updated. (Project rates in the 1st column and media rates in subsequent columns.)

Project rate 23.98 24 25 29.97 29.97d 30 50 59.94 60
23.98 N/A ? X ? ? ? ?
24 X N/A X X
25 N/A
29.97 X ? N/A ? ? ? ?
29.97d X X N/A
30 X X N/A
50 N/A
59.94 X X N/A
60 X X N/A

X = implemented and unit tested
? = should work in theory, being reciprocals of already-tested rate pairs, but have not been explicitly unit tested

Authoring

  • Make all model attributes settable
  • Make all model children mutable / settable
  • Add a mechanism to be able to author elements by supplying their absolute start timecode value
  • API ergonomics and considerations

Refactors

  • Extensively implement throwing methods with propagated error handling
  • Refactor FCPXML to remove resources parameter in inits and force them to parse exclusively from the XML, including parsing out resources when necessary
  • Aggregate accumulation of Timecode can result in subframe aliasing where Final Cut Pro shows timecode for elements (markers, captions, keywords, etc.) 1-2 subframes off
    • may need to store atomically as CMTime or Fraction and convert to Timecode ad-hoc
    • not an issue when times fall on frame boundaries and do not have subframes
  • API adjustments and refactors to enable writing model contents to disk as FCPXML

Technical

  • Add pretty debugDescription output for Any* enum cases, etc.
  • Performance optimization: concurrency for FCPXML file parsing to increase performance
    • Each XML element's child notes could be parsed in parallel to speed up parsing. And that could happen with every element in the hierarchy.
  • Make types Sendable

To Investigate

  • Bundle FCPXML DTDs and validate input FCPXML files?
  • FCPXML note is a child element for clips, but an attribute for annotations
  • Implement predefined formats?
    Not sure if it's possible to sometimes only have the format string and no attributes like frameDuration, in which case we would need to add frame rate info for each one of these to derive it
  • read/write FCP clipboard contents (see #26)

Testing

  • Audio/video role attribute parsing & storage in structs
  • Edge cases involving new-line characters in notes or other text elements
  • Occlusion where elements with non-0 lanes extend past clip(s) beneath them

Cubase XML: Calculating event times when tempo ramp events are used

Issue

Cubase tempo tracks can contain one or more jump or ramp tempo events. A jump is an immediate jump to a new tempo. A ramp is a gradual ramp between two tempo events.

Currently, DAWFileKit can accurately determine event start times (and timecodes) when "jump" tempo events are used exclusively. However, DAWFileKit cannot yet properly calculate event start times if one or more "ramp" tempo events are used. The time calculation that Cubase uses needs to be intuited.

Observations

The ramp is linear when in the musical domain (bars + beats). However this results in a curved ramp when in the real time domain.

For consideration, this is a Cubase project used in DAWFileKit unit tests (after being exported as a Track Archive XML file). The ramp curves can be observed in Cubase by changing the visible time scaling mode.

Bars + Beats Linear:
bars-beats-linear

Time Linear:
time-linear

Current Progress

The unit test involving the Cubase project shown above is already written, but the four marker events that overlap and trail the tempo ramps have been commented out, as they will fail currently since the calculations are not yet correct.

#warning(
"> TODO: these four asserts are correct, but will fail for now until tempo ramp events are implemented"
)
let track1event7 = track1?.events[safe: 6] as? Cubase.TrackArchive.Marker
// XCTAssertEqual(track1event7?.startTimecode.stringValue, "01:00:26:02")
_ = track1event7 // silence warning
let track1event8 = track1?.events[safe: 7] as? Cubase.TrackArchive.Marker
// XCTAssertEqual(track1event8?.startTimecode.stringValue, "01:00:29:09")
_ = track1event8 // silence warning
let track1event9 = track1?.events[safe: 8] as? Cubase.TrackArchive.Marker
// XCTAssertEqual(track1event9?.startTimecode.stringValue, "01:00:31:24")
_ = track1event9 // silence warning
let track1event10 = track1?.events[safe: 9] as? Cubase.TrackArchive.Marker
// XCTAssertEqual(track1event10?.startTimecode.stringValue, "01:50:25:07")
_ = track1event10 // silence warning

The currently incomplete calculation is found here:

case .rampToNext:
#warning("> TODO: This calculation is not accurate, it is merely approximate.")
// Cubase (and other DAWs like Logic Pro X) have mysterious tempo ramp calculation
// algorithms
// I was not able to precisely reverse engineer the algo Cubase uses
// This is as close as I could get to approximate the calculation

Pro Tools 2023.12 New Markers Features

https://www.avid.com/resource-center/whats-new-in-pro-tools-202312

Pro Tools 2023.12 adds new marker features such as multiple marker rulers and improved marker window ("memory location" window) filtering and filter presets.

It also adds more control over how markers on tracks and markers in rulers are imported and exported - but only to/from Pro Tools sessions it seems. And its session file format remains fairly proprietary and unreadable/unwritable.

For interchange with other software, there seems to be no new features. We are still limited to:

  • Session Info Text file export (which DAWFileKit extensively supports)
  • MIDI file import/export (DAWFileKit supports exporting MIDI files with markers and will be improved in future; importing is on the to-do list)
  • AAF/OMF do not carry marker information (and are notoriously difficult to deal with as file formats any how)

It would be worth checking to see if the session info text file's format has changed in any way with the new updates.

The last update round, Pro Tools introduced track markers (#20) and it did not alter the text file formatting at all. Track markers are merged with ruler markers and exported in the same single marker list in the text file, without any additional information regarding their usage or associated track name from the session.

ProTools Session Info Text: TextEdit Character Encoding

Issue

In Pro Tools during ExportSession Info as Text, if the text encoding is set to TextEdit (which is the default selection), it lossily converts various non-standard/extended characters in text strings (such as Marker name / comments).

Common Examples

Character Description Exported Notes
U+2013, En Dash Ð
U+2014, Em Dash Ñ Keyboard input character substitution on macOS
U+2019, Right Single Quote Õ Sometimes when copying from Microsoft Word; should be an apostrophe
U+2026, Ellipsis É Keyboard input character substitution on macOS

Example 'UTF8' Format

T R A C K  L I S T I N G
M A R K E R S  L I S T I N G
#   	LOCATION     	TIME REFERENCE    	UNITS    	NAME                             	COMMENTS
1   	01:00:00:00  	480000            	Samples  	Test Ellipsis…                   	
2   	01:01:00:00  	3360000           	Samples  	Test Em Dash —                   	
3   	01:02:00:00  	6240000           	Samples  	Test En Dash –                   	
4   	01:03:00:00  	9120000           	Samples  	Right Side Quote’s Not An Apostrophe	

Example 'TextEdit' Format (Pro Tool's default when exporting)

T R A C K  L I S T I N G
M A R K E R S  L I S T I N G
#   	LOCATION     	TIME REFERENCE    	UNITS    	NAME                             	COMMENTS
1   	01:00:00:00  	480000            	Samples  	Test EllipsisÉ                   	
2   	01:01:00:00  	3360000           	Samples  	Test Em Dash Ñ                   	
3   	01:02:00:00  	6240000           	Samples  	Test En Dash Ð                   	
4   	01:03:00:00  	9120000           	Samples  	Right Side QuoteÕs Not An Apostrophe	

Proposed Solutions

  • Add note to documentation suggesting users export using UTF-8 format and not TextEdit format.
  • A heuristic could be employed to convert some of the most common lossy conversions back to their original Unicode characters.

Pro Tools 2023.6: Track Markers

  • Add support for Pro Tools track markers

Not yet released, but it was announced that the next update to Pro Tools (perhaps 2023.5 or 2023.6) will have a new feature called Track Markers.

IMG_4241

Once the update is released, hopefully there will be a way to import/export these markers and then add support to DAWFileKit to extract them in addition to the already-existing session markers.

Cross-platform XML library

Look into finding a cross-platform solution for read/write of XML files.

XML is used by Cubase's Track Archive XML file format.

(XMLNode / XMLDocument are part of Foundation but only available on macOS.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.