free-audio / clap Goto Github PK

CLAP_EVENT_NOTE_EXPRESSION and MIDI 2.0 per-note controllers do exactly the same thing (with the possible exception of CLAP_NOTE_EXPRESSION_VOLUME and CLAP_NOTE_EXPRESSION_TUNING which work like their VST3 counterparts). What is a host supposed to use if it can choose?

FWIW: I appreciate some people don't like a raw MIDI API, but IMO it would be better to ditch CLAP_EVENT_NOTE_ON / CLAP_EVENT_NOTE_OFF / CLAP_EVENT_NOTE_EXPRESSION / CLAP_EVENT_MIDI in favor of CLAP_EVENT_MIDI2. Efforts to make it look easier just make it a bigger mess I'm afraid. There can be some utility conversion routines of course, there's just no need to burden the host/plugin interface specs with 3 ways to do the same thing if one way would do. It's asking for problems.

That said: CLAP makes a lot of sense and it's easy to implement so far :-), It's just the notes/MIDI1.0/MIDI2.0 thing that doesn't seem right.

A comment could be added to clap_vst3_converter.vst3_plugin_id that says it's a 32 digit hex string (as opposed to a GUID like {xxxx-xxxx-...}).

That's what I figured out it probably is, anyway ;-)

I know of this project, because some random dude joined a discord, posted a link to it, and left.

That's spamming. Please, don't be a dick.

We could add clap_event.flags and a flag CLAP_EVENT_FLAG_IS_LIVE.
One possible drawback is that may lead to non reproducible rendering as the host records the input and then schedules it without the flag is_live.

entry.h mentions only /Library/Audio/Plug-Ins/CLAP for macOS. AU/VST2/VST3/LV2 use both the system and user Library folder, so I guess CLAP should do the same.

The CLAP_EVENT_NOTE_ON event type should support sending the MIDI 2.0 note-on "attribute type" (8 bits) and "attribute data" (16 bits) fields.

The recently added comments in events.h say "plugins are encouraged to be able to handle note events encoded as raw midi or midi2". Either CLAP_EVENT_NOTE_ON must be able the send full MIDI 2.0 note-ons, or plugins must be forced to support raw midi2 note-ons.

http://www.indiana.edu/~emusic/cntrlnumb.html

Perhaps the is_main / is_cv / in_placebool fields of clap_audio_port_info should be turned into a single uint32 field with flags. This would make adding new flags in the future easier.
Shouldn't there be an is_SidechainInput ?

For the sake of completeness:

1. The new UMP MIDI messages use 'normal' small messages for sending sysex data. This means CLAP_EVENT_MIDI2 can send sysex data. That's very inconvenient in a typical DAW/plugin scenario. I suggest adding a comment like this to events.h:

"While it's technically possible to send sysex data using CLAP_EVENT_MIDI2, hosts/plugins are not expected to support this. Use CLAP_EVENT_MIDI_SYSEX for sysex data."

2. UMP introduces a new 8 bits sysex data type (as opposed the the 7 bits sysex we all know). If this actually gains traction a CLAP_EVENT_MIDI_SYSEX8 message type could be added in the future. I think this can be ignored for now.

There should be a way to send higher resolution values for MIDI controllers. Could be done by changing clap_event_midi to this:

typedef struct clap_event_midi {
uint32_t port_index;
uint8_t data[3]; //MIDI message with valid note velocity
bool hiresvalueisvalid; //if true: hiresvalue contains a higher resolution value that can be
//used instead of the 7 (or 14) bits value in data[]
double hiresvalue; [[-1..1] for pitchbend, [0..1] for controllers and channel/poly aftertouch
}

Alternatively MIDI 2.0 protocol messages could be used of course, but that would require the receiving end to support them.

EDIT: on second thought this should probably just be left to CLAP_EVENT_MIDI2...

I'm mentioning this just to make sure it's not an oversight. If you think it's not a problem then it's fine with me.

VST3 plugins have two states: one for the processor and one for the editor (ui). clap_vst3_converter.convert_state accepts the processor state only. That probably isn't a problem for most - if not all - plugins as there's nothing important in their editor state.

A second clap_istream_t parameter could be added for the editor state, to make sure no plugins are left out in the cold.

Suggested with https://github.com/justinfrankel/reaper-sdk/blob/main/sdk/reaper_plugin_fx_embed.h as a source of inspiration.

Start a graph based engine, in C. It should support:

• abstract events input
• abstract sound input
• abstract sound output
• do plugin delay compensation

Let's do PDC later :)

Hello, this is a request for a keyswitch extension.
It's functionality usually found in samplers, which makes certain keys of midi keyboards switch to a different timbral variation.

The keyswitches can be packed together in groups, which are usually adjacent keys with a same color.

The purpose of the extension would be to expose the keyswitchs to a host, such that they can appear on a virtual keyboard, and/or have the switch names appear on the notes of a piano roll. (similar to clap_plugin_note_name)

This exists in vst3, with a severely lacking documentation.
https://steinbergmedia.github.io/vst3_doc/vstinterfaces/classSteinberg_1_1Vst_1_1IKeyswitchController.html

In essence, I believe it should be

• list of keyswitches, having a name, and the key + channel + port as identifier
• keyswitches can optionally refer to a keyswitch group that contains them
• keyswitch group can have a name
• plugin can invoke host's changed to reload the switches

illustration:

FR: Add a "isPlayedLive" flag to the flags part of a MIDI event (notes etc.)

The reasoning behind this is:
A plugin that connects a hardware, a DSP processing card or has other reasons for higher latency can improve the user experience by ditching certain aspects like sample accuracy in exchange for a snappy feeling when played live.

To allow musicians to experience a snappy feeling, the original behavior of all notes that had a timestamp of 0 to be played as soon as possible it mentioned in the VST2 documentation.
In VST2 the timestamp 0 of a note event was designated to be "play as fast as possible".

see Documentation of struct vst_midi_event:

VstInt32 noteOffset
offset (in sample frames) into note from note start if available, else 0

Some of hosts didn't follow that rule so there is a change of 1/blocksize that this does not work correctly.

Therefore, VST2.4 introduced the kVstMidiEventIsRealtime flag.

from the documentation of VstMidiEventFlags:

kVstMidiEventIsRealtime
means that this event is played life (not in playback from a sequencer track).
This allows the Plug-In to handle these flagged events with higher priority, especially when the Plug-In has a big latency (AEffect::initialDelay)

By the time of VST2, a timestamp of 0 was quite a good sign to distinguish notes in a playback from those played live. At least Cubase guaranteed this, but there was always the chance of 1/(buffersize) that it is actually a timestamped note, especially because other hosts may not give the guarantee.

For a host it is easy to decide when this flag is being set: as soon as a track is in „live mode“ and the events are coming from a „real“ MIDI input.

This flag is also implemented in VST3, see documentation of VST (VST/3/doc/vstinterfaces/ivstevents_8h_source.html):

struct Event
00101 {
00102    int32 busIndex;            
00103    int32 sampleOffset;        
00104    TQuarterNotes ppqPosition; 
00105    uint16 flags;           
00106    
00108    enum EventFlags
00109    {
00110       kIsLive = 1 << 0,       
00111       
00112       kUserReserved1 = 1 << 14,  
00113       kUserReserved2 = 1 << 15   
00114    };

and Audio Units (AU Property kMusicDeviceProperty_DualSchedulingMode):

	@constant		kMusicDeviceProperty_DualSchedulingMode
	@discussion			Scope:				Global
						Value Type:			UInt32
						Access:				write
						Some instrument units need to distinguish realtime note and control events (such as from 
						incoming MIDI) from sequenced or pre-scheduled events. To support this, a host application 
						may set this property to 1. If the instrument unit returns a value of noErr, it supports 
						an alternate interpretation of the inOffsetSampleFrame parameter for the following 
						functions:
							MusicDeviceMIDIEvent
							MusicDeviceStartNote
							MusicDeviceStopNote
							AudioUnitSetParameter
						Once the host sets this property to 1 and the instrument unit returns noErr, the 
						inOffsetSampleFrame field becomes a bitfield:
							kMusicDeviceSampleFrameMask_SampleOffset = 0xFFFFFF // AND with this to obtain sample offset
							kMusicDeviceSampleFrameMask_IsScheduled = 0x01000000
						The IsScheduled bit should be set on events which are being scheduled ahead of time from 
						a prerecorded track. The IsScheduled bit should be clear on events which are being sent 
						to the instrument unit in response to realtime events, such as incoming MIDI or control 
						changes in a view.

the whole fd-support.h file does not compile, since read/write/close etc. don't work with void* types.

A minor optimization:

The clap_beattime...clap_sectime fields could be moved to the end of the clap_event_transport struct. This would make the struct 4 bytes smaller.

Hi there! This is an awesome project, thank you for working on it and making it available.

I am wondering if we could please add a clap event type for the new Universal Midi Packets (UMP) defined in the Midi 2.0 spec that has recently been published. There should be examples for defining them in Apple's Core MIDI headers (MIDIMessages.h), and JUCE has a template based implementation to account for the variable packet sizes.

I would suggest defining structs for each packet size, which are comprised of uint32_t fields; ie, packets come in sizes of 32, 64, 96, and 128 bits.

I'm no expert in the standard, but I am happy to help in any way to get MIDI 2.0 added to CLAP.

Thank you,
-Nick

Hi!

I was looking through the APIs (lots of changes since the last time I did that), and everything looks very clean. I just had a couple small notes I wanted to mention. Because who knows, maybe one day I'll have to implement clap support in yabridge. ;) Feel free to not do anything with this if it's not useful or of it goes against clap's design goal.

For future reference, this was written for clap 0.15.0/commit 24a3df6.

• The channel maps currently only support predefined sets of channel layouts. I actually quite like this because 99.9% of the use cases will involve processing either mono, stereo, 5.1, or 7.1 surround audio, but a long-standing complaint for VST3 (and one of the few places where VST2 still offers more functionality) is the inability to support exotic ambisonic setups or other non-standard channel layouts. See VST3 SDK issue #28 and this thread on their forum. I'm not actually sure what the best approach would be where you can have both clap's current simple model and a way for plugins to support exotic channel layouts, but I thought it might be worth mentioning since this is one of the complaints against VST3 clap has not solved yet.

• The second thing I wanted to ask about is forward compatibility. This is a tough one because forward and backward compatibility is often where APIs get most of their complexity from. What is the current plan for extending functionality to an existing interface? For instance, if at some point after clap 1.0's release it's deemed necessary to add another flag or more information to the audio port info, then how will this work? The usual options are to either completely replace the extension by a _v2 version of the extension, to add padding at the end of the structs in the hope that any new fields will fit in there, or to begin every struct with a version field so you can cast the pointer to that struct to a newer version of the struct depending on which version you support. But none of those options are ideal. So I was just wondering, are there already plans for how clap will handle this?

Anyways, I'm excited for clap's future, and I was very happy to see initial clap support added in Bitwig 4.1's betas. The polyphonic automation and modulation alone is going to be a game changer for sure. So let me know what you think!

Looks like typedefs and variables may not share the same name. Might be a good idea to follow the common practice by using clap_version_t for the typedef.

/home/dude/projekte/rip/ext/clap/include/clap/host.h:10:17: error: declaration of ‘clap_version clap_host::clap_version’ changes meaning of ‘clap_version’ [-fpermissive]
   10 |    clap_version clap_version; // initialized to CLAP_VERSION

(Compile with -pedantic)

It would be nice to have OSC capabilities in the way that parameters could send their movements out as control data in OSC over the network and also may receive such messages.

This could serve as an inspiration maybe: https://plugins.iem.at/docs/osc/

There should be a way for the plugin to know if it can send MIDI 2.0 to the host. There's no clap_plugin_event_filter for the host which the plugin could use.

Is it possible to support an arbitrary amount of languages for things like the clap_plugin_descriptor and parameter names?

I think it would be best to use the language tag system as defined in RFC5646: https://datatracker.ietf.org/doc/html/rfc5646

Perhaps it could work something like this where each language version is separated by a ;, similar to the list of keywords. Also if the user doesn't supply a [<lang>] tag, then the host assumes that only a single english version is given. (Please excuse any wrong words or grammar mistakes in this example, english is my only language).

clap_plugin_descriptor_t my_plugin_descriptor {
   .clap_version = CLAP_VERSION;
   .id = "com.me.myplugin"
   .name = "[en]My Plugin;\
            [es]Mi Plugin;\
            [fr]Mon Plugin",
   .vendor = "[en]me;\
              [es]me;\
              [fr]me",
   .url = "[en]https://en.my-website.com;\
           [es]https://es.my-website.com;\
           [fr]https://fr.my-website.com",
   .manual_url = "[en]https://my-website.en-manual.pdf;\
                  [es]https://my-website.es-manual.pdf;\
                  [fr]https://my-website.fr-manual.pdf",
   .support_url = "[en]https://en.my-website.com/support;\
                   [es]https://es.my-website.com/support;\
                   [fr]https://fr.my-website.com/support",
   .version = "1.0.1",
   .description = "[en]goes beep-boop;\
                   [es]va beep-boop;\
                   [fr]va beep-boop",
   .keywords = "[en]synth;[en]beep;[en]boop;\
                [es]sintetizador;[es]beep;[es]boop;\
                [fr]synthétiseur;[fr]beep;[fr]boop;",
   .plugin_type = CLAP_PLUGIN_INSTRUMENT
};

Another way we could go about this is to change the const clap_plugin_descriptor_t *desc; field in the clap_plugin struct to a method that takes the language tag as a parameter. Then the user can decide if they want to return the given language or just return a default language if they don't support the given language:

typedef struct clap_plugin {
   clap_plugin_descriptor_t (*desc)(const char* lang_tag, uint32_t lang_tag_size);
   
   ...
}

I also see that clap_param_info has a constant size for its name and module name buffer, so either that needs to be changed or we do the second thing again where we pass in the language tag as a parameter:

// Copies the parameter's info to param_info and returns true on success.
// [main-thread]
bool (*get_info)(const clap_plugin_t *plugin,
                int32_t              param_index,
                clap_param_info_t   *param_info,
                const char* lang_tag,
                uint32_t lang_tag_size
);

It may also be necessary to either assert a specific text encoding standard like UTF-8 or have a separate "text encoding type" field on the descriptor.

There should be a way for the host to detect if the plugin can receive MPE.

The clap_event_param_value and clap_event_param_mod param_id/cookie aren't in the same order:

typedef struct clap_event_param_value {
alignas(4) clap_event_header_t header;

// target parameter
void *cookie; // @ref clap_param_info.cookie
alignas(4) clap_id param_id; // @ref clap_param_info.id

typedef struct clap_event_param_mod {
alignas(4) clap_event_header_t header;

// target parameter
alignas(4) clap_id param_id; // @ref clap_param_info.id
void *cookie; // @ref clap_param_info.cookie

clap_event_param_value should probably be changed to match clap_event_param_mod, that makes the struct a little smaller.

When the api matures, CLAP should definitely feature a clap percussion synthesizer as one of its demos, because... I dunno, it just feels right.

There is already a crate with Rust bindings, so maybe it also could be added to the readme:

https://github.com/glowcoil/clap-sys

Some misc requests related to parameters.
These are use cases mostly related to wrapper plugins of other formats, which have their own parameter mappings (sfz, jsfx).

• changing the parameter visibility
  When visibility is switched off, the parameter should no longer appear from host's generic UI.
  It's a feature present in Reaper's jsfx, possibly also in vst3 with some subtleties.

• loose range constraints
  Perhaps a flag which tells the host explicitly to not clamp value to min/max, let value be how it is.
  For example, Reaper allows a plugin to automate a parameter outside of the nominal range, in jsfx min&max both to 0 is valid and working.
  IIRC LV2 has it the other way around, a flag requires host to force values into range.
  Either way, the objective is to make explicit the host's clamping behavior.

• trigger parameter
  A trigger parameter works no different than a boolean, but can allow the host represent it as a non-checkable push button. Present on Reaper jsfx generic UI.

• Also needs an event to notify the host that the list changed

The idea is to have a tool that could be used for testing:

• ./clap-console -plugin myplugin.so -midi score.midi -in input.wav -out output.wav -plugin-state state-file

So it could be used to do testing.

Structure packing isn't necessarily standardized between all compilers targetting the same platform. The VST SDK enforces the same alignment for all supported compilers on each system. CLAP doesn't do this yet, so structs like clap_event_note end up with a packing that might end up differently on different compilers (as the natural placement of a double value cannot be at byte 12, which would be the next free space after 3 32-bit integers). In practice this means the double will be placed at offset 16 instead.

I'd suggest to have a well-defined structure packing for all structs in CLAP (doesn't even have to be different from what is currently being done, it just needs to be defined at all), and/or avoid structs with non-natural alignment to begin with (in this particular example double could be replaced by float if the reduced precision is agreeable, but for other structs that may not always be possible, especially as soon as pointers are involved).

If you decide against issue #48 the data field of clap_event_midi should be turned into an uint32_t. Or a 4-byte aligned "uint8_t data[4]", or a union of both.

The current "uint8_t data[3]" isn't easy to read/write in one go as an integer, and it's not 4-byte aligned.

AFAICS the behavior in case a plugin doesn't implement clap_plugin_event_filter is undefined. That can be a problem in some cases. For example:

• a host might want to use CLAP_NOTE_DIALECT_MIDI_MPE instead of CLAP_NOTE_DIALECT_CLAP if the plugin doesn't support CLAP_EVENT_NOTE_EXPRESSION.
• a host probably doesn't want to show UI for some features if the plugin doesn't support CLAP_EVENT_PARAM_MOD.

Just a style issue: the clap_supported_dialects / clap_preferred_dialect fields of clap_note_port_info should probably be renamed to supported_dialects / preferred_dialect.

(moving from the conversion on b902efa)

For user-local plugins, the two options currently considered as mentioned in b902efa are ~/.clap and ~/.local/lib/clap.

The arguments for ~/.clap are that it's in line with what users would expect (as ~/.vst, ~/.vst3 and ~/.lv2 will likely also exist on their system), while the argument against it is that it clutters up the home directory. My personal argument against ~/.local/lib/clap is that it's not a standardized directory, and there's no associated environment variable defined in the XDG Base Directory specification for controlling its location. And a third option is $XDG_DATA_HOME/~/.local/share which is meant for program data that can be shared across systems, but that may of course also not be a very discoverable location for users that try to install CLAP plugins from downloaded archives.

Does clap_plugin_event_filter.accepts(... CLAP_EVENT_TRANSPORT) apply to clap_process.transport?

If yes:
This should be documented

If no:
Perhaps it should. Calculating tempo and timesig isn't a trivial operation, so you might want to avoid doing that for plugins that don't need it anyway. Perhaps not as big an issue as it used to be with today's computers, but it does add up, especially when audio buffer size is small.

Apple has already deprecated all 'old' MIDI APIs in favor of UMP versions. Microsoft will likely use UMP for MIDI 1.0 too when a MIDI 2.0 API appears. That's simply the way the new MIDI is designed, and the way MIDI 1.0 will live on in a world that moves to MIDI 2.0. This will make clap_event_midi old fashioned pretty soon.

It would make sense to change clap_event_midi to use UMP instead of traditional 2/3 byte messages. Two helper function can be added to convert between traditional and UMP MIDI.

The dilemma is that everybody will want to use UMP for MIDI 1.0 two years from now, but most people don't know that yet ;-)

PS: Many issues are solved, and it looks good. Great work! :-)

AFAICS there's no way for a plugin to report reverb tail length like kAudioUnitProperty_TailTime (AU) / effGetTailSize (VST2) / IAudioProcessor.GetTailSamples (VST3). A host may use this to calculate the duration of a song.
Could be implemented similar to clap_plugin_latency.

Finish the midi parser and add tests.

clap_event_midi2 contains UMP packets. UMP can use either the MIDI 1.0 or MIDI 2.0 protocol (see MIDI_2-0_Specification_Version_1-0/M2-104-UM_v1-0_UMP_and_MIDI_2-0_Protocol_Specification.pdf from midi.org). Currently there's no way to know what protocol the receiving end actually supports.

Proposal:
Add something like "Sending MIDI 1.0 protocol messages using CLAP_EVENT_MIDI2 is not allowed, use CLAP_EVENT_MIDI for MIDI 1.0 instead" to events.h.