C library access to full Praat functionality about praat HOT 80 OPEN

Yes, coders can be expected to figure things out. I am reluctant, though, to advise keeping old versions of the API. That is, newer versions tend to work better with newer platforms, tend to have fewer bugs in the areas that you use, and will give you the joy of having only one version to account for. Just as an example: I recently tried to link a one-year-old .a file on the Mac, and Apple complained about my not having included "bitcode"! The problem is always acute for GUIs (just made Praat compatible with macOS 10.12), but even the simplest C code can cause problems. That is, old versions become obsolete not only because of API incompatibilities, but also because of developments outside our control, such as you get with mixing the various glibc versions on Linux.

But of course, if you can handle all that... your wrappers can of course have their own subset of Praat commands and their own names. For me it's relevant at the moment what I will be able to provide from my side, and the basic menu-command-level API could be flexible enough for others as a first step. There will be lots of issues to solve for that one already, such as handing over exceptions to Python, setting the preferences to Python-sensible values (UTF-8 for text output, I presume), and unpacking the Python argument list in C.

from praat.

Totally agree.

This is something a lot of people (mostly developers) want. I certainly have been thinking about it for a while, and we spoke about it with @PaulBoersma in August last year, although I think it has had a relatively low priority.

Just for the record, there are some independent initiatives like this out there, although none seem to have produced an entirely usable library:

• libpraat
• tala

There are also other attempts at writing wrappers for different platforms and environments. Like PraatR, there is also rPraat and mPraat.

There are probably loads more I am not aware of, but it would be good to develop an official library that others could use to unify these efforts.

from praat.

The project file for the Mac has a target libpraat.a, which includes everything except the main function. Such a library is very easy to make, also for other platforms. To turn it into a working executable, it only has to be linked with main_Praat.o. Do you think it would be useful to include this in the makefiles, e.g. with make a (the static archive) and make so (the shareable object, i.e. the dynamic library)? Such a library would contain lots of stuff that you wouldn't need for a web service, for instance, but at least it would contain everything you do need.

A problem might be the C++ formulations of all the functions. That is, the linker that you use should understand that. Perhaps some crucial functions (say, the script-running function) could be turned into having a C interface rather than a C++ interface, so that they are callable from Python with the help of a simple ctypes module. Could that be useful?

from praat.

Now I think of it, it's probably better to just have a limited API that can call exactly the same Praat commands that a Praat script can call, namely all menu commands available in the Objects window and the Picture window. Along the lines of Python's subprocess.call command:

praat.call("Create Sound from Formula...", ["whiteNoise", 1, 0.0, 1.0, 44100, "randomGauss(0,0.1)"])

from praat.

I think a limited API of the sort you describe is probably better in the end, but having a direct C interface could probably be a quick-and-dirty solution for the time being. I guess it depends on how much extra work an API would be.

from praat.

Such a library would contain lots of stuff that you wouldn't need for a web service, for instance

Couldn't we just break those into smaller ones? Praat is a complex beast with many different parts that are not necessarily related. The shell is different from the plotting features, which are different from the commands from the objects and editor windows. Couldn't those be in, say, libpraatshell, libpraatpicture and libpraatcore? That would solve that problem, and allow people to better tailor what they need.

Then again, having everything in the same library would be infinitely better than not having any library at all.

it's probably better to just have a limited API that can call exactly the same Praat commands that a Praat script can call, namely all menu commands available in the Objects window and the Picture window

I think what would be ideal would be to dogfood the library with the Praat GUI. That would give a good indication of whether its API was sufficient or not, and I think it would ultimately lead to clearer, better organised code. Not to mention that it would also provide an example of how to use it.

I guess it depends on how much extra work an API would be.

Libraries are as good as their APIs, so I think some work should be put into the design of a good one. If the interface is not good, chances are people will not want to use it.

Along the lines of Python's subprocess.call command:

praat.call("Create Sound from Formula...", ["whiteNoise", 1, 0.0, 1.0, 44100, "randomGauss(0,0.1)"])

This approach is good in that it is simple, but I've never been quite sure about the names of Praat commands, because they are very strictly linked to the labels of the buttons in the GUI. This ultimately leads to command names that can get very unwieldly (eg To PointProcess (periodic, peaks)..., To PointProcess (periodic, cc)..., Create formant table (Pols & Van Nierop 1973)).

The first two of those examples, for instance, could be distinguished by their signature and could be internally represented by a single exposed command: Sound->toPointProcess(args), or PointProcess->new(Sound, args), or something like that.

I don't think we need to assume that the names of the commands exposed by the library must always be the same ones used in the GUI, and indeed this might be a good chance to think of better names for some of them, when it makes sense.

from praat.

Praat has more than 8000 commands, and it would be a hellish job to convert all of them to short names (by hand, given your examples), and to document the relation. In my proposal (it is not new), the Praat manual as it is would automatically be the manual for the API, and the existing menus would define the available functions. By the way, the "single exposed command" you mention is already there: if there are menu commands "Draw" and "To PointProcess" for a Sound object, this means that there are C++ functions Sound_draw and Sound_to_PointProcess with the same arguments. As for your "clearer, better organised code" idea, the code as it is now has been optimized for clarity and organization already. For instance, the code can be compiled without GUI, or without graphics, or without interface files, and everything else still works; for instance, Sound.cpp contains a function Sound_draw, but that function does not require any graphics implementation to be present. (Yes, I know that there is one exception to full separation in the code; good for you if you can find it.)

OK, the Python API will look more or less like this (assuming there is a single Praat object):
import praat
praat.load ("libpraat.so")
version = praat.version ()
result = praat.do (command, args [, nocheck=false]) (numeric result)
result = praat.do_str (command, args) (string result)
result = praat.do_array1 (command, args) (numpy vector result)
result = praat.do_array2 (command, args) (numpy matrix result)
praat.selectObject (id_or_name,...)
praat.plusObject (id_or_name,...)
praat.minusObject (id_or_name,...)
praat.removeObject (id_or_name,...)

from praat.

All I'm trying to say is that having a Praat library is something a lot of people want (me included), and I think it would be better to have a library that is not only available, but good as a library: easy to use, clear, concise, etc.

The way I see it, the very fact that "Praat has more than 8000 commands" is a clear sign that we can cut some stuff away. We certainly don't need access to all of them through the same API (and I imagine we might want access to some things through the API that are not currently available as commands).

There's no need to rush into having something out quickly. We've lived without a Praat library for a long while now, so my opinion is that we could take our time to think as a group whether the decisions that have been made in the past continue to be the best ones. We can start with a spec and share it with the rest of the people who would actually be using it, and take it from there.

Maybe @psibre, @ghedlund, @usagi5886, or @drammock have something to say?

from praat.

The way I see it, the very fact that "Praat has more than 8000 commands" is a clear sign that we can cut some stuff away.

Not really. For instance, R also has 8000+ commands, and would you like to make some of its packages unavailable? Probably not. Same with other comprehensive field-wide toolsets (e.g. Python with a deep-learning package installed). The users should be able to decide what parts of the software are relevant for them and what aren't. And users differ from each other.

from praat.

I don't think that comparison works. R has no notion of "command", it uses functions which can be created and redefined at runtime. There's technically no limit to the number of those that can be made available. The same with Python.

And in both cases we are also talking about languages that are extendible by loading additional libraries (or modules, or packages, or however they are called). Many of those "commands" will be available as part of one of those additional libraries... each of which will have an API that will likely (hopefully!) have fewer than 8000 "commands" (whatever those may be in that particular case).

So no one is talking about making those packages "unavailable". I'm talking about designing an API for the Praat library which is powerful and versatile enough to (at least) power the GUI (that's what I meant before), but as simple and concise as it can be so that it is easy to use. If the library is successful, then we'd see users coming to it that have never used the Praat GUI before, and I can imagine them scratching their heads as to why they need to use commands with spaces, parentheses, numbers, ampersands and trailing ellipses in their names.

The Praat interface is married to its past because of backwards compatibility. All I'm suggesting is that we seize this opportunity to create a more streamlined interface for its library with developers in mind.

It will be a lot of work, that's for sure. But in any case, this is not something that should be done by a single person. There's an entire community willing to help.

from praat.

FWIW, I'm fine with just calling praat to run scripts from whatever toolchain (typically a Gradle build). The only headache is figuring out whether an installed version is pre or post 6.0, but I can lock down the Praat dependency version. So personally, I don't need a C library to use Praat in new and exciting ways.

from praat.

I am generally sympathetic with @jjatria's idea that many of Praat's commands could be unified, for example the four commands To LPC (foo) (where foo is one of autocorrelation, covariance, burg, or marple) by adding a new argument that specifies which method/algorithm to use. It would be nice if that were done as part of the standard library, but it doesn't have to be; it's also possible to build that into wrappers/bindings. For example, a python wrapper for the LPC functions could be:

def sound_to_lpc(sound, order=16, window=0.025, step=0.005, preemph=50.0,
                 tol1=1e-6, tol2=1e-6, algorithm='autocorrelation'):
    '''Create LPC object from Sound.
    sound: praat.Sound
        The praat Sound object on which to operate
    order : int
        Prediction order
    window: float
        Window size in seconds
    step: float
        Step size in seconds
    preemph: float
        Preemphasis in Hertz
    tol1, tol2: float
        Tolerance values, ignored unless algorithm == 'marple'
    algorithm: str
        Which algorithm to use
    '''
    command = 'To LPC ({})'.format(algorithm)
    args = [order, window, step, preemph]
    args = args.extend([tol1, tol2]) if algorithm == 'marple' else args
    praat.selectObject(sound)
    lpc = praat.do(command, *args)
    return lpc

Of course this means that the bindings may vary from language to language. Either way, it's a lot of work. If the library were in fact more "unified" in the sense that @jjatria suggests, I might be more likely to create/contribute to python bindings, but for right now at least I don't have strong daily cravings to be able to call praat from python, so for me the urgency is low.

from praat.

Thanks @psibre and @drammock for reacting.

As for "To LPC...", the buttons are separate because not all of these commands have the same arguments (the same goes for @jjatria's suggestion for the "To PointProcess (xxx)..." commands). If you find the separation excessive, it could be repaired by changing the menu commands (at the cost of having unused arguments in some cases), which has sometimes happened in Praat's history (while keeping the older commands working at the same time). That is, unification of commands is not an issue that is specific to a library interface, and does not seem to constitute an argument for or against translating each and every command. Several commands already have a "method" argument, for instance; the choice between separating them or not has been taken in each case with a view at the argument list and typical use cases. Such decisions are typical of software design in general; many of them necessarily constitute compromises, as in this example.

I don't understand @jjatria's point about R not having 8000+ commands. That they're called functions instead seems hardly relevant. That they are divided into packages: well, that's the same in Praat (where the packages aren't dynamically loadable, that's true; that's related to Praat having much larger GUI demands than e.g. RStudio, and to the increasing number of cross-links between packages in the manual). As for "simple and concise": that's in the eye of the beholder, because if you make only a subset available, then the next user will require a subset that you didn't make available.

As for spending thousands of hours to create a Python binding for all interesting commands: be my guest, but you would also have to create documentation for each command (or function). And after having done that, you should try to keep the documentation synchronized with the Praat manual, which always develops. Quite likely, the binding will have become obsolete the moment it exists, unless the Praat manual is the documentation for the commands, but that only seems possible if the names of the commands are identical.

from praat.

unification of commands is not an issue that is specific to a library interface, and does not seem to constitute an argument for or against translating each and every command.

I agree. I'm not sure I fully understand @jjatria's argument either, but I think maybe it hinges on the issue of praat commands containing spaces and punctuation? Is it that fact which makes it necessary to say things like praat.do("To LPC (marple)", args) instead of the more widespread patterns praat.to_lpc_marple(sound, args) (what one might call the library.function() pattern) or SoundObject.to_lpc_marple(args) (the object.method() pattern)?

from praat.

I would use an official 'libpraat' if it existed. However, building a library which meets the needs of various developers may prove challenging to say the least. For example, I'm more interested in working with Java bindings to the C++ objects LongSound, Formants, etc., than working with an abstraction for Praat scripting or UI menu calls. This has the advantage of allowing the programmer to manipulate objects directly and avoid using the Praat object window. The calls end up looking like Praat scripts, just in another language as most DO_ commands have equivalent function calls which are easily exportable to C and linked in a shared library.

E.g., loading formants in Java

final double xmin = ...;
final double xmax = ...;

final Sound sound = longSound.extractPart(xmin, xmax, 1);
final Formant formants = sound.to_Formant_burg(
            formantSettings.getTimeStep(),
            formantSettings.getNumFormants(),
            formantSettings.getMaxFrequency(),
            formantSettings.getWindowLength(),
            formantSettings.getPreEmphasis());

The libpraat sitting in my repository list was built to support the few objects in the fon package I needed to work with in Java using the companion jpraat library. My intentions are only to wrap those objects I need, but would be happy to assist anyone who wants more available.

from praat.

This has the advantage of allowing the programmer to manipulate objects directly and avoid using the Praat object window.

The object window is just a view on an underlying list of available objects. When running Praat from the command line, for instance, your script manipulates the objects in that list, without there being any window. The difference between

praat.selectObject (sound)
praat.do ("To Pitch...", [0, 75, 600])

and

praat.call (sound, "To Pitch...", [0, 75, 600])

is one of syntax only, and both are equally easy to implement. The same goes for

sound.do ("To Pitch...", [0, 75, 600])

although I'm not sure that the latter works for multiple objects in Python as well. That is, drawing a Sound and a Pitch together could be either

praat.call ([sound, pitch], "Draw...", [0, 0, 0, 0, true])

or

[sound, pitch].do ("Draw...", [0, 0, 0, 0, true])

but can the latter syntax be implemented in Python?

from praat.

can the latter syntax be implemented in Python?

No, [sound, pitch].do("Draw...", [0, 0, 0, 0, True]) would not work in python.

from praat.

I know it is irrelevant what they are called (commands, functions, etc). What I was trying to say was that the Praat "commands" and the Python "functions" do not exist in the same level: you can create and redefine one of those at runtime. Saying that it is OK for Praat to have 8000+ commands because Python has 8000+ functions makes little sense to me. That's all.

As for the interface, I am not saying that the unification of commands is on its own a sufficient argument for giving the API a second look. What I am saying is that the release of a Praat library offers a good chance to ask ourselves if the API could be improved. And if it could, why it shouldn't.

Other than unification, there are plenty of examples of commands that I think could be redesigned to make them easier to use. Here are some odd name choices in the TextGrid commands alone:

TextGrid->Get start time
TextGrid->Get end time

Why not just "Get start" or "Get end"?

TextGrid->Get total duration

Why "total"?
And why is there no "Get interval duration"?

TextGrid->Get starting point... tier interval
TextGrid->Get end point... tier interval

Why "starting" if the other one was "start"?
If the idea is for the name to be informative, why not "Get interval start / end"?

TextGrid->Get interval at time... time
TextGrid->Get low interval at time... time
TextGrid->Get high interval at time... time

Why not a single "Get interval at time" that returned a float?

TextGrid->Get interval edge from time... time
TextGrid->Get interval boundary from time... time

I don't even know what these do, and they are not documented

TextGrid->Get label of interval... tier interval
TextGrid->Set interval text... tier interval text

Is it "label" or "text"?
Why the change in word order? What's wrong with "Get interval text" (or "label")?

etc.

from praat.

Yes, neither "start time" nor "starting time" are 100% fine, especially if next to "end time" (for which there no "ending time" option); either the English grammar or the parallelism will suffer (natives' judgments appear to diverge). By the way, both formulations work in a script, and the names on the buttons have changed several times, because our constraint ranking changed. We have "total duration" because there are parts (intervals) that have their own durations ("Get grid duration"?). Get interval edge from time and Get interval boundary from time should be documented (they look for exactly matching times of boundaries, and otherwise return 0). I agree that "Get end of interval" is clearer than "Get end point".

from praat.

This has the advantage of allowing the programmer to manipulate objects directly and avoid using the Praat object window. The calls end up looking like Praat scripts, just in another language as most DO_ commands have equivalent function calls which are easily exportable to C and linked in a shared library.

@ghedlund , This is what the conversation is about, though with a focus on C, Python, etc. I guess the idea is to make it easily portable, e.g. to Java.

from praat.

I think I'll follow @drammock's approach with sound_to_lpc, i.e. I'll create the small low-level API that I proposed and anybody else can create wrappers for the part of Praat they think could be useful for some subset of Praat users. My API would work for C and Python, and Java and R interfaces can follow.

from praat.

It would be great if we can have some time to look at the low-level interface you propose before it's final, so that we can make comments and suggestions about things that can be improved, or that other people think might be useful to have. I don't think it's a good idea for all of this to be the work of a single person (and we seem to agree that some commands have less-than-ideal names that could be improved).

If this is going to provide low-level access, it would also be great if it allowed for direct object manipulation, like @ghedlund mentioned, which would take care of cases like #52 and make the integration with other platforms that much easier.

This would also mean that, if objects can be passed as arguments, we could do away with the entire selection mechanic and the separation between editor and object window commands, which is useful in the GUI but a source of endless headaches for inexperienced script writers.

from praat.

If that's the plan, I think it would be possible to take a principled approach, whereby current command names can be consistently converted to more user-friendly (?) function names. That is, for To LPC (marple)... (which I assume is subsumed under some Sound object or something) to become sound_to_lpc, you're taking out the subtype (match the part in parentheses?), lower-casing everything, replacing spaces with _, removing the ...., etc. If done consistently, I imagine any changes to or addition of names could be semi-automated.

from praat.

A consistent and predictable naming scheme should definitely be used, but I'd discourage a blind automatic conversion of the existing interface (not sure how automated is "semi-automated").

Doing so would reproduce all the inconsistencies that exist in the current interface (like the ones I mentioned), the commands that have more than one name (to accommodate the changes in button labels throughout the years), the ones that only make sense within a GUI (like the ones in the Editor window), the ones that really do not belong as is (like the ones that generate example datasets), etc.

from praat.

Please don't use the conversion to get rid of perceived inconsistencies in naming. Such inconsistencies should be tackled in Praat's menus, not in a conversion to a library.

Automation is possible only if commands are not being unified at the same time. Unification can only be done by hand on a case-by-case basis. Any unification deemed necessary should be tackled in Praat's menus, not in a conversion to a library.

If renamings and reshufflings are done in the conversion, the result of the conversion will get outdated as soon as anything changes in Praat's menu commands. Please consider keeping the process modular: renamings and reshufflings by hand at the Praat menu command level, and conversion to underscores automated at the API level. By the way, we used to think a long time about automated conversions, but never found a scheme that works transparently enough (predictability in two directions is needed, because the entries in the manual have spaces and parentheses, not underscores).

I don't yet see how clarity is served by collapsing spaces and parentheses into underscores, or by merging upper and lower case, but if consistency of formatting is an issue, then it will of course be fairly easy to write an algorithm for converting the complete Praat interface description to the standard Python function orthography.

from praat.

@jjatria the ones that really do not belong as is (like the ones that generate example datasets)

Who says so? Of course people do use them. Why would you keep them away from Pythonists who like to debug their algorithms? R contains lots of example datasets as well. Really.

from praat.

@jjatria Doing so would reproduce all ... the commands that have more than one name (to accommodate the changes in button labels throughout the years)

And rightly so. If Praat script writers have that freedom, and the Praat developers have a freedom to introduce more consistency in the names of menu commands, why shouldn't Pythonists profit as well?

from praat.

I never said that people didn't use those commands to generate example datasets. I said that I didn't think they belonged as part of the API of a Praat library.

Likewise, I have never considered the fact that commands have multiple names something people can "profit" from or that offers "freedom". I see it more as a source of confusion and unnecessary complexity.

For some reason that I cannot comprehend, it seems as if you think the fact that Praat is overly complicated is a positive, something to be proud of, that gives users freedom. Or maybe, despite all the things I've mentioned, you just think that Praat is not overly complicated and is "optimised for clarity and organization". Either way, we disagree.

I think having a Praat library that allows other projects to benefit from the algorithms implemented in Praat would be a great thing to have. I think the existing Praat interface has serious problems in design and usability. I think developing a library would be a great opportunity to revisit some of those issues and think as a community about ways in which they can be improved. I think we can take our time to design a good interface, and I think it would be well worth the effort to do so. But I seem to be the only one that thinks this.

Bottomline, I think that I have a very different idea of what a Praat library should look like and do, and I don't think I have many chances of swaying your opinion.

from praat.

I don't yet see how clarity is served by collapsing spaces and parentheses into underscores, or by merging upper and lower case, but if consistency of formatting is an issue, then it will of course be fairly easy to write an algorithm for converting the complete Praat interface description to the standard Python function orthography.

I think I'm missing something. I believe I understood before that the various menu items / commands are all linked directly to underlying functions. Is there a one-to-one mapping between them? If so, presumably those functions don't have spaces, parentheses, ellipeses, etc. in their names? Is it an unsafe leap to make from that to assuming that the function names are somehow related to the menu names? And that mapping between them isn't hardcoded, but is stored as a list somewhere? (Because otherwise, how do you localise Praat?) I'm trying to understand why so much extra work has to be put into this if the functions already exist, and already have a non-menu naming scheme.

Also, if there's already a mapping between them somewhere, no one has to strip parentheses and stuff in order to keep API and GUI/scripting documentation in sync.

from praat.

I think having a Praat library that allows other projects to benefit from the algorithms implemented in Praat would be a great thing to have. I think the existing Praat interface has serious problems in design and usability. I think developing a library would be a great opportunity to revisit some of those issues and think as a community about ways in which they can be improved. I think we can take our time to design a good interface, and I think it would be well worth the effort to do so. But I seem to be the only one that thinks this.

For the record, I basically agree with this. My point of view is more like this, though: as long as there's going to be renaming of stuff, maybe it's a good opportunity to think about why stuff is named the way it is. It's not necessarily the right time for a complete overhaul of the code or naming conventions though..

from praat.

@PaulBoersma If Praat script writers have that freedom, and the Praat developers have a freedom to introduce more consistency in the names of menu commands, why shouldn't Pythonists profit as well?

I'm actually on @jjatria's side on this point. Changes to an API are common occurrence, and a variety of (best?) practices such as semantic versioning, or an announced period of deprecation (with runtime warnings) prior to fully retiring old aspects of syntax, exist to support such changes. Perpetuating legacy command aliases into a new API is only going to be useful for users who learned those older ways of doing things, and failed to notice (or refused to adopt) the newer ways. Remember that some (many? most??) users of a praat API will have little or no experience with the praat GUI or praat scripting, and so will not have learned the "old ways". To that kind of user, having several different ways of doing the same thing will probably look like "cruft" at best, and could be confusing.

from praat.

To that kind of user, having several different ways of doing the same thing will probably look like "cruft" at best, and could be confusing.

And many of the rest of us would rather not be stuck with the old ways—and are happy enough and capable enough to learn the new ways.

from praat.

I believe I understood before that the various menu items / commands are all linked directly to underlying functions. Is there a one-to-one mapping between them? If so, presumably those functions don't have spaces, parentheses, ellipses, etc. in their names?

That is 95% correct, but that is about implementation, i.e. about keeping the code navigable for Praat-internal developers. Thus, indeed, there is a C++ function Sound_to_Pitch_ac(), with a Sound as its first argument. It is hard, though, to expose these function directly in an API, the main cause being that the interface functions (i.e. the mappings from menu commands to the underlying functions) add argument checking and other kinds of checks that cannot be 100% automated. So via libpraat.so you would have access to the underlying functions from your C++ code, for instance, and these underlying functions will work correctly if you call them in the way they are supposed to be called, but a safe API should add the required error checking, which is what Praat's interface functions have accomplished. Therefore, accessing the underlying function Sound_to_Pitch_ac() should be done through the interface function that the manual refers to as Sound: To Pitch (ac)..., which checks that some of the arguments are non-negative, for instance. From Python, this could indeed be called sound_to_pitch_ac(), but that would be a new name for the user, the orthography of Sound_to_Pitch_ac() being an implementation detail. But as said, this is possible. Thus, sound_to_pitch_ac() (exposed in the Python API) calls Sound: To Pitch (ac)... (exposed in Praat's menus and in the manual), which calls Sound_to_Pitch_ac() (an implementation detail highly relevant to Praat's developers but irrelevant to the users of Praat or libpraat.so).

There are many other reasons beside error checking to keep the API separate from the implementation, such as having to make sure that in a graphics call such as Sound: Draw... the correct graphics location is chosen: the Picture window, a manual page, a PNG file...

from praat.

maybe it's a good opportunity to think about why stuff is named the way it is

Yes indeed. It is good that @jjatria finds many problems with the names. One criterion he mentions is clarity. For instance, Get starting point... can be called that way because it is in a menu labelled Query interval tier, but indeed Get starting time of interval... would be much clearer, especially in a script, where it's out of context (and Query commands are on average used more often in scripts than chosen directly from the menu). Another criterion @jjatria mentions strongly, though, is brevity, and on that criterion Get starting time of interval... is not better than Get starting point.... My point here is that there are multiple criteria, and that often these are inherently conflicting, so that the current result is often a (conscious) compromise. Different people may rank the different criteria differently, but it would be good if we all understood that these tensions exist. These days, I am thinking that clarity should be primary, so that Get starting time of interval... could show up in Praat's TextGrid Query menu soon. (Get interval start... is shorter but makes it less clear you're getting a time.)

from praat.

Perpetuating legacy command aliases into a new API is only going to be useful for users who learned those older ways of doing things, and failed to notice (or refused to adopt) the newer ways.

Yes, we would not necessarily need pre-2016 names for the commands. But in the past their names have often required changing, and I see no reason to think that that will be different in the future. Thus, in 1995 we had Sound: To Spectrum, and with the advent of new Fourier methods it had to be changed to Sound: To Spectrum..., with the added Fast argument. We kept the other name in, so that existing scripts would not break. People easily keep their scripts for ten years, if not more. The same will happen in the future. We now have Sound: To Intensity..., but when we add another method in the future, the name may have to be disambiguated to Sound: To Intensity (smooth)... or so, while we keep the old name in. We will hide the menu command, but keep the old name accessible for scripts. You talk about refactoring Praat's menu structures, and this can indeed happen now, but it will equally surely continue to happen in the future, as the field of speech analysis progresses.

I agree that creating an API for a new platform (Python) may be a good time for renewing the command set, especially if you think the new command set will be fixed, but if in 2021 we want backwards compatibility to 2016, we will have to accept the possibility of (future) multiple names, unless Python users are much more satisfied with removing "cruft" and "legacy" and with accepting "deprecation" than other users. I doubt this, considering how many Pythonists still use Python 2.7, almost 8 years after Python 3.0 was introduced.

For Praat I have often chosen in favour of backwards compatibility. I realize that that is Microsoft-API style, not Apple-API style. The cause of the similarity between Microsoft and Praat in this respect is probably that Microsoft and Praat are about software. We don't maximize the selling of new phones, for which it is better to introduce incompatibilities. Well, I guess it's just a choice; the R people do break their code every two years or so, annoying (or educating?) their users.

from praat.

@PaulBoersma ...in the past their names have often required changing, and I see no reason to think that that will be different in the future... if in 2021 we want backwards compatibility to 2016, we will have to accept the possibility of (future) multiple names...

I don't think we have to accept multiple names. Another approach is to change the API as needed (using semantic versioning and a deprecation period to ease the transition), and ensure that older versions of the API are still available for users who want/need to run old scripts as-is, leaving it up to the user to take responsibility for documenting which version(s) of their dependencies work for their projects. I don't think this is strictly a difference between hardware-centric vs software-centric perspectives on software development; to me it seems that lots of cross-platform software projects (free and non-free) function quite well on this model.

I think I am less concerned than you are about strict preservation of backward compatibility, and more concerned about keeping my source code as orderly as possible (a subjective notion to be sure) so that future improvements are as easy as possible for me (the developer). Of course I don't want improvements to leave users in the lurch, but I see keeping old versions available as a sufficient good-faith effort toward supporting the software I write. Of course (as previously mentioned) I am unlikely to contribute to a praat C library directly (though I may write wrappers for it), so my opinion should perhaps not carry much weight in deciding how the C library is structured.

from praat.

leaving it up to the user to take responsibility for documenting which version(s) of their dependencies work for their projects

Well, researchers who use Praat scripts typically combine scripts of various ages. This is why I advise people to always use the newest version of Praat, which is supposed to be able to run scripts created at any time during the last ten years (yes, we do have a deprecation period...). With short-term deprecations, there may exist no Praat version that runs the whole workflow.

Consider a real use case, and you may become more sympathetic to my standpoint. In Belgium, the state encourages speech pathologists to use a certain Praat script for measuring the "Acoustic Voice Quality Index". These people are professionals in their field, but cannot be bothered at all to fix problems with Praat or their script. For instance, the script will exit with an error message if the user happened to have recorded the patient's speech with the microphone switched off, and I get complaints about that. That's just for illustrating the level of self-help the users are able to perform. This is a situation in which breaking an API would cause an irresponsible amount of stress. The author of the script, Youri Maryn, creates a new version every three years, on the basis of problems reported (mostly cases in which the script failed to check for erratic situations). I don't think we can afford to add API incompatibilities to those problems.

Backward compatibility usually (though indeed not always) costs very little. Take, for instance, the case that @jjatria labelled as a case of a "serious problem in design or usability", namely the visual inconsistency in the names Get start time versus Get starting point. I remember the long deliberations we had about these names in 2004. When I Google these terms again in 2016, I again count that "start time" occurs 30 times more often than "starting time", and "starting point" occurs 100 times more often than "start point". So there's an asymmetry here, which can be expressed as an odds ratio of 3000. That's the English language! Looking at the actual examples on Google, it seems that "start point" is really not very good, whereas "starting time" is acceptable, and indeed a Praat script accepts Get starting time as an intentional synonym for Get start time, simply because the asymmetry in the English language is easy to overlook when you are writing a script. So here we actually considered the synonym useful, and didn't deprecate when we went from "starting time" to "start time" in 2004... So, again, many of these decisions were conscious and have a story behind them, and continuing to reconsider command names is a natural way to proceed. @jjatria, you like to change some command names? Well, I agree, and I like to assert that doing so is easier if the old names continue to work, keeping the Belgian speech paths really happy. Oh, and the visual Get starting point inconsistency? It will completely go away once the command is called Get start time of interval.

from praat.

@PaulBoersma Your example is illustrative, and shows that your choices about maintaining backward compatibility within Praat are well motivated with regard to the Praat user base. But won't the user base of _a Praat API_ all be at least somewhat proficient as computer programmers? Isn't it fair to expect that population to take responsibility for tracking versions of their dependencies, and to avoid chaining together several scripts of different vintages that require different versions of the API?

I won't press the point any futher, since I have already acknowledged that I eschew backwards compatability for selfish reasons (and so I must defer to your preferences if you're the one maintaining the API). Moreover, as someone wrapping the API, I would be free to ignore synonymous commands and would suffer less (or not at all) from your upstream changes if you maintain backwards compatability, so I don't stand to lose anything if you do things your way instead of mine.

from praat.

@PaulBoersma nailed it when he described the separation between interface and implementation. But in his view those were the only two levels, so there is but one interface, and it must serve for both the GUI (and the scripting language) and the library. This is, I think, an error, and what I was thinking what I said that I thought we had different ideas of what the library should be.

(Incidentally, when I talk about "the GUI" I'm really talking about what @PaulBoersma refers to as "the interface functions", because those two - and the scripting language - seem to be currently indelibly linked. This - and not the TextGrid commands - is one of those "serious design problems" I had in mind).

Currently, because there exists but a single interface, anytime there should be a change in the GUI (to reflect changing tendencies, to clarify ambiguities, to fix inaccuracies, etc) that means a new command that must remain there for backwards compatibility. If the API is yet another way to access this same interface (in addition to the GUI and the scripting language), then any of these changes will result in a change to the API, and public APIs should not change. (This also means that the GUI cannot be translated, for example).

What I've been trying to propose is that the library is developed as a new layer in between these two. That way the graphical interface (the GUI, or the current interface) and the programmable interface (the API, or what would be exposed by the library) exist in different levels. The GUI makes API calls in the background (which should not be linked to the labels of the buttons in that GUI), and the API hides the implementation. That way both ends can change and the API stays the same.

And because it would be a new interface (a programmable interface), it could target a very specific set of stake holders (programmers) and learn from the mistakes of the past regarding, for instance, the development of new techniques and algorithms and the problematic name choices.

PS. I think renaming Get starting point... for Get start time of interval... would only be a good idea if all those commands changed in a similar and predictable way.

from praat.

to reflect changing tendencies, to clarify ambiguities, to fix inaccuracies

If you want that to happen in the menu commands, you will also want that to happen in the functions accessible in the library. I don't see how a library can be kept fixed while Praat itself keeps improving.

By the way, I object to calling the development of new techniques a "mistake from the past"; they were usually conscious decisions that were appropriate in their context. I also object to your framing of the problems you perceive as "facts". Praat is being developed and will continue to be developed as our field progresses. Our inexpensive way of maintaining backward compatibility (by hiding but not immediately removing the commands) is a conscious strategy that intends to flexibly react to new developments in the field while not breaking existing scripts, which are often of a high quality themselves and on which people have sometimes spent hundreds of hours. You will be advising people to script Praat from Python rather than from Praat script; this advice will make sense only if we provide at least the same services from Python scripting as from Praat scripting, and that involves slow deprecation. I think ten years is not excessively long.

It is easy to have the user decide. I will include a function in the API (both in Praat script and in Python) for setting the level of adherence to the newest command names. As in other software that does this, there will be three levels:

1. calling a deprecated command is fine
2. calling a deprecated command issues a warning, but the script will run on
3. calling a deprecated command issues an exception, which stops the script

from praat.

I don't see how a library can be kept fixed while Praat itself keeps improving.

This would be easy to do if the API existed in a different layer between the GUI and the underlying internal implementation. This is precisely what I've been suggesting, and indeed how many other programs with both a GUI and scripting features are designed (or indeed with localisable GUIs). Just to take two very different examples, this is true for Inkscape, and for Kate: they have a GUI, and an API, and both of these interfaces can be different because they target different sets of users and aim at solving different sets of needs. The case of Praat would be different, but these examples show that you can have more than one different interface.

As an exercise, here are some statements. Let me know when we start to disagree:

1. Different users have different requirements.
2. Most of the time, users can be grouped into sets based on those requirements.
3. The requirements of users in the same group are similar.
4. Users interact with the program using an interface.
5. An interface needs to be tailored to the needs of its stake holders.
6. The needs of some groups of users will sometimes not be compatible.
7. No interface can accommodate the needs of all users.
8. There is no need to have a single interface.
9. Having too many interfaces is also a problem.

Currently, Praat has but one interface. The commands available through the GUI are the same as those available through the scripting language. This means that the labels on the buttons of the GUI are the same as the names of the commands for the scripting language. This is the case despite the fact that the GUI and the scripting interfaces aim at normally different sets of users, and that these users will have different requirements: what is good in a GUI is not the same as what is good in a scripting language.

I do not suggest we change the scripting language, since that would break backwards compatibility. I care about that as much as you do, and that ship has sailed.

What I do suggest is that a library is developed as an additional interface, a programmable interface, that aims at the needs of developers. And I do not mean Python developers. I'm not one of them and I would not suggest anyone to write anything in Python (I don't know where you got that idea). I mean the developers of the Praat GUI (which should use the API in the background, to solve the problem you raised in your previous message), the developers of other C and C++ applications that use Praat algorithms (who will need the C library this issue is about), the developers that will write the language bindings for whatever language they may use, etc.

I object to calling the development of new techniques a "mistake from the past"

Me too. I never said that the development of a new technique was a mistake. I did say that there had been mistakes in the past (when talking about the design of the Praat interface), and I said that some of those mistakes were about the development of new techniques. Specifically, the development of a new technique is not a mistake; changing the API is a mistake (or "reveals a mistake", if the change is a correction).

this [...] will make sense only if we provide at least the same services from Python scripting as from Praat scripting

Not at all. There are a lot of aspects of Praat scripting that are irrelevant from within a high level language. Most of them will already have arrays and hashes, so those do not need to be provided. The same goes for most of the numeric and string functions (with maybe a few exceptions, like those dealing with the trigraphs), and those for file I/O. I would argue the same goes for the object selection, pauses, forms, and the editor commands, which only really make sense within Praat. And if you press me, I'd even suggest that the picture commands should not be a part of the library (which I already mentioned in my second comment).

I also object to your framing of the problems you perceive as "facts".

I don't know what you mean by this.

from praat.

@jjatria good to see some numbering. Let's try some more of that:

1. Documentation is half of the work of making a tool usable.
2. If there are two interfaces to the same underlying functionality, it would be good not to have to document the functionality twice.
3. Providing proper subsets of the functionality to various identifiable groups of users is a nightmare.
4. The nightmare can be prevented if all users have access to all the functionality.
5. If a user has access to all the functionality, but needs only a proper subset of it, it would be very good if the parts of the functionality the user does not need, does not intrude.

In Praat, point 5 is a design criterion. Praat has lots of commands about "Optimality Theory", but you will never see those if you don't start by touching the single "Optimality Theory" submenu in the New menu.

With "this will make sense only if we provide at least the same services from Python scripting as from Praat scripting" I of course did not mean that features of the Praat scripting language have to be reused. Those are already present in Python, so we automatically "provide" them by having the API. Thus I agree with you, no arrays and hashes (except inside "Formula..." commands, I guess), fewer numeric functions (but conversion of auditory features?), no object selection indeed, no pauses, no forms, and no editor commands indeed. As for drawing pitch contours to a PNG file, this seems hard to do directly from Python, so I would choose for the drawing commands to stay, at least for the time being (everybody is free not to use them); eventually, an integration with a Python canvas will be better.

Specifically, the development of a new technique is not a mistake; changing the API is a mistake (or "reveals a mistake", if the change is a correction).

This one is interesting. I was just looking at the command To Spectrum, which we had until 2000. Then we included an algorithm for non-FFT versions, so we had to split this into To Spectrum (fft) and To Spectrum (dft). Now, the latter formulation was slightly ambiguous, because an FFT is a special case of a DFT, and since we could collapse the two in the manual anyway, we chose to combine them into a single To Spectrum... in 2004, with "Fast" as a boolean argument, with the manual explaining that if "Fast" is true the algorithm will work faster but the result will be based on a zero-padded sound if the number of samples is not a power of 2. Both of these API changes I would consider normal results of refactoring the interface, which is a normal type of evolution in software. Although the changes were in a sense "corrections" (or at least improvements), there wasn't anything in this process that I would consider much of a "mistake".

Why was I looking at To Spectrum? I just scanned the whole of Praat for deprecations, and added the deprecation year wherever possible. There was large-scale refactoring of menu commands in 2004, 2007 and 2011. It's now time to remove the pre-2004 set, perhaps with appropriate error messages until 2024 ("The command XX no longer exists for a YY object; use ZZ instead"), where ZZ can be a new name, or a new command with a different (usually extended) argument set, or sometimes a sequence of two commands (e.g. saving two Sound objects to a stereo file was no longer needed when Sounds could have multiple channels in 2007, and you could instead use Combine to stereo followed by one of the normal Save commands).

By the way, some stuff has really been removed from the Praat scripting language already. Does anybody remember the following syntax?:

Font size... 11
let top = 'Viewport_top' - 0.1
let bottom = 'Viewport_bottom' + 0.1
let width = 'Viewport_width'
Viewport... 0 'width' 'top' 'bottom'

The let directive was deprecated in 1998 and removed in 2003. Apple's new Swift language has it again; it also has variable substitution, which Praat script has also almost gotten rid of.

from praat.

I'm glad you liked the numbering, but you didn't actually tell me where we started disagreeing. From your answer, though, it seems as if the point where we diverge is that you do think that a single interface can satisfy everybody's requirements. This, I think, explains the reason for some of the existing design flaws.

Like that, there are a lot of other things I've mentioned as arguments in support of an intermediate interface (an API for a library) that you haven't addressed. Here's some bullet points now:

• Having one would allow for changes in the GUI, present and future, while sustaining a stable API.
• Allowing for changes in the GUI would in turn allow for localisation.
• A new API (targeted at developers) would offer the chance for adding corrections and improvements (and reducing the overall complexity) without the need to break compatibility.
• A new well designed API would not force the new users to learn command names that are designed to look good in a GUI they've never seen or used.

And here are some new ones:

• Nobody has said anything about "providing [...] subsets of the functionality" to different sets of users. The same functionality could be provided by different interfaces, each of which addresses the requirements of different media and users (what is good for a GUI is not what is good for a language, what an end user needs is not the same as what a developer needs, etc).
• Having a single interface for all users is good for the developer, but bad for the users. Since the interface is designed primarily with the GUI in mind, the users that are most affected are developers. The library would address this issue.

there wasn't anything in this process that I would consider much of a "mistake"

If not in that process, then before. Tying the interface to the GUI in the first place meant that these things would almost certainly happen. I've said before that public APIs shouldn't change, but I understand that sometimes those changes are inevitable. The current interface design, however, has a built-in tendency to change. That I think is a mistake.

no object selection indeed

Good! I'm happy to see this. Your first "draft" of how it might look had it in there, and I was worried.

drawing pitch contours to a PNG file [...] seems hard to do directly from Python, so I would choose for the drawing commands to stay

This issue is not Python-specific, nor even Python-related. Whether generating PNG files is easy or hard in Python is irrelevant to whether generating PNG files should be part of the library.

I don't think this belongs as part of the library because I think that the library should be as general as it can be, to be usable in as many different contexts and environments where it is applicable, and I don't know if there is a sensible way to implement general purpose plotting features. As long as the developer has access to the internal objects (which I think should be a must), they can read them and plot them in whatever way they find better suited to their specific needs.

And if they need to plot PNG files, and they are using Python (for some reason), and that is difficult to do in Python... then that's not really a problem that Praat should try to fix.

from praat.

I'm glad you liked the numbering, but you didn't actually tell me where we started disagreeing.

Oh, I agree with all of them, including the last one:

.9. Having too many interfaces is also a problem.

We're talking here about two or three interfaces, aren't we? And if one starts implementing separate interfaces for different subsets of users, one could get "too many" of them, which was one of my points, so how could I disagree?

Perhaps I disagree a bit with this one:

.7. No interface can accommodate the needs of all users.

If the interface has all the functions, then any sub-niche of users can pick what they need, I would think. Of course, any user could have a need that is not answered by anything in Praat, so in that sense your statement is true, although I don't see the relevance to the API we're gonna create.

from praat.

if they need to plot PNG files, and they are using Python (for some reason), and that is difficult to do in Python... then that's not really a problem that Praat should try to fix.

I was not talking about plotting in Python being difficult. I was talking about plotting a Pitch object being difficult. The logic of it is difficult to get right in a general-purpose language; you would basically have to rewrite Pitch_draw() and the functions it calls, which is non-trivial. Pitch curves are discontinuous and you need linear interpolation between adjacent points as well as constant extrapolation to fill up the frames (no linear extrapolation, because pitches cannot be negative). The problem has been solved in Praat, and therefore in the Praat library, and we don't want to burden users with solving the same problem, probably by going through a series of mistakes. So it is preferable that our solution to pitch drawing is made available to the users of the library.

from praat.

Localisation.

Praat is English-only. Why?

1. Localizing Praat means not only translating the 8000+ menu commands, but also translating the 1500+ manual pages.
2. Translating is not enough. The field develops and Praat develops: there will be many more manual pages in the future. Without maintenance of the translation, there is no localization.
3. Localization of large flexible programs can therefore only be afforded by large companies, who employ people in all countries for this.
4. Even for big companies, localization is difficult. While Apple's Dutch localization used to be excellent in the nineties, much of it nowadays sounds like a literal translation from the English and can only be understood by translating it back, as has been the case for Microsoft software from the start. Microsoft Excel has the menu command "Cut cells" in the Insert menu, which is meant to refer to "The cut cells", i.e. "The cells that have been cut"; however, "Cut cells" can also mean "Cut the cells", i.e. "Please cut the cells I selected", and that is how Microsoft has been showing this command in the Dutch version for 15 years now.
5. Fortunately, Praat is intended for users who can read research articles, and except perhaps in China and Brazil the majority of researchers write their research papers in English.

Now, there have indeed been translations of the manual in other languages. Most of them contain mistakes, but more importantly, they all have become obsolete. So we weight the above 5 arguments against localization with the big argument in favour of it (helping more users), and there hasn't been a moment in which the balance was in favour of localization. We are planning to localize the text around the pictures, e.g. having Time represented as Zeit or Temps or Tid depending on the language the user selects from the Language menu in the Picture window.

Perhaps you are optimistic about some of these issues for Praat (a large cooperative and knowledgeable user base, perhaps), but I am not. When users ask for features, and I ask them for solving the related interface problem, they never write me back...

from praat.

the existing design flaws.

I object to the word "the".

from praat.

arguments in support of an intermediate interface [between underlying functions and GUI]

the [current] interface is designed primarily with the GUI in mind

There are already two interfaces in Praat. One is really a GUI: when you want to look into a Sound and modify it, you can use the Sound window, which has zooming, scrolling, mouse-selecting, mouse-dragging, and clicking on parts. The other is the commands in the Objects window. I would say that the latter set is intermediate between the underlying functions and the GUI. This set is more comprehensive than what direct manipulation in the GUI allows; many commands have arguments (which is more coding-like than GUI-like); and many commands, such as "Get start time of interval" are really used more often in scripts than by hand. So we already have two sets, with the set in the Objects window providing a functionality that is a proper superset of that of the set in the editing window. That is why I see the set of commands that people would expect from an API to the library as roughly corresponding to the set of commands in the Objects window, while the commands in the editor windows, whose meaning depends on context and settings, will not be included.

from praat.

You still haven't addressed the points I made. Indeed, many of the things you address in your responses are either things I haven't actually said, or things that are not the important parts of what I say.

As an example: you said that you didn't understand how the API could be stable (which is a requirement of a good API, by the way) if the GUI changed. I gave an example of a way to achieve this, gave examples of applications that used this approach, mentioned the benefits of this approach for users and developers, and mentioned localisation as a bonus that is also made possible. Your response was about why you were against localisation*.

You did touch on your impression that "there are already two interfaces in Praat". While of course this is true in one way (there is a GUI and a scripting language), this is not the case in the sense that matters. Both of these interfaces are the same: the same functions, with the same names, with the same parameters and the same behaviours, etc. You say that this is by design. But in that case I call that bad design, because it ignores the fact that the needs of a GUI and the needs of a scripting language are different. If there are indeed two interfaces, then it would be better if they were both designed to meet the requirements of its respective stake holders, which is not currently the case.

I would say that the latter set is intermediate between the underlying functions and the GUI

Then I haven't made myself clear. If a change in the label in a button of the GUI means also changing (adding) a command in the scripting language, then the latter is not intermediate between anything. It's the same interface (only one of them doesn't have clicky bits).

As for the plotting commands:

I was not talking about plotting in Python being difficult

I was confused by you saying that the argument for including those commands in the interface was that "drawing pitch contours to a PNG file [...] seems hard to do directly from Python".

The problem [of interpolating the pitch curve] has been solved in Praat [...] So it is preferable that our solution [...] is made available to the users of the library.

I completely agree with this. But computing the values that need to be plotted, and plotting them (to a PNG, to whatever a "Python canvas" might be, etc) are entirely different animals. So let me re-phrase: "As long as the developer has access to [the appropriate data from the] internal objects (which I think should be a must), they can read [it] and plot [it] in whatever way they find better suited to their specific needs."

Perhaps you are optimistic about some of these issues for Praat (a large cooperative and knowledgeable user base, perhaps), but I'm not

To quote you, "this one is interesting". Praat is a software for which there are not that many alternatives, certainly not with the same features. It is Free Software, and for a little more than a year now, available in what is probably the world's most popular software collaboration platform, with 69 people actively following its development (and counting). It has been going for more than a decade, and has 3000+ subscribers in its mailing list.

So why shouldn't there be a "large cooperative and knowledgeable user base" interested in contributing with the development? Because I agree: there isn't.

I would say that this is (at least partially) due to the fact that there are many barriers for this community to develop. I would say that some of those barriers include the lack of an open development plan, a coding style that is not really idiomatic, the lack of a public stable library API that makes sense, a transparent and open decision-making process that involves the community, etc.

I would also say, however, that you are right in thinking I am optimistic about this: otherwise I would not be here. But if you are not optimistic then it doesn't matter if anybody else is.

(*) Incidentally, I also have many issues with your aside about localisations: there are tons of successfully localised programs that are not managed by "large companies who employ people in all countries", including many that are Free Software developed by volunteers; and the difficulties you mention regarding the development of a good localisation are no argument against designing software with an architecture that supports localisation.

from praat.

Both of these interfaces are the same: the same functions, with the same names, with the same parameters and the same behaviours, etc. You say that this is by design. But in that case I call that bad design, because it ignores the fact that the needs of a GUI and the needs of a scripting language are different. If there are indeed two interfaces, then it would be better if they were both designed to meet the requirements of its respective stake holders, which is not currently the case.

Yes, it's by design. One goal is to have an easy entrance into the scripting language. One can create a script for garnishing a pitch drawing for one's research article by writing a sequence of ten menu commands and then run it. Easy to adapt later. The "history" mechanism can also help. And it's great for documentation. If somebody asks me what the scripting commands are for a Sound object, then the simple answer is "whatever you see in the menu when you select a Sound object."

I am stil very happy with that early design decision. It prevents double documentation and double names for the commands, and it enforces a one-to-one relation between the menu commands in the Objects window and the object commands in the scripting language. You now made it clear that you regard this as bad design. So be it. I find it at least transparent. I agree it is the reverse from what was usual when Praat was conceived in 1992, when applications tended to build their "GUIs" (actually menu commands) on top of a command-line syntax; I found that an example of bad design: in SPSS, for instance, you could do some things in the "GUI" and you had to do other things in the "syntax". Oh, SPSS and RStudio still work this way.

the fact that the needs of a GUI and the needs of a scripting language are different

Is that a "fact"? Users can handle stuff by choosing sequences of menu commands, but when they want to automate that, they start to use scripting. Many scripters need the same algorithms as other users, and many scripters have evolved from being non-scripting users. Why learn a new language to access those algorithms? Yes, the scripting language has control structures and arrays, so a scripter can benefit from this additional functionality, and has to learn how to work with them, but half of what people on average write into a script has a one-to-one correspondence with what they can do in the menus, so the user/scripter does not have to double their algorithm knowledge skills. Handling a comprehensive program is difficult enough as it is.

from praat.

So let me re-phrase: "As long as the developer has access to [the appropriate data from the] internal objects (which I think should be a must), they can read [it] and plot [it] in whatever way they find better suited to their specific needs."

I did mean to say that this is the difficult part. As far as I know, there is no generic plotting function for drawing a pitch curve in any language, including Python. Such a curve is discontinuous and in some first-degree entropy-maximizing view requires linear interpolation together with constant extrapolation, i.e., you don't just connect the measurement points. Drawing it in a different way is of course possible, but would violate the semantics of what a pitch curve is. The algorithm Praat uses may not be ideal in all respects; if somebody invents a better algorithm, we will incorporate it. My point is that a Pythonist who uses the Praat library should not be forced to reinvent the wheel just because we didn't make the drawing command available.

So why shouldn't there be a "large cooperative and knowledgeable user base" interested in contributing with the development?

I did not say that. I was talking about maintaining 1500+ manual pages that are continually extended and improved. There have been Praat beginner's manuals in the past. Very few of them received any updates. None of them received updates for longer than five years. Updating French manual pages involves much more work than updating a beginner's manual.

a coding style that is not really idiomatic

Wow, that sounds interesting. Any specific ideas for improvement of my coding style? We transferred all of Praat to C++11 around June 2015, getting rid of error-prone casting; any new steps? I'm all ears. Will C++14 solve any of the remaining awkwardnesses, such as our multiple inclusion of header files for automatically generating reading, writing and copying code? This would be my challenge to the developer community: how to make the code even neater/clearer/cleaner/more concise. Perhaps we could start a new Issue about this.

from praat.

Such a curve is discontinuous and [...] requires linear interpolation together with constant extrapolation

Maybe I'm being thick, but what then is the problem with having some function that returns such an interpolated curve without actually drawing it to some device? The curve is returned as a list, which can then be passed to whatever graphic library makes more sense in the specific context.

the fact that the needs of a GUI and the needs of a scripting language are different

Is that a "fact"?

Uhm... yes (and you don't need the scare quotes). One uses visual cues, the other is text-based. The media are different, the users are different, the use cases are different, etc. There is a reason there are zero books dealing with the design principles of both programmable and graphical interfaces, but many books dealing with the design of each of them separately.

In Amazon (YMMV):

• "gui api design": 3 results (none relevant)
• "api design": 2430 results
• "gui design": 6612 results

Any specific ideas for improvement of my coding style?

Certainly.

I do not consider myself a C or C++ coder, so I bet there are a lot of other things that could be improved. But a good place to start would be to do away with the idiosyncratic (=non-idiomatic) use of macros for attribute accessors and class definitions (and you don't have to take my word about them not being a good idea).

This would be my challenge to the developer community: how to make the code even neater/clearer/cleaner/more concise

Introducing automatic testing and continuous integration would be a plus, as would be to get rid of the gazillion warnings that pop up during compilation. Those might mean additional code, but they would definitely improve the quality of the code.

The directory structure of the project could also be a bit clearer: the only reason I can imagine for there being both a sys and a dwsys directory is that the files in one of them were probably written by David, which hardly seems like a good criterium for file separation.

The build process might also be made a bit smarter. Avoiding recursive make might be a good idea, as would be perhaps using something like autoconf or waf instead of having to manually copy files from makefiles/.

I think preferring standard solutions to standard problems would also help. One good example would be the generation of documentation, which is currently done with a Praat-specific solution when there are other battle-tested approaches that do not require a personal implementation. This would also apply to the automated tests: there are currently automated tests for the interpreter, but those also do not use a standard solution even though one is *ahem * available.

And you know what else I think would fall in this category? Introducing an intermediate API between the GUI and the underlying implementation, so that the code is more compartmentalised and it is easier to programmatically interact with Praat algorithms from outside Praat.

Perhaps we could start a new Issue about this.

Sounds good.

So why shouldn't there be a "large cooperative and knowledgeable user base" interested in contributing with the development?

I did not say that.

No. I guess it was I who said it. But I do stand by it. Why do you think there is no such user base? I've already said why I think it's the case.

from praat.

Maybe I'm being thick, but what then is the problem with having some function that returns such an interpolated curve without actually drawing it to some device? The curve is returned as a list, which can then be passed to whatever graphic library makes more sense in the specific context.

Bravo! That's exactly what happens when you run Praat without a GUI. To implement drawing on a Python canvas, we "just" need to implement a driver for that.

the fact that the needs of a GUI and the needs of a scripting language are different
Is that a "fact"?
Uhm... yes (and you don't need the scare quotes). One uses visual cues, the other is text-based.

OK, we're talking here about three levels. One of them is what I would call the GUI, which in Praat is how people handle e.g. the Sound window. Another level is programming. The third (intermediate) level is the menu structures in the Objects window. You call it a GUI (in the above quote), but it's actually just a way to access commands with arguments, without having to remember the names of the commands and the order and number of the arguments. You don't see this level often in application programs (though by now there are some others), and it is a level with its own dynamics. We introduced it in 1992 in order to be able to quickly supply functionality before or in addition to a true GUI, without forcing users to a command line interface. So one could call it a "visual CLI" or so. Whether the needs of this intermediate level and the scripting language overlap or not: well, I think they do.

a good place to start would be to do away with the idiosyncratic (=non-idiomatic) use of macros for attribute accessors and class definitions

Most object-oriented languages have a root class, called Object or so. C++ is a language without such a root class, and indeed without class information. Stroustrup had his reasons. In Praat we do need shared behaviour, especially for all objects that you see in the Objects window, like reading, writing, and copying. For reading we need the mapping from a class name (e.g. "Pitch) to a class (e.g. Pitch), which C++ doesn't provide. The use of macros for class definitions (I don't see examples of attribute accessors) has to do both with the shared behaviour and with the need for class information. If you know of a more "idiomatic" way to do this, then just tell us; saying that our current method is wrong may be easier than providing the better alternative.

Introducing automatic testing

We have a runAllTests.praat script, which calls many test scripts. I agree the set is not complete. If you have anything to add, please send us your test scripts.

get rid of the gazillion warnings that pop up during compilation

They are mainly in code provided by others, which we really don't want to touch. Wherever warnings are in our own code, we try to get rid of them.

Why do you think there is no such user base? I've already said why I think it's the case.

Yes, you seemed to suggest design by committee.

from praat.

I'm not sure sometimes whether you read what I write.

We have a runAllTests.praat script, which calls many test scripts.

I know. And I actually mentioned it in my previous comment. But these tests only cover the interpreter. Praat has no tests for the code itself (without going through the scripting language), it has no continuous integration, and those tests use an ad-hoc solution for a standard problem (which are also things I mentioned in my previous comment, and also things you chose to ignore).

[The compiler warnings] are mainly in code provided by others, which we really don't want to touch

Isn't this a smelly smell.

Code provided by others into a project (and accepted into its code base) becomes part of the code in the project, and as such its maintainance falls under the responsibility of the project maintainer. If the project maintainers don't want to touch it, then that sounds to me like a ticking time bomb. If we understand "our" as "the Praat team's", then this is "our code".

I don't see examples of attribute accessors

I guess you didn't follow the link I provided. I meant these macros.

If you know of a more "idiomatic" way to do this, then just tell us; saying that our current method is wrong may be easier than providing the better alternative

Of course it is easier. But the fact that I don't know the solution to a problem does not mean that the problem suddenly disappears, nor that it is not helpful to point out that the problem exists.

And there you go with the scare quotes again. It's not like I invented the term "idiomatic" when talking about programming languages. If you think this is totally idiomatic, then I'd be happy to see examples of other similar projects written in the same way. If there are none, then by definition this is not.

you seemed to suggest design by committee.

Did I? I don't think I did. This is a straw man if there ever was one. From your own link:

One of these was that project teams should be egalitarian; in other words, that everyone should have an equal say and that decisions are democratic. This leads to Design by Committee.

I never suggested the development team should be egalitarian, nor that decisions should be democratic, and I never said anything to deny the fact that you are the project lead and the "designated architect" (to quote the same page again).

But (lucky us) there is a long way from involving the community in a transparent decision-making process (which is I guess the one aspect of the things I mentioned that you chose to focus on), and an egalitarian committee-based design process with no clear lead.

Maybe I'm being thick, but what then is the problem with having some function that returns such an interpolated curve [a a list] without actually drawing it to some device?

Bravo!

Thank you! :D

But this is exactly what I had said before. Let me re-phrase again, this time in a less general form: "As long as the developer has access to [the list of values of the appropriately interpolated curve], they can read [it] and plot [it] in whatever way they find better suited to their specific needs."

And (unless I'm understanding the term "driver" in some specially obtuse way) we don't need to implement a driver to draw into a Python canvas or any other such device: we can just let the developer figure out how to best take the values in that list and draw them onto the screen. Like I said before, "that's not really a problem that Praat should try to fix".

So one could call it a "visual CLI" or so. Whether the needs of this intermediate level and the scripting language overlap or not: well, I think they do.

I don't agree with much of what you wrote here (the idea of a "visual CLI" as something different from a "GUI" doesn't make much sense to me), but I feel delving deeper into it will distract from the topic at hand.

I think the levels that you bring up do not really contribute much to understanding things, because all of these still use the same interface, and this is an interface that is aimed at making the users of those tools happy.

What I've been trying to say is that there is a different set of users, which so far hasn't been addressed in the design of the Praat interface (programmable, graphic, or whatever the visual CLI is), and I fear that when I say "programmable" you think of Praat script writers. The users I mean are developers of other C and C++ applications, and of other higher-level languages, who may want to interact with Praat from outside Praat itself. Users who may want to write alternative GUIs for specific purposes, or integrate acoustic analysis features into their existing tools. Those are the users for which there is currently no interface available, and who would be the only users of a library.

The thing is, regardless of how much thought we put into a design, the litmus test for whether it is good or not will be whether people use it. And I would say that the minimal use of Praat among developers (not script writers) is not just because there is no library available, but because the interface as it is right now, and that you want to reproduce in the library, is not appropriate for those users. In all my years of knowing about Praat, I have never heard of a single developer that fell into the category described above who a) knew Praat and b) liked its interface.

If you go ahead and produce a library with an API that simply reproduces the existing one, I see only three possible outcomes:

1. People will not use it.
2. People will use it via additional wrappers, which will have to come up with their own interface resulting in a fragmented outcome.
3. People will use it because they must, dislike it while doing so, and move on to another tool as soon as they can.

The alternative would be to come up with an API that targeted those users (a single additional set of users, which is unlike any of the existing ones), and provide them with a unified, versatile and sensible interface that matched their needs and expectations. That's the approach I favour.

from praat.

I don't see examples of attribute accessors
I guess you didn't follow the link I provided. I meant these macros.

There was no code trick we enjoyed as much as "my", which occurs 28,000 times in the code, and used to occur twice as often when the code was in C. The need for "my" is due to a restriction in C++, which, other than other object-oriented languages, does not allow the developer to extend their classes. Thus, in Sound_to_Pitch() there is a logical self or this object (the Sound), but it has to be expressed explicitly. That is, one cannot have Sound::to_Pitch(), because one doesn't want this function declared in Sound.h, because one doesn't want Sounds to know about Pitches, and the reverse (modular programming, code encapsulation, right?). If you know of a solution, please tell us.

from praat.

But this is exactly what I had said before. Let me re-phrase again, this time in a less general form: "As long as the developer has access to [the list of values of the appropriately interpolated curve], they can read [it] and plot [it] in whatever way they find better suited to their specific needs."

Oh, but you literally said "As long as the developer has access to [the appropriate data from the] internal object", by which I thought you meant "as long as the developer has access to the pitch points," and we now seem to agree that that does not suffice. Great! Now, how do we transfer the appropriately interpolated curve to the Python user (rather than the pitch points)? Isn't that something quite analogous to the Draw command? Well, we'll see what the best way is in which we can transfer the drawing data.

By the way, we are marking the deprecation years everywhere in the code now. Approximately 6 percent of the current commands used to have a different name. The main causes: a split in functionality (inner and outer viewports, creating tables with and without column names, tab-separated versus comma-separated table files), changing terminology in the majority of applications (in the 1990s "Write", nowadays "Save"), and indeed some clarifications (Edit -> View & Edit). Six percent since 2004; that's not a terribly unstable interface, I estimate.

from praat.

I never suggested the development team should be egalitarian, nor that decisions should be democratic

Good. Then I must have over-interpreted your statement "a transparent and open decision-making process that involves the community." I agree it has other interpretations.

from praat.

The users I mean are developers of other C and C++ applications, and of other higher-level languages, who may want to interact with Praat from outside Praat itself. Users who may want to write alternative GUIs for specific purposes, or integrate acoustic analysis features into their existing tools. Those are the users for which there is currently no interface available, and who would be the only users of a library.

OK, the users I was thinking of were people who want to script Praat-like functionality and find that combining that with SciPy and TensorFlow and the like yields additional possibilities over what direct Praat scripting can provide. Perhaps that's the same group, defined with a slightly different focus. I think they can all get by by getting a header file to the full functionality and select the part they need.

from praat.

And there you go with the scare quotes again. It's not like I invented the term "idiomatic" when talking about programming languages. If you think this is totally idiomatic, then I'd be happy to see examples of other similar projects written in the same way. If there are none, then by definition this is not.

No scare quotes; I quoted the word because it was a term you used (as with "fact" before; also not your invention). But examples of other projects? I like to think that Praat is much bigger (in the good sense of the word, i.e. more functionality for the intended users, more algorithms to choose from) than other large projects written in the same time, and the "my" idiom was one of the big time-savers that helped in achieving that (the "visual CLI" was an even bigger one).

from praat.

To inject an aside that skirts most of the big issues discussed here:

how do we transfer the appropriately interpolated curve to the Python user (rather than the pitch points)? Isn't that something quite analogous to the Draw command? Well, we'll see what the best way is in which we can transfer the drawing data.

This is actually quite easy for Python, at least. Praat finds the pitch points, passes NaN wherever pitch is undefined, and matplotlib will do the right thing, plotting a connected line with gaps where the NaNs are. In fact, the NaNs only need to be in either the x or y values, so you could return a dense vector of times and a sparse vector of pitch estimates. No driver or drawing required.

from praat.

@drammock Does this do "linear extrapolation"? For instance, if there is a lonely voiced frame with two unvoiced frames at its sides, will it draw a small horizontal line with a length equal to a frame duration, or will it draw just a point? In general, what will it do beyond the first and last points of each stretch?

from praat.

So I guess I was right: you really don't read what I write.

I never suggested the development team should be egalitarian, nor that decisions should be democratic

Good. Then I must have over-interpreted your statement

So maybe now I can actually get you to answer the actual question: why do you think there isn't a 'large cooperative and knowledgeable user base' interested in contributing with the development?

you literally said "As long as the developer has access to [the appropriate data from the] internal object", by which I thought you meant "as long as the developer has access to the pitch points,"

Then you were once again arguing with a straw man. I specifically used that word to mean that the developer does not need a command that plots the data (because we don't know what device they'll be using), but they do need the data. What data? Whatever data is necessary for their current task. In other words: the appropriate data. For some things this will be the pitch points, for others it will be something else.

how do we transfer the appropriately interpolated curve to the [...] user?

As a list.

Isn't that something quite analogous to the Draw command?

Don't know. What do you mean by "analogous"? It is not if you consider the actual drawing of those points (and not just their calculation) as part of "the Draw command", which is what you implied before when you were talking about drivers and Python canvases and whatnots.

I think they can all get by by getting a header file to the full functionality and select the part they need.

They can also "get by" without a library at all. The whole discussion is not what they can "get by" with, but what is it that they need, and how reasonable it is to meet those needs.

However, I am happy to concede the point regarding the "full functionality" since I've never argued against it. I guess one of the parts you didn't read was the part where I said that this is not about limiting the functionality, but about providing a different interface to that functionality. One that will actually be used by the users who would want a library. And we've already agreed that this is compatible with selecting the things that are appropriate.

But examples of other projects? I like to think that Praat is much bigger [...] than other large projects written in the same time, and the "my" idiom was one of the big time-savers that helped in achieving that (the "visual CLI" was an even bigger one).

This is really disheartening. You say that the code is "optimized for clarity and organization", and yet you proudly boast of your clever my "trick" and the use of non-idiomatic code throughout its code base, part of which you "really don't want to touch" despite the fact it issues multiple warnings upon compilation.

I guess you never really did click on that link I gave you, but it shows Praat 18th in a list of "the worst real-world macros abuse" in StackOverflow, exclusively because of your my "trick". And with good reason: clever code is smelly code, because it is obscure for everyone who is not your particular flavour of clever.

I guess the worst part of this is that you blame this on apparent limitations of the C++ language, and yet you are unable to find any examples of any other application using a similar approach, because Praat is so "much bigger" (I'd just like to point out here that one of the symptoms of your dreaded "design by committee" is "excess complexity"). I would imagine that if this was truly a problem with C++, then there would be others who have also been forced to come up with similar solutions to this problem. The fact that there aren't makes me think that the problem is not with C++, but with your design.

from praat.

As for "idiomatic", it is well known that C++ has multiple idioms, e.g. one for inheritance and one for template programming, and that they don't mix well. Almost everybody doing C++ therefore settles on a subpart of it in their projects (e.g. minimizing templates, or no multiple inheritance). The term "idiomatic" is also used for a consistent coding style within a project, and we find that more important than the similarities (or not) with the coding style of other projects. That is, there should be as few surprises as possible when reading code fragment X for people who have seen many other code fragments in the same project. The "my" trick is a huge readability device for us. Indeed, the parts supplied by others (PortAudio, eSpeak, FLAC, MAD) are in different styles (all different among each other as well), and we don't want to touch them too much, because (1) that will lead to bugs, which are difficult to find because we are not the authors, and (2) their authors will come with upgrades so we would have to redo all that editing.

When writing "much bigger", I sort of expected (because of earlier statements) that you would equate this with "excess complexity", and so you did, but size and complexity are really not the same. Whether big means complex depends on how the parts depend on each other. Praat's structure has been designed as flat, i.e. one can add a module about, say, three-dimensional modelling of tongue movements without affecting the rest of Praat; the module can be included in the application with just one line of code at the highest level (e.g. main.cpp). Flatness is how projects that model a whole world (here, the world of speech research, or at least some non-negligible parts of it) have to cope with the size of their subject matter.

Let's close the "big issues" part of this thread, and focus on how the task can be done. I'll send an explicit proposal for the C interface.

from praat.

It's been 8 months since the last post on this topic. Are we any closer to seeing that "explicit proposal"?

from praat.

Do "Create C interface..." from the Technical menu, and you will see a beginning, at least as far as the fixed commands are concerned. When you browse them, please skip the first eight. The others are also rough. Thus,

/* Menu command "Create PitchTier..." */
PraatObject Praat_PitchTier_create (
	const char *name,   // e.g. "empty"
	double startTime,   // e.g. 0.0 s
	double endTime   // e.g. 1.0 s
);

might become

/* Menu command "Create PitchTier..." */
/* For a description see http://www.fon.hum.uva.nl/praat/manual/Create_PitchTier___.html */
PraatPitchTier Praat_PitchTier_create (
	const char *name,   // e.g. "empty"
	double startTime,   // e.g. 0.0 s
	double endTime   // e.g. 1.0 s
);

i.e. with stronger typing and with a link to the manual. This list is generated automatically from the form descriptions, and the functions called go through the same error checking as you would get when using these commands from the GUI or from a script. The commands from the dynamic menus will have PraatObjects (or PraatSound, PraatPitchTier...) as their first arguments.

from praat.

Hello everyone, I am working on a language identification task using prosodic features. I know how to extract pitch and formants and all using Praat manually. But as I am working with a large dataset, I would like to know on how can I automate the task of prosodic features extraction using praat. thanks

from praat.

This is out of topic here. Please check first whether the Praat scripting language can do what you need.

from praat.

okay sir. sorry sir. I am new to github and don't know much so I thought raising my query over here could be solved by you all. So where should I raise my queries related to Praat scripting?

from praat.

@Shashwat121, you can post a question to relevant areas of stackexchange or stackoverflow.

P.S., @usagi5886 does something like what you're after in his dissertation. Please direct inquiries elsewhere, though.

from praat.

@PaulBoersma, I assume the fact that you closed this issue means significant progress has been made. Could you share an update on it?

from praat.

@Shashwat121, you could become a member of the Praat discussion list.

from praat.

@jonorthwash, sorry you're right, I thought I was re-closing after Shashwat121 re-opened.

from praat.

@PaulBoersma Okay sir, Could you share me the link for the Praat Discussion list? That would be very helpful.

from praat.

Go to the Praat website and click on Praat User List.

from praat.

Dear all,

If I just want to get the pitch only with .cpp and .h file, how I could Do to solve the problem ?

If there are some main functions or test files with .cpp file ,that would be great !

Thanks a lot.

from praat.

Dear all,
If you can understand Chinese, and share your great things about Praat in C/C++,

Please come into the QQ group.

from praat.

@ucasiggcas the GitHub issues are not for usage questions. Please use the praat users mailing list instead.

from praat.

@ucasiggcas the GitHub issues are not for usage questions. Please use the praat users mailing list instead.

Emm，
I know , but in P.R.China I can't open the link you said

from praat.

Indeed, that company seems to have been blocked since September 2018.

from praat.

I compiled praat from source on linux and I see that creating a c interface from the menu is available in this version, but I don't know if it's really usable and I couldn't find any help on how to use anywhere (including discussion list, website). I even try to find something in the source code but still not found it.
where can I find some information about it?

from praat.