aichaos / rivescript.next Goto Github PK

Related to #1, list some of the important design considerations that should go into the new language. I'll give some examples to start with.

• Unicode as a first-class citizen. The new language should accept that user messages will oftentimes contain foreign symbols and special characters and it should be able to handle these gracefully. When possible, the user's untainted original message should be available at all stages of the reply process. They should be able to tell the bot their e-mail address and have that string make it all the way through to a custom object macro without losing any meaningful characters along the way.
• Flexible wildcard matching patterns: support syntaxes like *2 to catch "exactly two words" or *~4 to catch "up to four words" and so-on.
• Support out-of-band data transfers in both directions: the bot author could attach an arbitrary key/value store along with the message request and the bot should be able to return similar with its reply.

This issue is reserved for discussion about the problems in the current version of RiveScript.

Use a bullet list for each major point, to make it easy to scan through the comments and tally up all the problems.

I'll start with some that I've encountered:

• The line-delimited nature of the language feels claustrophobic.

Especially when trying to do something like have a random reply in response to a condition check. You either have to do this:

+ my name is *
* <formal> == <bot name> => {random}
^ Oh wow, we have the same name!|
^ My name is also <bot name>!|
^ We have the same name!{/random}
^ <set name=<formal>>
- <set name=<formal>>Nice to meet you, <get name>!

...or you use an {@redirect} to a different trigger. Neither is very ideal.

• You can't easily nest conditionals. You have to chain together a bunch of {@redirects} to check multiple vars for the same trigger.
• Tags can't be mixed and matched in arbitrary order. Only the get/set/bot/env/sub/add/div/mult tags work this way, but all the other tags have a fixed order in which they're called. You can't <set> based on the output of a <call> tag, for example.
• The difference in syntax where some tags have <angled> brackets and others have {curly}. Related to the above point, this made the parser harder to write for processing tags.
• %Previous is hard to work with: a trigger with a %Previous is its own "top level" trigger, disjointed from the previous trigger that it refers to. It makes it hard to keep your code straight if you're trying to design a more complicated discussion around these.
• Everything is too "text based", as the unit of knowledge transfer is the text string. This is particularly annoying with object macros, where the args passed in has to be a sentence of words (you can't easily use JSON as the input data), and the result of the macro is also a string that gets substituted in for the <call> tag. Instead it should be more data-driven, with a struct being used in all places where additional metadata can be carried throughout the bot in the replying process.

This issue is reserved to discuss syntax ideas for how the successor to RiveScript might look.

Provide example code snippets for how you think "real world" code might look, and explain your reasoning.

I'll start:

I think the new language should resemble Python and be whitespace sensitive (no curly brackets for blocks). This would:

• Make the language easier to parse. I don't care too much for writing a proper interpreter with a lexer, parser, tokenizer, abstract syntax tree and language grammar.
• Improve the end user's code readability. Languages with curly braces can be minified and compressed down to one long line of code, for example, which isn't very user friendly to read.

Line noise should also be kept minimal, meaning there shouldn't be too many 'special symbols' in a source file. A user unfamiliar with the language should be able to grok what is happening without running for the documentation.

I also like a "callback style" approach, where you could almost treat the user triggers as function definitions at the top level of the source. The commands that follow the trigger would be processed imperatively (top to bottom). What RiveScript calls "tags" that don't affect the bot's output text should be handled completely separately from that output text; for example instead of a <set> tag to set a user variable, have a dedicated "set variable" statement in the language.

This syntax idea is by no means final and are just some preliminary thoughts on how to do some common tasks:

// A trigger definition, the parent of all reply-related things.
// The ":" after the keyword "on" is optional, and that's true of all the keywords.
// You'd see "on:" most often in triggers to clearly distinguish where the trigger
// begins, and on other keywords when readability is needed (e.g. "say:" would
// be a valid way to write the "say" keyword, too).
on: hello bot
    // The "say" keyword would send one "sentence" of reply to the user.
    // You can use `say` many times and each sentence is added to the
    // end of the reply, with a space separator.
    say Hello, human!

// Conditionals
on: hello
    if <get name> is defined:
        say Hello again, <get name>!
    else:
        say Hello there!

// Setting user variables is done out-of-band with the reply parts.
// But getting a user variable is done in-line because you need to control
// the format and presentation of the bot's reply.
on: my name is *
    set name = <formal>
    say It's nice to meet you, <get name>!

// For random replies maybe have a "randomize" keyword, where we switch
// to a YAML style array syntax; you'd have a bulleted list of steps to pick from.
// A randomize step could have multiple actions, like this example the bot might
// say "I'm good, thanks for asking. And you?"
on: how are you
    randomize:
        - say I'm good,
          say thanks for asking.
        - say I'm doing well.
    randomize:
        - say And you?
        - say How are you doing?

// Callback for further discussions (like %Previous)
on: knock knock
    say Who's there?
    expect: *
        say <sentence> who?
        expect: *
            say Haha! <sentence>! That's hilarious!
    expect: banana
        say I'm not gonna fall for that old trick!

// Some users of RiveScript wanted to associate out-of-band, arbitrary data
// with the bot's reply. This could be supported with an "oob" tag, like in AIML
// on Pandorabots. The "oob" tags could be key/value pairs, short-lived (only
// for the current reply request), and returned along with the reply as a JSON
// serializable object. If you want to you could put literal JSON text into a value
// and parse it on your own if you have more advanced needs.
on: send a message to * and say *
    oob recipient = <star1>
    oob message = <star2>
    say OK, I'll send "<star2>" to the user <star1>.

Additional thoughts:

• Replace some tags in RiveScript with "standard macros" for the new language. For example instead of a <formal> tag to capitalize names, have a standard function do that job. Calling syntax for the function is still up in the air; if it looks like ChatScript you might write ^formal(<star1>) to formalize the first wildcard. This also adds flexibility for end users to extend the language more easily: RiveScript has uppercase/lowercase/formal/sentence case, but let the users define their own formatter that they can use just as easily from their code.
• Extending a line of code across multiple should use a trailing backslash \ instead of its own command.
• Callbacks (%Previous) should be grouped under its parent triggers. It was madness that they were treated as a separate top-level trigger in RiveScript: it meant they needed a lot of special treatment and black magic to keep them sorted out from the 'normal' triggers.
• Unicode wasn't a design consideration in the beginning. To stay on the safe side RiveScript removes as much as it can from messages, which is not always ideal and you can lose important formatting and special symbols from the original message.