Right now, the documentation has a few examples for a few features and a table with language syntax, but there's very little information on how to get started with a new project. There should be a "Getting Started" page that helps people set up the structure for a project and how to use databind with it. The file structure follows normal datapacks, but not everyone will know how to set one up.

Some things to do:

• "Getting Started" page
  • Installation
  • Project creation with databind create
  • Explanation of main.databind versus other files
  • Building
  • How to use multiple files
• Reformat or update other pages if they're hard to understand or missing information

WIth the current --generate-func-tags argument, function tags in minecraft/tags/functions are generated for functions specified in the config, or the default functions (load and tick). However, it's not currently possible to have multiple functions in one tag with this setup. It should be possible to tag functions in-code instead of having to manually create the tag files when using multiple namespaces or multiple functions that should be tagged.

The syntax could look something like this:

:func example :tag load :tag secondTag
# etc
:endfunc

or:

:func example
:tag load
:tag secondTag
# etc
:endfunc

The second would probably be better since it would allow multiple tags on one function without creating long lines.

Right now, the integration tests are set up via a bunch of files that each contain only a couple of functions. Moving these test functions into files with more general purposes (eg. file_structure_tests.rs and file_contents_tests.rs) would mean less files in the tests folder and a more clear idea of where to place new tests.

It's a bit weird to have some .databind files that generate their own functions and some that don't. It would probably be best to remove the function_out_exclusions config option and make what it does default for all files. This would mean that every .databind file should only contain function definitions.
This behavior makes the most sense since tagging can only be done in function definitions. Another option would be to keep the current system but to allow :tags at the top of files, but the function_out_exclusions config option can be confusing and is unlike any other language.

Documentation mentioning function_out_exclusions should also be updated.

The compile() function has to do some extra work when it comes to tokens such as Token::VarName. It could mean that a variable is being modified, returning a test for an execute if, being deleted, or, when #45 is closed, returning something for a scoreboard player operation. Having unique tokens for each of these cases (eg. ModVarName, TestVarName, DelVarName, and OpVarName) would meant that the compiler doesn't have to look at the last token to find out what it should do with the variable name it's given.

Right now, the tests for the contents of while loops have to use globs to find the correct files. If more than one while loop was to be used in a test, then it would take some extra code to find out which output file was generated by which loop. This problem gets more complicated with if/else statements (#84), since they generate more files and some files won't always exist.

Adding a CLI option to pass a list of chars to use instead (eg. --chars aaaa,bbbb) could solve this problem, since the output files will always have the same characters at the end.

I think that Clap can deal with multiple values passed for the same argument, but I'm not sure if it's comma-separated, or if you pass the argument more than once (eg. --chars aaaa --chars bbbb), or something else.

Right now, a function might look like this:

func example
say Hello, World!
end

It would be easier to tell what's where if tabbing in lines inside functions was recommended, like this:

func example
    say Hello, World!
end

Or this:

func example
    var i := 10
    while tvar i matches 1..
        var i -= 1
    end
end

Documentation examples and test .databind files should also be changed.

Having a variables.toml file or something similar to define global variables could make it easier to let users customize settings. An example of a variables.toml file could look like this:

delay_in_ticks = 500
do_something = true
do_something_else = false

True or false values would be converted to 1 or 0.

The globals could then be referenced in a .databind file by using some preceding character, like a %:

func load
tag load
    var do_something := %do_something
    var do_something_else := %do_something_else
    var delay := %delay_in_ticks
end

func tick
tag tick
    execute if tvar do_something matches 1 run tellraw @a "do_something is true"
    execute if tvar do_something_else matches 1 run tellraw @a "do_something_else is true"
end

Pull request #7 added a :tag keyword and removed the generate_func_tags config option. Most documentation examples still use files named load.databind and tick.databind, but these files are no longer automatically tagged. Examples should be replaced with single main.databind files using functions tagged with :tag, like this:

example/data/example/functions/main.databind

:func load :tag load
tellraw @a "Loaded"
:endfunc

:func tick :tag tick
tellraw @a "Ticking"
:endfunc

Transpiled output should be able to stay the same, assuming that the function names stay the same.

Something similar to a while loop is already possible using functions (can be seen in the loop function example), but it can get a bit messy with a lot of similarly-named functions and variables. Creating syntax for a while loop could make things easier. Something like this could work:

:var i .= 5
:while :tvar i matches 1..
say Still in loop!
:var i -= 1
:endwhile
say i is 0

With currently implemented databind functions, this would be roughly equivalent to this:

:var i .= 5
:func main_loop
execute if :tvar i matches 1.. :call loop_condition
:endfunc

:func loop_condition
say Still in loop!
:var i -= 1
:endfunc

:call main_loop

Datapacks support tags by creating a /tags folder under a namespace folder. These tags are referenced using a #. For example:

kill @e[type=#namespace:my_tag]

The support for comments means that this like in a .databind file would be compiled to:

kill @e[type=

The tag and anything following it is removed since the compiler thinks it's a comment. Comments should only be possible if the # is at the beginning of a line, not partway through.

The tokenize() function contains many copies of this code:

if self.current_char.is_whitespace() {
    tokens.push(Token::VarName(current_keyword));
    building_keyword = false;
    building_token = Token::None;
    current_keyword = String::new();
    first_whitespace = false;
} else {
    current_keyword.push(self.current_char);
}

Moving this to another function that takes a token as an argument would make the code cleaner. It would also make it much easier to change how tokens are added, since it'd be in one place instead of as many places as tokens.

Any other common copy+pasted code throughout the project could also be moved to a new function in the same file.

When trying to use global macros with the files !macros.databind and main.databind on Linux, the error A non-existant macro test was called was printed. It seems like files with ! at the beginning either aren't being tokenized first, or aren't being recognized as global macros. The files in question are from PR #95 and can be seen here.

PR's to fix this should probably branch off of #95 to make sure that the new test is there.

When running with either the --generate-func-tags flag or generate_func_tags = true in config, even with an empty inclusions list, everything is transpiled properly. However, if either the flag is missing or the config setting is false, all of the files will be put directly in the <project>.databind folder instead of a data subfolder.

I am unsure why this is happening right now.

One main issue to track things that are very broken.

• Fix output folder
  When running databind with no arguments, the output folder is
  created relative to the directory the command was run is as opposed to
  the root of the project.
• Fix problems with tagging
  Tags of folders that are not in subfolders have a preceding slash before
  the function name (eg. namespace:/funcname instead of namespace:funcname)
• Fix while loops breaking
  .databind files seem to stop being transpiled after a while loop, and the while loop
  functions are not generated. I'm not sure what's causing this.

Add support for a TOML configuration file that lets you configure what/how Databind transpiles.

It should look something like this:

databind.toml

random_var_names = false
var_display_names = false
# Whether to auto-generate function JSON in minecraft/tags/functions
generate_func_json = true
# Functions not to generate JSON for
func_json_exclusions = [
    "main",
    "loop1"
]

Any values that aren't lists should also be passable as CLI arguments. An optional --config/-c argument should be added to specify the location of a config file. If the argument is not passed, Databind should look for a config file in the directory it's being run in. Otherwise, use default settings.

Databind should be able to run properly on Windows, macOS, and Linux distributions without problems. Adding tests that properly check the output of files instead of just that they're there would also help ensure that output doesn't vary on different operating systems.

The vars.ini file added in #92 lets developers define variables that can be used across the program in an easily configurable way. However, the .ini filetype isn't meant for this and doesn't support some features that might be useful, like multiline strings or other types. Using an .ini file also means extra dependencies.

Switching over to a .toml file would mean using existing dependencies and would also add support for more types. These types would be converted to strings, but it would mean a better-looking file better suited for something like this.

The ability to compile single files is likely not going to be used much as single .mcfunction files are not very useful on their own. It also means having to document additional features and making sure that new features are supported for single-file compilation.

Documentation mentioning single-file compilation should also be updated.

Currently there is a databind create command that can be used to make a project, but it could be easier to build. The current system that the CLI uses to build is just to pass a path, for example databind ./my_project. There are options for output, configuration, etc., but it would be nice to be able to just run databind and have it build based on the configuration file.

The generation of the configuration file should be able to use the default values for the Settings struct, but the output directory should be specified instead of left blank. Building by passing a path to the CLI should still be allowed, but it should also look for a databind.toml file in the target directory. Running databind should work no matter the current directory of the command line. For example, it should work whether you're in the project's root, in the data folder, or in a functions folder.

The folder structure from databind create could also use some improvement. It should look more like this:

project_name
│   databind.toml
└───src
    │   pack.mcmeta
    └───data
        └───project_name
            └───functions
                    main.databind

This way, the generated folder will also contain the output folder instead of generating a new one.

• Make databind create generate a databind.toml with default settings
• Make databind create generate the aforementioned file structure
• Replace <project_name>.databind as the default output folder with out or something similar
• Allow databind alone to be run in a project and detect the config file
  • Find databind.toml file no matter the current directory in the project

Databind is currently missing integration tests for while loops and replacement definitions. Adding integration tests for these will make sure that no code accidentally breaks a feature and goes unnoticed for some time.

• While tests
• Replacement tests

Maybe a bit ambitious, but the ability to add functionality to Databind with extensions could make it applicable to more projects. If this is going to be done, it should probably be done when the language's syntax has become more stable.
I was thinking that using PyO3 to let people add functionality with Python would be easiest for most developers. This would also mean creating a new Rust library, adding a new codebase and more to maintain. This is likely not a good idea to do right now, since bugs are still being ironed out in the main project. But once most bugs are fixed, this could be a nice addition.

Since the tokenizer will find any keyword no matter the context and try to tokenize it, it's impossible to use the keywords anywhere else, even in tellraw or say commands. Adding an escape character that escapes an entire keyword would help solve this problem, like this:

call func1
say ^call func1

which would become:

function namespace:func1
say call func1

If a developer wanted to use the escape character at the beginning of a word, they could just put two of them:

say ^^example

which would become:

say ^example

The transpile function currently returns an enum called TranspileReturn. With single-file compilation being removed, this is no longer necessary. It should only return the equivalent of what is currently called MultiFileAndMap.

I accidentally merged #84 without properly testing the code in-game, and just found out that they don't actually work for a couple of reasons.

I'll try to fix this as soon as I get the chance.

If you want to perform a scoreboard operation on two databind variables, you'd currently have to do it like this:

scoreboard players operation --databind var1 = --databind var2

It would be easier to have a keyword that becomes --databind var1 or --databind var2. The :tvar keyword almost does this, but it also has score at the beginning (eg. :tvar var1 -> score --databind var1). Adding a new keyword like :gvar to get just what you'd need to perform an operation would be a good idea. That way, the above could be written as such:

scoreboard players operation :gvar var1 = :gvar var2

Adding another keyword like :varop or something that becomes scoreboard players operation would shorten the above even further to the following:

:varop :gvar var1 = :gvar var2

The transpile function returns the transpiled contents of the .databind file, even though Databind files are now only meant to contain function definitions. The main function throws away the contents of the default file, so it doesn't need to be returned.

After #54 is closed, it would be worth considering removing the : prefix from all keywords. It was initially added to make tokenization easier, but the rewritten tokenizer won't need the prefix to identify keywords. Removing the colon would also make Databind look more like a regular language instead of something foreign. Documentation would also need to be updated.

The config file's output is relative to where databind was run as opposed to where the config file is. This can cause problems when trying to test the config file's output. Output should be relative to the config file regardless of whether the config file has an output folder specified.

Shorthand to delete objectives could be useful in functions that "uninstall" a datapack and could benifit from a shorter way to remove objectives they've created. Adding one or both of the keywords listed in the issue title (or something similarly named) could help to solve this.
These keywords would both do the same thing, so it would be up to preference which one to use. In practice, it would look something like this:

:delvar var_or_objective1
:delobj var_or_objective2

compiles to:

scoreboard objectives remove var_or_objective1
scoreboard objectives remove var_or_objective2

An if/else statement:

Example:

func condition
tag load
    var i := 10
    if tvar i matches ..9
        say var i is less than 9
        var i = 10
        say switched var i to 10!
    else
        say var i is 10 or above
        var i = 10
        say switched var i to 10!
    end
end

This would be compiled similar to a while loop, creating functions to run the if, or else.

Compiled:

function example:condition

execute if score example matches ..9 run function example:if_abcd
execute unless score exaple matches ..9 run function example:else_abcd

function example:if_abcd:

say var i is less than 9
scoreboard players set example i 10
say switched var i to 10!

function example:unless_abcd:

say var i is 10 or above
scoreboard players set example i 10
say switched var i to 10!

For some reason, having two or more !def lines causes all but the first not to work. The !def lines are excluded from the output, but the text is not replaced. For example, this code:

!def t1 test1
!def t2 test2

func main
    say t1
    say t2
end

Compiles to:

say test1
say t2

A keyword similar to :call for predicates and tags would be a good addition since they use a similar syntax of namespace:thing, such as:

execute if predicate namespace:predicate
execute as @a[predicate=namespace:prediate]

or:

kill @e[type=#namespace:tag]

Support for tags or the second form of predicate might be harder due to the way that Databind tokenizes by spaces, but predicates should be possible just by copy+pasting and tweaking the code for :call.

Functions tagged in Databind using the tag keyword are added to a HashMap that's turned into tag files at the end of compilation. The problem with this is that any function tags that already contained things will be overwritten. Since the project already uses serde and serde_json, it wouldn't be hard to check if each tag .json file already exists and, if it does, add it to the TagFile struct created at the end.

I've marked this issue as a bug because Databind is meant to be able to be implemented alongside normal mcfunctions, or to be gradually integrated into a project.

This settings hasn't been updated for almost every required feature of Databind. It's likely that someone might try to use it, see that it doesn't work, and thing that either the entire project doesn't work or that the feature is broken (and they'd be right). Removing it altogether instead of trying to update it would probably be easiest at this point.

When running databind with no args in a project, the output folder is generated relative to the directory the command was run in, as opposed to the project root.
I'll add more information if I find anything that might help.

Minecraft's scoreboard players operation command has more functionality than the current assignment operators that Databind has. Changing these assignment operators to use the scoreboard operations would make Databind variables more useful and improve the objective shorthand.

Due to the documentation's use of sphinx.ext.autosectionlabel, some labels like example or compiled are reused many times, creating warnings. While these warnings won't actually affect anything, they could detract from actual warnings or errors if they fill the output with warnings.

Anywhere with duplicate labels should be updated to have unique ones related to the docs page (eg. example to simple-function-example in simple_function.rst).

The ability to define multiline replacements with :def would make it a lot more useful. For example, if you want to make a file that lets people configure your pack by defining functions, having a definition that inserts a few lines to set up the function. Something like this:

:mdef MAKE_VARS
:var var1 .= 0
:var var2 .= 1
:enddef

Replacement should still work as single-line replacements do, replacing any occurrence of the replacement name (in this case MAKE_VARS).

Right now, the tokenize() function will look for a colon preceded by a space before attemting to turn something into a keyword, and anything else is put into a token called NonDatabind. There are some problems with this system. In the way it's implemented, it's impossible for a token to start with another (eg. you couldn't have :def and :deffunc, since it would find def and stop looking for new keywords). It's also impossible to make a token that doesn't begin with :, which is pretty unusual for any language.

If the tokenizer was rewritten to just treat anything separated by whitespace as a separate token, then there would be a lot more freedom as to what tokens could be named, etc. Any space-separated tokens that weren't recognized by Databind would just be assumed to be normal Minecraft commands and kept as-is. However, this is not an issue for this PR. The following:

func test
say hi
endfunc

Could be tokenized as something like: [Token::DefineFunc, Token::FuncName("test"), Token::NewLine, Token::NonDatabind("say"), Token::NonDatabind("hi"), Token::NewLine, Token::EndFunc].

It would be nice to be able to start a new Databind project just from the CLI instead of having to set up the files and folders yourself. The generated folders/file should look like this:

project_name
├── data
│   └── project_name
│       └── functions
│           └── main.databind
└── pack.mcmeta

The command should be something like databind create --name <project_name> --description <description> --path <path>. The only required arguments should be name. Description can default to nothing and the path should default to ..

Currently, you are unable to nest macro definitions like you can with if statements or while loops. The following definition...

!def macro_1($name)
    !def macro_2_$name()

    !end # End 1
!end # End 2

...would result in macro_1 being ended prematurely due to the line marked End 1. Since this functionality is undocumented, it should be considered a bug.

If support for nested definitions like this is added, then it the compiler would need to be updated to prioritize parsing of macro definitions before anything else. In the following situation:

!def macro_1($name)
    !def macro_2_$name()
        say Example
    !end
!end

func main
    ?macro_1("test")
    ?macro_2_test()
end

The macro macro_2_test will not have been defined yet.
I'm not sure of the best way to fix this right now.

The text replacement definitions can be useful, but adding Rust-like macros could also be useful. If you want to make a pack that lets users define their own functions to add functionality but you don't want to make them copy+paste a bunch of boilerplate, then you could use a macro to do it for them. Something like this could work:

!macro on_event
!takes $event
!takes $name
!takes $run
func $name
tag tick
obj $name $event
execute as @e[scores={$name=1..}] run $run
endfunc
!end

The above could be used as something like:

?on_event minecraft.used:minecraft.carrot_on_a_stick item_used say Hello!

The macro call would be converted to the following Databind code:

func item_used
tag tick
obj item_used minecraft.used:minecraft.carrot_on_a_stick
execute as @e[scores={item_used=1..}] run say Hello!
endfunc

The syntax is just an idea, so there might be better ways to implement it.
If this issue is closed, !def definitions should be removed.

Support for text replacement in a similar manner to C language #define's would be nice to have for repetitive commands or portions of commands.
For example, if you frequently use a command like execute as @a[scores={example_obj=1..},nbt= etc., then you could define a replacement and reuse that instead. In addition to making long lines shorter, it would also make it easier to change one part of the command. Instead of trying to change it everywhere, making it possible to accidentally miss a line and create a bug, you only have to change one line at the top of your file.

The syntax could look something like this:

:def LONG_EXECUTE execute as @a[scores={custom_item_obj=1..},nbt={SelectedItem:{id:"minecraft:carrot_on_a_stick",tag:{custom_item:1b}}}] at @s

:func custom_item_stuff
LONG_EXECUTE run summon minecraft:lightning_bolt
:endfunc

The code for while loops is already effectively a preprocessor, so that code could be reused or reformatted to include finding text replacement definitions.

Function calls are outputted with forward slashes prepended to them (eg. :call function becomes function namespace:/function). This is most likely a problem with get_subfolder_prefix in main.rs. The line :call function should become function namespace:function, without the prepended slash.

Every test starts with very similar code that looks something like this:

let mut path = tests::resources();
path.push("some_test_directory");

let out = TempDir::new("some_test_directory").expect("Could not create tempdir for test");

tests::run_with_args(
    "cargo",
    &[
        "run",
        "--",
        path.to_str().unwrap(),
        "--ignore-config",
        "--out",
        out.path().to_str().unwrap(),
    ],
    None,
);

Since this code is used in basically every test, it would be a good idea to move the code to a function that returns out and, if it's used (I can't remember if any tests use it or not), path.

The word "transpile" is used both in the source code for function/file names and in the documentation. More people are probably familiar with the word "compile" and would recognize it easier than "transpile." Renaming files, structs, functions, etc. and updating the docs to use "compile" might be a good idea.

• Update code to use "compile"
• Update docs to use "compile"

Right now, Databind will recompile every file in a project when built, even though it's unlikely that every file in a project has changed. In large projects, this can be a significant problem. I'm not certain how other incremental compilers make sure to only recompile changed files, but doing something like creating a .databind folder in a project's root with the SHA256 hashes of files could work.

In an example implementation, the following tree:

src
└──data
    └──namespace
        └──functions
               main.databind

would result in the following .databind folder:

.databind
└──src
    └──data
        └──namespace
            └──functions
                   main.databind.info

Before file's signatures are checked, the file modification date should be checked. There's no reason to check the hash of a file if it hasn't been modified. This could be stored in the same file as the hash, resulting in the following contents for main.databind.info:

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 1626831659

Where e3b0c442... is the file's hash, and 1626831659 is the file's modification time in seconds.

Some datpacks have subfolders under their namespaces to keep code more organized, like this:

mypack
│   pack.mcmeta
└───data
    └───mypack
        └───functions
            │   main.mcfunction
            └───cmds
                    a_command.mcfunction

This could look like this as a Databind project:

mypack
│   databind.toml
└───src
    │   pack.mcmeta
    └───data
        └───mypack
            └───functions
                │   main.databind
                └───cmds
                        commands.databind

This will successfully compile if the subfolders only contain untagged functions, do not contain calls to any other functions, and do not use while loops. However, this is unlikely.

You can already call a function from commands.databind from main.databind using :call cmds/func_name, but you can't call main from commands.databind.

Since this is already a feature of datapacks and Databind is mostly meant to be a superset of datapacks, this will be considered a bug.

• Fix tagging functions in subfolders
• Fix calling functions from a file in a subfolder

Fixing function calls would also fix while loops due to the way they were implemented.

Due to the implementation of functions, there cannot be nested functions even if they would not be nested in the output. This also means that while loops cannot be used in functions defined with :func, since a while loop becomes two functions.

Replacing the in_function boolean in transpile.rs with an integer that counts the current depth could help fix this problem. The current_function variable would also need to be changed to a Vec<String> of function names instead of just the name of the current function.