Git Product home page Git Product logo

aqua's Introduction

Aqua

release npm

Aqua is an open-source language for distributed workflow coordination in p2p networks. Aqua programs are executed on many peers, sequentially or in parallel, forming a single-use coordination network. Applications are turned into hostless workflows over distributed function calls, which enables various levels of decentralization: from handling by a limited set of servers to complete peer-to-peer architecture by connecting user devices directly. Aqua is the core of the Fluence protocol and a framework for internet or private cloud applications.

Usage

The easiest way to use Aqua is through Fluence CLI with aqua command.

Other ways of using Aqua are described in USAGE.md.

Documentation

Comprehensive documentation and usage examples as well as a number of videos can be found in Aqua Book. Aqua Playground demonstrates how to start writing Aqua and integrate it into a TypeScript application. Numerous videos are available at our YouTube channel.

Repository Structure

  • api - Aqua API for JS
  • aqua-run - Aqua API to run functions
  • backend - compilation backend interface
  • compiler - compiler as a pure function made from linker, semantics and backend
  • model - middle-end, internal representation of the code, optimizations and transformations
    • transform - optimizations and transformations, converting model to the result, ready to be rendered
    • test-kit - tests and test helpers for the model and transformations
  • linker - checks dependencies between modules, builds and combines an abstract dependencies tree
  • parser - parser, takes source text and produces a source AST
  • semantics - rules to convert source AST into the model
  • types - data types, arrows, stream types definitions and variance

Support

Please, file an issue if you find a bug. You can also contact us at Discord or Telegram. We will do our best to resolve the issue ASAP.

Contributing

Any interested person is welcome to contribute to the project. Please, make sure you read and follow some basic rules.

License

All software code is copyright (c) Fluence Labs, Inc. under the AGPLv3 license.

aqua's People

Contributors

akim-bow avatar alari avatar boneyard93501 avatar coder11 avatar diemyst avatar fluencebot avatar folex avatar github-actions[bot] avatar gurinderu avatar inversionspaces avatar justprosh avatar mikhail-1e20 avatar nahsi avatar renovate[bot] avatar shamsartem avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aqua's Issues

Logical expressions inside if statement

If would be nice to have logical expressions inside if statement. For example:

func ifElseNumCall(condition1: u32, condition2: u32):
    if condition1 == 1 and condition2 == 2:
        Println.print("it is 1 and  2")
    else:
        Println.print("it is not 1 and 2")

or

func ifElseNumCall(condition1: u32, condition2: u32):
    if condition1 == 1 or condition2 == 2:
        Println.print("it is either 1 or  2")
    else:
        Println.print("it is neither 1 nor 2")

Derive WASM facade modules from Aqua

In the Fluence network, computations are organized into services and then distributed over the network. Services may contain different logic inside, hidden behind a facade module that provides public API. Usually, facade modules are type definitions for requests and responses, public functions, and a sequence of calls to internal modules wrapped with simple business logic, authorization checks, and similar straightforward thing.

Currently, facade modules need to be developed in a different language, probably Rust. For developers who are not feeling comfortable with Rust, it makes a high entry barrier.

Another problem is understanding of every details of the Fluence protocol, including call_service notation, the meaning of tetraplets, deployment configuration of the security perimeter.

If we could make this simple logic in Aquamarine, and compile to Wasm so that a bunch of .aqua files are compiled into bunch of .ts, .air, and .wasm and .json (for blueprints) files, it would greatly simplify the flow and hide a lot of unnecessary complexity inside the compiler.

The easiest way to go is to compile some specially flagged Aqua services to Rust, which then compiles to Wasm using Rust toolchain.

  • Contra: Rust toolchain needs to be installed

Example to consider:

facade MyService:
  use module "sdhfasdf" as Bar
  func foo(x: u32, token: bool [from app.is_authorized]) -> bool:
      Bar.callFunc()
      if x == 4:
         true
      else
         false

Export clause

Aquamarine makes it easy to decompose big functions into small ones. However, that doesn't mean everything needs to be compiled into the compilation target.

Public functions should be marked with a label. Example:

-- Will not be compiled, but can be used:
func foo():
  ...

-- Will be compiled, can be used e.g. from Typescript:
pub bar():
  foo()

Named arguments in service function definitions

Currently, service functions are parsed from the name of a function and its arrow type definition, e.g.:

service Foo:
   bar: u32, u32, u32, u32 -> bool

It would be nice to allow function heads instead to name arguments and make it easier for service users to comprehend, e.g.:

service Foo:
   bar(offset: u32, limit: u32, month: u32, year:  u32) -> bool

It doesn't affect semantics as these argument names are never used indeed.

Built-in option type

Aqua provides array type with [] and stream type with * added in front of a basic type. Option type represents a field where either one value or no value might exist, and can be described with ?.

alias MaybeBool: ?bool

Option types can be used with streams and arrays as a back end, to avoid changes in Wasm/Rust part.

Here's some pseudocode to consider:

func foo(arg: string) -> bool:
  v <- Service.call(arg)
   <- v

func bar(input: bool) -> ?bool:
  -- Will be backed by array in the scope, can be further provided as `[]bool`, `?bool` types.
  variable: ?bool
  if input == true:
    variable <- foo("input is true")
  -- May have one value, or not
  <- variable


func bar2(input: bool) -> bool:
  -- Will be backed by array in the scope, can be further provided as `[]bool`, `?bool` types.
  variable: ?bool
  if input == true:
    variable <- foo("input is true")
  else:
    variable <- foo("input is false")
  -- We can typecheck that variable is always defined
  <- variable!

Improve error messages

Here is an example:

4 func a() -> string:
5     r <- "a"
      ^=======
      Syntax error, expected: OneOfStr(65,List(<-, else, for, if, on, otherwise, par)), InRange(65,A,Z), InRange(70,a,z)
6     <- r
7 

The actual error is that "a" should be a function call or an ability with a function call.
What expected for example:

4 func a() -> string:
5     r <- "a"
              ^===
      Syntax error, expected: `function or service call`
6     <- r
7 

Streams support

Stream is a major feature of Aquamarine.

Stream is an append-only CRDT. It is not a linear log, as different peers may write to a single stream in parallel. Merging is done in a conflict-free manner so that the merging node observes a stream with incrementing versions. Streams may differ on different peers but are consistent within a single peer.

Syntax example:

func foo(xs: []u16):
   -- First, set the type for the stream & check its visibility
   streamVar: *u16

   -- Write to stream several times
   for x <- xs:
      streamVar <- x

   -- Iterate over the stream
   for v <- streamVar:
      Local.print(v)

Things to consider:

  • Backpressure pattern: inside for over a stream, push to that stream, forcing a new iteration. Inside, go from producer to consumer.
  • Pass stream as an argument to functions: can be considered as a read-only array or read-write stream argument
  • Init peer's callbacks can't have streams as output value; not sure about input value.

Add logging to compilation steps

For easier debugging and understanding of what happens during compilation, it would be nice to add logs and the ability to turn them on.

Pass anonymous function as a last argument

Syntax sugar to simplify the composition.

Without sugar:

func shouldBeAnonymous(x: u32) -> bool:
   true

-- Imagine a complex library function, like onKademliaNeighborhood(key, cb)
func foo(xs: []u32, cb: u32 -> bool):
   for x <- xs:
       cb(x)

func actuallyUsed():
   xs <- Discovery.getXs()
   foo(xs, shouldBeAnonymous)

With sugar:


func foo(xs: []u32, cb: u32 -> bool):
   for x <- xs:
       cb(x)

func actuallyUsed():
   xs <- Discovery.getXs()
   foo(xs) do x: -- x is the local parameter, type is derived from the foo definition
      true

Reassignment expression

Proposition: provide an ability to assign values to other names in the scope.

It can help with reusing long value extractors and refactoring.

data Prod:
   value: string

func doSmth(arg: Prod):
    v = arg.value
    fn(v)
    -- the same as:
    fn(arg.value)

Add hops out of a par branch iff it affects the next operations

After #100, for each branch of par, the compiler adds network hops to exit from that branch context to the next context where join behavior may occur.

These hops are necessary only if data from the par branch is used in the code below. The compiler should not add hops when it's not necessary.

Respect `on...via` path in callbacks, `on` sequences

Aquamarine provides a rich syntax and semantics to express topology that needs to be processed accordingly. Consider the following example:

-- Function expressed to the end user
-- cb is a function that needs to be called on init peer, which is accessible via relay r
func foo(cb: bool -> u32):

  -- get to peer x via relay y they're connected to, but first get to init peer's relay r
  on x via y:
     -- call some service locally on x
     flag <- Calc.flag()
     -- return value via cb: get to y, then get to r, then go to init peer, then call a function locally
     cb(flag)

  -- Need to get to r, then w, then z
  on z via w:
     -- Call function locally
     check <- Calc.check()
     -- Get to w, then to r, then to init peer, and call cb with provided value
     v <- cb(check)
     -- Get to r, then to w, then to z, and call function locally
     Calc.notifyLocal(v)

  -- Get to w, then to r, then to init peer, and call function locally
  cb(true)
  -- No topology shifts, just call cb again
  cb(false)

Things get even more complicated when par is used.

Prelude with built-in services

Fluence nodes have a number of built-in services.

Need to provide service types for them and add to all aqua scripts as a prelude, so that built-in services can be used with no need to describe or import them.

Derive service types from blueprints

Service definition is required in Aquamarine in order to call strictly typed host functions on peers.

Currently, a developer needs to duplicate the type definition provided with the FCE-compatible facade module inside the Aqua script, which is cumbersome.

The proposition is to add a syntax sugar / language feature to fetch these type definitions into the scope from the Fluence network. Example:

use blueprint "blueprint-id" as MyService

func foo():
  MyService "serviceId"
  MyService.someResolvedFunction()

The compiler should parse the file, find use blueprint expression, connect to the Fluence network with fluence-sdk, find a peer providing this blueprint, ask it to provide the API, download json, transform it into blueprint-id.aqua file, save to caches, import as MyService.

#35 and some other work needs to be done before that.

Compile to scheduled scripts format

The Fluence node provides a built-in function that allows AIR script scheduling. This script must be prepared properly:

  • It cannot have arguments, return value
  • It has no relay, needs no xor wrapper
  • It might need access to some configuration data, e.g. list of nodes, service id, etc
  • It must be deployed, which involves a separate tool

Aqua compiler currently does not support all of that.

Code with for, and and if doesn't compile

This code does not compile:

service Test("test"):
	getBool: -> bool
	getList: -> []string

func f():
	list <- Test.getList()
	for user <- list:
		on "peer" via "relay":
			isOnline <- Test.getBool()
		if isOnline:
			Test.getBool()

The error message is:

8               on "peer" via "relay":
9                       isOnline <- Test.getBool()
10              if isOnline:
   ^==============
   Syntax error, expected: EndOfString(180,213)
11                      Test.getBool()

If you "remove" the for statement the code compiles. If you indent the if statement (putting it's execution onto the "peer") the code compiles.

app_config.json integration/replacement

When a distributed/p2p application is developed, it needs to be deployed. Deployment to the Fluence network can be described as app_config.json. But Aqua language has no bindings to it: it can't import values from the app config to use node ids and service ids, can't generate app_config or any other means of deployment.

We need to integrate deployment configuration into Aqua workflow.

Try-catch syntax

It's convenient to have try-catch syntax, which matches perfectly AIR's xor and %last_error% semantics.

Example:

func foo():
  try:
     -- function body
  catch e:
     -- e is of type LastError, can access its structure and recover
     Local.println(e.errorMessage)

Converts to (xor FUNC_BODY HANDLE_ERROR)

Configurable indentation for generated air

Currently nested statements are indented by 1 space everywhere the air used. That makes reading and debugging the scripts difficult (I always indent everything to 4 spaces before starting to do anything with the script), especially when they are embedded as text in .ts files.

Suggestion: increase the default indentation to 4 spaces or make it configurable via a switch. It will improve the "debuggability" of the air scripts

Model (middle-end) refactoring

Each aqua file, once semantically analyzed, provides a model that later used to generate the target code.

Currently model loses a lot of data that will be needed later:

  • Data types, service types, and all the other data that does not go to air
  • Errors in semantic analysis
  • Compilation logs

Model is not used when it must be (in linking).

Refactoring of the model will help with generating services from Aqua (#37) and linking (like #97, #62).

Missing hops to gracefully exit par branches

Imagine client AA and BB. AA sets user=BB and executes this script. In that case, <- "" will try to send result from BB to AA directly, and will fail if BB is behind a relay.

Aqua

func test(user: string, relay_id: string) -> string:
    on relay_id:
        Op.identity()
    on user:
        Op.identity()
    par Op.identity()
    -- BOOM
    <- ""

Generated AIR

(xor
 (seq
  (seq
   (seq
    (seq
     (call %init_peer_id% ("getDataSrv" "relay") [] relay)
     (call %init_peer_id% ("getDataSrv" "user") [] user)
    )
    (call %init_peer_id% ("getDataSrv" "relay_id") [] relay_id)
   )
   (seq
    (seq
     (seq
      (call relay ("op" "identity") [])
      (call relay_id ("op" "identity") [])
     )
     (call relay ("op" "identity") [])
    )
    (par
     (seq
      (call relay ("op" "identity") [])
      (call user ("op" "identity") [])
     )
     (call %init_peer_id% ("op" "identity") [])
    )
   )
  )
  (call %init_peer_id% ("callbackSrv" "response") [""])
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

AIR that reproduces same behaviour, but is more readable

(xor
    (seq
        (call relay ("op" "identity") [])
        (seq
            (par
                (seq
                    (call relay ("op" "identity") [])
                    (call user ("returnService" "run") ["hi, user!"])
                )
                (call %init_peer_id% ("returnService" "run") ["calm before the storm"])
            )
            ;; BOOM
            (call %init_peer_id% ("returnService" "run") ["DONE"])
        )
    )
    (call %init_peer_id% ("returnService" "run") [%last_error%])
)

Ability passing

As of today, Aquamarine is made in terms of arrows, which are built from data types.

Abilities are provided as a kind of FFI for Aqua: service ability facades the functions of a service which can be called on a peer with (call AIR instruction. In order to be used, ability must be resolved in the scope of the current peer (within current on scope).

The next step of the language evolution is to pass abilities resolution along with the arrow arguments and results. Example:


-- Cannot be public, needs Ab to be provided into scope
func {Ab}fooNeedsAb():
   -- Compiles using the Ab from the scope
   Ab.call()

func callAbOnServicesWithAb():
   abPeers <- Discovery.resolveAb()
   for a <- abPeers:
      -- Requires #32 sugar
      on Ab a:
          -- Peer id and service id are passed into the function
          fooNeedsAb()

Syntax additions:


-- Capture from the call scope
type ArrowNeedsFooBar: {Foo, Bar} u32, bool -> ()

-- Brings Foo, Bar into current scope
type ArrowProvidesFooBar: -> {Foo, Bar} ()

type Reprovide: {Foo} -> {Bar} ()

func {Foo}foo():
  ...

func foo() -> {Bar} u32:
   ...

func foo(arr: {Foo} -> ()):
   ...

Add constants

Example:

const PEER = "SOME_PEER_ID"

func example() -> string:
    Peer PEER
    res <- Peer.call()
    <- res

Why:
To have a possibility to use predefined constants in Aquamarine in functions and services. Because right now you need to forward variables from TypeScript manually to Aqua to use them

Add build config file

Passing everything through CLI options is not very reusable.

Need to provide a way to give the compiler a configuration file, and use CLI options just to override it.

Incorrect AIR if argument is called 'relay'

Aqua

func test(relay: string):
    on relay:
        Op.identity()

Generated AIR

(xor
 (seq
  (seq
   (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   (call %init_peer_id% ("getDataSrv" "relay") [] relay) ;; BOOM: duplicated variable
  )
  (seq
   (call relay ("op" "identity") [])
   (call relay ("op" "identity") [])
  )
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

Variable is undefined, but it should be

Aqua file content

service Peer("peer"):
	is_connected: string -> bool
	
service Test("test"):
	getAllUsers: -> []User

data User:
	peer_id: string
	relay_id: string

service ServiceOnClient("service"):
	call: -> ()

func initAfterJoin():
	allUsers <- Test.getAllUsers()
	for user <- allUsers par:
		on user.relay_id:
			isOnline <- Peer.is_connected(user.peer_id)
		if isOnline:
			on user.peer_id via user.relay_id:
				ServiceOnClient.call()
		ServiceOnClient.call()

Error message is:

/mnt/c/wsl/aqua-errors/./aqua/error1.aqua:20:24
19              if isOnline:
20                      on user.peer_id via user.relay_id:
                          ^^^^===========
                          Undefined name, available: isOnline
21                              ServiceOnClient.call()

Cannot make a function that returns a literal

Aqua:

    <- "Hello, World!"

Error:

Unknown model: EmptyModel(compiler state monoid empty |+| Root contains not a script model, it's EmptyModel(Function body is not a funcOp, it's EmptyModel(Return makes no model)))

Elm code generator

Aquamarine is compiled to AIR by default and can be wrapped with Typescript, using Fluence SDK. In this case, it is convenient to use from the Typescript project with all the types, promises. Typescript code can be used in Elm projects as well, but it requires a lot of boilerplate code for ports.

We have all the needed information to generate the right JS and Elm code on the Aqua compile stage. Generating Elm code will make Elm developers much more productive with Aqua.

Improve import behavior

Basically imports create a graph of dependencies between source files. Currently if you have a cycle in that graph the compiler will hang and crash with out of memory.

Quick fix: display an error message if a cyclic dependency has been found

Better fix: parse import statements, put them into a graph, do a topological sort on that graph, and include files in that order.

Add `module ... exports ...` header expression

Every Aqua file introduces various types, functions, and might import from other files.

Currently, everything is being combined and accessible from the resulting file.

a.aqua:

type Smth = u32

func smth(v: Smth, cb: Smth -> bool) -> bool:
  res <- cb(v)
  <- res

b.aqua:

import "a.aqua"

func foo(x: Smth, y: Smth) -> Smth:
  -- do smth with x, y

When b.aqua is compiled, it emits both foo from b.aqua, and smth from a.aqua.

The proposition is to add an exports expression to enumerate what functions and types are to be exported from this file. At the end of the file, remove everything from the compiler state (or the model?) except the exports.

Improve parse errors

Currently, the parser often claims that end of the file is expected instead of pointing to the exact problem in the code.

improve error messages

Error messages are not always optimal.

service Local("returnService"):
  run: string -> ()                      <-- this is the problem
service Echo("service-id"):
    echo: []string -> []string
service Greeting("service-id"):
    greeting: string, bool -> string
-- simple "manual" example without fold but multiple nodes and services
func manual_seq_example(echoList: []string, greeter_on: bool, greeter_off: bool, node_1: string, node_2: string, node_3: string, e_service: string, gr_service_2: string, gr_service_3: string):
  on node_1:
    Echo e_service
    echo_res <- Echo.echo(echoList)
   Local.run(echo_res)                    <-- because echo_res is an array
  on node_2:
    Greeting gr_service_2
    res2 <- Greeting.greeting(echo_res[0], greeter_on)
  Local.run(res1)
  on node_3:
    Greeting gr_service_3
    res3 <- Greeting.greeting(echo_res[2], greeter_off)
  Local.run(res3)

which resulted in:

npm run compile-aqua:air
> [email protected] compile-aqua:air /Users/bebo/localdev/fluence-examples/aqua-echo-greeter
> aqua-cli -i ./aqua/ -o ./compiled-air -a
java -jar /Users/bebo/localdev/fluence-examples/aqua-echo-greeter/node_modules/@fluencelabs/aqua-cli/aqua-cli.jar -m node_modules -i ./aqua/ -o ./compiled-air -a
21     Echo e_service
22     echo_res <- Echo.echo(echoList)
23    Local.run(echo_res)
   ^=====================
   Syntax error, expected: EndOfString(639,2046)
24   on node_2:
25     Greeting gr_service_2
20     Echo e_service
21     echo_res <- Echo.echo(echoList)
22    Local.run(echo_res)
   ^=====================
   Syntax error, expected: EndOfString(502,754)
23
24   on node_2:
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] compile-aqua:air: `aqua-cli -i ./aqua/ -o ./compiled-air -a`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] compile-aqua:air script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/bebo/.npm/_logs/2021-04-26T18_02_10_559Z-debug.log

which wasn't all that diagnostic. once i removed some code, i got the pertinent error message:

/Users/bebo/localdev/fluence-examples/aqua-echo-greeter/./aqua/echo_greeter_examples.aqua:24:13
23     echo_res <- Echo.echo(echoList)
24   Local.run(echo_res)
               ^^^^^^^^=
               Types mismatch, expected: Scalar(string), given: []Scalar(string)
25   on node_2:

Incorrect air for if statement without else branch

Example aqua:

import "builtin.aqua"

func ifWithNoElse(condition: u32):
    if condition == 1:
        Peer.identify()

Generates actual output:

(xor
 (seq
  (seq
   (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   (call %init_peer_id% ("getDataSrv" "condition") [] condition)
  )
  (match condition 1
   (call %init_peer_id% ("peer" "identify") [])
  )
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

Expected output (match wrapped with the xor statement):

(xor
 (seq
  (seq
   (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   (call %init_peer_id% ("getDataSrv" "condition") [] condition)
  )
  (xor
   (match condition 1
    (call %init_peer_id% ("peer" "identify") [])
   )
   (null)
  )
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

Allow `[]string` in `via` clause, compile to `fold`

Having a peer behind a relay, or not behind a relay, is a common pattern in Fluence. To handle it, we need to be able to write the same code for both situations. One way to have it done is to allow passing []string to via part of on expression and compile it with fold.

`on Ability x:` sugar

Abilities are made to tell the compiler how to call certain functions in the context of a peer, mainly reflecting external service function types.

on clause currently provides the scope of a peer's ID to get to the right peer and call functions local to it.

Knowing what to call where is a very common pattern that would be nice to have as syntax sugar. Example:

data Service:
  peer_id: string
  service_id: string

data App:
  foo: Service

service Foo:
  bar: -> ()

func fooBar(app: App):
  -- desugared:
  on app.foo.peer_id:
     Foo app.foo.service_id
     Foo.bar()
  
  -- sugar:
  on Foo app.foo:
     Foo.bar()

This syntax is intended to correlate with appConfig.json, supported by the Typescript fluence-sdk. Eventually, it can be brought into Aquamarine to reduce the number of things to learn.

Import files

Aquamarine language is designed for reusable composition & topology. Hard to imagine reusability in copy-paste fashion, when there's no way to share a script file to be independently worked on.

Imports are the first step to enable script reusability.

Example:

-- Import textually into the scope
import "Smth.aqua"

-- Import as an ability. Requires changes in the type definitions
use "Smth.aqua" as Smth

-- Expose just the named functions and types
use (funcA, funcB, DataC) from "test/Smth.aqua"

Imports should be defined in the file header, parsed via the top-level ScriptExpr, resolved before semantic evaluation is started.

Incorrect variable naming when a variable with the same name is used in two interconnected functions

Aqua code:

service Demo("demo"):
	get42: -> u64

func one() -> u64:
    variable <- Demo.get42()
    <- variable

func two() -> u64:
    variable <- one()
    <- variable

actual generated air for the function two

(xor
 (seq
  (seq
   (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   (call %init_peer_id% ("demo" "get42") [] variable0)
  )
  (call %init_peer_id% ("callbackSrv" "response") [variable])
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

expected

(xor
 (seq
  (seq
   (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   (call %init_peer_id% ("demo" "get42") [] variable0)
  )
  (call %init_peer_id% ("callbackSrv" "response") [variable0])
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

variable0 instead of variable or the vice versa.

Expose relay and %init_peer_id% as variables

relay and %init_peer_id% are very commonly used variables in air. Both of the variables are available already (relay` is being injected for the every generated RequestFlow) so it won't change the current contract.

As a workaround it is possible to inject these variables with a custom handler which is very tedious.

Non-destructive collection operations

Aquamarine interpreter & protocol provides decent security guarantees, including data consistency: if data is provided by a peer as a result of a function call, then it's checked that this peer got the right arguments, and the result is not modified by any other party.

This puts limitations on what is possible to do in AIR and therefore in Aquamarine. The key issue is working with streams: almost any function called on a stream will consume it, meaning that the origin of the result is missed. Consider the following example:

func has41() -> bool:
  $stream: u32
  for x <- xs par:
     on x.peer_id:
        -- add some numbers to the stream
        $stream <- LocalRandom.rand()
  t <- CollectionService.contains($stream, 41)
  <- t

This function walks around a part of the network, providing random values into the stream. When the random value is provided, computations are moved back to the initial peer scope. What we want to check is whether the stream contains the number 41, but how to get it?

And once we got 41, we want to return true. If the whole stream has no 41 in it, we return false.

It seems that this semantic is highly desired (e.g. it's needed to implement iterative routing algorithms, or "get first X values matching the criteria"), but can't be expressed in terms of service calls.

The proposition is to add to the language, as well as the interpreter, a set of collection operations that doesn't destroy the initial value but derives the result from it.

Let A, B be collections (arrays or streams).

  • Get data from a variable, derive its type
  • Check if the variable has a certain feature (map to bool)
  • Convert a tuple to something, e.g. bool (check for inclusion)

Possible functions:

  • union: union(A, B)
  • intersection: inter(A, B)
  • subtraction: sub(A, B)
  • partition: part(A, pred)
  • product (?): prod(A, B)

Element-wise:

  • is_empty(A)

  • ord:: a, a -> -1, 0, 1

  • inclusion: contains(A, a)

  • addition with respect to order: add(A, a, ord)

  • order: order(A, ord)

  • reducer: e.g. size?

  • take: lim(A, int) (can be done with partition, if has access to index)

  • Intersection, is_empty, inclusion, difference -- especially important for streams

  • Order a stream, e.g. by reducing into a new stream (re canonicalize)

The return "callback" doesn't respect the relay

Aqua code

func join(user: User) -> EmptyServiceResult:
	app <- AppConfig.getApp()
	on app.user_list.peer_id:
		UserList app.user_list.service_id
		res <- UserList.join(user)
	<- res

Generated air actual

(xor
 (seq
  (seq
   (seq
    (call %init_peer_id% ("getDataSrv" "user") [] user)
    (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   )
   (seq
    (call %init_peer_id% ("fluence/get-config" "getApp") [] app)
    (seq
     (call relay ("op" "identity") [])
     (call app.$.user_list.peer_id! (app.$.user_list.service_id! "join") [user] res)
    )
   )
  )
  (call %init_peer_id% ("callbackSrv" "response") [res])
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

Generated air expected

(xor
 (seq
  (seq
   (seq
    (call %init_peer_id% ("getDataSrv" "user") [] user)
    (call %init_peer_id% ("getDataSrv" "relay") [] relay)
   )
   (seq
    (call %init_peer_id% ("fluence/get-config" "getApp") [] app)
    (seq
     (call relay ("op" "identity") [])
     (call app.$.user_list.peer_id! (app.$.user_list.service_id! "join") [user] res)
    )
   )
  )
  (seq
      (call relay ("op" "identity") [])
      (call %init_peer_id% ("callbackSrv" "response") [res])
   )
 )
 (call %init_peer_id% ("errorHandlingSrv" "error") [%last_error%])
)

The callback is expected to go through the relay.

Commented string in the end of a file breakes compilation

Example:

service Testo("testo"):
    getString: string -> string

service LocalPrint("lp"):
    print: string, string -> ()


func initAfterJoin(me: string, myRelay: string, friend: string, friendRelay: string) -> string:
    on friend via friendRelay:
        str1 <- Testo.getString("friends string")
    on friend via friendRelay:
        str2 <- Testo.getString("friends string via")
    par LocalPrint.print(str1, str2)
    <- "finish"

-- alala

Expected this to be compiled

Don't print an error if an aqua file has no functions but used on another aqua file

println.aqua

service Println("println"):
    print: string -> ()

main.aqua

import "println.aqua"

func callArrowFunc(a: string -> string):
    res <- a("hello")
    Println.print(res)

logs:

ITERATE, can handle: List(FileModuleId(/home/diemust/git/minimal-aqua/src/aqua/callArrow.aqua))
proc = Set(FileModuleId(/home/diemust/git/minimal-aqua/src/aqua/on.aqua), FileModuleId(/home/diemust/git/minimal-aqua/src/aqua/println.aqua))
FileModuleId(/home/diemust/git/minimal-aqua/src/aqua/callArrow.aqua) dependsOn Set(FileModuleId(/home/diemust/git/minimal-aqua/src/aqua/println.aqua))
COMBINING ONE TIME 
call combine cats.data.IndexedStateT@7a4db5d5
MONOID COMBINE EmptyModel(compiler state monoid empty) EmptyModel(compiler state monoid empty |+| Root contains not a script model, it's EmptyModel(Service with ID defined))
Unknown model: EmptyModel(compiler state monoid empty |+| Root contains not a script model, it's EmptyModel(Service with ID defined))

What expected:
logs without this error

Unknown model: EmptyModel(compiler state monoid empty |+| Root contains not a script model, it's EmptyModel(Service with ID defined))

a call of a function that returns void will throw a timeout

Aqua:

func callArrowFunc(a: string -> string):
    res <- a("hello")

typescript:

await callArrowFunc(client, (a: string) => {
        return "Hello, " + a + "!"
    })

if we call this in typescript with await it timeouts.
Solution:
generate a function that returns void without promise

Compile the Aquamarine compiler into JS with ScalaJS

A big part of the Aquamarine ecosystem is developed with Javascript and Wasm, which is barely usable from Scala directly.

We need to compile aqua-c into a Javascript package. It will enable:

  • Seamless webpack integration
  • Compilation in browser
  • Using the interpreter and fluence-sdk in the compilation flow, different tooling like fldist, proto-distributor

Integration with existing JS/TS tooling is important as it makes it possible to hide this tooling from the developer, provide lean and concise interface within the language itself.

`par` does not respect variables visibility

Currently, you can do smth like:

on x:
    y <- z()
par foo(y)

This means that foo is executed with y argument in parallel with fetching y from z. But this is not possible.

Names interpreter should create a new Frame for each branch of par to forbid such behavior.

Failover capability with for ...try

it would be useful to be able to manage failover for nodes and services with a for...try statement. inputs would be a list of deployed node-id, service-id tuples where services (for now) are not stateful. the for ... try loop would iterate through the list and break when node and service were reached. Stylized example:

service Local("returnService"):
  run: string -> ()

service Greeting("service-id"):
    greeting: string, bool -> string

func greeting(name: string, greeter: bool, target_tuple: []string) -> string:
  err_msg: string
  success: bool = false
  for (node, srvc) <- target_tuple:
    try:
      on node:
        Greeting srvc
        res <- Greeting.greeting(name, greeter)
        Local.run(res)
        success = true
    catch:
        err_msg <- res
    
  if success:
      <- res
  <- err_msg

Cannot compare two constants with different number types in `if` statement

Example:

service Demo("demo"):
	get4: u64 -> u64

const bbb = 5
const ttt = -2

func two(variable: u64):
    if bbb == ttt:
        Demo.get4(variable)
    else:
        Demo.get4(variable)

Compilation error:

7 func two(variable: u64):
8     if bbb == ttt:
                ^^^=
                Types mismatch, expected: Literal(number), given: Literal(signed)
9         Demo.get4(variable)

But it should be possible to compare two numbers if there is no explicitly specified types

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.