moonlibs / tarantoolapp Goto Github PK

View Code? Open in Web Editor NEW

32.0 6.0 1.0 111 KB

App starter & dependency manager for Tarantool application server

License: MIT License

Lua 92.03% Makefile 1.85% Shell 1.49% CMake 3.93% C 0.70%

tarantool luarocks rockspec-files deps appstarter

tarantoolapp's Introduction

tarantoolapp

App starter for Tarantool application server

Installation

tarantoolapp is a cli tool so it is better to install it globally to system:

$ luarocks install tarantoolapp

Or to install most recent version

$ luarocks install --server=http://luarocks.org/dev tarantoolapp

In order for this to succeed it is required to have both Tarantool and Luarocks repos in ~/.luarocks/config.yaml:

➜ cat ~/.luarocks/config.lua
rocks_servers = {[[http://rocks.tarantool.org]], [[https://luarocks.org]]}

Bootstrap an application

To create a project template for application with name myapp in current directoty:

$ tarantoolapp create myapp

Full command is:

$ tarantoolapp create [-t <template>] [-p <path>]
       [--description <description>] [--version <version>] [-h] <name>

Parameters to `tarantoolapp create`:

<name> - Desired project name
<template> - template to use. Available templates: (basic, luakit, ckit) (default: basic)
<path> - path to directory where to setup project (default is ./{your_project_name})
<description> - Project description (default: Tarantool App)
<version> - Project version (default: scm-1)

There can be special options defined for a selected template, for example for a basic template:

<basic_use_spacer> - Use spacer or not.

These options are defined in a templates/<template_name>/config.yaml file and in command line are prefixed with a template name.

Install dependencies

$ tarantoolapp dep

This command installs dependencies, specified in the meta.yaml file in current folder (luarocks and Lua 5.1 are required).

Full command is:

$ tarantoolapp dep [-m <meta_file>] [-t <tree>] [-h]
       [--only [<only>] ...]

Parameters to `tarantoolapp dep`:

<meta_file> - path to meta.yaml file (default: ./meta.yaml)
<tree> - path to directory that will hold the dependencies (default: .rocks)
<only> - install only these sections (deps, tntdeps or localdeps). Separated with spaces (e.g. --only deps tntdeps)

meta.yaml

meta.yaml can have the following sections:

name - package name
version - package version
deps - list of paths to rockspec files or package names (each is installed using luarocks install command)
tntdeps - list of paths to rockspec files or package names (each is installed using tarantoolctl rocks install command)
localdeps - list of paths to local rockspec files (each is installed using luarocks make command). You can specify dependency in either of 2 following formats:
- ./local/path/to/package/package.rockspec if *.rockspec file is in the package root
- ./local/path/to/package/rockspecs/package.rockspec:./local/path/to/package - specify the package root after the colon

Getting help

$ tarantoolapp -h

$ tarantoolapp create -h

$ tarantoolapp dep -h

tarantoolapp's People

Contributors

Stargazers

Watchers

Forkers

bofm

tarantoolapp's Issues

Update module version from master branch.

Hello.

Could you please create a new tag with current master branch, because in a master branch we have fix for tarantool version 2.3+

Thanks a lot.

libraries are not enough after creating the package rpm

Hello!

I created application with template basic.
Edited meta.yaml with content:

name: reds
version: scm-1
deps:
  - https://raw.githubusercontent.com/moonlibs/config/master/config-scm-1.rockspec
  - https://raw.githubusercontent.com/moonlibs/package-reload/master/package.reload-scm-1.rockspec
  - inspect
tntdeps:
  - metrics
  - http
  - avro-schema
  - expirationd

In rpm/reds.spec for create rpm(first steps) changed string

tarantool dep.lua --meta-file ./meta.yaml --tree ./%{name}-%{version}-%{release}-rocks

tarantoolapp dep --meta-file ./meta.yaml --tree ./%{name}-%{version}-%{release}-rocks

After creating a package, several libraries(rocks) are missing
part output after run (make rpm)

[reds] Installing dep 'inspect'
[reds] luarocks --server=https://luarocks.org --server=http://rocks.tarantool.org install inspect --tree=/root/rpmbuild/BUILD/reds/reds-scm-20200221.0950-rocks...
Installing https://luarocks.org/inspect-3.1.1-0.src.rock...
Using https://luarocks.org/inspect-3.1.1-0.src.rock... switching to 'build' mode
Updating manifest for /root/rpmbuild/BUILD/reds/reds-scm-20200221.0950-rocks/lib64/luarocks/rocks
inspect 3.1.1-0 is now built and installed in /root/rpmbuild/BUILD/reds/reds-scm-20200221.0950-rocks (license: MIT <http://opensource.org/licenses/MIT>)

[reds] Installed dep 'inspect'


[reds] Installing tarantool dep 'metrics'
[reds] tarantoolctl rocks --server=https://luarocks.org --server=http://rocks.tarantool.org install metrics...
Installing http://rocks.tarantool.org/metrics-scm-1.rockspec
Cloning into 'metrics'...
remote: Enumerating objects: 51, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 51 (delta 1), reused 29 (delta 1), pack-reused 0
Receiving objects: 100% (51/51), 22.64 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1/1), done.
metrics scm-1 is now installed in /root/rpmbuild/BUILD/reds/.rocks (license: BSD)

[reds] Installed tarantool dep 'metrics'

Library installation paths are different

This problem after my edition(into spec or meta.yaml) or not ?

tarantoolapp is broken with the last commit

d45cbc1

tarantoolapp create app1 --basic_use_spacer=y
cd app1

nano app/init.lua

...

function app.init(config)
    log.info('app "app1" init')

    box.spacer = require 'spacer'({
        migrations = config.migrations,
        automigrate = true,
        keep_obsolete_spaces = true,
    })
    require 'schema'

    for k, mod in pairs(app) do if type(mod) == 'table' and mod.init ~= nil then mod.init(config) end end
end

...

nano app/schema.lua

local spacer = require 'spacer'

spacer:space({
    name = 'object',
    format = {
        { name = 'id', type = 'unsigned' },
    },
    indexes = {
        { name = 'primary', type = 'tree', unique = true, parts = { 'id' } }
    }
})

make dep
make run

logs

FG=1 CONF=./conf.lua tarantoolctl start tt.lua
Starting instance tt...
1st load. loaded: fiber, ffi, key_def, io, console, swim, digest, json, uri, decimal, crypto, net.box, internal.argparse, log, jit.opt, uuid, fio, pwd, internal.trigger, jit.p, jit.vmdef, os, string, debug, jit.profile, socket, box.internal.sequence, tap, coroutine, strict, utf8, pickle, msgpack, jit.dis_x86, box.backup, jit, jit.v, buffer, box, yaml, xlog, errno, bit, box.internal, table.clear, help, jit.zone, table, msgpackffi, serpent, net.box.lib, jit.bc, csv, jit.dump, jit.util, help.en_US, jit.bcsave, box.internal.session, error, package, fun, table.new, math, title, merger, _G, box.internal.space, tarantool, jit.dis_x64, http.client, clock, iconv
config	table: 0x010ccfb558	loading config 	./conf.lua
Run console at unix/:./.tnt/tt.control
started
mkdir ./.tnt/tt
started logging into a pipe, SIGHUP log rotation disabled
2020-07-12 10:41:39.706 [8324] main/102/tt C> Tarantool 2.3.2-1-g9be641b
2020-07-12 10:41:39.706 [8324] main/102/tt C> log level 5
2020-07-12 10:41:39.706 [8324] main/102/tt I> mapping 117440512 bytes for memtx tuple arena...
2020-07-12 10:41:39.707 [8324] main/102/tt I> mapping 134217728 bytes for vinyl tuple arena...
2020-07-12 10:41:39.708 [8324] main/102/tt I> instance uuid e3f583c0-982b-4a2f-83cc-ef04b9950552
2020-07-12 10:41:39.709 [8324] iproto/101/main I> binary: bound to 127.0.0.1:3301
2020-07-12 10:41:39.709 [8324] main/102/tt I> initializing an empty data directory
2020-07-12 10:41:39.718 [8324] main/102/tt I> assigned id 1 to replica e3f583c0-982b-4a2f-83cc-ef04b9950552
2020-07-12 10:41:39.718 [8324] main/102/tt I> cluster uuid 318e5a25-a650-485f-9a38-3dff5b9c580c
2020-07-12 10:41:39.718 [8324] snapshot/101/main I> saving snapshot `./.tnt/tt/00000000000000000000.snap.inprogress'
2020-07-12 10:41:39.720 [8324] snapshot/101/main I> done
2020-07-12 10:41:39.720 [8324] main/102/tt I> ready to accept requests
2020-07-12 10:41:39.720 [8324] main/106/checkpoint_daemon I> scheduled next checkpoint for Sun Jul 12 12:34:27 2020
2020-07-12 10:41:39.720 [8324] main/102/tt I> set 'memtx_memory' configuration option to 104857600
2020-07-12 10:41:39.720 [8324] main/102/tt I> set 'listen' configuration option to "127.0.0.1:3301"
2020-07-12 10:41:39.720 [8324] main/103/lua reload.lua:160 E> Cleanup 21, running
2020-07-12 10:41:39.722 [8324] main/102/tt I> app "tt" init
/Users/username/tt/app/init.lua:18: attempt to call a table value
stack traceback:
	./tt.lua:26: in function <./tt.lua:25>
	/Users/username/tt/app/init.lua:18: in function </Users/username/tt/app/init.lua:15>
	[C]: in function 'xpcall'
	./tt.lua:21: in main chunk
	[builtin#25]: at 0x010a5db74d
	[C]: in function 'pcall'
	/usr/local/bin/tarantoolctl:548: in function 'cmd_function'
	/usr/local/bin/tarantoolctl:1016: in function 'process'
	/usr/local/bin/tarantoolctl:1430: in function 'main'
	/usr/local/bin/tarantoolctl:1441: in main chunk
make: *** [run] Error 1

Refactoring: remove Lua path-related duplicate code in /test.sh and /init.lua

There is the same code piece in Bash repeated in test.sh in basic/ and vshard/ templates. Also there seems to be the same code in Lua in **/init.lua. Those pieces of code are related to Lua path and cpath definition. It would be great if that code was moved to a single place to avoid copy-paste.

[Bug] package.reload() clears all tuples in the second space

Related issue: igorcoding/tarantool-spacer#13
Steps to reproduce

tarantoolapp create example --basic_use_spacer=y
cd example
make dep

nano app/init.lua

box.spacer = require 'spacer'({
        migrations = config.migrations,
        automigrate = true, -- add this
})

nano app/schema.lua

local spacer = require 'spacer'

spacer:space({
    name = 'object',
    format = {
        { name = 'id', type = 'unsigned' },
    },
    indexes = {
        { name = 'primary', type = 'tree', unique = true, parts = { 'id' } }
    }
})

spacer:space({
    name = 'object2',
    format = {
        { name = 'id', type = 'unsigned' },
    },
    indexes = {
        { name = 'primary', type = 'tree', unique = true, parts = { 'id' } }
    }
})

make run

tarantool> box.space.object:insert({1})
---
- [1]
...

tarantool> box.space.object2:insert({2})
---
- [2]
...

tarantool> box.space.object:select()
---
- - [1]
...

tarantool> box.space.object2:select()
---
- - [2]

tarantool> package.reload()
2020-06-18 00:05:16.482 [47222] main/102/example I> 2nd load. Unloading {spacer, schema, app, spacer.version, spacer.transformations, spacer.migration, spacer.util, spacer.stmt, spacer.ops, spacer.inspect, config, inspect, spacer.compat, mod1, spacer.fileio}
2020-06-18 00:05:16.482 [47222] main/102/example I> package.reload:cleanup...
2020-06-18 00:05:16.485 [47222] main/102/example I> app "example" destroy
2020-06-18 00:05:16.485 [47222] main/102/example I> package.reload:cleanup finished
config	table: 0x010cc4a638	loading config 	./conf.lua
2020-06-18 00:05:16.491 [47222] main/102/example I> app "example" init
2020-06-18 00:05:16.507 [47222] main/117/lua reload.lua:160 E> Cleanup 1, running
---
...

tarantool> box.space.object:select()
---
- - [1]
...

tarantool> box.space.object2:select()
---
- []
...

Implement lock files for tarantoolapp dep

require('tarantool').version >= "1.7.3" is false with 1.10

tarantoolapp/templates/basic/template/t/tnt/init.lua

Line 24 in dd75d6b

if require('tarantool').version >= "1.7.3" then

I found this in old deprecated appstarter, but it seems that there's still a bug in new code.

Docker

It would be great to add the following features to the templates:

Add Dockerfile.dev to build an image for development (with tests and debug utils).
Add Dockerfile.prod to build an image for deployment to production (without tests).
Add docker-dev, docker-prod, docker-test, docker-run recipes to the Makefile.
Maybe add cli parameter to set Tarantool version or Docker image tag to be used in the templates.

dep command exit code is 0 even if there is an error

root@71e208ae72fc:/opt/tarantool/zzz# cat meta.yaml
name: zzz
version: scm-1
deps: []
tntdeps:
  - https://raw.githubusercontent.com/zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
root@71e208ae72fc:/opt/tarantool/zzz# tarantoolapp dep && echo $?
Tarantool version: 1.9.0-4-g195d4462d
Using the following options:
---
--tree: .rocks
--meta-file: ./meta.yaml
--only:
--luarocks-config: /root/.luarocks/config.lua
...

[zzz] Already have proper rocks servers
[zzz] Installing dependencies...
[zzz] Installing tarantool dep 'https://raw.githubusercontent.com/zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'
[zzz] tarantoolctl rocks install https://raw.githubusercontent.com/zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz...

Error: No results matching query were found.
/usr/local/share/lua/5.1/tarantoolapp/commands/dep.lua:72: [zzz] tarantoolctl rocks install https://raw.githubusercontent.com/zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz failed
stack traceback:
	...l/lib/luarocks/rocks/tarantoolapp/scm-1/bin/tarantoolapp:79: in function <...l/lib/luarocks/rocks/tarantoolapp/scm-1/bin/tarantoolapp:78>
	[C]: in function 'error'
	/usr/local/share/lua/5.1/tarantoolapp/commands/dep.lua:72: in function 'tarantoolctl_install'
	/usr/local/share/lua/5.1/tarantoolapp/commands/dep.lua:184: in function 'run'
	...l/lib/luarocks/rocks/tarantoolapp/scm-1/bin/tarantoolapp:76: in function <...l/lib/luarocks/rocks/tarantoolapp/scm-1/bin/tarantoolapp:75>
	[C]: in function 'xpcall'
	...l/lib/luarocks/rocks/tarantoolapp/scm-1/bin/tarantoolapp:74: in main chunk
0

Copy of file instead of symlink on project creation

When create a project test with tarantoolapp create test in directory test we have 2 files with same content:

test.lua
init.lua

Expected behavior: test.lua is a symlink to init.lua

Info about system:

System Version: macOS 10.13.5 (17F77)
Kernel Version: Darwin 17.6.0

Required tarantool version

Create command uses tarantool built-in fio.mktree(), which is not accesible via tarantool version < 1.9 (tested with 1.6, 1.7.6, 1.7-centos and 1.8.1)

Actually fio.mktree() is specified in docs on 1.7 (http://www.tarantool.io:8090/ru/doc/1.7/reference/reference_lua/fio/#fio-mktree)
But I am still unable to call it (using official docker image 1.7.6):

/opt/tarantool # tarantool
Tarantool 1.7.6-0-g7b2945d6c
type 'help' for interactive help
tarantool> require('fio').mktree
---
- null
...

Error stacktrace:

/usr/local/share/lua/5.1/tarantoolapp/commands/create.lua:232: attempt to call field 'mktree' (a nil value)
stack traceback:
	...lib/luarocks/rocks/tarantoolapp/1.0.2-1/bin/tarantoolapp:54: in function 'mktree'
	/usr/local/share/lua/5.1/tarantoolapp/commands/create.lua:232: in function 'run'
	...lib/luarocks/rocks/tarantoolapp/1.0.2-1/bin/tarantoolapp:51: in function <...lib/luarocks/rocks/tarantoolapp/1.0.2-1/bin/tarantoolapp:48>
	[C]: in function 'xpcall'
	...lib/luarocks/rocks/tarantoolapp/1.0.2-1/bin/tarantoolapp:47: in main chunk

Suggestion: specify required tarantool version

File 'templates' not found

New day, new problems.

I has installed environment on the ubuntu server with following commands

sudo apt-get install lua5.1
sudo apt-get install luarocks
sudo apt-get install tarantool
sudo luarocks install tarantoolapp

And now every call of tarantoolapp returns next

ubuntu@ip-172-31-93-224:~/app$ tarantoolapp dep
Tarantool version: 1.6.7-591-g7d4dbbb
/usr/local/lib/luarocks/rocks/tarantoolapp/1.0.3-1/bin/tarantoolapp: ....luarocks/share/lua/5.1/tarantoolapp/commands/create.lua:72: file 'templates' not found:
    can't open /home/ubuntu/.luarocks/share/lua/5.1/templates: /home/ubuntu/.luarocks/share/lua/5.1/templates: No such file or directory
    can't open /home/ubuntu/.luarocks/share/lua/5.1/tarantoolapp/commands/templates: /home/ubuntu/.luarocks/share/lua/5.1/tarantoolapp/commands/templates: No such file or directory
    can't open /home/ubuntu/.local/share/templates: /home/ubuntu/.local/share/templates: No such file or directory

We're solve this problem by modifying create.lua, but it still a problem.

This bug shows on the 1.6.x, 1.10.x works fine

Config aware tests in templates

The templates should contain examples of tests which can get parameters from conf.lua and perform app.init(config), preferably in setup-teardown helper functions.

Docker: attempt to concatenate field 'lua_extension' (a nil value)

Thanks for the app!

When i'm trying to run tarantoolapp --help inside of a docker container (tarantool/tarantool:2.x-centos7), it failes with error:

[root@72ef0f4d6703 tarantool]# tarantoolapp
bash: tarantoolapp: command not found
[root@72ef0f4d6703 tarantool]# luarocks install tarantoolapp
Installing http://luarocks.org/repositories/rocks/tarantoolapp-1.0.4-1.src.rock
Missing dependencies for tarantoolapp 1.0.4-1:
   luarocks-fetch-gitrec (not installed)
   lua-resty-template ~> 1.9 (not installed)

tarantoolapp 1.0.4-1 depends on luarocks-fetch-gitrec (not installed)
Installing http://luarocks.org/repositories/rocks/luarocks-fetch-gitrec-0.2-1.src.rock

luarocks-fetch-gitrec 0.2-1 is now installed in /usr/local (license: MIT)

tarantoolapp 1.0.4-1 depends on lua-resty-template ~> 1.9 (not installed)
Installing http://luarocks.org/repositories/rocks/lua-resty-template-1.9-1.src.rock

lua-resty-template 1.9-1 is now installed in /usr/local (license: BSD)

tarantoolapp 1.0.4-1 is now installed in /usr/local (license: MIT)

[root@72ef0f4d6703 tarantool]# tarantoolapp --help
Tarantool version: 2.3.0-117-g4c2d1ef
/usr/local/share/tarantool/luarocks/core/path.lua:47: attempt to concatenate field 'lua_extension' (a nil value)

I'm kinda new to lua and tarantool, so maybe it's obvious what i'm doing wrong :)

...luarocks/rocks-5.3/tarantoolapp/1.0.3-1/bin/tarantoolapp:9: module 'tarantoolapp.argparse' not found

tarantoolapp create myapp
Tarantool version: 1.10.2-1-ge0017ad
...luarocks/rocks-5.3/tarantoolapp/1.0.3-1/bin/tarantoolapp:9: module 'tarantoolapp.argparse' not found:
no field package.preload['tarantoolapp.argparse']
no file './tarantoolapp/argparse.lua'
no file './tarantoolapp/argparse/init.lua'
no file './tarantoolapp/argparse.dylib'
no file './tarantoolapp/argparse.so'
no file '/Users/pavel/dev/lua/.rocks/share/tarantool/tarantoolapp/argparse.lua'
no file '/Users/pavel/dev/lua/.rocks/share/tarantool/tarantoolapp/argparse/init.lua'
no file '/Users/pavel/dev