The underlying JDBI library already supports these.

Hi @andykram,

I tried to migrate (after rebasing it on the master branch), but when I try migrate it as described in the realm, I'm getting this error:

Error: Could not find or load main class db

Then I tried it this way:

java -Ddw.prestoCoordinator=http://localhost:8080 \
 -Ddw.s3AccessKey=KEY \
 -Ddw.s3SecretKey=SECRET \
 -Ddw.s3Bucket=airpal-project \
 -Ddw.metaStoreConfiguration.connectionUrl=jdbc:mysql://localhost:3306/metastore \
 -Ddw.metaStoreConfiguration.password=JUST_A_PASSWORD \
 -Ddw.metaStoreConfiguration.userName=SOME_USER \
 -Duser.timezone=UTC \
 -cp target/airpal-*.jar com.airbnb.airpal.AirpalApplication db migrate reference.yml

But now I get the error:

reference.yml has the following errors:
  * dataSourceFactory.url may not be null (was null)
  * dataSourceFactory.user may not be null (was null)

Do I really need to set these? If so; what should these URL's be? Or am I doing something else wrong?

Cheers

How to set up a mysql table permissions?
Thank you for your answer

The install is failing for me with the following error
Grgit : Unsupported major.minor version 51.0

I am using yosemite and I have java version details as follows

java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-466.1-11M4716)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-466.1, mixed mode)

Hi,

I am working on a machine that has no connectivity to the internet.
I have node and npm installed locally as below -

[root@x01tbipapp3a airpal-master]# which node
/home/mm/node-v0.12.7-linux-x64/bin/node
[root@x01tbipapp3a airpal-master]# which npm
/home/mm/node-v0.12.7-linux-x64/bin/npm
[root@x01tbipapp3a airpal-master]# which gradle
/home/mm/gradle-2.6/bin/gradle

However when I run the ./gradlew clean shadowJar -Dairpal.useLocalNode it still tries to download Gradle from the internet.

[root@x01tbipapp3a airpal-master]# ./gradlew clean shadowJar -Dairpal.useLocalNode
Downloading https://services.gradle.org/distributions/gradle-2.2.1-all.zip

Is there someway to change this.

Thanks,
Manish

I just cloned the repo and tried to build it but I get a weird gulp-notify 'not implemented' error.

∙ ./gradlew -Dairpal.prestoVersion=0.93 clean shadowJar -Dairpal.useLocalNode                                                                                  17:30  marco@mymachine
:cleanAssets UP-TO-DATE
:clean
:nodeSetup SKIPPED
:installAssets
:buildAssets
[17:31:13] Using gulpfile ~/playground/airpal/src/main/resources/assets/gulpfile.js
[17:31:13] Starting 'browserify'...
[17:31:13] Bundling app.js...
[17:31:13] Bundling plugin.js...
[17:31:13] gulp-notify: [Compile Error] not implemented
[17:31:13] gulp-notify: [Compile Error] not implemented
[17:31:13] Bundled app.js in 21 ms
[17:31:13] Bundled plugin.js in 22 ms
[17:31:13] Finished 'browserify' after 136 ms

My setup is:

∙ node -v                                                                                                                                                      17:31  marco@mymachine
v0.12.0
~
∙ gulp -v                                                                                                                                                      17:31  marco@mymachine
[17:31:23] CLI version 3.8.11

Hi,
what kind of changes wouldbe necessary to support other query engines ? Say Spark SQL or Impala?
Thanks

There are two functions (provideAWSCredentials and provideAmazonS3Client(AWSCredentials awsCredentials) ) that currently are set to manually taking in an accesskey or a secretAccessKey. I was wondering if we could push a functionality that would take credentials from an AWSCredentialsProvider for those who have the IAM roles set.

These functions are found in:
airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java

Hi can we use postgress or any database other than MYSQL? Please guide.

Could there be an online demo? (Know it connects to a live DB, which makes it tricky...) Would like to add it to react.rocks.

How difficult is it to remove the dependency on S3 (with reduced functionality)

I'm trying to test Airpal using prestoVersion=0.119 on Amazon EMR. I'm using java version 1.7.0_85. During the ":compileJava" step I get messages like below with many lines of warning and errors. Any ideas how to resolve?

"Unzipping /home/hadoop/.gradle/wrapper/dists/gradle-2.4-bin/1lebsnfoptv8qpa10w6kyy5mp/gradle-2.4-bin.zip to /home/hadoop/.gradle/wrapper/dists/gradle-2.4-bin/1lebsnfoptv8qpa10w6kyy5mp
Set executable permissions for: /home/hadoop/.gradle/wrapper/dists/gradle-2.4-bin/1lebsnfoptv8qpa10w6kyy5mp/gradle-2.4/bin/gradle
Download http://nodejs.org/dist/v0.10.33/node-v0.10.33-linux-x64.tar.gz
:cleanAssets UP-TO-DATE
:clean
:nodeSetup
:installAssets
:buildAssets
[16:23:40] Using gulpfile ~/airpal/src/main/resources/assets/gulpfile.js
[16:23:40] Starting 'browserify'...
[16:23:40] Bundling app.js...
[16:23:40] Bundling plugin.js...
[16:23:50] Bundled plugin.js in 9.4 s
[16:24:03] Bundled app.js in 23 s
[16:24:03] Finished 'browserify' after 24 s
:compileJava
warning: /home/hadoop/.gradle/caches/modules-2/files-2.1/io.airlift/units/0.116/ef6aa76ed76c5531e12cd48bda575e599ca8c0d8/units-0.116.jar(io/airlift/units/DataSize.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded."

I have some ideas to make it easier to start (I waste my time on it):

• add prestoCatalog in example
• add git clone [email protected]:airbnb/airpal.git in readme steps
• make prestoCoordinator: http://localhost:8080 by default
• add export NODE_ENV='development'
• make "browser-sync": "~2.6.5" in src/main/resources/assets/package.json
• add prebuild package (if possible)
• add precompiled javascript (if possible)

I have more questions:

1. What is partitions for?
2. Can we preview sql query result? (I can download, but can not see in browser)
3. Can we use only one schema from catalog? (tables list is too big)
4. Why do you choose java? Stack trace is so hard to read

app/build/plugin.js and app/build/app.js are returning 404 when I run Airpal on OS X so the application is failing to start. The app/stylesheet assets are loading correctly. Any ideas?

Hi @andykram,

Currently we need to call the API twice if we want both the column names and the column data. What's the reason behind this? Speed?

Cheers

i build airpal but my presto working on mysql but airpal still work on hive
so presto get error

SELECT table_catalog, table_schema, table_name FROM information_schema.tables WHERE table_catalog = 'hive'

i want change table_catalog ='myqsl' and i search everywhere can't find this Parameter

Currently the docs do not explain supported configuration options such as changing the default catalog or schema. We should have a Markdown document which explains all the ways Airpal can be configured.

I kept getting mismatched input when I use the unnest operator in my query

Hi there,
Installing on an EC2 server running Amazon Linux, with the user set as "ec2-user", I've cloned the git repo, made everything in the airpal folder executable, and tried running the ./gradlew script with:

• useLocalNode (tried v0.10.33, v0.12.0, v0.12.2)
• Gradle's own Node version (i.e. no flags on "clean shadowJar" command)

Both ways end in a build failure at the same point, with the following error message (see gist):
https://gist.github.com/curry36/ee6e3bdd788a58db761b

Thanks!

I get a weird error when I'm trying to run Airpal connecting to presto. I'm running presto-server 0.107.
Any idea how to solve this?

Command used:
java -server -Duser.timezone=UTC -cp build/libs/airpal-*-all.jar com.airbnb.airpal.AirpalApplication server reference.yml

WARN  [2015-08-08 01:41:55,233] com.google.common.cache.LocalCache: Exception thrown during refresh
! java.lang.IllegalStateException: scheme has not been set
! at com.google.common.base.Preconditions.checkState(Preconditions.java:173) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at io.airlift.http.client.HttpUriBuilder.build(HttpUriBuilder.java:234) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.facebook.presto.client.StatementClient.buildQueryRequest(StatementClient.java:87) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.facebook.presto.client.StatementClient.<init>(StatementClient.java:81) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.QueryRunner.startInternalQuery(QueryRunner.java:30) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.core.execution.QueryClient.executeWith(QueryClient.java:39) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.SchemaCache.queryMetadata(SchemaCache.java:78) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.SchemaCache.access$000(SchemaCache.java:35) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.SchemaCache$1.load(SchemaCache.java:58) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.SchemaCache$1.load(SchemaCache.java:54) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.loadAsync(LocalCache.java:2325) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! ... 7 common frames omitted
! Causing: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: scheme has not been set
! at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2331) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:170) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.loadAsync(LocalCache.java:2326) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.refresh(LocalCache.java:2389) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache.refresh(LocalCache.java:4090) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$LocalLoadingCache.refresh(LocalCache.java:4843) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.SchemaCache$3.run(SchemaCache.java:123) [airpal-0.1.0-SNAPSHOT-all.jar:na]
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
! at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]

[SOLVED]

I have run everything but I am not sure why I can't query and it always return "Catalog hive does not exist". What am i missing here?

Thanks.

In presto-server-0.96/etc/catalog dir, I created a connector named "hive.properties". zzzz

I don't understand this, can you give me a detailed said? Thanks

allGroup = com.airbnb.shiro.UserGroup
allGroup.groups = all
allGroup.permissions = access, read::, write::
allGroup.defaultConnector = hive
allGroup.defaultSchema = default
allGroup.timeout = 40m
allGroup.accessLevel = User

We just noticed the following in the built in presto UI:

Does anyone know why this is happening?

Airplay UI is empty and no default queries are populating

Good morning, a great god, I have a question want to ask, trouble to solve, thank you very much.
How the airpal corresponding database table set user permissions?

Command used : gradle clean build .

                                      ^

TypeError: Cannot read property 'prototype' of undefined
at Object.exports.inherits (util.js:556:43)
at Object. (/Users/bismoymurasing/Desktop/OpenSource/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/http-proxy/lib/http-proxy/index.js:108:17)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/Users/bismoymurasing/Desktop/OpenSource/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/http-proxy/lib/http-proxy.js:4:17)
at Module._compile (module.js:456:26)
:buildAssets FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':buildAssets'.

  Process 'command '/Users/bismoymurasing/.gradle/nodejs/node-v0.10.33-darwin-x64/bin/node'' finished with non-zero exit value 8

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2 mins 2.425 secs
BISMOYs-MacBook-Air:airpal bismoymurasing$

Hello,

I am getting the following issues when installing the airpal.

[ec2-user@ip-172-31-37-150 airpal]$ sudo ./gradlew -Dairpal.prestoVersion=0.85 clean shadowJar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-client/0.85/presto-client-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-root/0.85/presto-root-0.85.pom
Download https://repo1.maven.org/maven2/io/airlift/airbase/28/airbase-28.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.85/presto-spi-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-main/0.85/presto-main-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.85/presto-parser-0.85.pom
Download https://repo1.maven.org/maven2/io/airlift/http-client/0.97/http-client-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/airlift/0.97/airlift-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/json/0.97/json-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/bootstrap/0.97/bootstrap-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/concurrent/0.97/concurrent-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/node/0.97/node-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/configuration/0.97/configuration-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/discovery/0.97/discovery-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/event/0.97/event-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/floatingdecimal/0.2/floatingdecimal-0.2.pom
Download https://repo1.maven.org/maven2/io/airlift/http-server/0.97/http-server-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jaxrs/0.97/jaxrs-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jmx/0.97/jmx-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jmx-http/0.97/jmx-http-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/log/0.97/log-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/log-manager/0.97/log-manager-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/stats/0.97/stats-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/trace-token/0.97/trace-token-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/units/0.97/units-0.97.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-client/0.85/presto-client-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.85/presto-spi-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-main/0.85/presto-main-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.85/presto-parser-0.85.jar
Download https://repo1.maven.org/maven2/io/airlift/http-client/0.97/http-client-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/json/0.97/json-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/bootstrap/0.97/bootstrap-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/concurrent/0.97/concurrent-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/node/0.97/node-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/configuration/0.97/configuration-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/discovery/0.97/discovery-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/event/0.97/event-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/floatingdecimal/0.2/floatingdecimal-0.2.jar
Download https://repo1.maven.org/maven2/io/airlift/http-server/0.97/http-server-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jaxrs/0.97/jaxrs-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jmx/0.97/jmx-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jmx-http/0.97/jmx-http-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/log/0.97/log-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/log-manager/0.97/log-manager-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/stats/0.97/stats-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/trace-token/0.97/trace-token-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/units/0.97/units-0.97.jar
:cleanAssets UP-TO-DATE
:clean
:nodeSetup UP-TO-DATE
:installAssets
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] string_decoder is also the name of a node core module.

[email protected] install /home/airpal/src/main/resources/assets/node_modules/bufferutil
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/bufferutil/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/bufferutil/build' CXX(target) Release/obj.target/bufferutil/src/bufferutil.o SOLINK_MODULE(target) Release/obj.target/bufferutil.node SOLINK_MODULE(target) Release/obj.target/bufferutil.node: Finished COPY Release/bufferutil.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/bufferutil/build'
npm WARN prefer global [email protected] should be installed with -g

[email protected] install /home/airpal/src/main/resources/assets/node_modules/contextify
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/contextify/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/contextify/build' CXX(target) Release/obj.target/contextify/src/contextify.o SOLINK_MODULE(target) Release/obj.target/contextify.node SOLINK_MODULE(target) Release/obj.target/contextify.node: Finished COPY Release/contextify.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/contextify/build'
npm WARN prefer global [email protected] should be installed with -g

[email protected] install /home/airpal/src/main/resources/assets/node_modules/utf-8-validate
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/utf-8-validate/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/utf-8-validate/build' CXX(target) Release/obj.target/validation/src/validation.o SOLINK_MODULE(target) Release/obj.target/validation.node SOLINK_MODULE(target) Release/obj.target/validation.node: Finished COPY Release/validation.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/utf-8-validate/build'
:buildAssets
[21:53:08] Using gulpfile /home/airpal/src/main/resources/assets/gulpfile.js
[21:53:08] Starting 'browserify'...
[21:53:08] Bundling app.js...
[21:53:08] Bundling plugin.js...
[21:53:16] Bundled plugin.js in 7.6 s
[21:53:27] Bundled app.js in 18 s
[21:53:27] Finished 'browserify' after 18 s
:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.7
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:13: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol: class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:66: error: cannot find symbol
implements AsyncHttpClient
^
symbol: class AsyncHttpClient
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:227: error: cannot find symbol
public <T, E extends Exception> HttpResponseFuture executeAsync(Request request, ResponseHandler<T, E> responseHandler)
^
symbol: class HttpResponseFuture
location: class OldJettyHttpClient
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:381: error: cannot find symbol
implements HttpResponseFuture
^
symbol: class HttpResponseFuture
location: class OldJettyHttpClient
/home/airpal/src/main/java/com/airbnb/airpal/core/execution/ExecutionClient.java:29: error: package org.jetbrains.annotations does not exist
import org.jetbrains.annotations.NotNull;
^
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:6: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol: class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:11: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol: class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:19: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol: class AsyncHttpClient
location: class QueryRunner
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:21: error: cannot find symbol
protected QueryRunner(ClientSession session, JsonCodec queryResultsCodec, AsyncHttpClient httpClient)
^
symbol: class AsyncHttpClient
location: class QueryRunner
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:42: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol: class AsyncHttpClient
location: class QueryRunnerFactory
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:44: error: cannot find symbol
public QueryRunnerFactory(ClientSessionFactory sessionFactory, AsyncHttpClient httpClient)
^
symbol: class AsyncHttpClient
location: class QueryRunnerFactory
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:39: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol: class AsyncHttpClient
location: class QueryInfoClient
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:42: error: cannot find symbol
public QueryInfoClient(AsyncHttpClient httpClient, JsonCodec queryInfoCodec)
^
symbol: class AsyncHttpClient
location: class QueryInfoClient
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:57: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol: class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:139: error: cannot find symbol
public AsyncHttpClient provideQueryRunnerHttpClient()
^
symbol: class AsyncHttpClient
location: class AirpalModule
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:175: error: cannot find symbol
@nAmed("query-runner-http-client") AsyncHttpClient httpClient)
^
symbol: class AsyncHttpClient
location: class AirpalModule
16 errors
:compileJava FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':compileJava'.

  Compilation failed; see the compiler error output for details.

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 47.727 secs

I get this file exceeded maximum configured filesize error. How do I increase the limit?. I am using AWS implementation of Airpal.

As discussed with Andy, it should be nice to have an reference.example.yml with all the configuration examples for the AirPal app.

cc: @andykram

At present,Airpal can not support multiple presto connector,but in actual situations,it has some applicatioin scenario,so Airpal plans to add multiple connector on web ui?

I'm trying to install airpal from scratch on Debian Wheezy with nodejs on it. Here is the build log from the command ./gradlew clean shadowJar -Dairpal.useLocalNode:

:installAssets
npm WARN unmet dependency /root/airpal/src/main/resources/assets/node_modules/babel-jest/node_modules/babel-core requires lodash@'^3.2.0' but will load
npm WARN unmet dependency /root/airpal/src/main/resources/assets/node_modules/lodash,
npm WARN unmet dependency which is version 2.4.2
npm WARN unmet dependency /root/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/socket.io/node_modules/engine.io/node_modules/ws requires commander@'~0.6.1' but will load
npm WARN unmet dependency /root/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/commander,
npm WARN unmet dependency which is version 2.3.0
:buildAssets

TypeError: Cannot read property 'prototype' of undefined
    at Object.exports.inherits (util.js:556:43)
    at Object.<anonymous> (/root/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/http-proxy/lib/http-proxy/index.js:108:17)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/root/airpal/src/main/resources/assets/node_modules/browser-sync/node_modules/http-proxy/lib/http-proxy.js:4:17)
    at Module._compile (module.js:456:26)
:buildAssets FAILED

I noticed that :installAssets is pulling these 2 dependencies from the airpal source tree rather than from the system, which causes warns. Versions available through npm are newer.

After running db migrate these are the only tables in the airpal schema:

job_outputs
job_tables
saved_queries
schema_version
tables

This is causing queries to fail with the following exception:

Exception thrown when executing query: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'airpal.jobs' doesn't exist

I have successfully run presto server against mysql.you can see when I try to access http://localhost:8081 ,it seems airpal is ok,but there is nothing shows

When try to receive all the saved queries of a user, you're getting all the runs. This seems to be incorrect.

 XHR finished loading: GET "http://localhost:8081/api/users/anonymous/queries".
2015-02-17 16:19:02.008MySavedQueries.react.js:1 [Object, Object, Object, Object]

The above result should return only one result, because I've got only one result saved.

Please point to steps if this is currently supported for user authentication

Whenever a file gets removed from cache due to it's size (maxiumWeight is by default 100MB), it's also deleted from the file system in the removalListener's procedure, which, I believe, is not the desired situation.

I removed the delete call from removalListener in my environment.

I'm facing this error when I type in the field "Temporary table name for query results".
Any idea how to fix this ?

Hi,

I am seeing that all queries launched to Presto from Airpal that take longer than 15 minutes fail with:
PrestoException: Query was canceled

I think that maybe Airpal is timing out the query for some reason.

We are using the latest Airpal with Presto 0.119 in Amazon EMR.

In the file shiro_static_users.ini we configured the timeout for the group to 40 minutes

In Presto's config.properties we have: query.client.timeout=20m

I don't know any other place where a timeout for this might be considered.

Any ideas why this is happening or how to increase the timeout?

If I do not specify any prestoVersion it works just fine.
When I do specify (I've just tried with 0.93 and 0.96) I get compilation errors.

See: https://gist.github.com/mfirry/492a502fa4b1a4246bfe

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Am I missing something? when I build the airpal, the following exception throwed:

:buildAssets
module.js:338
throw err;
^
Error: Cannot find module 'cl-strings'
at Function.Module._resolveFilename (module.js:336:15)
at Function.Module._load (module.js:278:25)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object. (/Users/ricdong/workspace/install/cdh/airpal-master/src/main/resources/assets/node_modules/browser-sync/lib/messages.js:3:16)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
:buildAssets FAILED

FAILURE: Build failed with an exception.

Hi, we are restricted from downloading using - Downloading https://services.gradle.org/distributions/gradle-2.2.1-all.zip.
So I have downloaded gradle manually and installed it. But when I run the command
./gradlew -Dairpal.prestoVersion=0.119 clean shadowJar
it still goes and looks for downloading. How do I change this in gradlew script to make it point to my already installed gradle? Thanks!

When I use hive catalog and mysql catalog together on my presto server, Airpal doesn't work. Does Airpal support this case or it only support Hive by itself?

when settting a new value to prestoSchema in reference.yml it does not make any effect

im using this version

Hey guys, I got this error while running ./gradlew -Dairpal.prestoVersion=0.85 clean shadowJar
It seems in ClientSessionFactory.java the constructors are using the ClientSession class which had a signature change so the parameters passed in don't match. Could anybody fix it?

:compileJava
/home/hadoop/airpal-master/src/main/java/com/airbnb/airpal/presto/ClientSessionFactory.java:36: error: constructor ClientSession in class ClientSession cannot be applied to given types;
return new ClientSession(server.get(),
^
required: URI,String,String,String,String,String,Locale,Map<String,String>,boolean
found: URI,String,String,String,String,String,Locale,boolean
reason: actual and formal argument lists differ in length
/home/hadoop/airpal-master/src/main/java/com/airbnb/airpal/presto/ClientSessionFactory.java:48: error: constructor ClientSession in class ClientSession cannot be applied to given types;
return new ClientSession(server.get(),
^
required: URI,String,String,String,String,String,Locale,Map<String,String>,boolean
found: URI,String,String,String,String,String,Locale,boolean
reason: actual and formal argument lists differ in length
/home/hadoop/airpal-master/src/main/java/com/airbnb/airpal/presto/ClientSessionFactory.java:60: error: constructor ClientSession in class ClientSession cannot be applied to given types;
return new ClientSession(server.get(),
^
required: URI,String,String,String,String,String,Locale,Map<String,String>,boolean
found: URI,String,String,String,String,String,Locale,boolean
reason: actual and formal argument lists differ in length
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
3 errors

First off, AirPal looks awesome, way to kill it :)

So some useful additions on the data preview:

• sort by a column / multiple columns
• filter for a value in a column / multiple columns
• show all rows in a partition (not as useful as the first 2)

use-case:
It's pretty common that you start with 1 particular value of interest. Maybe a client name, charge amount & date, etc. I remember during meetings or via e-mail getting asked to pull something from the db, which was really just a specific row. Add filtering & sorting to a table, then it lets any person familiar with Excel to get this kind of data out of AirPal.

implementation:
From FixedDataTable docs I don't believe they do virtualization... maybe I'm wrong. You can do this without requerying the table or crashing the DOM if you use SlickGrid (https://github.com/mleibman/SlickGrid). It just does virtual rendering for your table so you can have ~ 1M rows without destroying the DOM / js. Then the filtering or sorting is instant (<100ms from when I used it on ~1M rows).

I'm sure you'll still have partitions that need to be capped at 1M (maybe I'm forgetting the scale of most hive partitions), but for a lot of tables it's pretty useful when exploring the data or trying to check values quickly. For example, we can sort by date & by transaction size. Or find a row with a particular location & rating - all without writing SQL / waiting for requery.

One downside to SlickGrid, is it doesn't use HTMLs table algorithm to size columns (since the rows aren't actually part of the DOM). But you can take the max() of each column and render it offscreen to get the ~column widths.

Github button on http://airbnb.io/projects/aerosolve/ links to airbnb/airpal instead of to airbnb/aerosolve

When I build airpal with presto 0.117, there are errors

$ ./gradlew -Dairpal.prestoVersion=0.117 clean shadowJar
:cleanAssets UP-TO-DATE
:clean
:nodeSetup UP-TO-DATE
:installAssets
:buildAssets
[17:00:57] Using gulpfile /software/servers/github/airpal/src/main/resources/assets/gulpfile.js
[17:00:57] Starting 'browserify'...
[17:00:57] Bundling app.js...
[17:00:57] Bundling plugin.js...
[17:01:08] Bundled plugin.js in 11 s
[17:01:23] Bundled app.js in 26 s
[17:01:23] Finished 'browserify' after 26 s
:compileJava
警告: [options] 未与 -source 1.7 一起设置引导类路径
/software/servers/github/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:13: 错误: 找不到符号
import io.airlift.http.client.AsyncHttpClient;
......

The default build is ok, I can query presto using airpal with Firefox, but failed with Chrome.

Is prestoVersion really needed when I can query presto using airpal?
My presto version is 0.117, any potential problems?