qubole / presto-udfs Goto Github PK
View Code? Open in Web Editor NEWPlugin for Presto to allow addition of user functions easily
presto-udfs's People
Contributors
Stargazers
Watchers
Forkers
yixu34 codeaudit akwirick zhenchuan quboletest apoorv2711 kfitzgerald shubhamtagra praks276 zhuomingliang swarooppallapothu shaikidris johnistan tian123lang amoghmargoor quaker1 sczhaoqi mr-bear kc14 simonmayu archer-christ lewis-liu-001 cryptoe rameshbyndoor raunaqmorarka ashpreetbedi binlijin martint jackyfreezer007 jacob119 ericxsun dpolonsky shivyshiv qwalker wangwei216 nambastha ji3jin lishouyan moowei rubik-ai gilesalderson ameya2607 zx-hub yyyangzr ywww phillip2019 muskaya cyofeiyue lucaqy a123demi willhan123 angelzhan doublexxxxxx zxx150615 gucb1011 singer-bin jixuekang dengyaomo particlemedia jihongbin-git krishna-prasad-p-v fanluoopresto-udfs's Issues
Deploy jars to maven central using Maven
array_aggr function does not work
Cannot use in presto 0.151
My presto's version is 0.151, and after I move the .jar file to /plugin/udfs/ and restart my presto, it kept initializing and didn't work. How can I solve this problem? Is it because my presto's version too new?
Presto v0.142 compatibility
Need to update the code to make it compatible with Presto v0.142
Compilation errors on Presto .127+
I got the following errors while I was trying to build with Presto 0.127:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/skt1110551/workspace/presto-udfs/src/main/java/com/facebook/presto/qubole/udfs/UdfFactory.java:[19,36] cannot find symbol
symbol: class ParametricFunction
location: package com.facebook.presto.metadata
[ERROR] /Users/skt1110551/workspace/presto-udfs/src/main/java/com/facebook/presto/qubole/udfs/UdfFactory.java:[20,36] cannot find symbol
symbol: class ParametricAggregation
location: package com.facebook.presto.metadata
[ERROR] /Users/skt1110551/workspace/presto-udfs/src/main/java/com/facebook/presto/qubole/udfs/UdfFactory.java:[49,17] cannot find symbol
symbol: class ParametricFunction
location: class com.facebook.presto.qubole.udfs.UdfFactory
[ERROR] /Users/skt1110551/workspace/presto-udfs/src/main/java/com/facebook/presto/qubole/udfs/UdfFactory.java:[68,17] cannot find symbol
symbol: class ParametricAggregation
location: class com.facebook.presto.qubole.udfs.UdfFactory
[ERROR] /Users/skt1110551/workspace/presto-udfs/src/main/java/com/facebook/presto/qubole/udfs/UdfFactory.java:[70,39] cannot find symbol
symbol: class ParametricAggregation
location: class com.facebook.presto.qubole.udfs.UdfFactory
[INFO] 5 errors
Compilation broken in master branch
I cannot assign it to Apoorv.
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /media/ebs/rv_src/presto- udfs/src/main/java/com/facebook/presto/qubole/udfs/scalar/hiveUdfs/ExtendedDateTimeFunctions.java:
[120,48] cannot find symbol
ymbol: method toUnixTimeFromTimestampWithTimeZone(long)
location: class com.facebook.presto.qubole.udfs.scalar.hiveUdfs.ExtendedDateTimeFunctions
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.360 s
[INFO] Finished at: 2016-06-30T22:58:22+00:00
[INFO] Final Memory: 53M/536M
Change package names to com.qubole
I suggest com.qubole.presto since this is a project maintained by Presto.
Error when plugging in UDFs
Hi, I followed this tutorial and leveraged this github project and wrote a simple 'mysum' UDF function.
@ScalarFunction("mysum")
@Description("Returns summation of two numbers")
@SqlType(StandardTypes.BIGINT)
public static long sum(@SqlType(StandardTypes.BIGINT) long num1, @SqlType(StandardTypes.BIGINT) long num2) {
return num1 + num2;
}
I followed the below steps to plugin the UDF but Presto fails to import the function. Since, there is very little documentation about writing and plugging UDFs, any help would be much appreciated. Thanks in advance.
- ran
mvn compileandmvn package - copied the
.jarfile into plugins folder under presto (unzipped) directory/Users/nithin/presto-server-0.166/plugin/udfs/ - started coordinator using
/Users/nithin/presto-server-0.166/bin/launcher run - ran the UDF
select mysum(10,100)in Presto CLI, but throws error
Below is error log when I try to run my UDF ('mysum') in Presto CLI. It is quite evident that Presto is not able to find the UDF, so plugging in wasn't successful. How to fix that? Am I missing any step?
➜ Workspaces ./presto.jar --server localhost:8080 --catalog mysql --schema default --debug
presto:default> select mysum(99,100);
Query 20170228_183509_00002_vr5dt failed: line 1:8: Function mysum not registered
com.facebook.presto.sql.analyzer.SemanticException: line 1:8: Function mysum not registered
at com.facebook.presto.sql.analyzer.ExpressionAnalyzer$Visitor.visitFunctionCall(ExpressionAnalyzer.java:824)
at com.facebook.presto.sql.analyzer.ExpressionAnalyzer$Visitor.visitFunctionCall(ExpressionAnalyzer.java:255)
at com.facebook.presto.sql.tree.FunctionCall.accept(FunctionCall.java:111)
at com.facebook.presto.sql.tree.StackableAstVisitor.process(StackableAstVisitor.java:26)
at com.facebook.presto.sql.analyzer.ExpressionAnalyzer$Visitor.process(ExpressionAnalyzer.java:274)
at com.facebook.presto.sql.analyzer.ExpressionAnalyzer.analyze(ExpressionAnalyzer.java:231)
at com.facebook.presto.sql.analyzer.ExpressionAnalyzer.analyzeExpression(ExpressionAnalyzer.java:1406)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.analyzeExpression(StatementAnalyzer.java:1802)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.analyzeSelect(StatementAnalyzer.java:1623)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuerySpecification(StatementAnalyzer.java:800)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuerySpecification(StatementAnalyzer.java:188)
at com.facebook.presto.sql.tree.QuerySpecification.accept(QuerySpecification.java:127)
at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:27)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuery(StatementAnalyzer.java:550)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuery(StatementAnalyzer.java:188)
at com.facebook.presto.sql.tree.Query.accept(Query.java:94)
at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:27)
at com.facebook.presto.sql.analyzer.Analyzer.analyze(Analyzer.java:68)
at com.facebook.presto.sql.analyzer.Analyzer.analyze(Analyzer.java:60)
at com.facebook.presto.execution.SqlQueryExecution.doAnalyzeQuery(SqlQueryExecution.java:290)
at com.facebook.presto.execution.SqlQueryExecution.analyzeQuery(SqlQueryExecution.java:276)
at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:234)
at com.facebook.presto.execution.QueuedExecution.lambda$start$1(QueuedExecution.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
select mysum(99,100)
Adding support of traversing the whole `udfs` presto-dir and reading all udf jars in presto-server bootstrap
What I understood from the code is that this jar scans its own classes and adds all the udfs from there. What needed was to keep one's udfs separately. So for that dir. scan for valid udfs is needed.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.