Comments (14)
jeff-zou
commented on June 27, 2024
有报错么？
Francis-X ***@***.***> 于2022年5月7日周六 17:30写道：
… 好像不支持flink1.14.4版本
—
Reply to this email directly, view it on GitHub
<#7>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFNV7R7GXEB3W4BGVJX5FS3VIYZ3NANCNFSM5VKFCWPA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
测试了一下，是支持的。
只是flink1.14.4删除了blink相关的包，所以maven编译时会报错，直接使用下面引用运行不会有问题。
<dependency>
<groupId>io.github.jeff-zou</groupId>
<artifactId>flink-connector-redis</artifactId>
<version>1.0.11</version>
</dependency>
如果要自行编译的话请删除pom文件里的blink相关包引用。
jeff zou ***@***.***> 于2022年5月13日周五 13:56写道：
… 有报错么？
Francis-X ***@***.***> 于2022年5月7日周六 17:30写道：
> 好像不支持flink1.14.4版本
>
> —
> Reply to this email directly, view it on GitHub
> <#7>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFNV7R7GXEB3W4BGVJX5FS3VIYZ3NANCNFSM5VKFCWPA>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
from flink-connector-redis.
Francis-X
commented on June 27, 2024
第一次跑报错了，后面好像没有报错了，然后我自己实现了一套sink
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
新加了什么功能
Francis-X ***@***.***> 于2022年5月13日周五 15:05写道：
… 第一次跑报错了，后面好像没有报错了，然后我自己实现了一套sink
—
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFNV7R7KICWTAACZ5MTX5A3VJX5LZANCNFSM5VKFCWPA>
.
You are receiving this because you commented.Message ID:
***@***.***>
from flink-connector-redis.
Francis-X
commented on June 27, 2024
我底层用的Lettuce客户端
from flink-connector-redis.
Francis-X
commented on June 27, 2024
你好，我可以咨询你一些问题吗？
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
可以
Francis-X ***@***.***> 于2022年5月30日周一 20:59写道：
… 你好，我可以咨询你一些问题吗？
—
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFNV7R2K3IDZIVL5HYZW5UTVMS3TDANCNFSM5VKFCWPA>
.
You are receiving this because you commented.Message ID:
***@***.***>
from flink-connector-redis.
Francis-X
commented on June 27, 2024
请问你有过flink 大窗口checkpoint 超时问题经历吗？
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
有的，中间状态数据量太大了就会出现，你用的是flink哪个版本
Francis-X ***@***.***> 于2022年5月31日周二 11:49写道：
… 请问你有过flink 大窗口checkpoint 超时问题经历吗？
—
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFNV7R5NHKZ5TJILBQA4RETVMWD5VANCNFSM5VKFCWPA>
.
You are receiving this because you commented.Message ID:
***@***.***>
from flink-connector-redis.
Francis-X
commented on June 27, 2024
我用1.14.4和1.15.0都会有这个问题，超时达到次数就把taskmanager搞挂了，你是如何解决这个问题的呢？
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
是不是试了增量checkpoint也没用？ 如果没试过可以试一下，之前我的状态非常大用增量也没用，后面是调整业务实现方式，绕开了checkpoint, 大致思路是将任务拆分成两个，第一个任务先预处理，第二个任务做聚合计算时只需要对简单数据进行checkpoint. 可以加我v聊 zilong954224
from flink-connector-redis.
Francis-X
commented on June 27, 2024
对的，增量checkpoint也没用，我这边是因为窗口开了一个小时，数据来源频率是每10分钟会产生20多万数据
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
数据量不大呀，单条数据有多大，是其它问题吧，平常checkpoint能成功不？
Francis-X ***@***.***> 于2022年6月1日周三 13:56写道：
… 对的，增量checkpoint也没用，我这边是因为窗口开了一个小时，数据来源频率是每10分钟会产生20多万数据
—
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFNV7RZ7SLITLSHO57PE2M3VM33QVANCNFSM5VKFCWPA>
.
You are receiving this because you commented.Message ID:
***@***.***>
from flink-connector-redis.
jeff-zou
commented on June 27, 2024
close
from flink-connector-redis.
Related Issues (20)
- 能在入redis中增加支持原始BYTES入redis的方法么？ HOT 5
- Caused by: java.lang.ClassNotFoundException: io.netty.channel.ChannelHandler HOT 5
- sink.parallelism 参数无效 HOT 1
- Lettuce API Implicit Failure HOT 1
- Flink CDC同步MySQL数据到Redis时，Redis不能同步删除数据 HOT 5
- 哨兵模式下连接失败 HOT 3
- 异常问题 HOT 6
- 报错 no match redis HOT 4
- 对于redis消费相关问题 HOT 1
- 导入依赖运行test示例报错 HOT 3
- 测试里有个flushdb操作, 跑了一下测试case，测试redis数据全没了，差点要收拾包裹走人了 HOT 1
- hash结构怎么查询 HOT 1
- How to implement exactly once in redis sink function? HOT 1
- hset命令下ttl不断重置的问题 HOT 4
- Setting `command` option in TableDescriptor leads to Java error HOT 6
- [flink1.16.0]java.lang.UnsatisfiedLinkError: 'int io.netty.channel.unix.Socket.sendAddress(int, long, int, int)' HOT 5
- 适配flink 1.17.0 HOT 1
- 1.3.1 版本 能写入数据，但是读取不到数据 要怎么处理 HOT 2
- datastream sink的时候怎么设置ttl HOT 1
- io.lettuce.core.RedisConnectionException: Unable to connect to host:port HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flink-connector-redis.