xingwenge / canal-php Goto Github PK
View Code? Open in Web Editor NEWAlibaba mysql database binlog incremental subscription & consumer components Canal's php client[阿里巴巴mysql数据库binlog的增量订阅&消费组件 Canal 的 php 客户端 ] https://github.com/alibaba/canal
Alibaba mysql database binlog incremental subscription & consumer components Canal's php client[阿里巴巴mysql数据库binlog的增量订阅&消费组件 Canal 的 php 客户端 ] https://github.com/alibaba/canal
canal启动后,canal.log会一直报“something goes wrong with channel:[id: 0x7cb4dc0d, /xx.xx.x.xxx:41776 => /xx.xx.x.xxx:11111], exception=java.io.IOException: Connection reset by peer”,出现频率每分钟基本都有,但是canal可以正常监听到binlog数据。
搭建方式:单例Simple(docker容器)
客户端:php客户端(官方demo)
const TYPE_SOCKET = 1;
const TYPE_SWOOLE = 2;
const TYPE_SOCKET_CLUE = 3;分别代表什么意思
用倒是很想用,就是不知道流程怎么运转的,.zip下的文件里面也没有找到安装的入口
Socket operation failed: Broken pipe (SOCKET_EPIPE)
PHP Fatal error: Uncaught Socket\Raw\Exception: Socket operation failed: Broken pipe (SOCKET_EPIPE) in /data/docker/www/canal/vendor/clue/socket-raw/src/Exception.php:59
请问如果我可以查询到队列里的条数吗?如果生产的数据非常多,消费不及时的话。如果感知不到未消费的条数,就无法根据情况去判断要增加的消费进程数量
$client->connect($canalHost, $canalPort) 连接返回null 是连接canal 客户端 不成功吗
Exception 'Exception' with message 'get data error. error code:400, error message:packet type=CLIENTAUTHENTICATION is NOT supported!'
in /mnt/c/futu/ns_oss/vendor/xingwenge/canal_php/src/adapter/CanalConnectorBase.php:205
Stack trace:
#0 /mnt/c/futu/ns_oss/vendor/xingwenge/canal_php/src/adapter/CanalConnectorBase.php(151): xingwenge\canal_php\adapter\CanalConnectorBase->getWithoutAck()
#1 /mnt/c/futu/ns_oss/console/controllers/CanalController.php(107): xingwenge\canal_php\adapter\CanalConnectorBase->get()
docker 里面运行:
$client = CanalConnectorFactory::createClient(CanalClient::TYPE_SOCKET_CLUE);
$client->connect("zookeeper", 2181);
php1_1 | Warning: unpack(): Type N: not enough input, need 4, have 0 in /canal/src/adapter/clue/CanalConnector.php on line 29
php1_1 |
php1_1 | Notice: Trying to access array offset on value of type bool in /canal/src/adapter/clue/CanalConnector.php on line 29
php1_1 | Socket operation failed: Success
php1_1 | Socket operation failed: Broken pipe (SOCKET_EPIPE)
有以上报错,请问是不支持 HA 吗,还是我写法错了呢?
感谢!
看其他语言现在是支持的,现在是还没有添加吗?可否给个联系方式?
我搭建的单机canal和单个php客户端来进行消费,中间涉及重组数据再输出到es,所以消费效率不高,35条/s,消费方式是tcp,同步ack。请问在tcp模式下有什么好的办法提高消费效率吗?php客户端如何设置异步ack呢,有例子吗?
不支持集群吗?只能连接指定的canal-server吗?
客户端中,不能同步处理好mysql源表的blog类型的字符串,会有乱码的发生。乱码只产生于 中文字符,英文字符没有问题。
canal版本为1.1.14
canal-python pip客户端为0.4版本
如题,可以部署在nginx下么?
在消费一条sql的时候,由于一个错误导致canal-php崩溃退出,重启canal-php后,刚才错误的sql就过去了,相当于没有消费,这种问题怎么处理乃?
canal-server:1.1.4
按照 sample 样例进行测试,能够正常输出sql
但是打印的 RowData 为空,类型为 \Google\Protobuf\Internal\RepeatedField
并不是期望的 \Com\Alibaba\Otter\Canal\Protocol\RowData[]
查看了 example 和 canal 日志,并没有报错
不知道是什么原因
让我们用 compare require 安装呢?
PHP Warning: unpack(): Type N: not enough input, need 4, have 0 in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
Warning: unpack(): Type N: not enough input, need 4, have 0 in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
PHP Notice: Trying to access array offset on value of type bool in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
Notice: Trying to access array offset on value of type bool in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
unexpected packet type:0
PHP Warning: unpack(): Type N: not enough input, need 4, have 0 in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
Warning: unpack(): Type N: not enough input, need 4, have 0 in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
PHP Notice: Trying to access array offset on value of type bool in /opt/www/vendor/xingwenge/canal_php/src/adapter/swoole/CanalConnector.php on line 34
总是提示 Socket operation failed:Broken pipe(SOCKET_EPIPE) php fatal error: Uncaught Socket\Raw\Exception:Socket operation failed:Broken pipe(SOCKET_EPIPE) in vendor/clue/socket-raw/src/Exception.php:59 是什么问题
报错信息在 canal-php/src/adapter/clue/CanalConnector.php on line 29
我尝试打印 $data
变量,值为 ""。
当配置 canal 账户密码时有此错误提示。
$client->checkValid('canal', 'canal');
canal.properties 配置文件:
canal.user = canal
canal.passwd = E3619321C1A937C46A0D8BD1DAC39F93B27D4458
感谢答复~
Fmt::println 里的 $rowChange->getSql() sql语句为什么没有打印出来呢
-------> before
id : 29 update= false
catid : 6 update= false
modelid : 2 update= false
title : 案例展示 update= false
thumb : https://img.staticdj.com/5db9190f2c7cfa1b8dcd1572b1862965_1080x.jpg update= false
keywords : update= false
description : update= false
listorder : 0 update= false
status : 1 update= false
hits : 5 update= false
username : admin update= false
time : 1392974386 update= false
-------> after
这个没打印出 SQL语句,岂不是要根据
foreach ($rowChange->getRowDatas() as $rowData) {
switch ($evenType) {
case EventType::DELETE:
self::ptColumn($rowData->getBeforeColumns());
break;
case EventType::INSERT:
self::ptColumn($rowData->getAfterColumns());
break;
default:
echo '-------> before', PHP_EOL;
self::ptColumn($rowData->getBeforeColumns());
echo '-------> after', PHP_EOL;
self::ptColumn($rowData->getAfterColumns());
break;
}
}
里面的 增删改查 把每种情况的字段都取出来,然后根据条件 插入到ES ?
canal-server报错 exception=java.lang.IllegalArgumentException: illegal hex-string: canal
get data error. error code:400, error message:something goes wrong with channel:[id: 0x0f268ec7, /127.0.0.1:58478 => /127.0.0.1:11111], exception=com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:1007 is not the firstly:854
这个怎么搞
启动 两个或以上的任务 为什么不能监控成功
canal装在本机,端口也是默认,我执行了sample下的 index.php,提示如标题,请问这是什么原因引起?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.