salebab / phpkafka Goto Github PK

Hi,
is there any api to quickly get the length of each topic message queue? waiting for the answer, thanks!

I am getting the following error while using the php kafka client

It Looks like somehow the produce is getting confused between the topic name and the host name.

The code,
{noformat}

I tried running consume function from php in interactive mode and it crashes php. produce function on the other hand works very well.

There are no logs that I can figure out. Any pointers to debug this will help.

Hello,

I already install both phpkafka and librdkafka and it seems that the installation went well. Even following the steps you suggested in README.md I solved the problem of "Unable to load dynamic library '/usr/lib64/php/modules/kafka.so' - librdkafka.so.1".

Now, when I try to execute the sample code I get this error "PHP Fatal error: Class 'Kafka' not found in /var/www/html/kafka-test.php on line 3"

What am I missing?
Why PHP does not recognize the class Kafka?
Is there some specific place where I have to put my code within my public directory?

The code I'm using is below, I really appreciate any help on this.

when i 'make' phpkafka, it occured the followed issue:

[root@OA-test phpkafka]# make test
Build complete.
Don't forget to run 'make test'.

PHP Warning: PHP Startup: Unable to load dynamic library '/root/phpkafka/modules/kafka.so' - librdkafka.so.1: cannot open shared object file: No such file or directory in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library '/root/phpkafka/modules/kafka.so' - librdkafka.so.1: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/root/phpkafka/modules/kafka.so' - librdkafka.so.1: cannot open shared object file: No such file or directory in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library '/root/phpkafka/modules/kafka.so' - librdkafka.so.1: cannot open shared object file: No such file or directory in Unknown on line 0

how can i solve it??? please help me

@EVODelavega
Hi EVODelavega,
Can I ask some questions about offset which I am not quite sure about when I utilize phpkafka client:
https://github.com/EVODelavega/phpkafka

1. Does phpkafka client store offset of each topic partition? I have a try on function getPartitionsForTopic in Kafka.class.php, but it does not work.
2. For consuming message by invoking function consume($topic, $offset = self::OFFSET_BEGIN, $count = self::OFFSET_END), we must tell it the offset, right ?

For consuming messages, I store the offset of each topic partition in files, wrote a php script, which reads offset from the file, and then invokes function consume with the offset as parameter, after that, I will update the offset in the file by adding the message count which are consumed this time to the previous offset. Besides, I add a crontab in the server for executing that php script everything minute.

To be more concrete, if the offset in the file is 5, and 20 messages are consumed this time, the I will update the offset in that file to be 25. I update the offset each time messages are consumed because I do not want to consume same message repeatedly.

I am not sure if my solution of consumer is good or not, what I worry about is, for kafka broker, it will delete old log(segment) by some policy, I did not modify the default configuration of broker, so it will delete any log with a modification time of more than 7 days ago.

If the old logs are deleted by broker, not sure if the offset stored in the file will be incorrect.

Have you come across the same problem? Could you give me some suggestion about this ? Is there any better way to record the offset, and deleting old logs will not affect it.

Thanks!

ldd /usr/lib/php5/20100525/kafka.so
linux-vdso.so.1 => (0x00007fff2bdff000)
librdkafka.so.1 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90d3d42000)
/lib64/ld-linux-x86-64.so.2 (0x00007f90d430f000)

echo time();