armetiz / leezypheanstalkbundle Goto Github PK

This is a great bundle however I believe all the events here should be explained with a few words. https://github.com/armetiz/LeezyPheanstalkBundle/blob/master/src/Resources/doc/4-events.md

Here is my composer.json:

"require": {
       ...
        "leezy/pheanstalk-bundle" : "1.*"
    },

and here is the error:

$ php composer.phar update leezy/pheanstalk-bundle
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - leezy/pheanstalk-bundle 1.0.3 requires pda/pheanstalk dev-master -> no matching package found.
    - leezy/pheanstalk-bundle 1.0.2 requires pda/pheanstalk dev-master -> no matching package found.
    - leezy/pheanstalk-bundle 1.0.1 requires mrpoundsign/pheanstalk-5.3 * -> no matching package found.
    - leezy/pheanstalk-bundle 1.0.0 requires mteneur/pheanstalk-5.3 * -> no matching package found.
    - Installation request for leezy/pheanstalk-bundle 1.* -> satisfiable by leezy/pheanstalk-bundle 1.0.0, leezy/pheanstalk-bundle 1.0.1, leezy/pheanstalk-bundle 1.0.2, leezy/pheanstalk-bundle 1.0.3.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

leezy.pheanstalk.proxy.default service definition is invalid (it does not specify the PheanstalkInterface $pheanstalk argument) and thus the Symfony lint:container command fails because of it. As a temporary workaround I've redefined the definition of that service in my application so that linting the container doesn't fail in my CI pipeline.

Our use case have multiple beanstalk instances and seems like the bundle can work with only the last one listed, because of a missing clone statement.

Have a look at the following bit from ProxyCompilerPass with my comments:

// get service definition by reference
$pheanstalkDef = $container->getDefinition($pheanstalk['proxy']);

// modify the service definition
$pheanstalkDef->addMethodCall('setPheanstalk', [new Definition(Pheanstalk::class, $pheanstalkConfig)]);
$pheanstalkDef->addMethodCall('setName', [$name]);

// setting an alias to it
$container->setDefinition('leezy.pheanstalk.'.$name, $pheanstalkDef);

If the last line is as follows, the behavior would be exactly the same:

$container->setAlias('leezy.pheanstalk.'.$name, $pheanstalk['proxy']);

Let's discuss this. If you agree with me, we can change the line to clone the definition, rather than alias it.

$pheanstalkDef = clone $container->getDefinition($pheanstalk['proxy']);

My config

leezy_pheanstalk:
    enabled: true
    pheanstalks:
        primary:
            server: %beanstalkd_server%
            default: true
            proxy: lighthouse.core.job.pheanstalk

Where lighthouse.core.job.pheanstalk is service that extends Pheanstalk_Proxy and adds some extra functionality

But cache clear fails with error

  [Symfony\Component\DependencyInjection\Exception\InvalidArgumentException]  
  The service definition "lighthouse.core.job.pheanstalk" does not exist.   

I've debugged a little and found out that at the moment when LeezyPheanstalkExtension class creates DI Definition of connections there are only few services defined in container and my services is not set in container yet, though i configure it in common way it is done in Symfony.

A depecrated message is raised about the configuration of the leezy_pheanstalk node. From Symfony 5 this configuration is no longer possible. Do you plan to make any improvements?
Thank you in advance

Hi!

Thanks for building this just before I needed it. =)

When trying to set it up, I found two typos in the readme file.

In the deps file,

[LeezyPheanstalkBundle]
    git=git://github.com/pda/pheanstalk.git
    target=bundles/Leezy/PheanstalkBundle

should be

[LeezyPheanstalkBundle]
    git=git://github.com/armetiz/LeezyPheanstalkBundle
    target=bundles/Leezy/PheanstalkBundle

and in AppKernel.php,

new Leezy\PheanstalkBundle\PheanstalkBundle(),

should be

new Leezy\PheanstalkBundle\LeezyPheanstalkBundle(),

The beanstalkd requirement should be mrpoundsign/pheanstalk-5.3

When you try to ignore the default tube when beanstalkd contains only one tube ( the default tube ).
Pheanstalk client throw an Pheanstalk\Exception\ServerException.

So, use this option carefully until a solution can be found.

When trying to get this to work, I stumbled over the class name in the dependency injection defintion in LeezyPheanstalkExtension.

Currently, it says "Pheanstalk\Pheanstalk", but when using that I get a class not found error.

To get it to work, I had to change it to

$pheanstalkDef = new Definition("Pheanstalk", array ($server, $port, $timeout));

and add a line in autoload.php:

$loader->registerPrefixes(array(
    ...
    'Pheanstalk'       => __DIR__.'/../vendor/pheanstalk/classes',
));

Did I miss a simpler solution?

Pheanstalk Version 4 has been released meanwhile. Do you have any plans to support it?

I noticed that the exception type was fixed in the commands but just in master. Can you release 2.1.4 with those fixes? my composer is setup with "2.1.*"

cheers

Hi, I'm writing the bundle for Silex 💃

https://github.com/sergiors/pheanstalk-service-provider

Hi,

Could you please add a new tag for the current master version (ex. 1.0.4).

Installing the bundle with composer for version 1.* ends up with ConnectionFinder throwing "Class 'Pheanstalk' not found"...

Pheanstalk changed its class name from "Pheanstalk" to "Pheanstalk_Pheanstalk" - the change is visible in master branch, but is not tagged and so - is not available in 1.*.

Thanks,
ef

When the locator has no instance to return, it should throw an exception instead of returning null.

Returning null makes people to have to test the return of the locator and then eventually throw an exception and prevent them to chain function calls.

    function sendMessage(Message $message, string $tube): self
    {
        $this->pheanstalk_locator
            ->getPheanstalk($this->readNameFromConfig())
            ->useTube($tube)
            ->put($message)
            ;

        return $this;
    }

This code assumes the locator throws an exception if no pheanstalk instances could be returned. I can catch the exception if I want to have an alternate execution path, or let it as is if I expect the tube to exist in the configuration.

Especially true with getDefaultPheanstalk().

I would like to use this bundle on a project under SF 2.0, which does not use composer.

Is this bundle compatible with SF 2.0 ?
If it is, how can I install it through deps file ?

Thanks a lot for your work and help!

Hello!

After updating symfony to last 3.x, I see this warning:
The "leezy.pheanstalk.command.delete_job" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.

Not only for this command for all of it.
Before update was the deprecation message about it.

Can you fix it?
For the symfony 4.x all commands and services must be public or not used(as i understend).

For now i reinitialize commands with public option.

Thank you.

A connection is a different object for pheanstalk library.

Will lead to a BC.

Would it be useful to have data transformers?

For example, to register a given class as having a given service as transformer and later on to be able to put and fetch jobs from this type transparently with all the hydrating being made automatically by the bundle itself.

I think this may be a nice feature.
Any thoughts?

I've implemented in my project a proxy class, in order to edit the payload before is returned to the application. Wouldn't it better to have a post RESERVE command for that?

If so, PUT is the complementary command and deserve a post event too...

If the queue server is down, an exception is thrown and is blocking the access of the website.

Exceptions should be handled softly and logged with monolog.

Pheanstalk_Exception_ConnectionException: Socket error 13: Permission denied (connecting to localhost:11300)

Can you tag a new bugfix release that contains commit f36e096 please? Currently the debug panel doesn't work due to an exception on the latest stable Symfony release. That commit fixes it:

Block "javascript" on template "@WebProfiler/Profiler/base.html.twig" does not exist.

Would you be open to add a markdown extension to the CHANGELOG file to format it adding comparison link between tags, ie:

2.1.1...2.1.2

That would be useful, what to you think?

The warning call assumes the logger implements PSR-3. This is available since Symfony 2.2. For 2.1 projects this call will fatal.

It's probably easiest to change it to addWarning, since Monolog still contains this method (for BC I assume). And to change it to warning when you want to drop support for 2.1.

I am missing the feature, to be abled to prefix tubes per server.

Something like this:

leezy_pheanstalk:
    enabled: true
    pheanstalks:
        primary:
            server: 127.0.0.1
            port: 11300
            timeout: 60
            tube_prefix: production_
            default: true

What do you think?

When executing leezy:pheanstalk:stats command I get and exeption

Command "l:p:stats" is ambiguous (leezy:pheanstalk:stats, leezy:pheanstalk:stats-job and 1 more).

Maybe the problem is actually with Symfony console component, but the problem could be solved easily with just renaming stats command to stats-full or something like it.

After installing with composer successfully, I'm getting the following error with any command and in my code.

Hi,

I've never used Symfony so I have no idea how to install this.
I created a composer.json to download the package and it's dependencies, but don't know what to do next.
Might you have some reference or maybe advice as to how I would go by installing this?
Thanks!

Are there plans to do Symfony 5 compatibility?

When making heavy use of workers, there's a lot of logging going on, especially in long running commands. What's more is that all commands are logged with the info level, which can really clog up logs.

I'd like to propose two possible changes:

1. Log to a specific pheanstalk channel. That way you can include/exclude it by monolog configuration, send it to its own separate log file, etc.
2. Add a configuration option to disable the log listener, just like the profiler.

What do you think?

It is not possible to inject pheanstalks in my own bundles. My config:

leezy_pheanstalk:
    pheanstalks:
        default:
            server: localhost
            default: true

The service IDs leezy.pheanstalk and leezy.pheanstalk.default are generated by the ProxyCompilerPass. That means that in MyBundleExtension I cannot create a new Reference to these services because they do not exist yet. Effectively I cannot inject them. Symfony says:

[Symfony\Component\DependencyInjection\Exception\InvalidArgumentException]
The service definition "leezy.pheanstalk" does not exist.

Is there a specific reason why you create these services in a CompilerPass and not directly in the LeezyPheanstalkExtension?

Whats the best way to set this up? A tube per task? Or some logical tubes with the command being part of the payload, which in return calls the specific console command with its dependencies?

Or maybe another client thats listening to tubes, in return calling commands? Since long running tasks + PHP = WIZARD status.

Supervisord keeping up a command per tube, which in return calls another command to actually do something with it?

On each commands, check if pheanstalk connection is listening and add an error message if it's not.

It would be interesting to see if it's feasible to support different queuing system such as Amazon SQS or RabbitMQ.

I don't know how those 2 others queue are working and if it would be possible to integrate them implementing adapters for each one of them.

Anyone has experience with those queuing system and could comment on the feasibility?

I think it would be a great idea to integrate this bundle with the profiler to check the tubes and the jobs queuing.

I know that Sonata Block is doing it (to display the block on the page)

I might have a look this week or next week if I have time.

- The requested package leezy/pheanstalk-bundle could not be found in any version, there may be a typo in the package name.

When & Where log this information.
I guess, the best place to do this is when user require a specific action, like using a CLI Command.

Using a Proxy system, we could dispatch Event, Log them, and even in custom proxy add Security or Auto Serialization.

User have to be able to choose to use a Acme\PheanstalkProxy instead of the default one.

Here is the context:

Installed beanstalk bundle with composer "composer require leezy/pheanstalk-bundle"

I'm trying to using inside a command but i get this error

Cannot autowire service "App\Command\Worker\ProcessParserCommand": argument "$pheanstalk" of method "__construct()" references interface "Pheanstalk\Contract\PheanstalkInterface" but no such
service exists. You should maybe alias this interface to the existing "leezy.pheanstalk.proxy.default" service.

class ProcessParserCommand extends Command
{
    protected static $defaultName = 'app:worker:process-parser';

    /** @var PheanstalkInterface $pheanstalk */
    private $pheanstalk;

    protected function configure()
    {
        $this
            ->setDescription("Parse something")
        ;
    }



    public function __construct(PheanstalkInterface $pheanstalk)
    {
        $this->pheanstalk=$pheanstalk;

        parent::__construct();
    }
}

I'm testing this bundle and looks good so far.

I found a minor annoyance, while creating the connection (using latest stable). I named the connection default instead of primary just to use the same convention as doctrine and other packages. Then I added a job in an example-tube and looked to the profiler page. Some kind of display loop happens and the application display 11 jobs in 22 tubes.

Renaming the connection to primary fix all the weird behaviour. An easy fix is to add a validation contraint in the Configuration class. Something like:

    $nodeConnection
        ->validate()
        ->ifInArray(array('default'))
            ->thenInvalid('Invalid connection name "default"')
        ->end()

When open symfony development profiler I get
Block "javascript" on template "@WebProfiler/Profiler/base.html.twig" does not exist.

The actual dependency status of LeezyPheanstalkBundle requires "symfony/console" version 2.x. Would it be possible to have a ~2.3|3.0 in composer.json ?

Hello sir, we got some problem, after migrating to this bundle from https://github.com/pheanstalk/pheanstalk

Symfony Profiler does not work correctly, and showing us this message:
An error occurred while loading the web debug toolbar.

And we got some messages in our log file below.

[2018-12-08 20:25:08] request.CRITICAL: Uncaught PHP Exception UnexpectedValueException: "The profiler template "LeezyPheanstalkBundle:Profiler:pheanstalk.html.twig" for data collector "pheanstalk" does not exist." at /var/www/application/vendor/symfony/web-profiler-bundle/Profiler/TemplateManager.php line 90 {"exception":"[object] (UnexpectedValueException(code: 0): The profiler template \"LeezyPheanstalkBundle:Profiler:pheanstalk.html.twig\" for data collector \"pheanstalk\" does not exist. at /var/www/application/vendor/symfony/web-profiler-bundle/Profiler/TemplateManager.php:90)"} []

We are using:

• Symfony 4.1 (33a71ed) Hope this compatibility works correctly ;)
• leezy/pheanstalk-bundle 3.3

Can u help us? many thanks.

From #12 (comment) & @Maxwell2022

It would be a good idea to move some of them in Resources/doc like most of the bundle are doing.

In the README we should have the basic installation guide and the table of content (for additional feature).
Additional features would be:

Advance configuration (multiple servers)
How to use it in a controller
Command line tool
A simple Beanstalkd Manager
Profiler
Configuration reference (full config.yaml config options)

I'm using your bundle (v3.1) on a symfony 2.3 project. It throws a "Missing argument" exception on PheanstalkLogListener/ onCommand() function. (screenshot 1 bellow)
The missing exception is $eventname. if I change the function to remove that parameter and in the code, get the event name by: $event->getName(), it works (screenshot 2)

Screenshot 1:

Screenshot 2:

I upgraded to Symfony 3.4 and am now getting an depreciation message with phpunit.

Implementing "Symfony\Component\HttpKernel\DataCollector\DataCollectorInterface" without the "reset()" method is deprecated since version 3.4 and will be unsupported in 4.0 for class "Leezy\PheanstalkBundle\DataCollector\PheanstalkDataCollector"

There seems to be a memory leak in this bundle.
I have tried the same example using only the Pheanstalk library and it goes without the leak.

In order to reproduce it you can use this code sample:

$tube = 'YOUR_TUBE_HERE';
$hostname = 'YOUR_HOSTNAME';

$i = 0;
$iterations = 1000;

//$queue = new \Pheanstalk_Pheanstalk($hostname);
$queue = $this->getContainer()->get('leezy.pheanstalk.primary');

gc_enable();
while (true)
{
    ++$i;
    echo 'memory usage: '. memory_get_usage() . "\n";

    $job = $queue->reserveFromTube($tube);

    if ($i >= $iterations) {
        break;
    }

    gc_collect_cycles();
    usleep(10000);
}

You are going to notice the memory usage gets higher and higher, like so:

memory usage: 10276560
memory usage: 10550936
memory usage: 10555312

...

memory usage: 14987360
memory usage: 14991728
memory usage: 14996104
memory usage: 15000480
memory usage: 15004848
memory usage: 15009224

Here's my environment information, as it may be relevant: http://pastebin.com/yRQ2n3Xj

Hi,

I'm not sure if whether this is related to Ocramius or Leezy but when I try to install one or both of these packages with composer I'm getting error below. I tried certain version numbers for both packages but for some reason error won't go away. Any reason why?

Thanks

PHP Fatal error:  Class 'Leezy\pheanstalkBundle\Proxy\PheanstalkProxyInterface' not found in /var/www/html/local/fws/vendor/ocramius/proxy-manager/src/ProxyManager/Generator/ParameterGenerator.php on line 143