ongr-io / elasticsearchbundle Goto Github PK
View Code? Open in Web Editor NEWSymfony bundle for Elasticsearch with steroids
License: MIT License
Symfony bundle for Elasticsearch with steroids
License: MIT License
This will allow us to change testing bundle more freely.
$repository->execute($search, Repository::RESULTS_ARRAY) returns:
Array
(
[0] => Array
(
[title] => test_prod2
[description] => test_desc2
[price] => 7.79
)
[1] => Array
(
[title] => test_prod
[description] => test_desc
[price] => 0.99
)
)
There are some fields missing, particularly id
and score
. Is this by design?
Add support for 'is' getter for bool type document fields (getActive() => isActive()).
Now when document have private bool type field ($active) ElasticsearchBundle check is declarated set and get methods (getActive(), setActive()) for it.
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Mapping/MetadataCollector.php#L257
FAILURES! Tests: 306, Assertions: 650, Failures: 6, Errors: 1.
Terms aggregation has new field in response, script filter completely fails.
Currently it is not possible to define custom format for date type fields:
@ES\Property(name="dateField", type="date", format="YYYY-MM-dd h:i:s")
It should throw Document not found.
es:index:create - OK
app/console es:index:create --manager="default" - takes a long time
after clearing the cache, it creates an index imediately
But when the index is dropped, the problem persist when trying to create another index.
Also sometimes it takes longer to drop indexes.
duplicating code at
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L82
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L108
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L129
etc.
Consider the following code:
/**
* Demonstrates unexpected behaviour of ElasticsearchTestCase.
*/
class ESBProblem extends ElasticsearchTestCase {
/**
* Test for success - succeeds.
*/
public function testSuccess()
{
$repository=$this->getManager()->getRepository('AcmeTestBundle:Product');
usleep(250000);
$repository->execute($repository->createSearch());
}
/**
* Test for fail - fails.
*
* @expectedException \Exception
*/
public function testFail()
{
$repository=$this->getManager()->getRepository('AcmeTestBundle:Product');
// Let's remove usleep(250000) and see what happens.
$repository->execute($repository->createSearch());
}
}
Override elasticsearch-php serializer to pretty print json.
duplicating code at
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L82
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L108
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L129
etc.
For the given document mapping:
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* @var string
*
* @ES\Property(name="meta", type="object", objectName="AcmeDemoBundle:Meta")
*/
private $meta;
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
/**
* {@inheritdoc}
*/
public function getMeta()
{
return $this->meta;
}
/**
* {@inheritdoc}
*/
public function setMeta(Meta $meta)
{
$this->meta = $meta;
return $this;
}
}
With the meta object definition:
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
/**
* @ES\Object
*/
class Meta
{
/**
* @var string
*
* @ES\Property(name="title", type="string", index="not_analyzer")
*/
public $key;
/**
* @var string
*
* @ES\Property(name="value", type="string", index="not_analyzer")
*/
public $value;
/**
* {@inheritdoc}
*/
public function getKey()
{
return $this->key;
}
/**
* {@inheritdoc}
*/
public function setKey($key)
{
$this->key = $key;
return $this;
}
/**
* {@inheritdoc}
*/
public function getValue()
{
return $this->value;
}
/**
* {@inheritdoc}
*/
public function setValue($value)
{
$this->value = $value;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$meta = new \Acme\AcmeDemoBundle\Document\Meta();
$meta->setKey("title");
$meta->setValue("Some value");
$content->setMeta($meta);
$manager->persist($content);
$manager->commit();
When you try to get an existing document an error is fired
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
Fatal error: Argument 1 passed to Acme\AcmeDemoBundle\Document\Content::setMeta() must be an instance of Acme\AcmeDemoBundle\Document\Meta, instance of ONGR\ElasticsearchBundle\Result\ObjectIterator given.
I think that the problem is that the rawData must be converted to a document in this line https://github.com/ongr-io/ElasticsearchBundle/blob/master/Result/Converter.php#L90. Maybe a call to $value = $value->current(); fix that
For example, for the given document
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$manager->persist($content);
$manager->commit();
When you try to update an existing document you never can update it
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
$content->setName("Another Name");
$manager->persist($document); // The persist method never send an "update" bulk operation to elasticsearch
$manager->commit();
And look in the index
curl -XGET localhost:9200/my_index/my_type/_search?pretty
The content never change. I think that the problem is in the persist method of the Manager class https://github.com/ongr-io/ElasticsearchBundle/blob/master/ORM/Manager.php#L103-L107 that always sends the "create" operation. The Connection class never send any update option upsert, doc_as_upsert ... in the bulk method https://github.com/ongr-io/ElasticsearchBundle/blob/master/Client/Connection.php#L93-L115
Hi,
My symfony2 bundle is DemoBundl, but i want to customize the document_dir path set to DomoBundle/Search/Document. plz tell me how to configure?
This would be useful if tests are run on older versions when some features are not supported. We could skip them by knowing from documentation if they are supported by current running elasticsearch version or not.
Related: #60
Use these documents for set and get data.
For the given document
<?php
namespace Acme\AcmeDemoBundle\Document;
use DateTime;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* @var string
*
* @ES\Property(name="createdAt", type="date")
*/
private $createdAt;
/**
* Constructor.
*/
public function __construct()
{
$this->createdAt = new DateTime();
}
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
/**
* {@inheritdoc}
*/
public function getCreatedAt()
{
return $this->createdAt;
}
/**
* {@inheritdoc}
*/
public function setCreatedAt(DateTime $createdAt)
{
$this->createdAt = $createdAt;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$manager->persist($content);
$manager->commit();
When you try to get an existing document an error is fired
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
Fatal error: Argument 1 passed to Acme\AcmeDemoBundle\Document\Content::setCreatedAt() must be an instance of DateTime, array given.
In the elasticsearch index the createAt field has a array value, not a date value.
see
ElasticsearchBundle/Tests/Functional/ORM/RepositoryTest.php
Lines 235 to 239 in 1fc70e1
Index import isn't working with data set sizes of 1000 and its multiples, cause of the bulk size.
With data sets of sizes like 1001 or 999 it works as expected.
[Elasticsearch\Common\Exceptions\BadRequest400Exception]
{"error":"ElasticsearchParseException[Failed to derive xcontent from org.elasticsearch.common.bytes.BytesArray@1]","status":400}
Something is wrong with this logic.
This invalidation is wrong
Converted and document arrays keys are being checked 1:1. So if converted array has values after which is being unset, they all will shift.
Hey,
I am using a multifield to have the raw version of a field available for aggregations. I annotate the field in my document like this:
/**
* @var string
*
* @ES\Property(name="manufacturer", type="string", analyzer="simple", fields={@ES\MultiField(name="raw", type="string", index="not_analyzed")})
*
*/
private $manufacturer;
(ES
is mapped to ONGR\ElasticsearchBundle\Annotation
)
When I am trying to use the manufacturer.raw
field in my aggregation i get an empty result. The analyzed manufacturer
field works, but of course in an analyzed way, which is not what I want.
My kibana installation doesn't seem to know anything about the manufacturer.raw
field, either.
Am I missing something here?
Version of the bundle used: v0.2.0
Add the ability for mapping tool to compare tree levels by parsing precision.
Elasticsearch parses precision on mapping put, so the precision returned is actually tree levels.
To compare new mapping to the old one we need to parse it, since the old mapping is return by Elasticsearch and the new mapping is collected from documents. A few ideas of mine on how to solve this:
Create in cache virtual document classes with setters and getters. If document class already have setter or getter, use it if not create virtual.
$params was never initialized. Although not strictly required by PHP, it is generally a good practice to add $params = array(); before regardless.
Implement PrePersist and PreUpdate annotations. They should be written above methods which:
/Client/Connection.php two functions have identical documentation.
/**
* Send refresh call to index.
*/
public function refresh()
{
$this->client->indices()->refresh();
}
/**
* Send refresh call to index.
*/
public function flush()
{
$this->client->indices()->flush();
}
$foo = $repository->findBy(['foo' => 'a']); // Generates empty result set, nothing was found.
$foo->current();
Generates following notice:
Notice: Undefined index: in /var/www/vendor/ongr/elasticsearch-bundle/Result/AbstractResultsIterator.php line 96
There should be no notice, error or empty result is preferred.
Tests fails with elasticsearch 1.2.4 because this version does not have top_hits aggregation type.
Member variable "index_analyzer" is not in valid camel caps format
Member variable "saerch_analyzer" is not in valid camel caps format
ElasticsearchBundle/Annotation/MultiField.php
ElasticsearchBundle/Annotation/Property.php
Make sure that proper query structure is formed for each operation.
f.e. now update operation wouldn't execute.
There are some documents, which need fields like createdAt or updatedAt. There are useful extensions for doctrine, which we could remake for ESB and use:
https://github.com/l3pp4rd/DoctrineExtensions/blob/master/doc/timestampable.md
https://github.com/l3pp4rd/DoctrineExtensions/blob/master/doc/sluggable.md
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.