jeskew / amazon-es-php Goto Github PK

View Code? Open in Web Editor NEW

94.0 94.0 30.0 22 KB

An AWS Signature V4-signing handler for Elasticsearch-PHP

License: Apache License 2.0

PHP 100.00%

amazon-es-php's People

Contributors

Stargazers

Watchers

amazon-es-php's Issues

Error retrieving credentials from the instance profile metadata server. (Client error: `GET http://169.254.169.254/latest/meta-data/iam/security-credentials/` resulted in a `404 Not Found`

Hi
I got this error while connecting AWS Managed Elasticsearch. But My code does not provide any error on my local environment. I have shared my both environment specification below:

Local Environment:

Mac OS Catelina

AWS CLI version:

aws-cli/2.0.31 Python/3.7.4 Darwin/19.3.0 botocore/2.0.0dev35

aws-php-sdk:

"aws/aws-sdk-php": "3.90.6",

"jsq/amazon-es-php": "^0.3.0",

Dev Environment:

AWS EC2
Ubuntu 18.04.2 LTS (GNU/Linux 5.3.0-1019-aws x86_64)

AWS CLI version:

aws-cli/1.18.69 Python/3.6.9 Linux/5.3.0-1019-aws botocore/1.16.19

aws-php-sdk:

"aws/aws-sdk-php": "3.90.6",
"jsq/amazon-es-php": "^0.3.0",

I have check the aws credentials both of local and dev server using cat ~/.aws/credentials and they are the same.

[default]
aws_access_key_id =  xxxxxx
aws_secret_access_key = xxxxxxxxxxxx

I have added both of my local and dev server public ip on elasticsearch access policy.

Below I have added my PHP code:

Service class for getting client:

  class ElasticSearchService
{
    public function getElasticClient(){
        $provider = CredentialProvider::defaultProvider();
        $handler = new ElasticsearchPhpHandler('ap-southeast-1', $provider);
        $client = ClientBuilder::create()
            ->setHandler($handler)
            ->setHosts(['https://xxxx-xxxxx-xxxxxxxxxxxxxxx.ap-southeast-1.es.amazonaws.com:443'])
            ->build();
        return $client;
    }
}

Controller:

class SearchController extends Controller
{
 
   public function __construct(ElasticSearchService $elasticSearchService){
       $this->elasticSearchService = $elasticSearchService;
   }

public function  search(Request $request){
   try {
       $client = $this->elasticSearchService->getElasticClient();
       $content = $request->get("body");
         $parms = [
           "index" => "questions",
           "body" => [
               "size" => 5,
               "query" => [
                   "multi_match" => [
                       "fuzziness" => "AUTO",
                       "fields" => ["body"],
                       "query" => $content,
                       "minimum_should_match" => "50%"
                   ]
               ]
           ]
       ];
       $result = $client->search($parms);
       $data = array();
       foreach($result['hits']['hits'] as $d){
           array_push($data, $d["_source"]);
       }
       return MakeResponse::successResponse($data);
   }catch ( \Exception $exception){
       return MakeResponse::errorResponse("Something Went Wrong", $exception->getMessage());
   }
}

I got success data on my local environment. But on dev environment I got error response where the error message is :

Error retrieving credentials from the instance profile metadata server. (Client error: `GET http://169.254.169.254/latest/meta-data/iam/security-credentials/` resulted in a `404 Not Found` response:\n<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n\t\"http://www. (truncated...)\n)

Help me find my problem.

Release new version - 0.2.4

Hi,

Would you be able to release a version 0.2.4 to packagist? Would be nice to be able to use this fix:
#12

So far we've been running on the forked version, but it would be nice to run on the original package of course 😃

Cheers,

Bas Tuijnman
24i

AWS handler uses single index , how to create multiple index with multiple type and how laravel scout would know in which index need to import type.I m using laravel 5.5 , AWS ES instance 6.2.2 .
Scout only importing one model.
Any idea ?

Exception on newest version of elasticsearch

v5.3.2 causes a "Host must be a string" to be thrown.

$provider = CredentialProvider::fromCredentials(
    new Credentials($_SERVER['AWS_ACCESS_KEY_ID'], $_SERVER['AWS_SECRET_KEY'])
);

$handler = new ElasticsearchPhpHandler($_SERVER['AWS_REGION'], $provider);

$client = ClientBuilder::create()
                ->setHandler($handler)
                ->setHosts(['https://url.com:443'])
                ->build();

$results = $client->search($params);

Reverting to v5.3.0 fixes the issue.

This just started happening a few days ago.

I may eventually submit a PR for a fix, but don't have the time right now.

Create anonymous client

Is it possible to create an anonymous client, as per this example: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_anonymous.html

Any help would be appreciated

use of AWS ECS task iam role

Hi,

Does the bundle support ECS task iam roles as described in the following reference ?
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html

The aws php sdk supports it by looking for the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI env var.

If you implementation relies on the aws php sdk to retrieve credentials, i think it should work out of the box. Did you try?

Unable to set Connect Timeout

We are unable to configure either global or per-request connection timeout when using ElasticsearchPhpHandler.

Without the handler, this works, and connection timeout is correctly triggered after 1 second:

<?php
ClientBuilder::create()
  ->setConnectionParams([
    'client' => [
      'connect_timeout' => 1,
    ]
  ])
  ->setHosts(['http://10.10.3.22:81']) // some address that never responds
  ->build()
  ->get([ 'index' => 'test', 'id' => 1 ]);

But this does not work as expected and connection is timed out after 2 minutes instead:

<?php
ClientBuilder::create()
  ->setConnectionParams([
    'client' => [
      'connect_timeout' => 1,
    ]
  ])
  ->setHandler(new ElasticsearchPhpHandler('us-east-1'))
  ->setHosts(['http://10.10.3.22:81']) // some address that never responds
  ->build()
  ->get([ 'index' => 'test', 'id' => 1 ]);

Same thing when using per-request, this doesn't work, while not specifying a handler does

<?php
ClientBuilder::create()
  ->setHandler(new ElasticsearchPhpHandler('us-east-1'))
  ->setHosts(['http://10.10.3.22:81']) // some address that never responds
  ->build()
  ->get([
    'index'  => 'test',
    'id'     => 1,
    'client' => [
      'connect_timeout' => 1
    ]
  ]);

Is there a way around this? Thanks!

Trailing slashes on hosts cause invalid uri exceptions

I am not sure whether this should be fixed by sanitizing the host uris in https://github.com/elastic/elasticsearch-php or that this library should (also) handle this. But since the exception is thrown in this library I will create the issue in this repository.

We noticed this issue when creating a new Elasticsearch cluster on AWS and copying the endpoint uri from the Amazon Console. That uri contains a trailing slash.

Reproduction scenario

$handler = new \Aws\ElasticsearchService\ElasticsearchPhpHandler(
    'eu-west-1', 
    CredentialProvider::defaultProvider()
);
$client = \Elasticsearch\ClientBuilder::fromConfig([
    'hosts' => ['http://host.docker.internal:9200/'], 
    'handler' => $handler
], true);

$client->info()

This will lead to an InvalidArgumentException with the message "'Unable to parse URI: //'" to be triggered in the creation of a PSR-7 request at

amazon-es-php/src/ElasticsearchPhpHandler.php

Line 63 in b2b5634

$uri = (new Uri($ringPhpRequest['uri']))

The path of the uri is /, to get the information of the host / is added and thus the uri becomes //, which in itself is an invalid uri. I have only seem this issue occur with this operation, since the other operations add something to the path resulting in uris like //my_index. Those uris are excepted by the Uri constructor.

Maybe the best solution is to sanitize the host uris in https://github.com/elastic/elasticsearch-php by removing trailing slashes - I will try to raise an issue at that repository as well - or maybe this should also be handled in this repository.

Edit: issue on elastic/elasticsearch-php: elastic/elasticsearch-php#869

System details

PHP 7.2
jsq/amazon-es-php version 0.2.3 (latest stable)
elasticsearch/elasticsearch version 6.1.0 (latest stable)
OS: standard php:7.2-fpm Docker container

"No alive nodes found in your cluster" when checking for index existence

Hi! The setup I have is identical to what's described under "Basic Usage" in the documentation. The handler is loading credentials from env variables. In my code, I'm checking for index existence by $client->indices()->exists($params). Strangely this results in a Elasticsearch\Common\Exceptions\NoNodesAvailableException with message 'No alive nodes found in your cluster' error.

All other function calls, including creating index, deleting index, searching, adding mapping, work as expected but it's just this existence call that fails. Haven't had too much time to look into this issue, but was wondering if anyone has experienced something similar and if this might be an issue with this library? If I make the same function call against a local ES server and not AWS ES then it works.

Any insight is appreciated. Thanks!

Bug bad signing and 403

http://joxi.ru/52ab8eXiGVR64A
It signing one request and then merges it with original, so this is what creates the problem and really doesn't make sense. Coz, signed request already based on the original. So merging just spoils request

Error with OpenSearch

Hi,

I'm using the same code to add credentials for ES and OpenSearch.
It works great for ES, but fails with OpenSearch:
{“message”:”Authorization header requires ‘Credential’ parameter. Authorization header requires ‘Signature’ parameter. Authorization header requires ‘SignedHeaders’ parameter. Authorization header requires existence of either a ‘X-Amz-Date’ or a ‘Date’ header.}

$provider = CredentialProvider::fromCredentials(
new Credentials( $config['aws_access_key_id'], $config['aws_secret_access_key'] )
);
$handler  = new ElasticsearchPhpHandler( $config['aws_region'], $provider );
$client->setHandler( $handler );

Would you have an idea?

Compatibility with Opensearch-php

Is there any value in making this package compatible with opensearch-php?

Or would it make more sense to fork this into a new repo, modify it and name it specifically for this purpose? As far as I could find there are no implementations offering this for opensearch-php and it is also not included in that library.

Integration directly into Opensearch-PHP

Hey,

I am the maintainer of Opensearch-PHP and we want to implement exactly the same into the SDK directly. As you have invested time already into this topic and made it compatible even with this SDK, do you interest porting it directly into the library?

If not I would look myself how to integrate it properly. Anyway thanks for building this 💙

Undefined index: host

I can see that you've already fixed an issue with AWS Elastic Search, where host was renamed to Host in v5.3.2 🤦‍♂️

Would you be so kind as to add a release for this please?
We depend on your package and ES is no longer working for us now :(

Thanks and great project!

Antonio

ES 6.x Support

500 error when attempting to connect to index

I have the following code, but I keep getting a 500 error whenever I send an index call to the client. I cannot figure out where my issue is:

require_once "../vendor/autoload.php";
use Aws\ElasticsearchService\ElasticsearchPhpHandler;
use Elasticsearch\ClientBuilder;
$handler = new ElasticsearchPhpHandler('us-west-2');
$client = ClientBuilder::create()
    ->setHandler($handler)
    ->setHosts(['https://search-arc-bis-XXXXX.us-west-2.es.amazonaws.com:443'])
    ->build();
$params = array();
$params['body']  = array(
    'name' => 'Todd Coulson'
);
$params['index'] = 'item';
$params['type']  = 'item_books';
$client->index($params);

If I comment out the last line, no error comes up. It seems as though it could be a elasticsearch-php issue, but I cannot figure out where the issue is coming in. I am authenticating through the .aws/credentials file. Can you help me troubleshoot this?