Configuration issues with ES 2.2.1 about elasticsearch-readonlyrest-plugin HOT 27 CLOSED

Cool, can you paste your conf so I can reproduce the issue? Kibana version is needed too. Thanks for taking the time of reporting this, seriously helps!

Btw, is this issue affecting plugin version 1.9.1?

from elasticsearch-readonlyrest-plugin.

Ahhh, I rolled back. Let me deploy again. Which conf's should I paste? I have elasticsearch.yml, kibana.yml and nginx-kibana.conf

I can try plugin version 1.9.1 if it is compatible with ES 2.2.1...

from elasticsearch-readonlyrest-plugin.

elasticsearch.yml please :)

from elasticsearch-readonlyrest-plugin.

Oh, I can give that to you without redeploying (also what I added to it to support this plugin), one sec

from elasticsearch-readonlyrest-plugin.

elasticsearch.yml ::

cluster.name: TrainingProd
node.name: "Training1 - Wolverine"

path.conf: /data/elk-conf
path.data: /data/elasticsearch_data
path.logs: /data/elasticsearch_data/elastic_logs

bootstrap.mlockall: true

network.bind_host: 0.0.0.0
network.publish_host: 10.123.210.30
transport.tcp.port: 9300
http.port: 9200

gateway.recover_after_nodes: 3
gateway.recover_after_time: 5m
gateway.expected_nodes: 4

discovery.zen.minimum_master_nodes: 3
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["10.123.210.30", "10.123.210.31", "10.123.210.32"]

cloud:
azure:
storage:
account: "straz12"
key: "ffNXo6/f9vjknVeu5pnqDSkldjdDJSkjdskjSDDLSdklsSkj00aAov4PnANFXLxIhULw=="

#Box Type:
node.box_type: hot

Added to support readonlyrest ::

readonlyrest:
enable: true
response_if_req_forbidden: <.h1.>Forbidden<./h1.>
access_control_rules:

- name: Readers
  type: allow
  kibana_access: ro
  auth_key: readers:test123

- name: Managers
  type: allow
  kibana_access: ro+
  auth_key: manager:test456

- name: Admin
  type: allow
  kibana_access: rw
  auth_key: admin:test789

from elasticsearch-readonlyrest-plugin.

Unlike how it appears above, I do have the proper indenting, per your instructions :)

from elasticsearch-readonlyrest-plugin.

OK, have you set one of those HTTP credentials inside your kibana.yml?

from elasticsearch-readonlyrest-plugin.

Kibana.yml -

port: 5601
host: "0.0.0.0"

elasticsearch_url: "http://10.123.210.30:9200"
elasticsearch_preserve_host: true

kibana_index: ".kibana"

default_app_id: "discover"

request_timeout: 600000

shard_timeout: 0

verify_ssl: true

bundled_plugin_ids:

• plugins/dashboard/index
• plugins/discover/index
• plugins/doc/index
• plugins/kibana/index
• plugins/markdown_vis/index
• plugins/metric_vis/index
• plugins/settings/index
• plugins/table_vis/index
• plugins/vis_types/index
• plugins/visualize/index

from elasticsearch-readonlyrest-plugin.

lol @the markdown parser going crazy with unescaped yaml 😁

Anyway: you're missing the configuration of Kibana to use the HTTP credentials. Add these to your conf/kibana.yml

  elasticsearch.username: "admin"
  elasticsearch.password: "test789"

from elasticsearch-readonlyrest-plugin.

I shall go test! Will post back shortly

from elasticsearch-readonlyrest-plugin.

OK, I redeployed the same way as I did before, all of the same steps. I am able to authenticate with the different users and access Kibana, but one thing - using the 'Readers' user, I am able to add/remove index patterns from Kibana. Shouldn't that account not have any privileges like that in Kibana or?

Thanks for your help by the way!

from elasticsearch-readonlyrest-plugin.

I may have a thought about that......this is permissions for the REST API from ES not anything within Kibana

from elasticsearch-readonlyrest-plugin.

Simone - potentially bad news - I was able to update a document via Fiddler using HTTP POST when I specified the Authorization Header of the 'readers' user ;/

from elasticsearch-readonlyrest-plugin.

Ahh, so I need to supply 'methods' in the elasticsearch.yml file or else everything can do all ?

such as for readers, "methods: GET" so they can only utilize the REST API towards the ES via GET and no other methods?

UPDATE: Although adding "methods: GET" works for readers to prevent them from using anything other than GET calls, now that account can't login to Kibana, it just loops through the authentication prompt. Removing the 'methods: GET' allowed the user to access Kibana again (but unfortunately now they are back to being able to make calls besides GET)

from elasticsearch-readonlyrest-plugin.

Here is my current elasticsearch.yml -

readonlyrest:
enable: true
response_if_req_forbidden: <.h1.>Forbidden<./h1.>
access_control_rules:

- name: Readers
  type: allow
  methods: GET
  kibana_access: ro
  auth_key: readers:read123

- name: Managers
  type: allow
  kibana_access: ro+
  auth_key: manager:manage123

- name: Admin
  type: allow
  kibana_access: rw
  auth_key: admin:woot123

What I cannot currently do -

1. Log into Kibana with 'readers' - the authentication window loops
2. Use fiddler to attempt a 'POST' using the authorization header for 'readers'

What I can currently do -

1. Log into Kibana with 'manager' and 'admin'
2. Use fiddler to 'GET' using the authorization header for 'readers'
3. Use fiddler to 'POST' using the authorization header for 'admin'

What I need to be able to do -

1. Log into Kibana with 'readers' and NOT be able to add/remove index patterns
2. Anything that involves creating, updating or deleting with 'readers'

Hope this helps

from elasticsearch-readonlyrest-plugin.

I tried to reproduce the document update with cURL and it does not work.

I suspect you tried to contact ES from the same browser in which you were logged in, which automatically sent the authorization headers also from the Fiddler (it's a web app, right?).

I suggest you remove the methods rule (as Kibana needs to use POST) and try the same experiment you did of document update using the Fiddler from an incognito window (or even better, try the update from cURL).

This was my test:

1. with redonly rest disabled

curl -X PUT -d '{
    "counter" : 1,
    "tags" : ["red"]
}' "http://localhost:9200/test/type1/1"

Enable readonly rest plugin and try to update the document:

curl -X POST  -d '{
    "doc" : {
        "name" : "new_name"
    },
    "doc_as_upsert" : true
}' "http://localhost:9200/test/type1/1/_update"

I get a huge Forbidden response as I should.

from elasticsearch-readonlyrest-plugin.

I am using IE from an 'InPrivate' session. Not sure what the difference is since it is the same Authorization Header as it was when it was not an 'InPrivate' session.

I was still able to GET and POST using 'readers'

Are you using cURL from Windows? I guess that is the only other method of testing at this moment....

from elasticsearch-readonlyrest-plugin.

Damn I still can't reproduce this myself. This is my experiment, see if you'd do something different:

My ES 2.2.1 configuration

readonlyrest:
    enable: true

    response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin

    access_control_rules:

    - name: browser
      type: allow
      auth_key: user:pass
      kibana_access: ro

I have a document called 1 of mapping "type1" in the index "test".

curl -X GET  -H "Authorization: Basic dXNlcjpwYXNz" "http://localhost:9200/test/type1/1/"

{
  "_index": "test",
  "_type": "type1",
  "_id": "1",
  "_version": 2,
  "found": true,
  "_source": {
    "counter": 1,
    "tags": [
      "red"
    ],
    "name": "new_name"
  }
}

If I attempt to update it with a POST authenticated as "ro", Still I get forbidden.

curl -X POST -H "Authorization: Basic dXNlcjpwYXNz"  -d '{
    "doc" : {
        "name" : "new_name1"
    },
    "doc_as_upsert" : true
}' "http://localhost:9200/test/type1/1/_update"

Forbidden by ReadonlyREST ES plugin

Can you share an example of what document and how you are able to update an existing document? If you repeat the experiment in Chrome, it lets you copy the request as cURL command, which would be really valuable for me to debug this.

Copy as cURL (Chrome) http://www.lornajane.net/posts/2013/chrome-feature-copy-as-curl
Copy as cURL (Firefox/Firebug) https://hacks.mozilla.org/2013/08/firebug-1-12-new-features/#copyAsCURL

from elasticsearch-readonlyrest-plugin.

Simone -

I have an index which contains logs forwarded from an application. I am attempting to update the "message" field within a particular document to something like "Test". I am doing this by using Fiddler.

I realize you need an authorization header. To collect the authorization header from the read only user, I first opened up an InPrivate browsing session of IE (since my Fiddler isn't recording traffic from Chrome for some strange reason), then browsed to Kibana, logged in using the read only account 'readers', then checked Fiddler. I found the authorization header, and see that it converts to the 'readers' user (therefore confirming it is the correct authorization header).

Then, I select the 'Composer' tab in Fiddler, and use the following:

POST - http://10.123.210.30:9200/app-sp-2016.04.12/ULS/AVQLTAEw3Lk30-OY10Rx

10.123.210.30:9200 - instance 0 of Elasticsearch (I have four instances, using the first one to test)
app-sp-2016.04.12 - index
ULS - type
AVQLTAEw3Lk30-OY10Rx - document

Headers:

Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Fiddler
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: 10.123.210.30:9200
Authorization: Basic cmVhZAVyczpyMzAkM4J6Wg==
Content-Length: 37

cmVhZAVyczpyMzAkM4J6Wg== is the authorization header for the 'readers' user

Request Body:

{ "message": "Using Reader Again 3 or some other message" }

"message" is a field within the document.

When I click 'Execute' in Fiddler, it is successful. I can then initiate a 'GET' to the same URI, and see the updated "message" field in the response.

My concern - using the authorization header for the "readers" user allowed me to successfully "POST" to update the "message" field within the said document.

Make more sense?

from elasticsearch-readonlyrest-plugin.

I am not sure how I can repeat the experiment in Chrome as I am updating a field within an existing document and I am not too sure you can do that from the Kibana UI. But I did learn the way to retrieve the cURL command via Developer Tools in Chrome.

from elasticsearch-readonlyrest-plugin.

@reqless your description is nicely detailed, thank you. I tried to replicate your experiment as closely as I could (besides not having a windows machine to use fiddler).

It would be interesting to see the TRACE logs of elastic search, and grep the lines containing "checking request" so we could observe what's the action field that is being allowed to pass.

In my environment I read this:

[2016-04-13 17:25:42,021][TRACE][plugin.readonlyrest.acl  ] checking request:{ action: indices:data/write/index OA:/0:0:0:0:0:0:0:1:54295 M: POST}{
    "name" : "new_name1"
}
[2016-04-13 17:25:42,023][DEBUG][plugin.readonlyrest.acl  ] Discovered indices: test
[2016-04-13 17:25:42,023][DEBUG][plugin.readonlyrest.acl.blocks.rules.impl] KIBANA ACCESS DENIED { action: indices:data/write/index OA:/0:0:0:0:0:0:0:1:54295 M: POST}{
    "name" : "new_name1"
}

When I try to post the change to the document.

from elasticsearch-readonlyrest-plugin.

Hello Simone,

I have now tried to use the Google Chrome extension 'postman'. I was able to post again....this time I was on a different machine which has never accessed Kibana before. I was able to GET and POST using the 'readers' account.

I then cleared the browser history completely, passwords, cache, everything....opened up postman, entered the URI, selected POST, entered request body, initiated, it asked for credentials, I typed 'readers' account with password, clicked 'OK', came back successful.

from elasticsearch-readonlyrest-plugin.

Also, the field within the document I am updating is named 'message'.

I last updated it with this request body : "{ "message": "Using Reader Again 6" }"

I searched elasticsearch logs and grep'd keywords like 'again', 'Again', 'Using', 'Using Reader Again 6' with absolutely no results on that node.

from elasticsearch-readonlyrest-plugin.

Here is a log from ES when I made that POST request successfully :

[2016-04-13 18:13:15,904][INFO ][plugin.readonlyrest.acl ] checking request: { OA:/10.123.208.30:51257 M: POST }
[2016-04-13 18:13:15,904][INFO ][plugin.readonlyrest.acl ] Block Readershas matched: Readers match: true}

Then it just continues on...no other significant logs after that

from elasticsearch-readonlyrest-plugin.

To be super sure - I'd double check the rules are all interpreted correctly (and there's no indentation issues) by grepping the logs again and finding something like this:

[2016-04-13 22:24:01,116][INFO ][plugin.readonlyrest      ] [Akhenaten] Readonly REST plugin was loaded...
[2016-04-13 22:24:01,116][INFO ][plugin.readonlyrest      ] [Akhenaten] Readonly REST plugin is enabled. Yay, ponies!
[2016-04-13 22:24:01,141][INFO ][plugin.readonlyrest.acl  ] ADDING readonlyrest Rules Block :: { name: 'browser', policy: ALLOW}

Notice the ADDING line which lists all the rules in the block.

If that is not an issue, at this point the only idea I have is that the request is being forwarded to one of the nodes without the plugin installed (maybe the shard containing that document lives there!).

In order to verify to this: either you repeat the experiment on a single node, or you install the plugin and edit the plugin configuration in all the other nodes.

from elasticsearch-readonlyrest-plugin.

I executed the following on all four ES nodes ::

sudo docker exec -it elasticsearch plugin list

Result on all four nodes ::

Installed plugins in /usr/share/elasticsearch/plugins:
- readonlyrest

I am pretty confident in saying I installed readonlyrest on all of the nodes =)

Next, I checked elasticsearch.yml on each node, and I do verify the settings are in each.

Please see a link below for a screenshot from putty (to prove my indentation)::

http://i.imgur.com/gVxXtDK.png

from elasticsearch-readonlyrest-plugin.

@reqless this clearly needs more investigation, sorry I could not get to the bottom of this. But I want to come back to you after I'll have added some coverage of multi node environment. Let's track this specific thing in a more specific, separate issue: #52

from elasticsearch-readonlyrest-plugin.