unclear instructions about aws-glue-samples HOT 16 CLOSED

oh i think the problem is the IAM role with the glue managed policy gives access to
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::crawler-public*",
"arn:aws:s3:::aws-glue-*"
]
},
but your sample bucket starts with "awsglue-" so it doesn't match

from aws-glue-samples.

Thanks for pointing this out!

We will fix this in the documentation.

from aws-glue-samples.

I just ran into the same problem. AWS managed policy AWSGlueServiceRole provides access to "arn:aws:s3:::aws-glue-/". Would it be possible to move or copy data samples from s3://awsglue-datasets/examples to s3://aws-glue-datasets/examples?

from aws-glue-samples.

Running into same error as angelarw.
[49eaba68-d5c2-4245-803a-6ec9736422ea] ERROR : Error Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 8FF7A6AB88E7D7FA; S3 Extended Request ID: Wqplw7A9+PEVI5Cpg0hCwgm7UPEstv3Vbj+7sWDEo1/+6GaiFff2TE471wci3QR+yhZeO7NEtqA=) retrieving file at s3://awsglue-datasets/examples/us-legislators/all/persons.json. Tables created did not infer schemas from this file.

poorly done examples/instructions.

from aws-glue-samples.

I also had the same problem but it turned out that when you select the path for their bucket you have to specify that it an external path.

from aws-glue-samples.

I have the same as well, do we have to make to s3 bucket public to access make the bucket accessible to crawler?

from aws-glue-samples.

You can give "Full S3 Access" to Crawler IAM role to check if it works. Of course, after that you could secure policy as per your need. It worked for me.

from aws-glue-samples.

i am unable to crawler the json files and unable to create the tables

from aws-glue-samples.

@tejamuddada : what's the exact error?

from aws-glue-samples.

when i am trying to load the json files it is not creating exact no of tables as mentioned in example. It is not throwing an error .

from aws-glue-samples.

Had the same issue,
Fixed it by creating a custom policy by replacing references to "aws-glue" with "awsglue" and attaching to the role.

from aws-glue-samples.

also there's a typo in the s3 path (you have a dot at the end)
"s3://awsglue-datasets/examples/us-legislators."

Fixed this in c35f722.

Currently the developer guide (https://docs.aws.amazon.com/glue/latest/dg/console-crawlers.html) explains how to setup IAM role correctly for crawler. It already covers the issue reported here.

Resolving.

from aws-glue-samples.

Just want to share the tutorial where you could point the s3://awsglue-datasets/examples/us-legislators/all using AWS Glue Crawlers

4:48 of youtube.com/watch?v=zdaVBXfmSs0&t=5s

from aws-glue-samples.

You can give "Full S3 Access" to Crawler IAM role to check if it works. Of course, after that you could secure policy as per your need. It worked for me.

It works

from aws-glue-samples.

Writing a note that I had to figure this out from https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-samples-legislators.html which was a bad experience.

from aws-glue-samples.

I added the following to my datadest inline policy to get access to the public S3 bucket from the example:

"arn:aws:s3:::awsglue-datasets/examples/us-legislators/all/*"

That fixed it for me.

from aws-glue-samples.