local execution of aws glue about aws-glue-samples HOT 11 CLOSED

Hi @BardiaAfshin
I'm facing the same issue. Did you manage to resolve it?
Thanks.

Cheers x

from aws-glue-samples.

No, I have a support ticket from AWS, they basically came back with "this is proprietary issue", but I think that cannot be right answer. You can get their scala package working locally.

from aws-glue-samples.

I am also facing the same issue. Could Aws team provide any solution @BardiaAfshin ?

from aws-glue-samples.

Any answers about this issue?

from aws-glue-samples.

Any solution to this?? I am getting same issue.. i am trying to run aws glue on my ubuntu system..

ypeError Traceback (most recent call last)
in
----> 1 glueContext = GlueContext(sc)

~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in init(self, sparkContext, **options)
43 super(GlueContext, self).init(sparkContext)
44 register(sparkContext)
---> 45 self._glue_scala_context = self._get_glue_scala_context(**options)
46 self.create_dynamic_frame = DynamicFrameReader(self)
47 self.write_dynamic_frame = DynamicFrameWriter(self)

~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in _get_glue_scala_context(self, **options)
64
65 if min_partitions is None:
---> 66 return self._jvm.GlueContext(self._jsc.sc())
67 else:
68 return self._jvm.GlueContext(self._jsc.sc(), min_partitions, target_partitions)

TypeError: 'JavaPackage' object is not callable

from aws-glue-samples.

Recently we have updated the maven repository to solve issues for local development.
Can you try again and check if you still see the issue?

from aws-glue-samples.

Today I tried it on the latest Amazon Linux 2 and on my Macbook, and it worked without any errors.

If you still the error, can you describe details?

• Your environment (OS, version, etc.)
• How to replicate the issue
  • e.g. Starting "gluepyspark" command, putting the two lines of code
• Glue version: 1.0 or 0.9
• Check if you followed correct procedures
  • git checkout glue-1.0 (for Glue 1.0)
  • Used correct version of Spark package

from aws-glue-samples.

Thank you @moomindani - In my case, I had the jarsv1 not landing in the correct folder, but once that was fixed, I was able to run the job locally with no issue.

from aws-glue-samples.

Thank you @moomindani - In my case, I had the jarsv1 not landing in the correct folder, but once that was fixed, I was able to run the job locally with no issue.

that is correct if glue jars were not available in the classpath you will see this error.

from aws-glue-samples.

I followed instructions from aws docs - I see that in awslabs there is aws glue repo that has better instructions. BUT - for those wondering what jarsv1 is - like myself coming from aws docs, I followed these instructions and got it to work locally.

https://support.wharton.upenn.edu/help/glue-debugging#update-path-and-java-home-

from aws-glue-samples.

It seems that you are not seeing the issue now.
Please reopen this issue if you still see the issue.

from aws-glue-samples.