Can I run it in MAC I get the following error about aaia HOT 11 CLOSED

Hey @balavidya , I think it has something to do with following issue :-

neo4j/neo4j#12392

Can you please provide the following :-

i. neo4j version
ii. Output of command pip freeze

from aaia.

The Neo4j version is
Version: 4.0.2
Edition: Community
Name: neo4j
Databases: :dbs
Information: :sysinfo
Query List: :queries

pip freeze output :

awscli==1.16.207
AWSScout2==3.2.1
boto3==1.11.1
botocore==1.12.197
certifi==2019.9.11
chardet==3.0.4
colorama==0.3.9
configparser==4.0.2
docutils==0.14
fpdf==1.7.2
iampoliciesgonewild==1.0.6.2
idna==2.8
jmespath==0.9.4
neo4j==1.7.4
neobolt==1.7.13
neotime==1.7.4
netaddr==0.7.19
numpy==1.17.2
opinel==3.3.4
pandas==0.25.1
plotly==4.1.1
policyuniverse==1.3.2.0
psutil==5.6.3
pyasn1==0.4.5
pyjq==2.3.1
python-dateutil==2.8.0
pytz==2019.1
PyYAML==5.1
requests==2.22.0
retrying==1.3.3
rsa==3.4.2
s3transfer==0.2.1
six==1.12.0
urllib3==1.25.8

from aaia.

It does seems like the same issue as I mentioned earlier.
I am away from my system now and unable to test this.

Can you try editing the code in the below line and let me know :-

1. Aaia/Aaia/actions/aws/load_data/iam.py

   Line 1133 in 249a33c

   neo4j_driver = GraphDatabase.driver( neo4j_uri, auth=neo4j_auth,)

change it to

neo4j_driver = GraphDatabase.driver( neo4j_uri, auth=neo4j_auth,encrypted = False)  

2. Aaia/Aaia/actions/aws/load_data/organizations.py

   Line 329 in 249a33c

   neo4j_driver = GraphDatabase.driver(neo4j_uri, auth=neo4j_auth, )

change it to

neo4j_driver = GraphDatabase.driver( neo4j_uri, auth=neo4j_auth,encrypted = False)  

once the change is done. Try loading the data.

Let me know if this suggested change works for you.

from aaia.

After updating getting the following error

python3 Aaia.py -n default -a load_data
INFO:actions.aws.load_data.iam:[*] Loading AWS Managed Policies into neo4j instance for AWS account 'default'
Traceback (most recent call last):
File "Aaia.py", line 19, in
action.main(config,args)
File "Aaia/Aaia/actions/aws/main.py", line 28, in main
library.main(config,args)
File "Aaia/Aaia/actions/aws/load_data/main.py", line 14, in main
library.main(config, args)
File "/Aaia/Aaia/actions/aws/load_data/iam.py", line 1194, in main
loadAWSIAM(neo4j_uri,neo4j_user,neo4j_password,data_path,account_name)
File "/Users/bakannan/Aaia/Aaia/actions/aws/load_data/iam.py", line 1136, in loadAWSIAM
loadAWSManagedPolicies(neo4j_session,data_path,account_name)
File "/Aaia/Aaia/actions/aws/load_data/iam.py", line 92, in loadAWSManagedPolicies
neo4j_session.run(ingest_aws_managed_policies,Arn=policy['Arn'],AccountNo=policy['Arn'].split(":")[4],AttachmentCount=policy['AttachmentCount'],CreateDate=policy['CreateDate'],DefaultVersionId=policy['DefaultVersionId'],IsAttachable=policy['IsAttachable'],Path=policy['Path'],PermissionsBoundaryUsageCount=policy['PermissionsBoundaryUsageCount'],PolicyId=policy['PolicyId'],PolicyName=policy['PolicyName'],Description=policy['Description'],UpdateDate=policy['UpdateDate'])
File "/usr/local/lib/python3.7/site-packages/neo4j/init.py", line 499, in run
self._connection.fetch()
File "/usr/local/lib/python3.7/site-packages/neobolt/direct.py", line 422, in fetch
return self._fetch()
File "/usr/local/lib/python3.7/site-packages/neobolt/direct.py", line 464, in _fetch
response.on_failure(summary_metadata or {})
File "/usr/local/lib/python3.7/site-packages/neobolt/direct.py", line 759, in on_failure
raise CypherError.hydrate(**metadata)
neobolt.exceptions.CypherSyntaxError: The old parameter syntax {param} is no longer supported. Please use $param instead (line 1, column 27 (offset: 26))
"merge (A:AWSPolicy {Arn :{Arn} } )"
^

from aaia.

I have made changes in the code which should solve the issue.

@balavidya please pull the code again and test it and let me know.

from aaia.

HI cloned the project and re-ran the load data, it started to run I got the following errors.

python3 Aaia.py -n default -a load_data

INFO:actions.aws.load_data.iam:[] Loading AWS Managed Policies into neo4j instance for AWS account 'default'
INFO:actions.aws.load_data.iam:[] Completed loading AWS Managed Policies into neo4j instance for AWS account 'default'
INFO:actions.aws.load_data.iam:[*] Loading AWS Users into neo4j instance for AWS account 'default'
Traceback (most recent call last):
File "Aaia.py", line 19, in
action.main(config,args)
File "/Aaia/Aaia/actions/aws/main.py", line 28, in main
library.main(config,args)
File "Aaia/Aaia/actions/aws/load_data/main.py", line 14, in main
library.main(config, args)
File "/Aaia/Aaia/actions/aws/load_data/iam.py", line 1220, in main
loadAWSIAM(neo4j_uri,neo4j_user,neo4j_password,data_path,account_name)
File "//Aaia/Aaia/actions/aws/load_data/iam.py", line 1165, in loadAWSIAM
loadAWSUserNodes(neo4j_session,data_path,account_name)
File "/Aaia/Aaia/actions/aws/load_data/iam.py", line 385, in loadAWSUserNodes
account_number=getAWSAccountNo(data_path,account_name)
File "/Aaia/Aaia/actions/aws/load_data/iam.py", line 1088, in getAWSAccountNo
account_number=getAWSIamAccountAuthorizationDetailsInfo(data_path,account_name,jqQuery)[0].split(":")[4]
AttributeError: 'NoneType' object has no attribute 'split'

from aaia.

The line

assumes that there is atleast one user in your AWS account from whose arn Account No is parsed.

It seems logically that you dont have any Users in AWS account which is ultimately not reflecting in iam-get-account-authorization-details.json file.

Thats the reason the below query doesnt fetch anything .

So does you AWS account does not have any IAM users ?

Please note that the original issue that you have raised is resolved with the commit above. Hence am closing this issue. However , feel free to raise a new issue with this regard.

from aaia.

Also commited the fix for this in a recent commit. Try to pull it and run. FIxed getting the Account No from STS get caller identity and not from a user arn.

from aaia.

from aaia.

Anytime @balavidya . Thanks for checking it out . Also request you to run the collector script once again and then test loading into Aaia, as I have added some additional data(i.e sts caller identity) to be collected to solve the issues. Don't forget to do that.

Cheers !!

from aaia.

from aaia.