Hi awsx-Team,

I am getting the following issue:

`[*] Importing records from /opt/awspx/data/20200616201342551729_default.zip
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 935, in _connect
s.connect(resolved_address)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/awspx/cli.py", line 466, in
main()
File "/opt/awspx/cli.py", line 461, in main
args.func(args)
File "/opt/awspx/cli.py", line 193, in handle_ingest
handle_db(args)
File "/opt/awspx/cli.py", line 235, in handle_db
(success, message) = Neo4j.load(args.load_zip, db)
File "/opt/awspx/lib/graph/db.py", line 93, in load
Neo4j.stop()
File "/opt/awspx/lib/graph/db.py", line 70, in stop
if not Neo4j.isavailable():
File "/opt/awspx/lib/graph/db.py", line 43, in isavailable
auth=(Neo4j.username, Neo4j.password)
File "/usr/local/lib/python3.7/dist-packages/neo4j/init.py", line 181, in driver
return cls.bolt_driver(parsed.netloc, auth=auth, **config)
File "/usr/local/lib/python3.7/dist-packages/neo4j/init.py", line 194, in bolt_driver
return BoltDriver.open(target, auth=auth, **config)
File "/usr/local/lib/python3.7/dist-packages/neo4j/init.py", line 357, in open
pool = BoltPool.open(address, auth=auth, pool_config=pool_config, workspace_config=default_workspace_config)
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 510, in open
seeds = [pool.acquire() for _ in range(pool_config.init_size)]
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 510, in
seeds = [pool.acquire() for _ in range(pool_config.init_size)]
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 523, in acquire
return self._acquire(self.address, timeout)
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 395, in _acquire
connection = self.opener(address, timeout)
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 507, in opener
return Bolt.open(addr, auth=auth, timeout=timeout, **pool_config)
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 195, in open
keep_alive=pool_config.keep_alive,
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 1055, in connect
raise last_error
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 1045, in connect
s = _connect(resolved_address, timeout, keep_alive)
File "/usr/local/lib/python3.7/dist-packages/neo4j/io/init.py", line 949, in _connect
raise ServiceUnavailable("Failed to establish connection to {!r} (reason {})".format(resolved_address, error))
neo4j.exceptions.ServiceUnavailable: Failed to establish connection to IPv6Address(('::1', 7687, 0, 0)) (reason [Errno 99] Cannot assign requested address)`

I have tried to start the container with a different port but no idea why IPv6 instead 127.0.0.1 is used. Any idea how this can be changed?

Thanks!

I've tried to comment out class_s3ingestor section, but it still runs and its taking forever to scan a decent S3 environment. Any suggestion to run this without S3 ingestor and will future integration include thread module to speed up the process?

Hi, thanks for this project!

The INSTALL script pulls from a pure neo4j container and then installs a bunch of stuff. That slows deployment and means that it could break at anytime as libs change. Is there any reason you don't use a Dockerfile to bake all the necessaries into the project? Would you accept a PR for one?

Hi,

Could you provide a Docker image for arm64 that would work on newer Apple M1 chips please? It seems like the image available is amd64 and I'm not able to get it working on a more recent Mac. Thanks.

Need assistance in resolving the following error when trying to ingest IAM data. I already increased the 'dbms.memory.heap.max_size' in the neo4j.conf file within the AWSPX container but no success.

TransientError: {code: Neo.TransientError.General.OutOfMemoryError} {message: There is not enough memory to perform the current task. Please try increasing 'dbms.memory.heap.max_size' in the neo4j configuration (normally in 'conf/neo4j.conf' or, if you
you are running an embedded installation increase the heap by using '-Xmx' command line flag, and then restart the database.}

Verifying issue before reposting it

Autocomplete icons load only once an item has been selected when using firefox. Chip icons load after some delay.

Hi!

First, thanks for the tool.

I exported my env variables for STS tokens, and they are working fine, ie. if I do aws s3 ls or aws sts get-caller-identity, they work as expected (without having anything under .aws/config or .aws/credentials), in my host machine.

However, when I run sudo awspx ingest --env, it fails with the following output:
→ Awaiting response to sts:GetCallerIdentity
... bunch of output in table-format
NoCredentialsError: Unable to locate credentials

Since I would like to use env variables, I have not created any profiles, therefore the following command has no output:
sudo awspx profile --list

I also ran sudo docker exec -it awspx bash , and did env as well as trying to run a command such as aws s3 ls, it says unable to locate credentials. It seems that the STS tokens are not being passed to the docker container...

Am I doing anything wrong?

Any plans to support mac?

Author of CloudMapper here. First, awesome work! I just started looking at your project and the code looks very clean (especially in comparison to many of the other AWS security project code bases, ex. CloudMapper :) ). In your post you mentioned the use of AWS tag editor and that there were short-comings with that. I haven't seen that approach used before, and was surprised to not find a hundred List and Describe calls as is seen with other projects that try to identify all of the resources in an account. Where are you actually making the calls to get the resources for the account? I saw https://github.com/FSecureLABS/awspx/blob/136e64a2dc2f4abd931907dcbafe4b14663d4c64/cli.py#L235 which I believe then calls into this https://github.com/FSecureLABS/awspx/blob/136e64a2dc2f4abd931907dcbafe4b14663d4c64/lib/aws/ingestor.py#L802
But I wasn't able to follow how that collection actually happens? Also, what were the short-comings you ran into?

Also, there are some projects that have some similarities with this one that you might be interested in.

• Lyft's cartography has explored trying to collect all the resources in an account graph and make those queryable. You can see for example here how it does some parsing of resource policies on S3 buckets: https://github.com/lyft/cartography#which-s3-buckets-have-a-policy-granting-any-level-of-anonymous-access-to-the-bucket
• Duo's parliament (which I developed) parses the AWS docs for information about IAM privileges: https://github.com/duo-labs/parliament/blob/master/utils/update_iam_data.py . This generates a json file similar to your own https://github.com/FSecureLABS/awspx/blob/master/lib/aws/actions.py
• Duo's CloudMapper (which I also authored and maintain) has two things that might be of interest. First, is the ability to generate a graph of IAM relationships (users, roles, policies, groups). It is very basic as it doesn't try to understand the policies at all, just shows which principals have which policies. Some info about that functionality is at https://summitroute.com/blog/2019/03/11/cloudmapper_iam_report_command/ and the relevant screenshot included.

The second feature of CloudMapper I want to point out that is likely going to be more interesting to you is the recently added access_check command (recent as in I only added it a week ago). This uses parliament to parse the policies and understand what types of resources different privileges impact (ex. it automatically figures out that s3:GetObject is used on resources with an ARN of an S3 object). Then it tries to actually understand the policies by combining them all together, so if a user has been granted s3:GetObject on *, but also has a Deny * on *, then it understands that the user has no privileges. I've barely been able to document it, but I discussed it a little at https://github.com/duo-labs/cloudmapper/releases/tag/2.8.1

Anyway, looking forward to hearing about how you used the tag editor API and what those short-comings were.

Trying to run AWSPX against a profile created - the ingestor seems to run fine until it gets to the end, then I'm receiving a "List index out of range" error.

I tried limiting the ingestor to only IAM, EC2, Lambda - then also IAM, EC2 (with --services) but still get the same error.

Installed via INSTALL script with docker on Kali linux (Linux kali 4.19.0-kali4-amd64 #1 SMP Debian 4.19.28-2kali1 (2019-03-18) x86_64 GNU/Linux)

Traceback (most recent call last):

│ File "/opt/awspx/cli.py", line 385, in main │
│ 382 console.start() │
│ 383 │
│ 384 try: │
│ ❱ 385 args.func(args) │
│ 386 │
│ 387 except (KeyboardInterrupt, SystemExit): │
│ 388 console.stop() │
│ File "/opt/awspx/cli.py", line 145, in handle_ingest │
│ 142 ingestor = IngestionManager(session=session, console=console, services=arg│
│ 143 db=args.database, quick=args.quick, skip_actio│
│ 144 only_types=args.only_types, skip_types=args.sk│
│ ❱ 145 only_arns=args.only_arns, skip_arns=args.skip_│
│ 146 │
│ 147 assert ingestor.zip is not None, "Ingestion failed" │
│ 148 │
│ File "/opt/awspx/lib/aws/ingestor.py", line 79, in init │
│ 76 self.load_transitives() │
│ 77 │
│ 78 if not skip_actions: │
│ ❱ 79 self.load_actions() │
│ 80 │
│ 81 self.zip = self.save(db) │
│ 82 │
│ File "/opt/awspx/lib/aws/ingestor.py", line 241, in load_actions │
│ 238 │
│ 239 # Skip AWS::Domain principals │
│ 240 self.update(Elements(principal │
│ ❱ 241 for principal in resource_based_policy.p│
│ 242 if not principal.type("AWS::Domain"))) │
│ 243 │
│ 244 # Only actions beginning with sts:AssumeRole are valid │
│ File "/opt/awspx/lib/aws/ingestor.py", line 382, in update │
│ 379 def update(self, elements): │
│ 380 │
│ 381 for element in elements: │
│ ❱ 382 self.add(element) │
│ 383 │
│ 384 def add(self, element): │
│ 385 │
│ File "/opt/awspx/lib/aws/ingestor.py", line 401, in add │
│ 398 pass │
│ 399 │
│ 400 else: │
│ ❱ 401 self.console.info(f"Added {element.label()}: ({element})") │
│ 402 │
│ 403 │
│ 404 class SessionClientWrapper(object): │
│ File "/opt/awspx/lib/graph/base.py", line 61, in label │
│ 58 return [l for l in self.labels() │
│ 59 if l != self.class.name │
│ 60 ][0] │
│ ❱ 61 │
│ 62 def labels(self): │
│ 63 return sorted(list(self._labels)) │
│ 64 │
╰──────────────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

Create a wiki with articles about how to use & contribute to awspx. We have some of this content in text files in the repo itself, but it needs to be expanded.

Hello there,

It would be really nice to be able to use a flag in order to have the AWS credentials pulled from environment variables instead of using a profile file. In our case, we use https://github.com/segmentio/aws-okta for credential management. When we execute aws commands locally, aws-okta starts a subshell with the proper credentials in environment variables, so it would be useful to simply use those environment variables to kick off awspx. I am probably going to fork the repo to make this change, but will more than willingly put in a PR here if that's something you would be interested in. I will comment on this issue if I have a working fork in the near future.

Thanks for such a great tool!
Jake

I have assume role profile set, as well as using the assume role STS returned credentials set as environment variables, but non of the method works. Looking at cli.py, looks like assume role is supported but was not part of the parameters when creating new profile.

It seems to me that the attacks algorithm is not taking Deny policies into account when creating paths. I'm not sure how to debug it.

Assuming a policy like:

{
    "Version": "2012-10-17",
    "Statement": [
        {            
           "Effect": "Deny",            
           "Action": [              
                "iam:AttachUserPolicy"
            ],
            "Resource": ["*"]
        },
        {
            "Effect": "Allow",
            "Action": "iam:AttachUserPolicy",
            "Resource": "arn:aws:iam::174522763890:user/${aws:username}"
        }
    ]
}

I still see AttachUserPolicy attack paths coming from a user with that policy after running:

awspx attacks --only-attacks AddUserToGroup AttachUserPolicy CreatePolicyVersion PutUserPolicy --include-conditional-attacks --max-attack-depth 3

Is this a bug? Am I doing something wrong?

When using Fedora 33 and podman, the INSTALL script doesn't consider selinux, and the container won't be able to write to /opt/awspx

I changed the mount options to include the ":z" suffix, which relabeled the directory and the container could write to it.

I.e. -v $(MOUNT)/data:/opt/awspx:z

If I use the sample.zip or run through 'awspx ingest' (which does successfully connect and pull down env data after I plug in Key/Secret), when I try to go to browser ( I did edit 'INSTALL' to support a different web port + eth0 IP, so I could access remotely), I am getting this error:

"Neo4jError: the client is unauthorized due to authentication failure"

I only have limited experience w/neo4j, but I can't figure out how to properly connect into the front-end of this tool. I am able to go to the IP:7474, and upon connecting, I actually see some data related to my AWS environment -- so I must not be too far off. I changed the default web port to 8080, I know this is operator error :) any nudge or help is appreciated

Hi,

I am assessing an AWS account. When I run awspx with --assume-role I get the below error. But when I run without --assume-role it works fine. The following command I used.
awspx ingest --profile prof --services EC2 S3 Lambda IAM --assume-role arn:aws:iam::11111111111:role/some-role

Error

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.7/dist-packages/urllib3/connection.py:170 in _new_conn │
│ │
│ 167 │ │ │
│ 168 │ │ try: │
│ 169 │ │ │ conn = connection.create_connection( │
│ ❱ 170 │ │ │ │ (self._dns_host, self.port), self.timeout, **extra_kw │
│ 171 │ │ │ ) │
│ 172 │ │ │
│ 173 │ │ except SocketTimeout: │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/util/connection.py:73 in create_connection │
│ │
│ 70 │ │ │ LocationParseError(u"'%s', label empty or too long" % host), None │
│ 71 │ │ ) │
│ 72 │ │
│ ❱ 73 │ for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): │
│ 74 │ │ af, socktype, proto, canonname, sa = res │
│ 75 │ │ sock = None │
│ 76 │ │ try: │
│ │
│ /usr/lib/python3.7/socket.py:748 in getaddrinfo │
│ │
│ 745 │ # We override this function since we want to translate the numeric family │
│ 746 │ # and socket type values to enum constants. │
│ 747 │ addrlist = [] │
│ ❱ 748 │ for res in _socket.getaddrinfo(host, port, family, type, proto, flags): │
│ 749 │ │ af, socktype, proto, canonname, sa = res │
│ 750 │ │ addrlist.append((_intenum_converter(af, AddressFamily), │
│ 751 │ │ │ │ │ │ _intenum_converter(socktype, SocketKind), │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.7/dist-packages/botocore/httpsession.py:332 in send │
│ │
│ 329 │ │ │ │ assert_same_host=False, │
│ 330 │ │ │ │ preload_content=False, │
│ 331 │ │ │ │ decode_content=False, │
│ ❱ 332 │ │ │ │ chunked=self._chunked(request.headers), │
│ 333 │ │ │ ) │
│ 334 │ │ │ │
│ 335 │ │ │ http_response = botocore.awsrequest.AWSResponse( │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:756 in urlopen │
│ │
│ 753 │ │ │ │ e = ProtocolError("Connection aborted.", e) │
│ 754 │ │ │ │
│ 755 │ │ │ retries = retries.increment( │
│ ❱ 756 │ │ │ │ method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] │
│ 757 │ │ │ ) │
│ 758 │ │ │ retries.sleep() │
│ 759 │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/util/retry.py:507 in increment │
│ │
│ 504 │ │ """ │
│ 505 │ │ if self.total is False and error: │
│ 506 │ │ │ # Disabled, indicate to re-raise the error. │
│ ❱ 507 │ │ │ raise six.reraise(type(error), error, _stacktrace) │
│ 508 │ │ │
│ 509 │ │ total = self.total │
│ 510 │ │ if total is not None: │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/packages/six.py:770 in reraise │
│ │
│ 767 │ │ │ │ value = tp() │
│ 768 │ │ │ if value.traceback is not tb: │
│ 769 │ │ │ │ raise value.with_traceback(tb) │
│ ❱ 770 │ │ │ raise value │
│ 771 │ │ finally: │
│ 772 │ │ │ value = None │
│ 773 │ │ │ tb = None │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:706 in urlopen │
│ │
│ 703 │ │ │ │ timeout=timeout_obj, │
│ 704 │ │ │ │ body=body, │
│ 705 │ │ │ │ headers=headers, │
│ ❱ 706 │ │ │ │ chunked=chunked, │
│ 707 │ │ │ ) │
│ 708 │ │ │ │
│ 709 │ │ │ # If we're going to release the connection in finally:, then │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:382 in _make_request │
│ │
│ 379 │ │ │
│ 380 │ │ # Trigger any extra validation we need to do. │
│ 381 │ │ try: │
│ ❱ 382 │ │ │ self._validate_conn(conn) │
│ 383 │ │ except (SocketTimeout, BaseSSLError) as e: │
│ 384 │ │ │ # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout. │
│ 385 │ │ │ self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:1010 in _validate_conn │
│ │
│ 1007 │ │ │
│ 1008 │ │ # Force connect early to allow us to validate the connection. │
│ 1009 │ │ if not getattr(conn, "sock", None): # AppEngine might not have .sock │
│ ❱ 1010 │ │ │ conn.connect() │
│ 1011 │ │ │
│ 1012 │ │ if not conn.is_verified: │
│ 1013 │ │ │ warnings.warn( │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connection.py:353 in connect │
│ │
│ 350 │ │
│ 351 │ def connect(self): │
│ 352 │ │ # Add certificate verification │
│ ❱ 353 │ │ conn = self._new_conn() │
│ 354 │ │ hostname = self.host │
│ 355 │ │ tls_in_tls = False │
│ 356 │
│ │
│ /usr/local/lib/python3.7/dist-packages/urllib3/connection.py:182 in _new_conn │
│ │
│ 179 │ │ │
│ 180 │ │ except SocketError as e: │
│ 181 │ │ │ raise NewConnectionError( │
│ ❱ 182 │ │ │ │ self, "Failed to establish a new connection: %s" % e │
│ 183 │ │ │ ) │
│ 184 │ │ │
│ 185 │ │ return conn │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7fe595689898>: Failed to establish a new connection: [Errno -3] Temporary failure in
name resolution

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/awspx/cli.py:398 in main │
│ │
│ 395 │ │ console.start() │
│ 396 │ │
│ 397 │ try: │
│ ❱ 398 │ │ args.func(args) │
│ 399 │ │
│ 400 │ except (KeyboardInterrupt, SystemExit): │
│ 401 │ │ console.stop() │
│ │
│ /opt/awspx/cli.py:135 in handle_ingest │
│ │
│ 132 │ │ │ │ │ │ │ │ } │
│ 133 │ │ │ │
│ 134 │ │ │ assumed_role = session.client('sts').assume_role( │
│ ❱ 135 │ │ │ │ **assume_role_args)["Credentials"] │
│ 136 │ │ │ │
│ 137 │ │ │ session = boto3.session.Session( │
│ 138 │ │ │ │ aws_access_key_id=assumed_role["AccessKeyId"], │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/client.py:386 in _api_call │
│ │
│ 383 │ │ │ │ raise TypeError( │
│ 384 │ │ │ │ │ "%s() only accepts keyword arguments." % py_operation_name) │
│ 385 │ │ │ # The "self" in this scope is referring to the BaseClient. │
│ ❱ 386 │ │ │ return self._make_api_call(operation_name, kwargs) │
│ 387 │ │ │
│ 388 │ │ _api_call.name = str(py_operation_name) │
│ 389 │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/client.py:692 in _make_api_call │
│ │
│ 689 │ │ │ http, parsed_response = event_response │
│ 690 │ │ else: │
│ 691 │ │ │ http, parsed_response = self._make_request( │
│ ❱ 692 │ │ │ │ operation_model, request_dict, request_context) │
│ 693 │ │ │
│ 694 │ │ self.meta.events.emit( │
│ 695 │ │ │ 'after-call.{service_id}.{operation_name}'.format( │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/client.py:711 in _make_request │
│ │
│ 708 │ │
│ 709 │ def _make_request(self, operation_model, request_dict, request_context): │
│ 710 │ │ try: │
│ ❱ 711 │ │ │ return self._endpoint.make_request(operation_model, request_dict) │
│ 712 │ │ except Exception as e: │
│ 713 │ │ │ self.meta.events.emit( │
│ 714 │ │ │ │ 'after-call-error.{service_id}.{operation_name}'.format( │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/endpoint.py:102 in make_request │
│ │
│ 99 │ def make_request(self, operation_model, request_dict): │
│ 100 │ │ logger.debug("Making request for %s with params: %s", │
│ 101 │ │ │ │ │ operation_model, request_dict) │
│ ❱ 102 │ │ return self._send_request(request_dict, operation_model) │
│ 103 │ │
│ 104 │ def create_request(self, params, operation_model=None): │
│ 105 │ │ request = create_request_object(params) │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/endpoint.py:137 in _send_request │
│ │
│ 134 │ │ success_response, exception = self._get_response( │
│ 135 │ │ │ request, operation_model, context) │
│ 136 │ │ while self._needs_retry(attempts, operation_model, request_dict, │
│ ❱ 137 │ │ │ │ │ │ │ │ success_response, exception): │
│ 138 │ │ │ attempts += 1 │
│ 139 │ │ │ # If there is a stream associated with the request, we need │
│ 140 │ │ │ # to reset it before attempting to send the request again. │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/endpoint.py:256 in _needs_retry │
│ │
│ 253 │ │ responses = self._event_emitter.emit( │
│ 254 │ │ │ event_name, response=response, endpoint=self, │
│ 255 │ │ │ operation=operation_model, attempts=attempts, │
│ ❱ 256 │ │ │ caught_exception=caught_exception, request_dict=request_dict) │
│ 257 │ │ handler_response = first_non_none_response(responses) │
│ 258 │ │ if handler_response is None: │
│ 259 │ │ │ return False │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/hooks.py:357 in emit │
│ │
│ 354 │ │
│ 355 │ def emit(self, event_name, **kwargs): │
│ 356 │ │ aliased_event_name = self._alias_event_name(event_name) │
│ ❱ 357 │ │ return self._emitter.emit(aliased_event_name, **kwargs) │
│ 358 │ │
│ 359 │ def emit_until_response(self, event_name, **kwargs): │
│ 360 │ │ aliased_event_name = self._alias_event_name(event_name) │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/hooks.py:228 in emit │
│ │
│ 225 │ │ :return: List of (handler, response) tuples from all processed │
│ 226 │ │ │ │ handlers. │
│ 227 │ │ """ │
│ ❱ 228 │ │ return self._emit(event_name, kwargs) │
│ 229 │ │
│ 230 │ def emit_until_response(self, event_name, **kwargs): │
│ 231 │ │ """ │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/hooks.py:211 in _emit │
│ │
│ 208 │ │ responses = [] │
│ 209 │ │ for handler in handlers_to_call: │
│ 210 │ │ │ logger.debug('Event %s: calling handler %s', event_name, handler) │
│ ❱ 211 │ │ │ response = handler(**kwargs) │
│ 212 │ │ │ responses.append((handler, response)) │
│ 213 │ │ │ if stop_on_response and response is not None: │
│ 214 │ │ │ │ return responses │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:183 in call │
│ │
│ 180 │ │ this will process retries appropriately. │
│ 181 │ │ │
│ 182 │ │ """ │
│ ❱ 183 │ │ if self._checker(attempts, response, caught_exception): │
│ 184 │ │ │ result = self._action(attempts=attempts) │
│ 185 │ │ │ logger.debug("Retry needed, action of: %s", result) │
│ 186 │ │ │ return result │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:251 in call │
│ │
│ 248 │ │
│ 249 │ def call(self, attempt_number, response, caught_exception): │
│ 250 │ │ should_retry = self._should_retry(attempt_number, response, │
│ ❱ 251 │ │ │ │ │ │ │ │ │ │ caught_exception) │
│ 252 │ │ if should_retry: │
│ 253 │ │ │ if attempt_number >= self._max_attempts: │
│ 254 │ │ │ │ # explicitly set MaxAttemptsReached │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:277 in _should_retry │
│ │
│ 274 │ │ else: │
│ 275 │ │ │ # If we've exceeded the max attempts we just let the exception │
│ 276 │ │ │ # propogate if one has occurred. │
│ ❱ 277 │ │ │ return self._checker(attempt_number, response, caught_exception) │
│ 278 │
│ 279 │
│ 280 class HTTPStatusCodeChecker(BaseChecker): │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:317 in call │
│ │
│ 314 │ def call(self, attempt_number, response, caught_exception): │
│ 315 │ │ for checker in self._checkers: │
│ 316 │ │ │ checker_response = checker(attempt_number, response, │
│ ❱ 317 │ │ │ │ │ │ │ │ │ caught_exception) │
│ 318 │ │ │ if checker_response: │
│ 319 │ │ │ │ return checker_response │
│ 320 │ │ return False │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:223 in call │
│ │
│ 220 │ │ │ return self._check_response(attempt_number, response) │
│ 221 │ │ elif caught_exception is not None: │
│ 222 │ │ │ return self._check_caught_exception( │
│ ❱ 223 │ │ │ │ attempt_number, caught_exception) │
│ 224 │ │ else: │
│ 225 │ │ │ raise ValueError("Both response and caught_exception are None.") │
│ 226 │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/retryhandler.py:359 in _check_caught_exception │
│ │
│ 356 │ │ # caught_exception. That's what this class is being used for. If │
│ 357 │ │ # the MaxAttemptsDecorator is not interested in retrying the exception │
│ 358 │ │ # then this exception just propogates out past the retry code. │
│ ❱ 359 │ │ raise caught_exception │
│ 360 │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/endpoint.py:200 in _do_get_response │
│ │
│ 197 │ │ │ responses = self._event_emitter.emit(event_name, request=request) │
│ 198 │ │ │ http_response = first_non_none_response(responses) │
│ 199 │ │ │ if http_response is None: │
│ ❱ 200 │ │ │ │ http_response = self._send(request) │
│ 201 │ │ except HTTPClientError as e: │
│ 202 │ │ │ return (None, e) │
│ 203 │ │ except Exception as e: │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/endpoint.py:269 in _send │
│ │
│ 266 │ │ │ return True │
│ 267 │ │
│ 268 │ def _send(self, request): │
│ ❱ 269 │ │ return self.http_session.send(request) │
│ 270 │
│ 271 │
│ 272 class EndpointCreator(object): │
│ │
│ /usr/local/lib/python3.7/dist-packages/botocore/httpsession.py:352 in send │
│ │
│ 349 │ │ except URLLib3SSLError as e: │
│ 350 │ │ │ raise SSLError(endpoint_url=request.url, error=e) │
│ 351 │ │ except (NewConnectionError, socket.gaierror) as e: │
│ ❱ 352 │ │ │ raise EndpointConnectionError(endpoint_url=request.url, error=e) │
│ 353 │ │ except ProxyError as e: │
│ 354 │ │ │ raise ProxyConnectionError(proxy_url=proxy_url, error=e) │
│ 355 │ │ except URLLib3ConnectTimeoutError as e: │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
EndpointConnectionError: Could not connect to the endpoint URL: "https://sts.amazonaws.com/"

Hi,

thank you for the awesome tool.
often admins information provided in the form of json, through the "aws iam get-account-authorization-details > output.json"
is it possible for ingestor to work with the get-account-authorization-details data export file?

Hi,

I'm having a hard time running awspx on a remote machine, while I do intent to access the web console from my work laptop.

1. There is an install instruction, but not a run instruction. Install pulls, downloads and builds anew - thus time and data are lost. How do I just run this easily?

2. It seems just changing the HOST= line in INSTALL is not sufficient, as the webpage directs the browser to other local ports (such as 7687), not only that, the app does so directing the browser to HOST ip at :PORT, this wont do in cases where the local server is not aware of his external IP.
   How do I set awspx up for remote use, rather than just a local tool?

I started running awspx ingest. On realization that it was going to look at every object in a bucket that I didn't really care about, I deleted the bucket. Then awspx crashed with:

Traceback (most recent call last):
  File "/opt/awspx/lib/aws/ingestor.py", line 940, in get_object_acls
    obj.set("ACL", sr.ObjectAcl(bucket,key).grants)
  File "/usr/local/lib/python3.7/dist-packages/boto3/resources/factory.py", line 339, in property_loader
    self.load()
  File "/usr/local/lib/python3.7/dist-packages/boto3/resources/factory.py", line 505, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 626, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObjectAcl operation: The specified key does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/awspx/cli.py", line 406, in <module>
    main()
  File "/opt/awspx/cli.py", line 401, in main
    args.func(args)
  File "/opt/awspx/cli.py", line 134, in handle_ingest
    only_arns=args.only_arns, except_arns=args.except_arns)
  File "/opt/awspx/lib/aws/ingestor.py", line 900, in __init__
    self.get_object_acls()
  File "/opt/awspx/lib/aws/ingestor.py", line 946, in get_object_acls
    self._print("[!]", e)
  File "/opt/awspx/lib/aws/ingestor.py", line 57, in _print
    print(''.join(messages))
TypeError: sequence item 1: expected str instance, NoSuchKey found

I think it would be better to avoid the race somehow or ignore the failure.

Hello,

I noticed that creating a new profile and adding the access key, secret key, region and output format is not enough to get awspx to work in AWS GovCloud. You actually have to enter the region flag to get it to work. This looks like a bug because this is not required if ingesting data from AWS Public cloud.

How it works with AWS Public Cloud:

awspx ingest --profile Bob

How it works with AWS GovCloud:

awspx ingest --profile Bob --region us-gov-west-1

Just curious, I have a fairly simple AWS tenant that I'm testing in.

I got profile created with
awspx profile --create profilename
and then ran
awspx ingest --profile profilename.

It has made it to

Resolving actions and resources
[+] Identity based Policy for `arn:aws:iam::aws:policy/AmazonEC2FullAccess` resolved to 351 action(s)
[+] Identity based Policy for `arn:aws:iam::aws:policy/IAMFullAccess` resolved to 694 action(s)
[+] Identity based Policy for `arn:aws:iam::aws:policy/job-function/ViewOnlyAccess` resolved to 142 action(s)

and has been sitting there for about 90 minutes with no action. I've already Ctrl+c'd once and restarted and it's sitting at the same place again.

Im deploying my infrastructure with terraform:

resource "aws_s3_bucket_object" "xxxxxxxxxxxxxx" {
  bucket   = aws_s3_bucket.xxxxxxxx.bucket
  key      = "xxxxxxxxxxxxxx"
  source   = "xxxxxxxxxxxxxx"
  acl = "public-read-write"
}

Switching to just public-read, the ingestion dosent have any problems.

Ingestion fails with the following:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/awspx/cli.py:402 in main                                                                    │
│                                                                                                  │
│   399 │   │   console.start()                                                                    │
│   400 │                                                                                          │
│   401 │   try:                                                                                   │
│ ❱ 402 │   │   args.func(args)                                                                    │
│   403 │                                                                                          │
│   404 │   except (KeyboardInterrupt, SystemExit):                                                │
│   405 │   │   console.stop()                                                                     │
│                                                                                                  │
│ /opt/awspx/cli.py:150 in handle_ingest                                                           │
│                                                                                                  │
│   147 │   except ClientError as e:                                                               │
│   148 │   │   console.critical(e)                                                                │
│   149 │                                                                                          │
│ ❱ 150 │   ingestor = IngestionManager(session=session, console=console, services=args.services   │
│   151 │   │   │   │   │   │   │   │   db=args.database, quick=args.quick, skip_actions=args.sk   │
│   152 │   │   │   │   │   │   │   │   only_types=args.only_types, skip_types=args.skip_types,    │
│   153 │   │   │   │   │   │   │   │   only_arns=args.only_arns, skip_arns=args.skip_arns)        │
│                                                                                                  │
│ /opt/awspx/lib/aws/ingestor.py:82 in __init__                                                    │
│                                                                                                  │
│     79 │   │   self.load_transitives()                                                           │
│     80 │   │                                                                                     │
│     81 │   │   if not skip_actions:                                                              │
│ ❱   82 │   │   │   self.load_actions()                                                           │
│     83 │   │                                                                                     │
│     84 │   │   self.zip = self.save(db)                                                          │
│     85                                                                                           │
│                                                                                                  │
│ /opt/awspx/lib/aws/ingestor.py:275 in load_actions                                               │
│                                                                                                  │
│    272 │   │   │   │                                                                             │
│    273 │   │   │   │   acl = BucketACL(resource, resources) \                                    │
│    274 │   │   │   │   │   if resource.label() == "AWS::S3::Bucket" \                            │
│ ❱  275 │   │   │   │   │   else ObjectACL(resource, resources)                                   │
│    276 │   │   │   │                                                                             │
│    277 │   │   │   │   self.update(acl.principals())                                             │
│    278 │   │   │   │   self.update(acl.actions())                                                │
│                                                                                                  │
│ /opt/awspx/lib/aws/policy.py:645 in __init__                                                     │
│                                                                                                  │
│   642 │   │   │   │   │   │   statement["Principal"] = {"AWS": grantee["URI"]}                   │
│   643 │   │   │   │                                                                              │
│   644 │   │   │   │   # Handle Actions                                                           │
│ ❱ 645 │   │   │   │   statement["Action"] = self.AccessControlList[permission]                   │
│   646 │   │   │   │                                                                              │
│   647 │   │   │   │   # Handle Resources (Bucket and Objects in Bucket)                          │
│   648 │   │   │   │   statement["Resource"] = [resource.id(), resource.id() + "/*"]              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: unhashable type: 'list'

Thanks

awspx is failing to save the ingested data.
I think I've narrowed the issue down to something related to the IAM service ingestion.
This works:
awspx ingest --skip-attacks-all --services S3 --skip-types AWS::S3::Object --region us-east-1
This fails:
awspx ingest --skip-attacks-all --services IAM --region us-east-1

There is no error message- the ingest command just exists after printing NOTICE Saving ingested data and the directory for the .csv files has been created.

Hey, this isn't really a bug, more of a config issue that might be worth documenting to help others.

OS
MacOS Catalina 10.15.6

Issue
After launching the awspx docker container using docker-machine with the default settings I was unable to access the web services on localhost.

Remediation
Docker-machine and docker toolbox use 192.168.99.100 instead of localhost. I had to modify the docker run command to use 0.0.0.0 instead of 127.0.0.1:

-p 0.0.0.0:80:80
-p 0.0.0.0:7687:7687
-p 0.0.0.0:7373:7373
-p 0.0.0.0:7474:7474 \

Once changed I was able to access the interface here http://192.168.99.100.

Hopefully this will help other folks.

Every time I run awspx with --skip-types or the --services flag, I get the following error:

Traceback (most recent call last):
╭──────────────────────────────────────────────────────────────────────────────────────╮
│ File "/opt/awspx/cli.py", line 385, in main                                          │
│    382         console.start()                                                       │
│    383                                                                               │
│    384     try:                                                                      │
│  ❱ 385         args.func(args)                                                       │
│    386                                                                               │
│    387     except (KeyboardInterrupt, SystemExit):                                   │
│    388         console.stop()                                                        │
│ File "/opt/awspx/cli.py", line 145, in handle_ingest                                 │
│    142     ingestor = IngestionManager(session=session, console=console, services=arg│
│    143                                 db=args.database, quick=args.quick, skip_actio│
│    144                                 only_types=args.only_types, skip_types=args.sk│
│  ❱ 145                                 only_arns=args.only_arns, skip_arns=args.skip_│
│    146                                                                               │
│    147     assert ingestor.zip is not None, "Ingestion failed"                       │
│    148                                                                               │
│ File "/opt/awspx/lib/aws/ingestor.py", line 79, in __init__                          │
│      76         self.load_transitives()                                              │
│      77                                                                              │
│      78         if not skip_actions:                                                 │
│  ❱   79             self.load_actions()                                              │
│      80                                                                              │
│      81         self.zip = self.save(db)                                             │
│      82                                                                              │
│ File "/opt/awspx/lib/aws/ingestor.py", line 250, in load_actions                     │
│     247                                                                              │
│     248                     # This role trusts all IAM entities within this account  │
│     249                     if (action.source().type("AWS::Account")                 │
│  ❱  250                             and action.source().id().split(':')[4] == self.ac│
│     251                                                                              │
│     252                         self.update(Elements(Trusts(properties=action.propert│
│     253                                                     source=action.target(),  │
╰──────────────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

This happens when it gets to 'resolving policy information'

This does not happen when I just run ./awspx ingest however when I run it like this it will not show any connections/attacks etc in the web gui.

Running with the following flags:
./awspx ingest --skip-types "AWS::S3::Object"
./awspx ingest --services EC2, Lambda, IAM

This is running on a MacOS system, inside the docker container pulled down 2 weeks ago (right around the 10th of January)

Add command line support to update to the latest version.

Hi,
Im trying to visualize data that was ingested from multiple AWS accounts, with the same profile (using AssumeRole).
Unfortunately, it is only possible to load and visualize one DB at a time.
Is there any way that I am missing to do that?

Collapsing an expanded node doesn't remove all expanded nodes and edges.

Repeating an inbound/outbound actions search results in the front-end drawing multiple instances of the same action(s).

Hi,

I am assessing an AWS account with the following command sudo awspx ingest --services IAM

 INFO     Searching for attack (03/21): AssumeRole (iteration: 1 of max: 5)                                      
 DEBUG    [REDACTED]   
 DEBUG    Added:              (arn:aws:iam::<ID>:root)-->(arn:aws:iam::<ID>:role/CompanyAccountAccessRole)
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/awspx/cli.py:398 in main                                                                    │
│                                                                                                  │
│   395 │   │   console.start()                                                                    │
│   396 │                                                                                          │
│   397 │   try:                                                                                   │
│ ❱ 398 │   │   args.func(args)                                                                    │
│   399 │                                                                                          │
│   400 │   except (KeyboardInterrupt, SystemExit):                                                │
│   401 │   │   console.stop()                                                                     │
│                                                                                                  │
│ /opt/awspx/cli.py:157 in handle_ingest                                                           │
│                                                                                                  │
│   154 │   handle_db(args, console=console.item("Creating Database"))                             │
│   155 │                                                                                          │
│   156 │   if not (args.skip_attacks_all or args.skip_actions_all):                               │
│ ❱ 157 │   │   handle_attacks(args, console=console.item("Updating Attack paths"))                │
│   158                                                                                            │
│   159                                                                                            │
│   160 def handle_attacks(args, console=console):                                                 │
│                                                                                                  │
│ /opt/awspx/cli.py:172 in handle_attacks                                                          │
│                                                                                                  │
│   169 │   │   │   │   │   │   │   │   │   │      else ""),                                       │
│   170 │   │   │   │   │     console=console)                                                     │
│   171 │                                                                                          │
│ ❱ 172 │   attacks.compute(max_iterations=args.max_attack_iterations)                             │
│   173                                                                                            │
│   174                                                                                            │
│   175 def handle_db(args, console=console):                                                      │
│                                                                                                  │
│ /opt/awspx/lib/aws/attacks.py:1285 in compute                                                    │
│                                                                                                  │
│   1282 │   │   │   results = db.run(self.queries[pattern])                                       │
│   1283 │   │   │                                                                                 │
│   1284 │   │   │   for r in results:                                                             │
│ ❱ 1285 │   │   │   │   self.console.debug(f"Added: ({r['source']['Arn']})-->"                    │
│   1286 │   │   │   │   │   │   │   │      f"({r['grant']['Arn']})")                              │
│   1287 │   │   │                                                                                 │
│   1288 │   │   │   self.stats.append({                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'Arn'
                                                                                            

If I skip the check AssumeRole with --skip-checks AssumeRole awspx is running without errors.

Any idea how I could further debug this?

At present, awspx only maps actions that support Resource-Level permissions, because of the way it parses the AWS docs to determine which actions affect which resources. However, we may also be interested in actions that don't support Resource-Level permissions. Some of these actions don't affect specific resources (e.g. s3:ListBuckets) whereas others do (e.g. ec2:DescribeInstanceAttribute). For the sake of expediency, we will treat both cases the same and map all non-RLP actions to a single info node.

I just pulled awspx today and ran it for the first time. Unfortunately the NPM dependencies don't install correctly and the web server is not running.

Console output:

npm WARN npm npm does not support Node.js v10.21.0                                                                                  
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
npm WARN npm You can find the latest version at https://nodejs.org/
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "npm:": npm:vue-loader@^16.0.0-beta.3
npm WARN deprecated @hapi/[email protected]: joi is leaving the @hapi organization and moving back to 'joi' (https://github.com/sideway/joi/issues/2411)
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-08-27T20_10_57_164Z-debug.log

Log file:

0 info it worked if it ends with ok
1 warn npm npm does not support Node.js v10.21.0
2 warn npm You should probably upgrade to a newer version of node as we
3 warn npm can't make any promises that npm will work with this version.
4 warn npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
5 warn npm You can find the latest version at https://nodejs.org/
6 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'install' ]
7 info using [email protected]
8 info using [email protected]
9 verbose npm-session 1171aa578074561b
10 silly install runPreinstallTopLevelLifecycles
11 silly preinstall [email protected]
12 info lifecycle [email protected]~preinstall: [email protected]
13 silly install loadCurrentTree
14 silly install readLocalPackageData
15 silly install loadIdealTree
16 silly install cloneCurrentTreeToIdealTree
17 silly install loadShrinkwrap
18 silly install loadAllDepsIntoIdealTree
19 http fetch GET 304 https://registry.npmjs.org/cytoscape-dagre 429ms (from cache)
20 http fetch GET 304 https://registry.npmjs.org/codemirror 437ms (from cache)
21 silly pacote range manifest for cytoscape-dagre@^2.2.2 fetched in 444ms
22 silly pacote range manifest for codemirror@^5.53.2 fetched in 450ms
23 http fetch GET 304 https://registry.npmjs.org/@vue%2fcli-plugin-babel 499ms (from cache)
24 silly pacote range manifest for @vue/cli-plugin-babel@^4.3.1 fetched in 522ms
25 http fetch GET 304 https://registry.npmjs.org/core-js 521ms (from cache)
26 silly pacote range manifest for core-js@^3.6.5 fetched in 535ms
27 http fetch GET 304 https://registry.npmjs.org/vue-router 527ms (from cache)
28 silly pacote range manifest for vue-router@^3.1.5 fetched in 549ms
29 http fetch GET 304 https://registry.npmjs.org/@vue%2fcli-service 590ms (from cache)
30 silly pacote range manifest for @vue/cli-service@^4.3.1 fetched in 629ms
31 silly saveTree [email protected]
32 verbose stack Error: Unsupported URL Type "npm:": npm:vue-loader@^16.0.0-beta.3
32 verbose stack     at unsupportedURLType (/usr/lib/nodejs/npm-package-arg/npa.js:197:15)
32 verbose stack     at fromURL (/usr/lib/nodejs/npm-package-arg/npa.js:250:13)
32 verbose stack     at Function.resolve (/usr/lib/nodejs/npm-package-arg/npa.js:71:12)
32 verbose stack     at Object.keys.map (/usr/share/npm/lib/install/deps.js:445:20)
32 verbose stack     at Array.map (<anonymous>)
32 verbose stack     at BB.resolve.then (/usr/share/npm/lib/install/deps.js:444:47)
32 verbose stack     at tryCatcher (/usr/lib/nodejs/bluebird/js/release/util.js:16:23)
32 verbose stack     at Promise._settlePromiseFromHandler (/usr/lib/nodejs/bluebird/js/release/promise.js:512:31)
32 verbose stack     at Promise._settlePromise (/usr/lib/nodejs/bluebird/js/release/promise.js:569:18)
32 verbose stack     at Promise._settlePromiseCtx (/usr/lib/nodejs/bluebird/js/release/promise.js:606:10)
32 verbose stack     at Async._drainQueue (/usr/lib/nodejs/bluebird/js/release/async.js:138:12)
32 verbose stack     at Async._drainQueues (/usr/lib/nodejs/bluebird/js/release/async.js:143:10)
32 verbose stack     at Immediate.Async.drainQueues [as _onImmediate] (/usr/lib/nodejs/bluebird/js/release/async.js:17:14)
32 verbose stack     at runCallback (timers.js:705:18)
32 verbose stack     at tryOnImmediate (timers.js:676:5)
32 verbose stack     at processImmediate (timers.js:658:5)
33 verbose cwd /opt/awspx/www
34 verbose Linux 5.6.0-kali2-amd64
35 verbose argv "/usr/bin/node" "/usr/bin/npm" "install"
36 verbose node v10.21.0
37 verbose npm  v5.8.0
38 error code EUNSUPPORTEDPROTOCOL
39 error Unsupported URL Type "npm:": npm:vue-loader@^16.0.0-beta.3
40 verbose exit [ 1, true ]

Symlink /usr/local/bin/awspx to a location in $PATH if /usr/local/bin/ is not present .

Missing the iam sub-command before create-policy-version. I spot checked the rest in source and this looks like the only one missing it.

aws create-policy-version   \
  --policy-arn ...

Hello All,

Running Flags:
ingest --region=[single region] --except-types "AWS::S3::Object" --services="EC2"

Am getting a crash after 1-2hrs of runtime:

Traceback (most recent call last):
  File "/opt/awspx/cli.py", line 395, in <module>
    main()
  File "/opt/awspx/cli.py", line 390, in main
    args.func(args)
  File "/opt/awspx/cli.py", line 135, in handle_ingest
    only_arns=args.only_arns, except_arns=args.except_arns)
  File "/opt/awspx/lib/aws/ingestor.py", line 837, in __init__
    self.get_instance_user_data()
  File "/opt/awspx/lib/aws/ingestor.py", line 859, in get_instance_user_data
    InstanceId=name)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 272, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/botocore/client.py", line 576, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidInstanceID.NotFound) when calling the DescribeInstanceAttribute operation: The instance ID '[censored instance ID]' does not exist

Running this in docker on Mac OS 10.14.6. Did not have this issue with smaller aws regions.

If i am not mistaken, ASWPX takes a policy-centric approach to evaluating privesc paths as opposed to a principal-centric approach?. Is that right?

For example, I have a role that has two policies applied. One policy allows iam:passRole on resource:, and the other allows ec2:RunInstances on resource:. I can exploit the classic CreateInstanceWithExistingProfile privesc because the role has all the permissions it needs.

However this does not show up in AWSPX, and I'm really just asking if this is expected or unexpected on your end. If this is unexpected (a bug), I'm happy to provide more details on the roles, policies, instance, profiles, etc. that are involved. If expected, I suggest adding this to the FAQ like the note you have about not processing DENY actions.

Thanks!

Also, I gotta say that I really love AWSPX. Between the visualizations and the exploitation guidance, your tool really helped me put some major IAM privesc pieces together when I was ramping up in this area!

Hi all, attempting to install but it states /usr/local/bin is not in my PATH even though it is. I tried to install it regardless but cannot access the GUI. I verified the UFW is not active, and I can reach the server.

admin@awspx:~/awspx$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
admin@awspx:~/awspx$ sudo ./INSTALL
[*] '/usr/local/bin' isn't in your $PATH. Choose another location to save "awspx" [y/n]? y
1) /bin
2) /sbin
3) /snap/bin
4) /usr/bin
5) /usr/local/bin
6) /usr/local/sbin
7) /usr/sbin
#? 5

[*] Pulling "awspx" image...

latest: Pulling from beatro0t/awspx
d121f8d1c412: Pull complete
75deccc0fc24: Pull complete
96d87ee20b42: Pull complete
276bf8f477ec: Pull complete
871e64ea3282: Pull complete
702f4fa5465c: Pull complete
e7624eb4204f: Pull complete
d506fb82e3cf: Pull complete
46eebdeff943: Pull complete
dec13b9e4671: Pull complete
c540ebb420cf: Pull complete
Digest: sha256:ee7786dd03830cef671eb76ce4b1d7e59fcf63541908db8a01cf34b00675493b
Status: Downloaded newer image for beatro0t/awspx:latest
docker.io/beatro0t/awspx:latest

[*] Creating "awspx" container... and you're all set!

    The web interface (http://localhost) will be available shortly...
    Run: `awspx -h` for a list of options.

admin@awspx:~/awspx$ sudo systemctl stop ufw
admin@awspx:~/awspx$ sudo systemctl status ufw
● ufw.service - Uncomplicated firewall
     Loaded: loaded (/lib/systemd/system/ufw.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Fri 2020-12-18 12:58:16 UTC; 9min ago
       Docs: man:ufw(8)
    Process: 22657 ExecStop=/lib/ufw/ufw-init stop (code=exited, status=0/SUCCESS)
   Main PID: 362 (code=exited, status=0/SUCCESS)



`admin@awspx:~/awspx$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 2e:33:ea:53:60:a5 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet 192.168.10.48/24 brd 192.168.10.255 scope global ens18
       valid_lft forever preferred_lft forever
    inet6 fe80::2c33:eaff:fe53:60a5/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:00:32:09:c5 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:ff:fe32:9c5/64 scope link
       valid_lft forever preferred_lft forever
7: veth3c89b09@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
    link/ether be:77:fd:e2:46:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::bc77:fdff:fee2:4675/64 scope link
       valid_lft forever preferred_lft forever`





`C:\Users\admin>ping 192.168.10.48

Pinging 192.168.10.48 with 32 bytes of data:
Reply from 192.168.10.48: bytes=32 time<1ms TTL=63
Reply from 192.168.10.48: bytes=32 time<1ms TTL=63
Reply from 192.168.10.48: bytes=32 time<1ms TTL=63

Ping statistics for 192.168.10.48:
    Packets: Sent = 3, Received = 3, Lost = 0 (0% loss),`