Git Product home page Git Product logo

pyattck's Introduction

pyattck

Welcome to pyattck's Documentation

    .______   ____    ____  ___   .___________.___________.  ______  __  ___
    |   _  \  \   \  /   / /   \  |           |           | /      ||  |/  /
    |  |_)  |  \   \/   / /  ^  \ `---|  |----`---|  |----`|  ,----'|  '  /
    |   ___/    \_    _/ /  /_\  \    |  |        |  |     |  |     |    <
    |  |          |  |  /  _____  \   |  |        |  |     |  `----.|  .  \
    | _|          |__| /__/     \__\  |__|        |__|      \______||__|\__\

A Python package to interact with MITRE ATT&CK Frameworks

Current Version is 7.1.1

pyattck is a light-weight framework for MITRE ATT&CK Frameworks. This package extracts details from the MITRE Enterprise, PRE-ATT&CK, Mobile, and ICS Frameworks.

Why?

pyattck assist organizations and individuals with accessing MITRE ATT&CK Framework(s) in a programmatic way. Meaning, you can access all defined actors, malwares, mitigations, tactics, techniques, and tools defined by the Enterprise, Mobile, Pre-Attck, and ICS frameworks via a command-line utility or embedding into your own code base.

There are many reasons why you would want to access this data in an automated (scripted/coded) way but a few examples are:

  • Generate reports with additional details about a technique (or any object defined in the framework)
  • A build pipeline of detection rules with additional MITRE ATT&CK details for categorization
  • Quickly searching for specific details about a technique without navigating a web page

There are other benefits that pyattck provide as well which includes the ability to provide additional contextual data. You can find more information about this data here but the basics are that pyattck utilizes multiple open-source repositories to gather additional contextual data like commands used to execute a technique, country and other details about a malicious actor, other variants of malware similar to a defined tool/malware, etc.

This additional context is what makes pyattck truly powerful and enables people to build more robust testing and validation of their detection rules, validates testing assumptions, etc. Truly there are countless ways that pyattck could be utilized to help blue, red, and purple teams defend organizations (and themselves).

Features

The pyattck package retrieves all Tactics, Techniques, Actors, Malware, Tools, and Mitigations from the MITRE ATT&CK Frameworks as well as any defined relationships within the MITRE ATT&CK dataset (including sub-techniques).

In addition, Techniques, Actors, and Tools (if applicable) now have collected data from third-party resources that are accessible via different properties. For more detailed information about these features, see External Datasets.

The pyattck package allows you to:

  • Specify a URL or local file path for the MITRE ATT&CK Enterprise Framework json, generated dataset, and/or a config.yml file.
  • Access data from the MITRE PRE-ATT&CK Framework
  • Access data from the MITRE Mobile ATT&CK Framework
  • Access data from the MITRE ICS ATT&CK Framework
  • Access sub-techniques as nested objects or you can turn it off and access as normal technique
  • Access compliance controls (currently NIST 800-53 v5) related to a MITRE ATT&CK Technique
  • pyattck now utilizes structured data models. More information can be found at pyattck-data
  • Run an interactive console menu system to access pyattck data

Table of Contents

  1. Installation
  2. Usage Example
  3. Configuration
  4. Notes

Installation

You can install pyattck on OS X, Linux, or Windows. You can also install it directly from the source. To install, see the commands under the relevant operating system heading, below.

macOS, Linux and Windows:

pip install pyattck

Installing from source

git clone https://github.com/swimlane/pyattck.git
cd pyattck
python setup.py install

Usage example

To use pyattck you must instantiate an Attck object. Although you can interact directly with each class, the intended use is through a Attck object:

from pyattck import Attck

attack = Attck()

By default, sub-techniques are accessible under each technique object. You can turn this behavior off by passing nested_techniques=False when creating your Attck object.

As an example, the default behavior looks like the following example:

from pyattck import Attck

attack = Attck()

for technique in attack.enterprise.techniques:
    print(technique.id)
    print(technique.name)
    for subtechnique in technique.techniques:
        print(subtechnique.id)
        print(subtechnique.name)

You can access the following main properties on your Attck object:

  • enterprise
  • preattack
  • mobile
  • ics

Once you specify the MITRE ATT&CK Framework, you can access additional properties.

Here are the accessible objects under the Enterprise property:

For more information on object types under the enterprise property, see Enterprise.

Here are the accessible objects under the PreAttck property:

For more information on object types under the preattck property, see PreAttck.

Here are the accessible objects under the Mobile property:

For more information on object types under the mobile property, see Mobile.

Here are the accessible objects under the ICS property:

For more information on object types under the ics property, see ICS.

Interactive Menu Usage

To utilize the new interactive menu system within pyattck, you must set interactive to True. By doing so, it will launch the interactive console menu system.

Using a script your can launch this by running:

from pyattck import Attck

Attck(interactive=True)

Or you can also run interactive mode on the command line:

pyattck --interactive

Checkout a gif example below:

Configuration

pyattck allows you to configure if you store external data and where it is stored.

from pyattck import Attck

attck = Attck(
    nested_techniques=True,
    use_config=False,
    save_config=False,
    config_file_path='~/pyattck/config.yml',
    data_path='~/pyattck/data',
    enterprise_attck_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/merged_enterprise_attck_v1.json",
    pre_attck_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/merged_pre_attck_v1.json",
    mobile_attck_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/merged_mobile_attck_v1.json,
    ics_attck_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/merged_ics_attck_v1.json",
    nist_controls_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/merged_nist_controls_v1.json",
    generated_nist_json="https://swimlane-pyattck.s3.us-west-2.amazonaws.com/attck_to_nist_controls.json",
    **kwargs
)

By default, pyattck will (now) pull the latest external data from their respective locations using HTTP GET requests. pyattck currently pulls from the following locations:

You have several options when instantiating the Attck object. As of 4.0.0 you can now specify any of the following options:

  • use_config - When you specify this argument as True pyattck will attempt to retrieve the configuration specified in the config_file_path location. If this file is corrupted or cannot be found, we will default to retrieving data from the specified *_attck_json locations.
  • save_config - When you specify this argument as True pyattck will save the configuration file to the specified location set by config_file_path. Additionally, we will save all downloaded files to the data_path location specified. If you have specified a local path location instead of a download URL for any of the *_attck_json parameters we will save this location in our configuration and reference this location going forward.
  • config_file_path - The path to store a configuration file. Default is ~/pyattck/config.yml
  • data_path - The path to store any data files downloaded to the local system. Default is ~/pyattck/data

JSON Locations

Additionally, you can specify the location for each individual *_attck_json files by passing in either a URI or a local file path. If you have passed in a local file path, we will simply read from this file.

If you have used the default values or specified an alternative URI location to retrieve these JSON files from, you can additionally pass in **kwargs that will be passed along to the Requests python package when performing any HTTP requests.

Note

We understand that there are many different open-source projects being released, even on a daily basis, but we wanted to provide a straightforward Python package that allowed the user to identify known relationships between all verticals of the MITRE ATT&CK Framework.

If you are unfamiliar with the MITRE ATT&CK Framework, there are a few key components to ensure you have a firm grasp around. The first is Tactics & Techniques. When looking at the MITRE ATT&CK Framework, the Tactics are the columns and represent the different phases of an attack.

The MITRE ATT&CK Framework is NOT an all encompassing/defacto security coverage map - it is rather a FRAMEWORK and additional avenues should also be considered when assessing your security posture.

Techniques are the rows of the framework and are categorized underneath specific Tactics (columns). They are data points within the framework that provides guidance when assessing your security gaps. Additionally, (most) Techniques contain mitigation guidance in addition to information about their relationship to tools, malware, and even actors/groups that have used this technique during recorded attacks.

This means, if your organization is focused on TTPs (Tactics Techniques and Procedures) used by certain actors/groups then MITRE ATT&CK Framework is perfect for you. If you are not at this security maturing within your organization, no worries! The ATT&CK Framework still provides really good guidance in a simple and straightforward layout, but programmatically it is not straightforward--especially if you wanted to measure (or map) your security controls using the framework.

Developing and Testing

You can add features or bugs or run the code in a development environment.

  1. To get a development and testing environment up and running, use this Dockerfile.

  2. To use the Dockerfile run, cd to this repository directory and run:

docker build --force-rm -t pyattck .
  1. Next, run the docker container:
docker run pyattck

Running this calls the test python file in bin/test.py.

  1. Modify the test python file for additional testing and development.

Running the tests

Tests within this project should cover all available properties and methods. As this project grows the tests will become more robust but for now we are testing that they exist and return outputs.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning.

Change Log

For details on features for a specific version of pyattck, see the CHANGELOG.md.

Authors

See also the list of contributors.

License

This project is licensed under the MIT License.

Acknowledgments

First of all, I would like to thank everyone who contributes to open-source projects, especially the maintainers and creators of these projects. Without them, this capability would not be possible.

This data set is generated from many different sources. As we continue to add more sources, we will continue to add them here. Again thank you to all of these projects. In no particular order, pyattck utilizes data from the following projects:

.. toctree::
   :titlesonly:

   configuration
   pyattck/attck
   Dataset <https://github.com/swimlane/pyattck-data>
   Data Models <https://github.com/swimlane/pyattck-data>
   enterprise/enterprise
   preattck/preattck
   mobile/mobileattck
   ics/icsattck

pyattck's People

Contributors

aacienfuegos avatar actions-user avatar github-actions[bot] avatar isober avatar joshswimlane avatar mcarpenter avatar msadministrator avatar neo23x0 avatar nikkuman avatar peggyzimmerman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyattck's Issues

When resolving git-lfs references, json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

We are unable to access attck.enterprise.actors, attck.enterprise.techniques, etc because the code is downloading a json file that is actually a git-lfs reference and not valid json.

It fails when parsing the output of https://github.com/swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True.

We are using version 3.0.1.

Reproduction below:

Python 3.8.6 (default, Jan  8 2021, 15:34:15)
[Clang 12.0.0 (clang-1200.0.26.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyattck import Attck
>>> attck = Attck()
>>> attck.enterprise.actors
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/blah/lib/python3.8/site-packages/pyattck/attck.py", line 224, in enterprise
    self.__load_data()
  File "/blah/lib/python3.8/site-packages/pyattck/attck.py", line 285, in __load_data
    Attck.__ENTERPRISE_NIST_DATA_JSON = self.__datasets.get_data(
  File "/blah/lib/python3.8/site-packages/pyattck/datasets.py", line 66, in get_data
    return self.__get_json_data(
  File "/blah/lib/python3.8/site-packages/pyattck/datasets.py", line 48, in __get_json_data
    data = requests.get(url).json()
  File "/blah/lib/python3.8/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/blah/.pyenv/versions/3.8.6/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/blah/.pyenv/versions/3.8.6/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/blah/.pyenv/versions/3.8.6/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Pillow requirement (Python Imaging Library)

I noticed that builds on my build systems failed due to a missing libjpeg-dev that is required by Pillow 9.0.0 that is used in your project.

Could you explain why pyattck requires an imaging library?
I can't find any hints in the source code.

Create CI/CD pipeline to generate a correlated dataset

Create a CI/CD pipeline that will gather and correlate multiple datasets into one which will then be committed to this repository. Pyattck will download this pre-correlated/filtered dataset to use in retrieval of common attack data.

nist_controls_json bug

Encountering this error when using the pyattck library:

"Traceback (most recent call last):\n File "/app/Tasks/PythonQueue/aG_PWoywMX_NxMO7A/script.F0376B4DD8665F94DEDBE64EFC7BA36AE4B391493343E086DE7B74505FB2AA1D.py", line 77, in \n for technique in attack.enterprise.techniques:\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/attck.py", line 263, in enterprise\n from .enterprise.enterprise import Enterprise\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/enterprise/init.py", line 1, in \n from .enterprise import Enterprise\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/enterprise/enterprise.py", line 1, in \n from .technique import AttckTechnique\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/enterprise/technique.py", line 1, in \n from .attckobject import AttckObject\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/enterprise/attckobject.py", line 4, in \n class AttckObject(object):\n File "/home/swimlane/.pyenv/versions/3.6.11/lib/python3.6/site-packages/pyattck/enterprise/attckobject.py", line 17, in AttckObject\n nist_controls_json = Configuration.get_data('nist_controls_json')['objects']\nKeyError: 'objects'\n",
"stdOutput": null
}

It appears to be that the nist_controls_json object is not getting parsed correctly

Cant pull Tools from Techniques

I am trying to extract techniques, sub techniques, tactics, actors and tools/software:

for technique in attack.enterprise.techniques:
print(technique.id)
print(technique.name)
# you can also access generated data sets on aa technique
print(technique.command_list)
print(technique.commands)
print(technique.queries)
print(technique.datasets)
print(technique.possible_detections)
for subtechnique in technique.subtechniques:
print(subtechnique.id)
print(subtechnique.name)
# accessing tactics that this technique belongs to
for tactic in technique.tactics:
print(tactic.id)
print(tactic.name)

    # accessing mitigation recommendations for this technique
        for mitigation in technique.mitigations:
            print(mitigation.id)
            print(mitigation.name)

    # accessing actors using this technique
            for actor in technique.actors:
                print(actor.id)
                print(actor.name)
                for tool in technique.tools:
                    print(tool.name)

However, i keep getting the following error AttributeError: 'AttckTechnique' object has no attribute 'tools'

Is there a way i can pull the tools as well as the following information. I am trying to export this into a CSV/JSON file to be able to read it. Is there a way to export this out as at the moment i cant get the fields to match up

Certificate Verification Issues when using behind Proxy / Local Source Data

Hi team,

I am experiencing SSL Verify issues (seen below) when trying to utilize pyattck module behind a proxy due to the requests library attempting and failing (appropriately so) due to the proxy MitM all of my network traffic.

I have two suggestions on that would let me and other users in similar situations fix this without having to make permanent modifications to various pieces of our OS, requests library or otherwise;

  1. Add a flag to the instantiation of Attck class or other objects that can allow forcing verify=False for requests library globally in this module - this would immediately solve my problem.

  2. Allow users to point to local source files rather than attempting to get remotely - let me download all relevant source files, put them in a local directory and then point the entire module to the directory - maybe via an imported config file with each data source and location specified in the configuration. Currently these appear to be specified in the relevant Class file of each data source and interaction with the relevant repository is utilized to pull the file - allowing us to cache these locally and bypass network requests would significantly improve my quality of life in a zoned-off environment.

I love the idea of this module but it is unusable for me in my current environment due to neither of these being an option currently..

Fixed relationship links in preattack

Thanks Josh.

I might have come across another relationship issue. I'm seeing PreAttckActor class returning what looks like Mitre software "S000" for the IDs. Thoughts?

from pyattck import Attck
attack = Attck()

for actor in attack.preattack.actors:
    print(actor.id, actor.name)

Originally posted by @BigLeagueChew in #37 (comment)

Pyattck json.decoder.JSONDecodeError

When ever I try to run anything after re-installing 3.0.0 of pyattck i'm getting the following error:

for technique in attack.enterprise.techniques:

error - json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

tactics not implemented in techniques

This doesn't seem to work:

for technique in attack.techniques:
    print(technique)

    # accessing tactics that this technique belongs to
    for tactic in technique.tactics:
        print(tactic)

From usage examples https://pypi.org/project/pyattck/

Tactics also don't seem to be fully returned when using the tactic:

# accessing tactics
for tactic in attack.tactics:
    #print(tactic.id, [technique.id for technique in tactic.techniques])
    if 'T1098' in [technique.id for technique in tactic.techniques]:
        print(tactic.id)
for technique in attack.techniques:
    #print(technique.id,[tactic.id for tactic in technique.tactics]) - doesnt work
    #print(technique.id,[tactic.id for tactic in technique.tactic]) - works
    if technique.id == 'T1098':
        print([tactic.id for tactic in technique.tactic])
    
#https://attack.mitre.org/techniques/T1078/
#Tactics = Defense Evasion - confirmed, Persistence - confirmed, Privilege Escalation - confirmed, Initial Access - confirmed
#4 listed at site, only two returned. when using the tactic method of technique only one returned.

JSONDecodeError with pyattck version 4.0.0 and 4.0.1

Hi,

Started getting JSONDecodeError in version 4.0.0 and 4.0.1:

>>>from pyattck import Attck

>>>attack = Attck()
>>>attack.enterprise.techniques 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv/lib/python3.9/site-packages/pyattck/attck.py", line 266, in enterprise
    from .enterprise.enterprise import Enterprise
  File ".../venv/lib/python3.9/site-packages/pyattck/enterprise/__init__.py", line 1, in <module>
    from .enterprise import Enterprise
  File ".../venv/lib/python3.9/site-packages/pyattck/enterprise/enterprise.py", line 1, in <module>
    from .technique import AttckTechnique
  File ".../venv/lib/python3.9/site-packages/pyattck/enterprise/technique.py", line 1, in <module>
    from .attckobject import AttckObject
  File ".../venv/lib/python3.9/site-packages/pyattck/enterprise/attckobject.py", line 4, in <module>
    class AttckObject(object):
  File ".../venv/lib/python3.9/site-packages/pyattck/enterprise/attckobject.py", line 17, in AttckObject
    nist_controls_json = Configuration.get_data(Configuration.config_data.get('nist_controls_json'))['objects']
  File ".../venv/lib/python3.9/site-packages/pyattck/configuration.py", line 108, in get_data
    data = cls.__download_url_data(value)
  File ".../venv/lib/python3.9/site-packages/pyattck/configuration.py", line 16, in __download_url_data
    return request('GET', url, **cls.requests_kwargs).json()
  File ".../venv/lib/python3.9/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 4 (char 3)

Performance refactor

Hello @joshswimlane
I was working on a new refactor for improve the performance of pyattck.
I wanted to improve the memory usage, unfortunately it was not possible, but the time to parsing the file was huge improvement.

If you want i could create the pull request, it could be a new mayor version on the project.
Also i think it could useful create a test for the project.

Please give me feedback about what you think.

Here you are the results and test files used pyattck_test.zip:

Python 2.7

New code:

Memory:
	Max: 71.02Mb
	Min: 19.47Mb
	Median(6): 47.31Mb
Time mem: 0:00:00.353103

Old code:

Memory:
	Max: 78.98Mb
	Min: 17.62Mb
	Median(721): 70.28Mb
Time mem: 0:01:12.166692

Python 3.5

New code:

Memory:
	Max: 43.66Mb
	Min: 22.96Mb
	Median(5): 34.12Mb
Time mem: 0:00:00.223263

Old code:

Memory:
	Max: 42.43Mb
	Min: 21.35Mb
	Median(299): 42.16Mb
Time mem: 0:00:29.845125

Python 3.6

New code:

Memory:
	Max: 41.73Mb
	Min: 22.92Mb
	Median(5): 32.11Mb
Time mem: 0:00:00.221237

Old code:

Memory:
	Max: 45.91Mb
	Min: 21.18Mb
	Median(250): 40.53Mb
Time mem: 0:00:24.916879

Python 3.7

New code:

Memory:
	Max: 41.21Mb
	Min: 21.81Mb
	Median(5): 31.23Mb
Time mem: 0:00:00.232362

Old code:

Memory:
	Max: 41.25Mb
	Min: 21.00Mb
	Median(321): 41.04Mb
Time mem: 0:00:32.049158

Unnecessary pinned versions in install_requires and insecure Pillow version

Is your feature request related to a problem? Please describe.

While using pyattck, I noticed that the install_requires points to the requirements file having pinned library versions. This is a departure from best practice. Refer to https://packaging.python.org/discussions/install-requires-vs-requirements/

It is not considered best practice to use install_requires to pin dependencies to specific versions, or to specify sub-dependencies (i.e. dependencies of your dependencies). This is overly-restrictive, and prevents the user from gaining the benefit of dependency upgrades.

The current pinned version of Pillow==8.2.0 has several vulnerabilities which have been fixed in the latest version. I'm not able to pin Pillow to 8.4.0 because pyattck pins it to 8.2.0. Vulnerabilities:

Describe the solution you'd like

The more urgent request would be to allow Pillow>=8.2.0 . Overall, it would be nice to remove pinned versions.

Error for no module name PIL

No module named 'PIL'
  File "Exxx", line 14, in <module>
    for actor in attack.enterprise.actors:

I have installed all the dependencies. But I still have a problem.

My python version is 3.8.1. The pyattck version is 2.1.0

Convert class variables to instance variables.

When writing tests for the Attck class the class variables must be reset to None so that certain test which require them to be that way will pass. To reproduce the issue you can use the following test. If they are converted to instance vars the below test should fail.

@pytest.fixture
def attck():
    yield Attck()

def test_class_vars_set(attck):
    assert attck._Attck__ENTERPRISE_ATTCK_JSON is None
    assert attck._Attck__ENTERPRISE_GENERATED_DATA_JSON is None
    assert attck._Attck__ENTERPRISE_NIST_DATA_JSON is None
    techniques = attck.enterprise.techniques
    assert isinstance(techniques, list)
    assert isinstance(attck._Attck__ENTERPRISE_ATTCK_JSON, dict)
    assert isinstance(attck._Attck__ENTERPRISE_GENERATED_DATA_JSON, dict)
    assert isinstance(attck._Attck__ENTERPRISE_NIST_DATA_JSON, dict)


def test_class_vars_persist_tests(attck):
    assert isinstance(attck._Attck__ENTERPRISE_ATTCK_JSON, dict)
    assert isinstance(attck._Attck__ENTERPRISE_GENERATED_DATA_JSON, dict)
    assert isinstance(attck._Attck__ENTERPRISE_NIST_DATA_JSON, dict)

'NoneType' object is not subscriptable

I found a wrong population on the attck_obj in tools objects.

The root cause is on pyattck/tools.py:58

  • step to reproduce
from pyattck import Attck
m = Attck()
m.actors[0].tools[0].techniques

Raise the following exception

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-71-d9658118211f> in <module>()
----> 1 m.actors[0].tools[0].techniques

~\venv\lib\site-packages\pyattck\tools.py in techniques(self)
     34         from .technique import AttckTechnique
     35         technique_list = []
---> 36         for item in self.attck_obj['objects']:
     37             if 'relationship_type' in item:
     38                 if 'uses' in item['relationship_type']:

TypeError: 'NoneType' object is not subscriptable

Remove Pillow as Dependency

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Update Try except statements throughout project

Small PR attached for this one file issue. I lost a couple of hours to this (path with dash instead of underscore, arg!).

There are many other occurrences of except: pass or except: return None throughout the project. Almost all of those could/should be improved. I don't have the context for the original author's decision so difficult to say what they were expecting might fail. Example from technique.py:

        try:
            for item in self._RELATIONSHIPS[self.stix]:
                if item in item_dict:
                    return_list.append(PreAttckActor(preattck_obj=self.__preattck_obj, **item_dict[item]))
        except:
            pass

Where did the author think this might fail? Is that okay? What error is actually raised? Should it really be ignored?

I will not be working through these other items but they should probably be on somebody's TODO list.

Originally posted by @mcarpenter in #93 (comment)

Cannot extract DataSource's

I am trying to pull the DataSources for the techniques but I keep getting the follow error:

AttributeError: 'AttckTechnique' object has no attribute 'data_source'

2.1.0 and 2.1.3 return ValueError: No JSON object could be decoded.

from pyattck import Attck
attck = Attck().update(enterprise=True)
Traceback (most recent call last):
File "", line 1, in
File "C:\dev\swimlane_4_1_env\lib\site-packages\pyattck\attck.py", line 365, in update
self.__load_data(force=True)
File "C:\dev\swimlane_4_1_env\lib\site-packages\pyattck\attck.py", line 379, in __load_data
Attck.__ENTERPRISE_GENERATED_DATA_JSON = self.__datasets.generated_attck_data(force=force)
File "C:\dev\swimlane_4_1_env\lib\site-packages\pyattck\datasets.py", line 77, in generated_attck_data
datasets = self.__get_datasets()
File "C:\dev\swimlane_4_1_env\lib\site-packages\pyattck\datasets.py", line 101, in __get_datasets
return requests.get(self._DATASETS_URL).json()
File "C:\dev\swimlane_4_1_env\lib\site-packages\requests\models.py", line 892, in json
return complexjson.loads(self.text, **kwargs)
File "C:\dev\Python2714\Lib\json_init
.py", line 339, in loads
return _default_decoder.decode(s)
File "C:\dev\Python2714\Lib\json\decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\dev\Python2714\Lib\json\decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Traceback when calling actors from attack.techniques

Pyattck v2.0.5

for tactic in attack.tactics:
    for technique in tactic.techniques:
          for actor in technique.actors:
              print(actor)

results in:
'Traceback (most recent call last):\n File "c:\Users\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_resolver.py", line 193, in _get_py_dictionary\n attr = getattr(var, name)\n File "C:\Users\Documents\Projects\feeds\venv\lib\site-packages\pyattck\enterprise\technique.py", line 190, in actors\n for item in self.__attck_obj['objects']:\nTypeError: 'NoneType' object is not subscriptable\n'

Downloading fresh JSON (update) loops unnecessarily?

I want to utilise local config as much as I can, to help with speed. When you had the .update() method I would utilise that. Now with the latest update, I need to build my own "update function" to overwrite the existing config, and any normal call, would just use the local json.

I've tried updating as below (fresh virtualenv) using pyattck-4.0.3 on Python 3.9.2 from pip:

>>> from pyattck import Attck
>>> import logging
>>>
>>> logging.basicConfig(format = '%(asctime)s - %(levelname)s - %(message)s',level = 'DEBUG')
>>>
>>> attack_update = Attck(
...     use_config=False,
...     save_config=True,
...     data_path='/home/xakep/derp/'
... )
>>>
>>> if attack_update:
...     print('yes')
...
yes
>>>
>>> if attack_update.enterprise:
...     print('yes')
...

Only now begins the download of the JSON files, and appears to download the same files 6 times:

2021-07-21 12:19:24,522 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:26,339 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
2021-07-21 12:19:27,191 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:27,419 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/pre-attack/pre-attack.json HTTP/1.1" 200 85794
2021-07-21 12:19:27,464 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:27,628 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/mobile-attack/mobile-attack.json HTTP/1.1" 200 228573
2021-07-21 12:19:27,748 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:27,990 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:28,117 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:28,350 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:28,564 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:28,567 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:29,957 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:31,359 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:31,577 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:31,778 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:31,781 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:32,390 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:32,704 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:32,734 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:32,743 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:32,745 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:32,759 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:33,259 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:33,277 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
2021-07-21 12:19:34,169 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:34,185 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/pre-attack/pre-attack.json HTTP/1.1" 200 85794
2021-07-21 12:19:34,229 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:34,243 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/mobile-attack/mobile-attack.json HTTP/1.1" 200 228573
2021-07-21 12:19:34,359 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:34,375 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:34,496 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:34,535 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:34,549 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:34,551 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:34,575 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:36,646 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:36,673 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:36,679 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:36,682 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:36,696 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:37,041 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:37,057 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:37,090 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:37,114 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
2021-07-21 12:19:38,014 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:38,029 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/pre-attack/pre-attack.json HTTP/1.1" 200 85794
2021-07-21 12:19:38,071 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:38,086 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/mobile-attack/mobile-attack.json HTTP/1.1" 200 228573
2021-07-21 12:19:38,173 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:38,188 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:38,293 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:38,321 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:38,330 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:38,333 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:38,348 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:40,455 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:40,489 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:40,500 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:40,503 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:40,517 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:40,836 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:40,866 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:40,874 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:40,877 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:40,890 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:41,157 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:41,175 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
2021-07-21 12:19:42,030 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:42,046 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/pre-attack/pre-attack.json HTTP/1.1" 200 85794
2021-07-21 12:19:42,154 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:42,179 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/mobile-attack/mobile-attack.json HTTP/1.1" 200 228573
2021-07-21 12:19:42,283 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:42,299 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:42,419 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:42,450 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:42,460 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:42,463 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:42,479 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:44,938 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:44,963 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:44,970 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:44,974 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:44,995 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:45,334 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:45,350 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:45,421 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:45,437 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
2021-07-21 12:19:46,304 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:46,318 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/pre-attack/pre-attack.json HTTP/1.1" 200 85794
2021-07-21 12:19:46,365 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:46,379 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/mobile-attack/mobile-attack.json HTTP/1.1" 200 228573
2021-07-21 12:19:46,483 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:46,499 - DEBUG - https://raw.githubusercontent.com:443 "GET /center-for-threat-informed-defense/attack-control-framework-mappings/master/frameworks/nist800-53-r4/stix/nist800-53-r4-controls.json HTTP/1.1" 200 331153
2021-07-21 12:19:46,643 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:46,675 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/generated_attck_data.json?raw=True HTTP/1.1" 302 138
2021-07-21 12:19:46,687 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/generated_attck_data.json HTTP/1.1" 302 157
2021-07-21 12:19:46,690 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:46,704 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/generated_attck_data.json HTTP/1.1" 200 31396583
2021-07-21 12:19:48,808 - DEBUG - Starting new HTTPS connection (1): github.com:443
2021-07-21 12:19:48,840 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/blob/master/attck_to_nist_controls.json?raw=True HTTP/1.1" 302 140
2021-07-21 12:19:48,847 - DEBUG - https://github.com:443 "GET /swimlane/pyattck/raw/master/attck_to_nist_controls.json HTTP/1.1" 302 159
2021-07-21 12:19:48,850 - DEBUG - Starting new HTTPS connection (1): media.githubusercontent.com:443
2021-07-21 12:19:48,869 - DEBUG - https://media.githubusercontent.com:443 "GET /media/swimlane/pyattck/master/attck_to_nist_controls.json HTTP/1.1" 200 9105826
2021-07-21 12:19:49,200 - DEBUG - Starting new HTTPS connection (1): raw.githubusercontent.com:443
2021-07-21 12:19:49,230 - DEBUG - https://raw.githubusercontent.com:443 "GET /mitre/cti/master/enterprise-attack/enterprise-attack.json HTTP/1.1" 200 2910652
yes
>>>

I believe the issue may be from the iteration over the JSON file names, and instead of a single-download it's downloading multiple? Am I wrong? And how do I try to implement an "update" func/method to only overwrite the json that exists using the new version 4.

Ref:

def __save_data(cls):

        for json_data in ['enterprise_attck_json', 'pre_attck_json', 
                          'mobile_attck_json', 'nist_controls_json', 
                          'generated_attck_json', 'generated_nist_json']:
            if cls._check_if_url(getattr(cls, json_data)):
                path = os.path.join(cls.data_path, "{json_data}.json".format(json_data=json_data))
                data = cls.__download_url_data(getattr(cls, json_data))
                cls.__write_to_disk(path, data)

PS: Great framework!

Thanks

Bad paths to constructor do not raise error

Describe the bug
If a caller gives an incorrect JSON file path to the Attck constructor then this is silently ignored. This will likely lead to later runtime exceptions that are hard to diagnose.

To Reproduce
err.py:

#!/usr/bin/env python

from pyattck import Attck
attck = Attck(enterprise_attck_json='oopsy.json')
print(len(attck.enterprise.actors))

Running this produces error at the last line:

Traceback (most recent call last):
  File "./err.py", line 5, in <module>
    print(len(attck.enterprise.actors))
  File "/home/mjc/github/pyattck/pyattck/enterprise/enterprise.py", line 292, in actors
    for group in self.__attck['objects']:
TypeError: 'NoneType' object is not subscriptable

Expected behavior
The standard exception FileNotFoundError should be propagated from the call to open(). Instead, it is hidden from the caller by _read_from_disk() in configuration.py. I see two issues (anti-patterns) here:

  1. Check-then-open: just try to open the path as a file.
  2. Ignoring all subsequent exceptions (except: pass): remove.
    def __read_from_disk(cls, path):
        if os.path.exists(path) and os.path.isfile(path):
            try:
                with open(path) as f:
                    if path.endswith('.json'):
                        return json.load(f)
                    elif path.endswith('.yml') or path.endswith('.yaml'):
                        return yaml.load(f, Loader=yaml.FullLoader)
                    else:
                        raise UknownFileError(provided_value=path, known_values=['.json', '.yml', '.yaml'])
            except:
                pass
        return None

I found bug in technique.py line 71

original code in line 71
temp_list = phase['phase_name']

Because many att&ck techniques can be used in several phases of kill chain, so you should use append for variable 'temp_list'

Code in line 71 should be changed to :
temp_list.append(phase['phase_name'])

Add ability to set refresh time of data files.

Currently the data files are evaluated and refreshed when they are older than 30 days. I would be great if that was an option that users could override in both the instantiation of the Attck class as well as in the config file.

Exception loading attack = Attck()

When trying to instantiate Attck() i am receiving the following errror:

Traceback (most recent call last):
File "ingest.py", line 412, in
attack = Attck()
File "pyattck.py", line 20, in init
self.attck = MITRE_ATTCK_JSON_URL
File "\venv\lib\site-packages\pyattck\pyattck.py", line 36, in attck
self.attck = requests.get(value).json()
File "venv\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "_init
.py", line 348, in loads
return _default_decoder.decode(s)
File "Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "Python\Python37-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)

Default nist_controls_json URL has moved

Describe the bug
Default nist_controls_json URL has moved.

To Reproduce

import pyattck
attck = pyattck.Attck()
for actor in attck.enterprise.actors:
    pass

Returns...

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/attck.py", line 263, in enterprise
    from .enterprise.enterprise import Enterprise
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/enterprise/__init__.py", line 1, in <module>
    from .enterprise import Enterprise
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/enterprise/enterprise.py", line 1, in <module>
    from .technique import AttckTechnique
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/enterprise/technique.py", line 1, in <module>
    from .attckobject import AttckObject
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/enterprise/attckobject.py", line 4, in <module>
    class AttckObject(object):
  File "/Users/jb/Library/Caches/pypoetry/virtualenvs/kanata-content-7gMKnNcc-py3.9/lib/python3.9/site-packages/pyattck/enterprise/attckobject.py", line 17, in AttckObject
    nist_controls_json = Configuration.get_data('nist_controls_json')['objects']
KeyError: 'objects'

KeyError on Nist Controls JSON Load

I'm running into an issue running Pyattck==5.0.0 on python3.8.

I'm instantiating pyattck and loading just the enterprise JSON v8.2 and am receiving errors against the nist_controls_json object.

from pyattck import Attck

attack = Attck(
    nested_subtechniques=False,
    enterprise_attck_json="/home/-----/projects/------/enterprise-attack.json",
)
Traceback (most recent call last):
  File "/home/------/projects/-----/mitre/ingest.py", line 25, in <module>
    for technique in attack.enterprise.techniques:
  File "/home/------/.local/lib/python3.8/site-packages/pyattck/attck.py", line 263, in enterprise
    from .enterprise.enterprise import Enterprise
  File "/home/------/.local/lib/python3.8/site-packages/pyattck/enterprise/__init__.py", line 1, in <module>
    from .enterprise import Enterprise
  File "/home/---/.local/lib/python3.8/site-packages/pyattck/enterprise/enterprise.py", line 1, in <module>
    from .technique import AttckTechnique
  File "/home/-----/.local/lib/python3.8/site-packages/pyattck/enterprise/technique.py", line 1, in <module>
    from .attckobject import AttckObject
  File "/home/---/.local/lib/python3.8/site-packages/pyattck/enterprise/attckobject.py", line 4, in <module>
    class AttckObject(object):
  File "/home/---/.local/lib/python3.8/site-packages/pyattck/enterprise/attckobject.py", line 17, in AttckObject
    nist_controls_json = Configuration.get_data('nist_controls_json')['objects']
KeyError: 'objects'

Errors with 5.1.1

Since this morning I get this. I noticed that the changlog shows that a new version has been released yesterday.

from pyattck import Attck
ModuleNotFoundError: No module named 'pyattck'

I found bug in function def tactic(self) of technique.py

#2
Because the problem mentioned in #2 ,so function def tactic(self) in technique.py need be fixed too.
original code:

def tactic(self):
       '''Returns all tactics as a list that this technique is found in'''
       from .tactic import AttckTactic
       tactic_list = []
       for item in self.attck_obj['objects']:
           if 'x-mitre-tactic' in item['type']:
               if str(self._tactic).lower() == str(item['x_mitre_shortname']).lower():
                   tactic_list.append(AttckTactic(**item))
       return tactic_list

Since the attribute 'tactic' of technique is a list, so line 51 is comparing two string,a string type list and a string variable.Considering there are many techniques ,which are corresponding to several tactics, so you should traverse the list of self._tactic and compare each one with the
str(item['x_mitre_shortname']).lower().
My code is :

@property
 def tactic(self):
      '''Returns all tactics as a list that this technique is found in'''
      from .tactic import AttckTactic
      tactic_list = []
      for item in self.attck_obj['objects']:
          if 'x-mitre-tactic' in item['type']:
              for tactic_temp in self._tactic:
                  if str(tactic_temp).lower() ==  str(item['x_mitre_shortname']).lower():
                      tactic_list.append(AttckTactic(**item))
      return tactic_list
          

Malware Technique relationship is linked to Actors

Hello,

I believe the techniques() method on a AttckMalware class is incorrectly linked to Actor information instead of a Technique. I believe line 112 on malware.py needs to be changed to 'attack-pattern'.
See example below which prints Actor names instead of Technique names.

from pyattck import Attck
attack = Attck()

for mal in attack.enterprise.malwares:
    for technique in mal.techniques:
        print(technique.name)

Get tools directly from a technique

I'm trying to get the tools used for a specific technique and I see that the Technique object doesn't have a property with the tools.

Although the opposite is availible (getting the techniques related to a specific tool), I would like to get the tools from the technique itself, just like it's done with Actors, Mitigations, etc. I've tried to add a new propperty to technique.py and it seems to work fine (it's already being done with the tool's property in actors).

If there's a reason this isn't being done already, either because it's not possible or it's not how you are "supposed" to use the matrix, please let me know.

Deprecated attribute accessible in all objects

I want to see if the AttckObject is deprecated, at the moment this is only possible with techniques. I would like to do the same with the other objects.

Just like it's done with techniques, you would access the "deprecated" attribute. I beleive this can be done with the x_mitre_deprecated attribute in the JSON files.

I think this attribute is relevant for any type of AttckObject, but if not, it can be created only in certain types.

2.1.2 python 2 package from PyPi is different from repository

I've attached the datasets.py file from the python2 package on pypi. Line 83 and 84 appear to be different than what is in git. There is a print statement that dumps the cached data to screen followed by an input('press') statement. This is causing the generated_attck_data function to hang waiting for input.
datasets.txt

Missing Mitigations?

Hey There!

I'm not sure if I'm doing something wrong, or if maybe something is broken... I don't seem to be able to get any mitigations per the examples from the enterprise class. I tried both from PyPI and from installation from source with the same result.

https://github.com/swimlane/pyattck/blob/master/docs/enterprise/enterprise.md

This is the code I'm using (effectively just copied from the example).

attack = Attck()
attack.update()

for technique in attack.enterprise.techniques:
    print("ID %s" % technique.id)
    print("Name %s" % technique.name)
    # you can also access generated data sets on aa technique
#    print(technique.command_list)
#    print(technique.commands)
#    print(technique.queries)
#    print(technique.datasets)
#    print(technique.possible_detections)

    # accessing tactics that this technique belongs to
    for tactic in technique.tactics:
        print("Tactic ID: %s" % tactic.id)
        print("tactic name: %s" % tactic.name)

    # accessing mitigation recommendations for this technique
    for mitigation in technique.mitigations:
        print("Mitigation ID: %s" % mitigation.id)
        print("Migiation Name: %s" % mitigation.name)

    # accessing actors using this technique
    for actor in technique.actors:
        print("Actor Name: %s" % actor.name)

    print()

This is the result (snippet as it loops through all entries of course):

ID T1053
Name Scheduled Task/Job
Tactic ID: TA0002
tactic name: Execution
Tactic ID: TA0003
tactic name: Persistence
Tactic ID: TA0004
tactic name: Privilege Escalation

As you can see there doesn't appear to be any mitigations included... however looking at this entry within Mitre's web interface there are (or at least should be?) mitigations included in the dataset;
https://attack.mitre.org/techniques/T1053/

Do I need to explicitly load in mitigations upon initialization?

Did you really update the object's structure in bugfix version release?

Using 2.0.4

Using cached https://files.pythonhosted.org/packages/88/36/e2ac6dcc310a8d5654e0c06c8f935fc60cc700cbadcac05a0facb54e0bb1/pyattck-2.0.4-py3-none-any.whl

I have no errors.

Using 2.0.5

Downloading https://files.pythonhosted.org/packages/f5/50/80cac42fc2da54d3a924d5886c8c94034fa53121af36879d42bc3cb7ca38/pyattck-2.0.5-py3-none-any.whl (934kB)

i suddenly get:

AttributeError: 'AttckActor' object has no attribute 'aliases'

Did you really change the object's structure from 2.0.4 to 2.0.5?
Could you please introduce breaking changes in major (X.0.0) or at least feature (0.X.0) releases only?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.