rabobank-cdc / dettect Goto Github PK

Hello,
When you have a YAML file with many groups defined, and run
python dettect.py g -g output/group.yaml

The output shows the following error:

Traceback (most recent call last):
File "dettect.py", line 309, in
_menu(_init_menu())
File "dettect.py", line 243, in _menu
include_all_score_objs=args.all_scores):
File "/opt/DeTTECT/group_mapping.py", line 584, in generate_group_heat_map
write_file(stage, filename[:242], json_string)
File "/opt/DeTTECT/generic.py", line 357, in write_file
with open(output_filename, 'w') as f:
OSError: [Errno 36] File name too long: 'output/attack_all_apt41-(mitre-att&ck-data)_machete-(mitre-att&ck-data)_kimsuky-(mitre-att&ck-data)_soft-cell-(mitre-att&ck-data)_ta505-(mitre-att&ck-data)_silence-(mitre-att&ck-data)_wirte-(mitre-att&ck-data)_the-white-company-(mitre-att&ck-data)_temp.vel_1.json'

I've been investigating this error, and I've can find the problem. It is because in the line 584 in File "/opt/DeTTECT/group_mapping.py", creates a filename of 255 character as limit, but later in the next function, is added the path of this file "output/" (7 char) and the extension ".json" (5 char), so in the end, the filename is of 267 characters, so it isn't possible.

Then for I propose that you change the number in line 584 in File "/opt/DeTTECT/group_mapping.py" such as 200 , set a same filename like as other functions or you can choose the output file with new parameter.

I love your tool and I want to share this kind of issues, because I want to collaborate in this kind of projects.

Hi DeTTECT

I would like to seek assistance, I am having error change I change the path. I am using windows 10 and all version of application is the latest one.

Hi In regards to generating the attack_windows_all.json file based on all the ATT&CK techniques, I seem to receive an error in recognizing the keyword 'platform' in get_group_techniques line:179.

The error as received is reported below for your reference.

Traceback (most recent call last):
File "dettact.py", line 202, in
menu(init_menu())
File "dettact.py", line 153, in menu
generate_group_heat_map(args.groups, args.overlay, args.overlay_type, args.stage, args.platform, args.software_group)
File "/home/dev/DeTTACT/group_mapping.py", line 452, in generate_group_heat_map
groups_dict = get_group_techniques(groups, stage, platform, groups_file_type)
File "/home/dev/DeTTACT/group_mapping.py", line 179, in get_group_techniques json_platform = e['platform'] KeyError: 'platform'

HI,

When running in docker or locally, Im unable to see more data sources for instance "AWS CloudTrail logs" from the drop menu:

any idea on what am I missing?

It would be beneficial to map the products with their respective data retention times. As a defender, knowing how long the data is stored is crucial when conducting investigations.

Hi,

When attempting to map detection capabilities with the below command:
python3 dettect.py g -g g0022 -o sample-data/techniques-administration-endpoints.yaml -t detection

The comments field in the yaml file are not displayed in the generated json file. For example:

- technique_id: T1222
  technique_name: File Permissions Modification
  detection:
    applicable_to: [all]
    location:
    - ''
    comment: 'This is a test'
    score_logbook:
    - date:
      score: 1
      comment: 'This is a test'
  visibility:
    applicable_to: [all]
    comment: ''
    score_logbook:
    - date: 2019-03-01
      score: 1
      comment: 'This is a test'
      auto_generated: true

I am trying to add our internal detection rule names to the comment field so that it makes it easier for us to determine from the output which rules are covering the technique.

Command :
python3 dettect.py v -ft ABC/techniques-administration-empty-data-source-admin-file-windows-linux-aws-azure-office-365(9).yaml -fd ABC/data-sources-diageo2.yaml -o
Error:
Traceback (most recent call last):
File "dettect.py", line 309, in
_menu(_init_menu())
File "dettect.py", line 233, in _menu
generate_visibility_layer(file_tech, args.file_ds, True)
File "/home/admin1/DeTTECT/technique_mapping.py", line 46, in generate_visibility_layer
mapped_techniques_both = _map_and_colorize_techniques_for_overlaid(my_techniques, my_data_sources, platform)
File "/home/admin1/DeTTECT/technique_mapping.py", line 303, in _map_and_colorize_techniques_for_overlaid
tcnt = len([d for d in technique_data['detection'] if get_latest_score(d) >= 0])
File "/home/admin1/DeTTECT/technique_mapping.py", line 303, in
tcnt = len([d for d in technique_data['detection'] if get_latest_score(d) >= 0])
TypeError: '>=' not supported between instances of 'NoneType' and 'int'

It would be handy if there was a gh-pages branch for building static sites.

CyberChef & MITRE ATT&CK provide gh-pages branches for this purpose. This has enabled both to be used within plugins of 3rd party resources, like CALDERA's plugin "CALT&CK": https://github.com/mitre/caltack, as well as by multiple users for ad-hoc work.

On the readme file, the URL in "This page on the Wiki" is broken.

I think correct URL would be this.

Hi There,

I attempted to invoke DeTTECT library but received xlsxwriter module error. Right after that I installed xlsxwriter but the error remain same. What could be wrong ?
admin1@admin1-virtual-machine:~/DeTTECT$ python3 dettect.py
Traceback (most recent call last):
File "dettect.py", line 4, in
from interactive_menu import *
File "/home/admin1/DeTTECT/interactive_menu.py", line 2, in
from data_source_mapping import *
File "/home/admin1/DeTTECT/data_source_mapping.py", line 3, in
import xlsxwriter
ModuleNotFoundError: No module named 'xlsxwriter'

admin1@admin1-virtual-machine:~/DeTTECT$ pip install XlsxWriter
Collecting XlsxWriter
Using cached https://files.pythonhosted.org/packages/00/1f/2092a81056d36c1b6651a645aa84c1f76bcee03103072d4fe1cb58501d69/XlsxWriter-1.2.8-py2.py3-none-any.whl
Installing collected packages: XlsxWriter
Successfully installed XlsxWriter-1.2.8

Regards,
Nitin

Hi Marcus,

I noticed you have put out new features in your latest drop which is fantastic, havent yet had the chance to explore the docker instance but will do in the coming days. Many thanks.

I have had this nagging thing which I thought I should ask. I found that when we use the command
python3 dettect.py ds -f sample-data/data-sources-endpoints.yaml -y
to generate the techniques-administration file, it gets written to the output folder but it is not the same file that is used in the following command to obtain the detections map. In fact there are close to 300 lines in difference. There is also some formatting difference in the way the fields are populated in the two files.

I understand that was an example, but when I used the other file, it turned out a blank detections map. In fact I also noticed you have mentioned that detections and visibility need not be connected. What am I missing in that connection? I am trying to understand what it would take if I were to map the same my linux fleet and use the detections map using the thence generated techniques-administration file.

Is there any assumption underpinning this mode of execution?

Kind regards
Sri

There is a new parameter on several functions to skip/remove deprecated and revoked STIX objects.

Example:
https://github.com/OTRF/ATTACK-Python-Client/blob/0.3.6/attackcti/attack_api.py#L342

The following parameter is set to True : skip_revoked_deprecated

Hi,

There is a small inconsistency at the generation of the technique administration files.

The command python dettect.py ds -fd sample-data/data-sources-endpoints.yaml -y will generate a administration file without a '-' before the applicable_to key. Like this:

  - technique_id: T1001.001
    technique_name: Junk Data
    detection:
      applicable_to:
      - all
      location:
      - ''
      comment: ''
      score_logbook:
      - date: 
        score: -1
        comment: ''

However in the example there is a '-' before applicable_to, the output of the DeTT&CT editor also contains the '-'.
A YAML exception (duplicated mapping key) occurs if you add another applicable_to block without '-'.

root@f41cd8582e67:/opt/DeTTECT# python dettect.py g -g apt19 -p all -of test.json
[!] Group not part of the data set: apt19
root@f41cd8582e67:/opt/DeTTECT# python dettect.py g -g G0073 -p all -of test.json
[!] Group not part of the data set: g0073

It was the same for apt17, and some others I tested.

It works great for many of the others I am trying. I am sure I am missing something here and the issue is with a misunderstanding of mine.

Thanks in advance. I love the project.

After I try to update the techniques yaml file after I added a datasource, I get errors when I want to generate a new navigator layer:

So i have added packet capture logsource to my datasource file: data-sources-demo_withPcap.yaml

1. python dettect.py ds -ft input/techniques-administration-demo-all_update_driveby.yaml -fd input/data-sources-demo_withPcap.yaml --update

2. Error happens when:
   python dettect.py v -ft input/techniques-administration-demo-all_update_driveby.yaml -fd input/data-sources-demo_withPcap.yaml -l

One example of the error can be seen here (the line number might be off as I put in a try/except clause to provide you with some error data to work with)

Traceback (most recent call last):
File "/opt/DeTTECT/generic.py", line 591, in get_latest_score_obj
if not newest_score_obj or score_obj_date > newest_date:
TypeError: can't compare TimeStamp to datetime.date

yaml_object:ordereddict([('applicable_to', ['all']), ('comment', ''), ('score_logbook', [ordereddict([('date', datetime.datetime(2020, 10, 8, 13, 33, 1, 644981)), ('score', 2), ('comment', 'Datasource Packet Capture was added'), ('auto_generated', True)]), ordereddict([('date', TimeStamp(2020, 10, 8, 0, 0)), ('score', 1), ('comment', ''), ('auto_generated', True)])])])

score_obj:ordereddict([('date', TimeStamp(2020, 10, 8, 0, 0)), ('score', 1), ('comment', ''), ('auto_generated', True)])

newest_score_obj :ordereddict([('date', datetime.datetime(2020, 10, 8, 13, 33, 1, 644981)), ('score', 2), ('comment', 'Datasource Packet Capture was added'), ('auto_generated', True)])

Note that putting below code block into try/except in the end provided me a navigator json file I could load correctly

if not newest_score_obj or score_obj_date > newest_date:
    newest_date = score_obj_date
    newest_score_obj = score_obj

Hi!

I would like to automate the generation of ATT&CK Layers with DeTT&CT via GIT actions. Detecting if the generation succeeds would be much easier if the DeTT&CT CLI exits with a non-zero code after failure.
Especially for the health check of the yaml files, but also for example when a yaml does not exists.

Is this an idea or was it a deliberate choice to have only zero exit codes?

Alternative is to check the output of the DeTT&CT CLI with a second script and base the exit code on the output, however this would be not very easy and clean.

Greetings I get the following error when running the conversion any ideas please

[!] The below YAML file contains possible errors. It's recommended to check via the '--health' argument or using the option in the interactive menu:
- /mnt/c/Users/craig/Downloads/data-sources-new.yaml
Traceback (most recent call last):
File "dettect.py", line 365, in
_menu(_init_menu())
File "dettect.py", line 254, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, args.platform)
File "/home/craig/DeTTECT/data_source_mapping.py", line 24, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, platform, exceptions)
File "/home/craig/DeTTECT/data_source_mapping.py", line 261, in _map_and_colorize_techniques
total_ds_count = _count_applicable_data_sources(t, applicable_data_sources)
File "/home/craig/DeTTECT/data_source_mapping.py", line 240, in _count_applicable_data_sources
ds = ds.split(':')[1][1:]
IndexError: list index out of range

Hello,

I didn't found in Dettect the technique T1207. Is it normal ?
May I have to change the techniques-administration-endpoints.yaml files ?

Since few days, MITRE has added the subtechniques. Are there any code changes to be expected?

Thank you in advance,

generic.py

import os
import shutil
import pickle
from datetime import datetime as dt
from io import StringIO
from ruamel.yaml import YAML
from ruamel.yaml.timestamp import TimeStamp as ruamelTimeStamp
from upgrade import upgrade_yaml_file, check_yaml_updated_to_sub_techniques
from constants import *
from health import check_yaml_file_health

# Due to performance reasons the import of attackcti is within the function that makes use of this library.

local_stix_path = None


def _save_attack_data(data, path):
    """
    Save ATT&CK data to disk for the purpose of caching. Data can be STIX objects our a custom schema.
    :param data: the MITRE ATT&CK data to save
    :param path: file path to write to, including filename
    :return:
    """

    if not os.path.exists('cache/'):
        os.mkdir('cache/')
    with open(path, 'wb') as f:
        pickle.dump([data, dt.now()], f)


def load_attack_data(data_type):
    """
    By default the ATT&CK data is loaded from the online TAXII server or from the local cache directory. The
    local cache directory will be used if the file is not expired (data file on disk is older then EXPIRE_TIME
    seconds). When the local_stix_path option is given, the ATT&CK data will be loaded from the given path of
    a local STIX repository.
    :param data_type: the desired data type, see DATATYPE_XX constants.
    :return: MITRE ATT&CK data object (STIX or custom schema)
    """
    from attackcti import attack_client
    if local_stix_path is not None:
        if local_stix_path is not None and os.path.isdir(os.path.join(local_stix_path, 'enterprise-attack')) \
                and os.path.isdir(os.path.join(local_stix_path, 'pre-attack')) \
                and os.path.isdir(os.path.join(local_stix_path, 'mobile-attack')):
            mitre = attack_client(local_path=local_stix_path)
        else:
            print('[!] Not a valid local STIX path: ' + local_stix_path)
            quit()
    else:
        if os.path.exists("cache/" + data_type):
            with open("cache/" + data_type, 'rb') as f:
                cached = pickle.load(f)
                write_time = cached[1]
                if not (dt.now() - write_time).total_seconds() >= EXPIRE_TIME:
                    # the first item in the list contains the ATT&CK data
                    return cached[0]

        mitre = attack_client()

    attack_data = None
    if data_type == DATA_TYPE_STIX_ALL_RELATIONSHIPS:
->      attack_data = mitre.get_relationships()

Exception has occurred: InvalidJSONError
Invalid JSON was received from https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/objects/?match%5Btype%5D=relationship

Command: python dettect.py g

Hi,

In the current implementation, no distinction is made between not-specified technique detection scores and specified techniques with score -1. Both cases are not visible in the MITRE ATT&CK matrix.

It would be nice if all -1 detection scores where the date-field is non-empty would be visible in the matrix. Just only the comments without scoring / color would make it much clearer.

Currently, we often look at a technique in the attack-navigator and then do not know if the detection is that bad or if we just had not specified it yet.
You could, of course, assume that every technique has been filled-in, but in practice I think this works differently.

I currently work with other CTI teams and they have their own Navigator JSON files. I want to automate the Groups vs Coverage maps but every time they update their JSONs I have to manually update the YAML file.

Any chance DeTTECT can read the json and create a YAML Group file?

Possible Fields to add upon ingest into YAML or when you are able to import JSON from the Navigator
specific to Groups, but was curious of the LOE to add custom fields, some of these may have overlap already
[sorry for the list, just stood this up locally so I'm diving in right now]

Last Known Active
Services Used
Origins
Services Offered
Community Identifiers [additional Group names]
Customers
Target Nations
Victims
Target Industries / Sectors
Crimes
Reconnaissance
Weaponization
Delivery
Installation
C2
Actions & Objectives
Associated Malware
Monetization
Attack Vectors
Technical Tradecraft
Priority [personal]
Exploitation [CVEs]
Marketing
First Seen
Attribution

I someone can upload a youtube tutorial of blog how to use this. I'm lost. I can't follow the guide.

With the latest versions, the data source .yaml files generated no longer include references to individual TTP's, nor will the .json files generated by dettect.py ds -fd afterwards.

When trying to convert a data source yaml generated by the Editor to a json, the needs to be manually updated to have the new ATT&CK v9 data sources/data components as it currently contains ATT&CK v8 data sources. error is thrown.

Can this tool be used to map CVEs to Mitre ATT&CK Framework? Or any workaround for that in this tool?

Hi and thank you for the amazing project.

I am trying to generate a threat actor mapping for APT29 using the command python .\dettect.py g -g 'APT29'. While it completes successfully, it seems that the results are incomplete. For example, subtechnique T1546.003 shouldbe included in the resulting navigator layer but it is not.

As github prevents me from uploading the actual json file, I have attaching a screenshot of the navigator:

For any more information required please let me know.

-- Detect Tactics, Techniques & Combat Threats --
version 1.2.6

Menu: Data source mapping

Selected data source YAML file: input/sources.yaml

Options:

1. Only include data sources which match the provided EQL query:
2. Include all ATT&CK techniques in the generated YAML file that apply to the platform(s) specified in the data source YAML file: False

Select what you want to do:
3. Generate a data source layer for the ATT&CK Navigator.
4. Generate a graph with data sources added through time.
5. Generate an Excel sheet with all data sources.
6. Generate a technique administration YAML file with visibility scores, based on the number of available data sources
7. update the visibility scores within a technique administration YAML file based on changes within any of the data sources.
Past visibility scores are preserved in the score_logbook, and manually assigned scores are not updated without your approval.
The updated visibility are based on the number of available data sources.
8. Check the data sources YAML file for errors.
9. Back to main menu.

3
Writing data sources layer...
Traceback (most recent call last):
File "dettect.py", line 299, in
_menu(_init_menu())
File "dettect.py", line 184, in _menu
interactive_menu()
File "/opt/DeTTECT/interactive_menu.py", line 74, in interactive_menu
_menu_data_source(_select_file(MENU_NAME_DATA_SOURCE_MAPPING, 'data sources', FILE_TYPE_DATA_SOURCE_ADMINISTRATION))
File "/opt/DeTTECT/interactive_menu.py", line 304, in _menu_data_source
_menu_data_source(filename_ds)
File "/opt/DeTTECT/interactive_menu.py", line 301, in _menu_data_source
interactive_menu()
File "/opt/DeTTECT/interactive_menu.py", line 74, in interactive_menu
_menu_data_source(_select_file(MENU_NAME_DATA_SOURCE_MAPPING, 'data sources', FILE_TYPE_DATA_SOURCE_ADMINISTRATION))
File "/opt/DeTTECT/interactive_menu.py", line 276, in _menu_data_source
generate_data_sources_layer(file_ds)
File "/opt/DeTTECT/data_source_mapping.py", line 17, in generate_data_sources_layer
my_data_sources, name, platform, exceptions = _load_data_sources(filename)
File "/opt/DeTTECT/data_source_mapping.py", line 202, in _load_data_sources
exceptions = [t['technique_id'] for t in yaml_content['exceptions'] if t['technique_id'] is not None]
File "/usr/local/lib/python3.7/site-packages/ruamel/yaml/comments.py", line 753, in getitem
return ordereddict.getitem(self, key)
KeyError: 'exceptions'

I've just started using your great tool, but encuntered this error when trying to generate a data source layer for the ATT&CK Navigator.

Using data-sources-empty.yaml as template to document my data sources I noticed python dettect.py ds -fd sample-data/data-sources-endpoints.yaml -y to generate the techniques administration yaml file fails with below error message unless a string value for
exceptions:
- technique_id:
is specified. Specifying a random string will work. Would be nice if either the template mentioned this, had a default value set or the error handling would beadjusted.

Traceback (most recent call last):
File "dettect.py", line 296, in
_menu(_init_menu())
File "dettect.py", line 200, in _menu
generate_technique_administration_file(file_ds)
File "/root/Documents/dettect/DeTTECT/data_source_mapping.py", line 512, in generate_technique_administration_file
techniques_upper = list(map(lambda x: x.upper(), exceptions))
File "/root/Documents/dettect/DeTTECT/data_source_mapping.py", line 512, in
techniques_upper = list(map(lambda x: x.upper(), exceptions))
AttributeError: 'int' object has no attribute 'upper'

Files that need to be updated:

• .github/workflows/scripts/requirements_refresh_attack_data.txt
• requirements.txt

I tried to import some samples from https://github.com/rabobank-cdc/DeTTECT/tree/master/threat-actor-data but only one seems working fine at the Editor is ASCS.

For example https://raw.githubusercontent.com/rabobank-cdc/DeTTECT/master/threat-actor-data/20200220-FireEye.yaml

ASCS works fine https://raw.githubusercontent.com/rabobank-cdc/DeTTECT/master/threat-actor-data/20200520-ASCS.yaml

Just updated to 1.3.1 and the same problem as 1.3.0. It seems like only parsing techniques in the same line and not multiline.

Thanks

Please add the ability to either duplicate data source values or make bulk changes to an entire YAML within the DeTT&CT Editor

The command usage described in https://github.com/rabobank-cdc/DeTTECT/wiki/Threat-actor-group-mapping does not show how to make use of just the groups mentioned in groups.yml

Thus, I would like to add another command example as follows to the wiki:

python dettect.py g -g sample-data/groups.yml

Hi,

I'm trying to generate a layer to view in Attack and am encountering the below error:

Writing detection coverage layer with visibility as overlay...
Traceback (most recent call last):
File "dettect.py", line 366, in
_menu(_init_menu())
File "dettect.py", line 239, in _menu
interactive_menu()
File "C:\Users\test\Desktop\Dettect_new\interactive_menu.py", line 79, in interactive_menu
_menu_detection(_select_file(MENU_NAME_DETECTION_COVERAGE_MAPPING, 'techniques', FILE_TYPE_TECHNIQUE_ADMINISTRATION))
File "C:\Users\test\Desktop\Dettect_new\interactive_menu.py", line 362, in _menu_detection
generate_detection_layer(file_tech, filename_ds, True, None, None)
File "C:\Users\test\Desktop\Dettect_new\technique_mapping.py", line 30, in generate_detection_layer
mapped_techniques_both = _map_and_colorize_techniques_for_overlaid(my_techniques, my_data_sources, platform)
File "C:\Users\test\Desktop\Dettect_new\technique_mapping.py", line 341, in _map_and_colorize_techniques_for_overlaid
x['metadata'].append({'name': 'ATT&CK data sources', 'value': ', '.join(get_applicable_data_sources_technique(technique['x_mitre_data_sources'],
File "C:\Python38\lib\site-packages\stix2\base.py", line 216, in getitem
return self._inner[key]
KeyError: 'x_mitre_data_sources'

Can you please advise on what I'm doing wrong? Apologies if it's something obvious :)

Thanks,
Sherlon

Hi, when I tried to generate data based on selected groups "python dettect.py g -g 'fin7' -g 'cobalt group'", I received error below.

Traceback (most recent call last):
File "dettect.py", line 365, in
_menu(_init_menu())
File "dettect.py", line 290, in _menu
generate_group_heat_map(args.groups, args.overlay, args.overlay_type, args.platform,
File "/DeTTECT-master/group_mapping.py", line 572, in generate_group_heat_map
groups_dict = _get_group_techniques(groups, platform, groups_file_type)
File "/DeTTECT-master/group_mapping.py", line 211, in _get_group_techniques
found = _are_groups_found(groups_found, groups)
File "/DeTTECT-master/group_mapping.py", line 42, in _are_groups_found
if group_arg in group_aliases_lower or group_arg == get_attack_id(group).lower():
AttributeError: 'NoneType' object has no attribute 'lower'

I am trying to run DeTTECT on a network without external connectivity. I can see that when I generate a datasource layer from the sample data it reaches out to cti-taxii.mitre.org

$ python3 dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l
...
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='cti-taxii.mitre.org', port=443): Max retries exceeded with url: ...
...

Is it possibly to run the module without connecting out, by bringing in the static data?

Hi Marcus,

First of all, it's an absolutely fantastic tool from IR and hunting standpoint.Thanks for building it.
I'm editing inside Data Source and Techniques Xlsx files and looking for a way to convert those to yaml and then to json. I've tried converting xlsx to yaml using a python script but the output file wasn't supported by DeTTECT.

What is the best workaround for this issue ?

By the way, GUI editor looks good but I'm not sure if it's really useful as its definitely takes longer to fill than in excel.

Regards,
Nitin

These two functions are deprecated and have been merged into one:

• remove_revoked(attack_data)
• remove_deprecated(attack_data)

This function is new in version 0.3.6
https://github.com/OTRF/ATTACK-Python-Client/blob/0.3.6/attackcti/attack_api.py#L283-L290

Python Noob here, (sorry)

I tried and "think" I setup python correctly with all dependencies (using pip) on Windows 10 and Ubuntu but on both I always get the following error when trying to run dettact.py

Any help would be appreciated. If this is not an issue with the scripts/code and a python error please close the issue but I am really unsure.

Thank you

Traceback (most recent call last):
  File "dettact.py", line 202, in <module>
    menu(init_menu())
  File "dettact.py", line 27, in init_menu
    description='Create a heat map based on data sources, output data '
  File "C:\Python27\lib\argparse.py", line 1066, in add_parser
    parser = self._parser_class(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'aliases'

in the generic.py file, the use of yaml.load can be used to arbitrarily execute code, I would recommend switching it to yaml.safe_load. The code in question is as follows.

with open(filename, 'r') as yaml_file:
try:
yaml_content = yaml.load(yaml_file, Loader=yaml.FullLoader)

Would be fantastic if we could get coverage for the ICS and OT environments also. Fantastic app regardless however.

When using the command line argument --software-group, there is a note that it does not influence the scores. If the TA group uses a software/tool, which in turn covers a Technique/Subtechnique, shouldn't the Software TTP be added into the score of the heat map?

I'm curious as to what the design decision was for that caveat to be made, I might be missing something with regard to my understanding on how to track a group's known TTPs. After running a few tests, it is clear that a Group's TTP JSON layer does not necessarily include the Software TTP layer. Since this is the case, it seems that we should be adding the two JSON layers together to better understand the operational practices of the TA.

Thanks for the clarification.

I am finding that when generated an ATT&CK layer for visibility, when there are multiple data sources capable of detecting a technique, only the last data source's "Products" are being shown when I believe it should be appended to provide a full list of Products giving visibility.

After looking at the documentation for DETT&CT, I see that there are detection and visibility scores for techniques and data quality scores for data sources, but am unsure how they relate to each other. I looked at the sample YAML files and am still unclear on how data sources and techniques are correlated. Would you mind explaining this or showing me where I can find an explanation?

-Tim

Hello. I am having issues with the color not showing on the MITRE Navigator. I have dettect version v1.4.2 and I am using attack-navigator v4.2. However, when I upload the JSON file, it would upload without any issues, but the mapping isn't showing. I would like to continue to use the old version v8 of the MITRE attack. I was in the middle of a project and it will be too much to start over with version 9. Thanks!

I'm loving the tool - I'm just wondering if you have a best practices (or script) that would effectively make it so that I can map my existing data source spreadsheet to your taxonomic structure.

User Story:
I have CSV of my detections, rules, controls that contribute to them, and classification but I need to map to the standard ATT&CK format. I need to be able to import that existing CSV or use an 'input-able' form which imports the data within the CSV, to the technique-administration file (or data-source-administration file) rather than going one by one (although there ARE benefits to doing this one by one).

Thank you so much for making this tool. However, I am getting an error message when trying to convert YAML to JSON.

I used the command inside the folder where DETTECT is:

python dettect.py ds -fd a.yaml -l

a.yaml is what I named the file

The error message I get is:

[!] Cannot connect to MITRE's CTI TAXII Server

I've used this command last week and it worked great. I guess with the MITRE changes something happened.

Can you please help?

Thank you.

Hi,

I'm trying to overlay a threat actor group mapping with my detection coverage but I keep getting the below error whenever I select the techniques administration file.

Menu: Threat actor group mapping Options: 1. Software group: False 2. Platform: Windows 3. Groups: all 4. Overlay: - Groups: C:\Users\test\Desktop\DeTTECT\sample-data\groups-new.yaml - Type: group 5. EQL search: - Only include detection objects which match the EQL query: - Only include visibility objects which match the EQL query: - Include all 'score' objects from the 'score_logbook' in the EQL search: False Select what you want to do: 6. Generate a heat map layer. 9. Back to main menu. >> 4 1. Overlay with groups. 2. Overlay with detections. 3. Overlay with visibility. 4. No overlay. >> 2 Select the YAML file with techniques: Path: sample-data/ 1. sample-data\data-sources-empty.yaml 2. sample-data\data-sources-endpoints.yaml 3. sample-data\data-sources-partial.yaml 4. sample-data\groups-new.yaml 5. sample-data\groups.yaml 6. sample-data\techniques-administration-data-source-partial-windows.yaml 7. sample-data\techniques-administration-endpoints.yaml 8. sample-data\techniques-administration-v4.yaml 9. sample-data\UNC2452.yaml 14. Change path 15. Back to main menu. >> 8 Selected file: sample-data\techniques-administration-v4.yaml Press a key to continue -= DeTT&CT =- -- Detect Tactics, Techniques & Combat Threats -- version 1.4.2 Menu: Threat actor group mapping Options: 1. Software group: False 2. Platform: Windows 3. Groups: all 4. Overlay: - File: sample-data\techniques-administration-v4.yaml - Type: detection 5. EQL search: - Only include detection objects which match the EQL query: - Only include visibility objects which match the EQL query: - Include all 'score' objects from the 'score_logbook' in the EQL search: False Select what you want to do: 6. Generate a heat map layer. 9. Back to main menu. >> 6 [!] Unknown ATT&CK group: sample-data\techniques-administration-v4.yaml

This was done via the interactive menu.

Thanks,
Sherlon

Hello,

While executing dettect.py ds on an input .yaml file, I am receiving the below error:

[!] Cannot connect to MITRE's CTI TAXII server

What may be causing this?