In addition to the mitre_att metadata field already in this spec, there should also be mitre_mbc. This is a newer and more appropriare classification system for malware samples than ATT&CK framework. The capa project has aligned with MBC over the past year as well.

https://github.com/MBCProject/mbc-markdown/blob/master/yfaq/README.md

Rules that contain non-ASCII characters in metadata are inappropriately rejected. YARA reads the rules in as bytes, and processes the metadata strings similarly to strings.

There are more issues with escaped values in metadata strings (\x00, \x80-\xFF) than with printable non-ASCII strings.

Recommend accepting non-ASCII characters in rules, modifying rules that contain values that can be unescaped (\000, \x80-\xFF, etc.), and performing a test compilation of the rules with yara-python.
VirusTotal/yara#1242

yr = yara.compile(source=r'rule test:tag {meta: emoji = "👍" escaped = "\x7E" condition: true}')
... print(yr.match(data='')[0].meta)
{'emoji': '👍', 'escaped': '~'}

yr = yara.compile(source=r'rule test:tag {meta: emoji = "👍" escaped = "\x7F" condition: true}')
... print(yr.match(data='')[0].meta)
{'emoji': '👍', 'escaped': '\x7f'}

yr = yara.compile(source=r'rule test:tag {meta: emoji = "👍" escaped = "\x80" condition: true}')
... print(yr.match(data='')[0].meta)
Process finished with exit code -1073741819 (0xC0000005)

Hey guys,

I found a strange bug. Under certain conditions re-running yara_validator_cli.py with inplace modification adds unnecessary whitespace to a file

I've been using cccs-yara to help validate a repo of yara rules. As part of that workflow, the rules end up being checked pretty consistently with the flags -i -v for in place changes to be made. Rerunning the validation can add up to kilobytes of whitespace to a file.

A test yara rule which triggers the bug can be seen below:

rule TEST_WhitespaceFeedback {
    meta:
        author = "TEST"
        date = "2021-02-04"
        description = "Testing to see how repeated uses of the validator result in large whitespace getting added"
        version = "1.0"
        info = "TEST CASE"
        category = "INFO"
        id = "h6DuDsSCZf8tvpoqorxKB"
        fingerprint = "ef3a9feef8f376382ecb07a970b37790ba56b7adce6bf072acfb8c94e0858ba6"
        first_imported = "2021-02-04"
        last_modified = "2021-02-04"
        status = "TESTING"
        sharing = "TLP:WHITE"
        source = "TESTING"

    strings:
        $s1 = "test123"
                        
         $s2 = "abc456"
        $s3 = "blah blah blah"

    condition:
        all of them
}

Note there is whitespace added longer than the usual tabs between $s1 and $s2.

When validating Yara rules with imported modules, the validator appears to fail. Any ideas?

$ cat ~/test.yara

import "pe"

rule test
{
    meta:
    condition:
        pe.number_of_resources > 1
}

$ python yara_validator_cli.py -v ~/test.yara

Validating Rule file:                   /root/test.yara
🍩 Invalid Rule File:                   /root/test.yara
     - Errors:
       - test.yara:                     Error Compiling YARA file with yara:    'line 6: syntax error'

When creating a PR I noticed the README documentation seems to be out of date.

• Documentation suggests using yara_validator_cli.py. However that does not appear to exist when installing via the latest from master. Setup.py appears to install this component as yara_validator. May need to switch the README to just yara_validator instead and update associated usage.
• Also noticed the library yara_validator.py appears to have been changed to just validator.py but is still referenced using the old name.

I'd toss in a PR candidate for this, but my French is not so great. Depending on the desired documentation updates being made, would require more than I can offer in that department! :)

In a few situations, a valid rule fails to be written (using -c or -i) with the following error:

./CCCS-Yara/yara-validator/yara_file_processor.py", line 184, in strings_of_rules_to_original_file
    changed_rule_string = rule.rule_return.validated_rule.splitlines()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'splitlines'

These regular expressions appear to the cause of the issue:

Based on these expressions:

• meta and : must be on the same line, and be the only contents of that line.
• The following strings or condition and : must be on the same line, and be the only contents of that line.

This breaks otherwise valid rules:

rule testing 
{ meta:
    key = "value"
  condition:
    true
}

rule testing {
  meta:
    key = "value"
  strings: $re = /test/
  condition: $re
}

rule testing 
{ meta:
    key = "value"
  condition: true
}

And on the most extreme end:

rule testing { meta: key = "value" condition: true }

Hi folks,

Compiled some new rules and updated to the standard - however, one rule imports the dotnet module - which the YARA validator does not seem to like as I'm getting an error compiling with an invalid field name of typelib.

In the yara_file_processor.py I see several modules that are not defined - are they not supported natively in AL? Or is there another reason to not have these included?

Thanks!

When running yara_validator_cli.py, the last_modified metadata attribute is being overwritten with the current date whether the value already exists in the yara rule or not. This is only happening for last_modified and not first_imported.

I would think the expected behavior is to leave the last_modified date as-is if found in the rule metadata.

The scenario where i ran into this was when importing an external ruleset that already had last_modified dates set. I would like to keep those dates as-is for context.

The CCCS-Yara specification lists TLP:AMBER+STRICT as a valid sharing classification but yara_validator_cli.py reports:

• Field has Invalid Value: TLP:AMBER+STRICT.

Validation works just fine when using TLP:AMBER.

Hey guys, your tool is amazing.

I would like a feature to make warnings get treated as errors.

In my workflow, I want to allow the metadata entry "hash" but don't want to allow "hash1, hash2, etc." Right now, if hash1 is included then a warning is thrown that it is similar to hash. I want that to be treated by an error.

If theres already a way to accomplish that, I'd happily take some advice on it.

Thanks again.

Private yara rules are not reported by yara when they match. I believe many people uses them for rules such as:

private rule IsPE {
    meta:
        description = "Ientifies Portable Executable binaries that has a valid magic in DOS and NT header"
    condition:
        (uint16(0) == 0x5A4D or uint16(0) == 0x4D5A) and uint32(uint32(0x3c)) == 0x00004550
}

Because these rules are never reported, and they are mostly referenced by other rules, does it make sense to ignore them in the validation process by adding a flag in the CLI or an option in the py package ?

This will save us from being obliged to add meta attributes to these rules. What do you think ?

Cheers.

Yara validation appears to succeed when the rule doesn't contain a closing bracket. Example yara rule:

rule APT_CN_TestYara {
   meta:
      description = "Detect malware"
      author = "TEST"
      date = "2021-01-12"
      version = "1.0"

   strings:
      $s1 = "test"
   condition:
      $s1

In case its relevant, there is no newline at the end of the file. This was tested using the default CCCS_YARA.yml file from commit (91e66c4)

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

I wonder why you haven't looked a the thousands of published YARA rules before you've decided on the fields in your standard.

I recommend the following changes to align your specs with the best practices in the real world:

source > reference

^ source could imply that the rule was copied from somewhere else, although most often it's derived from a sample or blog article.

first_imported > date

what does first imported even mean? It implies that you "import" a YARA rule somewhere and limits the field to that use case.

last_modified > modified

The prefix last_ is unnecessary.

Recommended further fields currently missing:

minimum_yara

^ indicating the minimal YARA version required to run the rule. This is very important since new modifiers, module features and keywords get added every few months. (e.g. icontains, xor)

e.g.

minimum_yara = "4.2"

Furthermore, have you considered using the YARA tags instead of certain fields?
For tags of all kinds, one could add a field tags and format it as a comma separated list.

e.g.

rule EXPL_LOG_ProxyNotShell_OWASSRF_PowerShell_Proxy_Log_Dec22_1_RID3C7E {
   meta:
      description = "Detects traces of exploitation activity in relation to ProxyNotShell MS Exchange vulnerabilities CVE-2022-41040 and CVE-2022-41082"
      author = "Florian Roth"
      reference = "https://www.crowdstrike.com/blog/owassrf-exploit-analysis-and-recommendations/"
      date = "2022-12-22 21:14:11"
      score = 70
      customer = "demo"
      license = "CC-BY-NC https://creativecommons.org/licenses/by-nc/4.0/"
      
      tags = "CVE_2022_41040, CVE_2022_41082, DEMO, EXPLOIT, HKTL, LOG, SCRIPT, T1028, T1059_001, T1086, T1090"
      minimum_yara = "1.7"
      
   strings:
      $s1 = "/owa/mastermailbox%40outlook.com/powershell" ascii wide
      $sa1 = " 200 " ascii wide
      $sa2 = " POST " ascii wide
      $fp1 = "ClientInfo" ascii wide fullword
      $fp2 = "Microsoft WinRM Client" ascii wide fullword
      $fp3 = "Exchange BackEnd Probes" ascii wide fullword
   condition: 
      all of ( $s* ) and not 1 of ( $fp* )
}

Following your approach, e.g. with the fields mitre_att and sharing, we would get an ever growing specification with many more fields. A field that lists all tags would be more ... practical IMHO.

Looking at the differences between generate_logic_hash from plyara.utils and calculate_rule_hash from the CCCS-Yara validator_functions, I can't see a significant benefit from duplicating the capability.

Is there an issue with the plyara function that hasn't been raised there?

Hello,

Sometimes, it is useful to reference more than one hash in a yara rule, this comes often when we have a same malware family, but multiple versions.

What's the best way to do that ? I saw some people just separating hashes with a ,.

The following meta tag in the sample rule in the README.md file is not CCCS valid:

technique = "LOADER:MEMORYMODULE"

This is due to the colon in place. Removing this colon makes the rule valid, but I'm not sure if that is what is intended.