santandersecurityresearch / cryptobom-forge Goto Github PK
View Code? Open in Web Editor NEWTools and utilities needed to parse GitHub Multi-Repository Variant Analysis output
License: MIT License
Tools and utilities needed to parse GitHub Multi-Repository Variant Analysis output
License: MIT License
Hello, more of a query than an issue. The instructions say:
The parameter is versatile, accepting either:
A path to a single CodeQL output file, or
A directory path containing multiple CodeQL outputs.
But I'm unsure what this equates to? A .sarif
file , .bqrs
file? And does a particular query have to have been run to form it?
I've tried running with .sarif
files but get:
code_snippet = codeql_result['locations'][0]['physicalLocation']['contextRegion']['snippet']['text']
KeyError: 'contextRegion'
my .sarif
file is sarif-2.1.0:
{
"$schema" : "https://json.schemastore.org/sarif-2.1.0.json",
"version" : "2.1.0",
"runs" : [ {
"tool" : {
"driver" : {
"name" : "CodeQL",
"organization" : "GitHub",
"semanticVersion" : "2.15.1",
I've then rerun my .sarif
file generation and included the flag --sarif-add-snippets
and then run the command again:
python3.10/site-packages/cyclonedx/model/bom.py", line 535, in register_dependency
ref=target.bom_ref,
AttributeError: 'NoneType' object has no attribute 'bom_ref'
And on a directory containing .bqrs
files and I get the following which looks rather empty:
$ cat cbom.json
{
"metadata": {
"timestamp": "2023-12-07T11:17:16.309580+00:00",
"tools": [
{
"externalReferences": [
{
"type": "build-system",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/actions"
},
{
"type": "distribution",
"url": "https://pypi.org/project/cyclonedx-python-lib/"
},
{
"type": "documentation",
"url": "https://cyclonedx.github.io/cyclonedx-python-lib/"
},
{
"type": "issue-tracker",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/issues"
},
{
"type": "license",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/LICENSE"
},
{
"type": "release-notes",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/CHANGELOG.md"
},
{
"type": "vcs",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib"
},
{
"type": "website",
"url": "https://cyclonedx.org"
}
],
"name": "cyclonedx-python-lib",
"vendor": "CycloneDX",
"version": "4.2.2"
}
]
},
"serialNumber": "urn:uuid:5cb9d639-0a96-48e8-af23-e4e5471ea6ce",
"version": 1,
"$schema": "https://raw.githubusercontent.com/IBM/CBOM/main/bom-1.4-cbom-1.0.schema.json",
"bomFormat": "CBOM",
"specVersion": "1.4-cbom-1.0"
}
Thanks in advance.
In the cbom.json, instead of reporting the specifc line of the finding, I get multiple lines:
"detectionContext": [
{
"additionalContext": "from hashlib import md5",
"filePath": "sqli/dao/dummy.py",
"lineNumbers": [
1,
2,
3
]
},
My output.sarif
file gives two sections of detail on the finding: region
and contextRegion
. region
gives the precise line(s) and this would be more useful in the cbom (in my opinion).
"detectionContext": [
{
"additionalContext": "from hashlib import md5",
"filePath": "sqli/dao/dummy.py",
"lineNumbers": [
1
]
},
Have created #5 with code update for this.
Hi, I'm using an output.sarif
file with both md5 and sha1 results in it.
"results" : [ {
"ruleId" : "py/quantum-readiness/cbom/all-cryptographic-algorithms",
"ruleIndex" : 1,
"rule" : {
"id" : "py/quantum-readiness/cbom/all-cryptographic-algorithms",
"index" : 1
},
"message" : {
"text" : "Use of algorithm MD5"
},
"locations" : [ {
"physicalLocation" : {
"artifactLocation" : {
"uri" : "sqli/dao/dummy.py",
"uriBaseId" : "%SRCROOT%",
"index" : 0
},
"region" : {
"startLine" : 1,
"startColumn" : 21,
"endColumn" : 24
},
"contextRegion" : {
"startLine" : 1,
"endLine" : 3,
"snippet" : {
"text" : "from hashlib import md5\r\nfrom hashlib import sha1\r\nfrom typing import NamedTuple, Optional\r\n"
}
}
}
} ],
"partialFingerprints" : {
"primaryLocationLineHash" : "a3859dd903b217e8:1",
"primaryLocationStartColumnFingerprint" : "20"
}
}, {
"ruleId" : "py/quantum-readiness/cbom/all-cryptographic-algorithms",
"ruleIndex" : 1,
"rule" : {
"id" : "py/quantum-readiness/cbom/all-cryptographic-algorithms",
"index" : 1
},
"message" : {
"text" : "Use of algorithm SHA1"
},
"locations" : [ {
"physicalLocation" : {
"artifactLocation" : {
"uri" : "sqli/dao/dummy.py",
"uriBaseId" : "%SRCROOT%",
"index" : 0
},
"region" : {
"startLine" : 2,
"startColumn" : 21,
"endColumn" : 25
},
"contextRegion" : {
"startLine" : 1,
"endLine" : 4,
"snippet" : {
"text" : "from hashlib import md5\r\nfrom hashlib import sha1\r\nfrom typing import NamedTuple, Optional\r\n\r\n"
}
}
}
} ],
However I can see that the snippet which is being provided for both is:
"snippet" : {
"text" : "from hashlib import md5\r\nfrom hashlib import sha1\r\nfrom typing import NamedTuple, Optional\r\n\r\n"
This means that when utils.py
performs:
def get_algorithm(code_snippet):
match = _ALGORITHM_REGEX.search(code_snippet)
if match:
return match.group()
return 'unknown'
The match is "MD5" on both occasions.
MD5 was the finding, MD5 detected as algorithm
SHA1 was the finding, MD5 detected as algorithm
The actual code in the script is:
from hashlib import md5
from hashlib import sha1
from typing import NamedTuple, Optional
from aiopg import Connection
...
...
I shall look look into whether the snippet can be less verbose,
[SARIF formats only] Include code snippets for each location mentioned in the results, with two lines of context before and after the reported location.
Because at the moment this can cause a problem for the match = _ALGORITHM_REGEX.search(code_snippet)
because re.search
just finds the first match (vs .findall
which will return a list of all non-overlapping matches in the input string).
So with this particular code, when the cbom and cryptocheck are generated, I'm only ever getting the MD5 finding.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.