ninoseki / ioc-extractor Goto Github PK
View Code? Open in Web Editor NEWAn npm package for extracting common IoC (Indicator of Compromise) from a block of text
License: MIT License
An npm package for extracting common IoC (Indicator of Compromise) from a block of text
License: MIT License
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates are awaiting their schedule. Click on a checkbox to get an update now.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
@types/jest
, jest
)@typescript-eslint/eslint-plugin
, @typescript-eslint/parser
).github/workflows/test.yml
actions/checkout v3
actions/setup-node v3
package.json
commander ^11.0.0
get-stdin 9.0.0
@types/get-stdin ^7.0.0
@types/jest ^29.5.3
@types/node 20.4.10
@typescript-eslint/eslint-plugin 6.3.0
@typescript-eslint/parser 6.3.0
benny 3.7.1
coveralls 3.1.1
eslint 8.47.0
eslint-config-prettier 9.0.0
eslint-plugin-jest 27.2.3
eslint-plugin-prettier 5.0.0
eslint-plugin-simple-import-sort 10.0.0
husky 8.0.3
jest 29.6.2
lint-staged 13.2.3
prettier 3.0.1
ts-jest 29.1.1
ts-loader 9.4.4
ts-node 10.9.1
tsup ^7.2.0
typedoc 0.24.8
typescript 5.1.6
node >=18
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.
It would be great to add support for tracker type IOC (e.g. Google Analytics tracker)
The URL regex matches non-URL value.
var iocExtractor = require("ioc-extractor")
const input = 'www.hoge.com';
const ioc = iocExtractor.getIOC(input);
console.log(ioc.networks.urls); // => ["www.hoge.com"];
www.hoge.com
is a domain, not a URL.
For both emailRegex
and nonStrictEmailRegex
they do not account for dashes in the email addresses. By adding a dash like: [A-Za-z0-9_.-]
it should capture more addresses.
ioc-extractor/src/aux/regexes.ts
Lines 142 to 144 in 5815c6d
I am using your great library in my community project and found that when working on PDF plain-text it would be great if your tool could filter out the following two Unicode remnants:
U+201C : LEFT DOUBLE QUOTATION MARK {double turned comma quotation mark}
U+201D : RIGHT DOUBLE QUOTATION MARK {double comma quotation mark}
as you can see from the example these have been recognized as part of the domain names
Example
https://orkl.eu/libraryEntry/637b8178-1bb0-4e7e-a1aa-d00c18bbcdd4
--> click on "IOCs"
“cloudsafety.online
“drive-globalordnance.com
“landof-service.com
“network-storage-ltd.com
“nonviolent-conflict-service.com
“proxycrioisolation.com
“share-drive-ua.com
In my use cases, I never use Files
type IoCs (doc
, exe
, flash
, img
, mac
, web
and zip
).
Removing Files
type support means performance improvement so I'd like to drop the support.
If you need this kind of capability, I recommend to use the followings.
It cannot deal with a http[:]
type of defang technique.
var iocExtractor = require("ioc-extractor");
const input = 'http[:]//example1.com http://example2.com https[:]//example3.com';
const ioc = iocExtractor.getIOC(input);
console.log(ioc.networks.urls); // => ["http://example2.com"]
It would be good if it supports this type of defang/refang technique.
Hello, I really like this library, however recently I've come across a problem.
I noticed that when extracting IOC,
Input: (Valid URL)
https://1.1.1.1.domain.com:443
Expected output:
https://1.1.1.1.domain.com:443
Actual output:
https://1.1.1.1
I looked around and I believe that the url regex is causing the issue, specifically:
const url = `(?:${protocol})${auth}(?:localhost|${ipv4}|${domain})${port}${path}`;
Once localhost or ipv4 match is found, the group drops the rest.
This means this also produces unexpected output:
Input: (Valid URL)
https://localhost.domain.com:443
Expected output:
https://localhost.domain.com:443
Actual output:
https://localhost
Edit: Just to test my theory:
Input: (Valid URL)
https://1.1.1.domain.com:443
Expected output:
https://1.1.1.domain.com:443
Actual output: (this is correct)
https://1.1.1.domain.com:443
This shows that once ipv4 does not match, domain match is used instead.
Thank you for your time!
Edit 2:
I found a simple fix for the bug:
const url = `(?:${protocol})${auth}(?:${domain}|localhost|${ipv4})${port}${path}`;
cacador executable is a quite useful in CLI env.
pbpaste | cacador | jq .
So adding an executable is a good idea IMHO.
Use VerbalExpressions to make regular expressions more readable
When given an input www.atmarkit.co.jp
, the extractor returns atmarkit.co
as a domain.
I have noticed in my usage that researchers may simply report a url in the form: domain[.]tld/someURI
The extractIOC function is not returning these in the iocs.urls array. Just getting domain in the domains array.
Could add another option for strictURLs maybe that makes the https?:// part of the regex optional if the option is set to false. Or simply make it optional without an option setting.
Can the tool be used to search in depth of a folder with subfolders of many files?
Ty
I’m trying to install ios-extractor in a docker container and tested several npm versions…
I did not find one that does not return any error… :(
Which version do you recommend? Somebody has a Dockerfile to share?
Here is the most common error across the tested versions:
Syntax error: Unexpected token export
Tx!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.