ninoseki / ioc-extractor Goto Github PK

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update dependency eslint-plugin-prettier to v5.0.1
• chore(deps): update dependency ts-loader to v9.5.0
• chore(deps): update dependency typedoc to v0.25.2
• chore(deps): update dependency typescript to v5.2.2
• fix(deps): update dependency commander to v11.1.0
• chore(deps): update actions/checkout action to v4
• chore(deps): update dependency lint-staged to v15

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency prettier to v3.0.3
• chore(deps): update jest monorepo (@types/jest, jest)
• chore(deps): update dependency @types/node to v20.8.6
• chore(deps): update dependency eslint to v8.51.0
• chore(deps): update dependency eslint-plugin-jest to v27.4.2
• chore(deps): update dependency lint-staged to v13.3.0
• chore(deps): update typescript-eslint monorepo to v6.7.5 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

It would be great to add support for tracker type IOC (e.g. Google Analytics tracker)

• Google Analytics tracker ID format: UA-XXXXXXX-X

The URL regex matches non-URL value.

var iocExtractor = require("ioc-extractor")

const input = 'www.hoge.com';
const ioc = iocExtractor.getIOC(input);
console.log(ioc.networks.urls); // => ["www.hoge.com"];

www.hoge.com is a domain, not a URL.

For both emailRegex and nonStrictEmailRegex they do not account for dashes in the email addresses. By adding a dash like: [A-Za-z0-9_.-] it should capture more addresses.

I am using your great library in my community project and found that when working on PDF plain-text it would be great if your tool could filter out the following two Unicode remnants:

U+201C : LEFT DOUBLE QUOTATION MARK {double turned comma quotation mark}
U+201D : RIGHT DOUBLE QUOTATION MARK {double comma quotation mark}

as you can see from the example these have been recognized as part of the domain names

Example

https://orkl.eu/libraryEntry/637b8178-1bb0-4e7e-a1aa-d00c18bbcdd4

--> click on "IOCs"

“cloudsafety.online
“drive-globalordnance.com
“landof-service.com
“network-storage-ltd.com
“nonviolent-conflict-service.com
“proxycrioisolation.com
“share-drive-ua.com

In my use cases, I never use Files type IoCs (doc, exe, flash, img, mac, web and zip).
Removing Files type support means performance improvement so I'd like to drop the support.

If you need this kind of capability, I recommend to use the followings.

• Python:
  • cmu-sei/cyobstract
• Golang:
  • sroberts/cacador

It cannot deal with a http[:] type of defang technique.

var iocExtractor = require("ioc-extractor");

const input = 'http[:]//example1.com http://example2.com https[:]//example3.com';
const ioc = iocExtractor.getIOC(input);
console.log(ioc.networks.urls); // => ["http://example2.com"]

It would be good if it supports this type of defang/refang technique.

Hello, I really like this library, however recently I've come across a problem.

I noticed that when extracting IOC,

Input: (Valid URL)
https://1.1.1.1.domain.com:443

Expected output:
https://1.1.1.1.domain.com:443

Actual output:
https://1.1.1.1

I looked around and I believe that the url regex is causing the issue, specifically:

const url = `(?:${protocol})${auth}(?:localhost|${ipv4}|${domain})${port}${path}`;

Once localhost or ipv4 match is found, the group drops the rest.
This means this also produces unexpected output:

Input: (Valid URL)
https://localhost.domain.com:443

Expected output:
https://localhost.domain.com:443

Actual output:
https://localhost

Edit: Just to test my theory:

Input: (Valid URL)
https://1.1.1.domain.com:443

Expected output:
https://1.1.1.domain.com:443

Actual output: (this is correct)
https://1.1.1.domain.com:443

This shows that once ipv4 does not match, domain match is used instead.

Thank you for your time!

Edit 2:
I found a simple fix for the bug:

const url = `(?:${protocol})${auth}(?:${domain}|localhost|${ipv4})${port}${path}`;

cacador executable is a quite useful in CLI env.

pbpaste | cacador | jq . 

So adding an executable is a good idea IMHO.

Use VerbalExpressions to make regular expressions more readable

When given an input www.atmarkit.co.jp, the extractor returns atmarkit.co as a domain.

I have noticed in my usage that researchers may simply report a url in the form: domain[.]tld/someURI

The extractIOC function is not returning these in the iocs.urls array. Just getting domain in the domains array.

Could add another option for strictURLs maybe that makes the https?:// part of the regex optional if the option is set to false. Or simply make it optional without an option setting.

Can the tool be used to search in depth of a folder with subfolders of many files?
Ty

I’m trying to install ios-extractor in a docker container and tested several npm versions…
I did not find one that does not return any error… :(
Which version do you recommend? Somebody has a Dockerfile to share?
Here is the most common error across the tested versions:
Syntax error: Unexpected token export
Tx!