heinleinsupport / olefy Goto Github PK

Hi,

As I think problem in russian symbols, because "olevba3 -a -j -l error" work fine:

{
"vba_filename": "Лист3.cls",
"subfilename": "0L7RgdGC0YPQv9C7Li54bHM=X=",
"ole_stream": "_VBA_PROJECT_CUR/VBA/Лист3",
"code": null
}

In olefy we get:
rspamd python3[1123]: olefy ERROR default_exception_handler Fatal error: protocol.eof_received() call failed.
rspamd python3[1123]: protocol: <main.AIO object at 0x7f0468d61cf8>
rspamd python3[1123]: transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>>
rspamd python3[1123]: Traceback (most recent call last):
rspamd python3[1123]: File "/usr/lib/python3.7/asyncio/selector_events.py", line 823, in _read_ready__on_eof
rspamd python3[1123]: keep_open = self._protocol.eof_received()
rspamd python3[1123]: File "/usr/local/bin/olefy.py", line 172, in eof_received
rspamd python3[1123]: out = oletools(self.extra, tmp_file_name, lid)
rspamd python3[1123]: File "/usr/local/bin/olefy.py", line 104, in oletools
rspamd python3[1123]: out = bytes(out.decode("ascii").replace(' ', ' ').replace('\t', '').replace('\n', ''), encoding="ascii")
rspamd python3[1123]: UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1499: ordinal not in range(128)

thx

I made a fresh setup and followed the guide on Rspamd and here.

I can see the file is getting send to olefy , but looks like it's missing some data.
I enabled Debug.
Can some help debug the issue ?

Sep 27 17:02:33 atp-engine python3[4379]: olefy DEBUG data_received ('127.0.0.1', 37754) data received from new connection
Sep 27 17:02:35 atp-engine python3[4379]: olefy DEBUG data_received ('127.0.0.1', 37754) data received from new connection
Sep 27 17:02:42 atp-engine python3[4379]: olefy INFO eof_received No Rspamd-ID 405 bytes (stream size)
Sep 27 17:02:42 atp-engine python3[4379]: olefy ERROR eof_received No Rspamd-ID Protocol ERROR: no OLEFY/1.0 found
Sep 27 17:02:42 atp-engine python3[4379]: olefy INFO eof_received No Rspamd-ID ('127.0.0.1', 37754) response send: b'[ { "error": "Protocol error" } ]'
Sep 27 17:02:55 atp-engine python3[4379]: olefy DEBUG connection_made ('127.0.0.1', 38514) new connection was made
Sep 27 17:02:55 atp-engine python3[4379]: olefy DEBUG data_received ('127.0.0.1', 38514) data received from new connection
Sep 27 17:02:55 atp-engine python3[4379]: olefy INFO eof_received No Rspamd-ID 5 bytes (stream size)
Sep 27 17:02:55 atp-engine python3[4379]: olefy DEBUG eof_received ('127.0.0.1', 38514) PING request
Sep 27 17:02:55 atp-engine python3[4379]: olefy INFO eof_received No Rspamd-ID ('127.0.0.1', 38514) response send: b'PONG'

Hi,

olefy doesn't work here and errors out on debian buster.

on one system i get (not scanning):

Dec 08 08:27:12 debian python3[11842]: olefy ERROR default_exception_handler Fatal error: protocol.eof_received() call failed.
Dec 08 08:27:12 debian python3[11842]: protocol: <main.AIO object at 0x7f2d52a8a2e8>
Dec 08 08:27:12 debian python3[11842]: transport: <_SelectorSocketTransport fd=8 read=polling write=<idle, bufsize=0>>
Dec 08 08:27:12 debian python3[11842]: Traceback (most recent call last):
Dec 08 08:27:12 debian python3[11842]: File "/usr/lib/python3.7/asyncio/selector_events.py", line 823, in _read_ready__on_eof
Dec 08 08:27:12 debian python3[11842]: keep_open = self._protocol.eof_received()
Dec 08 08:27:12 debian python3[11842]: File "/opt/olefy/olefy.py", line 171, in eof_received
Dec 08 08:27:12 debian python3[11842]: rspamd_id = olefy_headers['Rspamd-ID'][:6] or ''
Dec 08 08:27:12 debian python3[11842]: KeyError: 'Rspamd-ID'

on the other system i get (scanning):

Dez 08 08:26:44 debian2 python3[13290]: olefy DEBUG data_received ('127.0.0.1', 57130) data received from new connection
Dez 08 08:26:44 debian2 python3[13290]: olefy ERROR default_exception_handler Fatal error: protocol.eof_received() call failed.
Dez 08 08:26:44 debian2 python3[13290]: protocol: <main.AIO object at 0x7f2bcfa980b8>
Dez 08 08:26:44 debian2 python3[13290]: transport: <_SelectorSocketTransport fd=8 read=polling write=<idle, bufsize=0>>
Dez 08 08:26:44 debian2 python3[13290]: Traceback (most recent call last):
Dez 08 08:26:44 debian2 python3[13290]: File "/usr/lib/python3.7/asyncio/selector_events.py", line 823, in_read_ready__on_eof
Dez 08 08:26:44 debian2 python3[13290]: keep_open = self._protocol.eof_received()
Dez 08 08:26:44 debian2 python3[13290]: File "/opt/olefy/olefy.py", line 171, in eof_received
Dez 08 08:26:44 debian2 python3[13290]: rspamd_id = olefy_headers['Rspamd-ID'][:6] or ''
Dez 08 08:26:44 debian2 python3[13290]: KeyError: 'Rspamd-ID'

All of the spam messages have a tooltip that gives you a rundown on what it actually found or what it's about, olefy does not? So I'm unsure why the email was blocked. I assume from a macro found in a file (as I said on my link)
I asked my question with screenshot is here:
rspamd/rspamd#3615

EDIT: someone answered on other thread

Port 10050 is already in use by Zabbiz Agent. Probably need to change the default configuration file, until the product has become popular.

Feature request for a little "ping? pong!" event via TCP to check if olefy is still alive.

When using application/octet-stream mime-type filter in Rspamd, many times it not an office file. Currently olefy just returnes "unhandled oletools error" instead of an concrete error message.

It'd be nice if you could analyse temporary files where oletools produced an error.

You can of course disable the deletion completely, but having an option to automatically delete the files that were handled correctly, but not the ones that failed, would help debug these problems and not get overwhelmed with every other file.

First of all - thank you for this rspamd extension. I've developed the MacroMilter as a possibility to integrated the olevba into the postfix several years ago. It's great to see some other approaches 😄 .

Do you have plans to add a whitelist for macros? My idea was to normalize the VBA code an generate a hash over it.
This will be provide the possibility to whitelist on the lowest level.

Hi, I see that olefy is calling olevba as an external script, and capturing its output + exit code. I think this is error prone and may lead to issues if we change the display or behaviour of olevba.

I would advise to import olevba and to use its python API instead:
https://github.com/decalage2/oletools/wiki/olevba#how-to-use-olevba-in-python-applications

Moreover, olevba is meant for malware analysts to look at the details of a file when they know it's suspicious or malicious.
If your goal is to decide (automatically) if a file is innocuous or suspicious, I would suggest to try mraptor instead. It is built on top of olevba, but applies a simple algorithm to detect suspicious keywords, and returns a clear result:
https://github.com/decalage2/oletools/wiki/mraptor

For example, it is used in the project MacroMilter for sendmail/postfix:
https://github.com/sbidy/MacroMilter

I have not yet documented the mraptor API, so for now you need to look at the code. But it's not a complex API.

olefy should scan a file using rtfobj when olevba reports the file is RFT and not OLE.

Since debian bookworm's pip installs oletools into a virtual python environment I'm not able to run olefy.
I'ld like to see an howto (install this under debian 12).