sfaci / masc Goto Github PK

When you create a new backup for a site, masc should check if a previous backup exists with the same name and maybe rename the new one adding something like '(1)' or similar (like browsers when you download twice the same file)

When a known files appears to be infected, masc should replace this file with a clean file from the clean installation

I've following the instruction but why always killed at 7%?

Develop a 'rollback' operation so that the user can restore his website with a previous backup created by masc

Currently masc delete directly every infected file if it detected some malware inside. Many times the website will be infected with specific scripts and the only action you can perform is to delete them. But we know that, sometimes, valid files are infected with injected code and removing the file is not the proper solution. We should delete the piece of maliciuos code and keep this file clean.

Currently, when you are going to clean a website, masc creates a backup with the name you provide (or with the no-name name when you don't provide it) and the website type as prefix, separated with the _ character.

Maybe there would be a goog idea to invent another way to save backups and its names.

Include Dockefile to ease the testing of the project.

If the Docker Image also exist in the DockerHub, would be great.

python masc.py -s -t custom -p test/

masc 0.2.2 (http://github.com/sfaci/masc)
Loading dictionaries and signatures. . .
Loading malware signature files |################################| 100%
Loaded 528746 malware signatures
Loading YARA rules . . . |################################| 100%
Some errors while reading yara rules. Some rules were not loaded
Loaded 717 YARA rules
done.
Loading web site . . .
done.
Making a backup . . .
A previous backups exists. Do you want to overwrite it? [y|N] y
Using no_name as backup name
done.
Searching for malware . . .
Scanning your website (ClamAV not found. Using only checksum and YARA rules databases) Traceback (most recent call last):
File "masc.py", line 98, in
results = cms.search_malware_signatures()
File "/wd/CMS.py", line 117, in search_malware_signatures
if entry.is_plain_text():
File "/wd/MascEntry.py", line 47, in is_plain_text
mg = magic.Magic(mime=True)
TypeError: init() got an unexpected keyword argument 'mime'

Create log files to save all the actions taken by masc so that the user can check later. In this way he can know what is happened

Add support for pyClamd to add new capabilities about virus detection
More info at http://xael.org/pages/pyclamd-en.html
https://www.decalage.info/en/python/pyclamd

It should check if the computer is connected to the Internet before downloading a clean installation. Otherwise an Python Exception raises

It seems ClamAV detects some WordPress files as malware. If you use ClamAV engine to scan the website some wordpress files appears as malware: wp-load.php, wp-mail.php and more.

It would be interesting to be able to ignore these files once you have checked that there are official files.

When masc is going to compare a web site with a clean installation (WordPress o Drupal), this is downloaded and unzipped in a temporary folder called cache. Currently the folder has to be created manually. It's better masc creates this folder automatically

Keep hashes and YARA databases in the Internet and download them when user execute masc for the first time and add an options to update them.

In this repository https://github.com/Yara-Rules/rules some people keep updated a lot of YARA rules. Maybe I could connect directly and load them in live

I don't know what to do with the unknown files where no malware is detected. They could be customize files or themes added to the clean installation.

Can I download the theme to check?

Can I offer this kind of file to remove by user manually?

Maybe I have to check manually these files and include them in a masc-ignore list? ( check #20)

Include the setup.py to make the project installable

The utility is causing the system to hang when loading yara rules. I have tried this in Kali and Parrot OS both running the latest versions and everything is updated. The loading of yara rules always ends up killing the scan.

Can anyone help me?

The function search_malware_signatures returns information about which malware is in every infected file. I thins it's a good idea to put a new option to print this information to the user

Now I filter the original result list to get only filenames

Add support for virustotal API.
I have researched and there is a test script virustotal_test.py with some code to use it.

More info at:

https://developers.virustotal.com/v2.0/reference#public-vs-private-api
https://www.virustotal.com/en/documentation/private-api/
https://www.virustotal.com/en/documentation/public-api/

Virus Total public API only allow 4 calls per minute and it seems private API is too expensive.
By the moment maybe I can only analyze with Virus Total some suspect files queueing task until it finishes

There are a couple of things that the linter that i use (Flake8) complains about (i excluded the line too long ones), these are the ones may be worth fixing:

• E703 statement ends with a semicolon (dictionary.py, line 40)
• E722 do not use bare 'except' (cms.py, lines 43, 93, 124 and 172)
• E713 test for membership should be 'not in' (cms.py, line 286)

These ones are minor but may also worth fixing:

• W292 no newline at end of file (print_utils.py, line 54)
• W391 blank line at end of file (drupal.py, line 40) (there are multiple blank lines actually)
• W391 blank line at end of file (joomla.py, line 21)
• W504 line break after binary operator (main.py, lines 52 and 137)
• W504 line break after binary operator (wordpress.py, line 77)
• E128 continuation line under-indented for visual indent (main.py, lines 53 and 138)
• F841 local variable 'spinner' is assigned to but never used (main.py, line 120)

Note: W504 and E128 are related to each other

PD: When solving W504, some flake8 versions could complain about W503, but seems like it's preferable that operators remain aligned.

• Flake8's W504 explanation
• PEP8's "Should a Line Break Before or After a Binary Operator?"

Currently masc creates some folders when you execute specific options:

• cache folder to store clean installations download during execution
• backups folder to store your backups
• logs folder to store logs

But these folders should be created at user home. Differents users can have differentes backups, for example.

I think the best approach is to keep these folders in a .masc folder in the user home directory.

In Dictionary.load_signatures, when I compile YARA rules, some of them throw an error.
Error are like this:
yara.SyntaxError: signatures/rules/gen_faked_versions.yar(27): undefined identifier "filename"

Maybe there are some difference in syntax between versiones of YARA/Python because using OWASP WebMalwareScanner with Python2 (it doesn't run with Python3) they run perfectly

This would be great to make ease the installation

Hi,

I'm trying to install the package onto an ubuntu server and I keep getting this issue ->

####################
ERROR: Command errored out with exit status 1:
command: /home/landon/app/appenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kcj9sfm_/masc/setup.py'"'"'; file='"'"'/tmp/pip-install-kcj9sfm_/masc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-kcj9sfm_/masc/pip-egg-info
cwd: /tmp/pip-install-kcj9sfm_/masc/
Complete output (7 lines):
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-kcj9sfm_/masc/setup.py", line 7, in
readme = pypandoc.convert_file('README.md', 'rst')
File "/home/landon/app/appenv/lib/python3.6/site-packages/pypandoc/init.py", line 135, in convert_file
raise RuntimeError("source_file is not a valid path")
RuntimeError: source_file is not a valid path
#######################

I'm not too sure what the issue is - but it seems to not like the README file. pypandoc doesn't accept the current path for some reason.

In this case, masc will try to use only local data. No Internet connection will be needed

Currently masc has specific support for WordPress and Drupal CMS but it's designed to support more CMS.
In the code you can encounter a CMS class where some generic methods are implemented and some specific classes to specific CMS.py (as WordPress.py and Drupal.py).

In fact, there is an empty class in the file Joomla.py but it's a good example about how to extends the CMS generic class to add specific support.

It would be great to improve masc adding support for another CMS or even some LMS (Moodle, for example) o any related known web application.

Create a web interface to manage the tool to scan local and remote sites

Hi,
This tool looks amazing !
Would it work with a custom wordpress folder structure ? I am using Roots/Bedrock.
If yes, would it be some additional steps to make it work with this kind of setup ?

Currently masc has to be executed in the host where the website is installed.

It would be interesting to add an option to scan remote sites. Maybe connecting remotely with the remote host o perhaps downloading the website and scanning it locally.

add functionality to test site throught url in wordpress, one function could be to test xmlrpc and check that the methods are available (mark more dangerous methods ) and if they don't use xmlrpc show how to disable

I'm trying to execute the below command as 'root' on ubuntu 20.04 + python 3.8.10:

masc --scan --site-type wordpress --path /var/www/html

But I only get error: "Can't open file or directory" like below for all files:

Scanning your website (Using ClamAV engine) \{'/var/www/html/wp-config-sample.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) /{'/var/www/html/wp-mail.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) \{'/var/www/html/wp-links-opml.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) /{'/var/www/html/wp-activate.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) \{'/var/www/html/wp-trackback.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) /{'/var/www/html/wp-admin/user/admin.php': ('ERROR', "Can't open file or directory")}
Scanning your website (Using ClamAV engine) \{'/var/www/html/wp-admin/user/user-edit.php': ('ERROR', "Can't open file or directory")}

Thanks to advice how to resolve this.

Principally masc search for malware, but it also tries to fix some issues that compromises the website security, such as wrong permission in some files and directories, empty directories and known files that reveal some details about the software (README, LICENSE, . . .).

It would be interesting think about new security issues that masc could fix and include them in the cleanup_site method that every specific CMS implements (Wordpress.py, Drupal.py or Custom.py for custom website).

The total number of rules and signatures loaded is higher than in the WebMalwareScanner case. I have to check that

Since the current setup.py was made in 2017, there are a couple of changes worth doing, as well as some fixes to comply with how pip and pypi handle packages.

Some of them are:

• Use setuptools instead of distutils
• Don't use pandoc
• Use a context manager for long_description
• Fix requirements:
  • Use install_requires instead of requires
  • Use correct requirements
• Don't specify download_url
• Add masc entrypoint

Currently there is a docker container with Debian + WordPress + masc ready to test in the docker hub. It has build some time ago for quick testing purpose.

It would be interesting automate the creation of this container to be always updated in the docker hub to perform some tests.

I think it would be very interesting include some manually-hacked websites to this container as well. There are two of them in the samples.zip. Even we can prepare some more.

Last released masc version on pypi is for 2017: https://pypi.org/project/masc/

I propose to create the release 0.3 and upload it.

Then change the install instructions in README.md to be pip install masc.

For development we can do pip install -e . after cloning the repo - this can be added to the wiki.

They have several signatures databases https://sanesecurity.com/usage/signatures/

Currently, when you need to recover an infected installation, you can restore a previous backup from this site done with masc. But you have to recover the entire backup.

Sometimes you will detect that only a specific file has been compromised, so it would be interesting to include an option to recover only this specific file (and maybe files) from backup