Refactor into a Python package about changeme HOT 6 CLOSED

Hi,

I had some time, so I wanted to help you to change the body mechanism in order to build your project as a python package. I don't want to change everything without your agreement, so before to do anything, I want to know how your see it.

My idea was to keep the "parsed" dictionary identical as it is right now. So the changeme file will still the same (only the way to retrieve the parsed file will be changed).

My idea was to create a class for each category. For example, all yaml files will become python classes.
These class will inherit from another class which will contain functions to get the information we need.

For example, here how I see the parent object:

class SubObject():
    all_services = []
    def __init__(self, name, creds, category, ...):
        self.name = name
        self.creds = creds
        self.category = category

        SubObject.all_services.append(self)

    def name(self):
        return self.name

    def creds(self):
        return self.creds

    def category(self):
        return self.category

    def all(self):
        # get all information of the class
        return {'category':self.category, ...}

And here is an example of a service class (I took Tomcat as example):

class Tomcat(SubObject):

    def __init__(self):

        category = 'web'
        ssl = False
        name = 'Apache Tomcat'
        default_port = 8080
        contributor = 'ztgrace'

        fingerprint = {
            'status': 401, 
            'url': [
                '/manager/html', 
                '/tomcat/manager/html'
            ], 
            'basic_auth_realm': 'Tomcat Manager Application'
        }

        auth = {
            'url': [
                '/manager/html', 
                '/tomcat/manager/html'
            ], 
            'credentials': 
                [
                    {'username': 'tomcat', 'password': 'tomcat'},
                    {'username': 'admin', 'password': 'admin'},
                    {'username': 'ovwebusr', 'password': 'OvW*busr1'},
                    {'username': 'j2deployer', 'password': 'j2deployer'},
                    {'username': 'cxsdk', 'password': 'kdsxc'}, 
                    {'username': 'ADMIN', 'password': 'ADMIN'}, 
                    {'username': 'xampp', 'password': 'xampp'}, 
                    {'username': 'tomcat', 'password': 's3cret'},
                    {'username': 'QCC', 'password': 'QLogic66'}, 
                    {'username': 'admin', 'password': None}, 
                    {'username': 'admin', 'password': 'tomcat'}, 
                    {'username': 'root', 'password': 'root'}, 
                    {'username': 'role1', 'password': 'role1'}, 
                    {'username': 'role', 'password': 'changethis'}, 
                    {'username': 'tomcat', 'password': 'changethis'}, 
                    {'username': 'admin', 'password': 'j5Brn9'}, 
                    {'username': 'role1', 'password': 'tomcat'}
                ], 
            'type': 'basic_auth', 
            'success': 
                {
                    'body': 'Tomcat Web Application Manager', 
                    'status': 200
                }
        }

It is possible to split the "auth" and "fingerprint" variables to multiple variables but an prefix will be needed (for example, auth_urls or fingerprint_urls).

Then, it will be necessary to instanciate once the classe and all information of all classes will be available on the "SubObject" classes. For example:

for f in SubObject.all_contacts:
    print f.name

Here it prints the name, but it could retrieve the same dictionary as the "parsed" one.

I used quite the same technic on my project, here is the classes used as object:
https://github.com/AlessandroZ/LaZagne/blob/master/Windows/lazagne/config/moduleInfo.py

And here is the an example of class:
https://github.com/AlessandroZ/LaZagne/blob/master/Windows/lazagne/softwares/chats/jitsi.py

This is only ideas so do not hesitate to share your ideas. I won't do any commits before to be agreed with you about all of that.

See you !

from changeme.

Hey @AlessandroZ,

You've got some ideas right inline with what I was thinking. I want to create classes for each type of cred (web, ssh, etc.), but do that via inheritance/mixins. So there'd be a base class that would have credential pairs, default port, name contributor but be an "abstract" class that you wouldn't instantiate directly like I've got here: https://github.com/ztgrace/changeme/blob/refactor/changeme/protocol/protocol.py. Then a "web" cred would inherit from the base class and add additional fields required for its type of auth and fingerprinting. Hope that makes sense.

I still want to keep the creds themselves as YAML files, I think that makes them easier to manage in the long run.

I started refactoring a few days ago, but started with some of the more core scanning components. You can check out what I'm working on here: https://github.com/ztgrace/changeme/tree/refactor

from changeme.

I was interested about changing your code into a pure python project because my idea was to embed your project into this project that I'm working on: https://github.com/n1nj4sec/pupy/

It's very useful working on internal network, it allows to load python code into the remote host. Before finding your project, I wanted to do one similar.
However, if your code have yml files, it would be necessary to write it on the disk, which is not a good idea.

I understand that you still want to keep this architecture but you could use your tool only on your localhost.
If you have started to do lots of modification, it would be more difficult for me to help you.

Anyway, I could help you if you changed your mind about yml files.

Have a good day !

from changeme.

That's a cool use case. We should make that happen.

There's probably a few ways we could make this work. Do you know how the import process works for pupy?

Here's some ideas off the top of my head without understanding the import process:

• distutils seems to be able to handle additional files https://docs.python.org/2/distutils/setupscript.html#installing-additional-files but you're thinking that would write them to disk
• There's loader modules for the powershell code in https://github.com/n1nj4sec/pupy/blob/b66857c3d8fa4764cdc0945cbfcce1a69c3db3af/pupy/pupygen.py, We could potentially do the same, but that looks a little rough to maintain.
• create a separate setup.py install target to pre-compile the yaml files into python script which should be really straight forward as the yaml files are just serialized data formats
• see if there's some dynamic importing that could happen at compile time

Converting the creds is as straight forward as the code below. There'd just need to be a change in the main code to look for either a python file or parse the yaml files directly.

# python
Python 2.7.11 (default, Dec  5 2015, 14:44:53)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import changeme
>>> creds = changeme.load_creds(None, None)
Loaded 42 default credential profiles
Loaded 79 default credentials

>>> with open("creds.py", "wb") as fout:
...     fout.write("""#!/usr/bin/env python
... creds = %s""" % creds)

Let me know what you think.

from changeme.

On pupy, the client embed the python interpreter which allow to load python code on memory without needed to write files on the remote host. The mechanism to load python files is done on the client code (C code). However, it could be done calling a simple function from the remote server.

For example, I added Lazagne on pupy, which is a pure python package (only python code), and it needs some dependency to work (such as sqlite3, impacket, etc.).
Here is the line where lazagne is loaded and above lines are how all dependencies are loaded using the same way:
https://github.com/n1nj4sec/pupy/blob/master/pupy/modules/lazagne.py#L48

Once the module is loaded on memory it is possible to call the wanted function to run on the remote host:
https://github.com/n1nj4sec/pupy/blob/master/pupy/modules/lazagne.py#L53

Because of you don't have a pure python code, it's not possible to call a function like that because the creds database could not be found on the disk. I had another idea which will allow to use it without changing everything.
On server side, all yml are parsed and loaded into a dictionary (or 2 dict, one containing fingerprints and the other one containing creds) which is what you do at this moment. Then, we call the remote function passing these dictionaries:
https://github.com/ztgrace/changeme/blob/master/changeme.py#L828

The best way would be to return from the scan function all credentials found, to avoid calling the "report_creds" function after.

So the yml files would be parsed on serve side and the content will be sent to the client. That will avoid writing it on the disk.

Another point, concerning the "shodan" option, I don't think it will be useful in an internal network. I try to see how it would be possible to load it on pupy but it requires lots of different dependencies (different .so files, etc). So it could be a pain to add it. So I suggest to let this option (if you want to keep it) but the import will be done only if the option is called and using a try / except to not break it if the dependency is not satisfied.

I could help you to integrate your project to pupy. However, now that I see that you have already changed a lot of things on your architecture, I don't know what I can do to help you.

So let me know when you think I could add your project into pupy, or tell me if you need help for your project. Right now, I don't know how I can help you.

Have a good day.

from changeme.

Completed the major refactoring with #21. Let's work on incorporating some of the other changes you suggested like moving off of lxml now that unit testing is working. The tests/http.py:test_jboss_scan_success test will let us know if the modifications work or not.

from changeme.