cisco-talos / first-server Goto Github PK

View Code? Open in Web Editor NEW

32.0 32.0 20.0 2.06 MB

License: GNU General Public License v2.0

Shell 0.06% Python 19.54% HTML 3.56% CSS 49.51% JavaScript 27.27% Dockerfile 0.06%

first-server's People

Contributors

Stargazers

Watchers

Forkers

secsandcyber p-state baderj icchy jellever liamjm peevtod demonduck pombredanne xabiugarte chehao lucamassarelli jimmy-sonny alirezachegini 5l1v3r1 duk-37 savobit empty2fill macoconne

first-server's Issues

Google Authenticator problem.

Hello,
Is there any way to disuse Google Authenticator, we are in China, you know that google is forbid in China,
google cannot be accessed directly, unless using a proxy.

Best regards.
Jessie Lee

Migration to Django ORM

Currently part of FIRST uses Django ORM, and the other part uses MongoDB. This was for quick prototyping, now it is time to update the project to stick with one technology.

can not register again

hello,
after register from web and install 3 engines, can not register second time
web always return 'User already exists'

can not installs engines

Hello,
there is an error when I use command in bash, Information is as follows:

#cd FIRST-server/server/utilities
#python engine_shell.py install first.engines.exact_match ExactMatchEngine [email protected]
install - Installs engine

Traceback (most recent call last):
File "engine_shell.py", line 341, in
shell.onecmd(' '.join(sys.argv[1:]))
File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
return func(arg)
File "engine_shell.py", line 142, in do_install
developer = User.objects(email=email).get()
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 276, in get
raise queryset._document.DoesNotExist(msg)
first.models.DoesNotExist: User matching query does not exist.
root@ubuntu:/home/jessie/FIRST-server/server/utilities#

best regards
Jessie Lee

[Feature Request] Make signature database downloadable

Hi,

I have a local install of the FIRST server running as we have a completely separate analysis environment which in no way talks to the internet. Is it possible to sync my local server with the community server or download the signature database somewhere so I can apply it locally?

Thanks!

Jelle

Invalid template library specified. ImportError [...] cannot import name getfullargspec

When running FIRST-server, the following error is produced:

Invalid template library specified. ImportError raised when trying to load 'django.contrib.admin.templatetags.base': cannot import name getfullargspec

This is error has been introduced in the last Django update (1.11.19):

https://groups.google.com/forum/#!topic/django-developers/B3Wf6yXCWqs

Downgrading to 1.11.18 temporarily solves the issue. One can update the 'install/requirements.txt' to match the correct django version: django==1.11.19

Assigning this issue to myself to keep track of the status of this issue.

first_api container is unable to connect to mysql

After the fresh installation of the FIRST server using Docker, I can not use it. Logs are provided below.

docker logs first_api_1

Using existing SSL certificate
[1st] IOError: [Errno 2] No such file or directory: '/home/first/first/../first_config.json'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/dist-packages/MySQLdb/__init__.py", line 84, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/MySQLdb/connections.py", line 164, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/first/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 350, in execute
    self.check()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 379, in check
    include_deployment_checks=include_deployment_checks,
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/commands/migrate.py", line 59, in _run_checks
    issues = run_checks(tags=[Tags.database])
  File "/usr/local/lib/python3.6/dist-packages/django/core/checks/registry.py", line 71, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/local/lib/python3.6/dist-packages/django/core/checks/database.py", line 10, in check_database_backends
    issues.extend(conn.validation.check(**kwargs))
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/validation.py", line 9, in check
    issues.extend(self._check_sql_mode(**kwargs))
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/validation.py", line 13, in _check_sql_mode
    with self.connection.cursor() as cursor:
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 255, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 232, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/dist-packages/MySQLdb/__init__.py", line 84, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/MySQLdb/connections.py", line 164, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.4. Set the 'ServerName' directive globally to suppress this message

docker logs first_mysql_1

2019-03-12T13:01:24.860546Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-03-12T13:01:24.878578Z 0 [Note] mysqld (mysqld 5.7.25) starting as process 1 ...
2019-03-12T13:01:24.932836Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-03-12T13:01:24.932898Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-03-12T13:01:24.932904Z 0 [Note] InnoDB: Uses event mutexes
2019-03-12T13:01:24.932908Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-03-12T13:01:24.932912Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-03-12T13:01:24.932916Z 0 [Note] InnoDB: Using Linux native AIO
2019-03-12T13:01:24.938259Z 0 [Note] InnoDB: Number of pools: 1
2019-03-12T13:01:24.960804Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-03-12T13:01:24.988673Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-03-12T13:01:25.160340Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-03-12T13:01:25.169485Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-03-12T13:01:25.206608Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-03-12T13:01:25.315532Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-03-12T13:01:25.315713Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-03-12T13:01:25.451939Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-03-12T13:01:25.467404Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2019-03-12T13:01:25.467461Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2019-03-12T13:01:25.468061Z 0 [Note] InnoDB: Waiting for purge to start
2019-03-12T13:01:25.520594Z 0 [Note] InnoDB: 5.7.25 started; log sequence number 12358972
2019-03-12T13:01:25.521210Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2019-03-12T13:01:25.522360Z 0 [Note] Plugin 'FEDERATED' is disabled.
2019-03-12T13:01:25.721923Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2019-03-12T13:01:25.724115Z 0 [Warning] CA certificate ca.pem is self signed.
2019-03-12T13:01:25.725024Z 0 [Note] InnoDB: Buffer pool(s) load completed at 190312 13:01:25
2019-03-12T13:01:25.730797Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2019-03-12T13:01:25.747044Z 0 [Note] IPv6 is available.
2019-03-12T13:01:25.767023Z 0 [Note]   - '::' resolves to '::';
2019-03-12T13:01:25.767138Z 0 [Note] Server socket created on IP: '::'.
2019-03-12T13:01:25.795319Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2019-03-12T13:01:25.836635Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.836739Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.836752Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.837143Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.837160Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.854110Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.897319Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:25.897370Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-03-12T13:01:26.189657Z 0 [Note] Event Scheduler: Loaded 0 events
2019-03-12T13:01:26.190141Z 0 [Note] mysqld: ready for connections.
Version: '5.7.25'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)

Perhaps the problem is that first_config.json is missing, but I cannot find documentation on how to properly configure this file.

can not login

hello,
when click login from web, it returns 403
the infomation is as follows:

Forbidden (403)
CSRF verification failed. Request aborted.
Help
Reason given for failure:
CSRF token missing or incorrect.

In general, this can occur when there is a genuine Cross Site Request Forgery, or when Django's CSRF mechanism has not been used correctly. For POST forms, you need to ensure:
Your browser is accepting cookies.
The view function passes a request to the template's render method.
In the template, there is a {% csrf_token %} template tag inside each POST form that targets an internal URL.
If you are not using CsrfViewMiddleware, then you must use csrf_protect on any views that use the csrf_token template tag, as well as those that accept the POST data.
The form has a valid CSRF token. After logging in in another browser tab or hitting the back button after a login, you may need to reload the page with the form, because the token is rotated after a login.
You're seeing the help section of this page because you have DEBUG = True in your Django settings file. Change that to False, and only the initial error message will be displayed.
You can customize this page using the CSRF_FAILURE_VIEW setting.

Best regards
Jessie Lee

Missing file extension

$ cd FIRST-server/server/utilities
$ python engine_shell install first.engines.exact_match ExactMatchEngine <developer_email>
$ python engine_shell install first.engines.mnemonic_hash MnemonicHashEngine <developer_email>
$ python engine_shell install first.engines.basic_masking BasicMaskingEngine <developer_email>

engine_shell is actually engine_shell.py

Add users by CLI

Currently there is not a streamlined way to use FIRST locally without OAuth.

Add a FIRST-server/server/utilities script to allow command line creation of new users

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
python manage.py shell
n = first.models.User(.........)
n.save()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Missing google_secret.json blocks docker-compose

COPY install/google_secret.json /usr/local/etc

there is no install/google_secret.json in the repo so the above line terminates the build

BasicMaskingEngine

The conversion from distorm3 to capstone allows for more architectures to leverage this engine, however, the intel based masking is not filled with as many masking abilities. Update the engine to contain equivalent functionality as was available with distorm3

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.