Git Product home page Git Product logo

antedebaas / dmarc-smtptls-reports Goto Github PK

View Code? Open in Web Editor NEW
73.0 10.0 15.0 934 KB

DMARC & SMTP-TLS Reports processor and visualizer

Home Page: https://hub.docker.com/repository/docker/antedebaas/dmarc-reports/general

License: GNU General Public License v2.0

PHP 64.80% CSS 0.17% Twig 33.30% Shell 1.28% Dockerfile 0.45%
dmarc dmarc-aggregate-reports dmarc-analyzer dmarc-assistant dmarc-parser dmarc-record dmarc-reports php php8 mta-sts

dmarc-smtptls-reports's Introduction

Contributors Forks Docker Stargazers Issues MIT License

About The Project

Dashboard DMARC-Reports SMTP-TLS-Reports

I needed a quick and easy way to view my dmarc and smtp tls reports. This is a simple symfony project that can read the inbox for new reports and process them. I you use it you should use it with a special sole-purpose email address.

(back to top)

Getting Started

Setup is pretty easy

Prerequisites

  • composer
  • mariadb (10.5+)/postgresql (16.1+)/sqlite
  • php (8.2+)
  • php-ctype
  • php-dom
  • php-fileinfo
  • php-fpm
  • php-iconv
  • php-imap
  • php-mbstring
  • php-pdo
  • php-pdo-mysql
  • php-pdo-pgsql
  • php-pdo-sqlite
  • php-phar
  • php-session
  • php-simplexml
  • php-tokenizer
  • php-xml
  • php-xmlwriter
  • php-zip

Installation

  1. Clone the repo
    git clone https://github.com/antedebaas/DMARC-SMTPTLS-Reports.git
  2. run update.sh to update project, clear its cache, this prevents symfony caching issues
    bash [root path of this project]/update.sh
  3. point the webserver root to the public/ directory
  4. point your webbrowser to http(s)://[yourhost]/setup and follow instructions
  5. run installservice.sh to install the systemd service and timer for automated mail checking
    bash [root path of this project]/installservice.sh
  6. if you put it on https://mta-sts.yourmdomain.ext it will provide an mta-sts policy file (https://mta-sts.yourmdomain.ext/.well-known/mta-sts.txt) you can edit the policy on the domain edit page.

(back to top)

Docker

  1. see the docker-compose.yml file for all variables and an example stack.
  2. MAILCHECK_SCHEDULE can be adjusted to check the mailbox more or less frequent using a cron syntax. you can also use one of the following keywords: monthly, weekly, daily, hourly, 15min
  3. run docker compose up

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

(back to top)

License

Distributed under the GPL v2 License. See LICENSE.txt for more information.

(back to top)

dmarc-smtptls-reports's People

Contributors

antedebaas avatar basti1508 avatar daygle avatar dependabot[bot] avatar gurglingtonic avatar keeper-of-the-keys avatar kimdre avatar wvogel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dmarc-smtptls-reports's Issues

This is not working on for me

Hi,

I tried running this on ubuntu 22.04 and it gave me lot of dependencies and it could not run. Then I tried with Docker and it gave to me weird error which forced to comment the name and containers stated working fine. I then pointed to imap.gmail.com and not a single mail was downloaded nothing appeared on dashboard.

Is this something you can help me with?

Bug that prevents mail deletion

There is an error hat will cause an http error 500 when setting DELETE_PROCESSED_MAILS=true in .env.local.

Line 92 of src/Command/GetReportsFromMailboxCommand.php reads:
$mailbox->deleteMail($mailId);

It has to be:
$mailbox->deleteMail($mailid);

I sent a pull request...

Error: ($policy_string_maxage) must be of type int, string given

After importing my 300+ emails, subsequent runs of /php bin/console -q app:getreportsfrommailbox
result in the following error:

[critical] Error thrown while running command "-q app:getreportsfrommailbox". Message: "App\Entity\SMTPTLS_Policies::setPolicyStringMaxage(): Argument #1 ($policy_string_maxage) must be of type int, string given, called in src/Command/GetReportsFromMailboxCommand.php on line 380"
[critical] Uncaught Error: App\Entity\SMTPTLS_Policies::setPolicyStringMaxage(): Argument #1 ($policy_string_maxage) must be of type int, string given, called in src/Command/GetReportsFromMailboxCommand.php on line 380

In SMTPTLS_Policies.php line 118:

  App\Entity\SMTPTLS_Policies::setPolicyStringMaxage(): Argument #1 ($policy_string_maxage) must be of type int, string given, called in src/Command/GetReportsFromMailboxCommand.php on line 380


app:getreportsfrommailbox

If I query the mySQL database:

select * from smtptls_policies order by policy_string_maxage limit 10;

I get the following:

+-----+------------------+-----------+-----------------+-----------------------+--------------------+----------------------+--------------------------+----------------------+
| id  | policy_domain_id | report_id | policy_type     | policy_string_version | policy_string_mode | policy_string_maxage | summary_successful_count | summary_failed_count |
+-----+------------------+-----------+-----------------+-----------------------+--------------------+----------------------+--------------------------+----------------------+
|  87 |                1 |        87 | no-policy-found | NULL                  | NULL               |                 NULL |                        1 |                    0 |
| 142 |                1 |       142 | no-policy-found | NULL                  | NULL               |                 NULL |                        1 |                    0 |
| 143 |                1 |       143 | no-policy-found | NULL                  | NULL               |                 NULL |                        1 |                    0 |
| 144 |                1 |       144 | no-policy-found | NULL                  | NULL               |                 NULL |                        1 |                    0 |
| 164 |                1 |       164 | sts             | NULL                  | NULL               |                 NULL |                        0 |                    2 |
| 166 |                1 |       166 | sts             | NULL                  | NULL               |                 NULL |                        0 |                    6 |
|   1 |                1 |         1 | sts             | STSv1                 | testing            |                86401 |                        1 |                    0 |
|   2 |                1 |         2 | sts             | STSv1                 | testing            |                86401 |                        1 |                    0 |
|   3 |                1 |         3 | sts             | STSv1                 | testing            |                86401 |                        6 |                    0 |
|   4 |                1 |         4 | sts             | STSv1                 | testing            |                86401 |                        2 |                    0 |
+-----+------------------+-----------+-----------------+-----------------------+--------------------+----------------------+--------------------------+----------------------+

If I update the NULL's and change them to 86401 everything runs normally.

Is there same way to know which messages those are so that I can send you a copy?

Sqlite upgrade error to v1.8.0

Hi,

When upgrading to 1.8.0 with an sqlite database, the table upgrade (migrations/sqlite/Version20240304200718.php) shows an error.
The query:

ALTER TABLE logs ADD COLUMN success BOOLEAN NOT NULL;

gives an error because sqlite expects a default value it seems for the not null column. I changed it to:

ALTER TABLE logs ADD COLUMN success BOOLEAN NOT NULL default false;

and it seems to be happy.

Regards,

Mark

BUG: Error 500 deleting Microsoft Corporation SMTPTLS reports

"500 Internal Server Error" while trying to delete a Microsoft Corporation SMTPTLS report

TLDR Error:

NOTICE: PHP message: [critical] Uncaught Exception: An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails (sys_dmarc_reports.smtptls_rdata_records, CONSTRAINT FK_65C68ADB2D29E3C6 FOREIGN KEY (policy_id) REFERENCES smtptls_policies (id))

Docker log:

2024/05/16 01:46:09 [error] 18#18: *183 FastCGI sent in stderr: "PHP message: [critical] Uncaught Exception: An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails (`sys_dmarc_reports`.`smtptls_rdata_records`, CONSTRAINT `FK_65C68ADB2D29E3C6` FOREIGN KEY (`policy_id`) REFERENCES `smtptls_policies` (`id`))" while reading response header from upstream, client: 172.17.0.1, server: _, request: "GET /reports/smtptls/delete/34 HTTP/1.0", upstream: "fastcgi://unix:/run/php-fpm.sock:", host: "emailreports.REDACTED.es", referrer: "https://emailreports.REDACTED.es/reports/smtptls?page=1&perpage=17"
172.17.0.1 - - [16/May/2024:01:46:09 +0200] "GET /reports/smtptls/delete/34 HTTP/1.0" 500 1017 "https://emailreports.REDACTED.es/reports/smtptls?page=1&perpage=17" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36" "83.34.0.121" 0.105 0.105 . -

Sample report

image

Thank you very much for the project!

Domain not auto-created

Hey ante,

First of thanks for this nice interface!

I recently setup this system for the first time, after finishing the setup the domains table was not populated, this prevented reports from being parsed as well as mta-sts.txt files to be generated.
I ended up manually adding an entry directly in SQL.

As I'm not sure if this is an issue of documentation or that my specific setup is strange in a way that messed up the process I'm first of all opening a ticket, more than happy to create a PR to add the documentation.

The domains page does not seem to have an "add domain" button either so the only option I seemed to have was SQL.

Thanks!

Checkmail button timeout

When I use the Check Mail Now button, I run into Gateway time-out.

I see this error in log.

Port 143
NOTICE: PHP message: [critical] Error thrown while running command "app:checkmailbox". Message: "["TLS\/SSL failure for mail.mailserver.com: SSL negotiation failed"]"

Port 993
NOTICE: PHP message: [critical] Error thrown while running command "app:checkmailbox". Message: "["IMAP SERVER BUG (invalid challenge): \"\"","Can not authenticate to IMAP server: [CLOSED] IMAP connection broken (server response)"]"
Please note that Port 143 works well with mail clients that are connected to the same mailbox.

I got this response from the mail server developer

If it works with your mail clients then it looks like an issue with your PHP code
Probably the TLS library is using a deprecated ciphersuite and it is not possible to negotiate a TLS connection

Latest logs empty

Hi,

The latest log windows is empty:
image

The web container have this error:
NOTICE: PHP message: [critical] Error thrown while running command "app:getreportsfrommailbox". Message: "Cannot create the attachments directory "/var/www/html/var/imap/attachments""
2024/03/13 14:07:12 [error] 17#17: *19 FastCGI sent in stderr: "PHP message: [critical] Error thrown while running command "app:getreportsfrommailbox". Message: "Cannot create the attachments directory "/var/www/html/var/imap/attachments""" while reading response header from upstream, client: 10.0.0.4, server: _, request: "GET /reports/checkmailnow HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm.sock:", host: "dmarcsrv.example.com", referrer: "https://dmarcsrv.example.com/logs"

Any help?

Thanks,
Pedro Paixão

Installation guide unclear

The "Setup is not pretty easy" statement is pretty exaggerated.
After installing the prerequisites it's unclear what steps need to be taken and what to configure where.
After cloning the repository and pointing the Apache2 configuration to the public folder in the best case one gets a blank page when accessing localhost in the browser. Next, a "Not found" error is returned when accessing "localhost/setup".

Please either provide step by step instructions or a real install script one can use which takes care of installing all dependencies from trusted sources and takes over the configuration.
Other than that it's impossible non-professionals to make use of your good prework.

Installed to web subfolders, Stylesheet not loeaded.

Hello,

Instead of using a new vhost on my apache config. I used the Alias directive.

Initially the icons on the left menu bar were HUGE. And I noticed that style.css file was not loading because it was linked as:
<link href="/style.css" rel="stylesheet">
(As seen by the browser's view source. Note the leading slash)

I don't know if the website is supposed to automatically detect and prepend any sub folder I would be using, but a quick glance at the source file, and it seems to be hard coded as-is.

In the meantime I've edited. the following file on my local setup, removing the leading slash, which temporarily fixes the problem.
./templates/base.html.twig

After editing I had to run the update.sh bash script.

I have some knowledge of PHP, but not enough to debug this in detail. I'm still configuring the tool. Need to setup the mail boxes.

Error on registration - TimeZone issues

When I try to register my first user I receive an 500 error on the web UI after I click the "Register" button (Error see below).
It seems like the mailer client isn't able to send the registration mail via SMTP (I tried both ports for secure SMTP, 465 and 587)
I also tried different passwords with different lengths and strengths.

My docker-compose.yml

name: dmarc-smtptls-reports
version: "2"
services:
  app:
    image: antedebaas/dmarc-reports:latest
    ports:
        - 8087:8080
    environment:
      TZ: Europe/Berlin
      DATABASE_TYPE: mysql
      DATABASE_HOST: db1.prod.<mydomain>
      DATABASE_PORT: 3306
      DATABASE_NAME: dmarcreports
      DATABASE_USER: dmarcreports
      DATABASE_PASSWORD: '<db_password>'
      MAILER_HOST: mail.<mydomain>
      MAILER_SMTPPORT: 587
      MAILER_IMAPPORT: 993
      MAILER_USER: dmarc-reports@<mydomain>
      MAILER_PASSWORD: '<mailer_password>'
    restart: unless-stopped

UI Error

Oops! An Error Occurred
The server returned a "500 Internal Server Error".
Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

Docker Log:

NOTICE: PHP message: [critical] Uncaught PHP Exception Symfony\Component\Mailer\Exception\TransportException: "Failed to authenticate on SMTP server with username "dmarc-reports@<mydomain>" using the following authenticators: "LOGIN", "PLAIN". Authenticator "LOGIN" returned "Expected response code "235" but got code "535", with message "535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6".". Authenticator "PLAIN" returned "Expected response code "235" but got code "535", with message "535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6"."." at /var/www/html/vendor/symfony/mailer/Transport/Smtp/EsmtpTransport.php line 225

2023/11/19 22:45:35 [error] 16#16: *7 FastCGI sent in stderr: "PHP message: [critical] Uncaught PHP Exception Symfony\Component\Mailer\Exception\TransportException: "Failed to authenticate on SMTP server with username "dmarc-reports@<mydomain>" using the following authenticators: "LOGIN", "PLAIN". Authenticator "LOGIN" returned "Expected response code "235" but got code "535", with message "535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6".". Authenticator "PLAIN" returned "Expected response code "235" but got code "535", with message "535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6"."." at /var/www/html/vendor/symfony/mailer/Transport/Smtp/EsmtpTransport.php line 225" while reading response header from upstream, client: 172.26.0.1, server: _, request: "POST /register HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm.sock:", host: "dmarc.<mydomain>", referrer: "https://dmarc.<mydomain>/register"

Fresh Install Error

Good morning,

I just installed the project on a server, checked that all packages were up to date with your recommendations.

I simply pull the project onto the server with GIT, and run update.sh.
Everything is going well except the Doctrine migration:

Voici l'erreur en console :

run migrations
[notice] Migrating up to DoctrineMigrations\Version20240124174018
[notice] CREATE TABLE domains (id INT AUTO_INCREMENT NOT NULL, fqdn VARCHAR(255) NOT NULL, PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8mb4 COLLATE `utf8mb4_unicode_ci` ENGINE = InnoDB
[error] Migration DoctrineMigrations\Version20230809184012 failed during Execution. Error: "An exception occurred while executing a query: SQLSTATE[HY000]: General error: 1 near "DEFAULT": syntax error"
[critical] Error thrown while running command "doctrine:migrations:migrate --no-interaction --query-time". Message: "An exception occurred while executing a query: SQLSTATE[HY000]: General error: 1 near "DEFAULT": syntax error"

In ExceptionConverter.php line 68:

  An exception occurred while executing a query: SQLSTATE[HY000]: General error: 1 near "DEFAULT": syntax error


In Exception.php line 28:

  SQLSTATE[HY000]: General error: 1 near "DEFAULT": syntax error


In Connection.php line 71:

  SQLSTATE[HY000]: General error: 1 near "DEFAULT": syntax error


doctrine:migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CONFIGURATION] [--em EM] [--conn CONN] [--] [<version>]

If I try to run the migration manually, php bin/console doctrine:migrations:migrate, I get the same error

Is there a recommendation for the SQLLite version?

Thank you

Feature Request: possibility to deactivate the registration

Hi Ante,

would it be possible to deactivate the register function through the envirement options? For example if someone only needs one single admin account and don't want to allow further registrations?

Motiv:
Especially if someone runs the parser on a public server (like I do), it could be useful for security to avoid any unwanted registrations.

As a workaround I have disallowed the register link through my reverse proxy, but it would be much easier if we could simply deactivate the registration directly in the docker compose file.

Many thanks.

Best...

Gateway time-out

When I use the Check Mail Now button, I run into Gateway time-out. This is happening after an update.

BUG (cosmetic issue): TLS-RPT with TLSA/DANE results looks strange

Hi Ante,

if you receive a TLS-RPT with DANE/TLSA results, it will look like this in the report overview:

2

The full report looks like this:

1

It simply looks like there is no nice button available for TLSA in the results column and that's maybe why there is just a red dot.
Is this something you could easily fix? Of course not urgent at all. It's only for the perfectionists. ;-)

Thank you lots.

Regards...

Question: how to get correct timezone in the logs?

Hi Ante,

it's just a cosmetic thing, but I like to know how to get the correct timezone in the logs?

I've already placed TZ: Europe/Berlin under environment in the docker-compose.yml, but there is always a 2 hour difference between the real local time and the time in the logs. Is there any way to get the correct information here?

Thank you very much.

Regards

Joe

Feature Request: Sort imap mail by oldest first.

I've saved my DMARC and TLS reports for years as I didn't have software to view the reports. Having found your software I set it up and had it parse the 300+ emails that were saved. The result isn't what I would have expected, as "Latest DMARC/TLS Reports" shows the oldest email first.

While looking around I stumbled on the following code from php's imap_fetch_overview page. I am wondering if you could implement something similar

<?php
$result = imap_fetch_overview($imap_stream, '1:10', 0);

usort($result, function($a, $b) {
  return($b->udate-$a->udate);
});
?> 

Schema validation errors

I've noticed some errors complaining about sts_version when trying to process some emails, so I've checked the schema and found out this:

 [FAIL] The entity-class App\Entity\SMTPTLS_Policies mapping is invalid:
 * The mappings App\Entity\SMTPTLS_Policies#SMTPTLS_MXRecords and App\Entity\SMTPTLS_MXRecords#policy are inconsistent with each other.


 [FAIL] The entity-class App\Entity\SMTPTLS_MXRecords mapping is invalid:
 * The association App\Entity\SMTPTLS_MXRecords#policy refers to the inverse side field App\Entity\SMTPTLS_Policies#mTASTSMXRecords which does not exist.

envelope_to, envelope_from and header_from all empty in DKIM report

I started using this tool a few days ago and have gathered a decent number of reports. When digging into them, I noticed that in the DKIM reports, the Envelope to, Envelope from and Header from fields are all empty.

I checked the database as well and those fields are all set to NULL, so it seems to be an issue with the parsing?

I can dig into this a bit more if needed, but maybe someone knows what is going on here. All the other info (IP, whether or not it passed, etc.) is all there.

Cron schedule not working

I am using v1.10.0.

I want to have a mail check every 6 hours. For that i set in the docker-compose.yaml following:
MAILCHECK_SCHEDULE: "* */6 * * *"

unfortunatly its not working and switchig back to every hour.

Logs:
2024-04-20 15:36:43,534 INFO Set uid to user 0 succeeded

2024-04-20 15:36:43,539 INFO supervisord started with pid 1

2024-04-20 15:36:44,543 INFO spawned: 'containerstartup' with pid 7

2024-04-20 15:36:44,547 INFO spawned: 'cron' with pid 8

2024-04-20 15:36:44,551 INFO spawned: 'nginx' with pid 9

2024-04-20 15:36:44,554 INFO spawned: 'php-fpm' with pid 10

Generate .env.local file

2024-04-20 15:36:44,569 INFO success: containerstartup entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)

Set cron schedule

Match not found, setting to run once every hour

Run migrations

[20-Apr-2024 15:36:44] NOTICE: fpm is running, pid 10

[20-Apr-2024 15:36:44] NOTICE: ready to handle connections

[OK] Already at the latest version ("DoctrineMigrations\Version20240317202800")

2024-04-20 15:36:44,795 INFO exited: containerstartup (exit status 0; expected)

2024-04-20 15:36:45,796 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2024-04-20 15:36:45,797 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2024-04-20 15:36:45,797 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Problems during installation/setup

Thank you for this project, that's exactly what I was looking for.
I'm not the biggest Synphony fan, but it´s ok.
Unfortunately, I have/had problems during the installation. But first things first:

Server version: 10.4.28-MariaDB
Server charset: UTF-8 Unicode (utf8mb4)
PHP version: 8.2.4 (XAMPP, Win10)

.env.local

  1. phpmyadmin creates passwords that contains a slash symbol (/). A slash is not accepted in the constant DATABASE_URL. The password is stored as a plain text string (.env.local)
  2. A similar scenario with passwords for the mail account: like an auto created password e.g. nx#22aZVyHM@U4. This time it's the @ symbol. @ is the separator between password and hostname in the MAILER_DSN constant.

Both cases caused a 500 server error. In the PHP error logfile, only one line was logged at a time:
[Fri Apr 12 14:43:05.629402 2024] [php:notice] [pid 7044:tid 2032] [client 192.168.0.62:61668] [critical] Uncaught Exception: Malformed parameter "url".

Maybe use base64 encrypted strings for password or syntax checking ...

Currently I'm stuck in a loop at the database schema updates:

    DoctrineMigrations\Version20240214121100
    DoctrineMigrations\Version20240302153203
    DoctrineMigrations\Version20240304200130
    DoctrineMigrations\Version20240317202800

SQL statements such as
'ALTER TABLE dmarc_reports_users RENAME COLUMN dmarc_reports_id TO report_id'
will not be executed.

The statement should look like this for my environment:
ALTER TABLE `dmarc_reports_users` CHANGE `dmarc_reports_id` `report_id`;

What would be the minimum MariaDB version?
Do I have to do all entries manually or is there a hack?

Thank you for your effort!

FeatureRequest: Mailbox Folder

Please add a posibility to select inside an Mailbox an imap folder for TLS Reports and one for dmarc Reports.

It should be possible to select separat imap Folders.

On Error, successfull reports are not stored

If you run app::checkmailbox and within the unread mails is one report, that does cause an error, all other reports from that unread mails are getting tagged read, but not processed to the DB.
The process should handle those errors better.

Here's a trace from when it fails:

$ php bin/console app:checkmailbox -vv
[critical] Error thrown while running command "app:checkmailbox -vv". Message: "Call to a member function __toString() on null"
[critical] Uncaught Error: Call to a member function __toString() on null

In CheckmailboxCommand.php line 117:
                                                  
  [Error]                                         
  Call to a member function __toString() on null  
                                                  

Exception trace:
  at /var/www/dmarc-smtptls-reports/src/Command/CheckmailboxCommand.php:117
 App\Command\CheckmailboxCommand->execute() at /var/www/dmarc-smtptls-reports/vendor/symfony/console/Command/Command.php:279
 Symfony\Component\Console\Command\Command->run() at /var/www/dmarc-smtptls-reports/vendor/symfony/console/Application.php:1049
 Symfony\Component\Console\Application->doRunCommand() at /var/www/dmarc-smtptls-reports/vendor/symfony/framework-bundle/Console/Application.php:125
 Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/dmarc-smtptls-reports/vendor/symfony/console/Application.php:318
 Symfony\Component\Console\Application->doRun() at /var/www/dmarc-smtptls-reports/vendor/symfony/framework-bundle/Console/Application.php:79
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/dmarc-smtptls-reports/vendor/symfony/console/Application.php:169
 Symfony\Component\Console\Application->run() at /var/www/dmarc-smtptls-reports/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49
 Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run() at /var/www/dmarc-smtptls-reports/vendor/autoload_runtime.php:29
 require_once() at /var/www/dmarc-smtptls-reports/bin/console:11

app:checkmailbox [--option1] [--] [<arg1>]

First install error

Dear all,
I'm trying a new install, but when I launch the command "bash update.sh", when he comes to command 'run migrations', i get this error:

`[critical] Error thrown while running command "doctrine:migrations:migrate --no-interaction --query-time". Message: "An exception occurred in the driver: SQLSTATE[HY000] [14] unable to open database file"

In ExceptionConverter.php line 76:

An exception occurred in the driver: SQLSTATE[HY000] [14] unable to open database file

In Exception.php line 28:

SQLSTATE[HY000] [14] unable to open database file

In Driver.php line 43:

SQLSTATE[HY000] [14] unable to open database file

doctrine:migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CONFIGURATION] [--em EM] [--conn CONN] [--] []`

In the .env file I see that it refers to a sqllite database, but is it possible to use mysql instead?
And if so, what parameters are needed to update in the .env file?

Thank you for your support!

Best regards

Feature request: Allow for smaller screen sizes

Maybe just a quick finger exercise for someone who is experienced with bootstrap:
It would be wonderful if the layout were a bit more compact, so that nothing wraps in the left column and in the two wide columns of the dashboard each report fit in one line without wrapping.

BUG: wrong counting for SMTP-TLS Reports

Hi Ante,

it looks like the SMTP-TLS Reports are wrong counted.

If I click on "Reports", I see this:

Screenshot 2024-05-19 132005

But in reality I only have received 46 SMTP-TLS Reports:

Screenshot 2024-05-19 132100

I have the impression that every SMTP report is somehow multiplied. Could that be?

Thanks as always.

Best

Jürgen

Feature Request: Manual Import/Upload?

I don't know how easy this would be to implement, but I'm very interested in aggregating and visualizing DMARC reports, but ideally without the tool in question actually talking to my mail server. Would it be possible to dump in a pile of DMARC reports, be they an MBOX or PST of them, a pile of EML email files, or a pile of just the XML report attachments, and have this collect and display them, without accessing the original mailbox?

Database Setup (Migration) not working

Hi together,

i treid to setup the DMARC Reportviewer but i got in trouble during the setup process. I entered my credentials 1000% right into the setup website, but when i click on Update database, nothing happens.
image

Can you please help?

Thanks,
Greets Michael

After registratiin or manual user creation no login possible

After manually creating a user as "[email protected]" with password and flagged verified for a single domain "role" no login with this user ist possible. PHP Logs Syntax error:

NOTICE: PHP message: [critical] Uncaught Exception: [Syntax Error] line 0, col 11: Error: Expected IdentificationVariable | ScalarExpression | AggregateExpression | FunctionDeclaration | "(" Subselect ")" | CaseExpression, got 'FROM'

After that I deleted the user and used "Register" to register a user and verify the user by an admin. Same error.

Setup process requires creation of admin user - NOT Working

Hi everyone,

in the setup process the admin user is asked to create, but this does not work in the current version, you are always redirected to the error page after the creation. But the user isnt in the database also. Can you fix this or how can I access the application logs?

Screenshot:
image

Screenshot 2:
image

Thanks,
Greets Michael

Feature request: amd/arm multiarch support (docker)

Hi guys,

If it should be possible at some point, I would be really excited about support for ARM architecture to run the dmarc-smtptls docker image also on the raspberry pi or ARM Hetzner server.

Many thanks and keep up the good work.

Kind regards from Spain...

BUG: twice or triple mail retrieval

Hi Ante,

I have noticed that for some time now, the mails have been retrieved three times in a row although according to CRON this should only happen once every hour:

Screenshot 2024-03-22 130009

As a result, if a new report is available in the mailbox, it is imported twice or three times. For example today:

import

I have checked the mailbox, but this report came in definitely only once and not repeated.

As always, many thanks for your work and support.

No Reports are

Hi,
I have found you project and it looks nice => I have installed.
The installation was without any issue.
The .env.local file was created, the database too.
I have create user in last step, all ok up to now.
I have placed the the installation directly to mta-sts.domain.ext => the mta-sts.txt file is correct
The extra email box works.

The issue is: the reports are not parsed.
There is always similar entry in log:

Mailbox checked: 0 new emails (0 domains, 0 mx), 0 new dmarc reports (0 records, 0 results), 0 new smtptls reports (0 policies, 0 mxmapping, 0 failure details)

It is not correct, the email are moved from new to cur subfolder of maildir.
There is no error in php log file.
There is no log file in dir project_folder/var/log.
I don't know where I can find next hint, what is wrong.

Thank you
Best regards
Robert

Feature Request: Mark reports as read/unread

Hi Ante,

I would kindly request another feature which could be useful:

Mark (all) reports as read/unread

Motivation:
due to technical problems it happens to me sometimes that I need to delete the database and to re-import all old reports. In this case I need to click every single report to mark them as read. It would be much easier if I could mark them all as read with just a single click as you can do similarly with emails.

As always, thank you very much!

Feature Request: change mail fetch cron job through environment (docker)

Hi @antedebaas,

I'm sorry if I'm bothering you again today with a feature request.

Would it be maybe possible to set the time of the cron job for fetching new mails through the env. of the docker compose file instead of hardcoding it in the docker image? For example something like: CRON_JOB: 0 1 * * *.
That would give more flexibility and everyone could decide at what time and how often the cron job should run.

Alternatively asked: is there a docker compose exec command to run the cron job manually (for example through a script)?

Once again many thanks for your great work.

Cheers..

Question and/or Feature Request: purge logs (automatically or manually)

Hi Ante,

is there any possibility to purge the logs easily?

If not, do you think it would be possible to create an automatism to purge the logs (for example) every 30 days?
Or at least a button which allows purging the logs through the GUI?

Motiv:
I don't really see the requirement to keep these logs for an unlimited period of time. To prevent the database from becoming unnecessarily bloated, I prefer to keep these logs just for a limited time.

As always, thank you very much for the great work.

Best regards....

Feature Request: make reports & domains deleteable in GUI

Hi Ante,

I would like to ask another feature which I guess could be really useful:

Make single reports & domains deleteable in the GUI

Motivation:
Sometimes it happens that a report (which my server sent out) returned undeliverable, because of mailbox full, wrong DMARC DNS record etc. If the returned mail contains still the xml report file, this undeliverable mail will also be imported into the database which confuses the overview, as I only want to see the reports I have received from others and not the ones I have sent myself.

So far, I have always deleted the entire database and re-imported all reports already received. However, it would be much easier if you could simply delete the "wrong" reports in the GUI with a simple click.

As I have no programming experience, I can't say whether this change would be a big effort, but if you have the time and motivation, you could perhaps have a look at it one day.

As always, many thanks for your effort.

Cheers...

Make Dmarc Reporter available as Subsite (e.g. Domain/dmarcreports)

Hi!

I'm currently trying to implement your software in some kind within the mailcow: dockerized stack as it would fit kinda nicely to the project in general.

However i came across the "issue" that the dmarc parser is always redirecting itself to / not a subsite. I can setup a nginx location for the dmarcparser but the redirects done by the dmarc parser will fail.

Maybe i'm just to dumb or something is hardcoded.

The example Location:

  location /dmarcparser/ {
    proxy_pass       http://dmarcparser:8080/;
    proxy_set_header Host      $http_host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_redirect off;
  }

Nice work, keep going :)

Cannot create the attachments directory

I am unable to retrieve mail and this is what I found in logs, can someone please share how I can get this resolved with docker installation?

NOTICE: PHP message: [critical] Error thrown while running command "app:getreportsfrommailbox". Message: "Cannot create the attachments directory "/var/www/html/var/imap/attachments""

[Feature Request] Users, Report and UI

Thank you for the wonderful work you're doing with this project. I would like to make some feature requests.

Users and Login

  • Manually add users as admin
  • User self password reset option on login
  • Welcome email for new users to get verified

Reports

  • Option to sort by: result, date, organization, reporter organization, domain
  • Table sortable
  • Click on the row to view the report instead of the 'Open Report' button alone
  • Add disposition to the result column

Charts and Graphs over a time period

  • View results over time in charts and graphs
  • Pie Chart: SPF Alignment, DKIM Alignment, DMARC Passage
  • Graph: SPF Results (soft fail, pass, none, fail), DKIM Results (pass, fail), SPF and DKIM Alignment (true, false), DMARC Passage (true, false), Message Disposition (reject, quarantine, none)
  • Count of all messages

Nginx config

How can i configure nginx to work with this project?

It seems it relies on .htaccess and nginx has no support for it.

I've tried the default Symfony nginx configuration with no avail. Getting 404 on /setup page.

Any help is appreciated!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.