Hello! I have noticed that the process of parsing and importing the emails from the IMAP connection fails if the attachment file contains a directory within the file archive.

Is it within the development roadmap to implement functionality to process both unnested and nested report formats -- it's something that I don't have control over, of course, and I'd like to be able to include the report contents in the system.

Thank you!

Your script created a folder called "failed" and it's full of Google reports. I'm assuming these are reports that couldn't be parsed. Not all Google reports are moved to the failed folder, but many are. Is there anything that can be done about this. Thanks!

I can't get dmarc-srg to retrieve the reports from imap.
The connection is correct on the panel, but when I run the fetch_reports I get this:

/php utils/fetch_reports.php PHP Fatal error: Uncaught Error: Call to undefined function Liuch\DmarcSrg\Mail\imap_utf8_to_mutf7() in /public_html/dmarc/classes/Mail/MailBox.php:230 Stack trace: #0 /public_html/dmarc/classes/Mail/MailBox.php(171): Liuch\DmarcSrg\Mail\MailBox->ensureConnection() #1 /public_html/dmarc/classes/Sources/MailboxSource.php(88): Liuch\DmarcSrg\Mail\MailBox->sort('SORTDATE', 'UNSEEN', false) #2 /public_html/dmarc/classes/Report/ReportFetcher.php(68): Liuch\DmarcSrg\Sources\MailboxSource->rewind() #3 /public_html/dmarc/utils/fetch_reports.php(117): Liuch\DmarcSrg\Report\ReportFetcher->fetch() #4 {main} thrown in /public_html/dmarc/classes/Mail/MailBox.php on line 230

Seems I need to have imap_utf8_to_mutf7 on the system, but this is only provided if libcclient exports utf8_to_mutf7 (explained here: https://www.php.net/manual/en/function.imap-utf8-to-mutf7.php)

The domain list ordering isn't working.
When we click on buttons, nothing happens.

If you want a ssl connection to an imap server the host entry in config.php of a mailbox
should be appended with :993

host = imap.yourdoain.com:993

But that is not sufficient, in classes/Mail/Mailbox.php the server string is created to be used in the imap_open call,
in case of a ssl connection it should end with '/ssl'
A simple fix:
$flags = '';
if (isset($params['novalidate-cert']) && $params['novalidate-cert'] === true) {
$flags = '/ssl/novalidate-cert';
}
else {
$flags = '/ssl';
}
$this->server = sprintf('{%s/imap%s}', $this->host, $flags);

Hello,

first of all, thank you for this very useful tool. Installed version 1.2 yesterday on Debian Buster with PHP 8.0.13 and it worked out of the box.

Because I don't really want to open every report in the GUI, I would like to have a button, which is marking all these reports as read.

Thanks anyway!

Hello! I have noticed that when filtering reports to view, the global number of emails and the percentage of DMARC compliant emails, partially compliant emails, etc, does not update when only viewing a specific domain's records. Would it be possible to update this tool so that when filtering to view DMARC reports for just a single domain name, the global statistics are updated to reflect the compliance and percent of compliance out of just the individual domain's reports?

Thank you!

dmarc-srg [error]: ErrorException: Creation of dynamic property Liuch\DmarcSrg\Report\Report::$db is deprecated in /var/www/html/classes/Report/Report.php:36

Solution declare "private db;" in class

dmarc-srg [error]: ErrorException: Using ${var} in strings is deprecated, use {$var} instead in /var/www/html/classes/Database/Mariadb/ReportMapper.php:598

Solution place dollar sign $ inside the curly braces

As discussed, I think it would be great to have the time setting set local and not server side, this would help with reading reports and a better 'out of the box' experience with the average user.

You mentioned using browser local which I think is a fantastic idea!

New user here. I've installed and configured dmarc-srg and trying to populate the database from an mbox with a history of DMARC emails and attachments. I've initialized the database, but when I run fetch_reports.php, there is an error about SORTDATE not being defined.

$ php utils/fetch_reports.php source=email
PHP Fatal error:  Uncaught Error: Undefined constant "Liuch\DmarcSrg\Sources\SORTDATE" in /var/www/orion.mydomain.com-443/html/dmarcreports/dmarc-srg/classes/Sources/MailboxSource.php:91
Stack trace:
#0 /var/www/orion.mydomain.com-443/html/dmarcreports/dmarc-srg/classes/Report/ReportFetcher.php(68): Liuch\DmarcSrg\Sources\MailboxSource->rewind()
#1 /var/www/orion.mydomain.com-443/html/dmarcreports/dmarc-srg/utils/fetch_reports.php(139): Liuch\DmarcSrg\Report\ReportFetcher->fetch()
#2 {main}
  thrown in /var/www/orion.mydomain.com-443/html/dmarcreports/dmarc-srg/classes/Sources/MailboxSource.php on line 91

What am I missing?

Hi, Thanks for the nice work on such a useful interface!

I managed to import some reports from mail and now I'm looking for a command (syntax) for cleaning up the current state.

But I can't find the right command or syntax.

Thanks for some suggestions and feedback.

/G

When I run utils/fetch_reports.php from the terminal, I get...
TLS/SSL failure for mail.myserver.com: SSL negotiation failed

Right now its only possible to filter the report list by status of SPF + DKIM, but not entries inside of the report itself.

Additional it would help if there is some kind of "show only failed validation" checkbox to filter all reports + listings to only show invalid entries.

Not trying to bug you... (..crickets...)

If you run the setup with the default admin settings in config.php

$admin = [
    // Set this value to null or remove this parameter to disable authentication
    'password' => '',
];

and then logout of the app, you get locked out

This would be a nice to have, as MySQL is way overkill for a database that only sees a few clients and runs batch jobs.
This would limit

• dependencies requirements
• attack surface
• ressources uses
• maintenance requirements

edit: I'll try a port

Is it possible to support .json files?

Best Regards

I try to upload report via .zip file with new domain name, but how can I set it to auto create a new domain name without I need to create it first? because when I setting this dmarc system, I will not open for all , just me, but if I need to create new domain one by one, and it not user friendly.

thanks

Hi,

I have some domains , and i like to get whole report in one email only.

Maybe add some solution to allow : all domains or a lits of domains ?

php utils/summary_report.php period=lastmonth
Chcek all know domains

php utils/summary_report.php domain=example.com;example.org period=lastmonth
For a lits of domains

I can create the pull requets (when i have time)

Hello,

I am trying to setup the tool. I don't plan to connect it to my mailbox. Instead, I want to upload DMARC reports manually or point it to a directory where my reports are saved. I keep getting this error message every time I refresh the page or try to do anything in the GUI.
[Sun Mar 26 15:46:04 2023] dmarc-srg [error]: Error: Call to undefined function Liuch\DmarcSrg\Mail\imap_errors() in C:\Users\**********\dmarc-srg\classes\Mail\MailBox.php:307

Is there something I am missing. Like I mentioned, I don't plan to use the email integration portion of the tool.

Thank you in advance,

Hey brother... first of all, great job! Been looking for a solid DMARC parser for use with with a LAMP stack for a while. Good stuff!

I'm using the email option as the source for our reports. The utilities are running fine via terminal but before setting them up as crons I wanted to see if there is a recommendation for the settings? i.e. fetching reports 1 per day, mailbox cleaner (1 per day?), DB admin, reportlog/reports cleaner?

Is it possible to send the summary report via email the same as the HTML report available in the dashboard?

I have had a read through the code and can't see a way to achieve this.

Just spotted this mail from my cron. The connection error itself was a problem by my mailserver, but that Fatal error seems to be some kind of bug.

Can't connect to mx.mydomain.de,993: No route to host
PHP Fatal error:  Uncaught TypeError: imap_close(): Argument #1 ($imap) must be of type IMAP\Connection, bool given in /var/customers/webs/mydomain/dmarc-view.mydomain.de/classes/Mail/MailBox.php:87
Stack trace:
#0 /var/customers/webs/mydomain/dmarc-view.mydomain.de/classes/Mail/MailBox.php(87): imap_close()
#1 [internal function]: Liuch\DmarcSrg\Mail\MailBox->__destruct()
#2 {main}
  thrown in /var/customers/webs/mydomain/dmarc-view.mydomain.de/classes/Mail/MailBox.php on line 87

Will check this tomorrow.

Running into this issue now.

I saw the closed cases and it looks like this issue happens with many components of the scripts but none of them mention it happening with DirectorySource::$path. Any thoughts? Would really love to get this tool up and running.

Thank you,

dmarc-srg was working ok with php7.3. For other reasons, I've had to upgrade to php7.4 and now I get a load of deprecated messages (which i can live with), but finally "SQLSTATE[HY000] [2002] No such file or directory"
Steps to debug that would be appreciated. Thanks.

PHP Deprecated: Return type of Liuch\DmarcSrg\Sources\Source::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/Sources/Source.php on line 60

Deprecated: Return type of Liuch\DmarcSrg\Sources\Source::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/Sources/Source.php on line 60
PHP Deprecated: Return type of Liuch\DmarcSrg\Sources\Source::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/Sources/Source.php on line 61

Deprecated: Return type of Liuch\DmarcSrg\Sources\Source::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/Sources/Source.php on line 61
PHP Deprecated: Return type of Liuch\DmarcSrg\ReportFile\ReportGZFileCutFilter::filter($in, $out, &$consumed, $closing) should either be compatible with php_user_filter::filter($in, $out, &$consumed, bool $closing): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/ReportFile/ReportGZFileCutFilter.php on line 33

Deprecated: Return type of Liuch\DmarcSrg\ReportFile\ReportGZFileCutFilter::filter($in, $out, &$consumed, $closing) should either be compatible with php_user_filter::filter($in, $out, &$consumed, bool $closing): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /usr/share/dmarc-srg-master/classes/ReportFile/ReportGZFileCutFilter.php on line 33
SQLSTATE[HY000] [2002] No such file or directory

Hi,

First, I think you are running the only php based DMARC project that I've come across that is solid.

Second, although the web interface admin panel says it connects to the mailbox for dmarc reports and lists the number of unseen reports in green, the reports are not loaded automatically into the database. I have set up a CRON-job to run the scripts listed in the readme file like so:

cd /var/www/dir/htdocs/dmarc && /usr/bin/php utils/fetch_reports.php

I have enabled debug messages, and the mailbox cleaner script results in the error No such host as host.domain.de (-1)"
Previously, when I set the mail server to localhost, I received the message Certificate failure for localhost: Hostname mismatch: /CN=host.domain.de (-1)"

Can you help getting the reports imported automatically?

Kind regards,

Mark

Looks like a DB connection problem.

I edited the config/conf.php file, and double checked on the command line that the DB user has full access to the database, and I configered a dmarc_ prefix, so there shouldn't be conflicts. The directory is also readable/writeable by the scripts. I hardly have experience with PDO, otherwise I'd dig deeper myself before asking here.

Using PHP 7.4.33 on Linux, '--with-pdo-mysql=shared,mysqlnd', Server version: 5.5.65-MariaDB MariaDB Server.

Looks like a brilliant tool, if I can get it to work :) Ideas anyone?

Cheers,
Jeroen

Very interesting bug.

When you click a report and bring up the DMARC Report, if you are on a slower connection (or accessing site via external webserver) and click it twice instead of once it will bring up the report and then the authentication dialog. you can click cancel and it will go away.

Can be easily re-produced by throttling the connection in dev mode on chrome/edge etc. and following above.

Feature request:

As title says: Please add RUF Support!
I love your tool!

As always, I'll start with praise for Aleksey and this project — excellent!

I suggest that filtering processed messages into subfolders would/should be an easy enhancement. I currently do this manually.

There seem to be 3 basic results that could be filtered:

done - No errors processing the message
failed - An error was encountered processing the message [This is currently implemented.]
other - The message does not appear to be a DMARC message

Thanks,

Will Mayall

Hi,

I use dmarc-srg on same server than email server (iredmail server)

Externaly : i can use starttls but not via localhost,
I set 'encryption' => 'none', for config.

When i try to get email :

$ php utils/fetch_reports.php 
PHP Notice:  Unknown: SECURITY PROBLEM: insecure server advertised AUTH=PLAIN (errflg=1) in Unknown on line 0

Then i try with

'encryption'      => 'starttls',
'novalidate-cert' => true,

$ php utils/fetch_reports.php 
TLS/SSL failure for localhost: SSL negotiation failed

To allow reading of email : i update debug.php to set ini_set('error_reporting', E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED); and now it work.

Would be a nice feature if you can add options to import the domains from the control panels using their API.
Almost all control panels allow to curl the a domain list using their API.
We won't need to type in hundreds or thousands of domains on the control panel or do an export to add these domains to the conf.php. It would simply import them automatically using the cron.

What do you think?

Filter feature isn't working on the Report List. When I click on it, the modal pops up, but the options are grayed out and it says "Error!".

Firefox console says "Failed to get a list of domains" on list.js:486:12

I checked and line 486 is about the error message, so that's not helpful.

I also tried it on Chrome and had the same problem with the same warning message.

Some records are encoded as 'application/x-gzip' (Amazon Route 53), but they aren't properly opened and processed.

Adding 'application/x-gzip' to the $mime_type array in classes/Mail/MailMessage.php:82 fixes this error when running utils/fetch_reports.php source=email:

Failed to get incoming report:
Error message:
- Failed to get the list of mailboxes
- Incorrect message: Attachment file type is not valid (application/x-gzip)
Email message metadata:
- From: XXXXXXXXXX [email protected]
- Date: Tue, 25 Oct 2022 13:21:44 -0400

but then it fails in 'classes/ReportFile/ReportFile.php':

Failed to get incoming report:
Error message:
- gzopen(): Filename cannot be empty

I'm unsure where to look next in order to fix this issue.

Thank you for this tool!

The reports are no longer listed but instead show: null is not an object (evaluating 'd1.toISOString')

As an aside, thank you so much for creating this tool! It is really fantastic.

Ironically, the reports emailed by summary_report.php are getting flagged as non-aligned for me because they're using the built-in PHP mailer which goes out through the hosting server, which I don't have control over on shared hosting, and therefore I can't configure it to sign using my DKIM key. :-) This came up as I was trying to chase down legitimate sources of non-aligned mail.

This could be avoided if it knew how to use SMTP (via the same account it's using for IMAP in my case)

I'm happy to submit a PR for this, but I have questions first to make sure I don't submit something you're not going to like. :-)

It seems like the accepted way to send mail via SMTP from PHP is to use PHPMailer (or another mailer library - I'm open if you have a preference). While you can just drop a copy of PHPMailer into your source, the easiest way to get it is to use composer to install it. Using composer actually gets you security benefits on GitHub: if PHPMailer ever has a security issue, GitHub will automatically file a ticket (and sometimes even a PR) against your project to update the dependency to the fixed version.

So with that said, are you willing for me to add a composer dependency to your project?

Here's what the patch will entail if I do it this way:

1. Add a .gitignore file that will ignore /vendor because that's where composer will download to, and you don't want the dependencies included in your source control. (Should probably add /config/conf.php to this as well so someone doesn't accidentally commit their production config file)
2. Add composer.json and composer.lock files to the project. composer.json will describe what the desired dependencies are (i.e. PHPMailer > some version). composer.lock will contain specific references to the version(s) last tested with so that chasing bugs is easier (everyone gets the same version).
3. End users who check out from source would need to run composer install to pull in PHPMailer.
4. A GitHub Action can be put in place to generate a zip file or tarball or both on your releases which already contains the results of composer install so the end user doesn't have to do it. I can supply that as part of the PR if you want.
5. config options for SMTP would be added to the sample config file
6. summary_report.php will check a) whether those config options are present, and b) whether PHPMailer is installed (in case they installed from source and didn't do composer install), and fall back on the existing mail() code if either check fails. This would ensure an update won't break people in case someone is auto-updating from git and not reading the changelog.

Thoughts?

Hi, Can you tell me how the imap will take automatically reports from the " Mailbox " and post into database,
i have to run some Cronjobs or what is the procedure to run " util/fetch_reports.php "

After upgrade to 1.7 from 1.4, i get this error in the list view.

Cannot read properties of null (reading 'toISOString')

I have tryed to make a frech install, in a new folder and different DB prefix.
same error message, after repport import.

dmarc.w-app.dk-1669117385866.log
dmarc.w-app.dk.har.txt

Hey, great project this app has so much potential!

I have spent the last few days looking under the hood, some very complex stuff, I am by no means a PHP wizard so there is plenty I don't understand...

I created a DarkTheme anyway, mainly for my poor eyes as looking at this on a white screen at night can certainly sting.

Let me know what you think!

https://github.com/kramwell/dmarc-srg_darktheme

Hello,

Now, it works, I would like create a cron job sending a report for each domain, such as:

domains="$(curl https://dmarc.dom.tld/domains.php -u pswd | gawk 'to be defined')" # to capture domains name
for i in domains
do
php utils/summary_report.php domain=$i period=lastmonth
done

My issue is that curl is not working with -u option...
Is there a way to capture domains with php domains.php
This command gives : PHP Fatal error: Uncaught Error: Call to undefined function Liuch\DmarcSrg\getallheaders()

A button in the web app to manually check for new reports.

Save going in to the backend to run via manual cron or SSH.

Somewhere easy to access (maybe on the side menu?)

I would like to use starttls connection to maildir
the error message:
imap-login: Disconnected: Connection closed: SSL_accept() failed: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, TLS handshaking: SSL_accept() failed: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol, session=<GqC79zvvFpp/AAAB>
Where I should start to correct the issue?
Thank you

When subdomains have different policies, or at least generate their own reports. Things stop working.
Any (easy) fix for this? Can I manually add the subdomains in the DB?

    try {
        $fqdn = $this->data['domain'];
        $domain = new Domain($fqdn);
        if (!$domain->exists()) {
            // The domain is not found.
            // It will automatically added a new domain if there are no domains in the table
            // or will throw an error otherwise.
            if (DomainList::count() !== 0) {
                throw new Exception('Failed to add an incoming report: unknown domain', -1);
            }

Is what errors out.

Thank you for sharing this much needed software!

I'm getting an error message when I try to use the web interface:

DB Error 1045: Access denied for user 'root'@'localhost' (using password: YES)

I have followed the instructions and added the correct creds to the conf.php file.

thanks

Thought about implementing TLS Reports? Same principle than DMARC Reports just another parser

Hello, our dmarc reports go to an office365 account and I have been made aware that on October 1, 2022 Microsoft is deprecating the basic auth access method:

https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-may-2022/ba-p/3301866

It seems that the only ways to access Outlook365 mail via a script will be using either OAUTH or the Graph API. I am aware of a project here on github which implements OAUTH for php-imap:

https://github.com/javanile/php-imap2

I am unsure if the standard php-imap library has implemented this functionality. "javanile" has claimed in comments responding to this feature request for other projects that his library is a drop-in replacement for the standard php-imap and merely requires changing calls to php2-imap. I hope this is true and would allow quick implementation of this request before Microsoft breaks things.

Thanks for your consideration and for this useful software.

Hi,

After update (via git pull) I'm gotting the error "d1 is null" in Reports list. It seems to fetch mails OK.

Thanks.

root@h2983452:~# cd /var/www/html && /usr/bin/php utils/fetch_reports.php
Failed to get incoming report:
Error message:
- Creation of dynamic property Liuch\DmarcSrg\Mail\MailAttachment::$encoding is deprecated

Firstly, great software. 👍

I was just wondering is it possible to send the report per domain to different email address.
I was thinking maybe as the current report is triggered:
php utils/summary_report.php domain=example.com period=lastndays:10
could this be adapted to
php utils/summary_report.php domain=example.com period=lastndays:10 [email protected]

on Justhost server the smtp reading reports from email not working.

Hi,

I'm very excited to find this DMARC parsing tool. I just need to get it working :-)

The web interface is working, the database is initialised and the web interface reports the IMAP server is accessible.

Now I'm trying to run php utils/fetch_reports.php from the command line.

Firstly I'm getting a PHP error in utils/MailBoxes.php:
Parse error: syntax error, unexpected 'list' (T_LIST), expecting identifier (T_STRING) in /volume1/web/dmarc-srg/classes/Mail/MailBoxes.php on line 53

It turns out "list" is a reserved keyword in PHP and cannot be used as a function name. So I renamed this private function to mList and this has improved things (although I've not found if/where "list" it is called yet!)

But then with that "fix" I get another PHP error:
Parse error: syntax error, unexpected '?' in /volume1/web/dmarc-srg/classes/Mail/MailBox.php on line 61

I'm wondering why I'm seeing PHP errors, when no one else has reported anything. Do you have any suggestions?
I'm using PHP Version 7.4.9