Comments (18)
I believe williamdes meant using this tool for testing purposes.
from dmarc-srg.
Thank you for looking into it -- if you would like to work together on this, I would be happy to privately provide a remote storage location (via S3) for you, to use for testing, etc. Just let me know!
from dmarc-srg.
Ahhh
For my use case, I wouldn't find it useful because the DMARC reports are sent to a SES email address, where the inbound emails are then automatically processed and stored into S3.
I'll keep it in mind for future reference, however.
from dmarc-srg.
Ahh, my bad, didn't realize that -- that makes a lot more sense
from dmarc-srg.
Hello! You can extract the files using any utility to put them any directory on your server and then run php utils/fetch_reports source=directory
. This method is not suitable for you?
from dmarc-srg.
The reason that this is not suitable is because the retrieval of a remote cloud storage location's contents just to sync onto the local machine requires custom code to be written and setup as a cron job onto the server itself, and it would then require either a separate web interface to allow the users of your system to trigger the fetch and download of the files directly, instead of being able to do so natively within the system. Further, by utilizing FlySystem, you will still support the native 'local directory' file retrieval method, but you will also be adding support for using remote cloud file storage locations in the same manner as a 'local' directory.
Does that make sense, and can I clarify anything further?
Thank you!
from dmarc-srg.
I hear you. I'll check out that project the other day. I'm not sure I have anything to test it on, though.
from dmarc-srg.
Is awesome to have a local S3 working storage in minutes
from dmarc-srg.
Is awesome to have a local S3 working storage in minutes
Wouldn't this solution require more than simply entering S3 IAM access credentials? Or am I not seeing what you are referencing?
from dmarc-srg.
https://min.io/
Is awesome to have a local S3 working storage in minutesWouldn't this solution require more than simply entering S3 IAM access credentials? Or am I not seeing what you are referencing?
This solution is a drop in replacement for AWS S3
I use this for on a client to emulate our production S3 bucket in a free and portable way.
So you can use it just like AWS S3
Credentials and stuff will work the same
from dmarc-srg.
@andrewhenke I've implemented these options for S3: key, secret, token, bucket, path, profile, endpoint, region. I haven't forgotten anything? Which options (names) do you use?
from dmarc-srg.
I should be able to tell you most accurately once I take a look at the implementation, but as of now, it looks like that should be everything that is needed! I'm extremely excited to be able to utilize this functionality.
On that note, is there any ability to be able to trigger the fetch/import of the emails from the storage location from within the UX of the web interface? That is the single biggest 'struggle' my team has with using the system on a enterprise level, because the ingestion of new emails requires a technical team member to access the server, instead of non-technical team members being able to trigger the ingestion of data via the web interface.
Thanks!
from dmarc-srg.
@andrewhenke I have just added implementation for this. Could you please test this commit on your system? See config/conf.sample.php for details.
Note: By default, successfully processed report files are deleted from the file system. Make sure that you use copies.
from dmarc-srg.
Certainly! I will do so in the morning -- I'm very excited to give it a try!
from dmarc-srg.
@liuch I wanted to double check with you -- does the code that you released support extracting the file attachment from the full email itself automatically, or do I need to write and utilize a AWS Lambda function that separates the actual DMARC report attachment from the email itself? I would prefer to not need to use Lambda, if this is something that the codebase will support, or already does support. Please let me know if you have any additional questions or need me to clarify further.
from dmarc-srg.
I guess I didn't read your first post carefully. I thought the bucket contained report files only (gz, zip, xml), i.e. attachments, not mail messages. My code doesn't work with messages saved as a file.
Could you tell me what is the format of the messages saved to the bucket? Is it *.eml or something else? Maybe I can add processing for such files.
from dmarc-srg.
No worries @liuch! By default, AWS stores the complete, raw email in the MIME format, which you can reference here in the AWS documentation, as well as RFC 2045. There are numerous MIME PHP processing libraries that are out there, such as php-mime-mail-parser which I found rather quickly through doing a few searches.
Does this help?
from dmarc-srg.
Thank you for the information. I won't promise to add such an implementation anytime soon. But I will definitely consider this possibility.
from dmarc-srg.
Related Issues (20)
- Error: Call to undefined function Liuch\DmarcSrg\getallheaders() HOT 10
- NP tag causes error to be logged HOT 2
- Feature Request: Button to manually retreive email HOT 6
- Multiple Mailboxes? HOT 2
- utils/fetch_reports.php does not list mailbox messages HOT 3
- Cant login HOT 20
- Error - Bad Message Number HOT 11
- JSON.parse: unexpected non-whitespace HOT 2
- Feature Request -- Export Summary Reports to CSV HOT 4
- admin login setting to assign the domains to user login HOT 1
- Forensic reports? HOT 4
- Ip address I wrong in reports. HOT 4
- Check on write access of config is incorrect HOT 1
- Fetch_report : failed to open stream HOT 9
- error message with PHP 7.4 HOT 1
- Feature request - Graphs in reports (duplicate #79) HOT 4
- Need install help HOT 18
- Feature Request: csv export HOT 4
- XML parsing fails for google reports HOT 13
- Bug: Menu Broken HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dmarc-srg.