• README-JA(日本語)
• What is Sisimai
  • Key features
  • Command line demo
• Setting Up Sisimai
  • System requirements
  • Install
    • From RubyGems
    • From GitHub
• Usage
  • Basic usage
  • Convert to JSON
  • Read bounce object
  • Callback feature
  • One-Liner
  • Output example
• Sisimai Specification
  • Differences between Ruby version and Perl version
  • Other specification of Sisimai
• Contributing
  • Bug report
  • Emails could not be parsed
• Other Information
  • Related sites
  • See also
• Author
• Copyright
• License

Sisimai is a Ruby library for analyzing RFC5322 bounce emails and generating structured data from parsed results. The Ruby version of Sisimai is ported from the Perl version of Sisimai at github.com/sisimai/p5-Sisimai.

• Convert Bounce Mails to Structured Data
  • Supported formats are Ruby(Hash, Array) and JSON(String)
• Easy to Install, Use.
  • gem install
  • git clone & make
• High Precision of Analysis
  • 2 times higher than bounceHammer
  • Support 23 known MTAs and 5 unknown MTAs
  • Support 22 major MSPs(Mail Service Providers)
  • Support 2 major Cloud Email Delivery Services(JSON format)
  • Support Feedback Loop Message(ARF)
  • Can detect 29 error reasons

The following screen shows a demonstration of Sisimai at the command line using Ruby(rb-Sisimai) and Perl(p5-Sisimai) version of Sisimai.

More details about system requirements are available at Sisimai | Getting Started page.

• Ruby 2.1.0 or later
  • Oj | The fastest JSON parser and object serializer
• Also works on JRuby 9.0.4.0 or later
  • JrJackson | A mostly native JRuby wrapper for the java jackson json processor jar

$ sudo gem install sisimai
Fetching: sisimai-4.22.2.gem (100%)
Successfully installed sisimai-4.22.2
Parsing documentation for sisimai-4.22.2
Installing ri documentation for sisimai-4.22.2
Done installing documentation for sisimai after 6 seconds
1 gem installed

$ cd /usr/local/src
$ git clone https://github.com/sisimai/rb-Sisimai.git
$ cd ./rb-Sisimai
$ sudo make depend install-from-local
gem install bundle rake rspec coveralls
...
4 gems installed
bundle exec rake install
sisimai 4.22.2 built to pkg/sisimai-4.22.2.gem.
sisimai (4.22.2) installed.

make() method provides feature for getting parsed data from bounced email messages like following.

#! /usr/bin/env ruby
require 'sisimai'
v = Sisimai.make('/path/to/mbox')       # or path to Maildir/

# Beginning with v4.22.7p2, both make() and dump() method of Sisimai class can
# read bounce messages from variable instead of a path to mailbox
f = File.open('/path/to/mbox', 'r');    # or path to Maildir/
v = Sisimai.make(f.read)

# If you want to get bounce records which reason is "delivered", set "delivered"
# option to make() method like the following:
v = Sisimai.make('/path/to/mbox', delivered: true)

if v.is_a? Array
  v.each do |e|
    puts e.class                # Sisimai::Data
    puts e.recipient.class      # Sisimai::Address
    puts e.timestamp.class      # Sisimai::Time

    puts e.addresser.address    # [email protected] # From
    puts e.recipient.address    # [email protected]   # To
    puts e.recipient.host       # example.jp
    puts e.deliverystatus       # 5.1.1
    puts e.replycode            # 550
    puts e.reason               # userunknown

    h = e.damn                  # Convert to HASH
    j = e.dump('json')          # Convert to JSON string
    puts e.dump('json')         # JSON formatted bounce data
  end
end

Sisimai.dump() method provides feature for getting parsed data as JSON string from bounced email messages like following.

# Get JSON string from parsed mailbox or Maildir/
puts Sisimai.dump('/path/to/mbox')  # or path to Maildir/

# dump() method also accepts "delivered" option like the following code:
puts Sisimai.dump('/path/to/mbox', delivered: true)

The way to read a bounce object retrived from Cloud Email Services as JSON using their API is the following code. This feature is available at Sisimai 4.20.0 or later.

#! /usr/bin/env ruby
require 'json'
require 'sisimai'

j = JSON.load('{"notificationType"=>"Bounce", "bounce"=>{"...') # JSON String
v = Sisimai.make(j, input: 'json')

if v.is_a? Array
  v.each do |e|
    ...
  end
end

As of present, Only Amazon SES and SendGrid are supported.

Beginning with Sisimai 4.19.0, make() and dump() methods of Sisimai accept a Lambda (Proc object) in hook argument for setting a callback method and getting the results generated by the method via Sisimai::Data.catch method.

#! /usr/bin/env ruby
require 'sisimai'
callbackto = lambda do |v|
  r = { 'x-mailer' => '', 'queue-id' => '' }

  if cv = v['message'].match(/^X-Postfix-Queue-ID:\s*(.+)$/)
    r['queue-id'] = cv[1]
  end
  r['x-mailer'] = v['headers']['x-mailer'] || ''
  return r
end

list = ['X-Mailer']
data = Sisimai.make('/path/to/mbox', hook: callbackto, field: list)
json = Sisimai.dump('/path/to/mbox', hook: callbackto, field: list)

puts data[0].catch['x-mailer']      # Apple Mail (2.1283)

More information about the callback feature is available at Sisimai | How To Parse - Callback Page.

% ruby -rsisimai -e 'puts Sisimai.dump($*.shift)' /path/to/mbox

[{"action": "failed", "subject": "Nyaan", "catch": null, "token": "08acf78323edc7923a783c04749dd547ab45c433", "alias": "", "messageid": "[email protected]", "listid": "", "smtpcommand": "MAIL", "smtpagent": "Email::Sendmail", "lhost": "localhost", "timezoneoffset": "+0900", "feedbacktype": "", "senderdomain": "neko.example.jp", "diagnostictype": "SMTP", "softbounce": 1, "deliverystatus": "5.0.0", "addresser": "[email protected]", "diagnosticcode": "550 Unauthenticated senders not allowed", "timestamp": 1528523769, "destination": "example.com", "recipient": "[email protected]", "reason": "securityerror", "replycode": "550", "rhost": "neko.example.jp"}]

The following table show the differences between Ruby version of Sisimai and Perl version of Sisimai. Information about differences between Sisimai and bounceHammer are available at Sisimai | Differences page.

1. See ./ANALYTICAL-PRECISION
2. Xeon E5-2640 2.5GHz x 2 cores | 5000 bogomips | 1GB RAM | Ruby 2.3.4

• Parser Engines
• Bounce Reason List
• Data Structure of Sisimai::Data

Please use the issue tracker to report any bugs.

Bounce mails which could not be parsed by Sisimai are saved in the repository set-of-emails/to-be-debugged-because/sisimai-cannot-parse-yet. If you have found any bounce email cannot be parsed using Sisimai, please add the email into the directory and send Pull-Request to this repository.

• @libsisimai | Sisimai on Twitter (@libsisimai)
• libSISIMAI.ORG | Sisimai | The successor to bounceHammer, Library to parse bounce mails
• Sisimai Blog | blog.libsisimai.org
• Facebook Page | facebook.com/libsisimai
• GitHub | github.com/sisimai/rb-Sisimai
• RubyGems.org | rubygems.org/gems/sisimai
• Perl verson | Perl version of Sisimai
• Fixtures | set-of-emails - Sample emails for "make test"

• README-JA.md - README.md in Japanese(日本語)
• RFC3463 - Enhanced Mail System Status Codes
• RFC3464 - An Extensible Message Format for Delivery Status Notifications
• RFC3834 - Recommendations for Automatic Responses to Electronic Mail
• RFC5321 - Simple Mail Transfer Protocol
• RFC5322 - Internet Message Format

@azumakuniyuki

Copyright (C) 2015-2018 azumakuniyuki, All Rights Reserved.

This software is distributed under The BSD 2-Clause License.