Git Product home page Git Product logo

mintapi's Introduction

mintapi

Build Status PyPI Version Code style: black

An unofficial screen-scraping API for Mint.com.

mintapi scrapes Mint.com by using Selenium/WebDriver to navigate a browser. Once logged in, mintapi allows programatic, automated access to Mint's UI.

IMPORTANT: mintapi 2.0 vs 1.x and breaking changes

We recently released 2.0, which supports (and only supports) the new Mint UI:

  • If your account has the new UI with the nav on the left, you'll need to install at least 2.0: pip install mintapi>=2.0
  • If your account still has the original UI with the nav on top, to use 2.0, you will need to specify --beta in your command-line options or submit beta=True when initializing the class. Otherwise, please install the latest 1.x release: pip install mintapi<2.0

Please note that due to data changes on the Mint.com side as well as various new features and changes on the mintapi side, there are several breaking changes in 2.0. Please see the CHANGELOG for details.

Installation

Install with pip from PyPi

pip install mintapi

Or for the latest release:

pip install git+https://github.com/mintapi/mintapi

Then simply:

mintapi --keyring --headless [email protected]

You will be prompted for your password, which will be stored securely in your system keyring, and use a headless (invisible) browser to log in and grab the account data. If this triggers an MFA prompt, you'll be prompted for the one-time code on the command line. MFA prompts default to SMS unless you specify --mfa-method=email. mintapi persists the browser session in $HOME/.mintapi/session to avoid an MFA in the future, unless you specify --session-path=None.

To simplify CLI invocation, you can specify a configuration file with the --config-file argument. For arguments such as --transactions, you can add a line in your config file that says transactions. For other arguments that have input, such as --start-date, you would add a line such as start-date=10/01/21. Reading email and password from config files is not supported. You must pass them as arguments directly or through a keyring.

For help, or just to chat with fellow mintapi users, please join us on Discord! :)

Complete Setup

MFA Authentication Methods

Mint accepts MFA in one of four ways: text, voice, email, or TOTP (Time-based One-Time Password). mintapi supports all of these options except Voice.

While you may disable MFA altogether, doing so is not recommended. Not only will it decrease your account security, but Mint will sometimes still email you a second factor code. So, for the least fragility, enable MFA.

As of v2.0, the mfa_method parameter is only required when using soft-token, or if your login flow presents you with the option to select which Multifactor Authentication Method you wish to use. Prior to v2.0, mfa_method is always required.

Option 1: TOTP

Set mfa_method to soft-token.

Obtain the seed for mfa_token as follows:

  1. Navigate to the Intuit MFA settings (or in your Mint setings page, navigate Intuit Account -> Sign In & Security -> Two-step verification).
  2. Token MFA is only allowed as a secondary option, so first enable one of the other options (text, email, voice).
  3. Now you're ready to setup token MFA. Select the Authenticator app option, and navigate through the prompts. If you already have TOTP enabled on your account, you will first have to disable and delete the old TOTP token before setting up a new one.
  4. When you get to the part where you see the QR code, copy the manual setup code that appears next to it. This is the token you pass to mfa_token in either the python api or from the command line. BE CAREFUL WHERE YOU STORE THIS, as anyone with it will be able to take over your Mint account.
  5. Complete the setup process by providing a generated code from your authenticator app when requested.

Option 2: Email

In order for mintapi to automate the retrieval of the MFA code from your email, your email provider must provide IMAP access. If you use IMAP in conjunction with keyring, then you can store your IMAP password (imap-password) in keyring. To do so, simply omit imap-password and you will initially be prompted for the password associated with your IMAP account. Then, on subsequent uses of your IMAP account, you will not have to specify your password.

Chrome

mintapi automatically downloads the latest stable chromedriver. For long term, automated deployments, verify that the particular chrome and chromedrive binaries you have downloaded work together, and use the --use_chromedriver_on_path flag to prevent mintapi from auto updating the chromedriver binary Keep these binaries separate from your regular installation to avoid accidental breakage via auto-update.

Use your distribution's package manager to install chromium and chromedriver:

# Debian/Ubuntu
sudo apt install chromium-browser chromium-chromedriver
# RHL/Fedora
sudo dnf install chromium-browser chromium-chromedriver
# Arch/Manjaro
sudo pacman -S chromium-browser chromium-chromedriver

You can also manually download a chromedriver version of your choice. To use your custom chromedriver in mintapi, you can either add it to your Python working directory or add it to your PATH as described in the Seleninum driver documentation.

(To use a browser other than Chrome/Chromium, see the python section below.)

Examples

Cron

The following cron job runs mintapi, looking for the chromium executable in /usr/bin, every day at 07:00:

0 7 * * * PATH=/usr/bin:$PATH mintapi --use_chromedriver_on_path --headless [email protected] my_password

Note that the PATH is only affected for this job, and will not change the environment for any other process.

This page has instructions for setting a scheduled task in Windows with Powershell. Running just once is as easy as:

$ENV:PATH = "C:\Program Files\Google\Chrome;$ENV:PATH"
mintapi --headless john@example.com my_password

Docker Image

You can also use the docker image to help manage your environment so you don't have to worry about chrome or chromedriver versions. There are a few caveats:

  1. Headless mode is recommended. GUI works but introduces the need to configure an X11 server which varies with setup. Google is your friend.
  2. Almost always use the flag --use-chromedriver-on-path as the chrome and chromedriver built into the docker image already match and getting the latest will break the image.
  3. If you want to persist credentials or your chrome session, you'll need to do some volume mounting.

To use the image:

docker run --rm --shm-size=2g ghcr.io/mintapi/mintapi mintapi [email protected] my_password --headless --use-chromedriver-on-path

AWS Lambda Environment

AWS Lambda may need a specific chrome driver with specific options. You can initialize Mint with your own pre-configured headless serverless chrome through a constructor:

driver = initialize_serverless_chrome_driver(...)
mint = mintapi.Mint(..., driver=driver)
...

Multi-Data Support

As of v2.0, mintapi supports returning multiple types of data in one call, such as: mintapi --accounts --budgets --transactions. When exporting multiple data types, you can either send it directly to stdout or you can export to a file via --filename. mintapi will create a file for each type of data, with a suffix based on the format. For example, if you run mintapi --accounts --transactions --filename=current --format=csv, then you will receive two files: current_account.csv and current_transaction.csv. The following table outlines the option selected and its corresponding suffix:

Option Suffix
accounts account
bills bills
budgets budget
transactions transaction
trends trends
categories category
investments investment
net-worth net_worth
credit-score credit_score
credit-report credit_report

Financial Data Trends

Mint supports providing some analysis of your financial data based on different types of "trends". Mint's requirements for accessing this data using mintapi is a bit more complex than the other endpoints.

Parameter Data Type Description
report_type ReportView.Options The type of report to generate.
date_filter DateFilter.Options The date window to analyze your trends.
start_date Optional[str] An optional beginning date (mm-dd-yy) to your trend analysis.
end_date Optional[str] An optional ending date (mm-dd-yy) to your trend analysis.
category_ids List[str] An optional list of category IDs to include in your trend analysis.
tag_ids List[str] An optional list of tag IDs to include in your trend analysis.
descriptions List[str] An optional list of descriptions to include in your trend analysis.
account_ids List[str] An optional list of account IDs to include in your trend analysis.
match_all_filters boolean Whether to match all supplied filters (True) or at least one (False)
limit int The page size of results.
offset int The starting record of your results.

Report Type

The Report Type is the type of report for which to generate trend analysis. The supplied value must be one of the following enum values:

Enum Value Description
1 Spending Over Time
2 Spending by Category
3 Spending by Merchant
4 Spending by Tag
5 Income Over Time
6 Income by Category
7 Income by Merchant
8 Income by Tag
9 Assets by Type
10 Assets Over Time
11 Assets by Account
12 Debts Over Time
13 Debts by Type
14 Debts by Account
15 Net Worth Over Time
16 Net Income Over Time

Financial Data Transactions

If you want to provide a more granular filtering of your financial data transactions, you can select from a variety of search filters that are sent to Mint.

Parameter Data Type Description
date_filter DateFilter.Options The date window for which to filter your transactions.
start_date Optional[str] An optional beginning date (mm-dd-yy) to your transaction filtering.
end_date Optional[str] An optional ending date (mm-dd-yy) to your transaction filtering.
category_ids List[str] An optional list of category IDs of transactions to include.
tag_ids List[str] An optional list of tag IDs of transactions to include.
descriptions List[str] An optional list of descriptions of transactions to include.
account_ids List[str] An optional list of account IDs of transactions to include.
match_all_filters boolean Whether to match all supplied filters (True) or at least one (False)
include_investment boolean Whether to include those transactions that are associated with an Investment Account.
remove_pending boolean Whether to remove those transactions that are still Pending.
limit int The page size of results.
offset int The starting record of your results.

Date Filters

The Date Filter is the date window for which to generate your trend analysis or for which to search transactions. The supplied value must be one of the following enum values:

Enum Value Description
1 Last 7 Days
2 Last 14 Days
3 This Month
4 Last Month
5 Last 3 Months
6 Last 6 Months
7 Last 7 Months
8 This Year
9 Last Year
10 All Time
11 Custom

If you select a Custom Date Filter, then start_date and end_date are required fields. Similarly, if you wish to use start_date and end_date, Custom Date Filter must be used.

From Python

From python, instantiate the Mint class (from the mintapi package) and you can make calls to retrieve account/budget information. We recommend using the keyring library for persisting credentials.

  import mintapi
  mint = mintapi.Mint(
    '[email protected]',  # Email used to log in to Mint
    'password',  # Your password used to log in to mint
    # Optional parameters
    mfa_method='sms',  # See MFA Methods section
                       # Can be 'sms' (default), 'email', or 'soft-token'.
                       # if mintapi detects an MFA request, it will trigger the requested method
                       # and prompt on the command line.
    mfa_input_callback=None,  # see MFA Methods section
                              # can be used with any mfa_method
                              # A callback accepting a single argument (the prompt)
                              # which returns the user-inputted 2FA code. By default
                              # the default Python `input` function is used.
    mfa_token=None,   # see MFA Methods section
                      # used with mfa_method='soft-token'
                      # the token that is used to generate the totp
    intuit_account=None, # account name when multiple accounts are registered with this email.
    headless=False,  # Whether the chromedriver should work without opening a
                     # visible window (useful for server-side deployments)
                         # None will use the default account.
    session_path=None, # Directory that the Chrome persistent session will be written/read from.
                       # To avoid the 2FA code being asked for multiple times, you can either set
                       # this parameter or log in by hand in Chrome under the same user this runs
                       # as.
    imap_account=None, # account name used to log in to your IMAP server
    imap_password=None, # account password used to log in to your IMAP server
    imap_server=None,  # IMAP server host name
    imap_folder='INBOX',  # IMAP folder that receives MFA email
    wait_for_sync=False,  # do not wait for accounts to sync
    wait_for_sync_timeout=300,  # number of seconds to wait for sync
    fail_if_stale=True, # True will raise an exception if Mint is unable to refresh your data.
	  use_chromedriver_on_path=False,  # True will use a system provided chromedriver binary that
	                                 # is on the PATH (instead of downloading the latest version)
    driver=None,        # pre-configured driver. If None, Mint will initialize the WebDriver.
    quit_driver_on_fail=True  # Quit from the browser and driver if an unexpected exception caught.
                              # Could be useful to set it to False if the ownership of the driver should not be owned by Mint object.
  )

  # Get account information
  mint.get_account_data()

  # Get budget information
  mint.get_budget_data()

  # Get transactions
  mint.get_transaction_data() # as pandas dataframe

  # Get transactions for a specific account
  accounts = mint.get_account_data()
  for account in accounts:
    mint.get_transaction_data(id=account["id"])

  # Get net worth
  mint.get_net_worth_data()

  # Get credit score
  mint.get_credit_score_data()

  # Get bills
  mint.get_bills()

  # Get investments (holdings and transactions)
  mint.get_investment_data()
  
  # Get tags
  mint.get_tag_data()
  
  # Get rules for assigning category and description
  mint.get_rule_data()

  # Close session and exit cleanly from selenium/chromedriver
  mint.close()

  # Initiate an account refresh
  mint.initiate_account_refresh()

  # you can also use mintapi's login in workflow with your own selenium webdriver
  # this will allow for more custom selenium driver setups
  # one caveat is that it must be based on seleniumrequests currently
  # seleniumrequests has most browsers already
  # it also has mixins for any browsers it doesn't have so the sky is the limit!
  from seleniumrequests import Firefox
  mint = mintapi.Mint()
  mint.driver = Firefox()
  mint.status_message, mint.token = mintapi.sign_in(
    email, password, mint.driver, mfa_method=None, mfa_token=None,
    mfa_input_callback=None, intuit_account=None, wait_for_sync=True,
    wait_for_sync_timeout=5 * 60,
    imap_account=None, imap_password=None,
    imap_server=None, imap_folder="INBOX",
  )
  # now you can do all the normal api calls
  # ex:
  mint.get_transaction_data()

Run it as a sub-process from your favorite language; pip install mintapi creates a binary in your $PATH. From the command-line, the output is JSON:

    usage: mintapi [-h] [--session-path [SESSION_PATH]] [--accounts] [--investments]
                   [--beta] [--budgets | --budget_hist] [--net-worth]
                   [--credit-score] [--credit-report]
                   [--exclude-inquiries] [--exclude-accounts] [--exclude-utilization]
                   [--start-date [START_DATE]] [--end-date [END_DATE]]
                   [--limit] [--include-investment] [--show-pending]
                   [--format] [--filename FILENAME] [--keyring] [--headless]
                   [--mfa-method {sms,email,soft-token}]
                   [--categories] [--attention]
                   [--transactions] [--transaction-date-filter]
                   [--trends] [--trend-report-type] [--trend-date-filter]
                   email [password]

    positional arguments:
      email                 The e-mail address for your Mint.com account (required)
      password              The password for your Mint.com account (if not supplied, --keyring must be provided)

    optional arguments:
      -h, --help            show this help message and exit
      --accounts            Retrieve account information (default if nothing else
                            is specified)
      --session-path [SESSION_PATH]
                            Directory to save browser session, including cookies. Used to prevent repeated
                            MFA prompts. Defaults to $HOME/.mintapi/session. Set to None to use a temporary
                            profile.
      --beta                Use the beta version of Mint
      --bills               Retrieve bills information
      --budgets             Retrieve budget information for current month
      --budget_hist         Retrieve historical budget information (past 12 months)
      --categories          Retrieve your configured Mint categories
      --config-file, -c     File used to store arguments
      --credit-score        Retrieve credit score
      --credit-report       Retrieve full credit report & history
      --exclude-inquiries   Used in conjunction with --credit-report, ignores credit inquiry data.
      --exclude-accounts    Used in conjunction with --credit-report, ignores credit account data.
      --exclude-utilization Used in conjunction with --credit-report, ignores credit utilization data.
      --net-worth           Retrieve net worth information
      --transactions, -t    Retrieve transactions
      --transaction-date-filter The date window for which to filter your transactions.  Default is All Time.
      --trends              Retrieve trend data related to your financial information
      --trend-report-type   The type of report for which to generate trend analysis.  Default is Spending Over Time.
      --trend-date-filter   The date window for which to generate your trend analysis.  Default is This Month.
      --start-date [START_DATE]
                            Earliest date for which to retrieve transactions.
                            Used with --transactions. Format: mm/dd/yy
      --end-date [END_DATE]
                            Latest date for which to retrieve transactions.
                            Used with --transactions. Format: mm/dd/yy
      --investments         Retrieve data related to your investments, whether they be retirement or         personal stock purchases
      --include-investment  Used with --transactions
      --limit               Number of records to include from the API.  Default is 5000.
      --show-pending        Retrieve pending transactions.
                            Used with --transactions
      --fail-if-stale       At login, Mint attempts to refresh your data.  If you wish to exit when the sync fails, use this option.
      --filename FILENAME, -f FILENAME
                            write results to file. If no file is specified, then data is written to stdout.  Do not specify the file extension as it is determined based on the selection of `--format`.
      --format              Determines the output format of the data, either `csv` or         `json`.  The default value is `json`.  If no `filename` is specified, then this determines the `stdout` format.  Otherwise, if a `filename` is specified, then this determines the file extension.
      --keyring             Use OS keyring for storing password information
      --headless            Whether to execute chromedriver with no visible
                            window.
	  --use-chromedriver-on-path
	  						Whether to use the chromedriver on PATH, instead of
              			  	downloading a local copy.
      --mfa-method {sms,email,soft-token}
                            The MFA method to automate.
      --mfa-token      The base32 encoded MFA token.
      --imap-account IMAP_ACCOUNT
      --imap-password IMAP_PASSWORD
      --imap-server IMAP_SERVER_HOSTNAME
      --imap-folder IMAP_FOLDER
                            Default is INBOX
      --imap-test           Test access to IMAP server
      --no_wait_for_sync    Do not wait for accounts to sync
      --wait_for_sync_timeout
                            Number of seconds to wait for sync (default is 300)
      --attention.          Get notice if there are any accounts that need attention


    >>> mintapi --keyring [email protected]
    [
      {
        "accountName": "Chase Checking",
        "lastUpdatedInString": "25 minutes",
        "accountType": "bank",
        "currentBalance": 100.12,
        ...
      },
      ...
    ]

Special Considerations

Email / Account Access

Because of the inter-connected nature of Intuit's products, when signing in to Mint for one account, you may see accounts associated with Intuit products other than Mint. If you do have multiple Intuit accounts, you should be aware that if one email is associated with two different usernames (and multiple Intuit products, such as TurboTax or Quickbooks), you may receive a prompt for Multifactor Authentication, even with a saved session. One possible solution is separating the two accounts to use two different emails. For many email clients, you can route different email addresses to the same account by using a suffix. For example, you could have email addresses "[email protected]" and "[email protected]" and receive emails for both in the "[email protected]" inbox.

Chrome version

NOTE: You must have Chrome or Chromium installed, on the stable track, and be up-to-date! If you run into a SessionNotCreatedException about "ChromeDriver only supports Chrome version XX", you need to update Chrome.

mintapi's People

Contributors

adamgreenhall avatar apiology avatar burkematthew avatar calvinchancan avatar chino avatar danshorstein avatar dantidote avatar dzg avatar echohack avatar egwoo avatar eyadgaran avatar felciano avatar hshore29 avatar hudcap avatar jaron-l avatar jasonkissinger avatar jbms avatar jprouty avatar mark-adams avatar mcronce avatar mplewis avatar mrooney avatar robwil avatar sherbang avatar sjstelmach avatar snowskeleton avatar tsdg112 avatar wendlinga avatar willbiddy avatar zsolteszku avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mintapi's Issues

Session has expired error

I am getting the following exception. Ideas?

Exception: Could not parse account data: 1Session has expired.

JSON pull issue

For some reason the JSON pulls are throwing an error with the request_and_check. Not sure why this happens now. It's saying the content type 'application/son;charset=UTF-8' does not match 'text/json'

Going to that site, the data looks fine and seems to be in JSON format.

Net Worth?

How hard would it be to add a function that grabs your net worth (All Assets, including investments minus All Debts)? Ps Thanks for your hard work!

TooManyRedirects: Exceeded 30 redirects.

I am getting the title's error when trying to get accounts (after successfully connecting). Any suggestions?

Its happening in api.py at line 249:
accounts = self.populate_extended_account_detail(accounts)

Session has expired.

I get an exception when i try to get accounts:

Could not parse account data: <error><code>1</code><description>Session has expired.</description><name></name><type></type></error>

File "./accounts/views.py" in fetchraw
    73.             print(mint.get_accounts())
File "./venv/lib/python2.7/site-packages/mintapi/api.py" in get_accounts
    171.             raise Exception('Could not parse account data: ' + response)

getting connected account(s) status

  1. Is there any way to confirm that mint is able to log into all accounts? I know it is possible from the web site (you get a warning or error - I don't remember)
  2. How about getting upcoming bills?

Exporting JSON to Chart.js

Hi,

I am new to programming, so please forgive me if this question is dumb.

Is there anyway to export the data retrieved by mintapi to a website so that I can use it on a chart? I would like to chart to update automatically, rather than manually (me typing the data in everyday).

Thank you.

Mobile app API

Does Mint's mobile app use the same API as its web app? It's possible that the api used on mobile is much simpler / more friendly. (Especially with regard to persisting login credentials, etc).

Transaction queries

It seems like it would be really easy to add the ability to search/query the transactions. Here is a sample URL that grabs a CSV for all results that match "staples":
https://wwws.mint.com/transactionDownload.event?query=staples&queryNew=&offset=0&filterType=cash&comparableType=8

Here is a query for all transactions that were $6.90:
https://wwws.mint.com/transactionDownload.event?query=6.90&queryNew=&offset=0&filterType=cash&comparableType=8

Here are all transactions between 100 and 200 dollars:
https://wwws.mint.com/transactionDownload.event?query=100.00..200.00&queryNew=&offset=0&filterType=cash&comparableType=8

Here is a query for things matching "geico" and between 100 and 200 dollars:
https://wwws.mint.com/transactionDownload.event?query=geico%2C%20100.00..200.00&queryNew=&offset=0&filterType=cash&comparableType=8

Here are all transactions from a particular account:
https://wwws.mint.com/transactionDownload.event?accountId=6379996&query=&queryNew=&offset=0&comparableType=8

Here are all queries categorized as "student loan" and occurring between 02/01/2015 and between "02/28/2015":
https://wwws.mint.com/transactionDownload.event?startDate=02%2F01%2F2015&endDate=02%2F28%2F2015&query=category%3A%22student%20loan%22&queryNew=&offset=0&exclHidden=T&filterType=cash&comparableType=8

From the above examples, it looks like it would need to be possible to pass the parameters to get_transactions():

  • query
  • accountId
  • startDate
  • endDate

And then simply append them to the query.

result = self.session.get(
            'https://wwws.mint.com/transactionDownload.event',
            headers=self.headers,
            params=params
            )

I may end up modifying get_transactions() myself, as I would like to make a tool that can modify transactions' categories based on a query. However, if I don't get around to actually doing it, at the very least I have outlined this in a ticket so if anybody is inclined to implement this, they can.

I appreciate that this API exists and hope that I get a chance to contribute to it!

Mint now requires Captcha

Running the sample code now throws this error:

mintapi.api.MintException: Challenge required, please log in to Mint.com manually and complete the captcha.

KeyError in get_budgets with budget in top-level category

mint.get_budgets()

/Library/Python/2.7/site-packages/mintapi/api.pyc in get_budgets(self)
    254         for direction in budgets.keys():
    255             for budget in budgets[direction]:
--> 256                 budget['cat'] = categories[budget['cat']]
    257
    258         return budgets

KeyError: 13

Max retries exceeded

I installed your script and have tried to use it a couple of times. I"m getting a "Max retries exceeded" error. Any ideas of what could be causing this?

Relevant info:


cherry:~/mint# pip install mintapi
Downloading/unpacking mintapi
Downloading mintapi-1.2.tar.gz
Running setup.py egg_info for package mintapi

Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/dist-packages (from mintapi)
Installing collected packages: mintapi
Running setup.py install for mintapi
changing mode of build/scripts-2.7/mintapi from 644 to 755

changing mode of /usr/local/bin/mintapi to 755

Successfully installed mintapi
Cleaning up...
cherry:~/mint# mintapi
Mint email: MY_EMAIL_HERE
Password:
Traceback (most recent call last):
File "/usr/local/bin/mintapi", line 18, in
accounts = mintapi.get_accounts(email, password)
File "/usr/local/lib/python2.7/dist-packages/mintapi/api.py", line 8, in get_accounts
if session.get("https://wwws.mint.com/login.event?task=L").status_code != requests.codes.ok:
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 395, in get
return self.request('GET', url, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, *_send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wwws.mint.com', port=443): Max retries exceeded with url: /login.event?task=L (Caused by <class 'socket.error'>: [Errno 104] Connection reset by peer)

Login failing

When I try to log in using a valid email and password I get a "Mint.com login failed[1]" message. Sanitized output follows:

'''
$ python mintapi/api.py email password
Traceback (most recent call last):
File "mintapi/api.py", line 110, in
main()
File "mintapi/api.py", line 106, in main
accounts = get_accounts(email, password)
File "mintapi/api.py", line 33, in get_accounts
raise Exception("Mint.com login failed[1]")
Exception: Mint.com login failed[1]
'''

Calling the API from within Python produces a similar result. I tried this today (about 5 minutes ago) if that helps.

pyflakes: Undefined name 'Mint'

There's a problem here:

api.py:18:          raise Exception("Failed to load Mint main page '{}'".format(Mint.START_URL))

The "Mint" object isn't defined anywhere.

Auth not working - again?

Anybody else having authentication issues - again? My scripts have been working for a long while, but last night (and still this morning) it stopped working. I'll double-check my code as well, but just wondering if anybody else is having issues.

Script doesn't work - invalid syntax

H:\Downloads\mintapi-master\mintapi-master\mint>"C:\Program Files\Python33\Pytho
n.exe" api.py myemailaddress mypassword
File "api.py", line 74
print json.dumps(accounts, indent=2)
^
SyntaxError: invalid syntax

content type 'text/html; charset=UTF-8' does not match 'text/csv'

File "/usr/local/lib/python2.7/site-packages/mintapi/api.py", line 337, in get_transactions_csv expected_content_type='text/csv' File "/usr/local/lib/python2.7/site-packages/mintapi/api.py", line 106, in request_and_check (url, content_type, expected_content_type)) RuntimeError: Error requesting 'https://wwws.mint.com/transactionDownload.event', content type 'text/html; charset=UTF-8' does not match 'text/csv'
Using version mintapi-1.18.

Transactions Error?

This is what I get when I try to get transactions (both command line and in the code)

Traviss-MacBook-Pro:mintest teamcoltra$ mintapi --transactions
Mint e-mail: [email protected]
Mint password: 
Traceback (most recent call last):
  File "/usr/local/bin/mintapi", line 9, in <module>
    load_entry_point('mintapi==1.9', 'console_scripts', 'mintapi')()
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 434, in main
    data = mint.get_transactions()
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 149, in get_transactions
    raise ImportError('transactions data requires pandas')
ImportError: transactions data requires pandas

No JSON object could be decoded

I found this thread: #73 and found the reference to the ius_session cookie. I was able to retrieve my ius_session cookie from a new attempt to login on Firefox. I tried this new cookie like this:
mint = mintapi.Mint(email=EMAIL, password=PASSWORD, ius_session=IUS_SESSION)
but get the following:

  File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 1580, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 964, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "Y:/python development/mint_to_evernote/mint.py", line 5, in <module>
    mint = mintapi.Mint(email=EMAIL, password=PASSWORD, ius_session=IUS_SESSION)
  File "Y:\python development\github\mintapi\api.py", line 60, in __init__
    self.login_and_get_token(email, password, ius_session)
  File "Y:\python development\github\mintapi\api.py", line 146, in login_and_get_token
    response = json.loads(response)
  File "C:\Python27\lib\json\__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\lib\json\decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python27\lib\json\decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

I am running code from origin/master that I just pulled this morning (9/14/2016

Login Doesn't Work...

running mint.get_accounts() from python, I get the following error:

Exception('Could not parse account data: ' + response)
Exception: Could not parse account data: <error><code>1</code><description>Session has expired.</description><name></name><type></type></error>

running from the command line, I get this:

Exception: Mint.com login failed[1]

JSON export should represent dates as ISO8601

The json exports seem to serialize dates in Javascript format:

"addAccountDateInDate": "datetime.datetime(2009, 12, 19, 19, 46, 47)", 

Rather than an ISO-based representation or similar. It would be better if these were represented as an ISO8601-compatible string.

sslv3 alert handshake failure

Getting this:

requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:581)

When attempting to use mintapi.get_accounts. This worked for quite sometime until about a week ago.

Add support for exporting accounts to CSV

When trying to export account details into an Excel-compatible CSV format I get a ValueError

$ mintapi $MINTUSERNAME --extended-accounts --include-investment -f account-details.csv
Traceback (most recent call last):
  ... details omitted ...
    raise ValueError('file type must be json for non-transaction data')
ValueError: file type must be json for non-transaction data

I though this used to work and it isn't clear to me why it wouldn't. Can someone confirm?

Workaround is to install something like https://csvkit.readthedocs.io/, exporting a json version, and then using in2csv, but this seems like it would be nice parsimonious functionality to have built in.

Issues with getting transactions

Hello! I'm having issues with getting the transactions into a dataframe. My login seems to work fine, but the below code indicates that the transactions aren't being interpreted as a CSV type file.

How do I fix this?


RuntimeError Traceback (most recent call last)
in ()
4 # Get transactions
5
----> 6 mint_data=mint.get_transactions() # as pandas dataframe
7
8 budgets=mint.get_budgets()

C:\Users\schung013\AppData\Local\Continuum\Anaconda2\lib\site-packages\mintapi\api.pyc in get_transactions(self)
363 """
364 assert_pd()
--> 365 s = StringIO(self.get_transactions_csv())
366 s.seek(0)
367 df = pd.read_csv(s, parse_dates=['Date'])

C:\Users\schung013\AppData\Local\Continuum\Anaconda2\lib\site-packages\mintapi\api.pyc in get_transactions_csv(self, include_investment)
335 ('?accountId=0' if include_investment else ''),
336 headers=self.headers,
--> 337 expected_content_type='text/csv'
338 )
339 return result.content

C:\Users\schung013\AppData\Local\Continuum\Anaconda2\lib\site-packages\mintapi\api.pyc in request_and_check(self, url, method, expected_content_type, **kwargs)
104 raise RuntimeError(
105 'Error requesting %r, content type %r does not match %r' %
--> 106 (url, content_type, expected_content_type))
107 return result
108

RuntimeError: Error requesting 'https://wwws.mint.com/transactionDownload.event', content type 'text/html; charset=UTF-8' does not match 'text/csv'

get_budgets returns only the most recent month, not the history for the past year

The json data returned from mint.com contains the full history of budget data, but the output from mintapi only shows the current month.

I'm not familiar with Python, but I suspect it is related to one of these sections of setting up and filling out the return data. I feel like there is a loop missing...

    # Make the skeleton return structure
    budgets = {
        'income': response['data']['income'][
            str(max(map(int, response['data']['income'].keys())))
        ]['bu'],
        'spend': response['data']['spending'][
            str(max(map(int, response['data']['income'].keys())))
        ]['bu']
    }

    # Fill in the return structure
    for direction in budgets.keys():
        for budget in budgets[direction]:
            budget['cat'] = self.get_category_from_id(
                budget['cat'],
                categories
            )

Pulling Budget Information?

I'd love to see the ability to pull in budgets as well. I'd be happy to implement, if you could point me in the right direction. Did you work this stuff out by just sniffing ajax requests when logged in to the site?

Net-worth and accounts don't save to output file

If you try to save either net worth or account listings to a JSON file from the commandline, you end up with an empty (0 byte) file. Looks like at the bottom of api.py:

json.dumps(data, f, indent=2)

should be

json.dump(data, f, indent=2)

since only the latter call has built-in support for writing to a file-like object.

Connection resets

After a long wait, all mint/api.py gives me is:

  File "/home/anholt/src/mintapi/mint/api.py", line 55, in <module>
    accounts = get_accounts(email, password)
  File "/home/anholt/src/mintapi/mint/api.py", line 9, in get_accounts
    response = session.post("https://wwws.mint.com/loginUserSubmit.xevent", data=data).text
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 399, in post
    return self.request('POST', url, data=data, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 357, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 460, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 354, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wwws.mint.com', port=443): Max retries exceeded with url: /loginUserSubmit.xevent (Caused by <class 'socket.error'>: [Errno 104] Connection reset by peer)

Getting SSL3 bad handshake error

About a week ago I started getting this error when running mintapi from the command line:

requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)

This happens whether it is called from code or commandline. I thought it might be a sporadic SSL issue with Mint but it doesn't seem to be going away.

I also did a pip install mintapi --upgrade which pulled down v1.16 and upgraded to requests 2.9.0. These did not fix the issue

Any thoughts on how to diagnose this further?

mint.get_transactions_json()

Got everything (that I am using so far) working with one exception. I am trying to limit how far this method reaches back in time. Here is my code:
first_of_month = datetime.date.today() first_of_month = datetime.datetime.strptime(str(first_of_month.year) + "-" + str(first_of_month.month) + "-1","%Y-%m-%d") print "getting transactions from " + str(first_of_month) + "..." transactions = mint.get_transactions_json(include_investment=False, skip_duplicates=False, start_date=first_of_month)
I checked the date and it looks good (1/10/16) but it is pulling transactions from years ago. Am I doing something wrong?

import of keyring should be lazy

Keyring doesn't work on some platforms (notably AppEngine). This could be fixed by lazy loading it in main() but it is still listed as a required package in setup.py. Should it still be a strict requirement?

selenium / chromedriver not working on OSX

After a successful install of mintapi, chromedriver, and selenium, I see this output.

➜  dashboard git:(master) ✗ mintapi
Mint e-mail: [email protected] <- I put my real email here
Mint password:
Traceback (most recent call last):
  File "/usr/local/bin/mintapi", line 9, in <module>
    load_entry_point('mintapi==1.23', 'console_scripts', 'mintapi')()
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 703, in main
    mint = Mint.create(email, password, ius_session=options.session, thx_guid=options.thx_guid)
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 74, in create
    mint.login_and_get_token(email, password, ius_session, thx_guid)
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 135, in login_and_get_token
    session_cookies = self.get_session_cookies(**data)
  File "/Library/Python/2.7/site-packages/mintapi/api.py", line 183, in get_session_cookies
    (e.args[0] if len(e.args) > 0 else 'No error message found.'))
RuntimeError: ius_session not specified, and was unable to load the chromedriver selenium plugin. Please ensure that the `selenium` and `chromedriver` packages are installed.

The original error message was: No error message found.

brew services list ->

chromedriver      started ... /Users/.../Library/LaunchAgents/homebrew.mxcl.chromedriver.plist

Date contains only the month and date but not the year

When I'm downloading my data, all my date fields are of the form, "Month date", (e.g. Oct 29). However, I wanna get also the year of the date and get something like Oct 29, 2015. Can you fix that please? I guess it's pretty simple.

Return parent and sub-categories?

Right now if a subcategory (e.g. "Office Supplies") shows up under two parent categories in mint (e.g. "Business Expenses" and "Home Office") then they are indistinguishable in the exported CSV and JSON files. Would it be possible to return both parent and sub-categories for Mint transactions, for example using a category identifier?

Budget Undefined Categories

Thank you for this great script! I've been using it successfully and even integrated it into an Alexa Skill. However, budgets don't seem to be working quite right for me. mintapi fetches my budgets, but the returned JSON leaves most of the cats as "Undefined". All of my budgets seem to be here and have the correct amounts, but I can't determine which budget they belong to.

Command

# /opt/python-2.7.2/bin/mintapi --budgets --session 123412341234 --thx_guid 123412341234 [email protected] password

Result

{
  "spend": [
    {
      "catTypeFilter": "Personal", 
      "ramt": 0, 
      "cat": "Gas & Fuel", 
      "pid": 14, 
      "amt": 82.16, 
      "isIncome": false, 
      "isTransfer": false, 
      "bgt": 140.0, 
      "ex": false, 
      "id": 56418315, 
      "st": 1, 
      "type": 0, 
      "isExpense": true, 
      "rbal": 57.84
    }, 
    {
      "catTypeFilter": "Personal", 
      "ramt": 0, 
      "cat": "Unknown", 
      "pid": null, 
      "amt": 90, 
      "isIncome": false, 
      "isTransfer": false, 
      "bgt": 700.0, 
      "ex": false, 
      "id": 56418316, 
      "st": 1, 
      "type": 0, 
      "isExpense": true, 
      "rbal": 610.0
    }, 
    {
      "catTypeFilter": "Personal", 
      "ramt": 0, 
      "cat": "Unknown", 
      "pid": null, 
      "amt": 96.25, 
      "isIncome": false, 
      "isTransfer": false, 
      "bgt": 150.0, 
      "ex": false, 
      "id": 56418312, 
      "st": 1, 
      "type": 0, 
      "isExpense": true, 
      "rbal": 53.75
    }, 
    {
      "catTypeFilter": "Personal", 
      "ramt": 0, 
      "cat": "Unknown", 
      "pid": null, 
      "amt": 0, 
      "isIncome": false, 
      "isTransfer": false, 
      "bgt": 50.0, 
      "ex": false, 
      "id": 56418311, 
      "st": 1, 
      "type": 0, 
      "isExpense": true, 
      "rbal": 50.0
    }
  ]
}

Could not parse account data

C:\Python27>python c:\temp\api.py
Mint email: [email protected]
Password:
Traceback (most recent call last):
File "c:\temp\api.py", line 61, in
accounts = get_accounts(email, password)
File "c:\temp\api.py", line 43, in get_accounts
raise Exception("Could not parse account data: " + response)
Exception: Could not parse account data: 1Sessi
on has expired.

CSV export ignores investment transactions switch

The --include-investment flag does not result in investment transactions returned if the target file is a CSV file. Looks like a simple fix at line 695 of api.py:

elif options.transactions:
    data = mint.get_transactions()

needs to be

elif options.transactions:
    data = mint.get_transactions(include_investment=options.include_investment)

Login fails

I'm getting the error "Exception: Mint.com login failed[1]". My code:

import mintapi

email = 'my_email'
password = 'my_password'

mint = mintapi.Mint(email, password)

The full traceback of the error is:

Traceback (most recent call last):
  File "/Users/ericbiewener/Google Drive/Personal/Dev/ππ Misc/mint/mintapi.py", line 8, in <module>
    mint = mintapi.Mint(email, pw)
  File "/Users/ericbiewener/Google Drive/Personal/Dev/ππ Misc/mint/mintapi/api.py", line 47, in __init__
    self.login_and_get_token(email, password)
  File "/Users/ericbiewener/Google Drive/Personal/Dev/ππ Misc/mint/mintapi/api.py", line 119, in login_and_get_token
    raise Exception('Mint.com login failed[1]')
Exception: Mint.com login failed[1]

I get the same error when trying to connect via the command line mintapi command.

Challenge/Captcha

When logging in, I continually get the error: Exception: Challenge required, please log in to Mint.com manually and complete the captcha.
I can log in via a browser, but that session never carries over to the python side. What is meant by 'log in manually?'

get_transactions python 3 compatibility

Hey I noticed that PR #42 was merged on May 25 and mintapi-1.10.1 was released on the same day. Were the python 3 improvements from that PR intended to be included as part of the 1.10.1 release? When I try downloading it, the updates don't seem to be there.

$ curl -O https://pypi.python.org/packages/source/m/mintapi/mintapi-1.10.1.tar.gz
$ tar xvfz mintapi-1.10.1.tar.gz
$ cat mintapi-1.10.1/mintapi/api.py | grep StringIO

    from StringIO import StringIO
    s = StringIO()

Whereas the most recent commits in the repo would show:

$ cat mintapi/api.py | grep StringIO

from StringIO import StringIO  # Python 2
from io import BytesIO as StringIO  # Python 3
    s = StringIO(result.content)

Would love to have the added functionality in python 3 if possible. Thanks!

Can the API support write operations?

Hi --

I've been toying with the idea of automating the creation of Mint splits based on detailed transaction info. For example, Amazon allows you to download a detailed order history, which shows you the different items you purchased in a single order. These appear as a single transaction in Mint labeled "Amazon" and typically marked Shopping. More useful for budgeting and analysis would be to split this into the individual purchased items and categorize accordingly (books, camera gear, computer, etc).

Mint obviously allows you to do this by hand by splitting a transaction and adding the line items for each sub-transaction. I'm wondering whether this operation could also be done via mintapi calls to push data back to Mint? If so then this is something I'd be interested in contributing as a side project.

Any thoughts?

Ramon

Is this using the most up-to-date/best data sources?

I've found multiple datasources and not sure if this is using the best ones, and if there is any tribal knowledge around which ones might stick around longer - it would be sad to invest a lot of effort into reverse engineering datasources that will be deprecated.

There is the "master controller" one - which this project uses for almost everything. I don't like it as much as it requires a more complex POST request, but it does seem to provide the most verbose response:
https://wwws.mint.com/bundledServiceController.xevent
It requires the token and a request ID.

There are these, which are cleaner, but aren't as verbose as the bundledServiceController:
https://wwws.mint.com/getJsonData.xevent?task=tags
https://wwws.mint.com/getJsonData.xevent?task=categories
https://wwws.mint.com/getJsonData.xevent?task=accounts - this doesn't show hidden accounts, but does show current balance which bundledServiceController.xevent doesn't
https://wwws.mint.com/getJsonData.xevent?task=transactions
They all seem to support pagination with "offset" parameter if needed. They don't require the token or request ID, only cookies.

There are these, which seem to need the oauth based authorization before accessing:
https://mint.finance.intuit.com/v1/accounts
https://mint.finance.intuit.com/v1/fis
(maybe more)

Also, is there a particular reason why the login process hits 3 endpoints? It appears that you only need to hit loginUserSubmit.xevent and you get back everything you need (token and cookies) to make any other followup requests.

Error getting transactions

I cannot get this library to get anything off mint.com. I am calling it like this:

mintapi -t 'username' 'password'

I get this error when trying to get the transactions:

RuntimeError: Error requesting 'https://wwws.mint.com/transactionDownload.event', content type 'text/html; charset=UTF-8' does not match 'text/csv'

Is this library currently working?

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.