Jun 21, 2020

Importing Stock Transactions Into Gnucash With Python

I use GnuCash, but keeping my 401(k) accounts up to date has always been a tedious, manual process.

I came up with the python snippet below to handle the import, but it was a pain to write since I couldn't find any examples for setting up stock or mutual fund transactions.

The SetSharePriceAndAmount method ended up being key, and I only found that by searching through the gnucash source.

The GncNumeric class also ended up being more of a pain to use than I expected. There's probably a better way to use it, but the 'multiple values by 1000000/100000' approach is working for me now.

I'm using the stock GnuCash and python-gnucash version 2.6.19 available in Ubuntu 18.04, so this stuck using python 2.7.

#!/usr/bin/python2.7

import csv
from datetime import datetime

import gnucash

session = gnucash.Session("xml://yourfile.gnucash")
book = session.book
root_account = book.get_root_account()

usd = book.get_table().lookup('ISO4217','USD')

# There's probably a better way to use 'Your:Retirement:Contributions' instead ....
contrib_acct = root_account.lookup_by_name("Your").lookup_by_name("Retirement").lookup_by_name("Contributions")

parent_acct = root_account.lookup_by_name("401k")

with open('your_transactions.csv', 'rb') as trans_csv:
  trans_reader = csv.reader(trans_csv, delimiter=',')

  # Skip over the first row since it's headers
  header = next(trans_reader)

  for description, date, fund_name, share_price_str, share_amount_str, amount_str in trans_reader:
    child_account = parent_acct.lookup_by_name(fund_name)

    posting_date = datetime.strptime(date,"%m/%d/%y")

    tx = gnucash.Transaction(book)
    tx.BeginEdit()

    tx.SetCurrency(usd)

    tx.SetDatePostedTS(posting_date)
    tx.SetDescription(description)

    sp1 = gnucash.Split(book)
    sp1.SetParent(tx)
    sp1.SetAccount(child_account)

    # GncNumeric(n,d) represents numbers as fractions of the form n/d, so GncNumeric(1234567/1000000) = 1.234567
    # There's probably a better way to do this...
    share_price = gnucash.GncNumeric(float(share_price_str)*(10**6), 10**6)
    share_amount = gnucash.GncNumeric(float(share_amount_str)*(10**6), 10**6)

    # share_price * share_amount == amount, so I could have used that instead using the value from the csv
    amount = gnucash.GncNumeric(float(amount_str)*(10**6), 10**6)

    # ( ˘▽˘)っ♨  This is the secret sauce for setting the number of shares and the price.
    sp1.SetSharePriceAndAmount(share_price, share_amount)

    sp2 = gnucash.Split(book)
    sp2.SetParent(tx)
    sp2.SetAccount(contrib_acct)
    sp2.SetValue(amount.neg())

    tx.CommitEdit()
session.save()
session.end()

Special thanks to this post for providing most of the code above.

Sep 24, 2017

Lessons from Wriing a Pylint Plugin

At work there's a python coding convention that I tend to overlook a lot. So when I post merge requests, there's a pretty good chance someone's going to call me out on this, which leads to a followup commit and another round of peer review. This can lead to an extra delay of a few hours until I notice the comments, switch context back to that merge request, making the changes, update the merge request and wait for another round of reviews. If I could find a way to check my code for this convention before posting the merge requsts, I could get my code merged in a few hours faster....

The Convention

The coding convention I cannot internalize is as follows: In python, the format method for strings will call the __format__ method on its arguments for you, so any code that looks like:

"interpolate these: {} {}".format(str(a), str(b))

Need only look like:

"interpolate me: {} {}".format(a, b)

The Pylint Plugin

So googling around led my to this Ned Batchelder post from a few years back. That post also led to a couple pylint plugins here. Looking at pylint's own format checker reminded me that I should also be handling keyword arguments.

From the post and sample code, it looked like I needed to define a checker class with a visit_callfunc method that would check when the 'format' method was used, and then check all the arguments to the format call and throw an error if any of them where a function call to str().

Here's what I eventually ended up.

To come up with this I used an embarassing amount of exploratory programming to figure out astroid. I wrote an initial visit_callfunc() method based on the sample code that didn't do much more than dump out all the data about the node argument via dir(node) and node.__dict__. Then I would call pylint with the plugin against some sample source with the error I was trying to plugin to report.

I run the plugin against the existing code and found one lingering case where the reviewers had allowed one of my unneccessary str() call into the codebase. It's been removed now.

Lessons Learned

  • pylint plugins are pretty powerful and I wouldn't shy away from writing another one. I'm on the lookout for other excuses to write another one.
  • https://greentreesnakes.readthedocs.io is a useful 'missing manual' for the python AST.
  • format() can take both positional and keyword arguments. My original pass at the plugin only supported positional arguments.
  • The bandit project exists and looks useful. I stumbled acros it while looking for other pylint plugins.