How to extract Curve data and make your own API (very nerdy content)

how-to

#1

Hi!

I’ve wanted to show you how I managed to write a script automatically extracting my own transaction data out of Curve for my own purposes and how you can do it too (if you know some basic Python). The premise behind is that you can conduct an analysis of your own spend habits the way you want to, or even trigger some actions every time you make a purchase. I don’t really have any specific reason for doing that, at least for now, but while programming I always wanted to work on real data and I am looking forward to finding new ways in which it could be used.

A little disclaimer here: first of all, I know that I can extract CSV files from the Curve app itself. What I’m trying to achieve is more of an automated, flexible solution that doesn’t need any further action on user’s side. On another note, I know that what I did is pretty much a bodge that can stop working any time. I made it mostly for fun and it shouldn’t be used in any professional projects.

Step 1: Getting your transaction data out of Curve sandbox
Well, Curve doesn’t offer any type of a public API that would let you ask for your data. Does it mean that there’s no way of getting it? Obviously not. Curve offers this awesome feature called Receipts, which sends you an e-mail every time a transaction is made or updated. So, the best thing would be activating that feature for all your underlying cards. That’s our first hook that will let us access our information. I haven’t found any way to easily get historical data in an automated way. If you have any ideas, let me know!

Step 2: Keeping your inbox tidy and sorting your incoming Receipts
An e-mail every time you buy anything? That’s a lot of e-mails. So, in order to keep your inbox clean you need to set up a filter that will redirect the receipts into a dedicated folder. This will also come in handy later. I use Gmail, so my filter looks like this:

This means, that every receipt e-mail I get will skip inbox and will be automatically moved to a seperate place. This way it doesn’t affect our mail account and makes our script run faster.

Step 3: Downloading e-mails using a Python script
This is where the fun begins. You basically need to write a script that will download all the messages sent by Curve. I used Python 3 and a imap_tools module. (Note: if you use Gmail, this won’t work unless you give an explicit permission for your account to be accessed via less secure apps in settings). Notice how reaching the right mails is simpler when you have a filter set up.

Step 4: Parsing those e-mails
How much you get out of these mails is really a matter of your determination. As for now, my script uses only the data provided in the subtitles. That is:

  • a merchant name,
  • date of the purchase,
  • amount and a currency.

In the future I will try to edit my script to include other data in the e-mails, such as

  • amount in original currency (for foreign transactions),
  • time of the purchase,
  • date and time of the updates (Go Back in Time and/or refunds),
  • the underlying card details.

There are also some details that are unavailable to get that way (at least in the current form), for instance:

  • MCC codes/categories
  • how much you saved on fees.

Step 5: Using your data
Now that you extracted all the data from the e-mails sent by Curve, you can process it however you want. For instance, you can find the stores that get too much of your money or those that you visit really often. You can categorise them the way you want and make more thoughtful decisions, as your own code can give you more detailed overview than Curve Insights. I also considered tweaking this idea, so purchases could act like triggers for other actions (IFTTT-like), but with recent Push notification delays that would be kinda pointless.

Here’s a snippet of how it worked out for me:
obraz

Again, I know that for most this is an extremely mundane and nerdy topic, so thank you for those who read it all. Please leave your feedback on this if you think you could contribute, you enjoyed it, or if you want to get the source code. I am especially interested in an opinion of someone from Curve team, as I’m not entirely sure if such things are encouraged by you (and I truly hope I didn’t break any Terms and Conditions :grin:).


#2

Hey @Pawel, this is very impressive!

Could you drop me a dm so I can discuss this further with you? :slight_smile:


#3

This is fantastic :clap: :heart_eyes: Just shared it with the whole Curve Team - let’s see what they say!