Rails 4 + PostgreSQL (Heroku): complex/scalable queries for analysis/reporting on data - postgresql

Question

Working on a financial application that tracks sales. However, I'm running into problems trying to create a schema for properly tracking the data for reports (the main point of the app).
A purchase is the foundation of the app. It has several associations (listed below). Each purchase is tracked via a year and month field. A year is the smallest unit a user may filter a report by, so I will only have to show data for each month in that year.
# purchase.rb model
class Purchase < ActiveRecord::Base
# Associations:
# belongs_to :partner
# belongs_to :purchase_type
# belongs_to :purchase_category
# Attributes:
# partner_id => association
# purchase_type_id => association
# purchase_category_id => association
# year => year in integer (2013, 2014, etc...)
# month => month in integer ("January" => 1, etc...)
# amount => amount a product sold for in cents ($10.00 => 1000)
# fee => fee for associated partner (if there is one) in cents ($2.00 => 200)
end
The problem is that I need to show an overview for a given year, which breaks things down by how many purchases were completed, what partners completed them, and what were the fee amounts. I solved that by having YearMetric and MonthMetric tables that are updated everytime a purchase is added/updated/removed. So you add a new purchase for a given year/month, and the corresponding YearMetric and MonthMetric rows are found and updated with +/- the appropriate amounts/fees.
This solution works well for the overview page. However, I also need to be able to view purchases in the context of partners, purchase_types, and purchase_categories. If I followed the same strategy as my overview report, I would have to add the following tables:
PartnerYearMetric, PartnerMonthMetric
PurchaseCategoryYearMetric, PurchaseCategoryMonthMetric
PurchaseTypeYearMetric, PurchaseTypeMonthMetric
So everytime I add a purchase, I would be doing up to 8 additional DB updates (8 finds and then 8 updates).
The items I'm reporting on are total purchases made, average purchases (historical comparison), total amounts/fees for the period, top partners by number of purchases and by most fee amounts, etc...
There has to be a better solution than this. "Live calculation" by updating 8 records for every 1 purchase seems a bit overkill.

Answer 1

What you're doing is maintaining materialized views of the data in the application. It's a form of denormalization. That can be OK as an optimization but should not be your first choice. It can be very error prone, especially in the presence of concurrency, and must be done quite carefully.
Instead, when you wish to generate a summary report, use an aggregate to SUM them, COUNT them, etc as appropriate. See aggregate functions in the Pg docs, rails Calculations, rails aggregates.
You may find it convenient to create a VIEW over the query you use, and then access the view from the application.
If you find performance of calculating the aggregates in real time for the summary to be a problem, and you cannot solve it with proper indexing and tuning, then you should think about denormalizing. Rather than maintaining your materialized views in the app, though, consider using triggers in the database; they're much easier to write in a concurrency-safe way.
You may also want to look up PostgreSQL 9.4's enhanced materialized views support.