How can I "cache" a mongoDB/Mongoose result to be used in my Express.js views and routes - mongodb

Question

What I'm trying to achieve is some sort of way to cache results of a mongoDB/Mongoose query that I can use in my views and routes. I'd need to be able to update this cache whenever a new document is added to the collection. I'm not sure if this is possible and if it is then how to do it, due to how the functions are asynchronous
This is currently what I have for storing the galleries, however this is executed with every request.
app.use(function(req, res, next) {
Gallery.find(function(err, galleries) {
if (err) throw err;
res.locals.navGalleries = galleries;
next();
});
});
This is used to get gallery names, which are then displayed in the navigation bar from a dynamically generated gallery. The gallery model is setup with just a name of the gallery and a slug
and this is part of my EJS view inside of my navigation which stores the values in a dropdown menu.
<% navGalleries.forEach(function(gallery) { %>
<li>
<a href='/media/<%= gallery.slug %>'><%= gallery.name %></a>
</li>
<% }) %>
The website I'm working on is expected to get hundreds of thousands of concurrent users, so I don't want to have to query the database for every single request if not needed, and just update it whenever a new gallery is created.

Answer 1

Take a look at cachegoose. It will allow you to cache any query you want and invalidate that cache entry each time a new gallery is created.
You will need something like this:
const mongoose = require('mongoose');
const cachegoose = require('cachegoose');
cachegoose(mongoose); // You can specify some options here to use Redis instead of in-memory cache
app.get(function(req, res, next) {
...
Gallery
.find()
.cache(0, 'GALLERY-CACHE-KEY')
.exec(function(err, galleries) {
if (err) throw err;
res.locals.navGalleries = galleries;
next();
});
...
});
app.post(function(req, res, next) {
...
new Gallery(req.body).save(function (err) {
if (err) throw err;
// Invalidate the cache as new data has been added:
cachegoose.clearCache('GALLERY-CACHE-KEY');
});
...
});
Although you could do something simpler caching the results manually in a variable and invalidating that cache when new galleries are added, I would advise you to take a look at that package instead.

Answer 2

I've created modern library for handling cache clearing automatically. In short it's more advanced and faster than cachegoose, because it's also caching Mongoose Documents instances in memory.
Speedgoose has autoclearing plugin, which you can set on a given schema, So it's not only ttl-based. It works on mongoose document events to clear results related to the document. So if the result of query XYZ is cached, and it was containing the record that was edited/removed, it will be removed from the cache. Same with adding and removing documents from schema. Those events might affect every cached result in the collection. So they are clearing the cache in the scope of a given model. I should be rather enough in 90% of cases.
In near future it will have many more features. And it's pretty easy to setup!
https://www.npmjs.com/package/speedgoose
Setup
import {applySpeedGooseCacheLayer} from "speedgoose";
import mongoose from "mongoose";
applySpeedGooseCacheLayer(mongoose, {
redisUri: process.env.REDIS_URI
})
Cache something!
//Caching both - query result, and instances of related Mongoose Documents in memory. It supports sort, select,project...etc.
model.find({}).sort({fieldA : 1}).cacheQuery()
//Caching only result
model.find({}).lean().cacheQuery()
//Works also with aggregation
model.aggregate([]).cachePipeline()