How do I store large files using Meteor? - mongodb

Question

I'm building a Meteor (meteorjs) app that needs to store and display PDF files, sometimes as large as 500Mb. GridFS doesn't seem to be integrated yet so I'm wondering if it's worth using Meteor in this case or stick to Rails.
Ideally I would not use S3 - I'd like to keep the files on my server.
UPDATE: it seems it's possible to connect outside of Meteor directly, I don't need PDFs to be automatically moved - and it likely doesn't make sense.
More specifically I'm now looking at:
MongoDB -> ElasticSearch using https://github.com/richardwilly98/elasticsearch-river-mongodb
Using the instructions at https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki

Answer 1

You can use GridFS inside meteor without touching any extra package
var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db; //grab the database object
var GridStore = MongoInternals.NpmModule.GridStore;
WebApp.connectHandlers.use('/someurl', function(req, res) {
var bigFile = new GridStore(db, 'bigfile.iso', 'r') //to read
bigFile.open(function(error, result) {
if (error) return
bigFile.stream(); //stream the file
bigFile.on('error', function(e) {...}) //handle error etc
bigFile.on('end', function() {bigFile.close();}); //close the file when done
bigFile.pipe(res); //pipe the file to res
});
});
However, the current GridStore/mongo (v1.3.x) used by Meteor is a bit dated, the newest verion is 2.x from http://mongodb.github.io/node-mongodb-native/2.0/api-docs/
The v1.x doesnt seem to pipe well so you may need to use the newer version
The second option
var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db; //grab the database object
var GridStore = Npm.require('mongodb').GridStore; //add Npm.depends({mongodb:'2.0.13'}) in your package.js
WebApp.connectHandlers.use('/someurl', function(req, res) {
var bigFile = new GridStore(db, 'bigfile.iso', 'r').stream(true); //the new API doens't require bigFile.open() and will close automatically on end
bigFile.on('error', function(e) {...}); //handle error etc
bigFile.on('end', function() {...});
bigFile.pipe(res); //pipe the file to res
});
In this example, I use the WebApp.connectHandlers, but of course you can use iron: router or something. I tried with a file of 500 MB and it pipes all well. You also need to set the res.writeHead(200) and other stuff such as content-type, etc