I'm working on a site with a MEAN stack scaffolded from this yeoman.io generator: https://github.com/DaftMonk/generator-angular-fullstack, and I'm trying to upload some image files to MongoDB in binary form. Here is my git repo for the project:
https://github.com/peter-atlanta/personal-site
I've followed #aheckmann's GIST to a tee: https://gist.github.com/aheckmann/2408370,
But I keep getting errors about how my files can't be found, i.e.
Error: ENOENT, no such file or directory '../../client/assets/images/github.png'
at Error (native)
at Object.fs.openSync (fs.js:500:18)
at Object.fs.readFileSync (fs.js:352:15)
at Immediate.<anonymous> (/Users/peterward/petergrayward/blog/server/config/imageToMongo.js:43:21)
at Immediate._onImmediate (/Users/peterward/petergrayward/blog/node_modules/mongoose/node_modules/mquery/lib/utils.js:137:16)
at processImmediate [as _immediateCallback] (timers.js:358:17)
Clearly, though, the png in question is located in that directory, and I've even tried moving the directory server-side to no avail.
Why can't a file/directory entry-point be found?
When it comes to storing small files in MongoDb, many answers which rely on Mongoose will also depend on using GridFS. If you are okay with simply storing files in MongoDB without relying on GridFS and Mongoose then try the following. (This answer is in TypeScript which I think is easier to read, but I can post the transpiled JavaScript if need be.)
import mongodb = require("mongodb");
const URI = 'mongodb://...';
var dbPromise = new Promise<mongodb.Db>(function (resolve, reject) {
mongodb.MongoClient.connect(URI, function (err, db) {
resolve(db);
});
});
import express = require("express");
var app = express();
import path = require("path");
import multer = require("multer");
var upload = multer({ dest: 'uploads/' });//saving files on filesystem seems to be a requirement
import fs = require("fs");
interface IFile {
name : string;
binary : mongodb.Binary;
}
//<input type="file" type="file" id="png1" name="png1" />
app.post("/file-upload", upload.single("png1"), function (req, res) {
let f = req.file;
fs.readFile(f.path, function (err, buffer) {
dbPromise.then(db => {
let file : IFile = {
name: f.originalname,
binary: new mongodb.Binary(buffer)
};
db.collection('files').insertOne(file, function(err, result) {
if (err) {
res.json(500, err);
}
console.log(`${result.insertedId} ${result.insertedCount}`);
});
});
})
});
Related
Following the manual from npm mongodb webpage(https://www.npmjs.com/package/mongodb) I created in progect directory and then specified a database directory by typing mongod --dbpath=/data? then I launched this file:
app.js
const { MongoClient } = require('mongodb');
// Connection URL
const url = 'mongodb://localhost:27017';
const client = new MongoClient(url);
// Database Name
const dbName = 'myProject';
async function main() {
// Use connect method to connect to the server
await client.connect();
console.log('Connected successfully to server');
const db = client.db(dbName);
const collection = db.collection('documents');
const insertResult = await collection.insertMany([{ a: 1 }, { a: 2 }, { a: 3 }]);
console.log('Inserted documents =>', insertResult);
// the following code examples can be pasted here...
return 'done.';
}
main()
.then(console.log)
.catch(console.error)
.finally(() => client.close());
and I got success, but folder data is still completely empty. What is wrong?
Actually, when you call "dbpath=/data" even being in your directory with this folder created, this command in Windows means that your db will be stored in folder "C:/ProgramFiles/mongodb/data". To specify your progects folder, use full path
I am a frontend developer trying to broaden my horizons, and making what will become a MERN application. I'm struggling with image uploads to mongodb.
First I used the express bodyparser:
app.use(express.urlencoded({ extended: true }));
and app.use(express.json());
when used like this I managed to upload the file fine, and the uploaded file showed up in MongoDB Compass.
I found out that this doesn't support multipart/form-data, so I've changed the bodyparser to busboy-bodyparser so that I can access both form-data and the file that is being uploaded. So I changed the bodyparser to:
app.use(busboyBodyParser());
and now it won't upload the request-data to mongodb.
My upload control looks like this:
const upload = require("../middleware/upload");
const uploadFile = async (req, res) => {
try {
req.file = req.files.file;
await upload(req, res);
if (req.file == undefined) {
return res.send(`You must select a file.`);
}
return res.send(`File has been uploaded.`);
} catch (error) {
console.log(error);
return res.send(`Error when trying upload image: ${error}`);
}
};
module.exports = {
uploadFile: uploadFile
};
the reason I've set req.file equals to req.files.file is because busboy-bodyparser sends the file from req.files.file and not req.file, and I thought that this change would make the request function properly, it did not.
My upload-middleware looks like this:
const promise = mongoose.connect(mongoURI, { useNewUrlParser: true, useUnifiedTopology: true });
const conn = mongoose.connection;
let gfs;
conn.once('open', () => {
gfs = Grid(conn, mongoose.mongo);
gfs.collection('uploads');
});
//create storage object
const storage = new GridFsStorage({
db: promise,
file: (req, file) => {
return new Promise((resolve, reject) => {
crypto.randomBytes(16, (err, buf) => {
if (err) {
return reject(err);
}
const filename = buf.toString('hex') + path.extname(file.originalname);
const fileInfo = {
filename: filename,
bucketName: 'uploads',
metadata: {
title: req.body.title,
orientation: req.body.orientation
}
};
resolve(fileInfo);
});
});
}
});
const uploadFile = multer({ storage }).single("file");
var uploadFilesMiddleware = util.promisify(uploadFile);
module.exports = uploadFilesMiddleware;
I believe this is the code that logs (node:15124) DeprecationWarning: Listening to events on the Db class has been deprecated and will be removed in the next major version.
(Use node --trace-deprecation ... to show where the warning was created)
which is another problem I'm unsure how to solve, but that's another problem for another day.
My end goal is to be able to send the file to mongodb, with the attached metadata (title and orientation).
with this code I'm able to get the "File has been uploaded" message from the upload-control, but in mongoDB compass no file/chunks has been uploaded. The uploads worked great on file-uploads (without the metadata) with the express-bodyparser, so when I changed that to the busboy-bodyparser I get both the file and the metadata as intended but it is not loaded into the db, which leads me to believe that the new bodyparser changes the request so that GridFsStorage no longer recognizes it and doesn't put the data into the db. But frankly I'm just speculating here, and I generally have very limited knowledge of backend.
I use the correct enctype on the form I believe:
<form
action="/upload"
method="POST"
enctype="multipart/form-data">
any tips or explanations is very much appreciated!
I am a complete beginner in backend, so don't be afraid to spell it our for me :)
I managed to fix it!
I'm unsure what caused it, but I believe that the req.body-fields hadn't been populated yet or something of that nature. I therefore switched out
metadata: {
title: req.body.title,
orientation: req.body.orientation
}
with: metadata: req.body and it just works.
For any other backend-newbie who might stumble upon this, also remember to name your inputs in html like this: <input name="title" type="text" /> it is the name-attribute that gets submitted with the html-form and gives the key to req.body, so that you can access for example req.body.title (which didn't work here, but still worth knowing)
I try to make a simple file upload REST interface using NEST.JS and MULTER -- but its not working. I am able to POST a binary file debug.log to the URL, and I see the "Hello undefined" message, but the uploaded file neither is created at the given folder uploads nor it is rejected because the extension is not correct - according to the file filter.
However, no exception or error is shown.
Why is multer not working?
Why is the #Uploadedfile() file shown as undefined?
Thanks
import { Controller, Post, Request, UseInterceptors, FileInterceptor, UploadedFile, HttpCode, HttpException, HttpStatus } from '#nestjs/common';
import { diskStorage, File } from 'multer';
const path = require('path');
const myStorage = diskStorage({
destination: function (req, file, callback) {
callback(null, './uploads/');
},
limits: { fileSize: 1000000 },
fileFilter: function (req, file, cb) {
const extension = path.extname(file.originalname).toLowerCase()
const mimetyp = file.mimetype
if (extension !== '.jpg' || mimetyp !== 'image/jpg') {
cb(new HttpException('Only images are allowed', HttpStatus.NOT_ACCEPTABLE));
}
cb(null, true);
},
filename: function (req, file, callback) {
callback(null, file.fieldname + '_' + Date.now() + '.jpg');
}
});
#Controller('documents')
export class DocumentsController {
#Post()
#HttpCode(HttpStatus.OK)
#UseInterceptors(FileInterceptor('file', { storage: myStorage }))
public async addDocument(#UploadedFile() file): Promise<any> {
console.log("Hello " + file);
}
}
I know it's a late reply for the person who asked this question, but the answer is for the people who will face it in the future.
First, check the nestjs documentation
If MulterModule registered in the *.module.ts file it creates the upload directory outside of the src folder. which is a good practice.
MulterModule.register({
dest: './upload',
});
If there is a need to change the destination directory use destination: String|callback from DiskStorageOptions or dest:string from FileInterceptor. For upload path use like src/../upload to keep the folder outside the src directory.
Other problems from the above code mentioned in the question:
I see the "Hello undefined" message
reason console.log("Hello " + file); here file is Object and trying concatenate with String.
change it to console.log("Hello ", file);
the extension is not correct, no exception or error is shown
fileFilter: function (req, file, cb) {
const extension = path.extname(file.originalname).toLowerCase()
const mimetyp = file.mimetype
if (extension !== '.jpg' || mimetyp !== 'image/jpg') {
cb(new HttpException('Only images are allowed', HttpStatus.NOT_ACCEPTABLE));
}
cb(null, true);
}
Here need to add a return.
I am trying to fill a pdf form using nodejs.
Im trying to use node-pdftk package for the same.Did following steps:
Installed pdftk for windows
mapped the path to the environment variables PATH
installed node-pdf package
`const express = require('express')
const app = express()
const cors = require('cors')
const pdftk = require('node-pdftk');
const bodyParser = require('body-parser');
var multer = require('multer'); // v1.0.5
var upload = multer();
app.listen(8000, () => {
console.log('Server started!')
});
var pdfPath='OoPdfFormExample.pdf';
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
var formdata = {
'Given Name': 'Swetha',
'Family Name': 'Gulapannavar'
}
app.post('/api/file', upload.array(), (req, res, next) => {
//var buffer=JSON.stringify(Buffer.from(req.body));
var buffer=Buffer.from(req.body)
pdftk
.input(pdfPath)
.fillForm(formdata)
.flatten()
.output()
.then(buffer => {
// Still returns a buffer
res.type('application/pdf'); // If you omit this line, file will
download
res.send(buf);
})
.catch(err => {
res.send(err.message)
// handle errors
});
});`
but i'm getting following error when i try to execute the same.
Write EPIPE error.
This could be caused by node-pdftk not being able to find the PDFtk binaries; your PATH variable may not be set correctly for the account running your web service. You can set the bin path directly inside of your application using node-pdftk's configure function, which is briefly described on the node-pdftk npm project page. If that doesn't work, try configuring the tempDir path.
I'm working on a project and I'm using a backend sails js, a MongoDB as database and front-end React js.
I have a problem on the upload image.
Here is the code Sails backend and when I test it in PostMan, I got this result (result test PostMan), and the image is not stored in the specified folder but a value containing id and name of file is inserted in MongoDB,
uploadFile: function (req, res) {
var image= req.file('avatar');
image.upload({
adapter: require('skipper-gridfs'),
uri: 'mongodb://localhost:27017/name_db.name_collection',
dirname: '../../assets/images/'
}, function (err, filesUploaded) {
if (err){
return res.header("Access-Control-Allow-Origin", "*");
/*res.negotiate(err);*/res.json(err);
}
else{
return
res.header("Access-Control-Allow-Origin", "*"); res.ok({
files: filesUploaded,
textParams: req.params.all()
});
}
});
},
result test postMan
And here is the Reactjs front-end code
_handleSubmit(e) {
e.preventDefault();
// TODO: do something with -> this.state.file
fetch('http://localhost:1337/uploadPhoto/logos', {
method: 'POST',
body: JSON.stringify({avatar:this.state.file})
})
.then((response) => response.json())
.catch((err) => { console.log(err); });
}
Sails version 0.12,
React version 15.5,
and MongoDB version 3.4.9.
Thanks for your help
There is no dirname parameter for skipper-gridfs adapter because the file won't be stored onto the local file system.
Skipper will store the file into the MongoDB database.
U need to send it as form Data
let formData = new FormData()
console.log(values);
await formData.append('profile_picture',values.profile_picture.rawFile,values.profile_picture)
await fetch('http://localhost:1337/api/moderators',{
body:formData,
method:'POST',
credentials:'include' //If Using Session for react-app instead of JWT token
})