Dropzone file not fully processed when calling processFile(File) - mongodb

I'm creating a Dropzone programmatically in my Meteor application and instead of posting Files to a url I'm storing them inside my MongoDB with CollectionFS / GridFS and call processFile(File) for each one by iterating over the getQueuedFiles array, like so:
dz.getQueuedFiles().forEach(function(element) {
Images.insert(element, function(err, fileObj){
if (err){
alert("Erreur!");
} else {
var imageId = fileObj._id;
arrayOfImageIds.push(imageId);
dz.processFile(element);
}
})
All goes well until the File sizes are bigger. Then, somehow the progress bar gets stuck halfway and the processFile is never finished (and the queue not emptied). Even when I just isolate the dz.processFile(element) and comment out the DB writing the file hangs in the client.
This is my Dropzone setup:
Template.logoUpload.onRendered(function (){
Dropzone.autoDiscover = false;
dz = new Dropzone("form#dropzone", {
dictDefaultMessage : "DROP YOUR LOGO HERE",
autoProcessQueue: false,
addRemoveLinks: true,
dictRemoveFile: "Delete",
maxFiles: 2,
maxFilesize: 5,
maxThumbnailFilesize: 5,
accept: function(file, done){
done();
},
init: function() {
this.on("addedfile", function(file) {
Session.set("files", this.files.length);
if (this.getAcceptedFiles().length > 1) {
alert("max 2 files allowed");
this.removeFile(file);
}
}),
this.on("removedfile", function(file) {
Session.set("files", this.files.length);
}),
this.on("queuecomplete", function(file, response){
console.log("SUCCESFULLY UPLOADED");
console.log(arrayOfImageIds);
Session.set("imageIds", arrayOfImageIds);
})
}
});
});
Anyone have an idea how to solve this?

Have you tried increasing the file chunk size?
var imageStore = new FS.Store.GridFS("images", {
chunkSize: 1024*1024 // optional, default GridFS chunk size in bytes (can be overridden per file).
// Default: 2MB. Reasonable range: 512KB - 4MB
});
Images = new FS.Collection("images", {
stores: [imageStore]
});

Related

How could I load video files from my library? Ionic 3

I observed that the Native File has not been supported by the Ionic View anymore see list here.
I am trying to get a video from my library by using Native Camera to access the videos. It can return me 3 different formats of path to my videos (DATA_URL, FILE_URI, and NATIVE_URI).reference to Native Camera here
I am currently using FILE_URI as recommended in this post. It returns something like "/storage/emulated/0/DCIM/Camera/VID_20180312_210545.mp4"
Please have a look at my code below. Aiming a better understanding, the current behavior is highlighted by comments with "//** comment ***" :
addVideoToOffer(){
this.platform.ready().then(() =>{
const options: CameraOptions = {
sourceType: this.camera.PictureSourceType.PHOTOLIBRARY,
destinationType: this.camera.DestinationType.FILE_URI,
mediaType: this.camera.MediaType.VIDEO,
}
this.camera.getPicture(options).then((data_uri) => {
this.readVideoFileasGeneral(data_uri);
});
});
}
readVideoFileasGeneral(data_uri) {
if(!data_uri.includes('file://')) {
data_uri = 'file://' + data_uri;
}
return this.file.resolveLocalFilesystemUrl(data_uri)
.then((entry: FileEntry) => {
//***it does not get in here***
this.presentQuickToastMessage(data_uri);
return new Promise((resolve)=>{//, reject) => {
entry.file((file) => {
let fileReader = new FileReader();
fileReader.onloadend = () => {
let blob = new Blob([fileReader.result], {type: file.type});
resolve({blob: blob, file: file});
};
fileReader.readAsArrayBuffer(file);
});
})
})
.catch((error) => {
this.presentQuickToastMessage(error);
//***it presents "plugin_not_installed" here***
});
}
I understand that I am having this message because Native File is not supported anymore (maybe reason of the plugin_not_installed message). However, I still have to do this task. So, if someone has any idea of what I could be using in order to have the selected videos in a blob, it would be great!
Thanks for reading until here,
Cheers,
Roger A L
makeFileIntoBlob(uri) {
// get the correct path for resolve device file system
let pathIndex = uri.indexOf('var');
let correctPath = uri.slice(+pathIndex);
this.file
.resolveLocalFilesystemUrl((this.platform.is('ios') ? 'file:///' : '') + correctPath)
.then(entry => (<FileEntry>entry).file(file => this.readFile(file)))
.catch(err => console.log('ERROR: ', err));
}
readFile(file) {
if(file) {
const reader = new FileReader();
reader.onloadend = () => {
const blob: any = new Blob([reader.result], { type: file.type });
blob.name = file.name;
console.log(blob);
return blob;
};
reader.readAsArrayBuffer(file);
}
}
You need to get rid of the /private/ and keep file:///, so that your path goes like file:///var/
I'm currently working on something similar.. I have the video recorded with media-capture and then I can display it within a normal video html tag.. if this is all you need then this code may help you...
this.mediaCapture.captureVideo({duration: 10, quality: 0}).then(
(data: MediaFile[]) => {
if (data.length > 0) {
let originname = data[0].fullPath.substr(data[0].fullPath.lastIndexOf('/') + 1);
let originpath = data[0].fullPath.substr(0, data[0].fullPath.lastIndexOf('/') + 1);
let alerta = this.alerts.create({
buttons: ['ok'],
message: this.file.externalDataDirectory
});
alerta.then(set => set.present());
this.file.copyFile(originpath, originname, this.file.externalDataDirectory, 'video.mp4')
.then(result =>{
let videopath = this.webview.convertFileSrc(result.nativeURL)
let video = (document.getElementById('myvideo') as HTMLVideoElement).src = videopath;
.... rest of the code
The problem raise when you try to use the native File plugin... converting files with any method (readAsDataURL, readAsArrayBuffer or readAsBinaryString) will never resolve, this is a known problem with the Ionic Native File plugin but is not taken care of...
What I did is to take the ionic native Filesystem and use it to read the file, this does read the file and get you with a base64 (pretty sure as I don't specify the encoding field) and then you can handle it the way you want...
const data = Filesystem.readFile({
path: result.nativeURL
})
.then(data =>{
...handle data as base64
...rest of the code

Mapbox GL JS: Style is not done loading

I have a map wher we can classically switch from one style to another, streets to satellite for example.
I want to be informed that the style is loaded to then add a layer.
According to the doc, I tried to wait that the style being loaded to add a layer based on a GEOJson dataset.
That works perfectly when the page is loaded which fires map.on('load') but I get an error when I just change the style, so when adding layer from map.on('styledataloading'), and I even get memory problems in Firefox.
My code is:
mapboxgl.accessToken = 'pk.token';
var map = new mapboxgl.Map({
container: 'map',
style: 'mapbox://styles/mapbox/streets-v10',
center: [5,45.5],
zoom: 7
});
map.on('load', function () {
loadRegionMask();
});
map.on('styledataloading', function (styledata) {
if (map.isStyleLoaded()) {
loadRegionMask();
}
});
$('#typeMap').on('click', function switchLayer(layer) {
var layerId = layer.target.control.id;
switch (layerId) {
case 'streets':
map.setStyle('mapbox://styles/mapbox/' + layerId + '-v10');
break;
case 'satellite':
map.setStyle('mapbox://styles/mapbox/satellite-streets-v9');
break;
}
});
function loadJSON(callback) {
var xobj = new XMLHttpRequest();
xobj.overrideMimeType("application/json");
xobj.open('GET', 'regions.json', true);
xobj.onreadystatechange = function () {
if (xobj.readyState == 4 && xobj.status == "200") {
callback(xobj.responseText);
}
};
xobj.send(null);
}
function loadRegionMask() {
loadJSON(function(response) {
var geoPoints_JSON = JSON.parse(response);
map.addSource("region-boundaries", {
'type': 'geojson',
'data': geoPoints_JSON,
});
map.addLayer({
'id': 'region-fill',
'type': 'fill',
'source': "region-boundaries",
'layout': {},
'paint': {
'fill-color': '#C4633F',
'fill-opacity': 0.5
},
"filter": ["==", "$type", "Polygon"]
});
});
}
And the error is:
Uncaught Error: Style is not done loading
at t._checkLoaded (mapbox-gl.js:308)
at t.addSource (mapbox-gl.js:308)
at e.addSource (mapbox-gl.js:390)
at map.js:92 (map.addSource("region-boundaries",...)
at XMLHttpRequest.xobj.onreadystatechange (map.js:63)
Why do I get this error whereas I call loadRegionMask() after testing that the style is loaded?
1. Listen styledata event to solve your problem
You may need to listen styledata event in your project, since this is the only standard event mentioned in mapbox-gl-js documents, see https://docs.mapbox.com/mapbox-gl-js/api/#map.event:styledata.
You can use it in this way:
map.on('styledata', function() {
addLayer();
});
2. Reasons why you shouldn't use other methods mentioned above
setTimeout may work but is not a recommend way to solve the problem, and you would got unexpected result if your render work is heavy;
style.load is a private event in mapbox, as discussed in issue https://github.com/mapbox/mapbox-gl-js/issues/7579, so we shouldn't listen to it apparently;
.isStyleLoaded() works but can't be called all the time until style is full loaded, you need a listener rather than a judgement method;
Ok, this mapbox issue sucks, but I have a solution
myMap.on('styledata', () => {
const waiting = () => {
if (!myMap.isStyleLoaded()) {
setTimeout(waiting, 200);
} else {
loadMyLayers();
}
};
waiting();
});
I mix both solutions.
I was facing a similar issue and ended up with this solution:
I created a small function that would check if the style was done loading:
// Check if the Mapbox-GL style is loaded.
function checkIfMapboxStyleIsLoaded() {
if (map.isStyleLoaded()) {
return true; // When it is safe to manipulate layers
} else {
return false; // When it is not safe to manipulate layers
}
}
Then whenever I swap or otherwise modify layers in the app I use the function like this:
function swapLayer() {
var check = checkIfMapboxStyleIsLoaded();
if (!check) {
// It's not safe to manipulate layers yet, so wait 200ms and then check again
setTimeout(function() {
swapLayer();
}, 200);
return;
}
// Whew, now it's safe to manipulate layers!
the rest of the swapLayer logic goes here...
}
Use the style.load event. It will trigger once each time a new style loads.
map.on('style.load', function() {
addLayer();
});
My working example:
when I change style
map.setStyle()
I get error Uncaught Error: Style is not done loading
This solved my problem
Do not use map.on("load", loadTiles);
instead use
map.on('styledata', function() {
addLayer();
});
when you change style, map.setStyle(), you must wait for setStyle() finished, then to add other layers.
so far map.setStyle('xxx', callback) Does not allowed. To wait until callback, work around is use map.on("styledata"
map.on("load" not work, if you change map.setStyle(). you will get error: Uncaught Error: Style is not done loading
The current style event structure is broken (at least as of Mapbox GL v1.3.0). If you check map.isStyleLoaded() in the styledata event handler, it always resolves to false:
map.on('styledata', function (e) {
if (map.isStyleLoaded()){
// This never happens...
}
}
My solution is to create a new event called "style_finally_loaded" that gets fired only once, and only when the style has actually loaded:
var checking_style_status = false;
map.on('styledata', function (e) {
if (checking_style_status){
// If already checking style status, bail out
// (important because styledata event may fire multiple times)
return;
} else {
checking_style_status = true;
check_style_status();
}
});
function check_style_status() {
if (map.isStyleLoaded()) {
checking_style_status = false;
map._container.trigger('map_style_finally_loaded');
} else {
// If not yet loaded, repeat check after delay:
setTimeout(function() {check_style_status();}, 200);
return;
}
}
I had the same problem, when adding real estate markers to the map. For the first time addding the markers I wait till the map turns idle. After it was added once I save this in realEstateWasInitialLoaded and just add it afterwards without any waiting. But make sure to reset realEstateWasInitialLoaded to false when changing the base map or something similar.
checkIfRealEstateLayerCanBeAddedAndAdd() {
/* The map must exist and real estates must be ready */
if (this.map && this.realEstates) {
this.map.once('idle', () => {
if (!this.realEstateWasInitialLoaded) {
this.addRealEstatesLayer();
this.realEstateWasInitialLoaded = true
}
})
if(this.realEstateWasInitialLoaded) {
this.addRealEstatesLayer();
}
}
},
I ended up with :
map.once("idle", ()=>{ ... some function here});
In case you have a bunch of stuff you want to do , i would do something like this =>
add them to an array which looks like [{func: function, param: params}], then you have another function which does this:
executeActions(actions) {
actions.forEach((action) => {
action.func(action.params);
});
And at the end you have
this.map.once("idle", () => {
this.executeActions(actionsArray);
});
I have created simple solution. Give 1 second for mapbox to load the style after you set the style and you can draw the layer
map.setStyle(styleUrl);
setTimeout(function(){
reDrawMapSourceAndLayer(); /// your function layer
}, 1000);
when you use map.on('styledataloading') it will trigger couple of time when you changes the style
map.on('styledataloading', () => {
const waiting = () => {
if (!myMap.isStyleLoaded()) {
setTimeout(waiting, 200);
} else {
loadMyLayers();
}
};
waiting();
});

Store image to Firebase storage from cordova camera plugins on Ionic

I've red some topics on the subject (e.g: Uploading image to Firebase Storage from Cordova app) but didn't find my answer...
I'm working on a IONIC project with the implementation of the ngCordova camera plugin to take picture and get pic from the librairy.
So I got the result as a image URI and I want to upload it in Firebase storage (as file or Blob).
Here is my code :
$scope.fromCamera = function() {
$ionicPlatform.ready(function() {
var options = {
quality: 75,
destinationType: Camera.DestinationType.FILE_URI,
sourceType: Camera.PictureSourceType.CAMERA,
allowEdit: true,
encodingType: Camera.EncodingType.JPEG,
targetWidth: 300,
targetHeight: 300,
saveToPhotoAlbum: true e
};
$cordovaCamera.getPicture(options).then(function(imageURI) {
window.resolveLocalFileSystemURL(imageURI, function (fileEntry) {
fileEntry.file(function (file) {
var reader = new FileReader();
reader.onloadend = function () {
// This blob object can be saved to firebase
var blob = new Blob([new Uint8Array(this.result)], { type: "image/jpeg" });
// Create the storage ref
var ref = storageRef.child('images/test');
// Upload the file
uploadPhoto(blob, ref);
};
reader.readAsArrayBuffer(file);
});
}, function (error) {
console.log(error)
});
});
});
};
I read the file and convert it into a Blob before uploading it into firebase. And I got an 'Encoding error' telling me "A URI supplied to the API was malformed, or the resulting Data URL has exceeded the URL length limitations for Data URLs."
I'm running it an chrome browser with the Cordova Mocks extension.
Any help is welcome!
Thanks
uploadPhoto() is my function to upload the file on firebase storage (and save the URL in firebase database)
var storageRef = Firebase.storageRef();
var databaseRef = Firebase.databaseRef();
var uploadPhoto = function(file, ref) {
var task = ref.put(file);
// Update progress bar
task.on('state_changed', function(snapshot){
// nothing
}, function(error) {
// Handle unsuccessful uploads
}, function() {
// Handle successful uploads on complete
$scope.downloadURL = task.snapshot.downloadURL;
$scope.actualKey = databaseRef.child('posts').push().key;
databaseRef.child('posts/' + $scope.actualKey).update({
url : $scope.downloadURL,
id : $scope.actualKey,
time : firebase.database.ServerValue.TIMESTAMP,
});
}
);
}
try changing...
[new Uint8Array(this.result)]
to just this
[this.result]
alternate approach using $cordovaFile
var fileName = imageURI.replace(/^.*[\\\/]/, '');
$cordovaFile.readAsArrayBuffer(cordova.file.tempDirectory, fileName)
.then(function (success) {
// success - get blob data
var imageBlob = new Blob([success], { type: "image/jpeg" });
// Create the storage ref
var ref = storageRef.child('images/test');
// Upload the file
uploadPhoto(imageBlob, ref);
}, function (error) {
// error
});
Instead of getting the path from the URI, in the code, I assume the following...
// modify the image path when on Android
if ($ionicPlatform.is("android")) {
path = cordova.file.cacheDirectory
} else {
path = cordova.file.tempDirectory
}
feel free to parse the path to get the directory

Chrome App FileReader

I'm trying to make use of the file system API in a Chrome App. I've tried all the sample code I can find and can't get a simple text file to read. I'm logging almost every step, and what seems to happen (or not happen) is everything stops the first time I reference a file reader object. It creates just fine, because I can log the .readyState, but after that I can't seem to even set an onload()event or execute a .readAsText().
Here's what I'm calling from a button:
function clickButton(){
chrome.fileSystem.chooseEntry({type: 'openFile', acceptsMultiple: false}, function(FileEntry){
if(chrome.runtime.lastError) {console.warn("Warning: " + chrome.runtime.lastError.message);}
else{
console.log(FileEntry);
var thing = new FileReader();
console.log(thing.readyState);
thing.onloadstart(function(){
console.log("Started loading " & FileEntry);
});
console.log("added onloadstart");
console.log(thing.readyState);
console.log(thing);
thing.readAsText(FileEntry);
console.log(thing.readyState);
console.log(thing.result);
}
});
document.getElementById("status").innerHTML = "I did something";
}
I did read somewhere that Chrome doesn't allow access to local files, but the chrome apps seem to be different. At least, the documentation seems to suggest that.
The only thing I end up with in my console is the FileEntry object.
https://developer.chrome.com/apps/app_storage#filesystem
I've used the example code right from the above link and still can't get it right. Anyone else have this issue or know what I'm doing wrong?
There is a difference between a FileEntry and a File. You need to call FileEntry's .file() method. So, replace
thing.readAsText(FileEntry);
with
FileEntry.file(function(File) {
thing.readAsText(File)
})
https://developer.mozilla.org/en-US/docs/Web/API/FileEntry#File
Try this code...
<!doctype html>
<html>
<script>
function handle_files(files) {
for (i = 0; i < files.length; i++) {
file = files[i]
console.log(file)
var reader = new FileReader()
ret = []
reader.onload = function(e) {
console.log(e.target.result)
}
reader.onerror = function(stuff) {
console.log("error", stuff)
console.log (stuff.getMessage())
}
reader.readAsText(file) //readAsdataURL
}
}
</script>
<body>
FileReader that works!
<input type="file" multiple onchange="handle_files(this.files)">
</body>
</html>
I've written a function to extract text from a file.
function getFileEntryText(fileEntry) {
return new Promise(function (resolve, reject) {
fileEntry.file(function (file) {
var fileReader = new FileReader();
fileReader.onload = function (text) {
resolve(fileReader.result);
};
fileReader.onerror = function () {
reject(fileReader.error);
};
fileReader.readAsText(file);
});
});
}
You can invoke this method like so:
getFileEntryText(fileEntry).then(function(text) {
// Process the file text here
}, function(error) {
// Handle the file error here
});
One thing I'm grappling with when working with the FileSystem is that every call is asynchronous. Having multiple levels of nested callbacks can make for code that's hard to read. I'm currently working around this by converting everything I can to a Promise.
for anyone who is interested, here's my final (working) code, complete with all the console.log()'s I needed to follow all those callbacks.
var chosenEntry = null;
function clickButton(){
console.log("Button clicked");
var accepts = [{
mimeTypes: ['text/*'],
extensions: ['js', 'css', 'txt', 'html', 'xml', 'tsv', 'csv', 'rtf']
}];
chrome.fileSystem.chooseEntry({type: 'openFile', accepts: accepts}, function(theEntry) {
if (!theEntry) {
output.textContent = 'No file selected.';
return;
}
// use local storage to retain access to this file
chrome.storage.local.set({'chosenFile': chrome.fileSystem.retainEntry(theEntry)});
console.log("local data set. calling loadFileEntry");
loadFileEntry(theEntry);
console.log("loadFileEntry called, returned to clickButton()");
});
}
function loadFileEntry(_chosenEntry) {
console.log("entered loadFileEntry()");
chosenEntry = _chosenEntry;
chosenEntry.file(function(file) {
readAsText(chosenEntry, function(result) {
console.log("running callback in readAsText");
document.getElementById('text').innerHTML = result;
console.log("I just tried to update textarea.innerHTML");
});
});
console.log("added function to chosenEntry.file()");
}
function readAsText(fileEntry, callback) {
console.log("readAsText called");
fileEntry.file(function(file) {
var reader = new FileReader();
console.log("Created reader as FileReader");
reader.onload = function(e) {
console.log("called reader.onload function");
callback(e.target.result);
};
console.log("calling reader.readAsText");
reader.readAsText(file);
});
}

Select multiple files from a FileUploadField in extjs

I'm looking at the example here and using the javascript provided from that example here. Basically what I want is a stand alone file chooser where I can select as many files as I want. The example that I've tried has a stand alone upload button but they don't let me shift highlight multiple files at the same time.
This code creates the button, but I can't load in multiple files at the same time:
var addFilesButton = new Ext.ux.form.FileUploadField({
buttonText: 'Add Files...',
buttonOnly: true,
listeners: {
'fileselected': function(fb, v){
var Record = myGrid.getStore().recordType;
var newFile = new Record({
fileName: v,
type: 'src',
version: '5.9',
});
myGrid.stopEditing();
myGrid.getStore().add(newFile);
myGrid.startEditing(0, 0);
}
}
});
Ext.ux.form.FileUpload uses HTML INPUT field to set a file to upload which is pretty normal thing to do.
If you are using HTML4, at most one file can be assigned to input file field.
However, from HTML5, there is a special attribute that you can set to accept multiple files.
I have modified the script accordingly and created a demo.
Note that HTML5 spec is still in draft. Feature compatibility table is available on caniuse.com
My MultiFileUploadField class:
MultiFileUploadField = Ext.extend(Ext.ux.form.FileUploadField, {
multiple: false,
createFileInput: function() {
this.fileInput = this.wrap.createChild({
id: this.getFileInputId(),
name: this.name||this.getId(),
cls: 'x-form-file',
tag: 'input',
type: 'file',
size: 1
});
if(this.multiple){
this.fileInput.dom.setAttribute('multiple', 'multiple');
}
},
bindListeners: function(){
this.fileInput.on({
scope: this,
mouseenter: function(){
this.button.addClass(['x-btn-over','x-btn-focus'])
},
mouseleave: function(){
this.button.removeClass(['x-btn-over','x-btn-focus','x-btn-click'])
},
mousedown: function(){
this.button.addClass('x-btn-click')
},
mouseup: function(){
this.button.removeClass(['x-btn-over','x-btn-focus','x-btn-click'])
},
change: function(){
var v = this.fileInput.dom.files;
this.setValue(v);
this.fireEvent('fileselected', this, v);
}
});
},
});