I am trying to create an app where users can upload files to the "wall". The idea is not important. Each file is added both to storage and to database as [fileName : url) under user's profile. Because of limitations regarding strings in database, I keep filenames without extensions (can't use dot symbol). Uploading works fine but I have a problem with deletion. Based on the file user picks to delete in app I get the filename(without extension). My delete function should delete the file from storage but it cannot locate file because of missing extension.
I use this function to upload the data:
let uploadTask = ref.putData(contents as Data, metadata: myMetadata, completion: { (metadata, error) in ...}
and the file itself:
let contents = try Data(contentsOf: url.standardizedFileURL)
Is it possible to upload "contents" without the extension? Or maybe is there another approach I'm missing?
Thanks for tips.
It's not a really good idea to store the names of random things (including files) as keys in Realtime Database. You could make this easier on yourself by pushing an object at a location in the database for each file uploaded. The pushed object could contain the name and location of the file in Storage, along with any other metadata about that file, and you won't have to mangle that data as long as it's stored in values rather than keys.
Related
For example, my podcast app has a list of all downloaded podcast, how do I get a list of all LockCachingAudioSource that has been downloaded using request() method?
When you create your LockCachingAudioSource instances, you can choose the location where you want them to be saved. If you create a directory for that purpose, you can obtain a directory listing using Dart's file I/O API. The directory listing will also show partially downloaded files and other temporary files, which you want to ignore. These have extensions .mime and .part.
Having explained that, here is a solution. First, create your cache directory during app init:
final cacheDir = File('/your/choice/of/location');
...
await cacheDir.create(recursive: true);
Then for each audio source, create it like this:
import 'package:path/path.dart' as p;
...
source = LockCachingAudioSource(
uri,
cacheFile: File(p.joinAll([cacheDir, 'yourChoiceOfName.mp3'],
);
Now you can get a list of downloaded files at any time by listing the cacheDir and ignoring any temporary files:
final downloadedFiles = (await _getCacheDir()).list().where((f) =>
!['mime', 'part'].contains(f.path.replaceAll(RegExp(r'^.*\.'), '')));
If you need to turn these files back into the original URI, you could either create your own database to store which file is for which URI, or you choose the file name of each of your cache files by encoding the URI in base64 or something that's reversable, so given a file name, you can then decode it back into the original URI.
I've tried three different ways to detect if a FileReference's original file is still existing (i.e. file has been deleted outside TYPO3 using SFTP or similar):
if($fileReference instanceof \TYPO3\CMS\Extbase\Domain\Model\FileReference) {
$isMissing = $fileReference->getOriginalResource()->getStorage()->getFile($fileReference->getOriginalResource()->getIdentifier())->isMissing();
$isMissing = $fileReference->getOriginalResource()->getOriginalFile()->isMissing();
$isMissing = $fileReference->getOriginalResource()->isMissing();
}
Only the first one give me the right isMissing() value.
The property isMissing is an database value, which is set if the storage detect an missing file. On getFile the storage check if the file is missing and set "isMissing" for the file. If you dont persist this to the database, the setting is get loose with the next call.
You can also call $isMissing = $fileReference->getOriginalResource()->getStorage()->hasFile($fileReference->getOriginalResource()->getIdentifier());
You can run the file indexer scheduler (TYPO3\CMS\Scheduler\Task\FileStorageIndexingTask) if you want to check frequently for deleted files. This should be required if you let change files externaly (like ftp).
So in google-cloud-storage if you upload more than one file with the same name to it the last will overwrite what was uploaded before it.
If I want to upload more than one file with the same name I should append some unique thing to the file name e.g. timestamp, random UUID.
But by doing so I'll lose the original file name while downloading, because I want to serve the file directly from google.
If we used the unique identifier as a folder instead of appending it to the file name, e.g. UUID +"/"+ fileName then we can download the file with its original name.
You could turn on Object Versioning which will keep the old versions of the object around.
Alternatively, you can set the Content Disposition header when uploading the object, which should preserve whatever filename you want on download.
instead of using object versioning, you can attach the UUID (or any other unique identifier) and then update the metadata of the object (specifically the content disposition), the following is a part of a python script i've used to remove the forward slashes - added by google cloud buckets when to represent directories - from multiple objects, it's based on this blog post, please keep in mind the double quotes around the content position "file name"
def update_blob_download_name(bucket_name):
""" update the download name of blobs and remove
the path.
:returns: None
:rtype: None
"""
# Storage client, not added to the code for brevity
client = initialize_google_storage_client()
bucket = client.bucket(bucket_name)
for blob in bucket.list_blobs():
if "/" in blob.name:
remove_path = blob.name[blob.name.rfind("/") + 1:] # rfind gives that last occurence of the char
ext = pathlib.Path(remove_path).suffix
remove_id = remove_path[:remove_path.rfind("_id_")]
new_name = remove_id + ext
blob.content_disposition = f'attachment; filename="{new_name}"'
blob.patch()
I use FilePicker (now called FileStack) and I wanted to know if it's possible to prevent duplicate file uploads to a single container. For example, if I allow users to up upload some music files, how do I prevent them from adding the same music file twice in the same upload instance? The starter code is below:
filepicker.pickAndStore(
{
mimetype:"image/*",
multiple: true
},
{
location:"S3"
},
function(Blobs){
console.log(JSON.stringify(Blobs));
}
);
To only reliable way for finding duplicates would be compare files MD5 hashes.
MD5 hash is a unique identifier and it is available via filepicker API.
Example call:
https://www.filepicker.io/api/file/hFHUCB3iTxyMzseuWOgG/metadata?md5=true
Response:
{"md5": "f31dbf9b885e315d98e136f1db0daf52"}
To be even more reliable you can also compare file size.
So what I would recommend is storing md5 hash and files size together with file links in the database.
'pickAndStore' method allows me to specify full path to the file, but I don't know it's extension at this point (file path has to be defined before file is uploaded, so it's not possible to provide a path with correct extension).
if I use 'pick' and then 'store' I have 2 files (because both methods uploads file to the s3). I can delete 'old' file, but it's not optimal and can be pain (take ages) with really big files.
Is there any better solution? Ideally to rename existing file.
Currently, there is no workaround for renaming file.
However, in our Javascript API v2 we are planing to add new callback function. onStart callback will be fired after user pick file but before file uploading. There could be option like renaming file based on original filename.
We will keep you updated.