Google Cloud Storage serves up old image when file names are the same - google-cloud-storage

Running into an issue with hosted images in Google Cloud Storage. When an image gets uploaded to a destination and has the same name as a file there it replaces the file, however Google continues to serve up the old image at the URL.
If I go into the bucket I can see the preview of the new image, however, clicking the link will open a window and show the old image.
How do I fix this so it shows the updated image?

So I had to set the max-age to 0 when uploading the image.
const options = {
expires: Date.now() + 1 * 60 * 1000, // 1 minute,
fields: {
'x-goog-meta-test': 'data',
'Cache-Control': 'max-age=0', <----------- this is what I needed
},
};

Related

Placing large image in Word Online not working

I am trying to place a large jpg image (size 17.15 MB) in Word Online. The code appears that the image was placed successfully, but it doesn't place the image in the document and there are no error messages. It is working properly in Word Desktop.
let options = {coercionType: Office.CoercionType.Image};
Office.context.document.setSelectedDataAsync(base64, options, function(obj){
console.log('obj', obj)
context.sync().then(function(){
console.log('complete')
}, function(err){
if(err){console.log('failed')}
});
});
obj is returning:
status: "succeeded"
value: {}
The code places properly for a smaller image and on Desktop. Is there another way to check if the image placed properly or an error message?

Protractor file upload with Saucelabs (Chrome 90) redirected to "data:text/html,<html></html>" after upload, works locally but not on Saucelabs

I am using protractor for a Node.js application with Chrome, in one test case I have to upload image and then click add/update button. The test case is working perfectly on my machine, but on Saucelabs it cant interact with the button after the image is uploaded. If I don't upload image its able to find button and click on it but when using with image upload it doesn't.
I have printed the current URL after image upload and it shows
"data:text/html,", whereas in the Saucelabs video I can see previous page.
Approaches I have tried
Tried to print all available handles and it only prints 1 handle i.e.
"handles are [chrome 90.0 Windows 10 #01-1]
CDwindow-B29A329D03022E36745F1A844177FAA8"
Print Current URL "data:text/html,"
Code
const remote = require('selenium-webdriver/remote');
browser.driver.setFileDetector(new remote.FileDetector());
// absolute path resolution
const path = require('path');
const fileToUpload = './Test_Images/' + imageName;
const absolutePath = path.resolve(__dirname, fileToUpload);
console.log('path is ', absolutePath);
// Find the file input element
const fileElem = element(by.css('input[type="file"]'));
//upload image
await browser.sleep(2000);
fileElem.sendKeys(absolutePath);
console.log('\n Image Uploaded for report');
//print current url after image upload
const currentUrl = (await browser.getCurrentUrl()).toString();
console.log('\n Current URL', currentUrl);
await browser.sleep(2000);
//get all handles
browser.getAllWindowHandles().then(function (handles) {
for (const handle of handles) {
console.log('\n\nhandles are\n', handle.toString());
}});
//click on the submit button
await adminReportsPage.createReportButton.click();
console.log('Report Submit button Clicked ');
The reason why this fails is because a remote machine in the cloud, as with Sauce Labs, does not has access to your local file system and your local machine has.
I'm not 100% sure if Protractor has a workaround for this, but with a framework like for example WebdriverIO, you have the option to use a method like uploadFile. That method will only work on Chrome and will translate the file to a base64 string so you can upload it form your local machine to the remove machine.
A working example can be found here
Last but not least, keep in the back of your mind that Protractor is deprecated and with the upcoming release of Selenium 4 you can't use it anymore because it doesn't support W3C. Also check the Protractor repository for more information.
#wswebcreation I have found a solution, it worked by changing the
const fileElem = element(by.css('input[type="file"]'));
to
const fileElem = browser.driver.findElement(by.css('input[type="file"]'));

Azure Data Lake HDFS upload file size limit

Does anyone know what is maximum size to upload file via Azure HDFS Rest API? (https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-rest-api).
I found someplace 256MB, some place 32MB, so wondering.
Or similar limits for other SDKs?
I was wrestling with the same problem some months ago and it turned out that the IIS which is in front of ADLS is setting the maxAllowedContentLength with default value of 30000000 bytes (or 28.6Mb). This essentially means that whenever we want to push anything bigger that 30Mb, that request never reaches ADL as IIS throws 404.13 before that. Reference.
As already suggested in the links, ADLS has a driver with a 4-MB buffer, I'm using the .NET SDK myself and following code has served me well
public async Task AddFile(byte[] content, string path)
{
const int fourMb = 4 * 1024 * 1024;
var buffer = new byte[fourMb];
using (var stream = new MemoryStream(content))
{
if (!_adlsFileSystemClient.FileSystem.PathExists(_account, path))
{
_adlsFileSystemClient.FileSystem.Create(_account, path);
}
int bytesToRead;
while ((bytesToRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
if (bytesToRead < fourMb)
{
Array.Resize(ref buffer, bytesToRead);
}
using (var s = new MemoryStream(buffer))
{
await _adlsFileSystemClient.FileSystem.AppendAsync(_account, path, s);
}
//skipped for brevity
In my tests, I am finding a maximum file size limit somewhere between 28MB and 30MB.
Using the Azure Data Lake Storage REST API, I have had no issues creating files as large as 28MB. However, when I try to create a file that is 30MB, I receive a 404 Not Found error.
The following references align with the file size limit and 404 error I am observing. The references are about the SDK, but it could be that the SDK is also calling the REST API under the covers. My tests are calling the REST API directly.
NotFound error on call to Data Lake Store Create
https://stackoverflow.com/a/41469724/10363

Page Blob download with Azure java SDK does not download the complete Blob

Trying to download a page blob of size 2 GB using java sdk and it fails with Storage exception because the file size downloaded does not match the actual file size
On multiple tries the same result is seen, although there is slight change in the downloaded file size. Setting timeout to maximum value also does not help.
Also, when I download the same vhd using the Azure portal, I see that the download completes but only partially. It is usually comparable to the one downloaded with SDK.
In the SDK code, I can see HTTpUrlConnection is being used. Could that be a problem ? The same code on a Windows machine has similar results but only the downloaded file is few more MB's in size but not complete.
Any thoughts on how to get it working ?
The code snippet used is
URI blobEndpoint = null;
String uriString = "http://" + "sorageaccount" + ".blob.core.windows.net";
blobEndpoint = new URI(uriString);
CloudBlobClient blobClient = new CloudBlobClient(blobEndpoint,
new StorageCredentialsAccountAndKey("abcd", "passed"));
CloudBlobContainer container = blobClient.getContainerReference(Constants.STORAGE_CONTAINER_NAME);
CloudPageBlob pageBlob = container.getPageBlobReference("http://abcd.blob.core.windows.net/sc/someimg.vhd");
System.out.println("Page Blob Name: " + pageBlob.getName());
OutputStream outStream = new FileOutputStream(new File("/Users/myself/Downloads/TestDownload.vhd"));
System.out.println("Starting download now ... ");
BlobRequestOptions options = new BlobRequestOptions();
options.setUseTransactionalContentMD5(true);
options.setStoreBlobContentMD5 (true); // Set full blob level MD5
options.setTimeoutIntervalInMs(Integer.MAX_VALUE);
options.setRetryPolicyFactory(new RetryLinearRetry());
pageBlob.download(outStream, null, options, null);
outStream.close();

How to delete a file while the application is running

I have developed a C# application, in the application the users choose a photo for each record. However the user should also be able to change the pre-selected photo with a newer one. When the user changes the photo the application first deletes the old photo from the application directory then copies the new photo, but when it does that the application gives an exception because the file is used by the application so it cannot be deleted while the application is running. Does any one have a clue how to sort this out? I appreciate your help
This is the exception
The process cannot access the file
'D:\My
Projects\Hawkar'sProject\Software\Application\bin\Debug\Photos\John
Smith.png' because it is being used by
another process.
//defining a string where contains the file source path
string fileSource = Open.FileName;
//defining a string where it contains the file name
string fileName = personNameTextBox.Text + ".png" ;
//defining a string which specifies the directory of the destination file
string fileDest = dir + #"\Photos\" + fileName;
if (File.Exists(fileDest))
{
File.Delete(fileDest);
//this is a picturebox for showing the images
pbxPersonal.Image = Image.FromFile(dir + #"\Photos\" + "No Image.gif");
File.Copy(fileSource, fileDest);
}
else
{
File.Copy(fileSource, fileDest);
}
imageIDTextBox.Text = fileDest;
First of all, you code is not good.
The new image is only copied if there is currently no image (else).
But if there is an old image, you only delete this image, but never copy the newer one (if).
The code should better look like this:
if (File.Exists(fileDest))
{
File.Delete(fileDest);
}
File.Copy(fileSource, fileDest);
imageIDTextBox.Text = fileDest;
This code should work, but if you are getting an exception that the file is already in use, you should check "where" you use the file. Perhaps you are reading the file at program start. Check all parts of your program you are accessing these user files if there are some handles open.
Thanks a lot for your help and sorry for the mistake in the code I just saw it, my original code is just like as you have written but I don't know maybe when I posted accidentally I've put like this. when the application runs there is a picturebox which the image of each record is shown, that is why the application is giving an exception when I want to change the picture because it has been used once by the picturebox , however I've also tried to load another picture to the picture box before deleting the original one but still the same. I've modified the above could if you want to examine it