I am trying to upload to Google Cloud Storage using the AWS SDK For Java 1.x. I have enabled interoperability mode and put AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the environment. I can list objects, but uploading is not working for me.
The code I am using is
AmazonS3 client =
AmazonS3ClientBuilder.standard()
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(
"http://storage.googleapis.com", "auto"))
.build();
TransferManager tm = TransferManagerBuilder.standard()
.withS3Client(client)
.build();
Upload upload = tm.upload(bucketName, key, new File(filename));
upload.waitForCompletion();
But the result I get is:
com.amazonaws.services.s3.model.AmazonS3Exception: The MD5 you specified in Content-MD5 or x-goog-hash did not match what we computed. (Service: Amazon S3; Status Code: 400; Error Code: BadDigest; Request ID: null; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1819)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1403)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1372)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384)
The same code works OK with S3 (minus the EndpointConfiguration)
The GCS docs say:
In the Cloud Storage XML API, chunked transfer encoding and V4 signatures cannot currently be used simultaneously. Some Amazon S3 tools use chunked transfer encoding along with signatures by default; you should disable chunked transfer encoding in such cases.
I found two ways to resolve this error
You can .withChunkedEncodingDisabled(true) when building the client:
AmazonS3 client =
AmazonS3ClientBuilder.standard()
.withChunkedEncodingDisabled(true) // <<<<<<<
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(
"http://storage.googleapis.com", "auto"))
.build();
You can switch to the https endpoint:
AmazonS3 client =
AmazonS3ClientBuilder.standard()
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(
"https://storage.googleapis.com", "auto"))
.build();
Related
I am new to Scala. I am trying to access REST API URl and trying to get json data from there using Scalaj-Http with Spark framework in local vm(Intellij). But with the following Code I am always getting Http error code 401 from code and the server log is responding with "new ssl session,TLS V1.2 No Client Cert.
The jks file that I am using seems ok with proper SSL Handshake and its installed on server side.
val url = "https://abcdef:1234/api/v1/get?q=abc"
val alias ="xxxxxx-1234 yyyyy"
val sslFactory = SSLFactory.builder()
.withIdentityMaterial("abc.jks","pass".tocharArray)
.withTrustMaterial("abc.jks","pass".tocharArray)
.withClientIdentityRoute(alias,url)
.build()
val optn = HttpOptions.sslSocketFactory(sslfactory.getSslSocketfactory)
val res = Http(url) //Here getting 401 res.code
.option(optn)
.option(HttpOptions.allowUnsafeURL)
.asString
Tried everything but unable to solve. Kindly help please
I got the code working as I have to discard this option option(HttpOptions.allowUnsafe
URL)
Thanks
I am trying uploading a image using below code.
String _urlsegment =
Explika.producaoFlag ? 'https://www.remoteserver.pt' : 'http://10.0.2.2';
var stream = http.ByteStream(
DelegatingStream.typed(compressedFileImage.openRead()));
var length = await compressedFileImage.length();
var uri = Uri.parse('$_urlsegment/explika/api/upload');
var request = http.MultipartRequest("POST", uri);
var multipartFile = http.MultipartFile('fotoaluno', stream, length,
filename: '${Explika.getAluno().id}.jpg');
request.files.add(multipartFile);
Everything work fine using localhost. When i try send image to remote server nothing happens.
The end point is ok: i tested it using postman and all worked well.
Summing up:
uploading using APP to LOCAL server - OK
uploading to LOCAL server using POSTMAN - OK
uploading using APP to REMOTE server - Fail (No error occurs but file does not reach server)
uploading to REMOTE server using POSTMAN - OK
Any ideas on what is going on? Do I need to enter any special permissions in the app manifest?
Flutter sends the images with mime type 'application / octet-stream'. The server was waiting for files with mime type 'image / jpeg', so it automatically rejected the file.
I have .net core WebApi application setup with swagger. An endpoint that responses with RedirectResult is not handled by swagger ui and shown up as Undocumented: TypeError: Failed to fetch
The redirect result itself contains a SAS url to Azure Storage Blob file.
I've tried to call this endpoint with SoapUI and Postman and the actual file appeared in response with application/octet-stream content type
The endpoint is built in format
Task<IActionResult> DownloadDocument([FromRoute] Guid id)
{
...
return Redirect(blobSasUri)
}
There is support for that or even some workaround or I've missed something?
From client (eg: https://localhost:8080/) we are passing the certificate related values and calling the rest services ( hosted on different port - https://localhost:446/serviceName).
The issue is like, when we try to pass the certificate , SSL handshake is happening correctly (no error on debug) , but the certificate value is not passed to the service hosted on another port. Certificate value is accessed in server code by referring to (X509Certificate)httpReq.getAttribute("javax.servlet.request.X509Certificate");
Note : We use Spring boot application which intenally runs on tomcat server.And desired CA authorised certificates, keystore and truststore are present in resource path in both the projects (client and service hosted). In rest service project config file, the client-auth is set to false.
Sample code snippet used to call rest service:
SSLContext sslContext = SSLContexts.custom().loadTrustMaterial(restserviceTruststore)
.loadKeyMaterial(restserviceKeyStore, password).build();
HttpClient client = HttpClients.custom() .setHostnameVerifier(SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
.setSslcontext(sslContext).build();
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(client));
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_XML);
HttpEntity<String> request = new HttpEntity<>(XML, headers);
response = restTemplate.postForObject(endpointURL, request, String.class);
Question:
1) From client what keystore and trustore should we need to pass to SSLContext? Is it server's keystore /truststore or clients?
2)What are the exact steps to be followed to resolve this issue.
I have a working spring-mvc application with rest services and some rest-assured tests which are fine :
#Test
public void createFoobarFromScratchReturns201(){
expect().statusCode(201).given()
.queryParam("foo", generateFoo())
.queryParam("bar", generateBar())
.when().post("/foo/bar/");
}
=> OK
Then I implemented a digest authentication. Everything is working well, now I have to log in to use my services :
curl http://localhost:8089/foo/bar
=> HTTP ERROR 401, Full authentication is required to access this resource
curl http://localhost:8089/foo/bar --digest -u user_test:password
=> HTTP 201, CREATED
But when I try to upgrade my tests with the most obvious function, I still have a 401 error :
#Test
public void createFoobarFromScratchReturns201(){
expect().statusCode(201).given()
.auth().digest("user_test", "password") // Digest added here
.queryParam("foo", generateFoo())
.queryParam("bar", generateBar())
.when().post("/foo/bar/");
}
=> Expected status code <201> doesn't match actual status code <401>
I found some clues with the preemptive() function, but it seems to be only implemented for basic :
// Returns an AuthenticatedScheme and stores it into the general configuration
RestAssured.authentication = preemptive().basic("user_test", "password");
// Try a similar thing, but it didn't work :
RestAssured.authentication = RestAssured.digest("user_test", "password");
Currently, I am trying to achieve two things :
I need to upgrade a couple of my tests to support digest
I need to amend the #Before of the rest of my tests suites (whose are not related to auth issues), to be already logged in.
Any ideas or documentation ?
Try enabling support for cookies in the HTTP client embedded inside Rest Assured with:
RestAssuredConfig config = new RestAssuredConfig().httpClient(new HttpClientConfig().setParam(ClientPNames.COOKIE_POLICY, CookiePolicy.BEST_MATCH));
expect().statusCode(201).given()
.auth().digest("user_test", "password") // Digest added here
.config(config)
.queryParam("foo", generateFoo())
.queryParam("bar", generateBar())
.when().post("/foo/bar/");
The HTTP client (and therefore Rest Assured) supports digest authentication and the configuration of RestAssured using the digest method works well.