Microsoft storage emulator missing headers with azure sdk 12.4.0 - azure-java-sdk

I'm using Microsoft storage emulator for development and in concrete, the method that gives me an error is the exists(), the error is a null pointer exception.
Looking deeply in the library, I was able to find that it was requesting a couple of headers, then I notice that the endpoint response from azure blob storage is different than from azure storage emulator:
From the emulator:
curl -G http://127.0.0.1:10000/devstoreaccount1/default?restype=container -v -H 'x-ms-version: 2019-07-07'
* Connected to 127.0.0.1 (127.0.0.1) port 10000 (#0)
> GET /devstoreaccount1/default?restype=container HTTP/1.1
> Host: 127.0.0.1:10000
> User-Agent: curl/7.51.0
> Accept: */*
> x-ms-version: 2019-07-07
>
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Last-Modified: Mon, 24 Feb 2020 18:21:08 GMT
< ETag: "0x8D7B95651C1DCC0"
< Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
< x-ms-request-id: 63875efa-4883-45fc-8dee-2481781ae3de
< x-ms-version: 2019-07-07
< x-ms-lease-status: unlocked
< x-ms-lease-state: available
< x-ms-blob-public-access: container
< Date: Mon, 24 Feb 2020 20:28:26 GMT
From storage:
curl -Gv https://account.blob.core.windows.net/default?restype=container -H 'x-ms-version: 2019-07-07'
> GET /roo?restype=container HTTP/1.1
> Host: account.blob.core.windows.net
> User-Agent: curl/7.51.0
> Accept: */*
> x-ms-version: 2019-07-07
>
< HTTP/1.1 200 OK
< Content-Length: 0
< Last-Modified: Mon, 24 Feb 2020 18:57:04 GMT
< ETag: "0x8D7B95B569707A4"
< Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
< x-ms-request-id: 9548781e-a01e-002f-0a51-eb0305000000
< x-ms-version: 2019-07-07
< x-ms-lease-status: unlocked
< x-ms-lease-state: available
< x-ms-has-immutability-policy: false
< x-ms-has-legal-hold: false
< x-ms-blob-public-access: container
< x-ms-default-encryption-scope: $account-encryption-key
< x-ms-deny-encryption-scope-override: false
< Date: Mon, 24 Feb 2020 20:29:01 GMT
<
* Curl_http_done: called premature == 0
* C
As you can see, there are a couple of headers missing in the storage emulator.
Is there a way to get around this ?
how recommended it is to wrap the exists() method on my own just to be able to use the development storage emulator ?
Versions:
StorageEmulator: 5.10
SDK: 12.4.0
Language: Java
Thanks.
Note: I have removed information as tls and ips from the curl responses.

Can you please try to use Azurite instead and start it with the --loose parameter to ignore unsupported headers?
https://github.com/Azure/Azurite/

Related

"Given request parameters are null or invalid" when using curl to submit URLs with IndexNow API

I try to submit URLs to the Bing search engine through IndexNow API using cURL, but I get an "invalid parameter" error, how to fix it?
Here is the PowerShell script
$header_host = '"Host: www.bing.com"'
$header_content = '"Content-Type: application/json; charset=utf-8"'
$url_index =
'"https://www.bing.com/IndexNow"'
$data = '{
"host": "xinlake.dev",
"key": "72d80bb8d9f645ad9e246e46d336c997",
"keyLocation": "https://xinlake.dev/72d80bb8d9f645ad9e246e46d336c997.txt",
"urlList": [
"https://xinlake.dev/blog/2022/05-voting-script/",
"https://xinlake.dev/blog/2022/05-private-channel/"
]
}'
& curl.exe -v --http1.1 --header $header_host --header $header_content --data $data $url_index
""
Here is the error log
* Trying 202.89.233.100:443...
* Connected to www.bing.com (202.89.233.100) port 443 (#0)
* schannel: disabled automatic use of client certificate
* schannel: ALPN, offering http/1.1
* schannel: ALPN, server accepted to use http/1.1
> POST /IndexNow HTTP/1.1
> Host: www.bing.com
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Type: application/json; charset=utf-8
> Content-Length: 278
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Length: 92
< Content-Type: application/json; charset=utf-8
< Expires: -1
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< X-Cache: CONFIG_NOCACHE
< Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
< X-MSEdge-Ref: Ref A: C06F23F2577843719494581907366B26 Ref B: BJ1EDGE0917 Ref C: 2022-06-15T10:20:00Z
< Date: Wed, 15 Jun 2022 10:20:00 GMT
<
{"code":"InvalidRequestParameters","message":"Given request parameters are null or invalid"}* Connection #0 to host www.bing.com left intact
Thanks, I have solved this by posting a JSON file.
Script changes
$data = '"path to the json file"'
& curl.exe -v --http1.1 --header $header_host --header $header_content --data #$data $url_index
JSON file
{
"host": "xinlake.dev",
"key": "72d80bb8d9f645ad9e246e46d336c997",
"keyLocation": "https://www.xinlake.dev/72d80bb8d9f645ad9e246e46d336c997.txt",
"urlList": [
"https://www.xinlake.dev/blog/2022/05-voting-script/",
"https://www.xinlake.dev/blog/2022/05-private-channel/"
]
}
Request and response
* Trying 202.89.233.101:443...
* Connected to www.bing.com (202.89.233.101) port 443 (#0)
* schannel: disabled automatic use of client certificate
* schannel: ALPN, offering http/1.1
* schannel: ALPN, server accepted to use http/1.1
> POST /IndexNow HTTP/1.1
> Host: www.bing.com
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Type: application/json; charset=utf-8
> Content-Length: 302
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: -1
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< X-Cache: CONFIG_NOCACHE
< Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
< X-MSEdge-Ref: Ref A: C9107410449C4D1A837A53C5CB1D392C Ref B: BJ1EDGE0416 Ref C: 2022-06-15T14:11:21Z
< Date: Wed, 15 Jun 2022 14:11:21 GMT
< Content-Length: 0
<
* Connection #0 to host www.bing.com left intact

What could cause a 302 redirect?

I have a wordpress site and I have a blog. When I try to share my blog post on FB http://videoefekt.cz/?p=472 , it shows the thumb of my homepage. I tried to change permalinks to clear, deactivate plugins and chceck .htaccess file, everything looks fine, but the FB sharing debugger shows I have 302 HTTP Redirect to my homepage, can anyone tell me what could cause this problem ?
Simulate the facebook crawler access to your site and it made redirect to home page when using ipv6 address. Possibly you have misconfig when the request is from ipv6 address.
curl -H "User-Agent: facebookexternalhit/1.1" "http://videoefekt.cz/?p=472" -vvvv -6
* About to connect() to videoefekt.cz port 80 (#0)
* Trying 2a02:2b88:1:4::a6...
* Connected to videoefekt.cz (2a02:2b88:1:4::a6) port 80 (#0)
> GET /?p=472 HTTP/1.1
> Host: videoefekt.cz
> Accept: */*
> User-Agent: facebookexternalhit/1.1
>
< HTTP/1.1 302 Found
< Date: Thu, 04 Jan 2018 04:16:32 GMT
< Server: Apache
< Pragma: no-cache
< Expires: Wed, 11 Jan 1984 05:00:00 GMT
< Cache-Control: no-cache, must-revalidate, max-age=0
< Link: <http://videoefekt.cz/wp-json/>; rel="https://api.w.org/"
< Set-Cookie: _icl_current_language=cs; expires=Fri, 05-Jan-2018 04:16:33 GMT; Max-Age=86400; path=/
< Set-Cookie: wfvt_3708104896=5a4daaa1e2090; expires=Thu, 04-Jan-2018 04:46:33 GMT; Max-Age=1800; path=/; HttpOnly
< Set-Cookie: PHPSESSID=8a1a1d9247ac9d9707b616c6c0ad2ccc; path=/
< Upgrade: h2,h2c
< Connection: Upgrade
< Location: http://videoefekt.cz/
< Vary: User-Agent
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<
* Connection #0 to host videoefekt.cz left intact

Empty response from Bitstamp.net

I am trying to use the private api from Bitstamp to query for my account's balance. I believe I am posting the right data as no error is returned but Bitstamp's answer is always empty
* Trying 107.154.249.133...
* Connected to www.bitstamp.net (107.154.249.133) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* NPN, negotiated HTTP1.1
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: OU=Domain Control Validated; CN=*.bitstamp.net
* start date: Apr 8 14:11:39 2015 GMT
* expire date: Apr 8 15:48:13 2018 GMT
* subjectAltName: www.bitstamp.net matched
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.;
OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate
Authority - G2
* SSL certificate verify ok.
> POST /api/v2/balance HTTP/1.1
Host: www.bitstamp.net
User-Agent: Bitstamp C++ API Client
Accept: */*
Content-Length: 134
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 134 out of 134 bytes
< HTTP/1.1 301 MOVED PERMANENTLY
< Access-Control-Allow-Headers: x-requested-with, Content-Type, origin, accept, cache-control
< Access-Control-Allow-Methods: POST, GET
< Access-Control-Allow-Origin: *
< Content-Language: en
< Content-Type: text/html; charset=utf-8
< Date: Fri, 08 Sep 2017 17:46:41 GMT
< Location: http://www.bitstamp.net/api/v2/balance/
< Server: Apache
< Strict-Transport-Security: max-age=63072000; includeSubDomains
< Vary: Accept-Language
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
< Connection: keep-alive
< Set-Cookie: visid_incap_99025=wHhR5uPzT92ReB9XbeNToIHXslkAAAAAQUIPAAAAAAAbNQJEog5ZojcqdOv0dFhd; expires=Sat, 08 Sep 2018 10:32:13 GMT; path=/; Domain=.bitstamp.net
< Set-Cookie: nlbi_99025=VyBXXwUOsQHckThX8F1n9AAAAABOW3tLZ0nUHyp47yinIimv; path=/; Domain=.bitstamp.net
< Set-Cookie: incap_ses_569_99025=04PeXe3KGFAAVw//8n7lB4HXslkAAAAA09BkGnsaiskl2dkFxzhFXQ==; path=/; Domain=.bitstamp.net
< X-Iinfo: 9-15073063-15060114 PNNN RT(1504892800858 31) q(0 0 0 -1) r(2 2) U6
< X-CDN: Incapsula
<
* Connection #0 to host www.bitstamp.net left intact
result: 0
response:
terminate called after throwing an instance of 'std::runtime_error'
what(): Bitstamp response doesn't contain result data
Any idea ?
Thanks a lot in advance !
Initially I had the balance path to be /balance, but after changing to /balance/ it worked and got 200 back with payload.
So instead of http://www.bitstamp.net/api/v2/balance use http://www.bitstamp.net/api/v2/balance/, note the trailing backslash

How do I tell spray-json on Akka HTTP to "assume application/json"?

Using spray-json on an Akka HTTP server; how can I "assume Content-type: application/json" for requests when not supplied?
Issue
curl localhost:12345/request -d'{"sample":"json"}'
doesn't work unless I add
-H "Content-type: application/json"
explicitly.
Why I ask
not ideal for showing succint examples on a README.md
annoying to add the header manually all the time while debugging
You should have defined you API to accept content-type json only, thats why it does not accept other content-types.
Just doing curl -XPOST -d 'content' URL will post your content as application/x-www-form-urlencoded (default behaviour).
See --verbose of curl POST, see Content-Type in example below
$ curl -XPOST -d '{"test" : "testValue"}' "https://jsonplaceholder.typicode.com/posts" -v
* Trying 104.31.87.157...
* Connected to jsonplaceholder.typicode.com (104.31.87.157) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate: sni233425.cloudflaressl.com
* Server certificate: COMODO ECC Domain Validation Secure Server CA 2
* Server certificate: COMODO ECC Certification Authority
* Server certificate: AddTrust External CA Root
> POST /posts HTTP/1.1
> Host: jsonplaceholder.typicode.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 22
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 22 out of 22 bytes
< HTTP/1.1 201 Created
< Date: Sat, 18 Mar 2017 04:54:41 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 51
< Connection: keep-alive
< Set-Cookie: __cfduid=d91d0ebbaa5f7b40a998d327ffce2b5671489812881; expires=Sun, 18-Mar-18 04:54:41 GMT; path=/; domain=.typicode.com; HttpOnly
< X-Powered-By: Express
< Vary: Origin, X-HTTP-Method-Override, Accept-Encoding
< Access-Control-Allow-Credentials: true
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: -1
< X-Content-Type-Options: nosniff
< Etag: W/"33-cwlRczABGtsoiZUP8lw9WSIt684"
< Via: 1.1 vegur
< Server: cloudflare-nginx
< CF-RAY: 3415986bb9f50c35-SEA
<
{
"{\"test\" : \"testValue\"}": "",
"id": 101
* Connection #0 to host jsonplaceholder.typicode.com left intact
}
You have to change your client headers to match what the API server accepts.
$ curl -H "Content-type: application/json" -XPOST -d '{"test" : "testValue"}' "https://jsonplaceholder.typicode.com/posts" -v
* Trying 104.31.87.157...
* Connected to jsonplaceholder.typicode.com (104.31.87.157) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate: sni233425.cloudflaressl.com
* Server certificate: COMODO ECC Domain Validation Secure Server CA 2
* Server certificate: COMODO ECC Certification Authority
* Server certificate: AddTrust External CA Root
> POST /posts HTTP/1.1
> Host: jsonplaceholder.typicode.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-type: application/json
> Content-Length: 22
>
* upload completely sent off: 22 out of 22 bytes
< HTTP/1.1 201 Created
< Date: Sat, 18 Mar 2017 04:53:22 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 38
< Connection: keep-alive
< Set-Cookie: __cfduid=d8e90de27fc1fa87fb42b763a4199ba7d1489812801; expires=Sun, 18-Mar-18 04:53:21 GMT; path=/; domain=.typicode.com; HttpOnly
< X-Powered-By: Express
< Vary: Origin, X-HTTP-Method-Override, Accept-Encoding
< Access-Control-Allow-Credentials: true
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: -1
< X-Content-Type-Options: nosniff
< Etag: W/"26-Zx6VPi+rfbM5YFlBzT2pzGEHcgg"
< Via: 1.1 vegur
< Server: cloudflare-nginx
< CF-RAY: 3415967b2af12a19-SEA
<
{
"test": "testValue",
"id": 101
* Connection #0 to host jsonplaceholder.typicode.com left intact
}
You can change the content-type on you server, and change whatever content you receive to JSON by yourself (which might not a good idea though)

PayPal REST API: trying to activate a billing plan, but getting "HTTP/1.1 401 Unauthorized"

I am building an EC site, where my customers can pay through PayPal API. Now, I am learning how it works, using Sandbox; referring to this page: https://developer.paypal.com/docs/integration/direct/billing-plans/
I have succeeded the prerequisites, i.e., having created a PayPal app,
gotten an access token, and made an API call. And then, I have succeeded making a billing plan, copying-and-pasting the sample command on the page, just replacing the Access Token.
Now, I want to activate the billing plan, but couldn't have succeeded.
The command used is, once again, copied-and-pasted from the page, as below.
curl -v -k -X PATCH https://api.sandbox.paypal.com/v1/payments/billing-plans/P-7DC96732KA7763723UOPKETA/ \
-H 'X-PAYPAL-OAUTH-CONTEXT: {"consumer":{"accountNumber":1181198218909172527,"merchantId":"5KW8F2FXKX5HA"},"merchant":{"accountNumber":1659371090107732880,"merchantId":"2J6QB8YJQSJRJ"},"apiCaller":{"clientId":"AdtlNBDhgmQWi2xk6edqJVKklPFyDWxtyKuXuyVT-OgdnnKpAVsbKHgvqHHP","appId":"APP-6DV794347V142302B","payerId":"2J6QB8YJQSJRJ","accountNumber":"1659371090107732880"},"scopes":["https://api.paypal.com/v1/payments/.*","https://uri.paypal.com/services/payments/futurepayments","openid"]}' \
-H 'Content-Type: application/json' \
-d '[{
"op": "replace",
"path": "/",
"value": {
"state": "ACTIVE"
}
}]'
First, as instructed, I just replaced the Plan ID "P-7DC96732KA7763723UOPKETA" with mine, provided in the previous response for creating the plan.
* Trying 173.0.82.78...
* Connected to api.sandbox.paypal.com (173.0.82.78) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_AES_256_CBC_SHA256
* Server certificate: api.sandbox.paypal.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> PATCH /v1/payments/billing-plans/P-8AX21799EN516221GJMBOECA/ HTTP/1.1
> Host: api.sandbox.paypal.com
> User-Agent: curl/7.43.0
> Accept: */*
> X-PAYPAL-OAUTH-CONTEXT: {"consumer":{"accountNumber":1181198218909172527,"merchantId":"5KW8F2FXKX5HA"},"merchant":{"accountNumber":1659371090107732880,"merchantId":"2J6QB8YJQSJRJ"},"apiCaller":{"clientId":"AdtlNBDhgmQWi2xk6edqJVKklPFyDWxtyKuXuyVT-OgdnnKpAVsbKHgvqHHP","appId":"APP-6DV794347V142302B","payerId":"2J6QB8YJQSJRJ","accountNumber":"1659371090107732880"},"scopes":["ttps://api.paypal.com/v1/payments/.*","ttps://uri.paypal.com/services/payments/futurepayments","openid"]}
> Content-Type: application/json
> Content-Length: 78
>
* upload completely sent off: 78 out of 78 bytes
< HTTP/1.1 401 Unauthorized
< Date: Fri, 17 Feb 2017 12:18:47 GMT
< Server: Apache
< paypal-debug-id: c46e304b4bb46
< Paypal-Debug-Id: c46e304b4bb46
< Set-Cookie: X-PP-SILOVER=name%3DSANDBOX3.API.1%26silo_version%3D1880%26app%3Dplatformapiserv%26TIME%3D669689432%26HTTP_X_PP_AZ_LOCATOR%3D; Expires=Fri, 17 Feb 2017 12:48:47 GMT; domain=.paypal.com; path=/; Secure; HttpOnly
< Set-Cookie: X-PP-SILOVER=; Expires=Thu, 01 Jan 1970 00:00:01 GMT
< Content-Length: 0
< Connection: close
< Content-Type: text/plain; charset=ISO-8859-1
<
* Closing connection 0
And I tried with some other parameter replacements, like the Client ID, but no luck.
Do I need to replace other parameters?
Or, have I done something wrong?
SOLVED.
On this page, I found simpler and more straightforward command
and made up below:
curl -v -X PATCH https://api.sandbox.paypal.com/v1/payments/billing-plans/<Plan ID>/ \
-H "Content-Type:application/json" \
-H "Authorization: Bearer <Access Token>" \
-d '[
{
"path":"/",
"value":{"state":"ACTIVE"},
"op":"replace"}
]'
Got the response as below:
* Trying 173.0.82.78...
* Connected to api.sandbox.paypal.com (173.0.82.78) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_AES_256_CBC_SHA256
* Server certificate: api.sandbox.paypal.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/payments/billing-plans/<Plan ID> HTTP/1.1
> Host: api.sandbox.paypal.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Type:application/json
> Authorization: Bearer <Access Token>
>
< HTTP/1.1 200 OK
< Date: Sun, 19 Feb 2017 11:36:50 GMT
< Server: Apache
< paypal-debug-id: 3ca31778e271e
< Content-Language: *
< Paypal-Debug-Id: 3ca31778e271e
< Set-Cookie: X-PP-SILOVER=name%3DSANDBOX3.API.1%26silo_version%3D1880%26app%3Dplatformapiserv%26TIME%3D1384360280%26HTTP_X_PP_AZ_LOCATOR%3D; Expires=Sun, 19 Feb 2017 12:06:52 GMT; domain=.paypal.com; path=/; Secure; HttpOnly
< Set-Cookie: X-PP-SILOVER=; Expires=Thu, 01 Jan 1970 00:00:01 GMT
< Vary: Authorization
< Content-Length: 1391
< Connection: close
< Content-Type: application/json
<
* Closing connection 0
{"id":"<Plan ID>","state":"ACTIVE","name":"Sample Plan",
...
"rel":"self","method":"GET"}]}
Got HTTP 200 and the state "ACTIVE".
Thanks for your trying to help, many thanks!