Background:
I have set up an IPFS server behind an HAProxy server.
I have written a Flutter client to connect to this IPFS server through the proxy and add a file.
Problem:
Everything works as expected when I run the Flutter client as a desktop app (on Macos), but I get a 403 error when I run the Flutter client as a web app.
Details:
The headers for the successful call (captured via tcpdump) are as follows:
REQUEST:
POST /api/v0/add HTTP/1.1
Host: po.segito.net
User-Agent: Dart/2.11 (dart:io)
Accept-Encoding: gzip
Content-Type: multipart/form-data; boundary=dart-http-boundary-fmjPP-TIwnDcY7pJGniid4grt9mdDADtazmb7Pm8sP_PRJkV1oY
Content-Length: 39611
--dart-http-boundary-fmjPP-TIwnDcY7pJGniid4grt9mdDADtazmb7Pm8sP_PRJkV1oY
content-type: application/octet-stream
content-disposition: form-data; name="asset"
RESPONSE:
HTTP/1.1 200 OK
Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Content-Type: application/json
Server: go-ipfs/0.6.0
Trailer: X-Stream-Error
Vary: Origin
X-Chunked-Output: 1
Date: Sat, 07 Nov 2020 09:48:28 GMT
Transfer-Encoding: chunked
The headers for the unsuccessful call is as follows:
REQUEST:
POST /api/v0/add HTTP/1.1
Host: po.segito.net
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:82.0) Gecko/20100101 Firefox/82.0
Accept-Encoding: gzip, deflate, br
Content-Type: multipart/form-data; boundary=dart-http-boundary-HvcRoK0lGd4+oahuNaOejbIBd-cID5+0n3GFvwwOY4ZqfitRP1s
Content-Length: 39611
--dart-http-boundary-HvcRoK0lGd4+oahuNaOejbIBd-cID5+0n3GFvwwOY4ZqfitRP1s
content-type: application/octet-stream
content-disposition: form-data; name="asset"
RESPONSE:
HTTP/1.1 403 Forbidden
Content-Type: text/plain; charset=utf-8
Vary: Origin
X-Content-Type-Options: nosniff
Date: Sat, 07 Nov 2020 09:38:55 GMT
Content-Length: 16
403 - Forbidden
Notes:
There is no CORS issue because all files and REST calls (including
the Flutter HTML files) are all served via the same HAProxy server.
The HAProxy is an SSL terminator.
A CURL request works as expected.
All other REST calls through the HAProxy works as expected.
Related
I am using traefik in kubernetes and I have a service deployed that is returning 103 Early Hint. I can confirm that it is working by directly querying the service, e.g.
curl -D - http://contra-web-app
HTTP/1.1 103 Early Hints
Link: <https://builds.contra.com>; rel="preconnect"; crossorigin
Link: <https://fonts.googleapis.com/css2?family=Inter:wght#400;500;600;700;900&display=swap>; rel="preload"; as="font"
Link: <https://builds.contra.com/3f509d0cc/assets/entry-client-routing.4f895d55.js>; rel="modulepreload"; as="script"; crossorigin
Link: <https://www.googletagmanager.com/gtag/js?id=G-96H5NXQ2PR>; rel="preload"; as="script"
HTTP/1.1 200 OK
cache-control: no-store
referrer-policy: strict-origin-when-cross-origin
x-frame-options: sameorigin
content-type: text/html
content-length: 9062
Date: Tue, 26 Jul 2022 20:34:19 GMT
Connection: keep-alive
Keep-Alive: timeout=72
However, requesting the same service through Traefik just returns 200 response:
curl -H 'host: contra.com' -D - http://contra-traefik.traefik/gajus
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Length: 11441
Content-Type: text/html
Date: Tue, 26 Jul 2022 19:51:48 GMT
Referrer-Policy: strict-origin-when-cross-origin
Set-Cookie: contra_web_app_service=394e7e912ad85b66; Path=/; Secure
Vary: Accept-Encoding
X-Frame-Options: sameorigi
At this point, I am unable to establish whether I am missing a configuration or if Traefik does not support it.
At the moment, we are trying to get CI working in our labs..
we have just followed the instructions on the concourse website.
We are able to login properly and have setup ~/.flyrc as recomended ion the concourse-ci.org and concoursetutorial.com websites.
We have noticed that most commands are returning with a 401 Unauthorized error.
We have gone ahead setup the audit logs https://concourse-ci.org/concourse-web.html#audit-logs
But it isn't clear where this writes to, help?
It is difficult at the moment to properly trace this. BTW this is our first exposure to concourse.
We would like to know why? and what we can do resolve this (to cross this huddle).
fly -t rdb-ci set-team --team-name a-team --local-user admin --github-org organization --verbose --print-table-headers --non-interactive
2019/07/10 22:02:37 GET /api/v1/info HTTP/1.1
Host: ci.example.org
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip
2019/07/10 22:02:37 HTTP/1.1 200 OK
Content-Length: 88
Connection: keep-alive
Content-Type: application/json
Date: Wed, 10 Jul 2019 21:02:37 GMT
Server: nginx/1.12.2
X-Concourse-Version: 5.3.0
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: deny
X-Xss-Protection: 1; mode=block
{"version":"5.3.0","worker_version":"2.1","external_url":"https://ci.example.org"}
setting team: a-team
role owner:
users:
- local:admin
groups:
- github:organization
apply team configuration? [yN]: y
2019/07/10 22:02:53 PUT /api/v1/teams/a-team HTTP/1.1
Host: ci.example.org
User-Agent: Go-http-client/1.1
Content-Length: 71
Content-Type: application/json
Accept-Encoding: gzip
{"auth":{"owner":{"groups":["github:organization"],"users":["local:admin"]}}}
2019/07/10 22:02:53 HTTP/1.1 401 Unauthorized
Content-Length: 14
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Wed, 10 Jul 2019 21:02:53 GMT
Server: nginx/1.12.2
X-Concourse-Version: 5.3.0
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: deny
X-Xss-Protection: 1; mode=block
not authorized
could not find a valid token.
logging in to team 'main'
2019/07/10 22:02:53 GET /api/v1/info HTTP/1.1
Host: ci.example.org
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip
could not reach the Concourse server called rdb-ci:
Get https://ci.example.org/api/v1/info: x509: certificate is valid for www.example.org, not ci.example.org
is the targeted Concourse running? better go catch it lol
A Laravel app is using the JSTree to display files.
If I get the tree under http://localhost:8000 I recive the correct tree.
We have a ngnix reverse Proxy setting to access the web site from behind a proxy.
But if I open the ngnix web site there are in some cases no data. The ajax response is correct, but JSTree doesn't render it.
Have anybody a idea?
First I tried out the jstree().last_error() function and it is a empty object.
Here are my header, I hope it helps:
Host: DOMAIN.de
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://DOMAIN.de/explorer/show/443
Content-Length: 6
Cookie: cartalyst_sentinel=eyJpdiI6I...iJ9; laravel_session=eyJp...J9
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
The response:
Cache-Control: private, must-revalidate
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html; charset=UTF-8
Date: Tue, 12 Apr 2016 06:37:12 GMT
Expires: -1
Host: DOMAIN.de
Pragma: no-cache
Server: nginx/1.4.6 (Ubuntu)
Set-Cookie: laravel_session=eyJpdi......3D; expires=Tue, 12-Apr-2016 08:37:35 GMT; Max-Age=7200; path=/; httponly
Transfer-Encoding: chunked
X-Powered-By: PHP/5.6.19
The PHP header:
header('Content-Type: application/json; charset=utf-8');
The problem is, with ngnix the response has another Content-Type. ngnix put "application/json" to "text/html".
Are there any options to modify this?
I am trying to get the data of an application which is build on Oracle Application Development Framework.
Whenver i try to wget the application it returns nothing..no content only returns below content
<html>
<head>
</head>
<body>
<meta HTTP-EQUIV="REFRESH" content="0; url=...">
</body>
</html>
I have also tried to wget the application by passing header values to wget, but it does not make any difference.
wget command was as below :
wget --header="Host:example.com" --header="User-Agent:Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:14.0) Gecko/20100101 Firefox/14.0." --header="Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header="Accept-Language:en-us,en;q=0.5" --header="Proxy-Connection:keep-alive" --header="Referer:http://example.com/Internet/faces/sbig_home" --header="GET:/Internet/faces/sbig_home?_afrLoop=19302770119376498&_afrWindowMode=0&_afrWindowId=null HTTP/1.1" http://example.com/
Expected result is html content of website application.
Below is the output of wget with --debug option(I have changed the url with example.com)
DEBUG output created by Wget 1.13.4 on linux-gnu.
URI encoding = `UTF-8'
--2014-05-14 09:41:18-- http://exaple.com/
Resolving exaple.com (exaple.com)... 202.59.244.48
Caching exaple.com => 202.59.244.48
Connecting to exaple.com (exaple.com)|202.59.244.48|:80... connected.
Created socket 3.
Releasing 0x08823ed0 (new refcount 1).
---request begin---
GET / HTTP/1.1
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:14.0) Gecko/20100101 Firefox/14.0.
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Host: exaple.com
Connection: Keep-Alive
Accept-Language: en-us,en;q=0.5
Proxy-Connection: keep-alive
Referer: http://exaple.com/Internet/faces/sbig_home
GET: /Internet/faces/sbig_home?_afrLoop=19302770119376498&_afrWindowMode=0&_afrWindowId=null HTTP/1.1
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Date: Tue, 06 May 2014 19:33:42 GMT
Last-Modified: Sat, 03 Aug 2013 10:45:27 GMT
ETag: "27400a-73-4e308c5f11fc0"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Type: text/html
Content-Language: en
Connection: Keep-Alive
Keep-Alive: timeout=20, max=999
Server: Oracle-Application-Server-11g Oracle-Web-Cache-11g/11.1.1.6.0 (G;max-age=0+0;age=0;ecid=99079302593879470,0:1)
Content-Length: 115
---response end---
200 OK
Registered socket 3 for persistent reuse.
Length: 115 [text/html]
Saving to: `index.html.2'
I'm getting error "Network unreachable: robots.txt unreachable" when trying to add my website on Google Webmaster tools -> http://www.hyponomist.com/
You can check my robots.txt at here and sitemap.xml at here
I have reading other posts here and there, but could not solve/understand. what is causing this issue. Also, I tried downloading a page with the Fetch as Googlebot tool but got same error.
Anyone knows?
Thanks in advance!
Your web server is returning a 503 error when the user-agent string says the request is from Googlebot, but 200 when it's from a browser. If you use an http diagnostic tool such as Fiddler (http://fiddler2.com/) you can see this.
If you use Fiddler to send the same request that a browser would send:
GET http://www.hyponomist.com/robots.txt HTTP/1.1
Host: www.hyponomist.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.72 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
The response is:
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Fri, 10 Jan 2014 21:34:42 GMT
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Retry-After: 18000
Last-Modified: Fri, 10 Jan 2014 20:43:28 GMT
Content-Encoding: gzip
If you change the user-agent to mimic Googlebot:
GET http://www.hyponomist.com/robots.txt HTTP/1.1
Host: www.hyponomist.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Then the response is:
HTTP/1.1 503 Service Temporarily Unavailable
Server: nginx/1.4.4
Date: Fri, 10 Jan 2014 21:35:25 GMT
Content-Type: text/html; charset=iso-8859-1
Content-Length: 234
Connection: keep-alive
Retry-After: 18000
Exactly why it's doing this, I can't tell you. 503 is normally the error sent when a server is temporarily overloaded, but that's clearly not the case here. Maybe your firewall is poorly configured, and has blacklisted Googlebot based on request frequency? Take a look at your firewall settings and your server config.
Removing the trailing slash (use http://www.hyponomist.com instead of http://www.hyponomist.com/) may help