Can we use nginx proxy for gRPC-web integration, without docker dependency? - grpc-web

Need to use NGINX without docker
I have tried using gRPC-web integration using the envoy proxy that had docker dependencies and so I moved to NGINX, how to use NGINX without docker dependencies?

Since there is no direct support for grpc-web from nginx, we can make a below hack in the config file of the nginx to work with both grpc-web and grpc calls.
server {
listen 1449 ssl http2;
server_name `domain-name`;
ssl_certificate `pem-file`; # managed by Certbot
ssl_certificate_key `key-file`; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
location / {
#
## Any request with the content-type application/grpc+(json|proto|customType) will not enter the
## if condition block and make a grpc_pass while rest of the requests enters into the if block
## and makes a proxy_prass request. Explicitly grpc-web will also enter the if block.
#
if ($content_type !~ 'application\/grpc(?!-web)(.*)'){
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Transfer-Encoding,Custom-Header-1,X-Accept-Content-Transfer-Encoding,X-Accept-Response-Streaming,X-User-Agent,X-Grpc-Web,content-type,snet-current-block-number,snet-free-call-user-id,snet-payment-channel-signature-bin,snet-payment-type,x-grpc-web';
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
proxy_pass http://reroute_url;
}
grpc_pass grpc://reroute_url;
}
}
The above config works based on the content-type mechanism, whenever the grpc-web makes a call to nginx, the content-type would be application/grpc-web and this content-type is not been handled by nginx with grpc_pass.
Hence, only those requests with content-type application/grpc+(proto|json|customType) works with grpc_pass while rest of the requests will work with proxy_pass.

You can look at what the dockerfile does and basically do it yourself outside of the docker image: https://github.com/grpc/grpc-web/blob/master/net/grpc/gateway/docker/nginx/Dockerfile
The main thing is basically to run make standalone-proxy, and run it as ./gConnector_static/nginx.sh. You will need a nginx.conf config file to specify where Nginx should receive and forward the gRPC-Web requests

Related

401 error on API over HTTPS, but not over HTTP

I'm trying to link with my mongoDB database, using axios, Vue, and my server is running NGINX.
In my App.vue file, where the axios requests are, I have this:
export default {
name: 'App',
data(){
return {
baseApiURL: '[HTTP or HTTPS]://example.com:4000/api',
This works over HTTP. When I change it to HTTPS, it doesn't. I have tried using the IP address, and using the domain address. Network tab on Chrome says:
Request URL: https://www.example.com/api/
Request Method: GET
Status Code: 401 Unauthorized
I don't understand what this means exactly.
My NGINX config:
server {
listen 80 default_server;
ssl on;
listen 443;
server_name example.com;
ssl_certificate /usr/src/app/ssl/domain.cert.pem;
ssl_certificate_key /usr/src/app/ssl/private.key.pem;
# vue app & front-end files
location / {
root /usr/src/app/dist;
try_files $uri /index.html;
}
# node api reverse proxy
location /api/ {
proxy_pass http://localhost:4000/;
}
}
I'm not sure if there's anything else I should include in this info, please let me know. I feel like it should be a small issue as it's working over HTTP.
Not really an "answer" as it doesn't solve the exact initial problem, but I solved my problem of my site not working over HTTPS by using Caddy Server instead of NGINX, which somehow does it automatically.

Nginx HTTPS subdomain redirect

I have a couple of websites configured with Nginx (nginx/1.10.3 (Ubuntu)) Server Blocks. Browsing to any of the configured domains listed below works as expected:
Those that are configured to use HTTPS get automatic HTTP -> HTTPS redirects (morgrowe.com for example). The issue is when I browse to a subdomain that doesn't exist (for example: doesnotexist.morgrowe.com). If I go to http://doesnotexist.morgrowe.com, I get redirected to the default Nginx page (which is what I want). However, if I go to https://doesnotexist.morgrowe.com, I get redirected to https://api.morgrowe.com.
This wouldn't be so bad, but if I go to https://doesnotexist.carpyslocksmiths.com, I also get redirected to https://api.morgrowe.com.
Where can I find the configuration that makes this happen? Ideally, I'd like the default nginx 404 page to appear like it does for http connections. I imagine I have to configure something in /etc/nginx/sites-available/default, but I don't know where to start. Here's my /etc/nginx/sites-available/default file contents:
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# include snippets/fastcgi-php.conf;
#
# # With php7.0-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php7.0-fpm:
# fastcgi_pass unix:/run/php/php7.0-fpm.sock;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
use a Default “Catch All” Server Block using _ in server_name both for http and https. _ is just an invalid value which will never trigger on a real host.
Please see example From below link
https://www.nginx.com/resources/wiki/start/topics/examples/server_blocks/

nginx redirect rule is redirecting everything to https even for other ports

Hello I have this config
server {
listen 82;
server_name myapp.mydomain.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl;
server_name myapp.mydomain.com
# ... remain of the stuff here
}
Before that I had other websites running in ports: 80, 3000 etc... now when I access http://myapp.mydomain.com automatically redirects me to my app (as is I were invoking port 82) and if I try to access another app running on 3000 port it tries to rewrite the https://myapp.mydomain.com:3000 as well... if I use the ip it works as expected (not the ssl part).
Full config can be found at:
https://gist.github.com/angvp/363f50ff8b8d345126adaf1595cd2523
Any ideas?
Ok after I start digging I had this on my nginx conf:
add_header Strict-Transport-Security max-age=15768000;
This is a security measure but that was causing all the subdomains even on different ports will try always https .. the correct way should be to have different subdomains per vhost per port..

Nginx dropping SSL handshakes with multiple virtual hosts

I have an Nginx server set up on an Ubuntu VPS with multiple hosts (1 IP). Previously, 1 host had certificates set up and NO redirecting (http allowed) and 1 host had certificates and forced HTTPS via 301. Now that I am attempting to force all of my hosts on SSL and force HTTPS, I am seeing that Nginx is dropping handshakes when I have more than 1 vhost with 301 directives. In particular, the error I am seeing is:
[error] 12370#0: *30 no "ssl_certificate" is defined in server listening on SSL port while SSL handshaking, client: (removed), server: 0.0.0.0:443
The issue definitely seems to be with my 301's, because if I exclude them I do not have an issue. What's the best way I can force HTTPS and non-www in my server blocks?
All of my vhosts are in /etc/nginx/conf.d, along with ssl.conf (listed below). I can provide nginx.conf if requested, but I didn't see anything that would seem useful in there.
example1.conf
server {
server_name www.example1.com example1.com;
return 301 https://example1.com$request_uri;
}
server {
listen 443;
server_name www.example1.com
return 301 https://example1.com$request_uri;
}
server {
listen 443 ssl;
server_name example1.com;
ssl_certificate /etc/letsencrypt/live/example1.com/cert.pem;
ssl_certificate_key /etc/letsencrypt/live/example1.com/privkey.pem;
root /var/www/example1.com;
location / {
try_files $uri $uri/ $uri.html =404;
}
access_log /var/log/nginx/example1.com.access.log;
error_log /var/log/nginx/example1.com.error.log;
}
example2.conf
same as example1.conf (except with example2.com instead of example1.com)
ssl.conf
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# Perfect Forward Security
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS +RC4 RC4";
# HSTS
add_header Strict-Transport-Security max-age=31536000;
Pointing out other obvious errors is also appreciated.
Solved (at least for now) by removing
server {
listen 443;
server_name www.example1.com
return 301 https://example1.com$request_uri;
}
It seems that since there's no cert specified, this block would get hit for all HTTPS requests and then the connection would get dropped.

Nginx and Flask-socketio Websockets: Alive but not Messaging?

I've been having a bit of trouble getting Nginx to play nicely with the Python Flask-socketio library (which is based on gevent). Currently, since we're actively developing, I'm trying to get Nginx to just work as a proxy. For sending pages, I can get this to work, either by directly running the flask-socketio app, or by running through gunicorn. One hitch: the websocket messaging does not seem to work. The pages are successfully hosted and displayed. However, when I try to use the websockets, they do not work. They are alive enough that the websocket thinks it is connected, but they will not send a message. If I remove the Nginx proxy, they do work. Firefox gives me this error when I try to send a message:
Firefox can't establish a connection to the server at ws:///socket.io/1/websocket/.
Where web address is where the server is located and the unique id is just a bunch of randomish digits. It seems to be doing enough to keep the connection live (e.g., the client thinks it is connected), but can't send a message over the websocket. I have to think that the issue has to do with some part of the proxy, but am having mighty trouble debugging what the issue might be (in part because this is my first go-round with both Flask-socketIO and nginx). The configuration file I am using for nginx is:
user <user name>; ## This is set to the user name for the remote SSH session
worker_processes 5;
events {
worker_connections 1024; ## Default: 1024
}
http {
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
server_names_hash_bucket_size 128; # this seems to be required for some vhosts
server {
listen 80;
server_name _;
location / {
proxy_pass http://localhost:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
}
I made the config file as an amalgam of a general example and a websocket specific one, but trying to fiddle with it has not solved the issue. Also, I am using the werkzeug Proxy_Fix call on my Flask app.wsgi_app when I use it in wsgi mode. I've tried it with and without that, to no avail, however. If anyone has some insight, I will be all ears/eyes.
I managed to fix this. The issues were not specific to flask-socketio, but they were specific to Ubuntu, NginX, and gevent-socketio. Two significant issues were present:
Ubuntu 12.04 has a truly ancient version of nginx (1.1.19 vs 1.6.x for stable versions). Why? Who knows. What we do know is that this version does not support websockets in any useful way, as 1.3.13 is about the earliest you should be using.
By default, gevent-socketio expects your sockets to be at the location /socket.io . You can upgrade the whole HTTP connection, but I had some trouble getting that to work properly (especially after I threw SSL into the mix).
I fixed #1, but in fiddling with it I purged by nginx and apt-get installed... the default version of nginx on Ubuntu. Then, I was mysteriously confused as to why things worked even worse than before. Many .conf files valiantly lost their lives in this battle.
If trying to debug websockets in this configuration, I would recommend the following steps:
Check your nginx version via 'nginx -v'. If it is anything less than 1.4, upgrade it.
Check your nginx.conf settings. You need to make sure the connection upgrades.
Check that your server IP and port match your nginx.conf reverse proxy.
Check that your client (e.g., socketio.js) connects to the right location and port, with the right protocol.
Check your blocked ports. I was on EC2, so you have to manually open 80 (HTTP) and 443 (SSL/HTTPS).
Having just checked all of these things, there are takeaways.
Upgrading to the latest stable nginx version on Ubuntu (full ref) can be done by:
sudo apt-get install python-software-properties
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:nginx/stable
sudo apt-get update
sudo apt-get install nginx
In systems like Windows, you can use an installer and will be less likely to get a bad version.
Many config files for this can be confusing, since nginx officially added sockets in about 2013, making earlier workaround configs obsolete. Existing config files don't tend to cover all the bases for nginx, gevent-socketio, and SSL together, but have them all separately (Nginx Tutorial, Gevent-socketio, Node.js with SSL). A config file for nginx 1.6 with flask-socketio (which wraps gevent-socketio) and SSL is:
user <user account, probably optional>;
worker_processes 2;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
sendfile on;
# tcp_nopush on;
keepalive_timeout 3;
# tcp_nodelay on;
# gzip on;
client_max_body_size 20m;
index index.html;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
# Listen on 80 and 443
listen 80 default;
listen 443 ssl; (only needed if you want SSL/HTTPS)
server_name <your server name here, optional unless you use SSL>;
# SSL Certificate (only needed if you want SSL/HTTPS)
ssl_certificate <file location for your unified .crt file>;
ssl_certificate_key <file location for your .key file>;
# Optional: Redirect all non-SSL traffic to SSL. (if you want ONLY SSL/HTTPS)
# if ($ssl_protocol = "") {
# rewrite ^ https://$host$request_uri? permanent;
# }
# Split off basic traffic to backends
location / {
proxy_pass http://localhost:8081; # 127.0.0.1 is preferred, actually.
proxy_redirect off;
}
location /socket.io {
proxy_pass http://127.0.0.1:8081/socket.io; # 127.0.0.1 is preferred, actually.
proxy_redirect off;
proxy_buffering off; # Optional
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
}
Checking that your Flask-socketio is using the right port is easy. This is sufficient to work with the above:
from flask import Flask, render_template, session, request, abort
import flask.ext.socketio
FLASK_CORE_APP = Flask(__name__)
FLASK_CORE_APP.config['SECRET_KEY'] = '12345' # Luggage combination
SOCKET_IO_CORE = flask.ext.socketio.SocketIO(FLASK_CORE_APP)
#FLASK_CORE_APP.route('/')
def index():
return render_template('index.html')
#SOCKET_IO_CORE.on('message')
def receive_message(message):
return "Echo: %s"%(message,)
SOCKET_IO_CORE.run(FLASK_CORE_APP, host=127.0.0.1, port=8081)
For a client such as socketio.js, connecting should be easy. For example:
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/socket.io/0.9.16/socket.io.min.js"></script>
<script type="text/javascript">
var url = window.location.protocol + document.domain + ':' + location.port,
socket = io.connect(url);
socket.on('message', alert);
io.emit("message", "Test")
</script>
Opening ports is really more of a server-fault or a superuser issue, since it will depend a lot on your firewall. For Amazon EC2, see here.
If trying all of this does not work, cry. Then return to the top of the list. Because you might just have accidentally reinstalled an older version of nginx.