Find a word in a file and then add a line before the next known word - sed

I am trying to add a comment "##INSERT-A NEW-LINE-HERE" in a file which looks like this:
location /test/ {
#keepalive_timeout 0;
proxy_http_version 1.1;
proxy_set_header Accept-Encoding "";
proxy_set_header X-Forwarded-For;
proxy_set_header Access-Control-Al;
proxy_pass https://test.company.in/;
proxy_set_header Host test.company.in;
#proxy_set_header Referer $http_referer;
##INSERT-A NEW-LINE-HERE
if ($http_referer ~* ^(https?\:\/\/)(.*?)\/test\/(.*?)$) {
set $real_referer https://test.company.in/$3;
}
proxy_set_header Referer $real_referer;
proxy_redirect http://test.company.in/ https://sample.company.in/test/;
proxy_redirect https://test.company.in/ https://sample.company.in/test/;
proxy_redirect / https://sample.company.in/test/
sub_filter_types *;
sub_filter 'href="/' 'href="/test/';
sub_filter 'href="../' 'href="../test/';
sub_filter 'href=../' 'href=../test/';
}
location /test_app1/ {
#keepalive_timeout 0;
proxy_http_version 1.1;
proxy_set_header Accept-Encoding "";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Access-Control-Allow-Origin https://test.app1.in;
proxy_pass https://test.app1.in/;
proxy_set_header Host test.app1.in;
#proxy_set_header Referer $http_referer;
##INSERT-A NEW-LINE-HERE
if ($http_referer ~* ^(https?\:\/\/)(.*?)\/test\/(.*?)$) {
set $real_referer https://test.app1.in/$3;
}
proxy_set_header Referer $real_referer;
proxy_redirect http://test.app1.in/ https://sample.company.in/test_app1/;
proxy_redirect http://sample.company.in/ https://sample.company.in/test_app1/;
proxy_redirect https://test.app1.in/ https://sample.company.in/test_app1/;
proxy_redirect https://test.app1.in https://sample.company.in/test_app1/;
proxy_redirect https://sample.company.in/ https://sample.company.in/test_app1/;
##DO-NOTINSERT-A NEW-LINE-HERE-AS-IT-IS-SECOND-IF-BLOCK
if ($http_referer ~* ^(https?\:\/\/)(.*?)\/test\/(.*?)$) {
set $real_referer https://test.app1.in/$3;
}
proxy_set_header Referer $real_referer;
proxy_redirect / https://sample.company.in/test_app1/;
}
I want to add a comment ONLY before first IF-BLOCK after every LOCATION-BLOCK
Here is what I have currently:
sed -i '/if ($http_referer/i
##INSERT-A NEW-LINE-HERE
' file.conf
Using this above code the comment appears above all IF BLOCKS and I only need this comment for first IF-BLOCK after every LOCATION-BLOCK

This might work for you (GNU sed):
sed '/^\s*location .*{$/{:a;n;/^\s*if /!ba;h;s/\S.*/INSERTED LINE\n/;G}' file
Match on location and then continue reading lines until another match of one beginning if.
Make a copy of if, replace the line (starting at the first non white spaced character) with the intended text (with a further newline).
Append the original line.
Repeat.
N.B. This assumes all such locations contain an if clause somewhere.

my guess is the code provided will insert the comment before every instance of 'if ($http_referer'. To only insert the comment before the first instance of 'if ($http_referer' after every 'location /test/ {', use the following code:
sed -i '/location \/test\//i\
##INSERT-A NEW-LINE-HERE
' file.conf

This sed command should do the job:
sed -i.bak '
/^[[:blank:]]*location.*{/,/^[[:blank:]]*if[[:blank:]]*(/!b
/^[[:blank:]]*if[[:blank:]]*(/i\
##INSERT-A NEW-LINE-HERE
' file

Related

Custom location for PGAdmin redirect to 'localhost/' nginx

I have a nginx server with a nodejs API on 'localhost/api' and PGAdmin4 on 'localhost/'. In this case everything works without problem, but as soon as I configure the location of PGAdmin4 in the nginx.conf on 'localhost/pgadmin4', when I click on the login button on the PGAdmin4 interface, I am redirected to 'localhost/' and do not access therefore to PGAdmin.
I have tried several solutions found here, but nothing works ..
Here is my nginx.conf file (pgadmin in proxy_pass is defined in my docker-compose.yml):
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events { worker_connections 1024; }
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /dev/null;
upstream app {
least_conn;
server app:3000 weight=10 max_fails=3 fail_timeout=30s;
}
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
server {
listen 80;
location /api/ {
proxy_pass http://app;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location /pgadmin {
proxy_pass http://pgadmin;
proxy_http_version 1.1;
proxy_set_header X-Script-Name /pgadmin;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
}
With apache24 as the reverse-proxy and pgadmin4 running standalone with uWSGI, I was successful with setting the X-Script-Name header in the reverse-proxy.
Alternatively, it works also properly with not setting the X-Script-Name header in the reverse-proxy, not rewriting URL path-component in the reverse proxy, but instead adding the following configuration to uWSGI:
route-run = addvar:SCRIPT_NAME=/pgadmin
route = ^/pgadmin(.*) rewrite:$1
This removes the relocating URL path-component from PATH_INFO and puts it into SCRIPT_NAME, and is independent of what webserver is used.
I am not familiar with nginx, but comparing Your quoted config with the documentation at https://www.pgadmin.org/docs/pgadmin4/latest/container_deployment.html#http-via-nginx I might assume Your proxy_path value is wrong, and should rather contain something like http://localhost:5050/.

Restrict SOAP calls using Nginx Reverse Proxy

We have a Nginx server configured for reverse proxying to protect an aging SOAP server that can't be patched.
I've been asked to restrict what calls clients can make from specific IP ranges.
I've no idea how to allow some SOAP calls but filter out others.
The Nginux config is
server {
listen 443;
listen [::]:443;
server_name shiny.nginx.server;
ssl on;
ssl_certificate "/public/directory/proxy.pem";
ssl_certificate_key "/secret/directory/proxy.key";
location / {
proxy_pass http://very.old.server:80/;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
auth_basic "Username and Password Required";
auth_basic_user_file /etc/nginx/.htpasswd;
}
}
Any ideas how I achieve this?
Thanks
I'v found a solution to this.
server {
listen 443;
listen [::]:443;
server_name shiny.nginx.server;
ssl on;
ssl_certificate "/public/directory/proxy.pem";
ssl_certificate_key "/secret/directory/proxy.key";
location / {
proxy_pass http://very.old.server:80/;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
auth_basic "Username and Password Required";
auth_basic_user_file /etc/nginx/.htpasswd;
if ($remote_addr ~ 192.168.1.1) {
set $is_user Y;
}
if ($http_soapaction ~ SOAP/Request/Goes/Here) {
set $user_allowed Y;
}
set $user_test "${is_user}${user_allowed}";
if ($user_test = YY) {
set $good yes;
}
if ($good != yes) {
return 403;
}
}
Not being able to do nested or complex tests made it a bit tricky...

How to change request_uri in nginx proxy_pass?

I am running a django application through gunicorn via unix socket and I have my nginx configuration which looks like this :
Current NGINX config File :
upstream django_app_server {
server unix:/django/run/gunicorn.sock fail_timeout=0;
}
server{
listen 80;
server_name demo.mysite.com;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://django_app_server;
break;
}
}
}
so my django is running on a unix socket here , lets say if it was running on localhost then it has a url which looks like :
http://127.0.0.1:8000/demo/app1
http://127.0.0.1:8000/demo/notifications
Main goal
so what i want to do is , when someone visit http://demo.mysite.com/app1 they can access via proxy pass the url : http://127.0.0.1:8000/demo/app1
It would have been really easy if i would be running django on localhost tcp port and i could have easy done this and it would have worked for me :
server{
listen 80;
server_name demo.mysite.com;
location / {
proxy_pass http://127.0.0.1:8000/demo/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
How do i achieve this with my current nginx configurtion ?
One approach is to use rewrite ... break, for example:
location / {
try_files $uri #proxy;
}
location #proxy {
rewrite ^ /demo$uri break;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://django_app_server;
}
See this document for details.

Nginx Reverse Proxy Redirect Issues

I have been struggling with this for some time now. My reverse proxy install essentially looks like this:
global_nginx: http://192.168.115.200/client1/ => docker_nginx: http://localhost:8877/ => docker_app: http://app:8080/
Reverse proxy is working fine, but I have trouble when my app is sending redirects. Essentially the app itself is just a Spring MVC application that redirects as following:
#Controller
public class Ctrl {
#RequestMapping(value = { "/" })
public String redir() {
return "redirect:home";
}
}
The result is that when I browse http://192.168.115.200/client1/ I am redirected to: http://192.168.115.200/home/ but should be redirected to http://192.168.115.200/client1/home/
This is the global_nginx conf:
location /client1 {
proxy_pass http://localhost:8877/.;
proxy_redirect http://localhost:8877/ /;
port_in_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
And this is my docker_nginx conf:
location / {
proxy_pass http://app:8080/;
proxy_redirect http://app:8080/ /;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
You need to add the /client1 prefix in your proxy_redirect directive.
Maybe:
proxy_redirect http://$host:8877/ /client1/;
You can have more than one proxy_redirect directive, if you need to match multiple conditions. See this document for details.

Nginx: dynamic proxy from url to subdomain

I'm trying to write a nginx conf to dynamically redirect a url to a webapp, based on the uri. I use proxypass directive.
For example, I want redirect http://www.example.com/clientname/ to http://clientname.internaldomain.local/webapp
So far, I only succeded replacing new host, but the uri is broken because I can't split it. (Now I can't even replace the host, dunno why ...)
Here is my actual non working conf:
server {
resolver 192.168.137.71;
listen 80;
server_name tomservpa1;
location ~ (^\/(.*)\/) {
set $ccehost $2;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://$ccehost.internaldomain.local:9780/webapp/$new_request_uri;
proxy_redirect off;
proxy_set_header Host $host;
}
}
Did someone already do something like that ?
Thanks.
You can use named captures in the location directive is avoid scope and ambiguity issues. For example:
location ~ ^/(?<ccehost>[^/]+)/(?<new_request_uri>.*)$ {
rewrite ^ /webapp/$new_request_uri break;
...
proxy_pass http://$ccehost.internaldomain.local:9780;
...
}
this is the way to do it: (set in production)
location ~* ^/(?<ccehost>.*)/(.*) {
proxy_intercept_errors on;
proxy_cookie_path ~*^/.* /;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_pass http://$ccehost.app.ptf:8080/app/$2;
}