How to use non-standard ports? - moovweb

I'm trying to get moov running on port 8080 but am getting the error:
$ curl -s -i http://mlocal.nytimes.com:8080/
HTTP/1.0 534 Internal Server Error
Connection: close
Content-Type: text/plain;
Content-Length: 69
Host header 'mlocal.nytimes.com:8080' did not match project rewriters
I'm starting the server with:
$ sudo moov server -p=8080 --auto-hosts
(It appears to work fine on port 80.)

There's an additional step you need to take when you manually specify a port to run on.
Go into the project files and open config.json
Append :8080 to the domain names specified like this:
"$.nytimes.com:8080 => www.nytimes.com",
etc..

Related

Wget works for some websites but not for others?

I am on centos 8 and using the wget command to download some files, I am able to do so on certain websites but not on others. here is an example that works for me
wget https://forums.centos.org/index.php
--2021-07-26 20:50:40-- https://forums.centos.org/index.php
Resolving forums.centos.org (forums.centos.org)... 35.178.235.168
Connecting to forums.centos.org (forums.centos.org)|35.178.235.168|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.php’
index.php [ <=> ] 49.80K 319KB/s in 0.2s
2021-07-26 20:50:41 (319 KB/s) - ‘index.php’ saved [50997]
and here is an example that doesn't
wget -d https://www.bioinformatics.babraham.ac.uk/projects/fastq_screen/_build/html/index.html
DEBUG output created by Wget 1.19.5 on linux-gnu.
Reading HSTS entries from /home/tuser1/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'index.html' (UTF-8) -> 'index.html' (UTF-8)
--2021-07-26 20:27:16-- https://www.bioinformatics.babraham.ac.uk/projects/fastq_screen/_build/html/index.html
Certificates loaded: 147
Resolving www.bioinformatics.babraham.ac.uk (www.bioinformatics.babraham.ac.uk)... 149.155.133.4
Caching www.bioinformatics.babraham.ac.uk => 149.155.133.4
Connecting to www.bioinformatics.babraham.ac.uk (www.bioinformatics.babraham.ac.uk)|149.155.133.4|:443... Closed fd 3
failed: Connection timed out.
EDIT: I have tried pinging as well, I can ping yahoo.com but cannot ping google.com. Some websites are working some are not with ping as well.
I have disabled the firewall (firewalld) and tried to use the curl -O command as well to download but have not found a solution for this. Please let me know if there is any way to fix this

How to add a package to the XQuery repository?

I am using cURL to execute commands using the REST interface of BaseX like this:
curl http://localhost:8984/rest/?command=repo+list
There are also commands to manage the XQuery module repository. I am especially interested in REPO INSTALL to install a package. Is it somehow possible to execute this command using cURL and the REST interface but without having the package already on the target server? I want to provide the file in the body of the cURL request, similar to adding a resource to a database (source) which goes like this:
curl -i -X PUT -T "etc/xml/factbook.xml" "http://localhost:8984/rest/factbook"
Trying
curl -i -X PUT -T "/tmp/foo.xar" http://localhost:8984/rest/?command=repo+install
Gives me
HTTP/1.1 404 Not Found
Content-Type: text/plain;charset=UTF-8
Content-Length: 18
Connection: close
Server: Jetty(9.4.18.v20190429)
No path specified.
Adding -H "Content-Type: application/x-xar" does not help either.
And replacing PUT with POST gives me
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
Date: Tue, 03 Mar 2020 09:53:21 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 46
Server: Jetty(9.4.18.v20190429)
"" (Line 1): Content is not allowed in prolog.
The following works in case of standard modules (replace user/pass/server if needed):
$ curl http://admin:admin#localhost:8984/rest/?command=repo+install+http://www.xqueryfunctions.com/xq/functx-1.0.1-doc.xq

In Minikube Buildroot OS wget: not an http or ftp url

I have Setup minikube in my Machine using Hyper-v in windows 10. All working fine, but when i tried to setup fannel network i execute following commannd.
wget http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Oupput:-
Connecting to raw.githubusercontent.com (151.101.192.133:80)
wget: not an http or ftp url: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
I tried some solution like to install wget and hash -r but not working.
Any idea or suggestion to solve this.
Thank you,
Note that the actual url your wget is trying to connect to is not an http but https url and the output you attached says:
wget: not an http or ftp url
which is true as https is neither http nor ftp url. It looks like your wget version supports only two mentioned protocols.
You can easily check it by issuing following commands:
wget -V | grep https
and
wget -V | grep ssl
I tried to reproduce it on a system possibly similar to the one you're using. For this purpose I created a buildroot Pod from advancedclimatesystems/docker-buildroot image:
kubectl run --generator=run-pod/v1 buildroot --image=advancedclimatesystems/docker-buildroot --command sleep 3600
and I attached to it by:
kubectl exec -ti buildroot /bin/sh
Once there, I tested out your wget command and it was successful. It's output in my system looks like this (note the 301 redirection to https url):
root#buildroot:~/buildroot# wget http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
--2019-12-31 16:04:27-- http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml [following]
--2019-12-31 16:04:27-- https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14416 (14K) [text/plain]
Saving to: 'kube-flannel.yml'
kube-flannel.yml 100%[=====================================================================================================================>] 14.08K --.-KB/s in 0s
2019-12-31 16:04:27 (53.3 MB/s) - 'kube-flannel.yml' saved [14416/14416]
As you can see it has built-in ssl and https support:
root#buildroot:~/buildroot# wget -V | grep ssl
+ntlm +opie +psl +ssl/openssl
-Wl,-z,relro -Wl,-z,now -lpcre -luuid -lidn2 -lssl -lcrypto -lpsl
ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a
root#buildroot:~/buildroot# wget -V | grep https
-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls

Unable to create marklogic rest api instance

I am trying to create rest-api instance with the following configuration:
rest-api.json
{
"rest-api": {
"name": "restdb-api",
"database": "restdb",
"port": "8003",
"xdbc-enabled": true,
"forests-per-host": 1,
"error-format": "json"
}
}
curl --anyauth --user admin:admin -i -X POST -d #"./REST/rest-api.json" -H "Content-type: application/json" http://localhost:8002/LATEST/rest-apis
The endpoint returns 201 created, but I am unable to access the created endpoint at http://localhost:8003. I have tried using other ports, but the same thing is happening. The port 8003 is not listening. Please help me solve this problem.
HTTP/1.1 401 Unauthorized
Server: MarkLogic
WWW-Authenticate: Digest realm="public", qop="auth", nonce="36473d01f5e45a:ND9/6NHD0sw9o2y/xad/uQ==", opaque="e9594a1b7e019a97"
Content-Type: text/html; charset=utf-8
Content-Length: 209
Connection: Keep-Alive
Keep-Alive: timeout=5
HTTP/1.1 201 Created
Server: MarkLogic
Content-Length: 0
Connection: Keep-Alive
Keep-Alive: timeout=5
Since you said you are running on a local docker container, you might need to publish the port.
See Docker Expose
Please note from the link that - "The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports."
Hope that helps!

Unable to configure nginx as mail proxy

I need to use nginx as a mail proxy. I am completely new to nginx and need some help with the configuration.
Here is what I did:
First I built a service that mocks the authentication services described here: http://wiki.nginx.org/NginxMailCoreModule. For example,
curl -v -H "Host:auth.server.hostname" -H "Auth-Method:plain" -H "Auth-User:user" -H "Auth-pass:123" -H "Auth-Protocol:imap" -H "Auth-Login-Attempt:1" -H "Client-IP: 192.168.1.1" http://localhost:8080/authorize
returns the following response header:
< HTTP/1.1 200 OK
< Content-Type: text/html;charset=ISO-8859-1
< Auth-Status: OK
< Auth-Server: 192.168.1.10
< Auth-Port: 110
Second I installed nginx on my mac after installing macports:
$ sudo port -d selfupdate
$ sudo port install nginx
Third I created an nginx.conf with the following:
worker_processes 1;
error_log /var/log/nginx/error.log info;
mail {
server_name <my mail server here>;
auth_http http://localhost:8080/authorize;
pop3_auth plain apop cram-md5;
pop3_capabilities "LAST" "TOP" "USER" "PIPELINING" "UIDL";
xclient off;
server {
listen 110;
protocol pop3;
proxy on;
proxy_pass_error_message on;
}
}
Here is what I got running nginx:
$ nginx -V
nginx version: nginx/1.2.4
configure arguments: --prefix=/opt/local --with-cc-opt='-I/opt/local/include -O2' --with-ld-opt=-L/opt/local/lib --conf-path=/opt/local/etc/nginx/nginx.conf --error-log-path=/opt/local/var/log/nginx/error.log --http-log-path=/opt/local/var/log/nginx/access.log --pid-path=/opt/local/var/run/nginx/nginx.pid --lock-path=/opt/local/var/run/nginx/nginx.lock --http-client-body-temp-path=/opt/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/opt/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/opt/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/opt/local/var/run/nginx/uwsgi_temp --with-ipv6
$ nginx
nginx: [emerg] unknown directive "mail" in /opt/local/etc/nginx/nginx.conf:6
The only mention of that error on the web brings up a discussion in Russian...
My questions:
Why am I getting this unknow directive?
Does my config look correct at first sight or am I missing some key component for the mail proxy to work using the authentication approach described here: http://wiki.nginx.org/NginxMailCoreModule?
I got the mail proxy working so I will answer my own questions for future reference:
nginx doesn't install support for mail by default
The following is needed for nginx to process the mail directive:
$ sudo port edit nginx
==> add --with-mail at the end of the config parameters
Then (re)install nginx
In the config I included, I was missing the events:
events {
worker_connections 1024;
}
An important clarification that got me stuck for a while: the authentication service (specified with auth_http) needs to return the mail server expressed as an IP address, not a host name.
Obviously for nginx to proxy on both inbound and outbound traffic, the smtp listener needs to be added. Similar approach as with the pop3 configuration. In my case, I used port 2525, so I had
server {
listen 2525;
protocol smtp;
}