Run Arango Shell (Arangosh) on a Kubernetes pod - kubernetes

I have set up an Arango instance on Kubernetes nodes, which were installed on a VM, as mentioned in the ArangoDB docs ArangoDB on Kubernetes. Keep in mind, I skipped the ArangoLocalStorage and ArangoDeploymentReplication step. I can see 3 pods each of agent, coordinators and dbservers in get pods.
The arango-cluster-ea service, however, shows the external IP as pending. I can use the master node's IP address and the service port to access the Web UI, connect to the DB and make changes. But I am not able to access either the Arango shell, nor am I able to use my Python code to connect to the DB. I am using the Master Node IP and the service port shown in arango-cluster-ea in services to try to make the Python code connect to DB. Similarly, for arangosh, I am trying the code:
kubectl exec -it *arango-cluster-crdn-pod-name* -- arangosh --service.endpoint tcp://masternodeIP:8529
In case of Python, since the Connection class call is in a try block, it goes to except block. In case of Arangosh, it opens the Arango shell with the error:
Cannot connect to tcp://masternodeIP:port
thus not connecting to the DB.
Any leads about this would be appreciated.

Posting this community wiki answer to point to the github issue that this issue/question was resolved.
Feel free to edit/expand.
Link to github:
Github.com: Arangodb: Kube-arangodb: Issues: 734
Here's how my issue got resolved:
To connect to arangosh, what worked for me was to use ssl before using the localhost:8529 ip-port combination in the server.endpoint. Here's the command that worked:
kubectl exec -it _arango_cluster_crdn_podname_ -- arangosh --server.endpoint ssl://localhost:8529
For web browser, since my external access was based on NodePort type, I put in the master node's IP and the 30000-level port number that was generated (in my case, it was 31200).
For Python, in case of PyArango's Connection class, it worked when I used the arango-cluster-ea service. I put in the following line in the connection call:
conn = Connection(arangoURL='https://arango-cluster-ea:8529', verify= False, username = 'root', password = 'XXXXX')
The verify=False flag is important to ignore the SSL validity, else it will throw an error again.
Hopefully this solves somebody else's issue, if they face the similar issue.
I've tested following solution and I've managed to successfully connect to the database via:
arangosh from localhost:
Connected to ArangoDB 'http+ssl://localhost:8529, version: 3.7.12 [SINGLE, server], database: '_system', username: 'root'
Python code
from pyArango.connection import *
conn = Connection(arangoURL='https://ABCD:8529', username="root", password="password",verify= False )
db = conn.createDatabase(name="school")
Additional resources:
Arangodb.com: Tutorials: Tutorial Python
Arangodb.com: Docs: Stable: Tutorials Kubernetes

Related

Installing Cloud SQL Proxy on Dataproc

I'm trying to install the Cloud MySQL Proxy on a Dataproc cluster with the initialization-action given as an example here. I removed all the hive-metadata part because I don't need it. The proxy is installed but I can't connect to my instance. The created cloud-sql-proxy.service file looks fine:
> cat /usr/lib/systemd/system/cloud-sql-proxy.service
[Unit]
Description=Google Cloud SQL Proxy
After=local-fs.target network-online.target
After=google.service
Before=shutdown.target
[Service]
Type=simple
ExecStart=/usr/local/bin/cloud_sql_proxy -dir=/var/run/cloud_sql_proxy -instances_metadata=attributes/additional-cloud-sql-instances
[Install]
WantedBy=multi-user.target
Also, if I try to get the value of attributes/additional-cloud-sql-instances it looks fine:
> /usr/share/google/get_metadata_value attributes/additional-cloud-sql-instances
> myproject-12345:europe-west1:my-db-instance=tcp:3333
Yet I can't connect to the instance. When I stop the service and start it again I see an error message:
> /usr/local/bin/cloud_sql_proxy -dir=/var/run/cloud_sql_proxy -instances_metadata=attributes/additional-cloud-sql-instances
2017/07/03 09:23:44 Ready for new connections
2017/07/03 09:23:44 Error on receiving new instances from metadata: metadata: GCE metadata "attributes/additional-cloud-sql-instances" not defined
Am I doing something wrong?
In the meantime I can get this to work by using -instances=myproject-12345:europe-west1:my-db-instance=tcp:3333 instead of using the metadata key, but I wonder why it doesn't work as provided in the example.
There's a subtle difference in how the Cloud SQL proxy interprets flags and how get_metadata_value interprets its input.
When using get_metadata_value, the attribute to read is assumed to be relative to
http://metadata.google.internal/computeMetadata/v1/instance/
while in the case of the Cloud SQL proxy, all paths should be relative to:
http://metadata.google.internal/computeMetadata/v1/
So if you're intention is to have the Cloud SQL proxy read instance metadata, pass in:
-instances_metadata=instance/attributes/additional-cloud-sql-instances

Issue connecting composer to Blockchain on Bluemix - identity or token does not match

I have fabric composer 0.72 installed on my mac, and I was able to follow this thread to get it connected to my Blockchain (v.61 of Fabric) on Bluemix.
fabric-composer-integration-with-bluemix-blockchain-service
Now I am trying to build an ubuntu (16.04) docker container and run composer-rest-server there. When I try to connect to my blockchain service from my docker container (using the same id, WebAppAdmin, that I used on my mac) I get an error:
Discovering types from business network definition ...
Connection fails: Error: Identity or token does not match.
It will be retried for the next request.
{ Error: Identity or token does not match.
at /home/composer/.nvm/versions/node/v6.10.3/lib/node_modules /composer-rest-server/node_modules/grpc/src/node/src/client.js:417:17 code: 2, metadata: Metadata { _internal_repr: {} } }
I tried copying the cert from my mac to my docker container:
/home/composer/.composer-credentials/member.WebAppAdmin
but when I did that I got a different error that says "signature does not verify". I did some additional testing, and I discovered that if I used an id that I had not previously used with composer (i.e. user_type1_0) then I could connect, and I could see a new cert in my .composer-credentials directory.
I tried deleting that container and building a new one (I dorked something else up) I could not use that same userid again.
Does anybody know how security and these certs are supposed to work? It would seem as though something to do with certificate generation/validation is tied to the client (i.e. hardware address), such that if I try to re-use an id on a different machine, the certs or keys or something don't match. I have a way to make things work, but it doesn't seem like it's the right way if I can't use the same id from different machines.
Thanks!
Hi i tried to recreate this by having blockchain running on a unix machine and then i copied my connection profile and certificate to my mac and then edited my connection profile to update the ip address and key store. I then did a composer network ping and it worked fine.
I am using composer v0.7.4 so you could try that?
I have also faced this issue, and concluded that
There is inconsistent behavior while deploying network using composer on Cloud environment includeing Bluemix. Problem is not with composer, but with fabric 0.6.
I am assuming that this issue is also indirectly related to following known bugs into fabric 0.6, which will not be fixed in fabric 0.6.
ERROR:
"
throw er; // Unhandled 'error' event
^
Error
at ClientDuplexStream._emitStatusIfDone (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:189:19)
at ClientDuplexStream._readsDone (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:158:8)
at readCallback (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:217:12)
"
So far, We have understood that following three JIRA are root cause , where essentially the cloud networking layer ends up killing the idle event hub connection after a period of inactivity and the fabric SDK cannot handle this.
https://jira.hyperledger.org/browse/FAB-4002 FAB-3310
https://jira.hyperledger.org/browse/FAB-3310
or FAB-2787
Conclusion:
There is no alternative way of fixing this issue with Bluemix or any cloud environment with fabric 0.6
You may not experience this issue with Fabric 1.0, but there is still possibilities as all above mentioned defects are not fixed yet.

installing kubernetes on coreos with rkt and automated script

I'm trying to install kuberentes with rkt on my real (not virtual) coreos servers at home using the scripts at https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/generic and I have some questions.
my etcd2 is using tls keys, I can't see anywhere in the script where I can define where the certificates are located.
can I supply a domain instead of IP for ADVERTISE_IP and CONTROLLER_ENDPOINT ?
when I tried to install kubernetes manually I needed start the rkt service api. it doesn't state in the documents that it needed here, does it mean that I don't need it if I use these scripts? or is it just something that's missing in the documents?
thanks!
update
Rob thank you so much for your response. I wasn't clear enough regarding etcd2. I already have etcd2 tls installed and properly configured on my coreos servers. so I configured my etcd servers in the controller-install.sh file:
export ETCD_ENDPOINTS="https://coreos-2.tux-in.com:2379,https://coreos-3.tux-in.com:2379"
but when I run the controller-install.sh script, it returns and repeat the following output:
Waiting for etcd...
Trying: https://coreos-2.tux-in.com:2379
Trying: https://coreos-3.tux-in.com:2379
Trying: https://coreos-2.tux-in.com:2379
Trying: https://coreos-3.tux-in.com:2379
...
so I was guessing it's because i didn't define etcd related tls certificates in the controller script and that is why it stuck in that faze.
on my macbook pro laptop I have the following alias configured:
alias myetcdctl="~/apps/etcd-v3.0.8-darwin-amd64/etcdctl --endpoint=https://coreos-2.tux-in.com:2379 --ca-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/ca.pem --cert-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/etcd1.pem --key-file=/Users/ufk/Projects/coreos/tux-in/etcd/certs/certs-names/etcd1-key.pem --timeout=10s"
so when I run myetcdctl member list I get:
8832ce6a269a7dac: name=ccff826d5f564c67abf35467306f80a0 peerURLs=https://coreos-3.tux-in.com:2380 clientURLs=https://coreos-3.tux-in.com:2379 isLeader=true
a2c0ac9708ef90fc: name=dc38bc8f20e64940b260d3f7b260430d peerURLs=https://coreos-2.tux-in.com:2380 clientURLs=https://coreos-2.tux-in.com:2379 isLeader=false
so I'm guessing that I don't really have a problem there.
any ideas?
thanks!
my etcd2 is using tls keys, I can't see anywhere in the script where I can define where the certificates are located.
These scripts don't start an etcd server. You will need to set one up manually and will be able to use TLS and as many nodes as you would like. This isn't clear in the current form of the document, I will attempt a PR to fix.
can I supply a domain instead of IP for ADVERTISE_IP and CONTROLLER_ENDPOINT ?
Only CONTROLLER_ENDPOINT be a domain name.
when I tried to install kubernetes manually I needed start the rkt service api. it doesn't state in the documents that it needed here, does it mean that I don't need it if I use these scripts? or is it just something that's missing in the documents?
These scripts include/start the rkt API service. As you can see below, it also has a Restart parameter set (source):
[Unit]
Before=kubelet.service
[Service]
ExecStart=/usr/bin/rkt api-service
Restart=always
RestartSec=10
[Install]
RequiredBy=kubelet.service

Openwhisk build in local vagrant - apihost issue

Hi I standup a local openwhisk instance following https://github.com/openwhisk/openwhisk#alternate-instructions-for-mac-developers - which mostly worked.
I can now create user using wskadmin and see the tables created in cloudant instance -
But after that when I try to invoke or list any action or package I get
ErrNo -2 - Service or Command not known
wsk package list or wsk action list
when I use verbose mode I see apihost is sent blank.
Any idea what's up with this and what should be apihost to begin with (I think it should be localhost but not working)
The API host is your Vagrant machine IP address. From here: https://github.com/openwhisk/openwhisk#using-cli-from-outside-vagrant-machine
The virtual machine IP address is 192.168.33.13 (see Vagrant file). From your host, configure wsk to use your Vagrant-hosted OpenWhisk deployment and run the "echo" action again to test.
wsk property set --apihost 192.168.33.13 --auth
The same question and answer appear here: https://github.com/openwhisk/openwhisk/issues/127

ejabberd on openshift - Failed RPC connection to the node

I am trying to configure ejabberd on DIY cartridge on openshift, following the guide here:
Erlang and Ejabberd on OpenShift
I followed everything successsfully up to here:
Next you can start ejabberd running the following 2 commands, which you’ll want to put in your .openshift/action_hooks/start script
there is no error and ejabberd seems to be started, but the next command:
$OPENSHIFT_DATA_DIR/erl_home/sbin/ejabberdctl register admin localhost password1234
failed with this error:
Failed RPC connection to the node ‘ejabberd#127.7.131.1′: {‘EXIT’, {badarg, [{ets,lookup, [local_config, ejabberdctl_access_commands], []}, {ejabberd_config, get_local_option, 1, [{file, "ejabberd_config.erl"}, {line,590}]}, {ejabberd_ctl, get_accesscommands, 0, [{file, "ejabberd_ctl.erl"}, {line,236}]}, {ejabberd_ctl, process,1, [{file, "ejabberd_ctl.erl"}, {line,199}]}, {rpc, ‘-handle_call_call/6-fun-0-’, 5, [{file, "rpc.erl"}, {line,205}]}]}}
Commands to start an ejabberd node: start Start an ejabberd node in server mode debug Attach an interactive Erlang shell to a running ejabberd node live Start an ejabberd node in live (interactive) mode
Optional parameters when starting an ejabberd node: –config-dir dir Config ejabberd: /var/lib/openshift/52c9674d5973ca7734000180/app-root/data//erl_home/etc/ejabberd –config file Config ejabberd: /var/lib/openshift/52c9674d5973ca7734000180/app-root/data//erl_home/etc/ejabberd/ejabberd.cfg –ctl-config file Config ejabberdctl: /var/lib/openshift/52c9674d5973ca7734000180/app-root/data//erl_home/etc/ejabberd/ejabberdctl.cfg –logs dir Directory for logs: /var/lib/openshift/52c9674d5973ca7734000180/app-root/data//erl_home/var/log/ejabberd –spool dir Database spool dir: /var/lib/openshift/52c9674d5973ca7734000180/app-root/data//erl_home/var/lib/ejabberd –node nodename ejabberd node name: ejabberd#127.7.131.1
I am not sure what causes the error... it seems like it is trying to connect to localhost (due to the nodename: ejabberd#127.7.131.1). However, I have sed every localhost smell from the previous commands on the blog.
Anybody ever encounter this before? Any clue on how to debug is also highly appreciated, as I am not very familiar with openshift or ejabberd as well as linux... Thank you very much in advance!
I wrote the blog post, and saw this error on a new DIY Application I created. It was due to a bug with erlang and the openssl package in the openshift box. I put in a patch in the erlang source so that new compilations will work.
The bug that was causing the issue was here: https://bugzilla.redhat.com/show_bug.cgi?id=1023017
You'll need to run using your host instead of localhost.
$OPENSHIFT_DATA_DIR/erl_home/sbin/ejabberdctl register admin <replacewithyourhost> password1234
For that
OPENSHIFT_DATA_DIR/erl_home/sbin/ejabberdctl register admin $OPENSHIFT_DIY_IP password1234
However it returns error as well:
{error_logger,{{2014,1,13},{21,11,24}},"Protocol: ~tp: register/listen error: ~tp~n",["inet_tcp",econnrefused]} {error
above
You need to kill all the ejabberd process including epmd.
I had this problem one time, My solution was to edit /etc/hosts(for windows- C:\Windows\System32\drivers\etc\hosts) file and make sure that there was a hostname entry for my public ip address and the domain I wanted ejabber to respond to.
0.0.0.0 hostname.domain.com hostname
1.1.1.1(your ip) your-hostname.your-domain your-hostname