How to connect to Dataproc jupyternotebook in VPC-SC - google-cloud-dataproc

Without VPC-SC, I can connect to Jupyter lab on dataproc. However when the VPC-SC is turned on, then the web component is disabled. I am aware of the VPC-SC limitation, however, what is the work around solution?

VPC-SC is designed to not permit data transfer across the security perimeter and this is essentially what any workaround would be.
VPC-SC is not currently supported but we are working on providing first-class support.

The workaround is to use an SSH tunnel to connect to the UI.
Since you mentioned accessing the UI without VPC-SC, I assume that you installed the Jupyter component and checked the "enable component gateway" button when you created your cluster.
If that is the case, then the port you want to connect to on the master node is 8443 and the URL path you want to use is "/jupyter/lab/".
So, for example, if you use the "GCLOUD COMMAND" options on the page I linked to, then the URL you would use after setting it up would be http://<your-cluster-name>-m:8443/jupyter/lab

Related

starting Mock Server automatically when running restbird docker container

i want to use the "Mock Server" feature provided with https://restbird.org/ .
When starting restbird via the provided docker image, it listens on localhost:8080 by default.
Unfortunately the configured "Mock Server" instances still need to be started via the Web Frontend as it is described in the documentation here:
https://restbird.org/docs/mock-run.html#run-mock-server
Is there a way to automatically start the "Mock Server" instances when running the docker image without logging in into the backend (via admin/admin) and clicking the "start" button ?
Reason is, i want to start the mock server inside a gitlab pipeline where i have no further interaction possibilities after the container has been started.
I can not find anything like this in the documentation.
Thanks a lot for any clues - or if it is not possible i cam make a feature request.
I have found the solution myself. One can start a specific Mock Server as described in:
https://restbird.org/docs/API-MockServer.html#start-specific-mockserver-case
This can be scripted into my pipeline after executing the docker run command.

Whatsapp Business API production setup not working

I am trying to configure or setup the production environment of whatsapp business api as mentioned in the link https://developers.facebook.com/docs/whatsapp/installation/prod-single-instance
I have done everything mentioned in this my dockers are also running on port:9090 as can be seen in the image
still I can't access it. Whenever I try to call https://localhost:9090 the error with "This site can’t be reached" occurs. Whatsapp business api does not have good documentation or tutorials till now. So this site is the only last way for me.
I had a similar problem which could be your case, I saw the docker containers OK but nothing was working. After a day searching I saw where it happened, my problem was I installed mysql MANUALLY (not docker container) in the same instance where docker is running and in db.env I just used 127.0.0.1, this was passed literally to docker container, then looking at a the wait_on_mysql.sh script, the whastapp docker containers were waiting util the mysql ip has conectivity to actually do something and was printing "MySQL is not up yet - sleeping" each second, of course they wouldn't find any conectivity.
Since my instalation is for development, and I am already using such database to other stuff, my solution was to use the 172.17.0.1(docker gateway of the containers) IP instead, then add two sets of network iptables rules to the host to redirect from the docker containers IP to the IP binded by mysql when using such port (3306, the default in my case). After that everything works well. I think there are better solutions, but I didn't want to go far on it, you should evaluate you case if apply.
check the command:
docker-compose logs > debug_output.txt
That gives you insight about whats happening, hope it can helps someone.
I think your setup is already complete. You just need to start with the registration process and start sending messages. The containers are up and running but calling https://localhost:9090 won't send you any response as this is not any specified API endpoint expected to be used.
Since you're using prod single instance, the documentation can be found here which seems pretty straight forward. https://developers.facebook.com/docs/whatsapp/installation/prod-single-instance
You seem to have completed till the 7 steps. The next step can be to perform a health check to make sure it is healthy. The API endpoint for that would be https://localhost:9090/v1/health https://developers.facebook.com/docs/whatsapp/api/health
Has your db also been setup?
I cannot see it in the docker screenshot.
Also - you have to accept the certificate, as it does not have a public CA issues certificate.

Sophos UTM VPN not accessible

I used the Sophos UTM 9.510 ha_standalone Cloudformation template (https://github.com/sophos-iaas/aws-cf-templates/blob/master/utm/9.510/standalone.template) and used defaults when possible. I did not use an existing ElasticIP, so it created it's own at (scrubbed) 50.12.12.123.
I gave a hostname at (for example) vpn.example.com and after creation, I created an A record for vpn.example.com to point to 50.12.12.123.
I don't have a license and just pay hourly for the AMI.
I understand that I should be able to hit https://vpn.example.com:4444 or https://50.12.12.123:4444 to see the admin panel. However, it times out and doesn't load anything.
When I deployed the stack, I got an email at the admin email I provided and it said REST daemon not running - restarted. I assume it restarted fine, since I have received no new emails, and the EC2 instance is running.
Has anyone else experienced this? Is there a step I'm missing? Aside from creating the Route53 record, I thought the Cloudformation Template should just work right out of the box.
The default security groups blocked traffic. I modified one of them to accept all traffic and the dashboard became accessible. I will now refine access further.

Location Specific Github Proxy

I have a work laptop that resides between a painful corporate proxy during the day. When I'm at home, I don't have to worry about a proxy unless I VPN in.
Is there a way to set-up an automatic github proxy, such that if I'm at work it'll use the corporate proxy, and if I'm at home, it'll remove proxy settings?
Or perhaps a way that attempt 1 is made with a proxy, and attempt 2 is made without?
Thanks for any suggestions!
You can set it manually through pre-defined scripts/functions, as described here:
See nwinkler/bash-it/plugins/available/proxy.plugin.bash
"When working from the office (where I have to use a proxy), I simply call enable_proxy, and when working from home, I call disable_proxy", as detailed here.
You could wrap this in a test, which tries a curl without and then with HTTP(S)_PROXY variable, in order to see which call is successful.
On Linux, that test could be part of your .bashrc, which would allow any new shell session to open itself with the right settings set or not.

AWS deployment without using SSH

I've read some articles recently on setting up AWS infrastructure w/o enabling SSH on Ec2 instances. My web app requires a binary to run. So how can I deploy my application to an ec2 instance w/o using ssh?
This was the article in question.
http://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/
Although doable, like the article says, it requires to think about servers as ephemeral servers. A good example of this is web services that scale up and down depending on demand. If something goes wrong with one of the servers you can just terminate your server and spin up another one.
Generally, you can accomplish this using a pull model. For example at bootup pull your code from a git/mecurial repository and then execute scripts to setup your instance. The script will setup all the monitoring required to determine whether your server and application are up and running appropriately. You would still need an SSH client for this if you want to pull your code using ssh. (Although you could also do it through HTTPS)
You can also use configuration management tools that don't use ssh at all like Puppet or Chef. Essentially your node/server will pull all your application and server configuration from the Puppet master or the Chef server. The Puppet agent or Chef client would then perform all the configuration/deployment/monitoring changes for your application to run.
If you with this model I think one of the most critical components is monitoring. You need to know at all times if there's something wrong with one of your server and in the event something goes wrong discard the server and spin up a new one. (Even better if this whole process is automated)
Hope this helps.