I'm trying to move to windows PowerShell instead of cmd
One of the commands I run often is for connecting to GCP compute engines using ssh and binding the machine's ports to my local machine.
I use the following template (taken from GCP's docs):
gcloud compute ssh VM_NAME --project PROJECT_ID --zone ZONE -- -L LOCAL_PORT:localhost:REMOTE_PORT -- -L LOCAL_PORT:localhost:REMOTE_PORT
This works great when using cmd but when I try and run it in PowerShell I get the following error:
(gcloud.compute.ssh) unrecognized arguments:
-L
8010:localhost:8888
What am I missing?
Related
I have a google cloud compute instance that I connect to with
INSTANCE_NAME='sam_vm'
gcloud beta compute ssh --zone "us-central1-a" $INSTANCE_NAME --project sam_project
when I try to connect with cmd-p then typing in that command, I get:
ssh: Could not resolve hostname gcloud beta compute ssh --zone "us-central1-a" "shleifer-v1-vm" --project $hf_proj: nodename nor servname provided, or not known
How can I connect?
run
gcloud compute config-ssh
then in the vscode remote-ssh popup,
ssh {instance_name}.{zone}.{project_name}
Even though it has been 2 years since this questions has been asked, I am providing instructions for those who are still trying to figure out how to set up ssh to GCP in VSCode on macOS.
Once you have Google Cloud CLI installed and have been granted access to the VM instance, login to your Google Cloud account by running in terminal:
gcloud auth login
Setup following variables which will be used in gcloud commands:
gcloud config set project <your project name>
gcloud config set compute/zone <your GCP zone>
Then you can establish SSH tunnel between VM and your local machine:
gcloud compute ssh [YOUR-VM-NAME] --tunnel-through-iap
Setup VS Code and install Remote Development extension.
Run the following command in terminal to obtain internal SSH command used by gcloud compute ssh to connect to VM:
gcloud compute ssh [YOUR-VM-NAME] --tunnel-through-iap --dry-run
The output may look similar to following:
/usr/bin/ssh -t -i /Users/<User_ID>/.ssh/google_compute_engine -o CheckHostIP=no
-o HashKnownHosts=no -o HostKeyAlias=compute.8972912327831725343 -o IdentitiesOnly=yes
-o StrictHostKeyChecking=yes
-o UserKnownHostsFile=/Users/<User_ID>/.ssh/google_compute_known_hosts
-o "ProxyCommand /Library/Frameworks/Python.framework/Versions/3.10/bin/python3
/Library/google-cloud-sdk/lib/gcloud.py compute start-iap-tunnel <instance name>
%p --listen-on-stdin --project=<your project name> --zone=<your zone> --verbosity=warning"
-o ProxyUseFdpass=no <Gcloud_Username>#compute.8972912327831725343
Copy this SSH command and remove ‘/usr/bin/’:
ssh -t -i /Users/<User_ID>/.ssh/google_compute_engine -o CheckHostIP=no
-o HashKnownHosts=no -o HostKeyAlias=compute.8972912327831725343 -o IdentitiesOnly=yes
-o StrictHostKeyChecking=yes
-o UserKnownHostsFile=/Users/<User_ID>/.ssh/google_compute_known_hosts
-o "ProxyCommand /Library/Frameworks/Python.framework/Versions/3.10/bin/python3
/Library/google-cloud-sdk/lib/gcloud.py compute start-iap-tunnel <instance name>
%p --listen-on-stdin --project=<your project name> --zone=<your zone> --verbosity=warning"
-o ProxyUseFdpass=no <Gcloud_Username>#compute.8972912327831725343
In VS Code:
Press 'Cmd' + 'Shift' + 'P'
Select 'Remote-SSH: Add New Host'
Press 'Enter'
Select the config file /Users/<User_ID>/.ssh/config
Press 'Cmd' + 'Shift' + 'P'
Select 'Remote-SSH: Connect Host'
Select Compute instance as Linux.
VS Code should establish SSH tunnel to the VM and you should be able to open root folder.
Hope that this will help someone save hours of efforts.
First execute gcloud compute config-ssh
Open ~/.ssh/config file
Find
Host {instance_name}.{zone}.{project_name}
HostName {ip_name_google_provided}
IdentityFile /Users/youruser/.ssh/google_compute_engine
UserKnownHostsFile=/Users/youruser/.ssh/google_compute_known_hosts
HostKeyAlias=compute.123423
IdentitiesOnly=yes
CheckHostIP=no
and convert to
Host {ip_name_google_provided}
HostName {ip_name_google_provided}
IdentityFile /Users/youruser/.ssh/google_compute_engine
UserKnownHostsFile=/Users/youruser/.ssh/google_compute_known_hosts
HostKeyAlias=compute.123423
IdentitiesOnly=yes
CheckHostIP=no
Simply use ip ({ip_name_google_provided}) at HostName as Host name and try to connect {ip_name_google_provided} as a solution.
Note: Problem is computer can not resolve DNS {instance_name}.{zone}.{project_name} from ssh config file.
I am trying to set up datalab from my chrome book using the following tutorial https://cloud.google.com/dataproc/docs/tutorials/dataproc-datalab. However when trying to set up an SSH tunnel using the following guidelines https://cloud.google.com/dataproc/docs/concepts/accessing/cluster-web-interfaces#create_an_ssh_tunnel I keep on receiving the following error.
ERROR: (gcloud.compute.ssh) Could not fetch resource:
- Project 57800607318 is not found and cannot be used for API calls. If it is recently created, enable Compute Engine API by visiting https://console.developers.google
.com/apis/api/compute.googleapis.com/overview?project=57800607318 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our sy
stems and retry.
The error message would lead me to believe my "Compute Engine API" is not enabled. However, I have double checked and "Compute Engine API" is enabled.
Here is what I am entering into the cloud shell
gcloud compute ssh ${test-cluster-m} \
--project=${datalab-test-229519} --zone=${us-west1-b} -- \
-4 -N -L ${8080}:${test-cluster-m}:${8080}
The ${} is for accessing the local environment variable. You set them in the step before with:
export PROJECT=project;export HOSTNAME=hostname;export ZONE=zone;PORT=number
In this case would be:
export PROJECT=datalab-test-229519;export HOSTNAME=test-cluster-m;export ZONE=us-west1-b;PORT=8080
Either try this:
gcloud compute ssh test-cluster-m \
--project datalab-test-229519 --zone us-west1-b -- \
-D 8080 -N
Or access the enviroment variables with:
gcloud compute ssh ${HOSTNAME} \
--project=${PROJECT} --zone=${ZONE} -- \
-D ${PORT} -N
Also check the VM you are trying to access is running.
I have installed gcloud/bq/gsutil command line tool in one linux server.
And we have several accounts configured in this server.
**gcloud config configurations list**
NAME IS_ACTIVE ACCOUNT PROJECT DEFAULT_ZONE DEFAULT_REGION
gaa True a#xxx.com a
gab False b#xxx.com b
Now I have problem to both run gaa/gab in this server at same time. Because they have different access control on BigQuery and Cloud Stroage.
I will use below commands (bq and gsutil commands):
Set up account
Gcloud config set account a#xxx.com
Copy data from bigquery to Cloud
bq extract --compression=GZIP --destination_format=NEWLINE_DELIMITED_JSON 'nl:82421.ga_sessions_20161219' gs://ga-data-export/82421/82421_ga_sessions_20161219_*.json.gz
Download data from Cloud to local system
gsutil -m cp gs://ga-data-export/82421/82421_ga_sessions_20161219*gz
If only run one account, it is not a problem.
But there are several accounts need to run on one server at same time, I have no idea how to deal with this case.
Per the gcloud documentation on configurations, you can switch your active configuration via the --configuration flag for any gcloud command. However, gsutil does not have such a flag; you must set the environment variable CLOUDSDK_ACTIVE_CONFIG_NAME:
$ # Shell 1
$ export CLOUDSDK_ACTIVE_CONFIG_NAME=gaa
$ gcloud # ...
$ # Shell 2
$ export CLOUDSDK_ACTIVE_CONFIG_NAME=gab
$ gsutil # ...
Is it possible to do silent deployment when using gcloud app deploy
When I run the command gcloud app deploy ./deployment/app.yaml --version v1 its always prompting for
Do you want to continue (Y/n)? Y
how to automate this?
is there any flag that we can pass in to mute this?
You're looking for the --quiet flag, available across all gcloud commands:
$ gcloud --help
--quiet, -q
Disable all interactive prompts when running gcloud commands. If input
is required, defaults will be used, or an error will be raised.
$ gcloud app deploy --quiet
Or also:
$ gcloud app deploy -q
I have been trying to apply my startup scripts to new Windows instances on Google Compute Engine as described here, however when I check the instances there is no trace of them ever being executed. Here is the gcloud command I am running:
gcloud compute instances create "my-instance"
--project "my-project"
--zone "us-central1-a"
--machine-type "g1-small"
--network "default"
--metadata "gce-initial-windows-user=my-user" "gce-initial-windows-password=my-pass"
--maintenance-policy "MIGRATE"
--scopes "storage-ro"
--tags "http-server" "https-server"
--image "https://www.googleapis.com/compute/v1/projects/windows-cloud/global/images/windows-server-2008-r2-dc-v20150110"
--boot-disk-type "pd-standard"
--boot-disk-device-name "my-instance"
--metadata-from-file sysprep-oobe-script-ps1=D:\Path\To\startup.ps1
I tried using all 3 startup types (sysprep-specialize-script-ps1, sysprep-oobe-script-ps1, windows-startup-script-ps1) but none worked. Can't see any indication in the Task Scheduler or Event Viewer either. The file on my system exists and does work when I run it manually. How can I get this working?
A good way to debug Powershell scripts is to have them write to the serial console (COM1). You'll be able to see the output of the script from GCE's serial port output.
gcloud compute instances get-serial-port-output my-instance --zone
us-central1-a
If there's no script you'll see something like:
Calling oobe-script from metadata.
attributes/sysprep-oobe-script-bat value is not set or metadata server is not reachable.
attributes/sysprep-oobe-script-cmd value is not set or metadata server is not reachable.
attributes/sysprep-oobe-script-ps1 value is not set or metadata server is not reachable.
Running schtasks with arguments /run /tn GCEStartup
--> SUCCESS: Attempted to run the scheduled task "GCEStartup".
-------------------------------------------------------------
Instance setup finished. windows is ready to use.
-------------------------------------------------------------
Booting on date 01/25/2015 06:26:26
attributes/windows-startup-script-bat value is not set or metadata server is not reachable.
attributes/windows-startup-script-cmd value is not set or metadata server is not reachable.
attributes/windows-startup-script-ps1 value is not set or metadata server is not reachable.
Make sure that contents of the ps1 file is actually attached to the instance.
gcloud compute instances describe my-instance --zone us-central1-a
--format json
The JSON dump should contain the powershell script within it.
Lastly, a great way to debug Powershell startup scripts is to write the output to the serial console.
You can print log messages and see them in the Google Developer Console > Compute > Compute Engine > VM Instances > (Instance Name). Then scroll to the bottom and click the expand option for "Serial console".
Function Write-SerialPort ([string] $message) {
$port = new-Object System.IO.Ports.SerialPort COM1,9600,None,8,one
$port.open()
$port.WriteLine($message)
$port.Close()
}
Write-SerialPort ("Testing GCE Startup Script")
This command worked for me, I had to make sure that the script was written in ascii. Powershell ISE writes with a different encoding that breaks gcloud compute.
gcloud compute instances create testwin2 --zone us-central1-a
--metadata-from-file sysprep-oobe-script-ps1=testconsole.ps1 --image windows-2008-r2