How to pass run id of experiment as tag information of model ?
I want to run experiment and register model with tag information with run id of experiment in az ml cli in Azure DevOps Build pipeline.
run experiment
az ml run submit-script -e test -d myenv.yml train.py
model register
az ml model register -n mymodel -p sklearn_regression_model.pkl --tag "run id"= ????
I can't figure out how to get run id from experiment run from az ml cli and pass it to --tag argument. Any idea ?
Thanks you all.
My requirements have changed and was able to code in Azure DevOps Pipeline.
With option -t run.json, experiment run informationn is stored in run.json
az ml run submit-script -e $(experiment) -d myenv.yml -t run.json train-titanic.py
I want to register model outside the experiment run using run.json.
az ml model register --name mlops-model --experiment-name $(experiment) -f run.json -t ../release-pipeline/model.json --asset-path outputs/decision_tree.pkl
enter image description here
The run ID information is passed along automatically if you register a model from a run. You do not need to manually tag it.
az ml run list --experiment-name experiment
This command returns a list of details about runs for this experiment , the run id should also be included.
To add or update a tag, use the following command:
az ml run update -r runid --add-tag quality='fantastic run'
For details ,please refer to this docs.
Related
I'm using below command in my terminal.(Im following the cypress support doc)
cypress run --env host=kevin.dev.local,api_server=http://localhost:8888/api/v1
my spec is like this.
I'm expecting to set variables as below:
host=kevin.dev.local
api_server=http://localhost:8888/api/v1
But it does not set the value of the "api_server". Instead it sets host with both values as below:
host=kevin.dev.local http://localhost:8888/api/v1
Pls support to get this resolved.
You shell might interpret some of the characters before passing them to Cypress.
When I run in Powershell the following command:
> cypress open --env a=1,b=2
I'll end up in the same situation like described.
When I run:
> cypress open --env "a=1,b=2"
It will correctly set two env variables a and b with correct values.
So, try using double quotes.
I would like to execute "cpdctl dsjob" explained in the document below URL.
https://dataplatform.cloud.ibm.com/docs/content/dstage/dsnav/topics/cli.html
I download cpdctl from below URL but can not use dsjob option.
https://github.com/IBM/cpdctl/releases/
Is there additional procedure for using dsjob option?
I setup configration and run "cpdctl job run" but "cpdctrl dsjob" option is not appear.
# ./cpdctl dsjob
Error: unknown command "dsjob" for "cpdctl"
Did you mean this?
job
Run 'cpdctl --help' for usage.
unknown command "dsjob" for "cpdctl"
Did you mean this?
job
I found that CPDCTL_ENABLE_DSJOB environment is required to use dsjob option.
export CPDCTL_ENABLE_DSJOB=1
# ./cpdctl dsjob Error: unknown command "dsjob" for "cpdctl"
Did you mean this?
job
Run 'cpdctl --help' for usage.
unknown command "dsjob" for "cpdctl"
Did you mean this?
job
# export CPDCTL_ENABLE_DSJOB=1
# ./cpdctl dsjob
The IBM DataStage service provides APIs to manage jobs, this command provides some of legacy dsjob commandline functionality.
Usage:
cpdctl dsjob [command]
Available Commands:
lprojects List Projects.
ljobs List Jobs.
run Manage Job Runs.
logdetail Print DataStage job run logs.
logsum Print Summary of Datastage job run logs.
lognewest Returns newest event id from DataStage job run logs.
jobinfo List Job Information.
migrate Migrate a exported legacy isx file into nextgen Datastage project.
lflows List DataStage flows.
compile Compile DataStage flows.
lenvs List Environments.
lhws List Hardware Specifications.
env-create Create Environment.
hwspec-create Create Hardware Specification.
Flags:
-h, --help help for dsjob
Global Flags:
--context string
--cpdconfig string
--output-path string
--raw-output
Use "cpdctl dsjob [command] --help" for more information about a command.
#
In my powershell script, I call the following azure function:
az repos import create --git-source-url https://incommunities#dev.azure.com/my-organisation/Templates/_git/$($Framework) --detect true --project $ProjectAlias --repository $ProjectAlias --requires-authorization
When running, it prompts the user for a Password/PAT token, e.g:
Git Password / PAT:
Is there a way to automatically pass the password/token to the user input without having to enter manually?
I attempted to pipe the value through, however this does not seem to work e.g
my-pat-token | az repos import create --git-source-url https://incommunities#dev.azure.com/my-organisation/Templates/_git/$($Framework) --detect true --project $ProjectAlias --repository $ProjectAlias --requires-authorization
Is this both a) possible and then if so b) how can I do this?
There are two approaches you can use, both come courtesy from this nice blog post which you'll probably want to read, as it talks about a bunch of Azure Devops tasks.
Use an environmental variable
These commands will check for the presence of an environmental variable and will use it instead of prompting.
To do this, set an environment variable called AZURE_DEVOPS_EXT_PAT to the value of your PAT. (More info on how these tokens work here from the Microsoft Docs)
# set environment variable for current process
$env:AZURE_DEVOPS_EXT_PAT = 'xxxxxxxxxx'
When you're automating things, just set this variable before running the Azure commands.
Pipe the value in
I am not as big of a fan of this sort of approach but you can "echo out" the PAT value and pipe that into a command, which might work. IMHO this is more fragile and frunky and I wouldn't advise it.
$pat | az devops login
I am running the following in a CLI task on Azure Devops (inline)
rem Create the Service Plan
call az appservice plan create --resource-group %RESOURCE_GROUP% --name %SERVICE_NAME% --sku B1
Whch works just fine. However, I'm having to scroll to see the whole thing and I have other commands which are even longer. So I'm trying to split it over multiple lines so I can see more clearly what is going on.
Looking at Microsoft Docs it looked like the solution was to put a backslash on the end of each line. So I tried:
I've tried:
rem Create the Service Plan
call az appservice plan create \
--resource-group %RESOURCE_GROUP% \
--name %APP_SERVICE_NAME% \
--sku B1
Which didn't work. I then read something that recommended the back-tick/back-quote at the end of each line:
rem Create the Service Plan
call az appservice plan create `
--resource-group %RESOURCE_GROUP% `
--name %APP_SERVICE_NAME% `
--sku B1
This also didn't work. Any help greatly appreciated.
Never mind. Worked it out. Turns out you need to use '^'
rem Create the Service Plan
call az appservice plan create ^
--resource-group %RESOURCE_GROUP% ^
--name %APP_SERVICE_NAME% ^
--sku B1
For me the ` character is working. I posted earlier that it wasn't but I was inadvertently using the wrong character.
(That post seems to have been deleted by a moderator though I'm not sure why - perhaps because it was more of a "me too" than an answer, but it did add that the above "^" solution doesn't work for me).
In case it was the same issue as I had, ensure the "Backtick" you use is the correct one (it's immediately to the left of "1" on my microsoft keyboard). I was trying to use a ' in the failed attempt as the CLI docs were reading made it hard to distinguish.
The character that you need to use to specify that the command is split over multiple lines, depends on the environment in where the CLI command is run.
In other words, it depends on the scriptType property of the AzureCLI#2 command.
Powershell/Powershell Core: use a backtick `
bash: use backslash \
batch: use accent circonflexe ^
from azure cloud shell, type AZ, then copy paste the az command with \ for multiple line will not work.
but there is a fix, you click on the + sign like a adding a folder sign, it will bring a full page Azure Cloud Shell window, change to Bash from the pull down (default is Poweshell), then you see prompt changed to name#Azure:~$, now you can use az commend with \ for multiple lines.
I've been trying to store the output of multiple az cli commands in a variable defined in my pipeline with 0 success.
This being my last attempt:
The way I try to make sure is getting pass to the var is by doing an echo, which it outputs this(in all attempts):
At the end what im trying to achieve is to get the key value stored to use later:
Any suggestions on how to do this in the Azure CLI task from Azure DevOps Pipeline?
PS: Have being trying some commands from shell and batch and must of the attempts failures are related to not recognizing commands(batch/shell) inside the script. Which is confusing since in Azure cli task Docs:
Answer
#4c74356b41 Answer helped a lot since I didn't know I could do query in azure cli commands to get a specific value of a command. But it didn't quite answer my questions. All that said, this link Set Output Variable in Azure CLI task on VSTS has the Answer to my question.
just use query path filtering, something like this:
--query 'properties.properties.sites[0].key' -o tsv
this should output only the key you are interested it. reading:
https://learn.microsoft.com/en-us/cli/azure/query-azure-cli?view=azure-cli-latest