Running bash script within a project folder of AzureDevOps - azure-devops

I have a script that is nicely performing all kinds of dependency installation and some manual works (NPM installation, some manual steps to do while setting up project) to setup a project before it is able to run. The script runs perfectly fine in a local environment.
Im now trying to build my pipeline in Azure DevOps, I realized I can't just fire the script right away. Running npm install inside the script is not actually running within my project folder but it always runs on the path /Users/runner/work
Question:
How can I execute the script within my project folder?
Sample code in my script file
set -e
# Setup project dependencies
npm install
# some mandatory manual work
.....
# Pod installation
cd ios
pod install
My AzurePipelines.yml
- task: Bash#3
inputs:
targetType: 'inline'
script: |
sh $(System.DefaultWorkingDirectory)/projectFolder/setup.sh
failOnStderr: true
Issue log from Azure (as you can see, the npm installation is not working due to incorrect path, hence further actions within pipeline will fail)
npm WARN saveError ENOENT: no such file or directory, open '/Users/runner/work/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/Users/runner/work/package.json'
npm WARN work No description
npm WARN work No repository field.
npm WARN work No README data
npm WARN work No license field.

Firstly I'd advise you to split your script into different steps of a single job or multiple jobs with many steps because this makes it easier to parallel them in the future allowing you to speed up the build time.
In order to execute your script directly from the project folder you can leverage the option working directory:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
./setup.sh
failOnStderr: true
workingDirectory: "$(System.DefaultWorkingDirectory)/projectFolder/"
However in your case you could point directly to the script, without the need to run it as a "script"
- task: Bash#3
inputs:
targetType: 'filePath'
filePath: "$(System.DefaultWorkingDirectory)/projectFolder/setup.sh"
failOnStderr: true
workingDirectory: "$(System.DefaultWorkingDirectory)/projectFolder/"
ref.: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/bash?view=azure-devops

Related

Running Behave Tests in an Azure Pipeline

I am trying to run some Behave(Python Cucumber) tests in an azure pipeline and I am getting this error:
/Users/runner/work/_temp/f1131f4b-92a8-4c36-92bc-c9cd539f281c.sh: line 1: behave: command not found
##[error]Bash exited with code '127'.
Finishing: Run behave tests
I am running the tests localy on my machine and they work fine and run. I have the tests in an Azure Git repo and this is my Azure Pipeline YAML, I am a noobPotato and could use some help/guidence :)
trigger:
- main
pool:
vmImage: 'macOS-latest'
steps:
- script: |
python -m pip install --upgrade pip
displayName: 'Install dependencies'
- script: |
export PATH=$PATH:$(python -m site --user-base)/bin
pip install --user behave
displayName: 'Add behave to PATH and install'
- script: |
behave
displayName: 'Run behave tests'
I have ried various ways of installing behave with the -m flag etc and also different ways of adding it to the Path but I am stumped and could use some help!

Alternatives for gnu tar when using cache#2 task in Azure DevOps

I'm having a hard time getting cache#2 task working as it seems when the build agent is windows OS, it requires to have GNU Tar installed in the machine and this seems it doesnt seem to be an option right now. This needs to run in the windows machine as its part of a stage and requires to collect the build output to produce the artifact.
Somewhere I read this could be an option but it doesn't work and its not on Microsoft's documentation: tool: '7zip'
- task: Cache#2
displayName: Restore from cache
inputs:
key: 'npm | "$(Agent.OS)" | $(projectDir)package-lock.json'
#restoreKeys: |
# npm | "$(Agent.OS)"
path: $(Pipeline.Workspace)/.npm
cacheHitVar: CACHE_RESTORED
verbose: true
- script: |
npm install
displayName: Install dependencies #, Build and Lint
workingDirectory: $(projectDir)
condition: ne(variables.CACHE_RESTORED, 'true')
The log says: ##[error]Failed to start the required dependency 'tar'. Please verify the correct version is installed and available on the path. This is accurate but I was wondering if there's any workaround or if I just can have the cache implemented and I have to install all dependencies on each run.
Btw, I'm using the default pool and its using windows v10.0.14393 and agent version 2.206.1
Thanks!
If it helps to anyone, I had to split my workflow into API and FE stages. The angular stage would run in the ubuntu agent that would have the tar installed. That fixed my problem and I also had the 2 stages running in parallel.

Azure Pipelines: Where is the output of my script?

I tried to build my Angular 13 app on a self-hosted agent and created the following YAML snippet for this:
- task: NodeTool#0
displayName: 'Install Node.js'
inputs:
versionSpec: '14.x'
- script: |
npm install -g #angular/cli
npm install
ng build --configuration production --aot
displayName: 'npm install and build'
workingDirectory: '$(Build.SourcesDirectory)/src'
I can observe the /s directory of the agent _work-directory and after my task was running, there is no node_modules folder or dist folder inside.
But also no console output.
If I remove the line "npm install -g #angular/cli" from the line, a node_modules folder gets created, but no dist-folder.
I am pretty sure that the installation of angular cli fails, but I do not get any error output in my window.
It just looks like this:
How can I get more logs to find out why the angular cli is not installing correctly? I saw that the "script" file that is executed on the agent puts an #echo off by default in front of the script.
Why is that?
How can I get some output to find my problem?
To get more detailed log from the pipeline you can add the variable system.debug and set the value to true in your pipeline.
For YAML pipelines, you can select Variables in the upper right corner of the YAML edit page.
Add a new variable with the name System.Debug and value true.
For more info about logs, please refer to Review logs to diagnose pipeline issues.

Where do the builds go after the pipeline is run in Azure DevOps?

I'm very new to Azure DevOps. I'm running npm run build in the pipeline.
I'm wonder where the dist folder goes? How do I get access to it for further processing?
The build completes without error.
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build
displayName: 'npm install and build'
In the agent you have 3 folders: a for artifacts, s for sources and b for binaries.
When the build start all the code downloaded to the s folder, so if you run npm run build the dist folder created there.
How do you access it? there are environment variables for all the folders, to the s folder the variable is $(Agent.SourcesDirectory), so you can take the dist from there in another task by $(Agent.SourcesDirectory)/Your App/dist (or more deeper, depend your app structure).
You can find here the list of the environment variables.

Not Loading Indexed Sources for nuget packages from Azure Devpos during debugging

Trying to use azure yml for build pipleline to publish symbols to allow nuget pkg to be debuggable usin azure devops. I see PDB files are donwloaded to the symbols cache folder but stepping thru is asking for source file location in visual studio, even when i have indexed the source code during publish symbol.
I have tried to enable different options in visual studio debugging but nothing seems to help
Here is my yml
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
name: $(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r) # need this for byBuildNumber verisonScheme nuget pack
# the build will trigger on any changes to the master branch
trigger:
- master
# the build will run on a Microsoft hosted agent, using the lastest Windows VM Image
pool:
vmImage: 'windows-latest'
# these variables are available throughout the build file
# just the build configuration is defined, in this case we are building Release packages
variables:
buildConfiguration: 'Release'
#The build has 3 seperate tasks run under 1 step
steps:
# The first task is the dotnet command build, pointing to our csproj file
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
projects: 'src/Common.Core/Common.Core.csproj'
- task: PublishSymbols#2
inputs:
symbolsFolder: '$(Build.SourcesDirectory)'
searchPattern: '**/bin/**/*.pdb'
indexSources: true
publishSymbols: true
symbolServerType: 'teamServices'
treatNotIndexedAsWarning: true
symbolsProduct: '$(Build.DefinitionName)'
symbolsVersion: '$(Build.BuildNumber)'
symbolsArtifactName: '$(name).Symbols_$(BuildConfiguration)'
# The second task is dotnet pack command again pointing to the csproj file
# The nobuild means the project will not be compiled before running pack, because its already built in above step
- task: DotNetCoreCLI#2
displayName: "dotnet pack"
inputs:
command: 'pack'
arguments: '--configuration $(buildConfiguration)'
packagesToPack: 'src/Common.Core/Common.Core.csproj'
nobuild: true
includeSymbols: true
versioningScheme: 'byBuildNumber'
# The last task is a nuget command, nuget push
# This will push any .nupkg files to the 'Nuget' artifact feed
# allowPackageConflicts allows us to build the same version and not throw an error when trying to push
# instead it just ingores the latest package unless the version changes
- task: NuGetCommand#2
displayName: 'nuget push'
inputs:
command: 'push'
feedsToUse: 'select'
packagesToPush: '$(Build.ArtifactStagingDirectory)/**/*.nupkg;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'myNuget'
allowPackageConflicts: true
I would expect that when i am debugging nuget packages with symbols enabled with idnexed soruce code, it automatically downloads the pdf file and the source code.
Visual Studio Settings for debugger
I see PDB files are donwloaded to the symbols cache folder but
stepping thru is asking for source file location in visual studio,
even when i have indexed the source code during publish symbol.
You should let the debugger know where to find your source files. First, please rename your xx.nupkg to xx.zip and check if it contains necessary source files.
After that you can right-click Solution in Solution Explorer=>Properties=>Debug Source Files,click the New Line option to add the path of your nuget source files to the debug source files setting.
I would expect that when i am debugging nuget packages with symbols
enabled with idnexed soruce code, it automatically downloads the pdf
file and the source code.
Maybe you can get some help from this issue. You can try setting the build action of the source files as C# compiler when you create the nuget package for .net core.