Azure pipeline how detect when pipeline stoped when script waiting for user input - azure-devops

I have a pipeline that is executing scripts and some of them are 3third party scripts.
in this case is Fastlane script.
Now in some rare cases, the Fastlane script is expecting the user to input value to stdin my question is how can I detect when a script is stoped and waiting to input. in this case, I like to throw an error and fail the task.
this is the pipeline task :
- script: |
fastlane release --verbose projectName:${{parameters.projectName}}
echo 'Done invoking Fastfile'
#failOnStderr: false
workingDirectory: '$(System.ArtifactsDirectory)/ios_artifacts'
displayName: 'run fastlane'
And it stack in this idle state :
2020-10-12T06:46:27.5309820Z INFO [2020-10-12 06:46:27.52]: [32m------------------[0m
2020-10-12T06:46:27.5310570Z INFO [2020-10-12 06:46:27.53]: [32m--- Step: sigh ---[0m
2020-10-12T06:46:27.5311280Z INFO [2020-10-12 06:46:27.53]: [32m------------------[0m
2020-10-12T06:46:27.5362760Z
2020-10-12T06:46:27.5386680Z +-------------------------------------+------------------------------+
2020-10-12T06:46:27.5387580Z | [32mSummary for sigh 2.162.0[0m |
2020-10-12T06:46:27.5388330Z +-------------------------------------+------------------------------+
2020-10-12T06:46:27.5389160Z | username | xxxx#xxxx.com |
2020-10-12T06:46:27.5389530Z | app_identifier | com.xxxx.xxxx |
2020-10-12T06:46:27.5389950Z | team_id | xxxx |
2020-10-12T06:46:27.5390320Z | adhoc | false |
2020-10-12T06:46:27.5390730Z | developer_id | false |
2020-10-12T06:46:27.5391490Z | development | false |
2020-10-12T06:46:27.5391860Z | skip_install | false |
2020-10-12T06:46:27.5392280Z | force | false |
2020-10-12T06:46:27.5392650Z | ignore_profiles_with_different_name | false |
2020-10-12T06:46:27.5393070Z | skip_fetch_profiles | false |
2020-10-12T06:46:27.5393450Z | skip_certificate_verification | false |
2020-10-12T06:46:27.5393860Z | platform | ios |
2020-10-12T06:46:27.5394230Z | readonly | false |
2020-10-12T06:46:27.5394630Z | fail_on_name_taken | false |
2020-10-12T06:46:27.5395400Z +-------------------------------------+------------------------------+
2020-10-12T06:46:27.5395640Z
2020-10-12T06:46:27.5396350Z INFO [2020-10-12 06:46:27.53]: Starting login with user 'xxxx#xxxx.com'
2020-10-12T06:46:27.5559840Z Reading keychain entry, because either user or password were empty
2020-10-12T06:46:27.5560420Z Loading session from environment variable
2020-10-12T06:46:27.7305500Z Session loaded from environment variable is not valid. Continuing with normal login.
2020-10-12T06:46:28.7278420Z Two-factor Authentication (6 digits code) is enabled for account 'xxxx#xxxx.com'
2020-10-12T06:46:28.7279570Z More information about Two-factor Authentication: https://support.apple.com/en-us/HT204915
2020-10-12T06:46:28.7279880Z
2020-10-12T06:46:28.7280520Z If you're running this in a non-interactive session (e.g. server or CI)
2020-10-12T06:46:28.7281300Z check out https://github.com/fastlane/fastlane/tree/master/spaceship#2-step-verification
How can i monitor this state ? and fail the pipeline ?

How can i monitor this state ? and fail the pipeline ?
As far as I know, this could not be possible. Currently, the azure devops pipeline does not provide such a built-in feature that can detect the script waiting for input and fail the pipeline. It should be an easy way to set the timeout you want for the task timeoutInMinutes: number # how long to wait before timing out the task to make it fail based on timeout, but it seems you don't want this.
You could add your request for this feature on our UserVoice site, which is our main forum for product suggestions. After suggest raised, you can vote and add your comments for this feedback. The product team would provide the updates if they view it.

Related

flyway sessions locking itself

I was trying to implement code through flyway:
create index concurrently if not exists api_client_system_role_idx2 on profile.api_client_system_role (api_client_id);
create index concurrently if not exists api_client_system_role_idx3 on profile.api_client_system_role (role_type_id);
create index concurrently if not exists api_key_idx2 on profile.api_key (api_client_id);
However flyway sessions were blocking each other and script is in "pending" state.
| Versioned | 20.1 | add email verification table | SQL | 2021-11-01 21:55:52 | Success |
| Versioned | 21.1 | create role for doc api | SQL | 2021-11-01 21:55:52 | Success |
| Versioned | 22 | create indexes for profile | SQL | 2022-10-21 10:23:41 | Success |
| Versioned | 23 | test flyway | SQL | | Pending |
+-----------+---------+----------------------------------------------+--------+---------------------+---------+
Flyway: Flyway Community Edition 9.3.1 by Redgate
Database: Postgresql 14.4
Can you please advice how to properly implement creating indexes concurrently in postgresql?
I've tried simply to kill blocking session and let the script to continue, however then implementation failed and scripts stayed in "Pending" status.

Fastlane issue with fastlane match: Couldn't decrypt the repo, please make sure you enter the right password

I'm trying to learn fastlane and I'm currently blocked at fastlane match: https://docs.fastlane.tools/actions/match/#fastlane
Following the docs, I installed fastlane.
I created a private repository for the profiles and certificates.
I created an SSH key using a passphrase, example sfsfsdf
I added the SSH key to the ssh-agent and uploaded it to GitHub.
I ran then bundle exec fastlane match init and adjusted the Matchfile
Matchfile
git_url("git#github.com:my-repository/certificates.git")
git_branch("main")
type "development" # The default type, can be: appstore, adhoc, enterprise or development
app_identifier("com.mydomain.ios.project.v1")
username("my-appstore-account#domain.co")
Then I ran bundle exec fastlane match development to generate and install the Development certificates and profiles, which generates the following output:
fastlane could not check for updates error: 776: unexpected token at '<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><BucketName>kits-crashlytics-com</BucketName><RequestId>VX3SA</RequestId><HostId>hnZRvv3hulf/q50pxV5S1co0jPLJq0KgI6/1EB0jQEI+dWlLedQA=</HostId></Error>'
[15:53:38]: Successfully loaded '/Users/my-user/my-project/fastlane/Matchfile' 📄
+----------------+--------------------------------------+
| Detected Values from './fastlane/Matchfile' |
+----------------+--------------------------------------+
| git_url | git#github.com:my-repository/
certificates.git |
| git_branch | main |
| type | development |
| app_identifier | com.mydomain.ios.project.v1. |
| username | my-appstore-account#domain.com |
+----------------+--------------------------------------+
+-----------------------+--------------------------------------+
| Summary for match 2.28.3 |
+-----------------------+--------------------------------------+
| git_url | git#github.com:my-repository/
certificates.git |
| git_branch | main |
| type | development |
| app_identifier | com.mydomain.ios.project.v1. |
| username | my-appstore-account#domain.com |
| keychain_name | login.keychain |
| readonly | false |
| team_id | TEAM_ID_NUMBER |
| verbose | false |
| force | false |
| skip_confirmation | false |
| shallow_clone | false |
| force_for_new_devices | false |
| skip_docs | false |
| platform | ios |
+-----------------------+--------------------------------------+
[15:53:38]: Cloning remote git repo...
[15:53:40]: Checking out branch main...
[15:53:40]: Enter the passphrase that should be used to encrypt/decrypt your certificates
[15:53:40]: This passphrase is specific per repository and will be stored in your local keychain
[15:53:40]: Make sure to remember the password, as you'll need it when you run match on a different machine
[15:53:40]: Passphrase for Git Repo:
I entered the passphrase I used for my SSH key (sfsfsdf) and I saw the following error:
[15:59:09]: Couldn't decrypt the repo, please make sure you enter the right password!
keychain: "/Users/my-user/Library/Keychains/login.keychain-db"
version: 512
class: "inet"
attributes:
0x00000007 <blob>="match_git#github.com:my-repository/certificates.git"
0x00000008 <blob>=<NULL>
"acct"<blob>=<NULL>
"atyp"<blob>="dflt"
"cdat"<timedate>=0x32303232303531303232353930395A00 "20220510225909Z\000"
"crtr"<uint32>=<NULL>
"cusi"<sint32>=<NULL>
"desc"<blob>=<NULL>
"icmt"<blob>=<NULL>
"invi"<sint32>=<NULL>
"mdat"<timedate>=0x32303232303531303232353930395A00 "20220510225909Z\000"
"nega"<sint32>=<NULL>
"path"<blob>=<NULL>
"port"<uint32>=0x00000000
"prot"<blob>=<NULL>
"ptcl"<uint32>=0x00000000
"scrp"<sint32>=<NULL>
"sdmn"<blob>=<NULL>
"srvr"<blob>="match_git#github.com:my-repository/certificates.git"
"type"<uint32>=<NULL>
password has been deleted.
[15:59:09]: Enter the passphrase that should be used to encrypt/decrypt your certificates
[15:59:09]: This passphrase is specific per repository and will be stored in your local keychain
[15:59:09]: Make sure to remember the password, as you'll need it when you run match on a different machine
[15:59:09]: Passphrase for Git Repo:
What am I doing wrong or not doing?
Thanks!
I'm facing a similar issue. So far the only solution I've seen is to add the password for the repo to the Matchfile as ENV["MATCH_PASSWORD"] = "[PASSWORD]".
This did not work for me, but it's the best I've found and it at least makes the error more obvious on the command line. My next attempt is going to be running fastlane match nuke and starting the process fresh.
This issue has more specifics about their reproducible steps and solution: https://github.com/fastlane/fastlane/issues/14879.

How to create a chrome extension that can run custom Catalon scripts on it?

I am looking for a way to build a custom Chrome extension that runs Katalon scripts.
From their main page I am not sure how to process and execute the outputted commands posts:
open | https://katalon-test.s3.amazonaws.com/demo-aut/dist/html/form.html |
click | id=first-name |
type | id=first-name | Alex
type | id=last-name | Smith

Airflow Scheduler not picking up DAG Runs

I'm setting up airflow such that webserver runs on one machine and scheduler runs on another. Both share the same MySQL metastore database. Both instances come up without any errors in the logs but the scheduler is not picking up any DAG Runs that are created by manually triggering the DAGs via the Web UI.
The dag_run table in MysQL shows few entries, all in running state:
mysql> select * from dag_run;
+----+--------------------------------+----------------------------+---------+------------------------------------+------------------+----------------+----------+----------------------------+
| id | dag_id | execution_date | state | run_id | external_trigger | conf | end_date | start_date |
+----+--------------------------------+----------------------------+---------+------------------------------------+------------------+----------------+----------+----------------------------+
| 1 | example_bash_operator | 2017-12-14 11:33:08.479040 | running | manual__2017-12-14T11:33:08.479040 | 1 | �� }�. | NULL | 2017-12-14 11:33:09.000000 |
| 2 | example_bash_operator | 2017-12-14 11:38:27.888317 | running | manual__2017-12-14T11:38:27.888317 | 1 | �� }�. | NULL | 2017-12-14 11:38:27.000000 |
| 3 | example_branch_dop_operator_v3 | 2017-12-14 13:47:05.170752 | running | manual__2017-12-14T13:47:05.170752 | 1 | �� }�. | NULL | 2017-12-14 13:47:05.000000 |
| 4 | example_branch_dop_operator_v3 | 2017-12-15 04:26:07.208501 | running | manual__2017-12-15T04:26:07.208501 | 1 | �� }�. | NULL | 2017-12-15 04:26:07.000000 |
| 5 | example_branch_dop_operator_v3 | 2017-12-15 06:12:10.965543 | running | manual__2017-12-15T06:12:10.965543 | 1 | �� }�. | NULL | 2017-12-15 06:12:11.000000 |
| 6 | example_branch_dop_operator_v3 | 2017-12-15 06:28:43.282447 | running | manual__2017-12-15T06:28:43.282447 | 1 | �� }�. | NULL | 2017-12-15 06:28:43.000000 |
+----+--------------------------------+----------------------------+---------+------------------------------------+------------------+----------------+----------+----------------------------+
6 rows in set (0.21 sec)
But the Scheduler that's started up on another machine and connected to the same MySQL DB is just not interested in talking to this DB and actually running these DAG runs and converting them to Task Instances.
Not sure what I'm missing in the setup here. So few questions:
When and how is the DAGS folder located at $AIRFLOW_HOME/dags populated? I think its when the webserver is started. But then if I just start the scheduler on another machine, how will the DAGS folder on that machine be filled up?
Currently, I'm doing airflow initdb only on the machine hosting the webserver and not on scheduler. Hope that is correct.
Can I enable debug logs for Scheduler to get more logs that could indicate what's missing? From the current logs it looks like it just looks in the DAGS folder on local system and finds no DAGS there ( not even example ones ) inspite of the config to load examples set as True.
Don't think it matters but I'm currently using a LocalExecutor
Any help is appreciated.
Edit: I know that I need to sync up DAGS folder across machines as the airflow docs suggest but not sure if this is the reason why Scheduler is not picking up the tasks in the above case.
Ok, I got the answer - It looks like the Scheduler does not query the DB until there are any DAGS in the local DAG Folder. The code in job.py looks like
ti_query = (
session
.query(TI)
.filter(TI.dag_id.in_(simple_dag_bag.dag_ids))
.outerjoin(DR,
and_(DR.dag_id == TI.dag_id,
DR.execution_date == TI.execution_date))
.filter(or_(DR.run_id == None,
not_(DR.run_id.like(BackfillJob.ID_PREFIX + '%'))))
.outerjoin(DM, DM.dag_id==TI.dag_id)
.filter(or_(DM.dag_id == None,
not_(DM.is_paused)))
)
I added a simple DAG in my local DAG folder on the machine hosting Scheduler and it started picking up other DAG instances as well.
We raised an issue for this - https://issues.apache.org/jira/browse/AIRFLOW-1934

Can I pass the results from match to sh in fastlane?

I have lane like:
lane :beta do
match(type: "appstore")
sh "ember cdv:build
--platform ios
--code-sign-identity='iPhone Distribution: Cannla Pte Ltd (856AP7L2GS)'
--provisioning-profile='5feb0088-c4dd-4ca2-84e6-4bbf7f319248'
--release"
pilot
end
Can I get the code signing identity and provisioning profile from match instead of manually setting them like this?
match will automatically fill your environment variables. Check out the fastlane code signing docs on what kind of variables are exposed and how you can use them in your Xcode project.
By default, these are the environment variables that are exposed for you
+---------------------+------------------------------------------------+--------------------------------------+
| Installed Provisioning Profile |
+---------------------+------------------------------------------------+--------------------------------------+
| Parameter | Environment Variable | Value |
+---------------------+------------------------------------------------+--------------------------------------+
| App Identifier | | me.themoji.release |
| Type | | appstore |
| Profile UUID | sigh_me.themoji.beta_appstore | 22a19b3a-7cf6-4997-95f2-9cbb4d33fe7e |
| Profile Name | sigh_me.themoji.beta_appstore_profile-name | match AppStore me.themoji.release |
| Development Team ID | sigh_me.themoji.beta_appstore_team-id | N8XAAASEU2 |
+---------------------+------------------------------------------------+--------------------------------------+