Device Farm test upload status sits in "INITIALIZED" status forever - aws-device-farm

I'm using AWS CLI to upload my android app and the tests.zip file to device farm. My test project type is Appium + Java + TestNG.
I'm able to upload the android app successfully, but while trying to upload the "tests.zip" the upload status shows INITIALIZED for ever, even though the upload is completed successfully.(using curl command)
I'm not able to proceed with my automated tests because of this issue. Any clue/suggestions would be really appreciated.
Below is the code snippet that I've been trying with so far and its output.
TEST_UPLOAD_RESPONSE=$(aws devicefarm create-upload --project-arn $DEVICE_FARM_PROJECT_ARN --name "zip-with-dependencies.zip" --type "APPIUM_JAVA_TESTNG_TEST_PACKAGE" --region "us-west-2" --output "json")
ANDROID_TEST_ARN=$(echo $TEST_UPLOAD_RESPONSE | jq '.upload.arn' | tr -d \")
TEST_PRESIGNED_URL=$(echo $TEST_UPLOAD_RESPONSE | jq '.upload.url' | tr -d \")
echo "Test upload url is :$TEST_PRESIGNED_URL"
echo "Test upload arn is :$ANDROID_TEST_ARN"
curl -T ./target/zip-with-dependencies.zip $APP_PRESIGNED_URL
sleep 5s
TEST_UPLOAD_STATUS=""
while [ "$TEST_UPLOAD_STATUS" != "SUCCEEDED" ]
do
TEST_UPLOAD_STATUS=$(aws devicefarm get-upload --arn $ANDROID_TEST_ARN --query upload.status)
TEST_UPLOAD_STATUS=$(echo $TEST_UPLOAD_STATUS | tr -d \")
echo "Test upload status : $TEST_UPLOAD_STATUS"
done
App upload url is :https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A949397191675%3Aproject%3Acd2fdd09-4c62-42f4-8e27-bde987554d70/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A949397191675%3Aupload%3Acd2fdd09-4c62-42f4-8e27-bde987554d70/22d28644-4c01-4b82-939c-8e39ee5a268e/zip-with-dependencies.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20190525T230634Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20190525%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=4ace7559023815966bbbd88af6357374bd511d021aeb9d222f9c08acdbb07ed5
App upload arn is :arn:aws:devicefarm:us-west-2:949397191675:upload:cd2fdd09-4c62-42f4-8e27-bde987554d70/22d28644-4c01-4b82-939c-8e39ee5a268e
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 41.2M 0 0 0 400k 0 536k 0:01:18 --:--:-- 0:01:18 536k
62 41.2M 0 0 62 25.6M 0 14.6M 0:00:02 0:00:01 0:00:01 14.6M
100 41.2M 0 0 100 41.2M 0 17.3M 0:00:02 0:00:02 --:--:-- 17.3M
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
Test upload status : INITIALIZED
.
.

Here is a document which shows the process you described
https://docs.aws.amazon.com/devicefarm/latest/developerguide/how-to-create-test-run.html#how-to-create-test-run-cli-step4
When you have the chance can you let me know if all these steps were followed?
The description suggests to me that the wrong upload url was used or that the upload failed during the curl command.
Please provide full output of the commands in the document and I should be able to help more.
Hth
-James

Related

Azure Rest API CI task completes but stuck on non-closing STDIO lingering process

As part of CI pipeline on ADO, I make REST API GET calls to get a list of requirement work items objects and a list of test results object. I sort and match the list and then I do multiple POST call to add information from that list as a attachment of my ADO test item object. Everything is done thought the ADO Matlab plugin task by using system function to execute curl through the command line via Matlab. Everything seem to work, I see the attachment on every test uploaded well and it even prints ''Done'' after all curl POST request to indicate, I'm done with my Matlab script.
I would expect the ADO CI job task to complete and pass to the next task in YAML file. But after running for 5 mins and completing everything, the job task stall and keep running for another 10h(Max ADO pipeline time)
The STDIO streams did not close within 10 seconds of the exit event from process 'C:\agent_work_tasks\RunMATLABCommand_28fdff80-51b4-4b6e-83e1-cfcf3f3b25a6\0.6.3\bin\run_matlab_command.bat'. This may indicate a child process inherited the STDIO streams and has not yet exited.
Any Ideas how to resolve this bug?
What I Tried:
I tried to run my script locally, I saw all my curl outputs via my matlab command window, but I saw that upon completion it opened a empty command prompt window.
looking for similar incidents and tried closing all processes(#4) as indicated here: https://developercommunity.visualstudio.com/t/the-stdio-streams-did-not-close-within-10-seconds/523146
I tried adding pause/wait in matlab script or via system command call to give time for asynchronous process to complete.
I added quit to my matlab script, and I tried using Taskkill/IM cmd.exe to kill all open command windows when my scripts ends.
Both didn't work on the CI pipeline. it still runs forever until i manually stop it or it reaches max time.
Error message & partial log:
2022-04-04T18:39:45.6947848Z curl -u :**ADOPAT** -X POST -H "Content-Type: application/json" -d "{\"stream\":\"***B64 encoded content***\", \"fileName\": \"requirementPath.txt\", \"comment\":\"Testattachmentupload\",\"attachmentType\":\"GeneralAttachment\"}" http://dev.azure.com/{organization}/{project}/_apis/test/Runs/**runID**/Results/**testResultID**/attachments?api-version=6.0-preview
2022-04-04T18:39:46.3140720Z
2022-04-04T18:39:46.3141125Z status =
2022-04-04T18:39:46.3141240Z
2022-04-04T18:39:46.3141367Z 0
2022-04-04T18:39:46.3141523Z
2022-04-04T18:39:46.3141598Z
2022-04-04T18:39:46.3141697Z cmdout =
2022-04-04T18:39:46.3141781Z
2022-04-04T18:39:46.3141992Z ' % Total % Received % Xferd Average Speed Time Time Time Current
2022-04-04T18:39:46.3142311Z Dload Upload Total Spent Left Speed
2022-04-04T18:39:46.3142510Z
2022-04-04T18:39:46.3142760Z 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
2022-04-04T18:39:46.3143126Z 100 354 100 139 100 215 269 417 --:--:-- --:--:-- --:--:-- 692
2022-04-04T18:39:46.3143506Z 100 354 100 139 100 215 269 416 --:--:-- --:--:-- --:--:-- 692
2022-04-04T18:39:46.3143948Z {"id":**attachementID**,"url":"http://dev.azure.com/{organization}/{project}/_apis/test/Runs/**RunID**/Results/**testResultID**/Attachments/**attachmentID**"}'
2022-04-04T18:39:46.3144224Z
2022-04-04T18:43:06.5720937Z done
2022-04-04T18:43:29.0053832Z ##[debug]Exit code 0 received from tool 'C:\agent\_work\_tasks\RunMATLABCommand_28fdff80-51b4-4b6e-83e1-cfcf3f3b25a6\0.6.3\bin\run_matlab_command.bat'
2022-04-04T18:43:39.0075174Z The STDIO streams did not close within 10 seconds of the exit event from process 'C:\agent\_work\_tasks\RunMATLABCommand_28fdff80-51b4-4b6e-83e1-cfcf3f3b25a6\0.6.3\bin\run_matlab_command.bat'. This may indicate a child process inherited the STDIO streams and has not yet exited.
2022-04-04T18:43:39.0076952Z ##[debug]The STDIO streams did not close within 10 seconds of the exit event from process 'C:\agent\_work\_tasks\RunMATLABCommand_28fdff80-51b4-4b6e-83e1-cfcf3f3b25a6\0.6.3\bin\run_matlab_command.bat'. This may indicate a child process inherited the STDIO streams and has not yet exited.
2022-04-05T04:10:04.2626714Z ##[debug]Re-evaluate condition on job cancellation for step: 'Matlab Rest API Test Update Action'.
2022-04-05T04:10:04.2919553Z ##[error]The operation was canceled.
2022-04-05T04:10:04.2932116Z ##[debug]System.OperationCanceledException: The operation was canceled.

I receive error 1 message when trying to run makepkg -sic for yay

I cloned yay through git clone https://aur.archlinux.org/yay.git. I Enter the directory and run makepkg -sic but I unfortunately get the error:
==> Making package: yay 10.1.0-1 (Mon 26 Oct 2020 06:25:36 AM +0330)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
-> Downloading yay-10.1.0.tar.gz...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 118 100 118 0 0 97 0 0:00:01 0:00:01 --:--:-- 97
100 339k 100 339k 0 0 103k 0 0:00:03 0:00:03 --:--:-- 168k
==> Validating source files with sha256sums...
yay-10.1.0.tar.gz ... Passed
==> Extracting sources...
-> Extracting yay-10.1.0.tar.gz with bsdtar
==> Starting build()...
go build -v -trimpath -mod=readonly -modcacherw -ldflags '-s -w -extldflags "-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now" -X "main.yayVersion=10.1.0" -X "main.localePath=/usr/share/locale/"' -buildmode=pie -o yay
go: github.com/Jguer/go-alpm/v2#v2.0.1: Get "https://gocenter.io/github.com/%21jguer/go-alpm/v2/#v/v2.0.1.mod": dial tcp 35.230.74.213:443: i/o timeout
make: *** [Makefile:127: yay] Error 1
==> ERROR: A failure occurred in build().
Aborting...
Are you getting the error consistently? If so, it looks like the host you are using to build does not have access to GoCenter.
exec: "gcc": executable file not found in $PATH
check prerequisites. see https://wiki.archlinux.org/index.php/Ar … Repository
But most of the time there is a problem with gcc, which can be solved in the following way :
pamac install base-devel
choose a gcc number from list

Concourse CI Windows Worker

I'm trying to setup a Concourse CI environment with a Windows 7 worker.
I have one machine under Ubuntu Server (16.04) hosting my TSA server and one worker (for the support of git resources), and a second one under Windows 7 hosting a worker.
Everything seems to work fine as:
I can login into the web ui
the fly -t my_concourseci workers command returns :
name containers platform tags team state version
ubuntu 1 linux none none running 1.1
windows7 0 windows none none running 1.1
the fly -t my_concourseci execute -c test.yml command returns:
executing build 146
initializing
running echo Hello World!
Hello World!
with the following content in test.yml file:
platform: windows
run:
path: echo
args: [ "Hello World!" ]
Nevertheless when I add an input to my task:
platform: windows
> inputs:
> - name: concourse
run:
path: echo
args: [ "Hello World!" ]
I get the following error:
executing build 148
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5698k 0 5698k 0 0 1948k 0 --:--:-- 0:00:02 --:--:-- 1949k
initializing
failed to stream in to volume
errored
If I look at the windows worker log I got this error that comes up:
{"timestamp":"1500642862.643555164",
"source":"baggageclaim",
"message":"baggageclaim.api.volume-server.stream-in.bad-stream-payload",
"log_level":1,
"data":{"error":"tar extract failed (exit status 2). output: \"\\ngzip: stdin: not in gzip format\\n/usr/bin/tar: Child returned status 1\\n/usr/bin/tar: Error is not recoverable: exiting now\\n\"",
"session":"2.1.8730",
"volume":"15bf1fc6-0727-4962-6c84-18446e54ab96"}
}
Any ideas about what can cause a not in gzip format error ? Knowing that if I run the exact same task on a linux platform every works fine.
platform: linux
image_resource:
type: docker-image
source: {repository: busybox}
inputs:
- name: concourse
run:
path: echo
args: [ "Hello World!" ]
----- STDOUT
executing build 149
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5699k 0 5699k 0 0 1917k 0 --:--:-- 0:00:02 --:--:-- 1918k
initializing
Pulling busybox#sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4...
sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4: Pulling from library/busybox
9e87eff13613: Pulling fs layer
9e87eff13613: Verifying Checksum
9e87eff13613: Download complete
9e87eff13613: Pull complete
Digest: sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4
Status: Downloaded newer image for busybox#sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4
Successfully pulled busybox#sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4.
running echo Hello World!
succeeded
Thanks.
Launch powershell as Administrator, and run the concourse_worker.exe from there. That worked for me.
I am running a Windows Server 2016 Base on AWS based on this AMI ami-e1876a98
Thanks for your answers, it helps me solve my problem.
I were starting my concourse worker within a MSYS 1.0 environment. The thing is that MSYS contains tar and gunzip binaries within the $PATH. When I started concourse worker inside a Power Shell or cmd.exe without any MSYS unix-like binaries in the $PATH it worked like a charm !
Note: Be sure to have no MSYS binaries in the $PATH Windows environment variable for this to work, especially check that Git-Bash environment tools are not added to your windows $PATH environment variable.
Thanks again.

How to test Unity projects with Travis CI?

I was following this specific guide: https://jonathan.porta.codes/2015/04/17/automatically-build-your-unity3d-project-in-the-cloud-using-travisci-for-free/ to test my unity project with travis CI but have been running into the same problem and don't understand how to change the 'installer' program to fix this problem:
$ ./Scripts/install.sh
Downloading from http://download.unity3d.com/download_unity/a6d8d714de6f/MacEditorInstaller/ Unity-5.4.0f3.pkg:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 16433 100 16433 0 0 16498 0 --:--:-- --:--:-- --:--:-- 16498
Installing Unity.pkg
installer: Error the package path specified was invalid: 'Unity.pkg'.
The command "./Scripts/install.sh" failed and exited with 1 during .
Your build has been stopped.
Here are the scripts I wrote using the tutorial:
.travis.yml
language: objective-c
osx_image: xcode61
rvm:
- 2.1.2
install:
- ./Scripts/install.sh
script:
- ./Scripts/build.sh
install.sh
#! /bin/sh
echo 'Downloading from http://download.unity3d.com/download_unity/a6d8d714de6f/MacEditorInstaller/Unity-5.4.0f3.pkg:
curl -o Unity.pkg http://download.unity3d.com/download_unity/a6d8d714de6f/MacEditorInstaller/Unity-5.4.0f3.pkg:
echo 'Installing Unity.pkg'
sudo installer -dumplog -package Unity.pkg -target /
Any help for this problem or guidance towards a solution would be greatly appreciated, thanks
Remove the colon at the end of the url inside install.sh
curl -o Unity.pkg http://download.unity3d.com/download_unity/a6d8d714de6f/MacEditorInstaller/Unity-5.4.0f3.pkg:
Unity-5.4.0f3.pkg: => Unity-5.4.0f3.pkg

Is it possible to see the output of cURL in the TravisCI console?

I am integrating HockeyApp with TravisCI, but when using the wrong HockeyApp Token value, I couldn't see any issue in the TravisCI console output. Running the CURL in Terminal quite clearly displayed the error:
{"errors":{"credentials":["api token invalid"]}}
is there anyway to display in the TraveisCI console, the results of cURL, when called from a build-and-upload.sh script?
I tried to do the following with our success:
rtn = $(curl http://....
echo $rtn
The return I got was interesting, but it was displayed even if the build wasn't uploaded.
Trying to echo the rtn value also clearly failed.
***************************
* Uploading to Hockeyapp *
***************************
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 216k 100 48 100 216k 27 124k 0:00:01 0:00:01 --:--:-- 124k
./scripts/sign-and-upload.sh: line 33: rtn: command not found
Doh! I was caught out by the space on either side of the =
rtn=$(curl http://....
This works!