Now, I am trying to run cts automatically with Python.
I can use that on single device but can't run on multiple devices.with same python codes.
this is the function to execute CTS testing and catching the output
CTS cannot start testing with --shard 2
['android-cts/tools/cts-tradefed', 'run', 'cts', '--plan', 'CTS-hardware', '--shards', '2']
CTS can start testing without --shard 2
['android-cts/tools/cts-tradefed', 'run', 'cts', '--plan', 'CTS-hardware']
It can also start testing with a terminal derictly.
android-cts/tools/cts-tradefed run cts --plan CTS-hardware --shards 2
My question is how can I start shards CTS testing with python to automatically test CTS?
Related
I am using Pytest to test a Python application with:
pytest -s --cov=myApp
However a process of my App runs asynchronously with Celery. The test module runs the Celery process properly but I don't get any coverage information.
Is is possible to get coverage from a process ran by celery?
I had a look at Celery testing but I don't want to test the asynchronously process directly/separately because I want to check how a process in MyAPP performs other actions with the task ID.
I also added the task module with --cov but I still don't get any coverage
I need to perform a single run of a BehaviorSpace experiment so that I can run the NetLogo model headless on Google cloud/AWS.
I tried writing a simple test code that just prints an output following the 'setup' command. However this prints the output twice. Am I doing something wrong? I tried entering 0 runs in parallel but this threw an IllegalArgumentException.
Here is the setup of the experiment:
Repetitions: 1
Setup commands: setup
Go commands: setup
Time limit: 1
Simultaneous runs in parallel: 1
Answered by Charles in the comment section:
I was running setup twice! Doh.
I am having an issue with kicking off test cases in Microsoft Test Manager from a script kicked off in Microsoft Release Manager. I can duplicate the issue when just running this command from powershell or command line. Here is the script:
C:\CODE\TCM\TCM.exe run /create /title:"Chads Example Test Case (Run from PowerShell)" /planid:31 /suiteid:2743 /configid:67 /settingsname:"DevWelisRemoteExecution" /testenvironment:"STAR_Regression" /collection:"http://tfssrv64:8080/tfs/DefaultCollection" /teamproject:QA /builddir:"\\tfssrv64\Builds" /include
Running this script returns a test run ID with no errors. I can immediately look in MTM and see the test run has started. It has a state of "pending". It eventually (some 20 minutes later) fails with the error "The test automation associated with the following test case could not be found: [48667]. Run the test case again using a build that contains the binary with the test automation.".
Facts: I can run the same test successfully, with a successful completion, from Microsoft Test Manager. (using the same settings specified in the script) Here are screen shots of the test runs from MTM.
Here is the MTM log of the successful test run.
The log of the successful test run.
Here is the same screen from the failed test run.
Here is the MTM log of the failed test run. Same test, ran from the above script.
The log of the failed test run.
Both test runs are using the same build number. Both test runs use the same test settings and configuration.
Any help would be greatly appreciated.....
I'm running my unit tests on a 16-core machine. I can't see any difference in elapsed time when using no parallelization parameter, --workers=1 and --workers=32. I'd like to confirm that NUnit really is running the expected number of simultaneous tests, so that I don't spend time hunting down a non-existent problem in my test code.
I have [Parallelizable] (default scope, ParallelScope.Self) on the common base class. It is defined not on any other class or method. I'm using nunit3-console, both via Jenkins and on my local command line.
Is there a way to tell that tests are running in parallel? NUnit is reporting the correct number of worker threads, but there's no report saying (for example) how many tests were run in each thread.
Run Settings
ProcessModel: Multiple
RuntimeFramework: net-4.5
WorkDirectory: C:\Jenkins\workspace\myproject
NumberOfTestWorkers: 16
I can log the start and finish times of each test then manually check that there's a reasonable number of overlaps; is there any simpler and more repeatable way of getting what I want?
I turned on --trace=Verbose which produces a few files. InternalTrace.<pid1>.log and InternalTrace.<pid2>.<dll_name>.log together contained a thorough description of what was happening during the tests. The per-agent log (the one with the DLL name in the log file name) was pretty clear about the state of parallelization.
16:13:40.701 Debug [10] WorkItemDispatcher: Directly executing test1
16:13:47.506 Debug [10] WorkItemDispatcher: Directly executing test2
16:13:52.847 Debug [10] WorkItemDispatcher: Directly executing test3
16:13:58.922 Debug [10] WorkItemDispatcher: Directly executing test4
16:14:04.492 Debug [10] WorkItemDispatcher: Directly executing test5("param1")
16:14:09.720 Debug [10] WorkItemDispatcher: Directly executing test5("param2")
16:14:14.618 Debug [10] WorkItemDispatcher: Directly executing test5("param3")
That third field looks like a thread ID to me. So I believe that the agent is running only one test at a time, even though (I think..) I've made all the tests parallelizable and they're all in different test fixtures.
Now I've just got to figure out what I've done wrong and why they're not running in parallel...
I could be mistaken but it seems to me that to set [Parallelizable] parameter is not enough to make tests run in parallel. You also need to create nodes that will run tests. So if you use Jenkins you have to create Jenkins Slaves and only then your tests will be run in parallel.
So what I want to tell is that as I understand there is no possibility to run tests in parallel on one PC.
If there is such a possibility (to run tests in parallel on the same machine) it would be really great and I struggle to hear about it!
I want to allow to run only 1 batch run at time. I tried to find it by using this code (from job listener):
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(jobExecution.getJobInstance().getJobName());
if (jobExecutions.size() > 1)
System.exit(2);
When I run 2 batches from different terminals this code doesn't work.
P.S. In dev environment I use HSQL.