I am having an issue with kicking off test cases in Microsoft Test Manager from a script kicked off in Microsoft Release Manager. I can duplicate the issue when just running this command from powershell or command line. Here is the script:
C:\CODE\TCM\TCM.exe run /create /title:"Chads Example Test Case (Run from PowerShell)" /planid:31 /suiteid:2743 /configid:67 /settingsname:"DevWelisRemoteExecution" /testenvironment:"STAR_Regression" /collection:"http://tfssrv64:8080/tfs/DefaultCollection" /teamproject:QA /builddir:"\\tfssrv64\Builds" /include
Running this script returns a test run ID with no errors. I can immediately look in MTM and see the test run has started. It has a state of "pending". It eventually (some 20 minutes later) fails with the error "The test automation associated with the following test case could not be found: [48667]. Run the test case again using a build that contains the binary with the test automation.".
Facts: I can run the same test successfully, with a successful completion, from Microsoft Test Manager. (using the same settings specified in the script) Here are screen shots of the test runs from MTM.
Here is the MTM log of the successful test run.
The log of the successful test run.
Here is the same screen from the failed test run.
Here is the MTM log of the failed test run. Same test, ran from the above script.
The log of the failed test run.
Both test runs are using the same build number. Both test runs use the same test settings and configuration.
Any help would be greatly appreciated.....
Related
We are using NUnit Test Engine to run test programatically.
Lokks like that after we add FrameworkPackageSettings.NumberOfTestWorkers to the Runner code, the test run for our Ui test hangs in execution. I'm not able to see at what time or event the execuiton hangs because Test Runned returns test result logs (in xml) only when entire execution ends
Is there a way to get test execution logs for each test?
I've added InternalTraceLevel and InternalTraceWriter but these logs are something different (BTW, looks like ParallelWorker#9 hangs even to write to console :) )
_package.AddSetting(FrameworkPackageSettings.InternalTraceLevel, "Debug");
var nunitInternalLogsPath = Path.GetDirectoryName(Uri.UnescapeDataString(new Uri(Assembly.GetExecutingAssembly().CodeBase).AbsolutePath)) + "\\NunitInternalLogs.txt";
Console.WriteLine("nunitInternalLogsPath: "+nunitInternalLogsPath);
StreamWriter writer = File.CreateText(nunitInternalLogsPath);
_package.AddSetting(FrameworkPackageSettings.InternalTraceWriter, writer);
The result file, with default name TestResult.xml is not a log. That is, it is not a file produced, line by line, as execution proceeds. Rather, it is a picture of the result of your entire run and therefore is only created at the end of the run.
InternalTrace logs are actual logs in that sense. They were created to allow us to debug the internal workings of NUnit. We often ask users to create them when an NUnit bug is being tracked. Up to four of them may be produced when running a test of a single assembly under nunit3-console...
A log of the console runner itself
A log of the engine.
A log of the agent used to run tests (if an agent is used)
A log received from the test framework running the tests
In your case, #1 is not produced, of course. Based on the content of the trace log, we are seeing #4, triggered by the package setting passed to the framework. I have seen the situation where the log is incomplete in the past but not recently. The logs normally use auto-flush to ensure that all output is actually written.
If you want to see a complete log #2, then set the WorkDirectory and InternalTrace properties of the engine when you create it.
However, as stated, these logs are all intended for debugging NUnit, not for debugging your tests. The console runner produces another "log" even though it isn't given that name. It's the output written to the console as the tests run, especially that produced when using the --labels option.
If you want some similar information from your own runner, I suggest producing it yourself. Create either console output or a log file of some kind, by processing the various events received from the tests as they execute. To get an idea of how to do this, I suggest examining the code of the NUnit3 console runner. In particular, take a look at the TestEventHandler class, found at https://github.com/nunit/nunit-console/blob/version3/src/NUnitConsole/nunit3-console/TestEventHandler.cs
Situation: VisualStudioCode (Browser library) runs a couple of .robot files (manually started)
Then it pauses because of an error...
At that point the process breaks and there is no final report.html
If you stop the run it doesn't generate an report.html that's not what you want. You actually want the results until that point. (or even better described: you still want the links output.xml, log.html and report.html)
you should be able to generate lag.htm and report.html using the rebot command. however you need output.xml for this. output.xml is created when you run the tests. when you break you will probobaly not have all the resources you need.
I would suggest to assign test timeout to the test that causes the pause. When the timeout is reached the test will be stoped automaticaly and you should have all reports. You can also set it globaly for all tests eg.:
*** Settings ***
Test Timeout 2 minutes
I am using Pytest to test a Python application with:
pytest -s --cov=myApp
However a process of my App runs asynchronously with Celery. The test module runs the Celery process properly but I don't get any coverage information.
Is is possible to get coverage from a process ran by celery?
I had a look at Celery testing but I don't want to test the asynchronously process directly/separately because I want to check how a process in MyAPP performs other actions with the task ID.
I also added the task module with --cov but I still don't get any coverage
We are using the Publish Test Results task PublishTestResults#2 to publish junit type results in a pipeline in Azure Devops. It has previously worked fine but is now hanging for 10mins (I think this is the default job timeout) and then failing, even though the results have been published. If I try to cancel the job when it starts hanging, the cancel request is ignored and the job continues to hang. Has anyone else experienced similar?
This is the log output whilst the task is hanging
Starting: PublishTestResults
==============================================================================
Task : Publish Test Results
Description : Publish test results to Azure Pipelines
Version : 2.160.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results
==============================================================================
##[warning]An error occurred while sending the request.
Publishing test results to test run '1033544'.
TestResults To Publish 11, Test run id:1033544
Test results publishing 11, remaining: 0. Test run id: 1033544
Async Command Start: Publish test results
We eventually nailed this down to only occurring on agents running as a service, rather than interactively. The problem is that our agents are behind a proxy. We provided the proxy settings as per the instructions but it appears the Publish Test Results task doesn't use that settings, so we had to provide it to the environment that runs the service as well by editing runsvc.sh:
export HTTP_PROXY=http://ourproxy:8080/
export NO_PROXY=localhost,127.0.0.1,localaddress
export HTTPS_PROXY=http://ourproxy:8080/
Publishing test results in AzureDevops Hangs
According to the error message:
[warning]An error occurred while sending the request.
It shows that an error was encountered while sending the request, you could enable the debug log by changing the default variables system.debug to true.
If it worked fine previously and you haven't changed your build definition, then the problem should be caused by your network or Azure devops agent server. You could use private agent to check if it related to the hosted agent.
Besides, for the cancel request is ignored and the job continues to hang, you could set the Build job cancel timeout in minutes in the build options:
So, it won't hang your job all the time.
Hope this helps.
We had the same issue and it really looks like a proxy problem. Since we don't need result files uploaded, the following 'input' helped:
publishRunAttachments: false
I recently deployed Botium v1.7.5 and have been getting "Job failed: Error: Worker finished with error" error. The bot does not detect any user utterance at all and ends the test session with a fail.
I included screenshots of my test case settings, test result and excel sheet containing test cases
I created a test set from manually chatting with my bot, through a feature in Botium, ran it, and it passed.