I am trying to run a single set of a single set of tests within the e2e Kubernetes tests. I am quite confused as to how the tests are organized, is there a comprehensive list of all the tests?
Thanks!
Assuming, the tests are placed at ./tests/e2e path in the repository.
If the tests are written in go, they are mostly written using a standard testing library or ginkgo framework.
For running the tests written using standard testing package
Add the tags to a specific test at the start of your test file like,
// +build <my-test>
Run the tests by specifying the tag name, go test -v ./tests/e2e -tags <my-test>
For running the tests written using ginkgo
go test -ginkgo.dryRun ./tests/e2e/... to list all the tests in the package.
go test -ginkgo.focus "<regex>" ./tests/e2e/... to run specific tests mentioned in the focus regex field.
go test -ginkgo.skip "<regex>" ./tests/e2e/... to skip the specific tests mentioned in the regex field
If you have e2e.test binary, you can list all available test by setting following flag: ./e2e.test --ginkgo.DryRun. Then if you want a single test, type: ./e2e.test --ginkgo.Focus="<name of your test>", pay attentention that all special characters in the test name must be escaped. For example, if you want run only conformance tests: --ginkgo.Focus="\[Conformnce\]".
Just in case, the right way of running particularly focused e2e tests is described officially here: https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md
it would be something like this:
go run hack/e2e.go -- --test --test_args="--ginkgo.focus=${matching regex}"
Related
I must be missing something obvious. I've got a couple of jobrunr jobs where i'm using the lambda enqueue format version 5.1.6. Like this:
JobId jobId = BackgroundJob.<MyService>enqueue(x -> x.doWork());
I would like to validate the plumbing and work in the jobs is executing via some integration tests with Spring, but don't see the options to run now, eager mode, etc? Thanks
You can't, I'm afraid.
You can mock the JobScheduler and capture the args. JobRunr itself is also tested very well so if you pass a job, you can rest assured it will be enqueued.
You could also put the pollIntervalInSeconds to 5 and use awaitility then to verify your job executed. There are many examples of this in the JobRunr repo.
Trying to clean up some testing for IaC using Inspec, But hardcoding security_group_ids is a no go for obvious reasons.
Im trying to use the ruby sdk instead to pull down the id based of a name (ie like you do with Terraform data resources).
But we work from aws named profiles and while Inspec can connect to named profiles when i run the test ie :
inspec exec . -t aws://prod_account
Is it possible from Inspec to link the call to aws named profiles to ruby code within a control?
since inspec is written in ruby, you can embed any ruby code within your spec files. for instance, you can have a ruby code with an array and for each array have a spec code.
thus, you can implement a logic for collecting the security group ids and then iterate over them.
I am doing some initial one off setup using [BeforeTestRun] hook for my specflow tests. This does check on some users to make sure if they exist and creates them with specific roles and permissions if they are not so the automated tests can use them. The function to do this prints a lot of useful information on the Console.Writeline.
When I run the test on my local system I can see the output from this hook function on the main feature file and the output of each scenario under each of them. But when I run the tests via Azure DevOps pipleine, I am not sure where to find the output for the [BeforeTestRun] because it is not bound a particular test scenario. The console of Run Tests Tasks has no information about this.
Can someone please help me to show this output somewhere so I can act accordingly.
I tried to use System.Diagnostics.Debug.Print, System.Diagnostics.Debug.Print, System.Diagnostics.Debug.WriteLine and System.Diagnostics.Trace.WriteLine, but nothing seems to work on pipeline console.
[BeforeTestRun]
public static void BeforeRun()
{
Console.WriteLine(
"Before Test run analyzing the users and their needed properties for performing automation run");
}
I want my output to be visible somewhere so I can act based on that information if needed to.
It's not possible for the console logs.
The product currently does not support printing console logs for passing tests and we do not currently have plans to support this in the near future.
(Source: https://developercommunity.visualstudio.com/content/problem/631082/printing-the-console-output-in-the-azure-devops-te.html)
However, there's another way:
Your build will have an attachment with the file extension .trx. This is a xml file and contains an Output element for each test (see also https://stackoverflow.com/a/55452011):
<TestRun id="[omitted]" name="[omitted] 2020-01-10 17:59:35" runUser="[omitted]" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Times creation="2020-01-10T17:59:35.8919298+01:00" queuing="2020-01-10T17:59:35.8919298+01:00" start="2020-01-10T17:59:26.5626373+01:00" finish="2020-01-10T17:59:35.9209479+01:00" />
<Results>
<UnitTestResult testName="TestMethod1">
<Output>
<StdOut>Test</StdOut>
</Output>
</UnitTestResult>
</Results>
</TestRun>
Is there a way to add text to unit tests (Jasmine or Mocha) run in Karma - maybe with a console.log - and be able to output it? I understand using result.description (for example) in a reporter, but I don't know of any result.x that I can fill in from the test. I'm not sure, but I think the log4js in Karma cannot be used from the test, as the test runs in the browser and the log4js is on the server?
I have a feature that has prod tests to be run on prod server. Its tagged #prod. I have other non-tagged tests that can run in test envs.
In env.rb I have a Before do .. end that opens a new browser and sets up test to run in test env. I also have and a Before('#prod') block that sets up tests helpers to run in prod account.
When I run just the prod test, using cucumber --tags #prod, it runs prod tests. I expect only the Before('#prod') to run. However I see that both the BEfore blocks in env.rb file are run, creating 2 new browser instances and prod test running in second browser window. When my suite of 7 scenarios are done I have 7 blank open browser windows which my test util setup from non-tagged Before.
Given any valid test merchant account # features/step_definitions/ConsumerPortal.rb:1
We are in plain Before
We are in #prod Before
When login to pos portal as 'pos' user
I see the same happening for After tag to logout and close browser as well. Tagged one is run first and then the plain After tag. If I can read what tag a test has in my env.rb Before, I can setup accordingly. Is tehre a way to do this?
Use negation tags with your before tags.
#Before("~#prod")
#After("~#prod")
This will prevent the Before and After tags from running for all #prod tags.
Or else in general, all #Before and #After methods run for any tags.