There is a question: Is there a possibility to set tests to perform priority in selenoid.
Problem: There is a suite> 20 tests, correspondingly at startup it fills the queue. After that, another test is run. He gets to the end of the line.
Is there an option to make it run as soon as the browser is freed, without waiting for all the tests to run before it?
No, this is not possible in current implementation. All incoming requests have equal priority. Two alternatives:
I think such issues should be addressed in test framework of you choice. For example for py.test a quick search shows a plugin for ordering your tests: https://github.com/ftobia/pytest-ordering Not sure whether it works.
You could also install Ggr and use different Selenoids and quota names for different tests, but this seems to be too much complicated for your case.
Related
could someone explain please how Flutter/Dart tests are executed using test runner?
Are the tests executed synchronously or asynchronously?
Does the testing framework execute every single test synchronously, meaning that only a single test and test suite is executed at any single time?
Or does the testing framework only execute a single test at a time within a test suite, but are able to execute multiple test suites at the same time?
Or testing framework run all tests and test suites completely independent of each other at the same time, completely asynchronously?
This is important because it has a direct impact on the way we are or aren't able to structure our tests, especially when it comes to the set up and tear downs of tests, and the way we assert functionality is working correctly.
Thanks!
In general, dart test will execute many tests in parallel (the parallelism level varies based on CPU core count), but you can disable this with a command line flag.
You should not write tests with any inter-dependence (i.e. one test should not rely on some global state set up by another test). For example, you may find that because your laptop has a different CPU configuration to your CI server, your tests might pass locally but fail in CI due to different ordering.
If you have some setup logic that is very expensive, and needs to be reused between multiple tests, you can use setUpAll() to run some code once before every test in a test group, however this is still discouraged. Personally, I prefer to just join the tests into one long test, to keep all tests self-contained.
This has some advantages. For example, you can use --total-shards and --shard-index to parallelize tests in CI (by creating multiple jobs to each run a different subset of the test suite).
You can also randomize the order of your tests with --test-randomize-ordering-seed, to make sure you aren't accidentally setting up such dependences between tests that might invalidate their results (i.e. perhaps test2 only passes if it happens after test1, randomizing the ordering will catch this).
TLDR
Many tests run in parallel. Try to keep tests self-contained, with no dependence on the order of tests. Extract setup logic into functions and pass it into setUp. If you really really need the performance, you can try setUpAll but avoid it if possible.
When using the Java client to start a workflow in Candece "executionStartToCloseTimeoutSeconds" is required on the Workflow. If I have a workflow that can run for an indeterminate amount of time, how do I get around this restriction?
That was a mistake to require this value. The new version of the platform I'm working on (temporal.io) defaults this value to infinity.
an indeterminate amount of time
First of all I believe indeterminate amount of time is not infinite amount of time.
As letting a workflow execution run and grow infinitely is anti-pattern in Cadence workflow. See Recommendation #5 in this article https://longquanzheng.github.io/cadence-lab/book/learnings/what-should-be-in-a-workflow-or-an-activity-in-cadence.html
A good timeout value can protect your workflow grow infinitely.
Because it's not recommended to let workflow run forever, as it will cause potential performance issue in both worker and server, the original idea is to enforce client providing a timeout value. We didn't provide defaults, as it's difficult to have a reasonable default for all use cases.
A too small default values will be even worse, because no one like workflow timeouted in production unexpectedly. Even though you can use "Reset" command to reopen it.
A too big default value, like Maxim suggests, is slightly better than too small values. But I personally disagree because that induces client forget thinking about how long the workflow will run, and how long the workflow history will grow. This will also turn out to be a production issue at some points later.
The biggest issue I see is that this required option is not friendly. It should be compiling error instead of running error. I think this is probably we can improve in Cadence -- if this is a required field, make it required at coding experience. At the same time provide some hardcoded fake "infinite" value to help some edge cases may also make sense.
Back to you question, I would suggest using some fake infinite value if you think it's indeterminate. An good example here is in Cadence system workflow: https://github.com/uber/cadence/blob/11547ee6db5dd306cb507b263381a6ea94c3faf1/service/worker/scanner/workflow.go#L48
Basically I've got a Service which can work with two alternatives of ResourceSets. Let's say, the Service would optimally work with one Doctor and one Nurse, but it is also possible to work with only one Doctor if a Nurse isn't available.
Now, assuming the Doctor works slower without a Nurse, the Service's delay time must depend upon the resourceSet being employed at the moment (Doctor+Nurse or Doctor). Any idea how can I program this?
You should also have in mind that my model has various Services in parallel working in the same way, it's not just only one Service line.
Thanks!
You're using Services but, to me, using the combination of Seize, Delay and Release gives you more flexibility.
What I've done is set the resource choice according to the image bellow:
It is important to have the nurses prior to the doctors in the first set (for some reason anylogic would opt for using only the doctor if otherwise - even with a nurse available).
Than, I would write this code:
Which means that if the agent was only able to seize one resource it will take longer (15 is just a random value).
In the delay block, I would set the processing time to agent.processTime
The topology I'm using is this:
Obviously this is a workaround and will not work for every case. You can always change the conditions you verify. I couldn't find a way to check which resource set was picked by the seize operation. If you're in a hurry this will do the trick.
Hope that helps,
Luís
I'm running JUnit 4 with AnyLogic. In one of my tests, I need access to the Experiment running the test. Is there any clean way to access it at runtime? E.g., is there a static method along the lines of Experiment.getRunningExperiment()?
There isn't a static method that I know of (and, if there was, it might be complicated by multi-run experiments which permit parallel execution, although perhaps not since there's still a single Experiment, though there'd be thread-safety issues).
However, you can use getEngine().getExperiment() from within a model. You probably need to explain more about your usage context. If you're using AnyLogic Pro and exporting the model to run standalone, then you should have access to the experiment instance anyway (as in the help "Running the model from outside without UI").
Are you trying to run JUnit tests from within an Experiment? If so, what's your general design? Obviously JUnit doesn't sit as well in that scenario since it 'expects' to be instantiating and running the thing to be tested. For my automated tests (where I can't export it standalone because I don't use AnyLogic Pro), I judged that it was easier to avoid JUnit (it's just a framework after all) and implement the tests 'directly' (by having my model components write outputs and, at the end of the run in the Experiment, having the Experiment compare the outputs to pre-prepared expected ones and flag if the test was passed or failed). With AnyLogic Pro, you could still export standalone and use JUnit to run the 'already-a-test' Experiments (with the JUnit test checking the Experiment for a testPassed Boolean being set at the end or whatever).
The fact that you want to get running experiments suggests that you're potentially doing this whilst runs are potentially executing. If so, could you explain a bit about your requirements?
I have a test suite of ~ 1500 tests and they generally run and finish within 'reasonable time'.
Recently, however, I've changed parts of the code to use threads -- and now my builds fail from time to time by simply timing out. I imagine that a thread refuses to die and the build waits until reaching the maximum build time.
My problem is how to detect which test is causing the problem?
Can I activate some logging that shows me that a test has started/finished? I can of course be done by inserting code in every single test method - or just the fixtures, but that is A LOT of work that I'd rather avoid.
I'd suggest upgrading to NUnit 2.5 and decorating your tests with Timeout attribute, specifying maximum per-test run time. For example, you can put this in your AssemblyInfo.cs:
[assembly: NUnit.Framework.Timeout(100)]
NUnit will now launch each test in a separate thread and cancell it if it exceeds its time slot. However, this might be costly, so it's probably better to identify long-running tests and then remove assembly-level attribute in favor of test-fixture time slots. You can also override this on individual tests, assigning them more time to run.
This way you move the timeout/hang detection from CruiseControl.Net to NUnit and get information inside the report on the tasks that did not complete properly. Unfortunately there's no way for CC.Net to get this information when it has to kill the process because of timeout.