I am looking an ability to execute <tasks> on separate threads, with the ability to join those separate threads after execution is done... does such a thing exist within any version of NAnt?
From the comments: Parallel task execution on NAnt links to http://jayflowers.com/WordPress/?p=101
... and answering in a comment is bad form. See https://meta.stackexchange.com/questions/131515/many-questions-answered-via-comment-instead-of-answer#comment362688_131515
Related
I have several recipes who's do_compile task uses a lot of memory (lots of C++ templates). If I build the recipes at the same time, they exhaust the host machine of memory and the out-of-memory killer starts killing.
I've modified BB_NUMBER_PARSE_THREADS, BB_NUMBER_THREADS, PARALLEL_MAKE, and PARALLEL_MAKEINST numerous times, but it is not feasible to pick numbers for these variables that'll work well in all situations. For example, if I set BB_NUMBER_THREADS to 1 in order to get only one of these recipes to build at a time, I end up increasing the build time a lot when there are no changes (everything can be pulled from the cache). I don't feel like those are the right solution to my problem.
Is there any way to tell bitbake to only build one of these recipe's do_compile tasks at a time, but let other recipe's tasks build normally?
It isn't quite the answer you're looking for but you could have something like:
do_compile[lockfiles] = "${WORKDIR}/mylock"
which would require the task to take and hold the lock to execute, then you can be sure only one would run at a time.
I asked this auestion on NUnit-Discuss, but i realize that group is not very active, so i give it a try here:
We've been using MSTests until now for some functional tests.
I know, neither MSTest nor NUnit is really for functional test, but we need those tests with a simple integration in Visual Studio.
The tests will launch other executables, connect, do stuff, disconnect and kill the processes.
We're having trouble with MSTest in that it launched tests in a separate thread and seems that some execution is overlapping between tests, even when executed sequentially.
So i'm thinking about moving to NUnit.
The question i have is:
Can NUnit be configured in any way such as to give the following guarantees:
Tests will be executed sequentially, in an order that can be specified.
Tests will be executed from the same thread.
TearDown code of one test will have been fully executed before Setup code of a following test will be called.
If so, what would be that configuration, if any particular?
Thank you.
By default, NUnit does not execute any tests in parallel. If you never use the ParallelizableAttribute, then your tests run one at a time.
Of course, that does not mean your tests can't break NUnit, for example, by starting a thread or process that never terminates after the test thread terminates. NUnit only takes responsibility for the tests it runs itself.
NUnit does not guarantee that all tests will be executed from the same thread. That is a separate matter from parallelization, of course. Separate threads may be started for each thread, based on attributes you specify. You may, for example, designate some tests to run in a Single-threaded Apartment, while others run by default in an MTA. You might use the RequiresThreadAttribute, which asks NUnit to use a new thread for the test it decorates. You might use the SingleThreadedAttribute on a class, to indicate that all the code in that class runs on the same thread.
One trick, which is currently available but which may not exist in all future releases, is to specify --workers=0 on the command-line to nunit3-console. That tells NUnit to simply run the tests without creating any test workers and gives an execution path that more closely resembles that of NUnit V2.
So, in general, I think your needs can be met, but it could require some tinkering with your tests to make it work the way you want.
I am exploring Talend at work, I was asked if Talend supports batch processing as in running the job in multiple threads. After going through the user guide I understood threading is possible with sub jobs. I would like to know if it is possible to run the a job with a single action in parallel
Talend has excellent multi threading support. There are two basic methods for this. One method gives you more control and is implemented using components. The other method is implemented as job setting.
For the first method see my screenshot. I use tParallelize to load three files into three tables at the same time. Then when all three files are successfully loaded I use the same tParallelize to set the values of a control table. tParallelize can also be connected to tRunJob as easily as a subjob.
The other method is described very well here in Talend Help: Talend Help- Run Jobs in Parallel
Generally I recommend the first method because of the control it gives you, but if your job follows the simple pattern described in the help link, that method works as well.
Trying to troubleshoot an existing Talend job with many iterations and sub-jobs created by a developer who is no longer with the company. Ran into an issue with subjobs and hoping someone here can answer.
I know by reading the documentation that OnSubjobOk10 indicates that the job will execute after #10 is complete. But in a workflow with no names, how I do know which is Subjob#10? Can I assume it is the one from where the job-job connection is made?
Thanks in advance,
Bee
OnSubJobOK will make te next subjob work if the previous subjob finished without error, from help.talend:
OnSubjobOK (previously Then Run): This link is used to trigger the
next subjob on the condition that the main subjob completed without
error. This connection is to be used only from the start component of
the Job.
These connections are used to orchestrate the subjobs forming the Job
or to easily troubleshoot and handle unexpected errors.
I need to know that how can we run a single job in parallel with different parameters in talend.
The answer is straightforward, but rather depends on what you want, and whether you are using free Talend or commercial.
As far as parameters go, make sure that your jobs are using context variables - this is the preferred way of passing parameters in.
As for running in parallel, there are a few options.
Talend's studio is a java code generator, so you can export your job (it's just java code) and run it wherever you want. How you invoke it is up to you - schedule it, invoke it N times manually, your call. Obviously, if your job touches shared resources then making it safe to run in parallel is up to you - the usual concurrency issues apply.
If you have the commercial product, then you can use the Talend admin centre (TAC). The TAC allows you to schedule a job more than once with different contexts. Or, if you want to keep the parallelization logic inside your job, then consider using the tParallelize component in one job to run another job N times.