Hi I have a web service that is doing huge computation and is taking more than a minute.
I have generated the proxy file of the web service and then from my client end I am using the dll(of course I generated the proxy dll).
My client side code is
TimeSeries3D t = new TimeSeries3D();
int portfolioId = 4387919;
string[] str = new string[2];
str[0] = "MKT_CAP";
DateRange dr = new DateRange();
dr.mStartDate = DateTime.Today;
dr.mEndDate = DateTime.Today;
Service1 sc = new Service1();
t = sc.GetAttributesForPortfolio(portfolioId, true, str, dr);
But since it is taking to much time for the server to compute, after 1 minute I am receiving an error message
The CLR has been unable to transition from COM context 0x33caf30 to COM context 0x33cb0a0 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.
Kindly guide me what to do?
Thanks
Are you calling this from within a UI thread? If so, that's the problem. Put long-running operations on background threads, then marshal calls back to the UI to update it (e.g. with BackgroundWorker or Control.Invoke.)
You need to refactor you service interface to have something like the Begin/End pattern, and do the long work in the thread pool.
So the client calls once to start the operation, the server runs this in the threadpool.
Then, later, the client calls again to see if the process has completed (and get the results if it has).
If the long running task can report progress so the client gets more than "done"/"not done" so much the better.
Related
I am asking myself the question: "When should you use scala.concurrent.blocking?"
If I understood correctly, the blocking {} only makes sense to be used in conjunction with the ForkJoinPool. In addition docs.scala-lang.org highlights, that blocking shouldn't be used for long running executions:
Last but not least, you must remember that the ForkJoinPool is not designed for long-lasting blocking operations.
I assume a long running execution is a database call or some kind of external IO. In this case a separate thread pools should be used, e.g. CachedThreadPool. Most IO related frameworks, like sttp, doobie, cats can make use of a provided IO thread pool.
So I am asking myself, which use-case still exists for the blocking statement? Is this only useful, when working with locking and waiting operations, like semaphores?
Consider the problem of thread pool starvation. Say you have a fixed size thread pool of 10 available threads, something like so:
implicit val myFixedThreadPool =
ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))
If for some reason all 10 threads are tied up, and a new request comes in which requires an 11th thread to do its work, then this 11th request will hang until one of the threads becomes available.
blocking { Future { ... } } construct can be interpreted as saying please do not consume a thread from myFixedThreadPool but instead spin up a new thread outside myFixedThreadPool.
One practical use case for this is if your application can conceptually be considered to be in two parts, one part which say in 90% of cases is talking to proper async APIs, but there is another part which in few special cases has to talk to say a very slow external API which takes many seconds to respond and which we have no control over. Using the fixed thread pool for the true async part is relatively safe from thread pool starvation, however also using the same fixed thread pool for the second part presents the danger of the situation where suddenly 10 requests are made to the slow external API, which now causes 90% of other requests to hang waiting for those slow requests to finish. Wrapping those slow requests in blocking would help minimise the chances of 90% of other requests from hanging.
Another way of achieving this kind of "swimlaning" of true async request from blocking requests is by offloading the blocking request to a separate dedicated thread pool to be used just for the blocking calls, something like so
implicit val myDefaultPool =
ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))
val myPoolForBlockingRequests =
ExecutionContext.fromExecutor(Executors.newFixedThreadPool(20))
Future {
callAsyncApi
} // consume thread from myDefaultPool
...
Future {
callBlockingApi
}(myPoolForBlockingRequests) // consume thread from myPoolForBlockingRequests
I am asking myself the question: "When should you use scala.concurrent.blocking?"
Well, since that is mostly useful for Future and Future should never be used for serious business logic then never.
Now, "jokes" aside, when using Futures then you should always use blocking when wrapping blocking operations, AND receive a custom ExecutionContext; instead of hardcoding the global one. Note, this should always be the case, even for non-blocking operations, but IME most folks using Future don't do this... but that is another discussion.
Then, callers of those blocking operations may decide if they will use their compute EC or a blocking one.
When the docs mention long-lasting they don't mean anything specific, mostly because is too hard to be specific about that; is context / application specific. What you need to understand is that blocking by default (note the actual EC may do whatever they want) will just create a new thread, and if you create a lot of threads and they take too long to be released you will saturate your memory and kill the program with an OOM error.
For those situations, the recommendation is to control the back pressure of your app to avoid creating too many threads. One way to do that is to create a fixed thread pool for the maximum number of blocking operations you will support and just enqueue all other pending tasks; such EC should just ignore blocking calls. You may also just have an unbound number of threads but manage the back pressure manually in other parts of your code; e.g. with an explicit Queue, this was common advice before: https://gist.github.com/djspiewak/46b543800958cf61af6efa8e072bfd5c
However, having blocked threads is always hurtful for the performance of your app, even if the compute EC is not blocked. The latest talks by Daniel explain those in detail: "The case for effect systems" & "Threads at scale".
So the ecosystem is pushing hard the state of the art to avoid that at all costs but is not a simple task. Still, runtimes like the ones provided by cats-effect or ZIO are optimized to handle blocking tasks the best they can as of today, and will probably improve during this and next years.
I'm trying to build a web application that should be able to handle at least 15000 rps. Some of the optimizations I have done is increase the worker pool size to 20 and set an accept back log to 25000. Since I have set my worker pool size to 20; wil this help with the the blocking piece of code?
A worker pool size of 20 seems to be the default.
I believe the important question in your case is how long do you expect each request to run. On my side, I expect to have thousands of short-lived requests, each with a payload size of about 5-10KB. All of these will be blocking, because of a blocking database driver I use at the moment. I have increased the default worker pool size to 40 and I have explicitly set my deploy vertical instances using the following formulae:
final int instances = Math.min(Math.max(Runtime.getRuntime().availableProcessors() / 2, 1), 2);
A test run of 500 simultaneous clients running for 60 seconds, on a vert.x server doing nothing but blocking calls, produced an average of 6 failed requests out of 11089. My test payload in this case was ~28KB.
Of course, from experience I know that running my software in production would often produce results that I have not anticipated. Thus, the important thing in my case is to have good atomicity rules in place, so that I don't get half-baked or corrupted data in the database.
I have created a web application in jsf and it has a button.
If the button is clicked then it will go to the server side and execute the below function to save the data in a table and I am using mybatis for this.
public void save(A a)
{
SqlSession session = null;
try{
session = SqlConnection.getInstance().openSession();
TestMapper testmap= session.getMapper(TestMapper.class);
testmap.insert(a);
session .commit();
}
catch(Exception e){
}
finally{
session.close();
}
}
Now i have deployed this application in an application server JBoss(wildfly).
As per my understanding, when multiple users try to access the application
by hitting the URL, the application server creates thread for each of the user request.
For example if 4 clients make request then 4 threads will be generated that is t1,t2,t3 and t4.
If all the 4 users hit the save button at the same time, how save method will be executed, like if t1 access the method and execute insert statement
to insert data into table, then t2,t3 and t4 or simultaneously all the 4 threads will execute the insert method and insert data?
To bring some context I would describe first two possible approaches to handling requests. In this case HTTP but these approaches do not depend on the protocol used and the main important thing is that requests come from the network and for their execution some IO is needed (either access to filesystem or database or network calls to other systems). Note that the following description has some simplifications.
These two approaches are:
synchronous
asynchronous
In general to process the typical HTTP request that involves DB access at least four IO operations are needed:
request handler needs to read the request data from the client socket
request handler needs to write request to the socket connected to the DB
request handler needs to read response from the DB socket
request handler needs to write the response to the client socket
Let's see how this is done for both cases.
Synchronous
In this approach the server has a pool (think a collection) of threads that are ready to serve a request.
When the request comes in the server borrows a thread from the pool and executes a request handler in that thread.
When the request handler needs to do the IO operation it initiates the IO operation and then waits for its completion. By wait I mean that thread execution is blocked until the IO operation completes and the data (for example response with the results of the SQL query) is available.
In this case concurrency that is requests processing for multiple clients simultaneously is achieved by having some number of threads in the pool. IO operations are much slower if compared to CPU so most of the time the thread processing some request is blocked on IO operation and CPU cores can execute stages of the request processing for other clients.
Note that because of the slowness of the IO operations thread pool used for handling HTTP requests is usually large enough. Documentation for sync requests processing subsystem used in wildfly says about 10 threads per CPU core as a reasonable value.
Asynchronous
In this case the IO is handled differently. There is a small number of threads handling IO. They all work the same way and I'll describe one of them.
Such thread runs a loop which basically waits for events and every time an event happen it calls a handler for an event.
The first such event is new request. When a request processing is started the request handler is invoked from the loop that is run by one of the IO threads. The first thing the request handler is doing it tries to read request from the client socket. So the handler initiates the IO operation on the client socket and returns control to the caller. That means that the thread is released and it can process another event.
Another event happens when the IO operations that reads from client socket got some data available. In this case the loop invokes the handler at the point where the handler returned the control to the loop after the IO initiate namely it is resumed on the next step that processes the input data (like parses HTTP parameters) and initiates new IO operation (in this case request to the DB socket). And again the handler releases the thread so it can handler other events (like completion of IO operations that are part of other clients' requests processing).
Given that IO operations are slow compared to the speed of CPU itself one thread handling IO can process a lot of requests concurrently.
Note: that it is important that the requests handler code never uses any blocking operation (like blocking IO) because that would steal the IO thread and will not allow other requests to proceed.
JSF and Mybatis
In case of JSF and mybatis the synchronous approach is used. JSF uses a servlet to handle requests from the UI and servlets are handled by the synchronous processors in WildFly. JDBC which is used by mybatis to communicate to a DB is also using synchronous IO so threads are used to execute requests concurrently.
Congestions
All of the above is written with the assumption that there is no other sources of the congestion. By congestion here I mean a limitation on the ability of the certain component of the system to execute things in parallel.
For example imagine a situation that a database is configured to only allow one client connection at a time (this is not a reasonable configuration and I'm using this only to demonstrate the idea). In this case even if multiple threads can execute the code of the save method in parallel all but one will be blocked at the moment when they try to open the connection to the database.
Another similar example is if you are using sqlite database. It only allows one client to write to the DB at a time. So at the point when thread A tries to execute insert it will be blocked if the is another thread B that is already executing the insert. And only after the commit executed by the thread B the thread A would be able to proceed with the insert. The time A depends on the time it take for B to execute its request and the number of other threads waiting to do a write operation to the same DB.
In practice if you are using a RDBMS that scales better (like postgresql, mysql or oracle) you will not hit this problem when using the small number of connection. But it may become a problem when there is a big number of concurrent requests and there is a limitation in the DB on the number of client connections or the connection pool is used to limit the number of connections on the application side. In this case if there are already many connections to the database the new clients will wait until existing requests are finished and connections are closed.
I have a requirement to
read a Database table
process the data (dataCleanser)
to, increase the throughput, I implemented EJB timers (non persistent) that wake up every 5 minutes (10 of them) and do the above work.
The problem is 'back pressure', the dataCleanser can sometimes take like 12 minutes (makes an external API call) and when this happens, Websphere reports a hung thread.
In those cases, I would like to decrease the number of timers (say from 10 to 5) programatically.
I can do that only if the timer comes back reports its status of successful execution or exception or timeout
that way I can control the back pressure.
is there anyway to do that in Websphere 8?
To ask the question in another way
- can the EJB timers(with transaction_not_supported) invoke another EJB that have transaction timeouts?
- can those timeouts be caught in the calling timer code?
if that is not possible, what are the downsides of using the plain old infinite loop with sleep and then invoke and EJB(in turn calls dataCleanser) with transaction timeout?
one of the downside, I know is this becomes single threaded and I do not how to make 10 parallel executions like I would do with timers.
I had some similar issues with scheduling, I decided to re-schedule callbacks programmatically each time my logic ended, based on the results of processing. You can have the timer service injected:
#Resource
private TimerService timerService;
This can be in a superclass if there are multiple EJBs that need scheduling, with methods like:
protected void reschedule(long millis) {
timerService.createTimer(millis, null);
}
This way you can control beans individually. I would not try to make them control each other, since that would become difficult in a cluster with multiple JVMs.
I am working with Play Framework (Scala) version 2.3. From the docs:
You can’t magically turn synchronous IO into asynchronous by wrapping it in a Future. If you can’t change the application’s architecture to avoid blocking operations, at some point that operation will have to be executed, and that thread is going to block. So in addition to enclosing the operation in a Future, it’s necessary to configure it to run in a separate execution context that has been configured with enough threads to deal with the expected concurrency.
This has me a bit confused on how to tune my webapp. Specifically, since my app has a good amount of blocking calls: a mix of JDBC calls, and calls to 3rd party services using blocking SDKs, what is the strategy for configuring the execution context and determining the number of threads to provide? Do I need a separate execution context? Why can't I simply configure the default pool to have a sufficient amount of threads (and if I do this, why would I still need to wrap the calls in a Future?)?
I know this ultimately will depend on the specifics of my app, but I'm looking for some guidance on the strategy and approach. The play docs preach the use of non-blocking operations everywhere but in reality the typical web-app hitting a sql database has many blocking calls, and I got the impression from reading the docs that this type of app will perform far from optimally with the default configurations.
[...] what is the strategy for configuring the execution context and
determining the number of threads to provide
Well, that's the tricky part which depends on your individual requirements.
First of all, you probably should choose a basic profile from the docs (pure asynchronous, highly synchronous or many specific thread pools)
The second step is to fine-tune your setup by profiling and benchmarking your application
Do I need a separate execution context?
Not necessarily. But it makes sense to use separate execution contexts if you want to trigger all your blocking IO-calls at once and not in a sequential way (so database call B does not have to wait until database call A is finished).
Why can't I simply configure the default pool to have a sufficient
amount of threads (and if I do this, why would I still need to wrap
the calls in a Future?)?
You can, check the docs:
play {
akka {
akka.loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = WARNING
actor {
default-dispatcher = {
fork-join-executor {
parallelism-min = 300
parallelism-max = 300
}
}
}
}
}
With this approach, you basically are turning Play into a one-thread-per-request-model. This is not the idea behind Play, but if you're doing a lot of blocking IO calls, it's the simplest approach. In this case, you don't need to wrap your database calls in a Future.
To put it in a nutshell, you basically have three ways to go:
Only use (IO-)technologies whose API calls are non-blocking and asynchronous. This allows you to use a small threadpool / default execution context which suits the nature of Play
Turn Play into a one-thread-per-request Framework by drastically increasing the default execution context. No futures needed, just call your blocking database as always
Create specific execution contexts for your blocking IO-calls and gain fine-grained control of what you are doing
Firstly, before diving in and refactoring your app, you should determine whether this is actually a problem for you. Run some benchmarks (gatling is superb) and do a few profiles with something like JProfiler. If you can live with the current performance then happy days.
The ideal is to use a reactive driver which would return you a future that then gets passed all the way back to your controller. Unfortunately async is still an Open ticket for slick. Interacting with REST APIs can be made reactive using the PlayWS library, but if you have to go via a library that your 3rd party provides then you're stuck.
So, assuming that none of these are feasible and that you do need to improve performance, the question is what benefit would Play's suggestion have? I think what they're getting at here is that it's useful to partition your threads into those that block and those that can make use of asynchronous techniques.
If, for instance, only some proportion of your requests are long and blocking then with a single thread pool you risk all threads being used for the blocking operations. Your controller would then not be able to handle any new requests, irrespective of whether that request needs to call a blocking service. If you can allocate enough threads that this never happens then no problem.
If, on the other hand, you are hitting your limit for threads then by using two pools you can keep your fast, non-blocking requests snappy. You would have one pool servicing requests in your controller and calling into services which return futures. Some of these futures would actually be performing work using a separate pool of threads, but only for the blocking operations. If there is any portion of your app which could be made reactive, then your controller could take advantage of this while isolating the controller from the blocking operations.