I have implemented a queue on top of TC (Tokyo Tyrant to be specific). I am using memcache compatible function increment to keep track of queue head and tail. I just want to know what is the limit for the integers stored (64 bit?) by TC and what happens when increment function is called after the number stored is at maximum value allowed? Does it starts again from 1 or does it raise error?
Just got back a reply from Mikio (creator of TC).
Yes. 2^63 is the upper limit. It will start again from 0.
If you are building for a 32-bit system, you need to enable 64-bit offsets to be able to pass the otherwise 2 GB limit.
e.g.
./configure --enable-off64
Why does it matter, theres no way in hell you will be able to ever reach 2^64 if you start counting from 1.
Related
I use Kafka Streams 2.1 and created the following stream using Suppressed feature to process the aggregation of each whole minute:
originStream
.windowedBy(TimeWindows.of(Duration.ofSeconds(60)).grace(Duration.ofMillis(500)))
.aggregate(factory::createAggregation,
(k, v, a) -> a.aggregate(v),
materialized.withLoggingDisabled())
.suppress(untilWindowCloses(Suppressed.BufferConfig.unbounded()))
.toStream();
The rate of messages I receive is about 200 per second.
After a short time I see the GC starting to work very hard, and sometimes OOM errors.
Since I use a heap of 2GB and a record will not take more than 1KB, it is clear to me that something is wrong - there shouldn't be so many messages in a window of 1 minute to explode a 2GB heap.
So.. I took a heap dump, in which I see 5 InMemoryTimeOrderedKeyValueBuffer Objects taking more than 300MB each (total >1.5GB).
I dived some more into one of those, and saw that the smallest/highest timestamp in their sortedMap was 1,575,458,160,000/1,575,481,800,000. This means that the buffer holds messages during a period of 23,640,000 = 394 minutes.
To my understanding the buffer was supposed to be flushed, so that only the last minute will consume the memory - all other windows should have been evicted.
Am I doing something wrong?
Any help would be appriciated.
The problem should not be suppress() but the aggregation state store. By default, it has a retention time of 1 day. You can reduce the retention time by passing in Materialized.withRetention(...) into aggregate().
I am surprised that your heap dump shows InMemoryTimeOrderedKeyValueBuffer though, because this is the store used by suppress(). Hence, I am not 100% sure if reducing the retention time will fix the issue.
Btw: that there are a few bugs in suppress() in 2.1 version that are only fixed in 2.3 release and thus it's highly recommended to upgrade to 2.3 if you use suppress().
I've changed The BufferConfig to use max-bytes boundary:
Suppressed.BufferConfig.unbounded().withMaxBytes(10_000_000)
and that seem to solve the problem. I looked at the code, and don't understand why - because I see it now should have thrown an exception, but it doesn't.
So, I still don't understand something here, but the problem is solved for now.
After that I used Mattias J. Sax suggestions too, just to be even safer (Thanks).
Edit:
It happened again twice today. This means that what I did did not fix the problem (Although it may have changed its frequency).
Right now, I have no solution for this problem.
I'm starting to integrate libmemcached into my application and reading the documentation, there is a non-blocking mode flag. After a quick google, there seems to be a performance advantage to non blocking mode, but are there any disadvantages to running libmemcached in non blocking mode?
Of course there are. The disadvantage would only arise if you needed to ENSURE that the written value actually was written to memcached and did not fail. For example - you're using memcached to store a counter variable which has a sentinel that checks to see if the counter has reached a certain value before performing an operation.
In blocking mode - the memcached client will wait to get a write success response from memcached before proceeding and produce an error if it fails. This way you know the counter was updated. If you tell it to write in non-blocking mode, the client sends the request to increment the counter, but never waits to ensure that it really occurred. Because it doesn't wait, you code execution after the call resumes more quickly, but with the uncertainty of not knowing for sure the counter was ever incremented.
However, since memcached values are destroyed on a service restart (think system crash) you can't ever really be sure a value will be there. Also, with low-memory pruning you also cannot ever be sure the value is 100% correct as it may get pruned by the LRU algorithm - you'd need persistent storage to alleviate this uncertainty.
Given this inherent uncertainty, many people use non-blocking mode to get the performance gain because they can't ever be totally certain the counter value in memcached isn't reset/innacurate anyway, so why not get some performance for the tradeoff.
Hope this clarifies the issue. As a side note - MongoDB has non-blocking writes in persistent storage - which while awesome in its flexibility, gives people using non-blocking mode more of a false sense of security that the write will always succeed...
R
I have a large file I need to load into the cache (As a hash) which will be shared between perl processes. Loading the data into cache takes around 2 seconds, but we have over 10 calls per second.
Does using the compute method cause other processes to be locked out?
Otherwise, I would appreciate suggestions on how to manage the load process so that there's a guaranteed lock during load and only one load process happening!
Thanks!
Not sure about the guaranteed lock, but you could use memcached with a known key as a mutex substitute (as of this writing, you haven't said what your "cache" is). If the value for the key is false, set it to true, start loading, and then return the result.
For the requests happening during that time, you could either busy-wait, or try using a 503 "Service Unavailable" status with a few seconds in the Retry-After field. I'm not sure of the browser support for this.
As a bonus, using a time-to-live of, say, 5 minutes on the mutex key will cause the entire file to be reloaded every 5 minutes. Otherwise, if you have a different reason for reloading the file, you will need to manually delete the key.
Yesterday i understood in my Advanced Operating Systems class that there will be a limit in the number of processes that can be allowed to be placed in the Ready Queue.I would like to know that number for different operating systems.And also what happens when that number is exceeded? Meaning : what if more than that number of processes are created?
I tried to see what happens by running a small program which is
int main()
{
while(1)
system(fork());
return 0;
}
The system immediately hung.Can anyone explain why my system hung?
Some systems place no limit and will simply keep appending to a running queue as needed. There are options to restrict the maximum number of processes that a system can use but by default there are no restrictions (on some systems). On Linux you can change the ulimit which is processes per user and if you set it to something like 500 or less you will see that this program will not hang the system and will simply just run and use up CPU cycles from doing constant context switches.
By the way, what you're doing there is called a Fork Bomb and it is a small denial exploit used to cause a denial of service attack on a computer that does not have a limit on processes per user.
I'm using Perl sockets in AIX 5.3, Perl version 5.8.2
I have a server written in Perl sockets. There is a option called "Blocking", which can be set to 0 or 1. When I use Blocking => 0 and run the server and client send data (5000 bytes), I am able to recieve only 2902 bytes in one call. When I use Blocking => 1, I am able to recieve all the bytes in one call.
Is this how sockets work or is it a bug?
This is a fundamental part of sockets - or rather, TCP, which is stream-oriented. (UDP is packet-oriented.)
You should never assume that you'll get back as much data as you ask for, nor that there isn't more data available. Basically more data can come at any time while the connection is open. (The read/recv/whatever call will probably return a specific value to mean "the other end closed the connection.)
This means you have to design your protocol to handle this - if you're effectively trying to pass discrete messages from A to B, two common ways of doing this are:
Prefix each message with a length. The reader first reads the length, then keeps reading the data until it's read as much as it needs.
Have some sort of message terminator/delimiter. This is trickier, as depending on what you're doing you may need to be aware of the possibility of reading the start of the next message while you're reading the first one. It also means "understanding" the data itself in the "reading" code, rather than just reading bytes arbitrarily. However, it does mean that the sender doesn't need to know how long the message is before starting to send.
(The other alternative is to have just one message for the whole connection - i.e. you read until the the connection is closed.)
Blocking means that the socket waits till there is data there before returning from a recieve function. It's entirely possible there's a tiny wait on the end as well to try to fill the buffer before returning, or it could just be a timing issue. It's also entirely possible that the non-blocking implementation returns one packet at a time, no matter if there's more than one or not. In short, no it's not a bug, but the specific 'why' of it is the old cop-out "it's implementation specific".