when we run the show dbs command, the x-development database doesn't appear.
> show dbs
admin (empty)
local 0.203125GB
p-development 0.203125GB
p-production 3.9521484375GB
test (empty)
>
we want to make sure this database isn't taking server resources, but mongostat keeps listing it. why does this happen if the database doesn't exist?
[root#mongo]# mongostat
connected to: 127.0.0.1
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time
0 5 0 0 0 1 1 2.11g 4.86g 464m 0 x-development:0.0% 0 0|0 0|0 62b 2k 3 11:42:57
0 0 0 0 0 1 0 2.11g 4.86g 464m 0 x-development:0.0% 0 0|0 0|0 62b 2k 3 11:42:58
0 0 0 0 0 1 0 2.11g 4.86g 464m 0 x-development:0.0% 0 0|0 0|0 62b 2k 3 11:42:59
It don't think it's strange or wrong that mongo chooses to retain statistics of a deleted database. After all they are statistics.
Deleting the database does not change the fact that you ran queries against it before.
Related
I try to replace values per row that enclosed between two values within accepatable tolerance range/distance between the two values (non-zero). For example:
A=[0 0 0 0 1 *0 *0 *0 *0 *0 -1 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 -1 *0 *0 *0 1 0 0 0;
0 0 1 *0 *0 *0 *0 *0 *0 *0 -1 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 *0 -1 *0 *0 *0 1 0 0 0]
* indicates the possibles values to be considered as an enclosure. With tolerance of 6 (for instance), the values inside will be replaced with the first boundary value:
A_replaced=[0 0 0 0 1 *1 *1 *1 *1 *1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 *-1 *-1 *-1 -1 0 0 0;
0 0 1 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 *-1 *-1 *-1 -1 0 0 0]
As you can see, everything that can be considered as an enclosure is replaced by the first boundary value (including the end closure value) since the distance between the two boundaries does not exceeds 6 (smaller than tolerance). Is there vectorized method to do this for large size of matrix? Thanks in advance!
Because I've already answered half of this question, I will do it with two steps. Note that your phrase of the question here is not the fully general case, since you need treating matrices with some continuous non-zeros in the places here that you placed only 1. This is relying on your previous question and comments.
So first step is to find all the zeros sequences that their length is under the tolerance, and replace them with 1. Second step is your previous question- replace all non-zeros with the starting value of the sequence.
Pre-first-step will be to add zeros to the beginning of A, because doing it all with linear indexing and treat A as a long vector is a much comfort, but we don't want to catch zero-sequences that cross the columns. We can remove the zeros at the end.
A=[
0 0 0 0 0 0.5 0.9 -1 0.1 0.3 1 1 0 0 0 0.9 -1 0.1 0 0 0 0 0.3 -1 -1 0 0 0 0 0;
0 0 0 0 0 0.1 0.3 1 -1 -1 -1 -1 0 0 0 0 0 -1 -1 -1 0 0 1 1 1 0 0 0 0 0 ];
tol=6;
A=[zeros(size(A,1),tol), A]; % add zeros at beginning of each row
A=A.'; % working on transposed A let us index continuously the sequences
zero_starts=strfind([A(1)==0; A(:)~=0].',[1 0]).'; % indices where `0` sequences starts
zero_starts(end)=[]; % last sequance is not enclosed
zero_length=diff([0; find(diff(find(~A))>1)]); % length of each of these zero-sequences
zero_tol=find(zero_length<=tol); % zero-sequences that shorter than tolerance
zero_tol(strfind(diff([0; zero_tol].')-1>0,[1 0])+1)=[]; % remove each second of continuous zero-seqeunces
% replace all short zero-seqeunces with `1`
sl=arrayfun(#(x)0:x-1, zero_length(zero_tol),'uni',0);
A(repelem(zero_starts(zero_tol),zero_length(zero_tol))+cat(2,sl{:}).')=1;
Now, the second step is to replace each non-zero sequences with the first element of each sequence:
B=find(A);
A(A~=0)=repelem(A(B([0 ;find(diff(B)>1)]+1)),diff([0;find(diff(B)>1) ;length(B)]));
Lastly, we can reverse back A to its shape and size-
A=A(tol+1:end,:).'
A=[
0 0 0 0 0 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0 0 0 0 0.3 -1 -1 0 0 0 0 0;
0 0 0 0 0 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0 0 1 1 1 0 0 0 0 0 ];
I use MongoDB to store price events for stocks. Depending on what you want to screen, the number of event can rapidly grow to 1Go-2Go.
I run MongoDB on a single machine and it is taking longer and longer to load the data. I am not able to find a clear answer on the web if "sharding on a single server" is a benefit to read speed.
Is it the right path to increase the read speed?
insert query update delete getmore command flushes mapped vsize res faults locked db
0 1 395 0 1 395 0 63.9g 128g 2.53g 7484 prices:70.5%
0 0 14726 0 5 14728 0 63.9g 128g 2.48g 31555 prices:7.4%
0 0 0 0 0 1 0 63.9g 128g 2.48g 436 prices:0.0%
0 0 0 0 1 1 0 63.9g 128g 2.48g 0 prices:0.0%
0 0 0 0 0 1 1 63.9g 128g 2.49g 3877 .:83.9%
0 0 0 0 0 1 0 63.9g 128g 2.49g 0 prices:0.0%
I have a single mongod (2.0.2) running on a (Intel Xeon 8 core 2.1 Ghz, 32 GB RAM) server. Here are some sample stats from mongostat (usual normal calm day):
insert query update delete getmore command flushes mapped vsize res faults locked % idx miss % qr|qw ar|aw netIn netOut conn repl time
0 186 3 0 0 5 0 42.6g 87.8g 22.6g 2 0.8 0 0|0 0|0 35k 1m 319 M 20:36:00
0 177 3 0 0 4 0 42.6g 87.8g 22.5g 2 0.7 0 0|0 0|0 28k 993k 319 M 20:36:30
0 181 3 0 0 3 0 42.6g 87.8g 22.6g 1 0.6 0 0|0 0|1 28k 878k 319 M 20:37:00
0 177 4 0 0 4 0 42.6g 87.8g 22.6g 2 0.7 0 0|0 0|0 31k 851k 319 M 20:37:30
0 171 2 0 0 3 0 42.6g 87.8g 22.6g 2 0.4 0 0|0 1|0 25k 912k 319 M 20:38:00
0 133 1 0 0 3 0 42.6g 87.8g 22.5g 0 0.3 0 0|0 0|0 20k 673k 319 M 20:38:30
0 180 3 0 0 4 0 42.6g 87.8g 22.5g 1 0.6 0 0|0 1|0 29k 890k 319 M 20:39:00
But sometimes when there are 500-600 users online (I store visit counters in Mongo, so there are a lot of updates when visitors come) queries jump up to ~500 per second and read queue grows quickly and stays at around 40-50 for few minutes which makes scripts time out.
Can adding a replica set member on the same machine (I don't have any more physical servers) help me? I want to set the read preference to point to the secondary member so that writes on the primary instance do not block the reads.
I'm running a MongoDB instance using a Replica Set, when there are a lot of insert, I can see very weird statistics on faults and locked %.
How come locked % can be more than 100 ?!
Where does the faults happen, I have no logs mentioning any fault, does someone have any clue about what it means ?
insert query update delete getmore command flushes mapped vsize res faults locked % idx miss % qr|qw ar|aw netIn netOut conn set repl time
9 0 0 0 1 4 0 70.3g 141g 4.77g 20 124 0 0|0 0|1 1m 2m 10 socialdb M 18:49:49
18 0 0 0 3 1 0 70.3g 141g 4.77g 17 73.8 0 0|0 0|1 1m 2m 10 socialdb M 18:49:50
21 0 0 0 1 5 0 70.3g 141g 4.77g 18 104 0 0|0 0|1 1m 1m 10 socialdb M 18:49:51
20 0 0 0 3 1 0 70.3g 141g 4.78g 18 98.8 0 0|0 0|1 1m 3m 10 socialdb M 18:49:52
172 0 0 0 5 4 0 70.3g 141g 4.79g 133 72.8 0 0|0 0|0 7m 12m 10 socialdb M 18:49:53
76 0 0 0 3 1 0 70.3g 141g 4.8g 114 65.1 0 0|0 0|1 6m 10m 10 socialdb M 18:49:54
54 0 0 0 4 4 1 70.3g 141g 4.81g 45 90.6 0 0|0 0|1 2m 8m 10 socialdb M 18:49:55
85 0 0 0 4 2 0 70.3g 141g 4.84g 101 98.1 0 0|0 0|1 6m 11m 10 socialdb M 18:49:56
77 0 0 0 3 4 0 70.3g 141g 4.82g 78 74.5 0 0|0 0|1 4m 9m 10 socialdb M 18:49:57
72 0 0 0 3 1 0 70.3g 141g 4.84g 111 95.7 0 0|0 0|1 6m 10m 10 socialdb M 18:49:58
Is there a better (standard) monitoring tool, free ?
Not sure about the other two but this could be the answer to your first question, if you are using v2.2:
http://docs.mongodb.org/manual/reference/mongostat/The above page mentions:
locked:
The percent of time in a global write lock.
(Changed in version 2.2: The locked db field replaces the locked % field to more appropriate data regarding the database specific locks in version 2.2)
locked db:
New in version 2.2.
The percent of time in the per-database context-specific lock. mongostat will report the database that has spent the most time since the last mongostat call with a write lock.
This value represents the amount of time the database had a database specific lock and the time that the mongod spent in the global lock. Because of this, and the sampling method, you may see some values greater than 100%.
I have wrote a test script which did millions of updates(using update query) in a collection. Following is the mongostat output
insert query update delete getmore command flushes mapped vsize res faults locked % idx miss % qr|qw ar|aw netIn netOut conn time
0 0 21156 0 0 1 0 208m 2.45g 119m 0 81.7 0 0|8 0|9 2m 1k 10 12:52:11
0 0 20620 0 0 1 0 208m 2.45g 119m 0 82.5 0 0|8 0|9 1m 1k 10 12:52:12
0 0 21915 0 0 1 0 208m 2.45g 119m 0 81.9 0 0|8 0|9 2m 1k 10 12:52:13
0 0 21634 0 0 1 0 208m 2.45g 119m 0 82.1 0 0|8 0|9 2m 1k 10 12:52:15
0 0 19793 0 0 1 0 208m 2.45g 119m 0 81.8 0 0|8 0|9 1m 1k 10 12:52:16
0 0 22062 0 0 1 0 208m 2.45g 119m 0 81.9 0 0|8 0|8 2m 1k 10 12:52:17
0 0 23395 0 0 1 0 208m 2.45g 119m 0 81.9 0 0|8 0|8 2m 1k 10 12:52:19
The netIn says the total network in bytes per second, i hope. Is there any way to increase the size of netIn to some mb, so that i can increase the update statement per second.
I don't think you understand the netIn statistic. It isn't some limit but the actual amount of data received by MongoDB per interval sample. In other words, the netIn value will increase if you (can) do more updates.
Increasing update throughput itself may be possible but is very application specific.