Is it possible to tell whether a process/thread has the PF_NO_SETAFFINITY flag set? I'm running taskset on a series of process ids and some are throwing errors of the following form:
taskset: failed to set pid 30's affinity: Invalid argument
I believe this is because some processes have PF_NO_SETAFFINITY set (see Answer).
Thank you!
Yes - look at /proc/PID/stat's 'flag' field
<linux/sched.h
#define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_allowed */
Look here for details on using /proc:
http://man7.org/linux/man-pages/man5/proc.5.html
https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk65143
Example:
ps -eaf
www-data 30084 19962 0 07:09 ? 00:00:00 /usr/sbin/apache2 -k start
...
cat /proc/30084/stat
30084 (apache2) S 19962 19962 19962 0 -1 4194624 554 0 3 0 0 0 0 0 20 0 1 0 298837672 509616128 5510 18446744073709551615 1 1 0 0 0 0 0 16781312 201346799 0 0 0 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0
The flags are 4194624
Q: Do you mind specifying how you'd write a simple script that outputs
true/false based on whether you're allowed to set affinity?
A: I don't feel comfortable providing this without the opportunity to test, but you can try something like this...
flags=$(cut -f 9 -d ' ' /proc/30084/stat)
echo $(($flags & 0x40000000))
Related
I encountered this behaviour many times in many servers which processed lots of network connections.
# ss -nt state time-wait sport ne :80 and sport ne :10050 | sort -k3
0 0 127.0.0.1:13530 127.0.0.1:8888
0 0 127.0.0.1:21978 127.0.0.1:8080
0 0 127.0.0.1:32490 127.0.0.1:8080
0 0 127.0.0.1:42922 127.0.0.1:8080
0 0 127.0.0.1:50728 127.0.0.1:8080
0 0 127.0.0.1:51542 127.0.0.1:8888
0 0 127.0.0.1:6274 127.0.0.1:8888
0 0 127.0.0.1:65264 127.0.0.1:8888
0 0 172.16.40.100:10000 172.16.40.5:3010
0 0 172.16.40.100:10002 172.16.40.34:3010
0 0 172.16.40.100:10002 172.16.40.97:3020
0 0 172.16.40.100:10004 172.16.40.116:3010
0 0 172.16.40.100:10004 172.16.40.21:3010
0 0 172.16.40.100:10008 172.16.40.30:3010
0 0 172.16.40.100:10010 172.16.40.216:3020
0 0 172.16.40.100:10012 172.16.40.30:3010
0 0 172.16.40.100:10014 172.16.40.131:3010
0 0 172.16.40.100:10014 172.16.40.22:3010
0 0 172.16.40.100:10014 172.16.40.33:3010
This is a part of ss output. As you may see, there are several strings with duplicated outgoing time_wait sockets. Such as:
0 0 172.16.40.100:10002 172.16.40.34:3010
0 0 172.16.40.100:10002 172.16.40.97:3020
or
0 0 172.16.40.100:10014 172.16.40.131:3010
0 0 172.16.40.100:10014 172.16.40.22:3010
0 0 172.16.40.100:10014 172.16.40.33:3010
I googled this question but could not get a reasonable explanation of this topic.
Thanks a lot!
As you may see, there are several strings with duplicated outgoing time_wait sockets. Such as:
0 0 172.16.40.100:10002 172.16.40.34:3010
0 0 172.16.40.100:10002 172.16.40.97:3020
or
0 0 172.16.40.100:10014 172.16.40.131:3010
0 0 172.16.40.100:10014 172.16.40.22:3010
0 0 172.16.40.100:10014 172.16.40.33:3010
The lines in this display are connections, not sockets.
There are exactly zero 'duplicated sockets' here. There is a duplicated port, because at the server end the port is always the same. However either the client IP address or the client port is always different. Or both.
I wish to check whether a square unit cell, when repeated side by side, forms a valid "whole piece" or not. I am sorry that I may not have described it in an accurate way, but I really do not know how to say this using scientific terms. Let me give you two examples, one for "whole piece" and the other for "not whole piece".
1 represents rubber, and 0 represents air.
"whole piece"
0 0 1 0 0 0
0 0 1 0 0 0
1 1 1 0 1 1
0 0 1 1 1 0
0 0 1 0 0 0
0 0 1 0 0 0
"not whole piece"
0 0 1 0 0 0
0 0 1 0 0 0
1 1 1 0 1 1
0 0 0 1 1 0
0 0 1 0 0 0
0 0 1 0 0 0
As you can see, in the "whole piece" case, one can find a path (diagonal counted) from top to bottom and left to right, whereas in the "not whole piece", the path is broken. So if you lift the periodic structure up, part of it will fall apart.
How can I do it efficiently in MATLAB? Is there an efficient way? Built-in?
Update
Actually, even if bwconncomp() returns 1, the periodic structure may not be connected. Let's consider this example.
A =
0 0 0 1 0 0
0 0 1 1 0 0
1 1 1 0 1 0
0 0 1 1 1 1
0 0 1 0 0 0
0 0 1 0 0 0
>> bwconncomp(A, 4)
>> 1
If I show 4 A blocks, then it is like
A | A
-----
A | A
0 0 0 1 0 0 | 0 0 0 1 0 0
0 0 1 1 0 0 | 0 0 1 1 0 0
1 1 1 0 1 0|1 1 1 0 1 0
0 0 1 1 1 1 | 0 0 1 1 1 1
0 0 1 0 0 0 | 0 0 1 0 0 0
0 0 1 0 0 0 | 0 0 1 0 0 0
-------------------------
0 0 0 1 0 0 | 0 0 0 1 0 0
0 0 1 1 0 0 | 0 0 1 1 0 0
1 1 1 0 1 0 | 1 1 1 0 1 0
0 0 1 1 1 1 | 0 0 1 1 1 1
0 0 1 0 0 0 | 0 0 1 0 0 0
0 0 1 0 0 0 | 0 0 1 0 0 0
So as you can see, the 1 on the left side collides with the 0 on the right side of another A, causing the thing not a "whole piece". (Placed with no space deliberately to make it patent).
The term is "connected". Equivalently, each piece is called a connected component.
Use bwconncomp (Image Processing Toolbox) to determine the number of connectrd components. Let A denote your matrix. Then
c = bwconncomp(A,4);
returns a structure c whose NumObjects field indicates how many connected components there are in A. The input argument "4" is the number of neighbours you want to consider to defin connectedness: either 4 (don't take diagonals into account; that semes to be what you want) or 8 (include diagonals). So, just check if
c.NumObjects
is 1 ("whole piece") or more than 1 ("not whole piece"). In your examples, the first matrix has 1 connected component and the second has 3.
If you want it in one line:
getfield(bwconncomp(A,4),'NumObjects')==1
(or use Divakar's suggestion).
I am trying to solve the linear system of equations A'*x = B using Matlab's "mldivide" (the backslash operator) in the form:
x_transp = A'\b;
A is a square sparse matrix and that is all I know about it.
The problem is that the transpose has no effect at all, so the result of the previous line of code is the same than:
x = A\b;
So, x = x_transp. However, either if I use a new variable such that:
A_transp = A';
x_transpOK1 = A_transp\b;
or simply use:
x_transpOK2 = transp(A)\b;
the result is different (x_transpOK1 = x_transpOK2 ≠ x = x_trans).
This behavior occurs in Matlab version 7.9.0 (R2009b) but it does not happen in 7.12 (R2011a).
This, however, does not happen with silly examples I have tried (the behavior then is correct). The matrices that make this behavior arise are:
A =[0.01 -0.495 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 -0.495 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 1];
b = [8
4
0
0
0
0
0
0
0
0];
Is it some kind of precision issue? Am I making any fundamental error I cannot see?
The guys at Mathworks replied: it is a bug in the interpreter, which have been fixed in the next versions. There is no fix for 7.9.0 and they recommend the following workaround:
A_transp = A';
x = A_transp\b;
I guess this is a great example of the typical advice to always be up-to-date...
My original post on Matlab Answers
The bug report
After all the discussion, here is my answer:
#Mario_Exec.bat, it seems to me that you might want to take this to the Matlab Answers (mathworks.com/matlabcentral/answers) as maybe someone with knowledge of the actual code (ie a Matlab employee) might be able to help you more specifically. It is an interesting question but it seems there is more going on that might need more knowledge of the actual code and decision trees.
Please do post back here when you hear back. I am curious what they say!
In Perl, I generate a huge read-only data-structure once, then fork().
This is to take advantage of COW on RSS pages when forking. It works really well, but when a child process exits, it allocates all the RAM from itelf just prior dying.
Is there a way to avoid this useless allocation ?
Here is sample Perl code that shows the issue.
#! /usr/bin/perl
my $a = [];
# Allocate 100 MiB
for my $i (1 .. 100000) {
push #$a, "x" x 1024;
}
# Fork 10 other process
for my $j (1 .. 10) {
last unless fork();
}
# Sleep for a while to be able to see the RSS
sleep(5);
In the sample vmstat output, we can see that it first allocates only 100MiB, then after the 1rst sleep it allocates the whole for a short while, and then releases all of it.
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 1329660 80596 86936 0 0 21 18 160 25 0 0 100 0 0
1 0 0 1328048 80596 86936 0 0 0 0 1013 44 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1028 76 11 5 84 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1010 40 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1026 54 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1006 39 0 0 100 0 0
13 0 0 741156 80596 86936 0 0 0 0 1012 66 13 58 28 0 0
0 0 0 1329288 80596 86936 0 0 0 0 1032 60 0 0 100 0 0
Note: it seems it isn't a Perl version specific issue. As I tested 5.8.8, 5.10.1 & 5.14.2 and they all do exhibit this behavior.
Update:
As #choroba asked in comments, I also tried to undef the data-structure, but it seems that it triggers the memory-touching as the RAM is then allocated.
You can add the following snippet at the end of the first script.
# Unallocate $a
undef $a;
# Sleep for a while to be able to see the RSS
sleep(5);
Actually, as I found out myself, this behavior is a feature, and the answer lies in the Perl doc:
The exit() function does not always exit immediately.
Likewise any object destructors that need to be called
are called before the real exit.
If this is a problem, you can
call POSIX::_exit($status) to avoid END and destructor processing.
And indeed, adding it at the end of the original code sample does avoid the behavior.
# XXX - To be added just before ending the process
# Use POSIX::_exit($status) to end without allocating copy-on-write RAM
use POSIX;
POSIX::_exit(0);
Note: for this to work, the child has to exit also before the data-structure goes out of scope.
i am making image compression in matlab.
After i applied DCT on image and i had img matrix, i want to apply a threshold mask on that matrix.
mask = [1 1 1 1 0 0 0 0
1 1 1 0 0 0 0 0
1 1 0 0 0 0 0 0
1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0];
maskedImg = blkproc(img,[8 8],mask );
I used that function but it didnt work and i get error message:
Error in ==> blkproc at 67
[a, block, border, fun, params, padval] = parse_inputs(varargin{:});
According to latest Matlab documentation; closest blockproc syntax (for your case) is B = blockproc(A,[M N],fun). So apparently your mask really should be a function!
However, I recall that blkproc has been a valid Matlab function for a while ago, thus double check the proper way to call it by typing (in the command line) > help blkproc. (Al tough I'm quite confident that it will share the calling signature with blockproc [in this case]).