Does anyone know how to execute mpirun on specified CPUs? Though "mpirun -np 4 " specifies the number of CPUs used, what I here want to do is to specify CPU IDs.
The OS is CentOS 5.6 and MVAPICH2 is used on a single node with 6x2 cores.
Thank you for your cooperation.
Yes; new versions of mvapich2 use the hwloc library to enable CPU affinity and binding.
From the User Guide:
For example, if you want to run 4 processes per node and utilize cores
0, 1, 4, 5 on each node, you can specify:
$ mpirun_rsh -np 64 -hostfile hosts MV2_CPU_MAPPING=0:1:4:5 ./a.out
or
$ mpiexec -n 64 -f hosts -env MV2_CPU_MAPPING 0:1:4:5 ./a.out
In this way, process 0 on each node will be mapped to core 0, process
1 will be mapped to core 1, process 2 will be mapped to core 4, and
process 3 will be mapped to core 5. For each process, the mapping is
separated by a single “:”.
Related
I'm working with a large computing cluster with a SLURM workload manager that has four different subsections: we'll call them C1, C2, C3, and C4. Nodes in C1 and C2 have 28 cores, whereas those in C3 and C4 have 40 and 52 cores, respectively. I would like to be able to use all cores per node, but when I submit a job to the queue, I have no idea to which subsection it will be assigned and therefore don't know how many cores will be available. Is there a variable in SLURM to plug into --ntasks-per-node that will tell it to use all available cores on the node?
If you request a full node, with --nodes=1 --exclusive, you will get access to all CPUs (which you can check with cat /proc/$$/status|grep Cpus). The number of CPUs available will be given by the SLURM_JOB_CPUS_PER_NODE environment variable.
But the number of tasks will be one so you might have to adjust how you start your program and set the number of CPUs explicitly, for instance with an OpenMPI program a.out:
mpirun -np $SLURM_JOB_CPUS_PER_NODE ./a.out
I am using PostgreSQL 9.5 with a Java 8 Application on Windows OS (System: i5 2nd Generation). I noticed that when my application is in execution, there are several PostgreSQL processes / sub-processes that are created/removed dynamically.
These PostgreSQL processes use almost all of the CPU (>95%), due to which problem arises with other applications installed at my system.
I recently came to know about CPU affinity. For the time being, I am executing PowerShell script (outside of my Java application) which checks periodically and sets the desired value of cpu affinity for all PostgreSQL processes in execution.
I am looking for a way where I don't need to execute an external script(s) and/or there is one-time configuration required.
Is there a configuration supported by PostgreSQL 9.5 through which we can set max CPU cores to be used by PostgreSQL processes?
I looked for the solution, but could not find any.
There is no way to set this in the PostgreSQL configuration.
But you can start your PostgreSQL server from cmd.exe with:
start /affinity 3 C:\path\to\postgresql.exe C:\path\to\data\directory
That would allow PostgreSQL to run only on the twp “first” cores.
The cores are numbered 1, 2, 4, 8, 16 and so on, and you use the sum of the cores where you want PostgreSQL to run as argument to /affinity. For example, if you only want it to run on the third and fourth core, you would use /affinity 12.
This should work, since the Microsoft documentation says:
Process affinity is inherited by any child process or newly instantiated local process.
From the bind9 man page, I understand that the named process starts one worker thread per CPU if it was able to determine the number of CPUs. If its unable to determine, a single worker thread is started.
My question is how does it calculate the number of CPUs? I presume by CPU, it means cores. The Linux machine I work is customized and has kernel 2.6.34 and does not support lscpu or nproc utilities. named is starting a single thread even if i give -n 4 option. Is there any other way to force named to start multiple threads?
Thanks in advance.
I am very new to LSF. I have 4 nodes with with 2 sockets per node. Each node is having 8 cores. I have developed hybrid MPI+OpenMP code. I am submitting the job like the following which asks each core to perform one MPI task. So I loose the power of OpenMP.
##BSUB -n 64
I wish to submit the job so that each socket runs one MPI task rather than each core so that the cores inside the socket can be used for OpenMP. How can I build up job submit scripts to optimize the power of the Hybridization in my code.
First of all, the BSUB sentinels have to be preceded by a single # sign, otherwise they are skipped over as a regular comments.
The correct way to start a hybrid job with older LSF versions is to pass the span resource request and request nodes exclusively. To start a job with 8 MPI processes and 8 OpenMP threads each, you should use the following:
#BSUB -n 8
#BSUB -x
#BSUB -R "span[ptile=2]"
The parameters are as following:
-n 8 - requests 8 slots for MPI processes
-x - requests nodes exclusively
-R "span[ptile=2]" - instructs LSF to span the job over two slots per node
You should request nodes exclusively, otherwise LSF will schedule other jobs to the same nodes since only two slots per node will be used.
Then you have to set the OMP_NUM_THREADS environment variable to 4 (the number of cores per socket), tell the MPI library to pass the variable to the MPI processes, and make the library limit each MPI process to its own CPU socket. This is unfortunately very implementation-specific, e.g.:
Open MPI 1.6.x or older:
export OMP_NUM_THREADS=4
mpiexec -x OMP_NUM_THREADS --bind-to-socket --bysocket ./program.exe
Open MPI 1.7.x or newer:
export OMP_NUM_THREADS=4
mpiexec -x OMP_NUM_THREADS --bind-to socket --map-by socket ./program.exe
Intel MPI (not sure about this one as I don't use IMPI very often):
mpiexec -genv OMP_NUM_THREADS 4 -genv I_MPI_PIN 1 \
-genv I_MPI_PIN_DOMAIN socket -genv I_MPI_PIN_ORDER scatter \
./program.exe
I am using Cassandra 1.2 with the new MurMur3Partitioner on centos.
On a 2 node cluster both set up with num_tokens=256
I see that one node is using much more memory than the other after inserting a couple million rows with CQL3.
When I run the free command
it shows 6GB usage on the second node and 1GB on the seed node.
However, when running
ps -e -o pid,vsz,comm= | sort -n -k 2
It shows the java process using about 6.8GB on each node.
Note that I have
MAX_HEAP_SIZE="4GB"
HEAP_NEWSIZE="400M"
set in cassandra-env.sh on each node.
Can anyone provide some insight?
This is most likely related to the general difficulties around reporting accurate memory utilization in Linux, especially as it relates to Java processes. Since Java processes reserve and allocate memory automatically, what the operating system sees can be misleading. The best way to understand what a Java process is doing is using JMX to monitor heap utilization. Tools such as VisualVM and jconsole work well for this.