Thursday, January 10, 2013

How to change processor affinity in Windows and Linux

We have been playing multiple core processor systems for a long time. It is good to know that there are several cores available to run so many processes and threads in your system. The operating system is responsible how the processes are assigned to the cores. Usually you do not care how this process works. Actually modern operating system kernels are really successful to evenly distribute the load among the cores and can sometimes even migrate a thread or a process from one core to another if it is required. But sometimes you need to control or restrict the threads of a process to a subset of the cores. In Windows operating system you can easily restrict the threads to run on specific CPU's by using start command available since Windows XP.
Let's take a look at the start command line parameters:

s>start /?
Starts a separate window to run a specified program or command.

START ["title"] [/D path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED]
      [/NODE <NUMA node>] [/AFFINITY <hex affinity mask>] [/WAIT] [/B]
      [command/program] [parameters]

    "title"     Title to display in window title bar.
    path        Starting directory.
    B           Start application without creating a new window. The
                application has ^C handling ignored. Unless the application
                enables ^C processing, ^Break is the only way to interrupt
                the application.
    I           The new environment will be the original environment passed
                to the cmd.exe and not the current environment.
    MIN         Start window minimized.
    MAX         Start window maximized.
    SEPARATE    Start 16-bit Windows program in separate memory space.
    SHARED      Start 16-bit Windows program in shared memory space.
    LOW         Start application in the IDLE priority class.
    NORMAL      Start application in the NORMAL priority class.
    HIGH        Start application in the HIGH priority class.
    REALTIME    Start application in the REALTIME priority class.

    ABOVENORMAL Start application in the ABOVENORMAL priority class.
    BELOWNORMAL Start application in the BELOWNORMAL priority class.
    NODE        Specifies the preferred Non-Uniform Memory Architecture (NUMA)
                node as a decimal integer.
    AFFINITY    Specifies the processor affinity mask as a hexadecimal number.
                The process is restricted to running on these processors.

                The affinity mask is interpreted differently when /AFFINITY and
                /NODE are combined.  Specify the affinity mask as if the NUMA
                node's processor mask is right shifted to begin at bit zero.
                The process is restricted to running on those processors in
                common between the specified affinity mask and the NUMA node.
                If no processors are in common, the process is restricted to
                running on the specified NUMA node.
    WAIT        Start application and wait for it to terminate.
                If it is an internal cmd command or a batch file then
                the command processor is run with the /K switch to cmd.exe.
                This means that the window will remain after the command
                has been run.

                If it is not an internal cmd command or batch file then
                it is a program and will run as either a windowed application

                or a console application.

    parameters  These are the parameters passed to the command/program.

NOTE: The SEPARATE and SHARED options are not supported on 64-bit platforms.

Specifying /NODE allows processes to be created in a way that leverages memory
locality on NUMA systems.  For example, two processes that communicate with
each other heavily through shared memory can be created to share the same
preferred NUMA node in order to minimize memory latencies.  They allocate
memory from the same NUMA node when possible, and they are free to run on
processors outside the specified node.

    start /NODE 1 application1.exe
    start /NODE 1 application2.exe

These two processes can be further constrained to run on specific processors
within the same NUMA node.  In the following example, application1 runs on the
low-order two processors of the node, while application2 runs on the next two
processors of the node.  This example assumes the specified node has at least
four logical processors.  Note that the node number can be changed to any valid
node number for that computer without having to change the affinity mask.

    start /NODE 1 /AFFINITY 0x3 application1.exe
    start /NODE 1 /AFFINITY 0xc application2.exe

If Command Extensions are enabled, external command invocation
through the command line or the START command changes as follows:

non-executable files may be invoked through their file association just

    by typing the name of the file as a command.  (e.g.  WORD.DOC would
    launch the application associated with the .DOC file extension).
    See the ASSOC and FTYPE commands for how to create these
    associations from within a command script.

When executing an application that is a 32-bit GUI application, CMD.EXE

    does not wait for the application to terminate before returning to
    the command prompt.  This new behavior does NOT occur if executing
    within a command script.

When executing a command line whose first token is the string "CMD "

    without an extension or path qualifier, then "CMD" is replaced with
    the value of the COMSPEC variable.  This prevents picking up CMD.EXE
    from the current directory.

When executing a command line whose first token does NOT contain an

    extension, then CMD.EXE uses the value of the PATHEXT
    environment variable to determine which extensions to look for
    and in what order.  The default value for the PATHEXT variable


    Notice the syntax is the same as the PATH variable, with

    semicolons separating the different elements.

When searching for an executable, if there is no match on any extension, then looks to see if the name matches a directory name.  If it does, the START command launches the Explorer on that path.  If done from the command line, it is the equivalent to doing a CD /D to that path.

The option affinity is the one we are interested in.  This option takes one parameter which instructs the process scheduler that the process threads run only on the selected subset of the cores. The parameter is an integer given in base 16. For 2-core hyper-thread cpu, operating systems senses 4 logical CPU's and let's denote them as CPU0, CPU1, CPU2 and CPU3. In the table given below, you can easily follow how the parameter controls which cores are used:

Parameter CPU3 CPU2 CPU1 CPU0
0x01 - - - +
0x02 - - + -
0x03 - - + +
0x04 - + - -
0x05 - + - +
0x06 - + + -
0x07 - + + +
0x08 + - - -
0x09 + - - +
0x0A + - + -
0x0B + - + +
0x0C + + - -
0x0D + + - +
0x0E + + + -
0x0F + + + +

where + means the corresponding CPU is used, and - means the corresponding CPU is NOT used in the scheduling. To verify that the start command with its options is working properly, i have written the following very simple java code:

package com.example.console;

public class TestCpuAffinity {
    public static void main(String[] args) {
    System.err.println("Number of available processor: "+Runtime.getRuntime().availableProcessors()); 

The code simply prints the number of cores available out to the console. Here are the results:

start /affinity 0xF "running" /B java com.example.console.TestCpuAffinity
Number of available processors: 4

start /affinity 0xA "running" /B java com.example.console.TestCpuAffinity
Number of available processors: 2

start /affinity 0x1 "running" /B java com.example.console.TestCpuAffinity
Number of available processors: 1

start /affinity 0x2 "running" /B java com.example.console.TestCpuAffinity
Number of available processors: 1

start /affinity 0x3 "running" /B java com.example.console.TestCpuAffinity
Number of available processors: 2

It is easy to do the same thing in Linux with the help of taskset command which is basically use the same mask defined in Windows. The manual page of taskset is given below:

TASKSET(1)                    Linux User’s Manual                   TASKSET(1)

       taskset - retrieve or set a process’s CPU affinity

       taskset [options] mask command [arg]...
       taskset [options] -p [mask] pid

       taskset  is  used  to  set  or retrieve the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity.  CPU affinity is a scheduler property that "bonds" a process to a given set of CPUs on the system.  The Linux scheduler  will honor  the  given CPU affinity and the process will not run on any other CPUs.  Note that the Linux scheduler also supports natural CPU affinity: the scheduler attempts to keep processes on the same CPU as long as practical for performance reasons.  Therefore, forcing  a specific CPU affinity is useful only in certain applications.

       The  CPU  affinity  is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU.  Not all CPUs may exist on a given system but a mask may specify more CPUs than are present.
       A  retrieved  mask will reflect only the bits that correspond to CPUs physically on the system.  If an invalid mask is given (i.e., one that corresponds to no valid CPUs on the current system) an error is returned.  The masks are  typically  given  in  hexadecimal. For example,

              is processor #0

              is processors #0 and #1

              is all processors (#0 through #31)

       When taskset returns, it is guaranteed that the given program has been scheduled to a legal CPU.

       -p, --pid
              operate on an existing PID and not launch a new task

       -c, --cpu-list
              specify  a  numerical  list  of  processors  instead of a bitmask.  The list may contain multiple items, separated by comma, and
              ranges.  For example, 0,5,7,9-11.

       -h, --help
              display usage information and exit

       -V, --version
              output version information and exit

       The default behavior is to run a new command with a given affinity mask:
              taskset mask command [arguments]

       You can also retrieve the CPU affinity of an existing task:
              taskset -p pid

       Or set it:
              taskset -p mask pid

Here is the sample usage of taskset command on running firefox:

[root@server1 ~]# taskset -c 1 firefox &
[1] 1032
[root@server1 ~]# taskset -p 1032
pid 1032's current affinity mask: 2

No comments:

Post a Comment