If a process is in certain states (e.g., polling for messages in the event lib), then it can blissfully ignore SIGTERM when we try to order it to die. Unfortunately, the OS thinks the process actually did die, leading us to leave orphaned procs around.

The only sure way to kill the thing is with SIGKILL. After hours spent trying to debug this bizarre situation with a reliable reproducer, I finally tracked it down and fixed it.

Go figure...I sure can't.

This commit was SVN r22220.

orte/mca/odls/base/odls_base_default_fns.c

@@ -2959,9 +2959,9 @@ int orte_odls_base_default_kill_local_procs(opal_pointer_array_t *procs, bool se
              value. */
             kill_local(child->pid, SIGCONT);
             
-            /* Send a sigterm to the process.  If we get ESRCH back, that
+            /* Send a sigkill to the process.  If we get ESRCH back, that
              means the process is already dead, so just move on. */
-            if (0 != (err = kill_local(child->pid, SIGTERM))) {
+            if (0 != (err = kill_local(child->pid, SIGKILL))) {
                 orte_show_help("help-odls-default.txt",
                                "odls-default:could-not-send-kill",
                                true, orte_process_info.nodename, child->pid, err);