MTL_OFI: Changed Recv cancel to be non-blocking

Updated the OFI MTL's Recv cancel to be a non-blocking call to match
the MPI spec. Given fi_cancel succeeded, then it is expected that the
user will wait on the request to read the result of if the cancel has
completed.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com
(cherry picked from commit 25bdd118acd0f9d9c43662def0a00afde708992f)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>

ompi/mca/mtl/ofi/mtl_ofi.h

@@ -1213,14 +1213,8 @@ ompi_mtl_ofi_cancel(struct mca_mtl_base_module_t *mtl,
                 ret = fi_cancel((fid_t)ompi_mtl_ofi.ofi_ctxt[ctxt_id].rx_ep,
                                &ofi_req->ctx);
                 if (0 == ret) {
-                    /**
-                     * Wait for the request to be cancelled.
-                     */
-                    while (!ofi_req->super.ompi_req->req_status._cancelled) {
-                        opal_progress();
                     if (ofi_req->req_started)
                         goto ofi_cancel_not_possible;
-                    }
                 } else {
 ofi_cancel_not_possible:
                     /**