on Linux, sending the header and then the message data does severely
impact performances of ptl/tcp :
on the receiver, reading the data can often result in an PMIX_ERR_RESOURCE_BUSY
or PMIX_ERR_WOULD_BLOCK, which ends up degrading performances)
this commit send both header and message data at the same time via writev()
and makes ptl/tcp virtually as efficient as ptl/usock.
Short writev generally occur when the kernel buffer is full, so there is no
point for retrying in this case.
fwiw, no such degradation was observed on OSX.
Refs open-mpi/ompi#2657
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
a01960bee5