A common novice programmer error (at least among those using the
wrapping Perl module Net::SSH2), is to try to reuse channels.
This patchs detects that incorrect usage and fails with a
LIBSSH2_ERROR_BAD_USE error instead of hanging.
Signed-off-by: Salvador Fandino <sfandino-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>
The error message returned by libssh2_channel_open in case of a server side channel open failure is now more detailed and includes the four standard error conditions in RFC 4254.
... host cannot be NULL in line 525, because it is always
valid (e.g. at least set to "0.0.0.0") after lines 430 and 431.
Reported by Coverity CID 89807.
In one case, the error code from `_libssh2_transport_read` was being returned from `_libssh2_channel_write` without setting it as the last error by calling `_libssh2_error`. This commit fixes that.
Found when using a session whose socket had been inadvertently destroyed. The calling code got confused because via `libssh2_session_last_error` it appeared no error had occurred, despite one being returned from the previous function.
_libssh2_channel_read was using an arbitrary hard-coded limit to trigger
the window adjusting code. The adjustment used was also hard-coded and
arbitrary, 15MB actually, which would limit the usability of libssh2 on
systems with little RAM.
This patch, uses the window_size parameter passed to
libssh2_channel_open_ex (stored as remote.window_size_initial) plus the
buflen as the base for the trigger and the adjustment calculation.
The memory usage when using the default window size is reduced from 22MB
to 256KB per channel (actually, if compression is used, these numbers
should be incremented by ~50% to account for the errors between the
decompressed packet sizes and the predicted sizes).
My tests indicate that this change does not impact the performance of
transfers across localhost or a LAN, being it on par with that of
OpenSSH. On the other hand, it will probably slow down transfers on
networks with high bandwidth*delay when the default window size
(LIBSSH2_CHANNEL_WINDOW_DEFAULT=256KB) is used.
Signed-off-by: Salvador Fandino <sfandino@yahoo.com>
After filling the read buffer with data from the read queue, when the
window size was too small, "libssh2_channel_receive_window_adjust" was
called to increase it. In non-blocking mode that function could return
EAGAIN and, in that case, the EAGAIN was propagated upwards and the data
already read on the buffer lost.
The function was also moving between the two read states
"libssh2_NB_state_idle" and "libssh2_NB_state_created" both of which
behave in the same way (excepting a debug statment).
This commit modifies "_libssh2_channel_read" so that the
"libssh2_channel_receive_window_adjust" call is performed first (when
required) and if everything goes well, then it reads the data from the
queued packets into the read buffer.
It also removes the useless "libssh2_NB_state_created" read state.
Some rotted comments have also been updated.
Signed-off-by: Salvador <sfandino@yahoo.com>
Until now, the window size (channel->remote.window_size) was being
updated just after receiving the packet from the transport layer.
That behaviour is wrong because the channel queue may grow uncontrolled
when data arrives from the network faster that the upper layer consumes
it.
This patch adds a new counter, read_avail, which keeps a count of the
bytes available from the packet queue for reading. Also, now the window
size is adjusted when the data is actually read by an upper layer.
That way, if the upper layer stops reading data, the window will
eventually fill and the remote host will stop sending data. When the
upper layers reads enough data, a window adjust packet is delivered and
the transfer resumes.
The read_avail counter is used to detect the situation when the remote
server tries to send data surpassing the window size. In that case, the
extra data is discarded.
Signed-off-by: Salvador <sfandino@yahoo.com>
This partially reverts commit 03ca9020756a4e16f0294e5b35e9826ee6af2364
in order to fix extreme slowdown when uploading to localhost via SFTP.
I was able to repeat the issue on RHEL-7 on localhost only. It did not
occur when uploading via network and it did not occur on a RHEL-6 box
with the same version of libssh2.
The problem was that sftp_read() used a read-ahead logic to figure out
the window_size, but sftp_packet_read() called indirectly from
sftp_write() did not use any read-ahead logic.
When there's no window to "write to", there's no point in waiting for
the socket to become writable since it most likely just will continue to
be.
Patch-by: ncm
Fixes#258
When calling _libssh2_channel_receive_window_adjust() internally, we now
always use the 'force' option to prevent libssh2 to avoid sending the
update if the update isn't big enough.
It isn't fully analyzed but we have seen corner cases which made a
necessary window update not get send due to this and then the other side
doesn't send data our side then sits waiting for forever.
if there's not enough room to receive the data that's being requested,
the window adjustment needs to be sent to the remote and thus the force
option has to be used. _libssh2_channel_receive_window_adjust() would
otherwise "queue" small window adjustments for a later packet but that
is really terribly for the small buffer read that for example is the
final little piece of a very large file as then there is no logical next
packet!
Reported by: Armen Babakhanian
Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0130.shtml
When draining data off the socket with _libssh2_transport_read() (which
in turn has to be done so that we can be sure to have read any possible
window-increasing packets), this code previously ignored errors which
could lead to nasty loops. Now all error codes except EAGAIN will cause
the error to be returned at once.
Bug: http://www.libssh2.org/mail/libssh2-devel-archive-2012-03/0068.shtml
Reported by: Matthew Booth
Removed the automatic window_size adjustments from
_libssh2_channel_read() and instead all channel readers must now make
sure to enlarge the window sizes properly themselves.
libssh2_channel_read_ex() - the public function, now grows the window
size according to the requested buffer size. Applications can still opt
to grow the window more on demand. Larger windows tend to give higher
performance.
sftp_read() now uses the read-ahead logic to figure out a window_size.
The loop that waits for remote.close to get set may end up looping
forever since session->socket_state gets set to
LIBSSH2_SOCKET_DISCONNECTED by the packet_add() function called from the
transport_read() function and after having been set to
LIBSSH2_SOCKET_DISCONNECTED, the transport_read() function will only
return 0.
Bug: http://trac.libssh2.org/ticket/198
The attempts made to have _libssh2_channel_write() accept larger pieces
of data and split up the data by itself into 32700 byte chunks and pass
them on to channel_write() in a loop as a way to do faster operations on
larger data blocks was a failed attempt.
The reason why it is difficult:
The API only allows EAGAIN or a length to be returned. When looping over
multiple blocks to get sent, one block can get sent and the next might
not. And yet: when transport_send() has returned EAGAIN we must not call
it again with new data until it has returned OK on the existing data it
is still working on. This makes it a mess and we do get a much easier
job by simply returning the bytes or EAGAIN at once, as in the EAGAIN
case we can assume that we will be called with the same arguments again
and transport_send() will be happy.
Unfortunately, I think we take a small performance hit by not being able
to do this.
Some checks are better done in _libssh2_channel_write just once per
write instead of in channel_write() since the looping will call the
latter function multiple times per _libssh2_channel_write() invoke.
Neither _libssh2_channel_write nor sftp_write now have the 32500 size
limit anymore and instead the channel writing function now has its own
logic to send data in multiple calls until everything is sent.
The new function takes two data areas, combines them and sends them as a
single SSH packet. This allows several functions to allocate and copy
less data.
I also found and fixed a mixed up use of the compression function
arguments that I introduced in my rewrite in a recent commit.
When a call to _libssh2_transport_write() succeeds, we must return from
_libssh2_channel_write() to allow the caller to provide the next chunk
of data.
We cannot move on to send the next piece of data that may already have
been provided in this same function call, as we risk getting EAGAIN for
that and we can't return information both about sent data as well as
EAGAIN. So, by returning short now, the caller will call this function
again with new data to send.
The well known and used ssh server Dropbear has a maximum SSH packet
length at 32768 by default. Since the libssh2 design current have a
fixed one-to-one mapping from channel_write() to the packet size created
by transport_write() the previous limit of 32768 in the channel layer
caused the transport layer to create larger packets than 32768 at times
which Dropbear rejected forcibly (by closing the connection).
The long term fix is of course to remove the hard relation between the
outgoing SSH packet size and what the input length argument is in the
transport_write() function call.
As was pointed out in bug #182, we must not return failure from
_libssh2_channel_free() when _libssh2_channel_close() returns an error
that isn't EAGAIN. It can effectively cause the function to never go
through, like it did now in the case where the socket was actually
closed but socket_state still said LIBSSH2_SOCKET_CONNECTED.
I consider this fix the right thing as it now also survives other
errors, even if making sure socket_state isn't lying is also a good
idea.
Sending in NULL as the primary pointer is now dealt with by more
public functions. I also narrowed the userauth.c code somewhat to
stay within 80 columns better.
If an application accidentally provides a NULL handle pointer to
the channel or sftp public functions, they now return an error
instead of segfaulting.
To get the blocking vs non-blocking to work as smooth as possible
and behave better internally, we avoid using the external
interfaces when calling functions internally.
Renamed a few internal functions to use _libssh2 prefix when not
being private within a file, and removed the libssh2_ for one
that was private within the file.
This was triggered by a clang-analyzer complaint that turned out
to be valid, and it made me dig deeper and fix some generic non-
blocking problems I disovered in the code.
While cleaning this up, I moved session-specific stuff over to a
new session.h header from the libssh2_priv.h header.
By making 'data' and 'data_len' more local in several places in
this file it will be easier to spot how they are used and we'll
get less risks to accidentally do bad things with them.
clang-analyzer pointed out how 'data' could be accessed as a NULL
pointer if the wrong state was set, and while I don't see that
happen in real-life the code flow is easier to read and follow by
moving the LIBSSH2_FREE() call into the block that is supposed to
deal with the data pointer anyway.
clang-analyzer pointed out how 'data' could be accessed as a NULL
pointer if the wrong state was set, and while I don't see that
happen in real-life the code flow is easier to read and follow by
moving the LIBSSH2_FREE() call into the block that is supposed to
deal with the data pointer anyway.
