Sockets Python 3.5: Socket Server Hangs Forever On File Receive

I'm trying to write a Python program that can browse directories and grab files w/ sockets if the client connects to the server. The browsing part works fine, it prints out all dir

Solution 1:

I think I know what's the problem, but I may be wrong. It happened to me several times, that the entire message is not received in one recv call, even if I specify the correct length. However, you don't reach the end of stream, so your program keeps waiting for remaining of 8192 bytes which never arrives.

Try this: Sending file:

#bit that sends the filewithopen(msg.split()[1], 'rb') as r:
   data = r.read()
   # check data length in bytes and send it to client
   data_length = len(data)
   s.send(data_length.to_bytes(4, 'big'))
   s.send(data)
s.shutdown(socket.SHUT_RDWR)
s.close()

Receiving the file:

# check expected message length
remaining = int.from_bytes(clientsocket.recv(4), 'big')
d = open(inp, "wb")
while remaining:
    # until there are bytes left...# fetch remaining bytes or 4094 (whatever smaller)
    rbuf = clientsocket.recv(min(remaining, 4096))
    remaining -= len(rbuf)
    # write to file
    d.write(rbuf)
d.close()

Solution 2:

There are several issues with your code.

First:

clientsocket.send(m.encode('utf-8'))
data = clientsocket.recv(10000)

This causes the file to be partially loaded to data variable when you issue get statement. That's why you don't get full file.

Now this:

while True:
    rbuf = clientsocket.recv(8192)                
    if not rbuf:
        break
    ...

You indeed load full file but the client never closes the connection (it goes to s.recv() after sending the file) so if statement is never satisfied. Thus this loop gets blocked on the clientsocket.recv(8192) part after downloading the file.

So the problem is that you have to somehow notify the downloader that you've sent all the data even though the connection is still open. There are several ways to do that:

1. You calculate the size of the file and send it as a first few bytes. For example, say the content of the file is ala ma kota. These are 11 bytes and thus you send \x11ala ma kota. Now receiver knows that first byte is size and it will interpret it as such. Of course one byte header isn't much (you would only be able to send max 256 byte files) so normally you go for for example 4 bytes. So now your protocol between client and server is: first 4 bytes is the size of the file. Thus our initial file would be sent as \x0\x0\x0\x11ala ma kota. The drawback of this solution is that you have to know the size of the content before sending it.

2. You mark the end of the stream. So you pick a particular character, say X and you read the straem until you find X. If you do then you know that the other side sent everything it has. The drawback is that if X is in the content then you have to escape it, i.e. additional content processing (and interpretation on the other side) is needed.