Problem of file transfer in socket programming

[CKShia]

Hi i am a newbie in C programming especially in the socket area.
Supposely i am writing a FTP, and want to get and put file in the FTP.
However, i have encountered a bug/error when client is trying to get(download) file from server.

The problem is as follow:
First, client type get test.c. Then, server will start trasnfer test.c to client.
During the process of receiving test.c, client is actually received the file while he receive the error message at the same time.
Then, when i ls -l in both side, test.c in client receive some additional byte.
Which when i open the test.c, i found some weird character such as ^@ or space(s) appears at the end of the file.

i have attached a sample screen to better illustrated the situation as well as the portion source code of server & client.

client.c

while (1) {
    // get input from user and breaks the 'buf' into 'token[]'

    if ((nw = writen(sd, buf, nr)) < nr) { perror("send"); } // write stream

    // token[0] = "get"
    // token[1] = "test.c"
    if (strcmp("get", token[0]) == 0) {
        char file[512];
        memset(&buf, 0, sizeof (buf));
        memset(&file, 0, sizeof (file));
        recv(sd, buf, 4096, 0);
        // file = "test.c"
        strncpy(file, buf, strlen(buf) > 512 ? 512 : strlen(buf));

        printf("File '%s' receiving...\n", file);
        // write in binary
        FILE *fd = fopen(file, "wb+");
        memset(&buf, 0, sizeof (buf));
        int length = 0;
        while ((length = recv(sd, buf, SIZE, 0)) > 0) {
            fwrite(buf, sizeof (char), length, fd);
            if (strlen(buf) > 0) {
                printf("received: %d\n", strlen(buf));
                memset(&buf, 0, sizeof (buf));
            }
        }
        printf("File '%s' received.\n", file);
        fclose(fd);
        if (close(sd) < 0) {
            perror("close");
        }
    }

    if ((nr = readn(sd, buf, sizeof (buf))) <= 0) { perror("receive"); } // read stream
    buf[nr] = '\0';
}

server.c

while (1) {
    if ((nr = readn(sd, buf, sizeof (buf))) <= 0) { return; } // read stream
    buf[nr] = '\0';

    // token[1] = "test.c"
    memset(&buf, 0, sizeof (buf));
    strncpy(buf, token[1], strlen(token[1]) > 512 ? 512 : strlen(token[1]));
    send(sd, buf, 4096, 0);
    FILE *fd = fopen(token[1], "rb+");
    printf("\nFile '%s' transferring...\n", token[1]);
    memset(&buf, 0, sizeof (buf));
    int length = 0, total = 0;
    while ((length = fread(buf, sizeof (char), 4096, fd)) > 0) {
        printf("Length : '%d'\n", length);
        send(sd, buf, length, 0);
        memset(&buf, 0, sizeof (buf));
        total += length;
    }
    printf("File '%s' transfered (%d-byte)!\n", token[1], total);
    fclose(fd);

    nr = strlen(buf);
    nw = writen(sd, buf, nr); // write stream
}

Can anyone points out what mistake I've made in the program?
Or perhaps the step or file reading and writing is wrong in sequence?
Or... maybe any other case?

THANKS FOR GUIDING ME and APPRECIATE VERY MUCH = )

[histrungalot]

In the client code: On line 23 use the length instead of strlen. If you read 4096 bytes then there is not '\0' for strlen to stop at. That is why the first print out of the number of bytes received (4102) is over 4096. On line 10, 11, 19 & 25 you call memset with the address of buf/file (&buf or &file). It should should just be buf. You close the socket on line 30 but then read from it on line 35. I'm guessing that is where your receive error is coming from. Also, the code that you posted doesn't match the output of the client you posted. There is no print out '... received (%d - byte)!'.

In the server code: Line 6, 11 & 16 you call memset with the address of buf (&buf). It should should just be buf. Because of that error, line 22 might be returning non-zero and that is why you get extra data in the file. You are not closing the socket at the end, is that what you want?

[CKShia]

Hi histrungalot, thanks for your quick helping.
I have modified my coding based on your reply, such as removed the '&' as in all the memset.
And finally, got my file received correct.

Both file size in server and client 'test.c' is the same size and no weird character! =)

However, while client is receiving the file, the same thing happens in the first packet transferring.
Which is, server sends 4096 while client receive 4101 (this time is 4102). But after completed the transfer client receive the same size.
(as i ls -l in both terminal)

Another problem I suspected is in the line 35 in client.c
As what I shown previously the error message is showing receive() failed: Bad file descriptor,
yet now it show receive() failed: Success .
And as soon as client receive the file, it quits the program.

Here... Update my source code again:

client.c

while (1) {
    // get input from user and breaks the 'buf' into 'token[]'
    if ((nw = writen(sd, buf, nr)) < nr) { perror("send() failed"); } // write stream

    // token[0] = "get"
    // token[1] = "test.c"
    if (strcmp("get", token[0]) == 0) {
        int F_SIZE = 512;  // max file size
        int packet = 4096; // max packet size per transfer
        char file[F_SIZE];

        memset(buf, 0, sizeof (buf)); // removed '&'
        memset(file, 0, sizeof (file)); // removed '&'
        recv(sd, buf, packet, 0);

        // file = "test.c"
        strncpy(file, buf, strlen(buf) > F_SIZE ? F_SIZE : strlen(buf));
        printf("File '%s' receiving...\n", file);

        // write in binary
        FILE *fd = fopen(file, "wb+");
        memset(buf, 0, sizeof (buf)); // removed '&'

        int length = 0, total = 0;
        while ((length = recv(sd, buf, packet, 0)) > 0) {
            fwrite(buf, sizeof (char), length, fd);
            if (length > 0) { // first time receive the file will get 5 extra bytes..?
                              // second time onwards are correct.

                printf("received: %d\n", strlen(buf));
                total += length; // forgot to put this in first time, tat's y u can't c it
                memset(buf, 0, sizeof (buf)); // removed '&'

            } else {
                break; // if i comment or un-comment this, still the same
                       // result sent to client.
            }
        }
        printf("File '%s' received (%d-byte)!\n", file, total);

        fclose(fd);
        //if (close(sd) < 0) { perror("close"); } // comment out close socket
    }

    if ((nr = readn(sd, buf, sizeof (buf))) <= 0) { perror("receive"); } // read stream
    buf[nr] = '\0';
}

server.c

while (1) {
    if ((nr = readn(sd, buf, sizeof (buf))) <= 0) { return; } // read stream
    buf[nr] = '\0';

    // token[1] = "test.c"
    int F_SIZE = 512;  // max file size
    int packet = 4096; // max packet size per transfer

    memset(buf, 0, sizeof (buf)); // removed '&'
    strncpy(buf, token[1], strlen(token[1]) > F_SIZE ? F_SIZE : strlen(token[1]));
    send(sd, buf, packet, 0);

    FILE *fd = fopen(token[1], "rb+");
    printf("\nFile '%s' transferring...\n", token[1]);
    memset(buf, 0, sizeof (buf));

    int length = 0, total = 0;
    while ((length = fread(buf, sizeof (char), packet, fd)) > 0) {

        printf("Length : '%d'\n", length);
        send(sd, buf, length, 0);
        memset(buf, 0, sizeof (buf));
        total += length;

    }
    printf("File '%s' transfered (%d-byte)!\n", token[1], total);

    fclose(fd);
    close(sockfd);

    nr = strlen(buf);
    nw = writen(sd, buf, nr); // write stream
}

Is it possible is due to the socket close problem?
As client actually receive everything but the buffer in server return -1 to client yet the receive status indicate SUCCESS...
i am not sure what's happening why the buffer return -1.
Thanks ya

[CKShia]

Oops just found why the client received 4102 instead of the original size 4096.
In client.c line 30, it shoud be lenght instead of strlen(buf);

Now the only problem is the transferring part where client exit after file transferred. =)

[histrungalot]

In the server.c: You are trying to write to the socket on line 32 but you are done writing the file, why? Also, you are closing file descriptor sockfd on line 29. Is sockfd you listening socket, shouldn't you be closing sd?

In the client.c: on line 45 you are reading from sd after you have received all of the data, why? What line of code is printing 'nr : -1' and 'received() failed: Success'?

What is the client suppose to do after the file is received?
What is the server suppose to do after the file is sent?

[CKShia]

Also, you are closing file descriptor sockfd on line 29. Is sockfd you listening socket, shouldn't you be closing sd?

Oops, sorry again its typo, should be sd instead of sockfd =P

In the server.c: You are trying to write to the socket on line 32 but you are done writing the file, why?
In the client.c: on line 45 you are reading from sd after you have received all of the data, why?

Sorry but I am not sure what you are trying to point out the question. T_T
What I intend to do is evytime client input a command such as pwd, dir, get, put, or whatever command.
It will goes the logic like this:

1. client.c writen() forward input to server (forward command to server)
2. server.c readn() recv client input (receive command from client)
3. process the command
4. server.c writen() reply result to client (reply result back to client)
5. client.c readn() grab server input n print out (accept and print out the result in client)

What line of code is printing 'nr : -1' and 'received() failed: Success'?

is in client.c line 45

What is the client suppose to do after the file is received?
What is the server suppose to do after the file is sent?

Client suppose to have another prompt after the file is received.
Server suppose to wait for another input send by client.

Is is possible to help me re-structure my code?
I am getting a bit lost. =P

[histrungalot]

I might do a bad job here trying to explain myself but here we go.

I see that you are using writen/readn. The reason you use those are because TCP is a stream of bytes. When you call read you may not get the amount of data you ask for so we create a function readn that will not return from readn until all of the data that you ask for is read off the sock. This becomes inportant when sending a variable amount of data because how much data should the other side read before it has all of the data.

Example:
client sends "get testFile.c"  -> server reads ? bytes
client sends "get t.c"         -> server reads ? bytes

In the first case the server should read 14 bytes but in the second if the server was to call readn with 14, it would block there until it get the full 14 bytes. Not what we want. The total number of bytes for that one is only 7 bytes for a complete message.

So, you tried to fix this by sending and receiving 4096 bytes for all commands. That works except for when you start to send/read the file data. In the while loop that is sending the data, it will exit out of the while loop when all of the data in the file is read and fread return -1. At that point you call close(sd). That closes the socket, but that is not the beheaver you detailed above. I thought you wanted some thing like:

ftp> get test.c
File 'test.c' receiving...
received: 45
File 'test.c' received (45-byte)!
ftp> pwd
/home/user/test
ftp> 

Because you closed the socket, the while loop on the client side was able to break out of the while loop that was writing the data file. But again, you don't want to close the socket.

One way to fix this is to do like the real FTP does which is one port (21) for commands and another port (ephemeral) for data. Ephemeral ports are temporary ports assigned by a machine's IP stack. That might be a little much right now.

Another way would be to send the commands and use a delimiter to indicate there end. Like a '\n' would indicate the end of a command. Using a readline function on page: http://www.informit.com/articles/article.aspx?p=169505&seqNum=9 you could read the commands and not worry about how much to read because you will read until the '\n' is seen. Then go process that command. This is how I would see it working:

   client            c      server
ftp> get test.c   ------->  readline from socket "get test.c\n"
                            get the size of test.c and send that value 
                     c      back to the client with another command
readline          <------   readsize 6088
"readsize 6088\n"         
                     d
now you enter the <------   now enter while loop to send data to client
while loop that      d 
will use revc to  <------
read 6088 bytes      d 
from the socket   <------
to get all of the           All data is sent
data in test.c              

6088 received        c
readline          <------   Send new command "done test.c"

ready for next
user command
ftp> 

Or yet another way would be to send blocks of data. Using this method you send a message header which is a fixed size followed by the data. It would look like:

struct HEADER_S {
   int  len;
   char data[512];
};
Example:
HEADER_S msg;
sprintf(msg.data,"get %s",fileName);
msg.len = strlen(msg.data);

Using this you can do a readn of 4 bytes which give you the size of the rest of the message and the client and server always know how many bytes to read off the socket.
See http://tools.ietf.org/html/rfc959 sec 3.4.2 for Block Mode.

I don't know if I'm helping or confusing you more, but either way it's something to think about.