how to read a text file backwards?

[stanats]

i'm having this problem and i would like to ask for some ideas on how i will read a file backwards, that is, from end, read the file backwards(forward, naturally) by line..

thanks

[Narue]

Read it forward, store it in a reversible container, and then walk through the container backward. Or if you know the exact length of each line, you could open the file as binary and use fseek with read to get the lines in reverse. Or you could physically reverse the file before reading it. Or you could use recursion, but that's an exceptionally poor solution. ;)

[stanats]

Read it forward, store it in a reversible container, and then walk through the container backward. Or if you know the exact length of each line, you could open the file as binary and use fseek with read to get the lines in reverse. Or you could physically reverse the file before reading it. Or you could use recursion, but that's an exceptionally poor solution. ;)

what container would you suggest? is it possible to use some kind of strtok but backwards?

[Narue]

>what container would you suggest?
A linked list since you might not know how many lines are in the file.

>is it possible to use some kind of strtok but backwards?
No, the trick here is that you have to get the lines from the file into memory before you can do anything with them. So your problem boils down to either reading the entire file into memory, or using tricky seeking to actually read the file from end to beginning.

[stanats]

reading it line by line forward costs too much...

>No, the trick here is that you have to get the lines from the file into memory before you can do anything with them. So your problem boils down to either reading the entire file into memory, or using tricky seeking to actually read the file from end to beginning.

how would that tricky seeking be implemented?

[Salem]

If the text file is large (and constant), and you need to do this a lot, then reading the whole file once to produce an index is worthwhile.

The index being all the 'tell' positions for the start of each line, so at some future time, you can just 'seek' to the start of any given line and read it.

[stanats]

>If the text file is large (and constant), and you need to do this a lot, then reading the whole file once to produce an index is worthwhile.

the text file is large and not constant...the program constantly adds entries to the file, and the most recent is at the bottom, that is what I wanted to read first and not from the head...

[Narue]

>reading it line by line forward costs too much...
And reading it line by line backward doesn't? :icon_rolleyes: Regardless of your solution, you've got the penalty of reading the file. Unless it's a random access file (unlikely), that involves sequential front-to-back access. Oh, and have you tried any of the suggestions and proven that they all cost too much? I'm willing to bet that you haven't.

[jim mcnamara]

Consider calling stat() to get the filesize first. Open the file, then move the file pointer forward to some arbitrary place.

If you assume no last line is ever longer than say, 200 characters,
then fseek() to within 200 character size units of the end of the file.
fread in the last bit of the file into a buffer.

Now you can apply what Narue indicated - you have a container that you can "backwards read" from to get the last line of the file. If you need the last 10 lines, then adjust the file pointer positioning scheme.

If you want to see a generalized algorithm for this, consult the coreutils source for the "tail" utility at
www.gnu.org

[Salem]

> that is what I wanted to read first and not from the head
Seems to me like you want the 'tail' program.

Or in any event, read forwards from the point you last read to get any new data in the file.

In which case, the answer still stands. You record the 'tell' position of where you got to, then 'seek' to it sometime later, and carry on reading.

[stanats]

>And reading it line by line backward doesn't? :icon_rolleyes: Regardless of your solution, you've got the penalty of reading the file. Unless it's a random access file (unlikely), that involves sequential front-to-back access. Oh, and have you tried any of the suggestions and proven that they all cost too much? I'm willing to bet that you haven't.

i guess i did not but i inferred... reading it like that may just be the same as loading the file to memory..

i think jim's may be correct, but i'll just have to make sure that a certain line should be a certain length, say, pad some values to reach that certain length so that fseek-ing to a certain length produce a uniform result..

[echobase]

Hello all-

I'm new to C programming + this is my first post.

I'm also trying to write a progam that simply reads in a single text file, then copies the contents in reverse order to a new file. I read the above entry which suggests reading the file into a buffer (which I have done using fgets). I presume you would then find where the file ends in this buffer, and read this backwards into the new file. Is that reasonable? What I don't quite understand is how to traverse this buffer (or "container") and find the end of the file. If I knew how to do that I could probably figure out how to read the buffer backwards into the new file. Can someone give me a hint on this? I'd be most grateful.

Thanks-

echobase

[Ancient Dragon]

1. Open the file for read, call fseek() to seek to the end of the file, then call ftell() to get the length of the file. Alternatively you can get the file length by calling stat() or fstat().

2. Allocate a buffer pointer to the file size obtained in #1, above.

3. Read the entire file into that buffer -- you can probably use fread() to read the file all in one shot (assuming the file is small enough).

4. Use another char pointer to transverse the file from end to beginning of the buffer. Something like this example

char *ptr = 0;
char *iobuf = malloc( fileSize );
fread(iobuf,fileSize,fp);
for(ptr = iobuf; ptr >= iobuf; ptr--)
{
    // do something with the character at *ptr
}
free(iobuf);

Note the above code snipped does not contain any error checking, which you should add in your program.

[echobase]

First, thank you for you generous reply. The code that follows is what I have so far. It compiles, but when I run it, I get a runtime error indicating the fread function has failed. The code looks reasonable to me. Do you know why it would fail?

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{    
    long int filesize;   
    //int nblocks = 1; 
    FILE *in, *out;   

    /* check no. of arguments */
    if (argc != 3) {
        fprintf(stderr, "%s: incorrect arguments\n",
            argv[0]);
        return 1;
    }
    
    /* open the input file */
    if ((in = fopen(argv[1], "r")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[1]);        
        exit(1);
    }
    
    /* open the target file */
    if ((out = fopen(argv[2], "w")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[2]);        
        exit(1);
    }
    
    /* move pointer to end of input file */    
    if(fseek(in, 0L, SEEK_END) != 0) {
        perror("fseek(in, 0L, SEEK_END");
        exit(1);
    }    
    
    /* get current file position in input file*/
    filesize = ftell(in);        
    
    char *ptr = 0;
    char *iobuf = malloc(filesize);
    
    /* read input file into buffer */
    if(fread(iobuf, filesize, 1, in) != 1){        
        fprintf(stderr, "fread failed\n"); 
        exit(1);
    } 
    exit(0);
}

Thanks-
echobase

[echobase]

Sorry, dumb mistake. I meant to write !=0 instead of !=1. But can you tell me if in the for loop I should be reading each char in ptr into a new target array that should then be passed to an fwrite function for streaming to the target file? Am I on the right track?

thanks again-
echobase

[Ancient Dragon]

line 44 should have read < filesize because it is reading filesize bytes.

>>But can you tell me if in the for loop I should be reading each char in ptr into a new target array that should then be passed to an fwrite function for streaming to the target file

No you don't have to create another array. In the for loop I posted just use fputc() to write out each character one at a time. The operating system will buffer them up and optimize the write to disk.

[echobase]

Ancient-
I see why you recommended fputc - the OS buffers it ,and therefore it is still efficient. Also, I read more about malloc(), to understand that better. My program still doesn't work but I feel it's very close. There is still a runtime error with my fread function - but why?!!? I've looked at it carefully + feel all the parameters are correct. Here is the latest:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{    
    long int filesize;   
    int nblocks = 1; 
    FILE *in, *out;   

    /* check no. of arguments */
    if (argc != 3) {
        fprintf(stderr, "%s: incorrect arguments\n",
            argv[0]);
        return 1;
    }
    
    /* open the input file */
    if ((in = fopen(argv[1], "r")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[1]);        
        exit(1);
    }
    
    /* open the target file */
    if ((out = fopen(argv[2], "w")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[2]);        
        exit(1);
    }
    
    /* move pointer to end of input file */    
    if(fseek(in, 0L, SEEK_END) != 0) {
        perror("fseek(in, 0L, SEEK_END");
        exit(1);
    }    
    
    /* get current file position in input file*/
    filesize = ftell(in);        
    
    char *ptr = 0;
    char *iobuf = malloc(filesize);    
    
    /* read input file into buffer */
    if(fread(iobuf, filesize, nblocks, in) < filesize){        
        fprintf(stderr, "fread failed\n"); 
        exit(1);
    }    
   
    for(ptr = iobuf; ptr >= iobuf; ptr--)
    {
        fputc(*ptr, out);       
    }
    free(iobuf);
    
    exit(0);
}

thanks,
echobase

[~s.o.s~]

You are using the fread function incorrectly by swapping the position of the second and third parameter. The second parameter is not the size of the entire file, but the size of each chunk.

Try changing to fread(iobuf, nblocks, filesize, in) and it should work out to be fine.

[echobase]

That makes sense because the third parameter is the number of blocks to read... I changed it (see below) but it still doesn't work. It compiles, but fread still fails. The input file is just a simple .txt file with a 3 sentence paragraph, so I don't know why it should fail ....Oh, the humanity.

/* read input file into buffer */
    if(fread(iobuf, nblocks, filesize, in) < filesize){        
        fprintf(stderr, "fread failed\n"); 
        exit(1);
    }

echobase

[Ancient Dragon]

You are using the fread function incorrectly by swapping the position of the second and third parameter. The second parameter is not the size of the entire file, but the size of each chunk.

Try changing to fread(iobuf, nblocks, filesize, in) and it should work out to be fine.

I don't think it makes any difference -- I usually do it the way the OP posted and have never had a problem. But, as Narue will probably say, in general it may not be safe to do that.

echobase: post your current program and attach a copy of the file you are working with.

[~s.o.s~]

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
    size_t filesize;
    size_t nblocks = 1;
    FILE *in, *out;

    /* check no. of arguments */
    if (argc != 3) {
        fprintf(stderr, "%s: incorrect arguments\n",
            argv[0]);
        return 1;
    }

    /* open the input file */
    if ((in = fopen(argv[1], "rb")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[1]);
        exit(1);
    }

    /* open the target file */
    if ((out = fopen(argv[2], "wb")) == NULL) {
        fprintf(stderr, "%s: can't open %s\n",
                argv[0], argv[2]);
        exit(1);
    }

    /* move pointer to end of input file */
    if(fseek(in, 0L, SEEK_END) != 0) {
        perror("fseek(in, 0L, SEEK_END");
        exit(1);
    }

    /* get current file position in input file*/
    nblocks = ftell(in);
    rewind(in);

    char *ptr = 0;
    char *iobuf = (char*)malloc(sizeof(char) * nblocks);

    /* read input file into buffer */

    if(fread(iobuf, 1, nblocks, in) != nblocks){
        fprintf(stderr, "fread failed\n");
        exit(1);
    }


    fclose(in);
    fwrite(iobuf, 1, nblocks, out);
    /*
    for(ptr = iobuf; ptr <= iobuf; ptr++)
    {
        fputc(*ptr, out);
    }
    */
    free(iobuf);
    fclose(out);

    exit(0);
}

[echobase]

Hey you guys-

The program as it stands now is basically the version ~s.o.s~ just posted. The fread function now works beautifully,but there is no mechanism to reverse the text. (The statement in the for loop at line 55 was my unsuccesful attempt to do just that.) The program in its present state merely copies the input file to the target file from beginning to end. The fwrite function doesn't appear to be sophisticated enough to write the buffer in reverse, so don't you think I'll have to reverse the buffer somehow myself in between fread and fwrite? This goes back to my original idea of creating a new target char array for that purpose. So really the question is, how do I reverse iobuf? Right? I have attached a copy of the input text file I've been using.

echobase

[~s.o.s~]

No need of another buffer, a simple swap should do the trick.

int i, j;
    char ch;
    for(i = 0, j = nblocks - 1; i <= j; ++i, --j)
    {
        ch = iobuf[i];
        iobuf[i] = iobuf[j];
        iobuf[j] = ch;
    }
    fwrite(iobuf, 1, nblocks, out);

[echobase]

It works! ! Thanks very much to ~s.o.s~ and Ancient Dragon.