Read csv file 10 records at a time

[bryancan]

I have a potentially very large csv file that I need to read in and display, but only a few records at a time. I want to be abe to page thru the file using offset and records variables.

What I have is the following, but it really doesnt work well at all and if I hit the end of the file it errors:

def showMail(self,offset,records):
        """
        read in N lines of mail items
        """
        mails = []
        ifile = open('some.csv', "r")        
                
        data = csv.reader(ifile ,delimiter=',', quoting=csv.QUOTE_ALL)
        
        if offset > 0:
            for i in range(0,((offset*records)+records)):
                if i > (offset*records)-1:
                    mails.append(data.next())
                else:
                    data.next()
        else:
            for i in range(0,records):
                mails.append(data.next())
            
        ifile.close()
        return mails

Is there a better way of doing this that I am just too new to know?
Thanks

[jlm699]

Is there a better way of doing this that I am just too new to know?

Here's a suggestion: don't use csv reader. It's more overhead than you need for a csv file. Just read the line, strip off the white space (newline and trailing/leading spaces) and then split on the commas.

def showMail(self,offset,records):
        """
        read in N lines of mail items
        """
        mails = []
        ifile = open('some.csv', "r")        
        data = iter(ifile)
        
        if offset > 0:
            for i in range(0,((offset*records)+records)):
                if i > (offset*records)-1:
                    mails.append(data.next().strip().split(','))
                else:
                    data.next()
        else:
            for i in range(0,records):
                mails.append(data.next().strip().split(','))
            
        ifile.close()
        return mails

Note that this is untested and might have a bug or two.

As far as when it "errors", does it raise an EOF/StopIteration failure? You can simply wrap your file reading code in a try/except block.