Read Multiple Files

[2ashwinkulkarni]

Hi,

I wanted to know if there is a way to read multiple csv files from a folder, process them one by one and write the output to corresponding output files. Also it is necessary that the if the name of the input file is foobar.csv then the name of t he output file has to be foobar_output.csv. I tried to use fileinput to iterate over a list of files but then i could not figure out how to manipulate the names of the multiple output file being generated.

thanks in advance

[shadwickman]

Why don't you just loop a function that opens each file one at a time and writes the processed information to the corresponding output file? Have an array of the .cvs files and for every one in the array, call the processing/writing function.

[jlm699]

To get a list of the contents of a folder you can use the os module's listdir() function.

Then loop over each item, and use split('.') on the name to add '_output' before opening the output file.

[woooee]

There are various tutorials on the web as well as on python.org . Try one of the tutorials so you can present some code that we can help you with.
http://www.faqts.com/knowledge_base/view.phtml/aid/23158/fid/552

[2ashwinkulkarni]

Thank you for the suggestions, it solved my problem. Here is the code I came up with. If anyone has the same problem this code is tested and works perfectly fine

## Code to iterate multiple files from a directory
## and process each and store them in an output
## file with a different name (root name is the original file name).
## split the name and path of the file in different parts

import csv
import os
import sys
import re
import glob

#[B]Path of the folder where all the files are stored.[/B]
path = 'C:\Documents and Settings\.........\My Documents\NEW WORKING FOLDER\TrafficFlow\June15'
# use join for multi platform compatibility
for infile in glob.glob(os.path.join(path, '*.csv')):
    
    # infile stores the complete path of the file
    print "Current File Being Processed is:  " + infile 

    #use split to seperate the path and name of the file
    (PATH, FILENAME) = os.path.split(infile)
    print " PATH is " + PATH
    print " FILENAME is " + FILENAME
    
    #use splitext() to seperate name of the file and the extension
    (ShortName, Extension) = os.path.splitext(FILENAME)
    print ShortName
    print Extension

    # then you can use each part in any way you want
    #Dont forget the double backslash to escape the single backslash :-)
    outputfilename1 = 'C:\..\Desktop\Output\\' + ShortName +   '_' + 'output' + Extension
    print " \n \n The path of the matched output file is:  " + outputfilename1