Convert csv data to a list of records (Python)

vegaseat 1 3K Views

Shows how to convert a comma separated value (csv) string to a list of records, and then do some data processing. The csv strings usually come from csv files created by spreadsheets.

''' csv_to_list_of_records.py
convert a comma separated value (csv) string to a list of records
then do some processing
tested with Python34  by  vegaseat  08apr2015
'''

import operator as op

class Persons:
    ''' mimics a record '''
    def __init__(self, name, age, weight, seniority, pay):
        self.name = name
        self.age = age
        self.weight = weight
        self.seniority = seniority
        self.pay = pay
        
    
# a csv test string with lines of 
# comma separated values (csv)
# could have come from a csv file
# name,age,weight,seniority,pay
csv_str = '''\
Bob,34,121,13,69000
Zoe,19,63,2,23300
Amy,45,59,5,45000
Frank,56,93,12,57000
Kaka,23,55,7,45000
Udo,27,65,3,39000
Vera,31,71,8,45000
Betty,22,77,5,45000'''

# convert to a list of records
record_list = []
for line in csv_str.split('\n'):
    name, age, weight, seniority, pay = line.split(",")
    record = Persons(name, age, weight, seniority, pay)
    record_list.append(record)

print("Do some data processing with the list of records:")
pay_list = []
seniority_list = []
for person in record_list:
    print("{:<8} makes ${} per year".format(person.name, person.pay))
    pay_list.append(float(person.pay))
    seniority_list.append(float(person.seniority))

print('-'*32)

print("Minimum pay = {}".format(min(pay_list)))
print("Maximum pay = {}".format(max(pay_list)))
sf = "Average pay in department = ${:0.2f}"
print(sf.format(sum(pay_list)/len(pay_list)))
print("Total seniority = {:0.1f} years".format(sum(seniority_list))) 
sf = "Average seniority in department = {:0.1f} years"
print(sf.format(sum(seniority_list)/len(seniority_list)))

''' result ...
Do some data processing with the list of records:
Bob      makes $69000 per year
Zoe      makes $23300 per year
Amy      makes $45000 per year
Frank    makes $57000 per year
Kaka     makes $45000 per year
Udo      makes $39000 per year
Vera     makes $45000 per year
Betty    makes $45000 per year
--------------------------------
Minimum pay = 23300.0
Maximum pay = 69000.0
Average pay in department = $46037.50
Total seniority = 55.0 years
Average seniority in department = 6.9 years

'''

print('-'*32)

# sorting, first by pay then name (case of equal pay)
for person in sorted(record_list, key=op.attrgetter('pay', 'name')):
    print("{:<8} {}".format(person.name, person.pay))
    
''' result ...
Zoe      23300
Udo      39000
Amy      45000
Betty    45000
Kaka     45000
Vera     45000
Frank    57000
Bob      69000

'''

Dani AI

Generated

Nice demo, , and +1 to @J-M DESMETTRE for the file-based approach. If your CSV has a header row, a small tweak makes the code safer and easier to maintain: read rows as dicts instead of tuples and convert types explicitly. That avoids fragile positional unpacking when columns are reordered or new fields are added. Also open with newline='' so the csv module handles line endings correctly, and consider encoding='utf-8-sig' to tolerate a BOM from Excel-generated files.

from dataclasses import dataclass
from csv import DictReader
from typing import List

@dataclass
class Person:
    name: str
    age: int
    weight: float
    seniority: int
    pay: float

def load_people(path: str) -> List[Person]:
    people: List[Person] = []
    with open(path, newline='', encoding='utf-8-sig') as f:
        rdr = DictReader(f)  # expects header: name,age,weight,seniority,pay
        for lineno, row in enumerate(rdr, start=2):
            try:
                people.append(Person(
                    name=row['name'].strip(),
                    age=int(row['age']),
                    weight=float(row['weight']),
                    seniority=int(row['seniority']),
                    pay=float(row['pay'])
                ))
            except Exception as exc:
                print(f"Skipping line {lineno}: {exc}")
    return people

Notes:

  • If your data is already a string (as in 's test), wrap it with io.StringIO(csv_text) and pass that to DictReader.
  • No headers? Provide fieldnames=[...] to DictReader.
  • Unknown delimiter? Try csv.Sniffer().sniff(sample_text) to detect it.
  • For currency, prefer decimal.Decimal instead of float.
  • For large files, make load_people a generator (yield) to avoid loading everything into memory at once.
Member Avatar for J-M DESMETTRE
J-M DESMETTRE 0 Newbie Poster

Hi,

Example reading the data from a "csv_str.csv" file:

# convert to a list of records
record_list = []
with open ("csv_str.csv",'r') as fi:        
    reader = csv.reader(fi,delimiter=',')
    for line in reader:
        name, age, weight, seniority, pay = line
        record = Persons(name, age, weight, seniority, pay)
        record_list.append(record)

Don't forget to import the csv library import csv

Member Avatar for vegaseat
vegaseat 1,735 DaniWeb's Hypocrite

Thanks for showing how to read the csv file directly. For testing purposes it was simpler to use the string in the code.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.

Edit Preview