Read delimited file

[1bennyd]

Hi Guys,

I am a complete newbie to C++. I have a background in R, and have recently taken the plunge into C++. Not having much success though. I am trying to solve an easy problem (no doubt) - reading data from a tab delimited file and inserting it into a 2D array.

The data I have been trying to read is:
1 5 7 8 90
2 67 4 9 0
3 6 54 5 4
3 23 4 2 8

Here is my code:

#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

int main() {
string s, line, segments;
vector<string> v, v2;
ifstream in("test.txt");
stringstream ss;
string data [4][5];

// read in each line
while(getline(in,line)) { 
	v.push_back(line);
}

cout << v[0] << endl; // Printing out these
cout << v[1] << endl; // gives the expected
cout << v[2] << endl; // results.
cout << v[3] << endl;

// Try and sort elements of each line
// into a 2D array:

for (int i=0; i < 4; i++) {
	ss << v[i]; // Suspected problem here
	while (getline(ss,segments,'\t')) {
		v2.push_back(segments);
	}
	for (int j=0; j < 5; j++) {
		data[i][j] = v2[j];
		cout << data[i][j] << endl;
	}	
}
}

Compiling and executing this code only prints out the elements of the first row of the input data. I suspect that "ss << v" is always pointing to the first element of the array. I am not sure how to remedy this though. If someone could give me a hand, or suggest a better method for solving this problem, I would be very grateful.

Ben.

[Freaky_Chris]

Thanks for the very nice post! Makes my life so much simpler.

The solution, you need to clear your stringstream before throwing the next string at it. Using ss.clear(); . Then in your loop that prints the results out, you need to change the way in which you access your vector elements in v2. Since you will always use 0..4, whereas you want 0..4, 5..9 , and so on. Only two minor changes. Here they are:

#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

int main() {
    string s, line, segments;
    vector<string> v, v2;
    ifstream in("test.txt");
    stringstream ss;
    string data [4][5];

    // read in each line
    while(getline(in,line)) {
        v.push_back(line);
    }

    cout << v[0] << endl; // Printing out these
    cout << v[1] << endl; // gives the expected
    cout << v[2] << endl; // results.
    cout << v[3] << endl;

    // Try and sort elements of each line
    // into a 2D array:
    for (int i=0; i < 4; i++) {
        ss << v[i]; 
        while (getline(ss,segments,'\t')) {
            v2.push_back(segments);
        }
        for (int j=0; j < 5; j++) {
            data[i][j] = v2[j+(i*4)]; // updated from 'j' to 'j+(i*4)'
            cout << data[i][j];
        }
        ss.clear(); // added to lcear ss buffer.
    }
    cin.get();
    return 0;
}

Also please be sure to do some error checking whenit comes to opening files.

Thanks,
Chris

[ArkM]

>I am trying to solve an easy problem (no doubt) - reading data from a tab delimited file and inserting it into a 2D array.
Well, try to solve THIS problem: you have 4x5 integers in the text file and you need to read them into 2D INTEGER array, not in string array or std::vector (why did you use vector? ).

/// Get 4x5 integers...
bool Zoo(int a[4][5], const std::string& fname)
{
    size_t i = 0, j = 0;
    ifstream f(fname.c_str());
    if (a)
    for (i = 0; i < 4; ++i)
    for (j = 0; j < 5; ++j)
        if (!(f >> a[i][j]))
            return false;
    return i == 4 && j == 5;
}
/// Five numbers per line!
bool Zoom(int a[4][5], const std::string& fname)
{
    size_t i = 0, j = 0;
    std::string line;
    ifstream f(fname.c_str());
    istringstream is;
    if (a)
    for (i = 0; i < 4 && getline(f,line); ++i) {
        is.str(line);
        is.seekg(0);
        for (j = 0; j < 5 && (is >> a[i][j]); ++j)
            ;
        if (j != 5) break;
    }
    return i == 4 && j == 5;
}
/// for example only
void print4x5(const int a[4][5])
{
    if (a)
    for (size_t i = 0; i < 4; ++i) {
        for (size_t j = 0; j < 5; ++j)
            std::cout << (j?"\t":"")
            << a[i][j];
        std::cout << '\n';
    }
    std::cout.flush();
}

const char* fname = "int4x5.txt";
/// Try this...
int main()
{
    int a[4][5];

    Zoo(a,fname);
    print4x5(a);

    a[0][0] = 666; // try again...

    Zoom(a,fname);
    print4x5(a);

    return 0;
}

[1bennyd]

Thanks for your time guys. It is much appreciated. I am very impressed with this forum.

I compiled Freaky_Chris's code (with the line "data[j] = v2[j+(i*4)];" changed to "data[j] = v2[j+(i*5)];") and everything worked great. But yeah, as ArkM pointed out, what I should really be doing is reading into a 2D int array.

So this led me to use ArkM's code. Everything compiled nicely. However, from a newbie perspective, I've been having a little trouble understanding whats going on. There are a few functions I am unfamiliar with. No doubt I have to do more c++ reading. Anyways, I have the following questions:

1. What exactly does the function "Zoo" do? Is it specifying the elements of a?
2. What exactly does the function "Zoom" do? Does it perform the same function as Zoo, just in a different way?
3. Assuming I have a file "test.txt" containing the data I originally posted, what parts of your code are needed to read the data into a 2D int array?

As you can see, I've really got no idea whats happening. Any help would be great.

Also, one more question: In practice, is reading the file into an array normally performed in two steps. 1) Determine the number of rows and columns of the data. 2) Read the data. Or, is there a way to dynamically define the 2D array?

Again, thanks for all the help!

Ben.

[ArkM]

I did not want to do your homework, that's why my code was not too straigtforward ;)
However it contains all elements you needed (and more).

A. 1,2: Both functions do the same job: see your original post specifications ;)
Of course, both functions specify elements of array (see f >> a[i][j] in Zoo and is >> a[i][j] in Zoom). Zoom garantees that every (of four) line contains five and only five numbers. Zoo reads 25 numbers separated by whitespace characters (blanks, tabs and newlines). For example, Zoo can read files where all numbers are placed by number per line etc.

A.3: You need Zoo or Zoom functions only: see main function in my snippet ;)
May be better call is:

if (Zoo(a,"test.txt")) {
    // you got data
} else {
    // wrong data were detected
}

Try the code with debugger then write your own code ;)

One more answer: of course, there are lots of methods to:
1. Determine how many rows/columns of numbers are contained in the source file.
2. Dynamically define the 2D array to get those data.
However it's the other story. No dynamically shaped 2D (3D, 4D and so on) arrays in C++. That's why you (and me) must write more codes. It's possible to get ready-to-use matrix class libraries for 2D arrays.

Some tips:
1. It's not so hard job to convert Zoom function to source file probe (how many rows and columns are... ).
2. Straightforward 2D-array generator sceleton:

int** dynMxN(int m, int n)
{
    int** pa2d = new int*[m];
    int*  parr = new int[m*n];

    for (int i = 0; i < m; ++i)
        pa2d[i] = parr + (i*n);
    return pa2d;
}
void delMxN(int** p2d)
{
    if (p2d) {
        delete [] *p2d;
        delete []  p2d;
    }
}
...
int** a2d = dynMxN(m,n);
... a2d[i][j] ...  // usual 2d array element accessor
delMxN(a2d);  // free storage

3. You must add number of rows and cols parameters to Zoo and Zoom functions in that case.

Good luck!

[1bennyd]

Hi guys. Embarrassingly enough, I'm not trying to program this simple task for homework. I'm actually a PhD student in Statistics, and I am trying to rewrite some of my R code in C++. Since data IO is so important for my needs, I thought I would start my C++ experience by trying to read a tab delimited file.

Thanks alot ArkM for your help. I read what you said, thought it over, and came up with the following solution for my problem.

// Reads data from a file that is tab delimited.
// Data is returned as a 2D array.

#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

float** ReadTable(const char* FileName, int NumRowCol[]) {

	// Declarations/Definitions
	string s, line, segments;
	vector<string> v;
	stringstream ss;
	int row_count = 0;
	int col_count = 0;
	ifstream in;
	in.open(FileName, ios::in);

	// Determine number of rows and columns in file.
	// Data must be tab delimited.
	while(getline(in,line)) { 
		row_count++;
		if (row_count == 1) {
			ss << line;
			while (getline(ss,segments,'\t')) {
				col_count++;
			}
		}
	}

	// Store number of rows and columns in array.
	NumRowCol[1] = row_count;
	NumRowCol[2] = col_count;

	// Declare arrays for storing and accessing
	// data that is read.
	float** pa2d = new float*[row_count];
	float*  parr = new float[row_count*col_count];

	// Reposition at start of stream buffer.
	in.clear();              
	in.seekg(0, ios::beg);   

	// Write data into array.
	for (int i = 0; i < (row_count*col_count); i++) {
		in >> parr[i];
	}

	// Prepare 2D array to be returned. 	
	// Define pointer position at start of each row.
	for (int i = 0; i < row_count; ++i) {
		pa2d[i] = parr + (i*col_count);
	}

	return pa2d;
}

void PrintMatrix(float** matrix, const int NumRows, const int NumCols) {
		for (int i=0; i < NumRows; i++) {
			for(int j=0; j < NumCols; j++) {
				cout << (j?"\t":"") << matrix[i][j];
				// Don't insert tab before first element
				// of each row.
			}
			cout << '\n';
		}
}

int main() {

// Define filename.
const char* FileName = "floats.txt";

// Create array to store the number of rows and
// columns in the data file.
int NumRowCol[2];

// Perform reading, storing and printing of data
// in the file.
float** data = ReadTable(FileName,NumRowCol);

// Print number of rows and columns in 2D array.
cout << NumRowCol[1] << " rows" << endl;
cout << NumRowCol[2] << " columns" << endl;

// Print contents of 2D array to console.
PrintMatrix(data,NumRowCol[1],NumRowCol[2]);

}

I'm not sure if I have used the most efficient method to determine the number of rows and columns in the file, but everything seems to be working OK.

Again, thanks everyone for your help. It is greatly appreciated.

Ben.

[ArkM]

The only error but there are lots of its occurencies ;):
You forgot that the 1st element index of array in C++ is 0 (zero). Now look at the NumRowCol indicies in your code...

It's not so good design idea to place matrix dimensions in array. An array contains similar values for loop processing. What's a loop can you invent to process matrix height and width? Better pass two variables by reference or pass (by reference) a structure with rows and columns members. It's a clear and simple solution.

Yet another defect: no error handling at all. For example, you count a number of columns in the 2nd line only. How about one-line file only or "jagged" file contents? What happens if input file contains not-a-numbers?

Well, we had eaten the big apples, only the small ones were left. Look at:

ifstream in;
    in.open(FileName, ios::in);

Are you sure that it's more robust code than clear and simple

ifstream in(FileName);

? ;)

[1bennyd]

Hey hey. You are 100% right, there were lots of mistakes in there. I am still getting used to zero indexing. It goes to show how good these forums are; I wouldn't have realized my mistakes if no one pointed them out.

Anyway, I believe I have fixed the errors. I also had a crack at some error handling. I used stringstreams to check if the data entries were non-numeric. I also implemented a test to see if all the rows were the same length. I made the program halt if any of these situations occured. This seemed sensible - you can't do much if a proper matrix cant be constructed.

Here is the code:

// Reads data from a file that is tab delimited.
// Data is returned as a 2D array.

#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <sstream>
#include <cmath>
#include <cstdlib>
using namespace std;

float** ReadTable(const char* FileName, int& RowNum, int& ColNum) {

string line;
ifstream in(FileName);
	
// Determine number of rows and columns in file.
// Program halts if rows are not of same length. 
while(getline(in,line,'\n')) {
	string segments;
	int ColsPerRow = 0; // Initialize counter.
	stringstream ss;
	ss << line;
	while (getline(ss,segments,'\t')) {
		ColsPerRow++;
	}
	if (RowNum == 0) {
		// Define number of columns in file as the number
		// of columns in the first row.
		ColNum = ColsPerRow;
	} else {
		if (ColsPerRow != ColNum) {
			cerr << "Row " << RowNum << " is not the same length "
			"as row 0." << endl;
			exit(0);
		}
	}
	RowNum++;
}

// Declare arrays for storing and accessing
// data from the file.
float** pa2d = new float*[RowNum];
float*  parr = new float[RowNum*ColNum];

// Reposition to start of stream buffer.
in.clear();              
in.seekg(0, ios::beg);   

// Write data to array:
for (int i = 0; i < (RowNum*ColNum); i++) {
	
	// Declarations.
	float in_float;
	char in_char;
	string in_s;
	stringstream in_ss;
	
	// Insert data entry into stringstream.
	in >> in_s;
	in_ss << in_s;

	// Convert data entry to float.
	in_ss >> in_float;

	if (!in_ss) {
	// Check if stream is in error state.
	// Halt program if it is.
		float r = floor(i/ColNum); float c = i - (r*ColNum);
	// (Convert 1D array position to 2D array position.)
		cerr << "Bad input: Entry [" << r << "," << c 
		     << "] put stream into error state." << endl;
		exit(0);
	} else {
		if (in_ss >> in_char) {
		// The attempt to extract a character is a test
		// to see if there is anything left in the stream
		// after we extract a float. Program is halted if
		// if a non-numeric entry is found.
		float r = floor(i/ColNum); float c = i - (r*ColNum);
		// (Convert 1D array position to 2D array position.)
		cerr << "Bad input: Entry [" << r << "," << c 
				 << "] is not numeric." << endl;
		exit(0);
		} else {
			// Everything is OK, build array.
			parr[i] = in_float;
		}
	}
}

// Prepare 2D array to be returned. 	
// Define pointer position at start of each row.
for (int i = 0; i < RowNum; i++) {
	pa2d[i] = parr + (i*ColNum);
}

return pa2d;
}

void PrintMatrix(float** matrix, const int RowNum, const int ColNum) {
// Print "matrix" to console.		
cout << '\n';
for (int i=0; i < RowNum; i++) {
	for(int j=0; j < ColNum; j++) {
		cout << (j?"\t":"") << matrix[i][j];
		// Don't insert tab before first element of each row.
	}
	cout << '\n';
}
}

int main() {

// Define filename.
const char* FileName = "floats_with_errors.txt";

// Initialize row and column counters.
int RowNum = 0;
int ColNum = 0;

// Perform reading of data file into 2D array.
float** data = ReadTable(FileName,RowNum,ColNum);

// Print contents of 2D array to console.
PrintMatrix(data,RowNum,ColNum);

return(0);

}

The code implements the following error handling.

1. Entries consisting of a mixture of numbers and characters are deemed as non-numeric.

2. Blank entries put the stream in an error state. This is identified and the program is halted.

3. Missing values result in varying row lengths. The code detects this and halts the program.

4. Mistakes, such as having two minus signs, put the stream in an error state. Again, this is identified.

Once again, thanks for the help.

I would be happy to hear any suggestions on how to improve the code.

Have a great Christmas!

Ben.