ios fstream (how to write and read binary data from and to a deque?

[Jsplinter]

I have data stored in a deque that I wish to write to disk using fstream. So far this is the test code I have written.

int j = 10000;
	deque<double> m_data;

	for(int i = 0; i < j; i++)
	{
		m_data.push_back(i);
	}	

	std::fstream myfile;
	myfile.open ("data2.bin", std::ios::out | std::ios::app | std::ios::binary );

	for(int i = 0; i < j; i++)
	{
		myfile.write((const char*) &m_data[i], sizeof(m_data));
	}
	myfile.close();

				
	deque<double> m_data2[sizeof(m_data)];
	ifstream myfileIN;
	myfileIN.open("data2.bin", std::ios::in | std::ios::binary);

	// get length of file:
	myfileIN.seekg(0, std::ios::end);
	int length = myfileIN.tellg();
	myfileIN.seekg(0, std::ios::beg);

	char* buffer = new char[length];
	myfileIN.read(buffer, length);

This code does run and compile just fine, BUT I have no idea how to go from the string stored in buffer back into a deque container. So, help, please.

Also, am I writing (serializing) this data in the most efficient manner?

Thank you!!

[nbaztec]

One way would be to read 8 bytes & push_back the values into the deque.

double dub;
dq.erase();  //drop elements, if any
while(!myfileIN.read(&dub,sizeof(double))
   dq.push_back(dub);
myfileIN.close();

[Jsplinter]

Well, I think you got me on the right track, but the read function takes a char* argument only. I took what you wrote and modified it as follows:

double dub;
	char *stopstring;
	char* buffer = new char[sizeof(double)];
	while(!myfileIN.read(buffer, sizeof(double)))
	{
		dub = strtod(buffer, &stopstring);
		dq.push_back(dub);
	}

This compiles, but it is not working. I don't know why this is so hard.

[nbaztec]

Maybe because the string you read *does not* have a NULL character.

[mitrmkar]

>> This compiles, but it is not working.

The loop condition is backwards (has ! ).
You are complicating the reading, to read a double at a time ..

ifstream myfileIN("data2.bin", ios_base::binary);
double dub;
// As long as sizeof(double) bytes is read ..
while(myfileIN.read(reinterpret_cast<char*>(&dub), sizeof(dub)).gcount() == sizeof(dub))
{
  dq.push_back(dub);
}

PS. If you'll be using strtod() , the buffer must be '\0'-terminated, like nbaztec noted.

[Jsplinter]

Well, I'm almost there thanks to you guys. I had been trying to use strcat to add the '\0' character to my char array, but this looks much better!

Yes, I see that the while condition was backwards now.

I've written this test program

int j = 10;
	deque<double> m_data;

	// Fill initial deque with values
	for(int i = 0; i < j; i++)
	{
		m_data.push_back(i);
	}	

	
	std::fstream myfile;
	myfile.open ("data2.bin", std::ios::out | std::ios::app | std::ios::binary );
	// Write deque to binary file
	for(int i = 0; i < j; i++)
	{
		myfile.write(reinterpret_cast<char*>(&m_data[i]), sizeof(m_data));
	}
	myfile.close();

			
	deque<double> dq;
	ifstream myfileIN;
	myfileIN.open("data2.bin",  std::ios::binary);

	double dub;
	//As long as sizeof(double) bytes is read
	while(myfileIN.read(reinterpret_cast<char*>(&dub), sizeof(dub)).gcount() == sizeof(dub))
	{
		dq.push_back(dub);
	}
	
	//Output the values of dq
	deque<double>::iterator iter(dq.begin());
	int count = 0;

	while(iter != dq.end() && count < j)
	{
		cout << dq[count] << " " << count << endl;
		count++;
		iter++;
	}

...which I think almost works. It outputs:

0 0
1 1
-2.53017e-098 2
1 3
-2.53017e-098 4
-2.65698e+303 5
2 6
3 7
-2.53017e-098 8
3 9

Very curious.

[mitrmkar]

When you write to the file, sizeof(m_data) gives sizeof(deque<double>) . Instead you need sizeof(double) .

[nbaztec]

Silly me, even I missed the while condition :P

[caut_baia]

#include <string>
#include <fstream>
#include <cassert>
#include <deque>
#include <iostream>

template <typename type,template <typename> class cont> struct stret {
   unsigned long size;
   enum {TYPE_SIZE=sizeof(type)};
   std::fstream file;
   type obj;
   
   stret (std::string flname,cont<type>& data)  {
      file.open(flname.c_str(),std::ios::binary|std::ios::out);
      size=data.size();
      assert(file.is_open());
      for (cont<type>::iterator i=data.begin();i!=data.end();++i)  {
         assert(file.write((unsigned char*) &(*i),TYPE_SIZE));
       }
      file.close();
    }
   stret (std::string flname,cont<type>& data,unsigned long sz)  {
      file.open(flname.c_str(),std::ios::binary|std::ios::in);
      assert(file.is_open());
      
      while (data.size()<sz)  {
         assert(file.read((unsigned char*) &obj,TYPE_SIZE));  
         data.push_back(obj);
       }
      file.close();
    }
  operator unsigned long () {return size;}
 };

int main ()  {
   unsigned long sz;
   std::deque<int> d(56,45);
   std::deque<int> c;

   sz=stret <int,std::deque> ("file",d);
   stret <int,std::deque> ("file",c,sz);
   copy (c.begin(),c.end(),std::ostream_iterator<int>(std::cout," "));
 }

[Jsplinter]

woo hoo!!! it works!!! marking as solved and I'm about to go through the solution by caut_baia... Thank you everyone! I think next time I'll be quicker to find my mistakes (i hope).

[Jsplinter]

@caut_baia, I want to understand your solution. It looks brilliant. I copied the code into a console application, but I get the Error: class template "std::deque" is not compatible with template template parameter "cont". That baffles me because I thought deque was a template class.