Member Avatar for sadsdw

Hi friends,

I wrote a code to build a matrix with specific calculations. When I'm using 3000 elements works well ... But I did a test with 150.000 (150.000 x 150.000) and I had problems.

" terminate called after throwing an instance of 'std::bad_alloc' "


I'm copying the code part of the problem:

void buildMatrix(ofstream &myfileOutSubsetZ, ofstream &myfileOutSubset,ofstream &myfileOut,ofstream &myfileLog,vector<string> &vetorInd)
{
	float **AAA = 0;
	AAA = new float *[vetorInd.size()];
	for(int i=0;i<vetorInd.size();i++)
		AAA[i]= new float [vetorInd.size()]; // PROBLEM!
        ...

Can you help me please?

Cheers!

  • 4 Contributors
  • 10 Replies
  • 2K Views
  • 22 Hours Discussion Span
  • Latest Post Latest Post by sadsdw

Dani AI

Generated

and are pointing you the right way: the matrix is simply too big. For n = 150000, nnsizeof(float) is about 90e9 bytes (~84 GiB). Even on a 64-bit build, that is beyond typical RAM and will almost certainly trip std::bad_alloc. There is a second issue in your snippet: allocating a float** with n separate new calls creates 150000 large, independent allocations, which is slow, fragile, and leaks memory if any row allocation fails mid-loop. Also, prefer size_t for sizes/indices so you do not risk overflow or signed/unsigned mismatches.

Catching the exception (as noted) only tells you the allocation failed; the fix is to not allocate O(n^2) memory. If you only need the results written out, stream them row-by-row (or in tiles) so peak memory stays O(n) or O(B^2). If the matrix is symmetric, store only one triangle; if it is sparse, use a compressed format. And remember the output itself will be ~90 GB, so write in binary and plan for disk space.

Here is a minimal row-at-a-time pattern that avoids the giant allocation and cleans up automatically:

#include <vector>
#include <fstream>

static inline float compute(size_t i, size_t j); // your formula

void buildMatrix_streaming(size_t n, std::ofstream& outBin) {
    outBin.exceptions(std::ofstream::failbit | std::ofstream::badbit);
    std::vector<float> row(n);
    for (size_t i = 0; i < n; ++i) {
        for (size_t j = 0; j < n; ++j)
            row[j] = compute(i, j);
        outBin.write(reinterpret_cast<const char*>(row.data()),
                     row.size() * sizeof(float));
    }
}

If you truly must hold blocks in memory for further work, tile the problem: pick a block size B that fits in RAM/cache (e.g., 512), compute a BxB block, process or flush it, and move on. This keeps memory bounded regardless of n.

Recommended Answers

All 10 Replies

Member Avatar for Insensus

Because 150.000 x 150.000 floats equals ~21GB of memory.
I don't think you have that much RAM in your computer.

Member Avatar for Narue

The new operator throws an exception of std::bad_alloc if the allocation fails.

[rhetorical question]
Did you catch the exception?
[/rhetorical question]

Member Avatar for sadsdw

Thanks Insensus and Narue!

Narue, No ... I didn't take the exception. I don't know ho to do this. Is it simple?

Cheers

Member Avatar for sadsdw

Narue, is it correct?

try
	{
		for(int i=0;i<vetorAni.size();i++)
				A[i]= new float [vetorAni.size()];


	}catch(bad_alloc)
	{
		cout << "Exception raised: " << bad_alloc << '\n';
	}
Member Avatar for sadsdw

Narue,

I tried this one but I couldn't take the message ... :(

try
    {
        for(int i=0;i<vetorAni.size();i++)
                A[i]= new float [vetorAni.size()];
    }catch(char * str)
    {
        myfileLog << "\nException raised: " << str << endl;
    }
Edited by Reverend Jim because: Fixed formatting
Member Avatar for Narue

Okay, instead of just guessing and writing random stuff, go read a C++ reference on handling exceptions. Or search google. Of course, there's still the issue of trying to allocate exorbitant amounts of memory.

commented: It was an easy walk to get past AD, now for the long climb to the #1 spot +17
Member Avatar for sadsdw

Thanks Narue!

Cheers

Member Avatar for sadsdw

The exception that happened is "St9bad_alloc".

Can you help me?

Member Avatar for mike_2000_17

The problem is clear, and has been answered already. std::bad_alloc is the exception that is thrown when dynamic memory allocation fails. Generally, the most likely reason why dynamic allocation can fail is because it would require more memory than you have available on your computer. When allocating a 150,000 x 150,000 array, it comes to no surprise that you run out of memory, that's about 84 Gb (I think that a system which can produce or at least emulate 84Gb of contiguous RAM memory is very rare).

Try to avoid needing such a large amount of memory at once in your program.

BTW, to catch the exception, you do:

#include <iostream>

int main() {
  try {
    float * huge_array = new float[150000 * 150000];
    //...
    delete huge_array;
  } catch(std::bad_alloc& e) {
    std::cout << "Could not allocate the required memory, failed with error: '" << e.what() << "'" << std::endl;
    return 1;
  };
  return 0;
};
Edited by mike_2000_17 because: n/a
Member Avatar for sadsdw

Thanks Mike! I will try to replace that ...
Cheers

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.