Freeing memory from vector of structs

[onako]

After I performed some operations on the vector of structs, I need to free the memory. I suppose clear() will not be sufficient, but I'm not sure how to perform individual delete.

typedef std::vector<VertexRAM> VecVertexRAM; // definition of a vector of structs

template <typename type>
void freeFromMemory(std::vector<type>& myVec) {
    typename std::vector<type>::iterator myIter=myVec.begin();
    while(myIter!=myVec.end()) {
	delete(*myIter);
	++myIter;
    }
    myVec.clear();
}

This outputs the following error:
Code:

error: type ‘struct VertexRAM’ argument given to ‘delete’, expected pointer

Any ideas on how to do this? Thanks

[Narue]

The vector handles its own memory. You only need to free pointers that were explicitly allocated with new.

[onako]

Here is the thing.
In a specific function, I need to perform operation on std::vector<VectorRAM>. Obviously, this vector consumes much memory. After I finish operation with this vector, I need to perform some 'other operations' which would consume additional memory (and, therefore, I would need to free it).
This means, in the middle of the function I would need to get rid of that vector consuming memory. Any help on this is welcome.
Thanks

[arkoenig]

Suppose you have an object of a type std::vector<VectorRam>. That object contains elements, each one of which is an object of type VectorRam.

If you want to free the memory that those VectorRam objects occupy, there are two ways to do so:

1) Free the vector itself. That is, if the vector is a local variable, allow it to go out of scope.

2) Reduce the number of elements in the vector. Calling myVec.clear() is one way to do that; it reduces the number of elements to zero. If you don't want to get rid of all of the elements, you could call myVec.erase or myVec.resize to get rid of just some of them.

What you cannot do is apply delete to individual elements of the vector. You can use delete only with a pointer to an object that was allocated with new ; in this case you do not even have a pointer, let alone a pointer to dynamically allocated memory.

[Narue]

Here is the thing.
In a specific function, I need to perform operation on std::vector<VectorRAM>. Obviously, this vector consumes much memory. After I finish operation with this vector, I need to perform some 'other operations' which would consume additional memory (and, therefore, I would need to free it).
This means, in the middle of the function I would need to get rid of that vector consuming memory. Any help on this is welcome.
Thanks

Here's reality. Even if you clear the elements of the vector, the capacity isn't guaranteed to decrease. The only way to truly free the memory owned by a vector is to force the vector object out of scope. There's a trick to accomplish this with a temporary though, and avoid waiting until your non-temporary object naturally goes out of scope:

std::vector().swap(myVec);

The idea is to create a new temporary vector of the size you want (in this case zero), then swap that vector with your non-temporary object. The capacity is also swapped which means that when the temporary goes out of scope almost immediately, the reserved memory is released for other usage.

[arkoenig]

Here's reality. Even if you clear the elements of the vector, the capacity isn't guaranteed to decrease. The only way to truly free the memory owned by a vector is to force the vector object out of scope.

Yes, but in this particular case, that probably doesn't matter.

The original poster was saying that the individual vector elements consumed large amounts of memory. In almost all cases, class objects that consume large amounts of memory do so by dynamically allocating that memory when they are constructed and freeing it when they are destroyed. To do otherwise requires fixing the amount of memory used during compilation, and has the side effect that every class element uses the same amount of memory, whether or not it needs to do so.

So if the class objects are sensibly designed, almost all of the memory occupied by each class object will be freed when the object is destroyed, and calling clear will work as I described. The extra memory consumed by the vector itself will be trivial in comparison to the memory consumed by its objects.

[Narue]

So if the class objects are sensibly designed, almost all of the memory occupied by each class object will be freed when the object is destroyed, and calling clear will work as I described. The extra memory consumed by the vector itself will be trivial in comparison to the memory consumed by its objects.

Drop your ego for a moment and look at the situation objectively. It's equally likely that we're looking at a large vector of large objects, in which case the memory owned by the vector is the culprit, not the memory allocated internally by each object. However, even if the problem is internally allocated memory the swap solution still works, while if the problem is not internally allocated memory the clear solution does not. I fail to see why you would advocate a solution that only works for an unconfirmed case (ie. you're exhibiting wishful thinking) and dismiss a solution that works in all cases.

[mike_2000_17]

Narue is right, the clear() function is not specifically guaranteed to free all the memory because for memory-optimization purposes, in general, the vector class can keep a large amount of allocated memory in anticipation of future reuse of the memory. So calling clear(), does call all destructors of the elements it contains, but it does not necessarily free the memory allocated. Narue's method will accomplish that (well I assume it will, taking Narue's word for it). Look at the difference between std::vector::size() and std::vector::capacity().

[onako]

Thank you for your messages. This is how (I believe) I resolved the issue:

template <typename VecType>
void freeFromMemory(VecType& myVec) {
     myVec.clear();
     VecType vert;
     myVec.swap(vert);
}

...but
Even by monitoring RAM consumption (using System Monitor in Ubuntu), after I free the memory consumed by vector (by some of the methods described above, such as swapping, for example) I see no decrease in RAM memory consumption (actually, sometimes there is a large drop, sometimes very little drop, and sometimes no drop at all). This is why I want to have really free memory. Are there guarantees that it the memory will be released?
Thanks again

[onako]

I still wonder whether

myVec.resize(0)

or consecutive calls to

myVec.pop_back()

might guarantee that the memory will really be released. If the swapping method is faster then the above (given that some of them really release the memory), I should use it. I hope a call myVec.clear() (as I did in the freeFromMemory) is not clashing with swapping.

[arkoenig]

I fail to see why you would advocate a solution that only works for an unconfirmed case (ie. you're exhibiting wishful thinking) and dismiss a solution that works in all cases.

1) Because I don't like premature optimization.

2) Because I think that before trying to explain this particular technique, it is important to be sure that the original poster understands why trying to apply delete to a vector element is unlikely to do what was intended.

So I think the right thing to do is to explain (and to urge the original poster to understand) the issues behind the original question before proceeding. Otherwise, we wind up with the original poster writing

myVec.clear();
VecType vert;
myVec.swap(vert);

without understanding why the call to clear is unnecessary, or why the second and third statements could have been written as

VecType().swap(myVec);      // This works...

but not as

myVec.swap(VecType());      // ...but this doesn't.

My teaching experience has shown me that the best time to explain these intricacies is only after the fundamentals are well understood.

Ego has nothing to do with it.

[Narue]

>I see no decrease in RAM memory consumption
Nor would I expect you to see anything definitive. The memory is still owned by your process, but free to be reused for other things within the process. There's a difference between being freed for reuse within the process and being freed for reuse by all processes. Your system monitor only looks at process-level memory consumption.

And no, calling resize, reserve, or pop_back is absolutely not guaranteed to release memory owned by the vector for elements.

[Narue]

Click to Expand / Collapse Quote originally posted by Narue ...
1) Because I don't like premature optimization.

2) Because I think that before trying to explain this particular technique, it is important to be sure that the original poster understands why trying to apply delete to a vector element is unlikely to do what was intended.

So I think the right thing to do is to explain (and to urge the original poster to understand) the issues behind the original question before proceeding. Otherwise, we wind up with the original poster writing

Failing to solve the problem by oversimplifying is not the same thing as premature optimization. Personally I'd find your excuse about explaining the swap technique more convincing if you had actually bothered to describe it at all rather than try palming off (and then attempting to defend) an incorrect solution to the stated problem.

If you're teaching a class and have control over the material as well as the projects, progression is fine. But when helping someone solve a real problem, offering a viable answer immediately and then explaining it is vastly more productive. So by all means show him why he was wrong to try using delete with a vector. Certainly offer options for doing it the right way. But if you withhold the best solutions for fear of confusing him, expect someone such as myself to cry foul.

[arkoenig]

I still wonder whether

myVec.resize(0)

or consecutive calls to

myVec.pop_back()

might guarantee that the memory will really be released. If the swapping method is faster then the above (given that some of them really release the memory), I should use it. I hope a call myVec.clear() (as I did in the freeFromMemory) is not clashing with swapping.

I assume that you're talking about the memory allocated by the vector , (i.e. the memory occupied, not allocated, by the vector 's elements) not the memory allocated by its elements.
In that case, pop_back will definitely not free that memory, because doing so would require invalidating all iterators to elements of the vector .

Calling resize(0) is permitted to free the vector 's memory, but is not required to do so. For that matter, the swap technique is not required to free the memory either; but it is harder to understand how one might go about implementing the C++ library in a way that causes it not to do so.

By far the most straightforward way of freeing the memoryh associated with an vector is to make the vector a local variable and let it go out of scope when you no longer need it. Of course, that technique may or may not apply to your particular program.

[arkoenig]

Failing to solve the problem by oversimplifying is not the same thing as premature optimization.

Failing to understand a problem thoroughly before trying to solve it is the kind of carelessness that often leads to premature optimization.

Evidently we have different ideas about what is important. Nothing wrong with that--but I would like to suggest that you think twice before assuming that everyone who disagrees with you is wrong.

[Kanoisa]

Um i dont know if this really helps but this peaked my curiosity so i made a program to test all these suggested methods.

I have attached the results(.txt file) in a zip file with the source codes i used which i explain here.

basically what it does is defines a struct containing 40 doubles; and then allocates a vector of 100 of these. pseudo code follows

bigStruct myBigStruct; //structure which contains 40 doubles so somewhat big
vecotr<bigStruct> memoryMuncher; //vector of these structures

//pause to allow memory usage reading to be taken

//add 100 big structs to vector

//pause to allow memory usage reading to be taken

//use whatever method to "clear" memory

//pause to allow memory usage reading to be taken

//end

The results i find somewhat strange as i dont quite understand how windows names memory so i took 2 readings. private data and virtual size.

with all methods except swap both values increased after adding 100 elements but did not decrease after using clear(), resize(0) or pop_back(); using swap decreased the private data only (i assume this is memory where my vars are) but not down as far as when the program started. the virtual size for swap did not change like the others.

Does this mean that none of the methods actually work except swap? and if so what extra memory is not free'd from swap?

PS. i had hoped this experiment would reveal the answer of which way worked. But unfortunately i can't provide any evidance other than swap worked better than others.

[arkoenig]

Otherwise, we wind up with the original poster writing

myVec.clear();
VecType vert;
myVec.swap(vert);

without understanding why the call to clear is unnecessary, or why the second and third statements could have been written as

VecType().swap(myVec);      // This works...

but not as

myVec.swap(VecType());      // ...but this doesn't.

It occurred to me just now that these remarks don't tell the whole story. Consider:

// Example 1
{
    VecType myVec;

    // Put a bunch of stuff in myVec

    VecType().swap(myVec);      // Clear myVec

    // Put a bunch of stuff in myVec again

    ...
}

// Example 2
{
    VecType myVec;

    // Put a bunch of stuff in myVec

    VecType temp;
    temp.swap(myVec);      // Clear myVec

    // Put a bunch of stuff in myVec again

    ...
}

These two examples are likely to have very different characteristics in terms of how much memory they consume. For the moment, I'll leave it as an exercise to figure out why. When you've figured it out, I think you will see why I say that there is more to this problem of controlling memory allocation than meets the eye, and perhaps you will understand better why I wanted to concentrate on the fundamentals first.

I am reminded of Brian Kernighan's two rules of optimization.

1) Don't do it.

2) [for experts only] Don't do it yet.

[mike_2000_17]

@Kanoisa:
What function did you use to obtain the total memory allocated on the process heap? I used to use AllocMemSize / AllocMemCount back in my days of Delphi programming, but I have never found one for C++, so I would like to know, even if it's just for windows. (You didn't include the main source file in your zip, memoryDealloc.cpp is missing from it)

@arkoenig:
Also, for the two examples you gave. First of all, if you are going to repopulate the vector myVec after the clear(), then who cares if the clearing method used actually frees the memory or not? Second, in the example 2, the "big memory" of the original myVec will not be destroyed until temp goes out of scope, at the final } in this example, as opposed to example 1 where the "big memory" will be destroyed at the semi-colon of the same line, i.e. the end of the expression's evaluation. That's all the difference I can see. But again, both examples are ridiculous because of my first point.

Finally,
@onako:
If you are really concerned about making sure that that vector is freed exactly when you "command" it to free and for some reason you cannot make your vector go out of scope when you want it to. I see an obvious solution:

Hold your vector as a pointer and allocate it with new and then delete it when you want to fully clear it. I think that should end the debate on which method is the best to clear the "capacity" allocation of a std::vector.

If for some weird reason that solution is not good for you, then you might want to consider going one level lower and use a C-style array with new[] and delete[], or make your own custom vector class based on a C-style array. There is nothing wrong with using the right tool for your job, and although std::vector is almost always nicer to use then arrays, sometimes it's not the right tool. std::vector is tuned for random-access, highly-dynamic memory, that means, lots of resizing back and forth, so if you have a BIG vector to allocate and use for a while then free it all at once, then a C-style array could be more appropriate.