Deallocating memory used by a STL map

[DarkNova]

I am having a much harder time than I should trying to deallocate the memory used by an STL map. Some example code:

void mainfunc()
{
  testmap();
  testvect();
}

void testmap()
{
  map<int, float> testmap;
  for (int i = 0; i < 1000000; i++) {
    testmap[i] = (float)i;
  }
  testmap.clear();
}

void testvect()
{
  vector<int> testvect;
  for (int i = 0; i < 10000000; i++) {
    testvect.insert(testvect.end(), i);
  }
  testvect.clear();
}

My code creates a STL map and then when it is done with it, I want the memory to be deallocated. I use a debugger and put a breakpoint in the mainfunc() and look at the memory used by the process. In this example, the testvect function works as I would expect, in that the for loop inserts a whole bunch of elements in the vector, so while it is in testvect() the memory usage increases and when testvect returns back to mainfunc(), the destructor is automatically called on the vector and the memory is deallocated.

However, the testmap() function doesn't work the same way. Memory usage goes up while it is inserting elements into the map but when the testmap() function returns and goes back to mainfunc(), the process is still using the same amount of memory. The memory is never getting deallocated. I feel like I must be doing something stupid as this seems so simple. Can anyone point out what I'm doing wrong? I'm using G++ 3.4.6. Thanks.

[Salem]

It goes up when you call testmap(), but does it go up again when you call testvec() ?

Using a process monitor to determine how much memory is in use can be misleading. When you delete memory in your program, it typically goes back into a pool of memory owned by the program to be re-used later, it is not returned to the OS.
If this is true, you will only see one jump in memory usage.

Allocating and freeing memory within the program is a matter of adjusting a few pointers (ie quick). Getting more memory from the OS usually involves a system call (slow), so the program run-time library works hard to avoid this as much as possible. Namely asking for large blocks, and not giving it back.

[vijayan121]

you could write a custom allocator that tracks calls to allocate and deallocate.

#include <map>

template< typename T > struct myallocator : public std::allocator<T>
{
  typedef std::allocator<T> base ;
  typedef typename base::size_type size_type;
  typedef typename base::difference_type  difference_type;
  typedef typename base::pointer pointer;
  typedef typename base::const_pointer const_pointer;
  typedef typename base::reference reference;
  typedef typename base::const_reference const_reference;
  typedef typename base::value_type value_type;
  myallocator() throw() {}
  myallocator( const myallocator& a ) throw() : base(a) {}
  template <typename X> myallocator(
             const myallocator<X>& ) throw() {}
  ~myallocator() throw() {}

  template <typename X> struct rebind
  { typedef myallocator<X> other; };

  pointer allocate( size_type sz, const void* hint = 0 )
  {
    // record alloc request eg. ++num_allocs ;
    return base::allocate(sz,hint) ;
  }
  void deallocate( pointer p, size_type n )
  {
    // record dealloc request eg. --num_allocs ;
    return base::deallocate(p,n) ;
  }
};

template<typename T> inline bool operator==(
               const myallocator<T>&, const myallocator<T>& )
{ return true; }

template<typename T> inline bool operator!=(
               const myallocator<T>&, const myallocator<T>& )
{ return false; }

int main()
{
  std::map< int, float, std::less<int>, 
                myallocator<int> > testmap;
  for ( int i = 0; i < 1000000; i++ )
  {
    testmap[i] = (float)i;
  }
  testmap.clear();
}

[DarkNova]

OK, thanks to both of you this is making more sense now. I tracked that the map was both allocating and deallocating memory as I would expect, and then I wrapped a for loop around the map creation/clear section so it would do it 5 times and the memory usage didn't go up past the initial increase because when the map got recreated it just reused the memory it had previously allocated. What was confusing me is that the memory behavior when a map is cleared is different than when the vector is cleared. When I clear the map, the memory stays allocated to my program, but when I clear the vector it is returned to the OS immediately. Is there any way to tell it to give the memory back to the OS? My program creates these giant map that takes up about 1 GB of RAM and then does some processing and the map is destroyed when it goes out of scope. At that point the program actually should be using less than 100 MB, however, it continues to use 1 GB of RAM because the RAM isn't given back to the OS. At what point will the memory be given back to the OS and is there any way to tell it that it will no longer be needed? Thanks for the great help.

[vijayan121]

> When I clear the map, the memory stays allocated to my program, but when I clear the vector it is returned to the OS immediately.
the vector allocation is for one large contiguous chunk of memory, the map makes many allocations of small chunks of memory. the allocator behaviour could be different in each case. (the behaviour varies depending on the implementation: microsoft's heap api on windows, dlmalloc used by gnu and either phkmalloc or jemalloc on bsd)

> Is there any way to tell it to give the memory back to the OS?
with the standard c++ library, implementing a custom allocator is the canonical way of specifying memory management policy. as the map has a large number of allocation and deallocation of small objects, a pool allocator could be used. one implementation is available in http://www.boost.org/libs/pool/doc/index.html

[daviddoria]

I found this:

Use std::vector<T>().swap( c ) to free vector memory

it seems to work!

[ggsddu]

hi DarkNova,
i test your example code in my redhat server, also with g++ 3.4.6.
in fact, i found below things, which may be useful to you:
1) it's not the STL cache the memory, but is the 'malloc' in glibc.
2) you can see the malloc status with the function 'malloc_stats()' or 'mallinfo()', which are defined in <malloc.h>
3) i run your example, and my result is as below:

void testmap()
{
/*
*  <====   at this point, malloc_stats() tell me that: 
*       system bytes     =          0
*       in use bytes     =          0
*/
  map<int, float> testmap;
  for (int i = 0; i < 1000000; i++) {
    testmap[i] = (float)i;
  }
/*
*  <====   at this point, malloc_stats() tell me that: 
*       system bytes     =          48144384
*       in use bytes     =          48005120
*/
  testmap.clear();
/*
*  <====   at this point, malloc_stats() tell me that: 
*       system bytes     =          48140288    <==== malloc cache the memory here
*       in use bytes     =          5120
*/
}

4) in my test, testmap() and testvect() perform the same, i don't know why your testmap() and testvect() perform differently. If you want to get more detail infomation about how malloc work, please see the chepter 3 in glibc manual. (http://www.gnu.org/software/libc/manual/)

[tux4life]

Hey ggsddu,

Don't you think your reply to this thread came quite late?
(The thread is already dead for over a year)

Another suggestion: Daniweb offers you the ability to wrap your code between
code tags, make use of it, your code will be much easier to read that way :)

Edit:: You must have edited your post (added code tags) while I was writing mine.

[ggsddu]

hey tux4life,

i recently meet the same problem, so i go to google, with the query "stl map memory deallocate", and this page is the 1st result given by google. But i found the answers in the thread not solve the problem. so i try to solve it myself.

i think i should share my finding to everyone else, no matter how old this thread is.