Limit size of a List

[bumsfeld]

Is there a way to limit size of a list in Python, so it does not use all memory?

[G-Do]

Hi bumsfeld,

I believe that the array data structure in the Numeric/NumPy package is both typed and non-resizable, just like C arrays (actually, I'm pretty sure that these data structures are C arrays wearing a Python hat). Check out:

http://numeric.scipy.org

[vegaseat]

Thanks G-Do for your information, so NumPy eventually replaces Numeric and Numarray. Interesting!

When I was coding in C, I used to sample lab data and put it into a circular or ring buffer to keep the thing from overflowing. This was an easy way to calculate a moving average.

You can do a similar thing in Python using a list and keeping track of the index. When the index reaches the limit you have set, new data starts at index zero again.

A queue makes this simpler, particularly the deque container, because you can add to the end and pop the front. Here is an example, I hope you can see what is happening:

# the equivalent of a circular size limited list
# also known as ring buffer, pops the oldest data item
# to make room for newest data item when max size is reached
# uses the double ended queue available in Python24
 
from collections import deque
 
class RingBuffer(deque):
    """
    inherits deque, pops the oldest data to make room
    for the newest data when size is reached
    """
    def __init__(self, size):
        deque.__init__(self)
        self.size = size
        
    def full_append(self, item):
        deque.append(self, item)
        # full, pop the oldest item, left most item
        self.popleft()
        
    def append(self, item):
        deque.append(self, item)
        # max size reached, append becomes full_append
        if len(self) == self.size:
            self.append = self.full_append
    
    def get(self):
        """returns a list of size items (newest items)"""
        return list(self)

    
# testing
if __name__ == '__main__':
    size = 5
    ring = RingBuffer(size)
    for x in range(9):
        ring.append(x)
        print ring.get()  # test
 
"""
notice that the left most item is popped to make room
result =
[0]
[0, 1]
[0, 1, 2]
[0, 1, 2, 3]
[0, 1, 2, 3, 4]
[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]
[3, 4, 5, 6, 7]
[4, 5, 6, 7, 8]
"""

[bumsfeld]

So the trick is to make size of ring buffer large enough to retain all pertinent data long enough?
I guess the answer is yes.

[wrybread]

Why not something like:

if len(mylist) > 5:
...del mylist[0]

[vegaseat]

Why not something like:

if len(mylist) > 5:
...del mylist[0]

Good idea, but if mylist is appended to in several different places in your code than it would not be very practical.