Member Avatar for james.lu.75491856 
def mu_cache(func,maxsize=128):
    "Most Used Cache decorator"
    cname = "__"+func.__name__+"_Cache"
    rname = "__"+func.__name__+"_Rankings"
    exec("global "+cname)
    exec(cname+" = {}") #here
    exec("global "+rname)
    exec(rname+" = {}")
    def wrap(*args,**kwargs):
            cache = eval(cname)
            rankings = eval(rname)
            try:
                    res = cache[(args,kwargs)]
            except:
                    res = func(*args,**kwargs)
                    cache[(args,kwargs)] = res
            else:
                    try:
                            res = rname[(args,kwargs)]
                    except:
                            res=0
                    rname[(args,kwargs)] = res+1
            def delete_smallest():
                global rankings
                smallest=None
                todelete=None
                for args in rankings:
                    r = rankings[args]
                    if (smallest == None) or r<=smallest:
                        smallest=r
                        todelete=args
                del cache[todelete]
            while len(cache)> maxsize:
                delete_smallest()
    return wrap
SyntaxError: unqualified exec is not allowed in function 'mu_cache' it contains a nested function with free variables (<pyshell#10>, line 3)

I have no idea what this means; and I am trying to make a copy of functools.lru_cache but with Most Used instead of Least Recently Used.

  • 3 Contributors
  • 4 Replies
  • 379 Views
  • 4 Hours Discussion Span
  • Latest Post Latest Post by sepp2k

Dani AI

Generated

The SyntaxError comes from using an unqualified exec inside a function that defines a nested function which closes over outer names. Unqualified exec can create or rebind names at runtime, so the compiler cannot determine which variables are locals vs. closure variables. Python therefore disallows that pattern. Removing exec and using explicit data structures is the correct fix.

Closures already provide persistent per-decorator state, so globals are unnecessary for “carry over between calls.” As explained, define cache/counts in the outer decorator and access them from the wrapper. If a single shared store for many decorated functions is required, use a module-level registry or a single object as suggested. Avoid bare except: for cache lookups; use explicit membership tests or dict.get(). Also ensure the wrapper returns the computed value (the original code omitted that).

A compact, safer MFU (most-frequently-used) decorator pattern:

from functools import wraps
from collections import Counter
from threading import RLock

def _make_key(args, kwargs):
    try:
        return (args, tuple(sorted(kwargs.items())))
    except TypeError:
        return repr((args, kwargs))  # fallback for unhashable elements

def most_used_cache(maxsize=128):
    def decorator(func):
        cache = {}
        counts = Counter()
        lock = RLock()
        @wraps(func)
        def wrapped(*args, **kwargs):
            key = _make_key(args, kwargs)
            with lock:
                if key in cache:
                    counts[key] += 1
                    return cache[key]
            result = func(*args, **kwargs)
            with lock:
                cache[key] = result
                counts[key] = counts.get(key, 0) + 1
                if len(cache) > maxsize:
                    victim = min(counts, key=counts.get)
                    cache.pop(victim, None)
                    counts.pop(victim, None)
            return result
        return wrapped
    return decorator

Notes: the key helper avoids unhashable dict objects by using a sorted tuple of items; that fallback may be imperfect (repr collisions) — for full robustness use a canonical serialization. For multi-threaded use the lock; for very large caches a heap or more advanced data structure will be more efficient than repeated min().

Recommended Answers

All 4 Replies

Member Avatar for sepp2k

If a function is nested in another function, you're not allowed to use unqualified exec in either of the two functions. Unqualified in this case means "without in". An explanation of why this limitation exists can be found here.

In your code you don't really need exec. To set and get global variables by name, you can just use globals()[name], but you don't even need global variables either. You can just define cache and rankings as local variables in mu_cache and then access them in wrap.

Another error in your code is that you use global rankings in delete_smallest. This will cause an error message about there being no global variable named rankings because rankings is a local variable in the wrap function, not a global variable. If you just remove that line, the error will disappear and you will be correctly able to access the rankings variable.

commented: very good remarks +14
Member Avatar for james.lu.75491856

I still need to set a global variable because It carrys over function calls

Member Avatar for Gribouillis

I still need to set a global variable because It carrys over function calls

Use a single global object

class _MuCacheData(object):
    def __init__(self):
        # define here arbitrary containers that you need
        pass

    def mu_cache(self, func, maxsize = 128):
        # write and return your wrapper here
        # Access persistent data through self
        pass


# A single instance is used for the decorator
mu_cache = _MuCacheData().mu_cache

Don't use exec() nor the global statement.

Edited by Gribouillis
Member Avatar for sepp2k

I still need to set a global variable because It carrys over function calls

No, you don't; local variables that are closed over from an outer function do carry over between function calls. Look at this:

def memoize(f):
    cache = {}

    def wrap(x):
        print("Contents of cache: {}".format(cache))
        if x not in cache:
            print("Calculating {}({})".format(f.__name__, x))
            cache[x] = f(x)
        return cache[x]

    return wrap

def double(x):
    return 2 * x

def square(x):
    return x*x

mdouble = memoize(double)        
msquare = memoize(square)

print("Double:")
mdouble(3)
mdouble(3)

print("\nSquare:")
msquare(3)
msquare(3)

Output:

Double:
Contents of cache: {}
Calculating double(3)
Contents of cache: {3: 6}

Square:
Contents of cache: {}
Calculating square(3)
Contents of cache: {3: 9}

As you see both msquare and mdouble have their own cache and those caches persist between calls to msquare and mdouble respectively. So this acts exactly like you want without any globals.

Edited by sepp2k
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.