General way to test objects for mutability

[hughesadam_87]

Hi,

I am trying to create the most general function possible to test if an object passed into it has mutability or not. With some prior help from pytony, it seemed that the best way to do this is to try to set and attribute. If the attribute can be set, it is mutable, otherwise it will force an error.

To make this function general, I need to be able to pick an attribute from the object. I thought the best way to do this would be to list the attributes, pick the first one listed and then overwrite it. It would be superfulous to try to overwrite every attribute, no?

So the steps are:

1. Take in object.
2. Make a deepcopy so as not to overwrite the object.
3. List its attributes.
4. Choose a single attribute.
5. Try running setattr on this.
6. Infer mutability based whether or not errors appear.

In regard to step 2, I"m confused on the most general way to get class attribtues in Python. From another website, the recommended way was:

>>> class new_class():
...   def __init__(self, number):
...     self.multi = int(number) * 2
...     self.str = str(number)
... 
>>> a = new_class(2)
>>> a.__dict__
{'multi': 4, 'str': '2'}
>>> a.__dict__.keys()
dict_keys(['multi', 'str'])

This seems messy and can it be guaranteed that all mutable python objects will have the dict variable? Does the presence of the dict variable automatically indicate mutability?

[lrh9]

Why do you need to check an object's mutability? Sounds like you need to solve a different problem or use a different solution. Technically you could try to make whatever changes you wanted to make and catch any AttributeError exceptions, but it still seems funny. So...?

[hughesadam_87]

I'm trying to write some functions that work the same way on two different objects types in my codes. If the function resets attributes or returns a new object depends on if the object is inherently mutable. In the future I may just make a set of mutable functions and a set of immutable functions if this type of behavior is not pythonic.

[griswolf]

I see no need to make the deep copy. Pick an attr, save its value, try to change it, then if it changed, put back the original.

But I'm with lrh9: A function that does two different things is an odd duck: The code that calls it cannot depend on the return value to be what it expects. Why not always return a mutated copy rather than only sometimes?

[lrh9]

So you are pretty much trying to make functions and methods that behave differently depending on an argument's type or behavior? That's not a very good idea. It's very difficult to ensure that you can appropriately handle all types or behaviors in one function or method. Even if you can, you make it very difficult for yourself or other developers.

If your function or method returns a copy when the object is immutable, what does it return when the object is mutable? The same object? Now imagine you call this function and have to work with the return value. You pretty much have to do the same mutability checking you did in your function or method to ensure you handle the return value properly. This introduces more code, more overhead, and more points where your software can break.

I don't know any standard solutions. What is your specific case? Do you have any code?

I ran into this problem myself when I was working on some trivial neural network code. I wanted to use either a pair of sequences or a pair of mappings to represent a pair of neuron inputs and corresponding neuron weights. If the inputs was a sequence I would use the index of each input to look up the corresponding weight, or if the inputs was a mapping I would use the key of each input to look up the corresponding weight. At first I decided to use the collections abstract base classes to check if the inputs was a sequence or mapping, but I eventually decided to make two different neuron types - one based on sequences and one based on mappings.

[Ene Uran]

Mutable objects can change their value but keep their id().

[hughesadam_87]

Sorry for the late reply. I've replied to several above posts.

I see no need to make the deep copy. Pick an attr, save its value, try to change it, then if it changed, put back the original.

I did this because there are very few cases where attribute setting can trigger events later in the code, for example in the Traits package from the Enthought Tool Suite. I will probably remove the copying behavior and just keep this in mind as a caveat.

So you are pretty much trying to make functions and methods that behave differently depending on an argument's type or behavior? That's not a very good idea. It's very difficult to ensure that you can appropriately handle all types or behaviors in one function or method. Even if you can, you make it very difficult for yourself or other developers.

That's true. I'm working on a project to store general record types in python. For immutable records, this is just an extension of namedtuples. I'd like for the immutable and mutable (object subclass) to be very similar and have a set of functions that works nearly identically on both of these. In some cases, this is troublesome because certain operations are intrinsically dependent on mutability. Since I am trying to alleviate these headaches from user, I want to user to only learn one function and then blissfully apply it to mutable and immutable records in general. Because I don't want to limit users to my particular classes, I think it would be better if these functions inspect mutability in a general sense (hence my motivation for the thread). I agree though, that it's not smart to make a function change behavior based on input.

If your function or method returns a copy when the object is immutable, what does it return when the object is mutable? The same object? Now imagine you call this function and have to work with the return value. You pretty much have to do the same mutability checking you did in your function or method to ensure you handle the return value properly. This introduces more code, more overhead, and more points where your software can break.

For mutable objects, I figured it would be less resource-intense to just modify them in place. For immutable objects, one has no choice but to make a new object.

I don't know any standard solutions. What is your specific case? Do you have any code?

I have a crude page together with a really sloppy api and intro. Will update it soon. Maybe it will give you the idea.

http://hugadams.github.com/pyrecords/
https://github.com/hugadams/pyrecords

[lrh9]

This just doesn't sound like a good idea. If it were me, I'd specify interfaces and let other programmers work out how to fulfill those interfaces.

For instance, the sum function expects an iterable of summable objects. It doesn't care if a programmer stores his or her data in a csv file, json file, shelve, or whatever. It's up to the programmer to get the data he or she wants to use in the correct structure; he or she has to fulfill the interface.

It's better. You don't have to worry about what other programmers are doing. You don't have to write code to fulfill the interface for every situation. You can just specify the interface and let other programmers fulfill it. It's more robust. They can handle situations that you can't foresee. It keeps your code lean.

P.S. When I asked what your case was, I was talking about the project that you are writing this system for. I assume you are writing this to address something in another project? If you aren't, you are "solving" a "problem" that doesn't need to be solved.

[hughesadam_87]

This just doesn't sound like a good idea. If it were me, I'd specify interfaces and let other programmers work out how to fulfill those interfaces.

For instance, the sum function expects an iterable of summable objects. It doesn't care if a programmer stores his or her data in a csv file, json file, shelve, or whatever. It's up to the programmer to get the data he or she wants to use in the correct structure; he or she has to fulfill the interface.

I agree with what you're saying and am looking for the best solution to my problem; however, I'm not redefining the function for every possible data type. I'm not doing something like:

def example(input):
   if type(input) == string:
     do something 
   elif type(input) == int:
     do something else

I probably will just end up making two functions, one which modifies objects in place and one that returns new objects. This way, the immutable objects will clearly error when attempts are made to manipulate in place. Maybe I will just make "in_place" a keyword in the function call.

P.S. When I asked what your case was, I was talking about the project that you are writing this system for. I assume you are writing this to address something in another project? If you aren't, you are "solving" a "problem" that doesn't need to be solved.

This project actually stemmed from my need to handle a certain type of data from bioinformatics. I'm not trying to solve a problem that 'doesn't need to be solved'. I think the point of Python packages, for example numpy, is to provide an API to users that eases the burdeon on them to reinvent certain wheels and provide an intelligent, systematic pattern for programming efficiently in that domain. In particular, the domain I'm trying to address is record keeping in Python, beyond what is provided by the namedtuple class and the common examplse of object subclassing. I'm attempting to improve and generalize the process for creating these objects and also providing a series of utility functions to handle common record manipulation operations. It may not prove a useful endeavor in the end, but I do think this is an area where python could improve.

THanks for the helpful discussion guys.

[lrh9]

Databases might be applicable. Python has the sqlite3 module for simple databases. Do you have some examples of the data you want to work with?

[hughesadam_87]

Hi Irh9,

You're right that I do need to look into sqlite and hdf5 more thoroughly before I try to emulate any of their functionality. The namedtuple example page Click Here actually demonstrates that namedtuples interface well to sqlite, and this suggests to me that they still have an important role apart from sqlite, but I do need to take a break and survey these packages.

I have to put this project on the shelf for a bit unfortunately, but I appreciate you're helpful discussion and insights. I will ressurect this thread in the future and hopefully you can continue to offer these good recommendations.

Thanks.