Posts by hughesadam_87

I use wingware IDE. Worth the money.

Good to hear. Perhaps mark this thread as solved? If you have further pandas issues, I'd recommend posting to the pandas mailing list, which is very active and extremely helpful.

The cached property is definatly the best suggestion. If not using that, I would get in the habit of using try: except in these situations. That is syntactially simple and very common, so others reading your code will be able to deduce your intent from the code.

Look at python library collections, particularlu the Counter class.

Your best bet is to load all the datafiles into a pandas dataframe. This is the de-facto structure for handling labeled array data, and also has a ton of utilities for data manipulation ESPECIALLY timestamp.

post an example of one of your files, maybe the first 30 lines, including header, and I can help you get it into a pandas dataframe.

http://pandas.pydata.org/

I can't really digest the entirety of your code just by looking at it, but another thing you might wanna look into is the polymonial support in numpy. There are some functions, mostly for fitting polynomicals of Nth order to other curves, that might be useful to you. For example:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.poly1d.html

Vega, does the experssion "with open(...) as fout" automatically close the file when the iteration completes?

outstring=' '.join([str(i) for i in range(0,105,5)])
file.write(outstring)

Also, it's a good habit not to use the word "file" as this is a python reserved file. I'd recommend outfile or just o.

Can you elaborate on this, your question isn't clear to me.

If you use numpy, then you want to avoid iterating item by item. In fact, the whole reason to use numpy, is that you don't have to do this. If you do do this, you get no speed increase over plain python (the speed increase is about 1000-fold btw).

You can do linear regression out of the box in python already:

http://glowingpython.blogspot.com/2012/03/linear-regression-with-numpy.html

If you are making your own for a class or something, you still want to avoid itearting over the array itself. It does take some practice to get used to this.

Thanks, this is awesome.

Can you put all the zipped values into its own list.

allvalues=[zip(ar1.values, br1.values, ar0.values, br0.values), zip(ar2.values, br2.values, ar1.values, br1.values) ... etc]

Then iterate over just this:

for zippedvals in allvalues:
   for w,x,y,z in zippedvals:
        val = (sqrt((w.data**2)+(x.data**2)))-(sqrt((y.data**2)+(z.data**2)))
        files[0].write(('%6.20f\n') % (val))

Hi guys,

This feels like something that should be possible in Python, and something that I feel like I should know how to do.

Image I have a list of numbers:

items=[10,20,30]

Let's say I wanted iterate over them and define two new variables, a and b.

a=[]
b=[]
for item in items:
  a.append(item*10)
  b.append(item*20)

I can also do this in two list comprehensions:

a=[item*10 for item in items]
b=[item*20 for item in items]

It feel like I should be able to do this in one single expression. Something like:

a,b=[(item*10),(item*20) for item in items]

Is this possible?

Thanks.

I actually added that while I was waiting for responses. I have the c2o or o2c as choices. cups2ounces or ounces2cups. Is that not needed? I am just new :\

Also you are saying to start with a "if" statement?

I'm saying you define two variables, c2o and o2c and then never use them.

Yes. Your code would literally only change at the bottom:

main()

becomes:

if __name__=='__main__':
    main()

The code should run fine and you won't notice any differences.

I noticed a few things.

First, there's no reason to put the entire code into a function (eg main). This is fine, but not mandatory. There's a special line at the bottom of your code that you should add so that if this program is called as an import from another program, then main() won't be executed. It will only be executed if you run the program (ie python thisprogram.py).

Change main() to:

if __name__=='__main__':
    main()

Note that the use of main() and main is just a coincidence. If your main() function was called "main_insane" then it would still be:

if __name__=='__main__':
    main_insane()

Second, why do you have these variables otc and cto?

Third, I would add an error if the use doesn't enter 1 or 2. Something like:

if choice == 1:
    ....
elif choice == 2:
   ...
else:
   raise InputError('Please enter 1 or 2!')

This will let the user know they screwed up.

Ok thanks guys. Let me look at your suggestions, reevaluate my problem, then come back if I still have questions.

Why not just split at "enclosure_url" rather than "title" and keep only the half you want?

Sorry for the late reply. This is exactly what I needed and works great. I honestly think this is the best implementation of property for costly operations. Perhaps thinks of a catchier named than cached_property_with_key2 and put it as a code snippet? I'd like to save this and use it in many future applications, so is there a way you'd like to be credited besided your daniweb account name?

Hi guys,

I am running a costly simulation and trying to optimize the output. I'd have a fixed range over which the parameter of interest can vary, and about 100 values it can take in between. We know the result we are looking for and want to find the parameter value that comes closest to this. Instead of iterating the parameter incrementally, from say 0 to 100, I'd like to implement some sort of binary search algorithm. I am sure there is a "best" value on this range, so there's no need to worry about duplicate values.

The algorithm in my head should go something like:

Parameter low bound (p=0): Value known
Parameter high bound (p=100): Value known

Compare=0-50 and 50-100

Choose range at which best estimate is found. For example 0-50 is closest.

Bisect this range again...
0-25 vs. 25-50

Can anyone recommend a module or something that is suited to this type of thing? I know bisect is the module of choice for binary searches, but am having trouble seeing exactly which aspects of it are suitable to my needs.

Thanks.

Awesome man, thanks for posting this. Let me play with it and bring back some feedback soon.

I think it's time for me to start using Python3

Thanks Gribouillis. That property decorator looks nice and is what I'll use if I can modify it to incporporate the following behavior.

class X(object):
   def __init__(self):
       self.a=50
       self.b=30

   @cached_property
   def c:
      '''Return c from pythagorian theorem'''
      return=sqrt(self.a**2 + self.b**2)

I'd like it to cache its value just like you code does; however, if I were to change a or b (eg:

self.a=52

I'd like the cache to be emptied and then recomputed the next time c is called. In essence, the property can check if any dependent variables have changed. If so, it updates c, otherwise it uses cached values. Basically, it's a "smart" cache.

Does behavior like this exist somehow? I know it can be implemented using properties in the enthought tool suite for example... but these are special object types whose purpose is to implement such behavior.

Thanks PyTony. I'll post the ones I tried, but first let me ask a clarifying question.

I'd like a property to implement the following behavior.

Say a property depends on two variables, a and b.

When I call the property, it returns a X b

If I call it again, it already has this value saved, so it merely returns it without recomputing a X b.

If a or b DO change and I call the property again, it will recompute the product a X b before returning it.

Some of the examples I'm seeing don't to the last part, aka recompute the value if the variable dependencies change.

I will try to implement the example you sent; however, it requires several module imports so I'll sift through all the source code there and take out the ones I need. But offhand, do you know if what I'm asking for is possible?

Hi guys,

I've downloaded 2 cached_property implementations from the internet and both did not work correctly for my programs. Does anyone here used a cached_property function that works really well and could share with me?

Also, python is going to round (9/5) to 2 since these are integers. Get in the habit of using floats: (9.0/5.0)

Install ubuntuu. It is a free oerating system, basically virus free and better than windows.

Bump on snippsat's reply...

That is almost certainly the way to do it if you ask me.

Bump tot he above poster, it is most certainly better to store variables so they may be accessed by attribute (eg movie.director)

Oh sweet.

To the OP, if you aren't able to get any help here, I expect the appengine has a mailing list and they'd be the best people to help in the scenario that no one here uses the engine.

Thanks woooee. I hesitate to use eval for the reason you pointed out. I like the idea of using len(); that's smart; however, it does restrict user input style (for example, I'd like 'False' in the field to also work.)

For my use case, the programs actually know ahead of time what field types ought to be. Therefore, I can take advantage of this since my program expects a boolean. Therefore, my solution is to do something like:

_boolean_states = {'1': True, 'yes': True, 'true': True,
                   '0': False, 'no': False, 'false': False}    

for field in expected_fields:
   if field == bool:
      entry=_boolean_states[entry.lower()]  #.lower() for case insensitivity

Stole this idea from the ConfigParser package actually.

I will keep these other methods in mind for other use cases, thanks for helping me out.

Are you sure that's something you can do out of the box with the software development kit provided by google? I know google provides a Python scripting layer for Android development, so that might be a good place to start.

Hi Irh9,

You're right that I do need to look into sqlite and hdf5 more thoroughly before I try to emulate any of their functionality. The namedtuple example page Click Here actually demonstrates that namedtuples interface well to sqlite, and this suggests to me that they still have an important role apart from sqlite, but I do need to take a break and survey these packages.

I have to put this project on the shelf for a bit unfortunately, but I appreciate you're helpful discussion and insights. I will ressurect this thread in the future and hopefully you can continue to offer these good recommendations.

Thanks.

This just doesn't sound like a good idea. If it were me, I'd specify interfaces and let other programmers work out how to fulfill those interfaces.

For instance, the sum function expects an iterable of summable objects. It doesn't care if a programmer stores his or her data in a csv file, json file, shelve, or whatever. It's up to the programmer to get the data he or she wants to use in the correct structure; he or she has to fulfill the interface.

I agree with what you're saying and am looking for the best solution to my problem; however, I'm not redefining the function for every possible data type. I'm not doing something like:

def example(input):
   if type(input) == string:
     do something 
   elif type(input) == int:
     do something else

I probably will just end up making two functions, one which modifies objects in place and one that returns new objects. This way, the immutable objects will clearly error when attempts are made to manipulate in place. Maybe I will just make "in_place" a keyword in the function call.

P.S. When I asked what your case was, I was talking about the project that you are writing this system for. I assume you are writing this to address something in another project? If you aren't, you are "solving" a "problem" that doesn't need to be solved.

This project actually stemmed from my need to handle a certain type of data from bioinformatics. I'm …

I would starting working with the Enthought Tool Suite. Their TraitsUI (and now the new Enaml framework) are worth investing the time into. They allow for really simple GUI's that are interactive from the door. In your case, it would be as simple as having a checkbox and when the user changes the value, the plot data in your chaco plot (chaco is a matplotlib wrapper for interactive plotting) would just change to whatever plot you waned. In fact, it would be easy to have all the plots on the same screen and the user can merely readjust the view to get one out of the frame for example.

Ya I would agree that the tradeoff for readability is worth it.

Thanks for sharing the snippet.

I know that HDF5 for python understands the concept of chunking and may be a good place to start if you're interested in learning more about handling large data.

Can't you do:

list_of_lists=[list(x) for x in list_of_tuples]

This should make a list of lists; however, as PyTony said, this step should not be necessary for implicit compatibility w/ numpy.

Well that just blew my mind.

Is it actually quicker to define the pali() function than it would be to just put that expression into the generator expression? EG

       max((a*b, a, b) for a in range(999, 100, -1) for b in range(a, 100000//a, -1) if str(a*b))==str(a*b)[::-1])

I ask because sometimes my list comprehensions get really crowded and I never think to do this.

PS, I'm trying to avoid the solution of using 1's and 0's in the file and doing bool(int('1')) for reasons that aren't germane. Is there another way?

Sorry for the late reply. I've replied to several above posts.

I see no need to make the deep copy. Pick an attr, save its value, try to change it, then if it changed, put back the original.

I did this because there are very few cases where attribute setting can trigger events later in the code, for example in the Traits package from the Enthought Tool Suite. I will probably remove the copying behavior and just keep this in mind as a caveat.

So you are pretty much trying to make functions and methods that behave differently depending on an argument's type or behavior? That's not a very good idea. It's very difficult to ensure that you can appropriately handle all types or behaviors in one function or method. Even if you can, you make it very difficult for yourself or other developers.

That's true. I'm working on a project to store general record types in python. For immutable records, this is just an extension of namedtuples. I'd like for the immutable and mutable (object subclass) to be very similar and have a set of functions that works nearly identically on both of these. In some cases, this is troublesome because certain operations are intrinsically dependent on mutability. Since I am trying to alleviate these headaches from user, I want to user to only learn one function and then blissfully apply it to mutable and immutable records in general. Because I don't want to limit …

Hi,

I'm having some difficulty reading in boolean values from a file, particularly false values. Python can understand false values as empty strings, as well as a few other ways. For example:

In [43]: bool(0)
Out[43]: False

In [44]: bool(False)
Out[44]: False

In [45]: bool('')
Out[45]: False

The problem is, anything I read in from a file is read in as a string. So if a file has these values, all the booleans will evaluate to true. For example:

In [46]: bool('0')
Out[46]: True

In [47]: bool('False')
Out[47]: True

Therefore, in my file, I chose to use empty strings to denote false values in my file. However, when I read these in and try to convert them, they yield TRUE! Here's an example of my data:

5450    0.0 Infinity    0.0 1.0 ''  ''

f=open('params.txt', 'r')
for line in f:
    sline=line.strip().split()
    print sline[5], type(sline[5]), bool(sline[5])

Results in:

'' <type 'str'> True

Am I going crazy, why doesn't this evaluate to False? bool('') is False....

Not sure how to do this but for handling the images, start with PIL (python image library). PyAudio is probably the goto package for handling audio in python and not sure about video.

I also should point out that if you do end up finding a way to use the python package "subprocess" the subprocess.call method knows to wait until the process is finished automoatically.

Don't use "List" as your variable name, this is a reserved python keyword. Try replacing it with something like "mylist".

Also, python's list have building count features so you can do something like:

for x in mylist:
print x, mylist.count(x)

I'm trying to write some functions that work the same way on two different objects types in my codes. If the function resets attributes or returns a new object depends on if the object is inherently mutable. In the future I may just make a set of mutable functions and a set of immutable functions if this type of behavior is not pythonic.

Hi,

I am trying to create the most general function possible to test if an object passed into it has mutability or not. With some prior help from pytony, it seemed that the best way to do this is to try to set and attribute. If the attribute can be set, it is mutable, otherwise it will force an error.

To make this function general, I need to be able to pick an attribute from the object. I thought the best way to do this would be to list the attributes, pick the first one listed and then overwrite it. It would be superfulous to try to overwrite every attribute, no?

So the steps are:

1. Take in object.
2. Make a deepcopy so as not to overwrite the object.
3. List its attributes.
4. Choose a single attribute.
5. Try running setattr on this.
6. Infer mutability based whether or not errors appear.

In regard to step 2, I"m confused on the most general way to get class attribtues in Python. From another website, the recommended way was:

>>> class new_class():
...   def __init__(self, number):
...     self.multi = int(number) * 2
...     self.str = str(number)
... 
>>> a = new_class(2)
>>> a.__dict__
{'multi': 4, 'str': '2'}
>>> a.__dict__.keys()
dict_keys(['multi', 'str'])

This seems messy and can it be guaranteed that all mutable python objects will have the dict variable? Does the presence of the dict variable automatically indicate mutability?

If your goal is to run a program from Python as in your original post, the only way I think of that working is that you call the program from a script like I mentioned. This can be done in windows if you are able to somehow call latex from the cmd line. Once you do this, you should be able to write a script in whatever language windows cmd line scripts are written in, then have python call that script as you did in the original post.

Let me ask you this... what exactly are you doing this for? I use latex and may be able to help you get around having to write a script in the first place. Secondly, latex I beleive has its own macro language so you may be able to write this script within latex itself. Last, you may want to consider downloading ubuntuu and installing it side-by-side of your windows build. Operations like what you are trying to do as well as many other situations you will likely encounter are more elegantly handled in a unix-based system, and the solutions are more likely to be scattered around the web.

I'm working on a package for the management and easy creation of mutable and immutable records. The idea is that we can either use object subclasses (mutable) or modified named tuples (immutable) as well as a bunch of very handy utilities/functions to create a record storage class in python. The page I made is really crappy and not put together yet but I will update it soon.

Source

Info

I know this API is horribly documented, and that's because sphinx generates the page automatically from the source files that have shabby doc strings. I'll update this soon to actually sound legitimate and also with more examples of input output.

I'd really like to add more functions to this package that are like the methods you added here. I made some general functions that do things like sort records by argument, filter etc... and I want to expand it. I'll try to update the pyrecords site soon to better reflect the current state of the programs, but since there's so many good code snippets here about records I REALLY hope you guys can help contribute (even if just knowledge). My email address is hughesadam87@gmail.com

Anyone interested in talking more, please let me know.

If you're using a unix-based operating system (aka not windows), you can use the subprocess module to execute shell commands. In effect, you would run latex at the command line through Python.

http://docs.python.org/library/subprocess.html

Looks like you are using windows though so I'm not quite sure. Sorry

Also, you should be careful with integer usage in this case. I would make num a float from the get go, otherwise division operations (aka 7/2) are going to be ounded to the nearest int. You can pass floats in to your function easily:

hailstone(float(i) )