Speeding up Python

[vegaseat]

I keep hearing that Python is about as slow as Java. Does anybody have experience with speeding things up?

[Narue]

>I keep hearing that Python is about as slow as Java.
Java can be comparable in speed to C++ or as slow as Christmas. It's not a good measure.

>Does anybody have experience with speeding things up?
Are you having problems with performance? If not, why bother? In my experience with Python, speed is rarely an issue if you actually bother to do things intelligently. Any moron can write code, but it takes work to write good code, and experience on top of that to write great code.

[Chainsaw]

Taking a clue from my managers at work, you should tell the compiler to work SMARTER, not HARDER. That will help.

Oh, gee, I guess it doesn't help employees either.

[vegaseat]

Thanks Narue and Chainsaw for your usual perspicacity.

I am just getting my feet wet with Python. If speed comes with experience, there might be hope. VPython is pretty impressive though! Just picked up info from another forum about "Swig" (?). A wrapper that allows Python to include C where speed might be of the essence.

[EAnder]

To speed up Python there are 3 things you could do that I know of:

Code Smarter: As ChainSaw pointed out. Try to simplify your programs. Don't rewrite things(there is a reason why a lot of modules are written in c, c++, and D. Not because they had to but because they are already compiled). Use threads and then reuse them. Make functions. Its the same as recycling.

Psyco(Did I spell the name right?): If you are using a 64 bit processor(I'm pretty sure its limited to a 64 bit) you can use the Psyco module and that speeds it up. I've heard its easy to use even for intermediate python programmers.

Compiling: I know python is interpreted but you can compile programs with py2exe(windows) or py2app(Mac), or PyInstaller(all i think)

[vegaseat]

To speed up Python there are 3 things you could do that I know of:

Code Smarter: As ChainSaw pointed out. Try to simplify your programs. Don't rewrite things(there is a reason why a lot of modules are written in c, c++, and D. Not because they had to but because they are already compiled). Use threads and then reuse them. Make functions. Its the same as recycling.

Psyco(Did I spell the name right?): If you are using a 64 bit processor(I'm pretty sure its limited to a 64 bit) you can use the Psyco module and that speeds it up. I've heard its easy to use even for intermediate python programmers.

Compiling: I know python is interpreted but you can compile programs with py2exe(windows) or py2app(Mac), or PyInstaller(all i think)

EAnder, thanks for your kind words. Python has proven to be quite addictive, and three and half years later I am still having fun. The Python interpreter has seen some architectual revamping and has gotten quit a bit faster over the years. Psyco compiles to 386 native code rather than virtual bytecode, and can speed things up 3 to 5 times. Narue was correct, I find that development speed is often more important then execution speed.

Programs like py2exe are actually programs that package your bytecode, the Python interpreter, and all required modules and resources into a self-extracting executable file. They are great, if you want to distribute your work to users without installed Python.

[woooee]

Pypy is the faster Python. I have never tried it though. http://codespeak.net/pypy/dist/pypy/doc/home.html

[Gribouillis]

I have some experience with speeding up python with swig. It allows you to access virtually any algorithm written in C or C++ from python. I used it for numerical analysis and also parsing. On such problems, this method is quit e efficient.

However, the most useful feature of swig is that you can wrap existing C librairies and access their functions from python. This method has been applied to many existing C librairies. Since thousands of potentially useful C libraries exist, this is a very good reason to experiment with swig, if you have a C compiler of course.

The usual warning applies: it's often difficult to predict exactly where are the botlenecks in your programs, which parts of the algorithms are going to slow down the whole program. So a good strategy can be to write a first version in python, and use profiling to determine which parts could be reimplemented in C. The golden rule is to avoid writing C code as much as possible, and speed up only key computational steps...