Hello Python community,
I've just finished building an app with Python and I'd like to share my experiences and get some feedback. This was from scratch (no code templates) all the way to compile and distribution on a Windows environment. Now all my colleagues at work are using the app with no problems.
I'll start off with some queries and questions that I had building the app, that I couldn't solve and then we'll move on to my experiences.
1. What is the standard way to put images on your toolbar buttons on other languages? Do you put your images in the install folder (i.e. the folder within the C:\Program Files\ folder) ??? or do you embed them in the code either directly or putting them into a DLL file or something. And is what I am doing right in terms of creating the toolbar buttons and images?
2. This is a related question. How do we normally package and distribute apps and their respective resource/image files? Is there something advanced here that I'm missing? Like embedding icon and toolbar images in the app/code or building DLL files and putting images in there?? Is a one-executable file possible for image-based toolbars and data resource files that are included in apps?
3. Anyone have any tips on creating a single executable file-based app? What are the pros/cons and are there any pitfalls and common errors? Is it possible to create a high level complex app with one executable file?
I'm not a software engineer by trade, so I don't understand the normal architectural procedures in terms of development and distribution.
If these questions aren't clear it's okay, it will be, just read along.
Program overview
So basically the program is a Data Retrieval app where you can get a range of data from Yahoo finance stock quotes to xml data. And then the program would do a range of stuff with that data like put it in an Excel Sheet as a CSV file and arrange it in a way that was standard for my/our needs.
Programming was pretty straightforward, as Python is such a great language with logical syntactical structure. All in all, it took me about 3 days to build the app from scratch, without much Python programming experience behind me and no standard templates. I learned a lot on the way. On my first go, this was a lot faster than say Java when I first started out. This is why I enjoy programming in Python, lots of programmer productivity for small apps.
The structure of the program is as follows:
- Standard imports
- one class - the menu bar/frame
- standard "def __init__(self,parent,id):" to initialize the frame/app.
- creation menu bar items
- creation of buttons
- creation of button binding events
- creation of toolbars
- creation of toolbar binding events
- event handlers and procedures to get data
- standard ending of the app with app.Mainloop()
So it was just one class (i.e. the frame) and then all the objects on the frame, and then the event handlers that would reference and initiate the data retrieval procedures:
import wx
import sys
import os.path
import subprocess
from urllib import urlretrieve
import zipfile
import win32com.client
class MenuBar(wx.Frame):
def __init__(self,parent,id):
wx.Frame.__init__(self,parent,id,'Data Retrieval', size=(450,250))
#... menubars, toolbars, etc ...
#... all the stuff in the middle as mentioned above
#... event handlers ...
if __name__=='__main__':
app=wx.App()
frame=MenuBar(parent=None,id=-1)
frame.Show()
app.MainLoop()My development setup is the following:
IDE: Netbeans/Jython, and Python's IDLE-GUI
GUI Toolkit: wxPython
Compile and Distribution: GUI2Exe and InnoSetup
Those were what I found to fit my needs and were very good tools from a productivity/stability/ease of use standpoint.
My recommendations and alternative setups are (and there are many) the following which I also found to be stable + high quality;
IDE: Eclipse, QTCreator, Stan's Python Editor (SPE), Komodo
GUI Toolkit: PyQT
Compile and Distribution: py2exe, NullSoft (NSIS), PyInstaller, PyFreeze. I tried NullSoft and py2exe and they both work fine. I prefer GUI2Exe over py2exe since it is GUI and more productive, however GUI2exe just uses py2exe and py2exe is a standard package for creating executables.
Problem 1:Bitmap and Image problems on wxPython on compile and creation of an executable.
The most annoying thing that I had problems with was dealing with the bitmaps, icons and images on my toolbar. They wouldn't work upon compile and execution. I knew how to reference images in Python when you were in your IDE at a testing stage, but that wasn't the issue ... the issue was referencing to images for distribution and execution of an app on an end-user's machine. A key problem with image files is when you reference them in your code you can do it three ways. Let's take creation of a toolbar with wxPython as an example;
1. by direct absolute referencing like so:
toolbar = self.CreateToolBar() #Toolbar creation.
toolbarDirectoryLocation = toolbar.AddSimpleTool(wx.NewId(), wx.Bitmap('C:\Documents and Settings\Admin\DataRetrievalApp\src\Documents-icon32.png',wx.BITMAP_TYPE_PNG), "Directory location", "Choose file directory to save to") #Toolbar object 2
toolbar.Realize()The problem with this is when you compile and package your code for execution your code it doesn't work!! It works when you are running it on your development machine with your IDE because the IDE knows where to look. However when you distribute your application to a different machine (i.e. let your work colleague use it on his computer) it won't work b/c he has saved it somewhere else.
or 2. relative referencing:
toolbar = self.CreateToolBar() #Toolbar creation.
toolbarDirectoryLocation = toolbar.AddSimpleTool(wx.NewId(), wx.Bitmap('Documents-icon32.png',wx.BITMAP_TYPE_PNG), "Directory location", "Choose file directory to save to") #Toolbar object 2
toolbar.Realize()But where will the app look to find 'Documents-icon32.png'? Once you distribute your app to the end user the standard file locations have changed.
This tiny little insignificant problem caused me all sorts of pain. I didn't know whether it was wxPython, or my code, or the compiler, so I went into a fox hole using different compilers like Nullsoft, py2exe and so on and trying all sorts of different code structures. After hitting my head on the table for about 1.5 days on this issue I finally found the solution.
Because you're trying to build your app so that it is platform independent what you have to do is write the code reference so it is independent and based around the end user. So Python must call upon the operating system and its file structure and produce a filepath that way:
homepath = os.path.abspath(os.path.dirname(sys.argv[0]))
toolbar = self.CreateToolBar() #Toolbar creation.
toolbarDirectoryLocation = toolbar.AddSimpleTool(wx.NewId(), wx.Bitmap(os.path.join(homepath,'Documents-icon32.png'),wx.BITMAP_TYPE_PNG), "Directory location", "Choose file directory to save to")
toolbar.Realize()By using os.path.join and setting homepath = os.path.abspath(os.path.dirname(sys.argv[0])), Python will recognize the file path that the end user has put the app on, and will create the absolute normalized file-path to reference to.
Other Python programmers had similar issues as I searched around on the web when exporting their code/app to different machines:
http://stackoverflow.com/questions/874625/wxpython-path-problems-when-exporting-a-bitmap
http://stackoverflow.com/questions/1270951/python-how-to-refer-to-relative-paths-of-resources-when-working-with-code-repos
And see here on the wxPython wiki FAQs: http://wiki.wxpython.org/Frequently%20Asked%20Questions#Why_do_the_paths_to_my_resource_files_work_in_testing.2C_but_fail_after_installation_on_Windows.3F
... under the title: "Why do the paths to my resource files work in testing, but fail after installation on Windows?"
That solved my first big issue.
Problem 2: Trying to create one-executable file and having similar bitmap/image file problems
The second big issue was compiling the code into one executable vs. letting the end user install a standard installation package in C:\Program Files\ <install folder here>
My preference was for one executable, so it could be as simple and quick as possible for the end user. However I had problems again, and again it was the image files on the tool bars and icons.
For some reason when you package your app in one executable InnoSetup (as well as NullSoft) either does not include the images that correspond to your toolbar buttons, or they put it in some place that the app doesn't recognize. So you get the same sort of error.
To see the error I get please see the image entitled; "wxPython error". As you can see it says "invalid toolbar button bitmap" so it's an issue with the bitmap images on the toolbar. I get this stdout message by the app ... the app runs for 2 seconds and it shuts down showing this message box quickly and then closing altogether. This is AFTER the app has been compiled and the execute file installed on a separate computer that is NOT the computer I developed the app on. i.e. the 'end user' or 'client'. This little error caused me a lot of pain. :icon_evil:
So what you had to do was to not create one executable, and instead create a package install with all the files placed into a standard C:\Program Files\<foldername here> ... sort of install, and then you place your images into the same folder as part of the install. Then the os.path.abspath(os.path.dirname(sys.argv[0]) line of code would work its magic and link up the image with the code reference - all on someone else's machine. They don't even need to install in c:\program files folder, it could be anywhere.
see the images:
- 'Package Installer" for the packaged Innosetup install technique
and
- "One Single Executable" for a single execute file app.
With the single executable you can put it anywhere, and send it around quickly without installing anything, which is quite attractive since installing packages for me, at work ... means I have to ask permission from the IT guys, and even then they might not let me install an external app, especially by an unknown publisher like me.
This is okay, and is standard, for most install type apps. But I would prefer to have just one file that is executable so that end users who are not computer saavy and especially short on time (like my work colleagues) can just get an executable through email and execute and get data quickly.
If anyone has a solution to this please let me know. I would really like to create one exe file that still holds images on it.
I searched around for this issue, and couldn't figure it out.
I read up about embedding your images into your code with Module wx.tools.img2py, but it seemed too complex for me, see here: http://www.wxpython.org/docs/api/wx.tools.img2py-module.html
If someone knows how to do this please let me know.
Also, how do other languages deal with this problem? Do they just put the images in the install folder on the end user's machine? Or do they embed it somehow?? For example see the image titled "WinRar" ... how did they get their images on the toolbars?? This is what I'm trying to do with my app, to build a similar interface.
As a result of this I had to create two versions:
i) an app with a standard installer package that would be installed in a file path defined by the end user (normally C:\Program Files\<folder_name>\
ii) and a single executable with no toolbars and no images.
See the files below:
'Appcomparison win64bit' --> how both apps look on a 64bit Windows Machine
and 'Appcomparison win32bit' --> how both apps look on a 32bit Windows Machine.
You can see clearly the single executable file-based app since it has no images and toolbar buttons/icons.
I'm still trying to figure this out, so if anyone has any idea please let me know.
Compile and Distribution
So after about 50 or so repetitive compiles and re-installs to clean out bugs and errors I got it working. I generally found that with the single executable file app, it takes longer to run ... i.e. get all the data and do all the parsing and putting it into an excel file and standardize the data. The data that is retrieved is stock data from the S&P500 - all 500 stocks.
The app with a packaged Innosetup install took about 8 seconds to complete, whilst the single executable took about 12 seconds.
I'm not sure why this is the case, but my feeling is that it takes longer to execute in a one-file executable due to the data being packed tightly into one file making it harder to run smoothly.
I found a good presentation on the web about compilation and distribution entitled "How I Distribute Python applications on Windows - py2exe & InnoSetup (#108)" from the PyCon2009 event by Brian Dorsey.
In conclusion I had a heck of a time building this small but very useful app and it has served a very productive purpose for us at work. It has been frustrating too, however not as frustrating as say, C# or Java.
I hope anyone out there who has experience in building and developing apps can figure out what I couldn't ... namely getting a one executable build and enabling toolbar buttons and their respective bitmaps/icons to work flawlessly.
I'm planning to focus on Python as my primary language, and then maybe extend it with Jython or IronPython if I need to scale up for performance reasons or for features that don't exist on Python. But Python is the core language around the stuff I build.
Hopefully I can get some answers to my questions. If you have ideas on how to help me improve my coding and development please let me know.
Cheers.
