Download file with HTTP, custom headers, and show percentage

[ohyeah]

Hi I just started out with python..
I'm trying to write some code that will request a file with HTTP GET, be able to send custom headers to the remote host, download this file and save it locally, and show some kind of percentage downloaded which updates WHILE its downloading.

I've scavenged the web for some tuts/examps/docs and pieced them together to come up with this:

import urllib
    import urllib2

    #here's how to download a file and save it remotely with urllib
    fileUrl = 'http://videos1.showmedo.com/ShowMeDos/wxPythonBeginners5AddingEventHandlers.flv'
    urlStream = urllib.urlretrieve(fileUrl, 'sampleVideoFile.flv', hookFunc)


    #docs say:    urlretrieve(url[, filename[, reporthook[, data]]])
    #not quite sure how to use the reporthook parameter to report on progress of download
    #i gathered this up from the web:
    def _reporthook(numblocks, blocksize, filesize, url=None):
    base = os.path.basename(url)
    try:
    percent =
    min((numblocks*blocksize*100)/filesize, 100)
    except:
    percent = 100
    if numblocks != 0:
    print str(percent)+'%')


    #i know how to send custom headers with HTTP GET with urllib2
    url = 'http://www.google.com'
    headers = {
    'User-Agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)',
    'Accept' :
    'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5',
    'Accept-Language' : 'fr-fr,en-us;q=0.7,en;q=0.3',
    'Accept-Charset' : 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'
    }

    #None = GET; set values to use POST
    reqObj = urllib2.Request(url, None, headers)

    urlObj = urllib2.urlopen(reqObj)
    response = urlObj.read()
    print response

    print urlObj.geturl()
    print urlObj.info()

    urlObj.close()

So I know how to request a webpage with HTTP GET with custom headers with urllib2. I read about downloading a file and saving it localy with urllib. Also urllib.urlretrieve has a parameter for reporting on download progress.. not sure how to use this.. maybe if someone could give me a working example. And if I could get some working code to somehow combine my use of urllib and urllib2 so I can DOWNLOAD, send CUSTOM HEADER, SAVE file, and show PROGRESS all together?

-thanks for reading

[EAnder]

To simplify your problem I suggest the urlgrabber module at: http://linux.duke.edu/projects/urlgrabber/

It also is able to show the percentage downloaded. Heres an example I found: http://thejuhyd.blogspot.com/2007/04/youtube-downloader-in-python.html

You could also use EasyDialogs(if your mac its standard, or you can use a windows port: http://www.averdevelopment.com/python/EasyDialogs.html for a gui progress bar) or Tkinter, or wxPython, or you could just create a console app that clears the screen every second or so and rewrites all the data including the new percentage.