Hi!
I don't remember where but i found this download function (geturl) that works great if i just send one url to the function.
But as you can see i want to execute geturl() for each line in url.txt witch contains url:s.
It still downloads but instead of a progressbar i just get newlines printed with "base" on it.
I don't get it, should i read the file in some other way?
def download():
for line in open('url.txt', 'r').readlines():
url = line
base = url[url.rindex('/')+1:]
geturl(url, base)
def _reporthook(numblocks, blocksize, filesize, url=None):
base = os.path.basename(url)
try:
percent = min((numblocks*blocksize*100)/filesize, 100)
except:
percent = 100
if numblocks != 0:
sys.stdout.write("\b"*70)
sys.stdout.write("%-66s%3d%%" % (base, percent))
def geturl(url, dst):
#print "get url '%s' to '%s'" % (url, dst)
if sys.stdout.isatty():
urllib.urlretrieve(url, dst,
lambda nb, bs, fs, url=url: _reporthook(nb,bs,fs,url))
sys.stdout.write('\n')
else:
urllib.urlretrieve(url, dst)
os.system("clear")PS.
The files i try to download is about 100mb and it often fails with:
urllib.ContentTooShortError: retrieval incomplete: got only 36438016 out of 112266771 bytes
Is there a better way to download files with python?