Member Avatar for revenge2

Does anyone know of any good urllib tutorials other than
& . Something which a little beginner friendly:| with good explanations.

-Thanks

  • 2 Contributors
  • 3 Replies
  • 222 Views
  • 6 Hours Discussion Span
  • Latest Post Latest Post by mn_kthompson

Dani AI

Generated

A few practical, beginner-friendly notes to go with the links and examples already posted.

As found, calling read() on a large response can exhaust memory and make the program (or machine) appear to freeze. For downloads, stream the response and write in binary to disk instead of loading the whole body into a string. ’s pointers to the stdlib docs are useful background; the examples below show patterns you can copy and run immediately.

import requests

url = 'https://example.com/largefile.zip'
with requests.get(url, stream=True, timeout=10) as r:
    r.raise_for_status()
    with open('largefile.zip', 'wb') as f:
        for chunk in r.iter_content(chunk_size=8192):
            if chunk:
                f.write(chunk)
import requests

s = requests.Session()
payload = {'username': 'you', 'password': 'secret'}
resp = s.post('https://example.com/login', data=payload, timeout=10)
resp.raise_for_status()
# subsequent requests reuse the session (cookies, headers)
dashboard = s.get('https://example.com/dashboard', timeout=10)

Quick tips: always open files in binary mode for non-text data ('wb'), set reasonable timeout values, call raise_for_status() or check status_code, and add a User-Agent header if a server rejects unknown agents. For file uploads use the files parameter of requests.post. If you must stick to the stdlib, the same principles apply: read and write in chunks, and on Python 3 use urllib.request (the old urllib2 names were reorganized). For most beginners the third-party requests library is simpler and safer for form posts, cookies, and streaming downloads.

Recommended Answers

All 3 Replies

Member Avatar for mn_kthompson

This one helped me a bit

Member Avatar for revenge2

This one helped me a bit

thanks mn_kthompson but thats a bit beyond me for the moment:) . Are there any other tutorials where it shows how to download files, fill in forms, send messages etc?.

----
My computer froze when i did this, LOL:D (was just messing about)

import urllib2
dload = urllib2.Request ('')
open = urllib2.urlopen (dload)
png = open.read()
print png
Member Avatar for mn_kthompson

Yeah, I probably shouldn't have given you that link. It is really more about parsing the html that you get rather than the intricacies of urllib.

One of the first things I would look at is this link. http://www.velocityreviews.com/forums/t326690-urllib-urllib2-what-is-the-difference-.html
It was helpful for me to understand the difference between urllib and urllib2 and what each can do. It's short, but simple.

Then check out the end of this page. http://docs.python.org/library/urllib.html It has a few examples of how to do simple stuff.

Unfortunately, my knowledge of urllib does not go beyond the simple...but I was able to make your code work. Check this out...

import urllib
pngfile = urllib.urlopen('')
filetowrite = open('ShadowBox.png','w')
filetowrite.write(pngfile.read())
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.