Member Avatar for mccarthp

Hello, I am very new to Python and have, what I think, is a rather simple question.

First, I am running Python 2.1 on Win XP. What I want to do is open a CF page after my process has run and pass the processID, that I will set, and the exit code value to it. The CF page will then insert the values to a SQL table. I have tried to just get Python to open a URL using the URLLIB but have been unsuccessful. Here is a sample what I have tried:

import urllib
url = ‘http://www.google.com’
page = urllib.urlopen(url)
pagedata = page.read()

Any help would be greatly appreciated. Thanks.

Also, if there is a way for Python to post to a SQL table directly without using an intermediate CF page I would be interested in learning how. Thanks again.

  • 5 Contributors
  • 4 Replies
  • 546 Views
  • 2 Years Discussion Span
  • Latest Post Latest Post by jrcagle

Dani AI

Generated

A quick diagnosis and two practical paths.

The failure in the original snippet is usually not urllib itself but the input: the string shown contains an HTML anchor tag and typographic (curly) quotes—Python needs a plain ASCII URL string. Replace the anchor with a bare URL (and use normal quotes), and either encode parameters into the query string or send them as form data. Also check Windows firewall/proxy settings as pointed out; a blocked process will behave like “urllib just doesn’t work.”

For calling your ColdFusion page (easy, reliable): use a modern HTTP client that handles encoding for you. The Requests library makes GET/POST with params trivial:

import requests

cf = "http://yourserver/insert.cfm"
params = {'processID': '1234', 'exit': '0'}

# GET -> http://yourserver/insert.cfm?processID=1234&exit=0
r = requests.get(cf, params=params, timeout=10)
print(r.status_code, r.text)

# or POST as form data
r = requests.post(cf, data=params, timeout=10)
print(r.status_code)

Requests takes care of percent-encoding and timeouts; install it with pip if your environment allows. (requests.readthedocs.io)

If you prefer to write directly to MySQL from Python (no CF page), use a proper DB driver and parameterized queries to avoid injection and to ensure transactions:

import mysql.connector

cnx = mysql.connector.connect(user='dbuser', password='pw', host='127.0.0.1', database='mydb')
cur = cnx.cursor()
cur.execute("INSERT INTO process_log (process_id, exit_code) VALUES (%s, %s)", (pid, exit_code))
cnx.commit()
cur.close()
cnx.close()

Do not expose your DB server to the public Internet; keep credentials on the server or use a secured API. (dev.mysql.com)

Notes on scraping/search and encoding: scraping Google HTML is fragile and against best practice—use the Google Custom Search JSON API for first-result lookup instead. If a page requires JavaScript rendering (as suggested) automate a browser (Selenium/Playwright) or COM only when necessary. When building query strings on Python 2.x, use urllib.urlencode() to form-safe strings. Also consider upgrading from Python 2.1—Python 2 is officially sunset; move to supported Python 3 releases when possible. (developers.google.com)

Recommended Answers

All 4 Replies

Member Avatar for vegaseat

Try this code, I think it's the one I posted on the "Starting Python" sticky here.

# if you are on the internet you can access the HTML code of a 
# given web site
# using the urlopen() method/function from the module urllib2

import urllib2

urlStr = 'http://www.python.org/'
try:
  fileHandle = urllib2.urlopen(urlStr)
  str1 = fileHandle.read()
  fileHandle.close()
  print '-'*50
  print 'HTML code of URL =', urlStr
  print '-'*50
except IOError:
  print 'Cannot open URL %s for reading' % urlStr
  str1 = 'error!'
  
print str1

First try to use it as is, then you can change the URL and give it another try. I know this one works and is tested with Python24 and a normal dialup connection.

Member Avatar for ryadav22

I am trying to open the page "http://www.google.co.in/search?q=test+filetype%3Adoc" using this but this is not reading the page.

I have a requirement of fetching the first URL and saving it in another file. Could you please help me in this.

My requirement is :
1. Go to google.
2. Search anything.
3. Get the URL for the first link
4. Save the URL.

Please help me in doing this.

Thanks,
Rajesh

Member Avatar for jrcagle

Also, if you are sitting behind a firewall, you might well get blocked unless you enable Python.

Jeff

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.