Problem sending file using urllib2

[Fluffybunny]

Hi

I am sending a text file to a remote app using urllib2. It all works well, but it just seems to take a very long time to send the data.

My code is basically doing :

TxData = open('OutputFile.txt')

data = {'name': 'value',
        'file':  TxData
        }

f = urllib2.urlopen(url, data)

s = f.read(2)   # Read only 2 bytes of data as the remote app only sends
                # an OK if the file is sent successfully 

f.close()
TxData.close()

As I said, it works fine as most of the files are less than 5kb but if I try to send a
file of, say, 100k it can take 10 minutes which seems ridiculous especially as the remote machine is sitting on our own LAN (i.e. it is not sending across the internet)

Any ideas why the data is taking so long to send, or clues as to how I can find out hat is happening?

Thanks

[TrustyTony]

Why not ftp? Or basic http open?

data = open('OutputFile.txt').read()
f = urllib2.urlopen(url, data)

[Fluffybunny]

I cant use ftp because I have no control over the remote app and that is requiring me to send using http POST.

I'm a real newbie to Python (and I'm not a programmer) and it took me about a week to work out how I could do it the way I did, so I do appreciate your sugestion as this looks so much neater.

I'll try your method and let you know how it goes. Thx very much!

[TrustyTony]

Why not ftp? Or basic http open?

data = open('OutputFile.txt').read()
f = urllib2.urlopen(url, data)

I have actually downloaded data, so that code is probably completely stupid. But from documentation I see request objects used, but you have not those. For example from Stack overflow:

data = urllib.urlencode(params)
url = host+page
request = urllib2.Request(url, data, headers)
response = urllib2.urlopen(request)

[Fluffybunny]

ok... I tried that and I got this error..

Traceback (most recent call last):
File "C:\CDR_Process\URL_Send_Test.py", line 21, in <module>
f = urllib2.urlopen(url, data)
File "C:\Python26\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python26\lib\urllib2.py", line 391, in open
response = self._open(req, data)
File "C:\Python26\lib\urllib2.py", line 409, in _open
'_open', req)
File "C:\Python26\lib\urllib2.py", line 369, in _call_chain
result = func(*args)
File "C:\Python26\urllib2_file.py", line 207, in http_open
return self.do_open(httplib.HTTP, req)
File "C:\Python26\urllib2_file.py", line 298, in do_open
return self.parent.error('http', req, fp, code, msg, hdrs)
File "C:\Python26\lib\urllib2.py", line 435, in error
return self._call_chain(*args)
File "C:\Python26\lib\urllib2.py", line 369, in _call_chain
result = func(*args)
File "C:\Python26\lib\urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error

Any ideas?

[TrustyTony]

Any help from this ActiveState Recipe?

"""
multipart.py
From http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
"""

import httplib, mimetypes

def post_multipart(host, selector, fields, files):
    """
    Post fields and files to an http host as multipart/form-data.
    fields is a sequence of (name, value) elements for regular form fields.
    files is a sequence of (name, filename, value) elements for data to be uploaded as files
    Return the server's response page.
    """
    content_type, body = encode_multipart_formdata(fields, files)
    h = httplib.HTTP(host)
    h.putrequest('POST', selector)
    h.putheader('content-type', content_type)
    h.putheader('content-length', str(len(body)))
    h.endheaders()
    h.send(body)
    errcode, errmsg, headers = h.getreply()
    return h.file.read()

def encode_multipart_formdata(fields, files):
    """
    fields is a sequence of (name, value) elements for regular form fields.
    files is a sequence of (name, filename, value) elements for data to be uploaded as files
    Return (content_type, body) ready for httplib.HTTP instance
    """
    BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
    CRLF = '\r\n'
    L = []
    for (key, value) in fields:
        L.append('--' + BOUNDARY)
        L.append('Content-Disposition: form-data; name="%s"' % key)
        L.append('')
        L.append(value)
    for (key, filename, value) in files:
        L.append('--' + BOUNDARY)
        L.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
        L.append('Content-Type: %s' % get_content_type(filename))
        L.append('')
        L.append(value)
    L.append('--' + BOUNDARY + '--')
    L.append('')
    body = CRLF.join(L)
    content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
    return content_type, body

def get_content_type(filename):
    return mimetypes.guess_type(filename)[0] or 'application/octet-stream'