using struct in python

[ggsr45]

hi,
I have the following dict which I want to write to a file in binary:

data = {(7, 190, 0): {0: 0, 1: 101, 2: 7, 3: 0, 4: 0}, 
        (7, 189, 0): {0: 10, 1: 132, 2: 17, 3: 20, 4: 40}}

I went ahead to use the struct module in this way:

packed=[]
for ssd, add_val in data.iteritems():
    # am trying to using 0xcafe as a marker to tell me where to grab the keys 
    pack_ssd = struct.pack('HBHB', 0xcafe, *ssd) 
    packed.append(pack_ssd)
    for add, val in data[ssd].iteritems():
        pack_add_val = struct.pack('HH', add, val)
        packed.append(pack_add_val)

The output of this is packed = ['\xfe\xca\x07\x00\xbe\x00\x00', '\x00\x00\x00\x00', '\x01\x00e\x00', '\x02\x00\x07\x00', '\x03\x00\x00\x00', '\x04\x00\x00\x00', '\xfe\xca\x07\x00\xbd\x00\x00', '\x00\x00\n\x00', '\x01\x00\x84\x00', '\x02\x00\x11\x00', '\x03\x00\x14\x00', '\x04\x00(\x00']

After which I write this as a binary file :

ifile = open('test.bin', 'wb')
for pack in packed:
    ifile.write(pack)

Here is what the binary file looks like:
'\xfe\xca\x07\x00\xbe\x00\x00\x00\x00\x00\x00\x01\x00e\x00\x02\x00\x07\x00\x03\x00\x00\x00\x04\x00\x00\x00\xfe\xca\x07\x00\xbd\x00\x00\x00\x00\n\x00\x01\x00\x84\x00\x02\x00\x11\x00\x03\x00\x14\x00\x04\x00(\x00'

It's all OK until I tried to unpack the data. Now I want to read the contents of the binary file and arrange it back to how my dict looked liked in the first place. This is how I tried to unpack it but I was always getting an error:

unpack=[]
while True:
chunk = ifile.read(log_size)
if len(chunk) == log_size:
    str = struct.unpack('HBHB', chunk)
    unpack.append(str)
    chunk = ifile.read(log1_size)
    str= struct.unpack('HH', chunk)
    unpack.append(str)

Traceback (most recent call last):
File "<interactive input>", line 7, in ?
error: unpack str size does not match format

I realize the method I tried to unpack will always run into problems, but I can't seem to find a good way in unpacking the contents of the binary file. I am trying to write a custom function for this, i know i could use pickle which will be straightfoward..Any help is much appreciated..

[woooee]

You have not declared log_size or log1_size anywhere and it appears the indentation is off under the while True, which is an infinite loop since you don't ever exit it. There is no reason from the code you posted to append to "packed" and then write each value from packed instead of just writing to the file directly. Other than that the error message is self explanatory and we of course don't know what the values of the variables are so there is no further way to help.

[ggsr45]

hi wooee,
sorry mybad here is what the log_size and log1_size are declared to
log_size = struct.calcsize('HBHB')
log1_size = struct.calcsize('HH')

[woooee]

You write one ('HBHB', 0xcafe, *ssd) and multiple ('HH', chunk) and then read one ('HBHB', 0xcafe, *ssd) and only one ('HH', chunk), then one ('HBHB', 0xcafe, *ssd) and one ('HH', chunk), etc. In addition to ('HBHB', 0xcafe, *ssd) write the number of items (length) written so you know how many to read.

[ggsr45]

oh ok, but how will i go about writing the number of items that i can read out later. U mean i can pack the length of items that i am going to write and than during the read i read that value out and used that to loop over the how many times i read my 'HH' chunk?