a C-like virtual-memory system for file data

[Tcll]

I've been working on this in other places so it's a little ahead of it's time,
but it still has a few minor issues... heh

here's an example to get started:

x = struct( 12, # struct size
    'x,y,z', # variable order
    x = bf32,
    y = bf32,
    z = bf32
)

# working stuff:
d = x()
d2 = x()

#dptr, d2ptr = ref( d, d2 ) # planned for future
dptr = ref( d )
d2ptr = ref( d2 )

deref( dptr, x ).set( 1.0, 1.0, 1.0 )
deref( dptr, x ).x.set( 2.0 )

# type testing (everything passes):

print 'test 1:', ['failed','passed'][type(x) == structType], '( type(x) == structType )'
print 'test 2:', ['failed','passed'][type(d) == x], '( type(d) == x )'
print 'test 3:', ['failed','passed'][type(d.x) == bf32], '( type(d.x) == bf32 )'
print 'test 4:', ['failed','passed'][type(d) == type(d2)], '( type(d) == type(d2) )'
print 'test 5:', ['failed','passed'][type(d) != type(header)], '( type(d) != type(header) )\n'

again, this is for file R/W, not RAM

anyways... the issue it hand is the variable order in structs...
I want to get rid of it, but I'm not sure how... :/

here's the code for struct():

class _STRUCT(object):
    import FILE

    def __init__(self): pass
    def __call__(self, size, order, **names):
        #print size, names

        class _SUB_STRUCT(object):
            _names = names
            _order = order #<-- TODO: get rid of this (perhapse with a hack) (we need the raw order)

            _addr = None
            _size = size

            def __init__(this, *values):
                #print this._names

                this._addr = self.FILE._current.offset # set the address of this struct

                write = bool(len(values))
                skip = this._size

                for vi,name in enumerate(this._order.split(',')):
                    this.__dict__[name] = this._names[name](values[vi] if write else '')
                    skip -= this.__dict__[name]._size

                self.FILE._current.offset+=skip

        self.FILE.DATA._TypesList.append(_SUB_STRUCT)
        return _SUB_STRUCT


global struct; struct = _STRUCT()
global structType; structType = type

you may also notice the structType isn't exactly what I want either
that was just something I had to do to get type(x) working... heh
(taken from the NoneType thing)

[Tcll]

atually I kinda lied in the code above... :P

deref( dptr, x ).set( 1.0, 1.0, 1.0 )
deref( dptr, x ).x.set( 2.0 )

I need the methods deref_set and set with the variable "set = None"

[TrustyTony]

I get only error from import FILE inside your second code, and also the bf32 in first code is undefined. But I would start from something like

from collections import OrderedDict

class Struct(object):
    def __init__(self, names, struct_type):
        self.struct = OrderedDict((name, struct_type(name)) for name in names.split())

    @property
    def struct_size(self):
        # size of elements without container overhead for skipping records
        return sum(get_size(n) for n in self.struct)

[Tcll]

FILE is a rather large code that manages the memory structures and pointer indecies for bf32 and and such.

even the class definitions for bf(4) aka bf32 are rather large in themselves...
it's quite a complex interface... heh

but at least it works :)

so anyways what's the expected input when calling struct??
and is the order called exactly as it was input??

what I'm hoping for is:

x = struct( 12, # struct size
    x = bf32,
    y = bf32,
    z = bf32
)

normally that would get ordered as y,x,z where the value for y would be the first read from the file no matter how it was ordered in your code.

with the way you're telling me, it looks like I'll need 2 lists

is there a python hack for functions that could pull this off??

keep in mind, my program's interpreter is distributed with it,
so everyone will get the hack. ;)

[TrustyTony]

I thought to call the Struct (if bf32 gets size 4 from function get_size):

x = Struct('x y z', bf32)

Assuming bf32() produces zero element of type bf32. OrderedDict keeps the order in order of keys. If you have multiple type of element in struct then you would need more than one type, one for each variable

x = Struct('x y z', int32, bf64, bf32)

and change the struct_type to *struct_type to iterate type for each member.

[Tcll]

yeh... looks interesting, but not exactly noob-friendly... heh

bf32 actually does quite a few things.
a blank call tells the class to read a value from the file-memory as well as associate the pointer address:

d = bf32()
ref( d ) >>> 0 # d._addr >>> 0

while a call with a float value allocates memory (if needed),
sets the memory to the converted value, and sets the address of that value:

d = bf32( 0.0 )

the address is for off-term stuff:

dptr = ref( d )
deref( dptr, bf32 ).set( 2.0 ) # works as expected

and I'm also getting it working with structs :)
(that code above)

deref( dptr, x ).set( 1.0, 1.0, 1.0 )

I like the name = type syntax...
I think noobs will really get to understand that,
especially when working with binary files.

from above, calling a struct actually causes the values to be read or written as the specified type:

d = x() #read
x( 1.0, 1.0, 1.0 ) #write

the size of the struct at the beginning is the memory that struct takes.
for example, look at the header struct in one of my test codes:

header = struct( 32, #allocation size
    'magic,datasize,dataoffset', #variable order

    magic = bu32, #string(4),
    datasize = bu32,
    dataoffset = bu32

    #the rest is automatically skipped

    )() #read

it has a memory footprint of 32 but only reads 12 bytes.
the string(4) part isn't built yet, so I substitued for now. :P

[Tcll]

haha
I take the no response as a:
TF2 Medic: "I have no idea!"

could this be done with a C extension?? :)

EDIT:
I'm in a mood right now because of what's currently going on in skype XD
sorry if my comment sounds a bit immature :P

[Tcll]

just found this in the related articled on another thread I've posted.

the original question on struct was solved in another thread:
https://www.daniweb.com/software-development/python/threads/488923/how-do-i-make-a-class-instance-act-as-a-class