Member Avatar for fatalaccidents

Hey guys,

I had a quick question. Let me try to explain what I'm trying to do here. I'm a bit new to numpy and I'm writing a parallelized code. It is working fine without the use of NumPy, but I'm trying to learn by getting it to work both ways (and I've heard that mpi4py is faster with NumPy as well).

So I have a variable amount of numpy column stacks, which as you can see in the code... I want to make vstacks out of all of them in some manner where I don't lose any data.

for i in range(len(local_files)):
    f1 = h5py.File(local_files[i], 'r')
    data = f1['Data']
    temp = data['Temperature']
    temp_day = numpy.array(temp[day_to_extract])
    press_head = data['Pressure_Head']
    press_day = numpy.array(press_head[day_to_extract])
    geo = f1['Geometry']
    nodes = geo['orig_node_number']
    orig_nodes = numpy.array(nodes['0'])

    local_hot_list = numpy.column_stack((orig_nodes,press_day,temp_day))

    print 'File ' + local_files[i] + ' read by proc', rank

Meaning, if there are 5 files in local_files, I would like to have all 5 local_hot_list's go into one stack like complete_hot_lists = numpy.vstack(h_list1, h_list2, h_list3, h_list4, h_list5) I can't think of an elegant way of doing this. So far I have been putting the local_hot_list's into a list, but I still can't figure out a good way of getting them all together to vstack them. Any help would be greatly appreciated! Thanks for your time.

  • 2 Contributors
  • 2 Replies
  • 362 Views
  • 3 Hours Discussion Span
  • Latest Post Latest Post by fatalaccidents
Member Avatar for Gribouillis

You mean perhaps something like

def local_hot_list(filename, day_to_extract):
    f1 = h5py.File(filename, 'r')
    data = f1['Data']
    temp = data['Temperature']
    temp_day = numpy.array(temp[day_to_extract])
    press_head = data['Pressure_Head']
    press_day = numpy.array(press_head[day_to_extract])
    geo = f1['Geometry']
    nodes = geo['orig_node_number']
    orig_nodes = numpy.array(nodes['0'])
    return numpy.column_stack((orig_nodes,press_day,temp_day))

complete_hot_list = numpy.vstack(tuple(local_hot_list(name, day_to_extract) for name in local_files))

?

Edited by Gribouillis
Member Avatar for fatalaccidents

Brilliant and very elegant. Thank you very much for your help. I've never seen an example that used a function call like that, but it's very pythonic.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.