python 3: os.walk issues

[Rebecca_2]

I have the following directory/file setup (this is simplified):

Ce  
+---top.txt
+---X0.0        
|     |  
|     +---Y0.0  
|     |     |
|     |     +---X0.0Y0.0Z0.0.dat
|     |     +---X0.0Y0.0Z0.05.dat   
|     +---Y0.05
|           |
|           +---X0.0Y0.05Z0.0.dat
|           +---X0.0Y0.05Z0.05.dat
+---X0.05
      |  
      +---Y0.0  
      |     |
      |     +---X0.0Y0.0Z0.0.dat
      |     +---X0.0Y0.0Z0.05.dat   
      +---Y0.05
            |
            +---X0.0Y0.05Z0.0.dat
            +---X0.0Y0.05Z0.05.dat

Within each Y directory, I need to make a 'psub' file, which contains appends a list of the .dat files to a copy of the file 'top.txt'.

I am attempting to do this using the os.walk function in python 3: however I am incurring two problems:
1. The new psub file appears in the X0.0 directory
2. Code does not list any filenames (presumably because it is not finding any) and then gives the error that it is unable to find the X0.05 directory.

Thus far, I have the following code:

import os

with open('top.txt', 'r') as reader:
    data=reader.read()
    for root, dirs, files in os.walk('.'):

        with open('psub', 'a') as writer:
            writer.write(data)
            for file in files:
                if file.endswith('.dat'):
                    print('gulp <' + names + '> ', end='', file=writer)
                    print(file.rsplit('.',1)[0], end='', file=writer)
                    print('.out', file=writer)

The resulting psub file should be:

#!/bin/bash
#MOAB -l walltime=48:00:0
#MOAB -j oe
#MOAB -N GULP-job
cd "$PBS_O_WORKDIR"
module load apps/gulp
#!/bin/bash
gulp <X0.00Y0.00Z0.00.dat> X0.00Y0.00Z0.00.out
gulp <X0.00Y0.00Z0.05.dat> X0.00Y0.00Z0.05.out

of which the first seven lines are in top.txt.

Any (simple) pointers as to where I am going wrong would be much appreciated. Cheers

[Gribouillis]

os.walk('.') means that you are traversing the current working directory with os.walk (as returned by os.getcwd()). If you run the code while in the X0.0 directory, os.walk will never see the X0.05 directory.

The current working directory does not change during the walk. To create psub in a subfolder, you must write

psub = os.path.join(root, 'psub')
with open(psub, 'a') as writer:
    ...

You can also unindent line 5 as the contents of 'top.txt' is stored in the string data.