Unique Walker with os.walk

[jimmy19]

I am trying to create a walker that goes through directories. Here are the inputs and outputs which I have partly working. I am using a test directory but I would like this to be done on any directory which is leading to some problems.

[IN]: print testdir  #name of the directory
[OUT]: ['j','k','l']  #directories under testdir

[IN]: print testdir.j
[OUT]: ['m','n']  # Files under testdir.j

Here is the code so far:

class directory_lister:
    """Lists directories under root"""
    def __init__(self,path):
        self.path = path
        self.ex = []
        for item in os.listdir(path):
            self.ex.append(item)
    def __repr__(self):
        return repr(self.ex)

This returns the directories and files but I have to manually assign the names of the directories.

testdir = directory_lister(path/to/testdir)
j = directory_lister(path/to/j)
etc

Is there a way to automate instances such that:

for root,dirs,files in os.walk(/path/to/testdir/):
    for x in dirs:
        x = directory_lister(root) #I want j = directory_lister(path/to/j), k = directory_lister(path/to/k) and l = directory_lister(path/to/l) here.

Can there be a:

class directory_lister:
    def __init__(self,path):
        self.path = path
        self.j = directory_lister(path + os.sep + j) # how to automate this attribute

The code above is wrong as the object x only becomes an instance but j,k,l have to be defined manually. Do I have to use another class or a dictionary with getattr but I always run into the same problem. If any extra information is required please ask, I hope I made this clear.

For Bonus points.....

Is there a way to add other complex functions, so when it gets to a file say testdir/j/p, it prints out the first line of file p.

[IN] print testdir.j.p
[OUT] 'First Line of p'

I have made a class for printing out the first line of the file:

class File:
    def __init__(self, path):
        """Read the first line in desired path"""
        self.path = path
        f = open(path, 'r')
        self.first_line = f.readline()
        f.close()

    def __repr__(self):
        """Display the first line"""
        return self.first_line

Just need to know how to incorporate it in the class. Thank you and sorry for asking so much...

[TrustyTony]

why you should use object hierarchy? Tell the reason for the code. The form looks fitting for recursive algorithm if you realy must do it this way. The names are not unique look for example doc directories or readme files.

[jimmy19]

Thank you for replying. I am trying to get a better understanding of python, OOP and data structures. It is somewhat of a self made challenge I set myself. I wanted to see how to use __getattr__ and assign dynamic attributes in this scenario.

[TrustyTony]

Maybe you could take some inspiration of my file search?
www.daniweb.com/software-development/python/code/316585

[TrustyTony]

Here one short interactive experiment:

>>> import os
>>> class directory_lister(list):
	def __init__(self, path):
		self.path = os.path.realpath(path)
		if os.path.isdir(path):
			for directory in os.listdir(path):
				if os.path.isdir(directory):
					self.append(directory_lister(os.path.join(path, directory)))
	def __repr__(self):
		return 'directory_lister(%r) %r' % (self.path, list(self))

	
>>> d = directory_lister(os.curdir)
>>> d
directory_lister('G:\\test') [directory_lister('G:\\test\\bck') [], directory_lister('G:\\test\\build') [], directory_lister('G:\\test\\Errors') [], directory_lister('G:\\test\\games_tiedostot') [], directory_lister('G:\\test\\gomoku') [], directory_lister('G:\\test\\GUI2Exe') [], directory_lister('G:\\test\\mancala') [], directory_lister('G:\\test\\Mastermind') [], directory_lister('G:\\test\\mypackage') [], directory_lister('G:\\test\\rename_digits') [], directory_lister('G:\\test\\test') [], directory_lister('G:\\test\\XorCrypting_SpeedTests') [], directory_lister('G:\\test\\__pycache__') []]
>>>

[jimmy19]

Thank you, this was very helpful but is it possible to make the it go one level down. As in I wanted to view the files in say G:\\\\test\\build\\, which is the problem area for me, do I have to generate it as an attribute, as in do build, Errors... etc become attributes using the __setattr__ overloader but which gives the problem of going down a second level...

[TrustyTony]

I do not understand it goes down until only files remain. List is it's parent so you could for example put in list filename, first line tuples in case of files, now it does nothing for files.

[jimmy19]

What I meant was that if you have a directory tree like

testdir/
      a/
       j
      b/
       d/
        p
       m
      c/
       k

m,k and j are files. With the code above I can get to a,b,c (level 1) but how do I get to testdir/b/d/ (level 2) using the same directory_lister class. I can do a dynamic attribute for a,b,c (__setattr__) but how do I do a dynamic attribute for level 2 to list p.

I hope I made this clear.

[TrustyTony]

The directory was without path. Here the class compared with os.walk

import os

class DirectoryLister(list):
    def __init__(self, path):
        self.path = os.path.realpath(path)
        #print self.path
        for fileobject in os.listdir(path):
            fileobject = os.path.join(self.path, fileobject)
            self.append(DirectoryLister(os.path.join(path, fileobject)) if os.path.isdir(fileobject) else fileobject)
            
    def __repr__(self):
        return 'DirectoryLister(%r) %r' % (self.path, list(self))

print DirectoryLister('testdir')
print list(os.walk(os.path.realpath('testdir')))

"""Output:
DirectoryLister('G:\\test\\testdir') [DirectoryLister('G:\\test\\testdir\\a') ['G:\\test\\testdir\\a\\j'], DirectoryLister('G:\\test\\testdir\\b') [DirectoryLister('G:\\test\\testdir\\b\\d') ['G:\\test\\testdir\\b\\d\\p'], 'G:\\test\\testdir\\b\\m'], DirectoryLister('G:\\test\\testdir\\c') ['G:\\test\\testdir\\c\\k']]
[('G:\\test\\testdir', ['a', 'b', 'c'], []), ('G:\\test\\testdir\\a', [], ['j']), ('G:\\test\\testdir\\b', ['d'], ['m']), ('G:\\test\\testdir\\b\\d', [], ['p']), ('G:\\test\\testdir\\c', [], ['k'])]
"""

However os.path.walk should be able to do most things you like with the callback function without need to reinvent the wheel.