Rookie - Copy creating dest folders if necessary

[yeticannotski]

Dear All,

I am struggling to write my first "useful" python script, which is a backup tool.

After hours and hours of trial and error, I finally got it to create two different lists.

The first list is that of the files to be copied:

['C:\\Folder\\Foo.txt', 'C:\\Folder\\Folder1\\Foo1.txt', 'C:\\Folder\\Folder1\\Folder2\\Foo2.txt', 'C:\\FolderB\\FooB.txt', 'C:\\FolderB\\FolderB1\\FooB2.txt']

The second one consists of combined absolute destination paths of such files (in the local folder of the script in a removable media)

target_paths = ['F:\\Programming\\backup_script\\Folder\\Foo.txt', 'F:\\Programming\\backup_script\\Folder\\Folder1\\Foo1.txt', 'F:\\Programming\\backup_script\\Folder\\Folder1\\Folder2\\Foo2.txt', 'F:\\Programming\\backup_script\\FolderB\\FooB.txt', 'F:\\Programming\\backup_script\\FolderB\\FolderB1\\FooB2.txt']

The issue is that, for instance, "shutil.copy(target_files[0], target_paths[0])" will only work if the destination folder already exists. The destination folder is not created automatically. I need the original folder structure, so saving all files in the current folder is not a solution.

shutil.copytree() won't solve my problem as well because I am not copying the whole source folder, but only some specific files (there is a selection according to the time stamps of the files).

Is there any simpler way to get around this other than a painful combination of "os.path.exists" and "os.mkdir"?

Many thanks in advance for any possible help.

Yeti

P.S. - For reference, I am attaching the script. I know the sytle is terrible (vomitable) and that I should use functions and etc., but this was supposed to be only a "proof of concept" version. The problem is specifically on line 78.

[yeticannotski]

Just found out that I cant attach a ".py" file. So here it goes:

#!\usr\bin\python3
# Filename: davback001.py

import os, shutil, time, ctypes, platform

all_files = []
target_files = []
target_folders = []
target_size = 0
baseback_folder = os.getcwd()
avail_space = 0
target_paths = []


#Load time of last back-up from list of tuples (log) on removable media, if not found, time must be less than epoch

if os.path.exists('backup.log'):
    in_logfile = open("backup.log", "r") # load previous back-up log
    previous_backlog = in_logfile.read()
    in_logfile.close()
    previous_backlog = eval(previous_backlog)
    last_backtime = previous_backlog[-1][2] #from the last tuple appended to the log list extract date of last backup
else:
    last_backtime = -10000000000 #applicable to a first backup, much before epoch and (hopefully) earlier than timestamp of any file to be saved

#Load list of target folders

if os.path.exists('target_foders.lst'):
    in_folderfile = open("target_folders.lst", "r") #load list of target folders
    target_folders = in_folderfile.read()
    in_folderfile.close()
    target_folders = eval(target_folders)
else:
    print()
    print("Could not locate the file: target_folders.lst")
    print()
    sys.exit()

#Search target folders and make list of files with more recent time stamps than last back-up (target_files)

for i in target_folders:
    for path, dirs, files in os.walk(i):
        for j in files:
            all_files.append(os.path.join(path, j))

for i in all_files:
    if os.path.getmtime(i) > last_backtime:
        target_files.append(i)

#compare size of target_files to available space in removable media

for i in target_files:
    target_size += os.path.getsize(i)

if platform.system() == 'Windows': #because "os.statvfs" is UNIX only
    avail_space = ctypes.c_ulonglong(0)
    ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(baseback_folder), None, None, ctypes.pointer(avail_space))
    avail_space = avail_space.value
else:
    avail_space = os.statvfs(baseback_folder).f_bsize

if avail_space < target_size: #To be implemented someday: calculate if, with the overwriting of files, relative avail_space is enough in spite of lack of absolute space 
    print()
    print("There is apparently not enough space for completing the backup!")
    print()
    sys.exit()
else:
    continue

#Copy/overwriting all target_files to removable media

for i in target_files:
    x = os.path.splitdrive(i)[1]
    x = x.strip('\\')
    x = os.path.join(baseback_folder, os.path.normpath(x))
    target_paths.append(x) 

for i in target_files: #GOT STUCK HERE!
    shutil.copyfile(i, os.path.normpath(target_paths[target_files.index(i)]))


#Get current timedate, get back-up size, number of files, make log tuple

now = time.time()
today = time.strftime("%a, %d %b %Y %H:%M:%S +0000", time.localtime(now))

log_tuple = (len(target_files), target_size, now) 
previous_backlog.append(log_tuple)

out_logfile = open("backup.log", "w")
out_logfile.write(previous_backlog)
out_logfile.close

print()
print("Backup of",len(target_files),"files carried out on",today,".")
print()

[Gribouillis]

I suggest to call the following function to ensure the target directories exist

def make_target_dirs(target_paths):
    for dirname in set(os.path.dirname(p) for p in target_paths):
        if not os.path.isdir(dirname):
            os.makedirs(dirname)

Edit: to attach a .py file, zip it first, then attach the zipped file.

[yeticannotski]

Dear Gribouillis,

Thanks for the help. I understand then that there is no easy shorcut, but in any event the answer was "os.path.isdir()" and not "os.path.exists()" as I originally thought.

Your code was much cleaner and clever, but at this phase I am rather sticking to the things I fully understand and I don't really get the "set ... for" you used (will study that though), so I came with this clumsy reinterpretation of your suggestion:

for i in target_paths:
    x = os.path.dirname(i)
    if not os.path.isdir(x):
        os.makedirs(x)

Thanks as well for the tip on zipping ".py" files.

Yeti