'os.path.getmtime' problem.

[lrh9]

I'm having a problem with 'os.path.getmtime'.

I've been doing some research into generating '.pyc' files and I am able to retrieve the source modification time (in seconds) as recorded in the '.pyc' file.

It is merely the second set of four bytes unpacked as a long integer.

import random, sys, os, struct

#Points to the random.pyc module.
testfilepath = os.path.join(sys.prefix, 'lib', 'random' + os.extsep + 'pyc')

istream = open(testfilepath, 'rb')

#Read and discard the first four bytes (the magic pyc identifier).
istream.read(4)

#The last modification time of the source according to the file.
modtime = istream.read(4)
istream.close()

#Convert the bytes to an integer.
modtime = struct.unpack('L', modtime)[0]

print('According to the pyc file the time the source file was last modified measured in the number of seconds from the last epoch is:')

print(modtime)

print('Retrieving source modification time using os.path.getmtime.')

#Points to the random.py module.
sourcefilepath = os.path.join(sys.prefix, 'lib', 'random' + os.extsep + 'py')

print('According to os.path.getmtime the time the source file was last modified measured in the number of seconds from the last epoch is:')

print(os.path.getmtime(sourcefilepath))

Oddly for me, the pyc time is one hour behind the time returned by os.path.getmtime. I suspect it is because of wonky dst errors. I'm using Windows XP SP3. I'd appreciate it if you would execute this code and report your results as well as your os and maybe suggest general purpose solutions to this problem.

[lrh9]

Did some experimentation, and I discovered that the py_compile compile function will generate a pyc file with an internal source modification time matching the source modification time.

Importing the module or reloading it did not induce the generation of a new pyc file.

Interesting.

[lrh9]

Unfortunately, the file I generated did induce a recompile on import.

[lrh9]

Looking in the py_compile module for a solution. Much to my surprise it actually generates the "pyc" files itself instead of depending on the interpreter to handle it.

[Gribouillis]

I had to modify your code to run it on my system. This is the modified version with the result that both times are the same

import random, sys, os, struct

lib = os.path.dirname(os.__file__)
sourcefilepath = os.path.join(lib, "random%spy" % os.extsep)
testfilepath = sourcefilepath + "c"

istream = open(testfilepath, 'rb')

#Read and discard the first four bytes (the magic pyc identifier).
istream.seek(0)
istream.read(4)

#The last modification time of the source according to the file.
modtime = istream.read(4)
modtime = struct.unpack('i', modtime)[0] # 'L' would require 8 bytes (x86_64 ?)
print("Modtime extracted from random.pyc file")
print(modtime)

istream.close()

#Convert the bytes to an integer.

print('Retrieving source modification time using os.path.getmtime.')
print('According to os.path.getmtime the time the source file was last modified measured in the number of seconds from the last epoch is:')

print(os.path.getmtime(sourcefilepath))

"""
My output --->
Modtime extracted from random.pyc file
1262995199
Retrieving source modification time using os.path.getmtime.
According to os.path.getmtime the time the source file was last modified measured in the number of seconds from the last epoch is:
1262995199.0
"""

I'm running Mandriva Linux 2010 with 64 bits.

[lrh9]

Interesting changes.

You are probably right about the struct unpacking.

I was using borrowed code.

I'm on a 32 bit system, so long automatically is the largest number of bytes that can fit in that number of bits - four bytes.

However, on a 64 bit system long is eight bytes.

Nice catch.

I didn't expect the error to appear in Linux because it uses UTC time internally without automatically adjusting for DST. Windows does automatically account for DST in local time.

However, according to earlier information I think the problem isn't that the timestamp I am using is invalid, but I am actually not generating a valid pyc file. I'll need to look into py_compile for more info.

[lrh9]

OK. I managed to generate the pyc file.

I think I was writing the bytes for the timestamp incorrectly.

[Gribouillis]

Why do you need to generate pyc files ?

[lrh9]

Why do you need to generate pyc files ?

I'm just building a custom importer using abstract methods in importlib.abc and one of the methods for a importer that supports loading compiled modules is the ability to write bytecode files.

I could've used py_compile, but the write_bytecode method accepts a module name and bytecode to write so I wanted to keep it in line with the importlib.abc specification.

[Gribouillis]

I'm just building a custom importer using abstract methods in importlib.abc and one of the methods for a importer that supports loading compiled modules is the ability to write bytecode files.

I could've used py_compile, but the write_bytecode method accepts a module name and bytecode to write so I wanted to keep it in line with the importlib.abc specification.

Interesting. I recently tried to write a custom importer too. I'll post it here when the small app that goes with it is released.

[lrh9]

Cool. Here is a post with some of my older work.

Aside from some minor changes in some function definitions and the new write_bytecode method it is fairly representative of what I have.

http://www.daniweb.com/forums/thread243780.html