Resources on building custom importer.

[lrh9]

I was wondering if anyone knows of any resources about building a custom importer.

I've found this: http://www.doughellmann.com/PyMOTW/sys/imports.html.

I am also aware of PEP 302 and PEP 369 and imp and importlib.

Are there any other resources I might need?

[lrh9]

I've read up on PEP 302 some more, and I've looked over importlib and the abstract base classes some more.

I think I'm understanding it better.

Basically, I need an object known as a finder first. I pass the name of the module I want to import to it, and then it returns a loader for the module. The loader is responsible for actually importing the module.

I haven't written the loader yet, but here is what I came up with for the finder.

from importlib import abc
import imp

class CopyImporter(abc.Finder, abc.PyPycLoader):

    def find_module(self, fullname, path = None):
        try:
            self.module = imp.find_module(fullname, path)
        except ImportError:
            return None
        else:
            return self

    def load_module(self):
        pass

Is this right? This object acts as both the finder and the loader, so it is appropriate for the finder to return itself.

If a loader can't be found, None is returned. If imp.find_module() can't find the module, then it raises an ImportError. That's why I handled the ImportError exception.

The main thing I'm concerned about is the fact that imp.find_module() doesn't handle dotted names. It can only find one module at a time. My question is what is a normal finder supposed to do? Is it supposed to handle dotted names by itself, or is that for import statement code to handle? The main reason I'm unclear on this is fullname is used to identify the module. Is that a full, dotted name? Also, does that refer to the whole modules or just the specified part?

[lrh9]

OK. I did some reading of PEP302, and apparently an import hook's find_module method only handles the qualified module. Basically, when importing a module such as ctypes.util, by the time find_module('ctypes.util') is called, ctypes has all ready been imported.

I tried registering my importer as a hook, but it didn't do anything.

[lrh9]

I managed to use the import hook.

http://www.daniweb.com/forums/thread251208.html

[lrh9]

I've been doing some more work, and this is what I have so far.

#!/usr/bin/env python

version = '3.1.1'

READ_BYTES = 'rb'
REPLACE_BYTES = 'w+b'

import sys

import imp

from importlib import abc

from tokenize import detect_encoding

import os.path

from os import linesep as LINESEP

from string import whitespace
NEWLINE = whitespace[2]

class CopyImporter(abc.Finder, abc.PyPycLoader):

    def __init__(self):
        pass

    def find_module(self, fullname, path = None):
        return self

    def load_module(self, fullname):
        imp.acquire_lock()
        imp.release_lock()
        
    def get_data(self, path):
        try:
            istream = open(file = path,
                           mode = READ_BYTES)
            data = istream.read()
        except:
            raise IOError
        finally:
            if istream:
                istream.close()
        return data

    def get_code(self, fullname):
        source_path = self.source_path(fullname)
        if source_path:
            try:
                data = self.get_data(source_path)
                return compile(source = data.replace(LINESEP.encode(), NEWLINE.encode()),
                               filename = source_path,
                               mode = 'exec')
            except:
                raise ImportError
        return None
    
    def get_source(self, fullname):
        source_path = self.source_path(fullname)
        if source_path:
            try:
                data = self.get_data(source_path)
                source = data.decode(detect_encoding(readline = open(source_path, READ_BYTES).__next__)[0])
                return source.replace(LINESEP, NEWLINE)
            except:
                raise ImportError
        return None
    
    def is_package(self, fullname):
        pass

    def source_path(self, fullname):
        return 'C:\\Python31\\Lib\\random.py'

    def source_mtime(self, fullname):
        source_path = self.source_path(fullname)
        try:
            if source_path:
                return int(os.path.getmtime(source_path))
            return None
        except:
            raise ImportError

    def bytecode_path(self, fullname):
        pass

    def write_bytecode(self, fullname, bytecode):
        if sys.dont_write_bytecode:
            return
        try:
            ostream = open(None,
                           mode = REPLACE_BYTES)
            ostream.write(bytecode)
        except:
            return False
        finally:
            if ostream:
                ostream.close()
        return True


def main():
    pass

if __name__ == '__main__':
    main()

Now I just need to implement the find_module, load_module, is_package, source_path, bytecode_path, and write_bytecode methods.