Python package namespaces

Namespaces are one honking great idea -- let's do more of those!
- Tim Peters, PEP20

We all know (or should know) how namespaces work in Python. You stick a module in a particular path and you can access it as path.to.my.module (as long as path is visible to Python using a number of methods). That's not what "namespace" in this post is referring to.

Here, I'm referring to is separating out a number of sub-modules under a common root namespace. This can be incredibly useful to disparate, but somehow related modules. Let's say I had the following directory setup in a project I've been working on:

foo/
    __init__.py
    bar/
        __init__.py
    baz/
        __init__.py

After a whole bunch of work, I found that I'd like to tease bar and baz out into their own base modules, giving users the ability to depend on only the modules that they care about rather than having to navigate the entire kitchen sink.

Now, this can all be done easily by following ugly conventions such as using the names foo_bar, foo_baz or other such name mangling, but it's not the effect I want. Aside from being ugly, it would force everyone dependent on the package upgrading to the new style to change their imports. As such, it's not a viable option.

We obviously can't just name a package foo.bar with the internal directory structure foo/bar/ and have another package foo.baz with the internal directory structure foo/baz. Surely, that will introduce an import collision..

Or will it?

That's where namespace_packages (from setuptools) comes into play. By using this feature, you can define a root namespace for each package to use. With the above example, if I simply set the namespace_package to "foo", then each package with the root foo will not collide with one another.

There are two important things to remember when using this:

• Nothing else but an init.py and other submodule directories can exist in the namespace directory
• The init.py should contain __import__('pkg_resources').declare_namespace(__name__) (required as of 0.7). It must not contain anything else.

http://www.python.org/dev/peps/pep-0382/
http://pythonhosted.org/distribute/setuptools.html#namespace-packages