Software contribution and development guide

Python packaging and __init__.py

The main concept that dictates what should be included into the __init__.py files is scope. Subpackages that are outside the main scope of the package should probably not be imported into in the top level __init__.py, while Classes that are vital to the usage but defined 2-subpackages deep into the package should probably be propagated upwards to the top level.

We only enforce one rule:

Imports trough __init__.py should always only propagate upwards, i.e. no cross level imports into __init__.py files.

It is recommended to expose all the main sub-packages (that are relevant to the main scope) within a package in the top __init__.py, e.g.

from . import subpackage

If the package name is package this allows for both import styles of

import package
a = package.subpackage.AClass()

as well as

from package import subpackage
a = subpackage.AClass()

It also makes sure that all sub-packages that are supposed to function on import are loaded by import package. In testing one can still import only a subpackage or even only a module of a subpackage.

There are of course exceptions: sub-packages that are not supposed to be immediately accessible without specific import due to e.g. long initialization times, optional dependencies or exposure of low-level or internal packages. Even though these might be in the main scope, they should not be exposed in the top level.

It is also a good idea to craft an __all__ list, even though wildcard import are generally not recommended, that determines what to expose if a wildcard import is performed, e.g.

from . import subpackage
from .module import important_function

__all__ = [
    'important_function',
]

will then allow

from package import *
x = important_function()

What to expose upwards

When segmenting code in sub-packages according to classes it is a good idea for usability to import the main class of that sub-package into the top or subpackage scope, e.g. in package/things/__init__.py

from .thing import Thing

and alternatively even in package/__init__.py

from . import things
from .things import Thing  # This import is optional

so that usage can be

import package

t = package.Thing()
# OR
t = package.things.Thing()

rather than

import package.things

t = package.things.thing.Thing()

The takeaway is: find what class or function is important and propagate that upwards so that it becomes easier for the user.

Cross-subpackage imports

If there is a need to import across subpackage branches sometimes this can create circular dependancies and generally unreadable imports, such as

from ...things.thing import Thing

In python the number of . (dots) represent the number of levels, so ... in this case means 2 levels above. But rather than this type of import it is probably better to do a absolute import such as

from package.things import Thing

and live with the possibility that if the package name gets changed we will have to also change the absolute imports.