Namespaces, Modules & Packages

I found the wikipedia page on namespaces not very helpful, and in fact did not find a concise and clear explanation elsewhere, so hopeful this post provides a clear and concise explanation of the basics, before getting to the more advanced. Contents:

• Basics
  • what is a namespace?
  • runtime vs compile time namespaces
  • global namespaces
  • object namespaces and “.”
  • function/method ‘local’ namespaces
    • Python
    • Kotlin
• More Advanced
  • class namespaces
  • module namespaces in Python
  • namespaces in Kotlin

What is a namespace?

A namespace is simple a dictionary (or using Kotlin Terminology: a map) of names, keyed by the names containing information about each name.

Each language has a variety of namespaces (or dictionaries of names) and has rules for which namespace(s) are in use at any one time. Consider the following Python Program

foo = "hello"
bar = "global"
parm = 5

def mydef(parm):
   bar = "test"
   other = "wow"
   print(foo,bar,parm,other)

mydef(7)
print(foo,bar,parm)
print(other)

# output from run:
hello test 7 wow
hello global 5
Traceback (most recent call last):
  File "C:\Users\ian\Documents\Dev\playarounds\test.py", line 15, in <module>
    print(other)
NameError: name 'other' is not defined  

Two ‘namespaces’ or dictionaries of names are in use in this code: 1) the global namespace and 2) the local namespace. Lines 1,2,3 all add an a name entry to the global ]namespace. The ‘def’ on line 5, adds ‘mydef’ to the global namespace. When running ‘mydef’, the parmeter(s) ‘parm’ are added to the local namespace, as are any other new names as with ‘bar’ and ‘test’ in this example.

It call be seen from the output data that the print inside ‘mydef’ accesses the global ‘foo’ as there is no local ‘foo’, but the locals for ‘bar’ and ‘parm’ which effectively hide their global namesakes as the local namespace is checked before then checking the global namespace.

This is a general rule for all languages. There can be multiple namespaces available at any one time to check for names, and there is a order or checking these namespaces that results in hiding names from namespaces checked later.

Name Scope

A name is said to be ‘in scope’ if the releveant namespace is currently available.

Runtime vs Compile-time Namespaces.

For dynamic interpreted languages such as Python, resolution of names within namespaces takes place at runtime, which allows the names available within a namespace to vary during program runtime. It also means that ‘reflection’ type features can be central to the language, as is the case with Python.

In compiled static languages, resolving names within namespaces normally takes place during compile time, which means name have usually been replaced by memory addresses or address offsets by runtime. With languages such as ‘C’, names and namespace are completely unavailable at runtime, and although JVM Kotlin does provide support through ‘reflection’ API, this increases code size and has limited cross platform availability.

Global Namespaces

Global namespaces are names that are available anywhere throughout the code.

As these names are accessible anywhere throughout the code, it follows that what they referenece should be available through out the code, so generally these are the name of global elements stored in global memory.

For a simple Python program with only a single file(or module), or a Kotlin program with a single package, the concept of a sinle ‘global’ namespace with names in that namespace accessible throughout the program is an workable model, but once programs grow, reality becomes a little more complex.

In fact global namespaces are Python divided in to module namespaces and in Kotlin are divided into package namespaces, and these concepts for larger programs are discussed below.

Object Namespaces and “.”

Each object has its own namespace. With the name of the object in a currently accessible namespace, the use of “.” following an object name provides access to the namespace of that object. First, a Kotlin Example.

class Foo{
    var foovar = 1
}
class Bar{
    var barfoo = Foo()
    var barvar = barfoo.foovar
}

fun main() {
    println(Bar().barfoo)
    println(Bar().barfoo.barvar)
}

The ‘Foo’ class defines a attribute ‘foovar’ which will be in the namespace of any ‘Foo’ object. The ‘Bar’ class similarly defines a ‘barfoo’ object, and can set the value of barvar to ‘Foo()’ because ‘Foo’ is in the global namespace. This class also defines a value ‘barvar’ which is set to ‘barfoo.foovar’ despite the ‘foovar’ name having been created in the Foo namespace. This is because using ‘.’ (dot notation) on the barfoo object provides access to the namespace of the barfoo object, which is of class Foo, and thus accesses the ‘Foo’ namespace, providing access to ‘foovar’.

In ‘fun main’, as ‘barvar’ is not preseent in the global namespace or the fun main local namespace, ‘barvar’ is not directly accessible. But the same principle as above applies allowing use of the ‘Bar’ namespace by using “.” (dot) notation on the instanced ‘Bar’ object. Futher, the ‘.’ dot notation can be chain, using a Bar object to obtain access to the Bar namespace, and then the Foo object within that Bar object to obtain access to the Bar namespace.

Now in Python

>>> class Foo:
	foovar = 1

>>> class Bar:
        barfoo = Foo()
	barvar = barfoo.foovar
	
>>> barvar
Traceback (most recent call last):
  File "<pyshell#16>", line 1, in <module>
    barvar
NameError: name 'barvar' is not defined
>>> Bar().barvar
1
>>> Bar().barfoo.foovar
1

This follows the exact same logic as the Kotlin example, although some interesting other Python specific features are also possible, and they are explained in the Python specific section of the class namespaces section below.

Function/Method ‘Local()’ Namespaces

For both Kotlin and Python, each function has its own ‘local’ namespace. This namespace includes function parameters, and all variables and other names defined within the function. Local() namespaces correspond to stack memory, the names within the local namespace are available only within the code of the function/method where the names are declared.

Inner Local namespaces take priority over outer namespaces, so names in global() can become hidden by names in the local() namespace, and for a function within a function, the names of the innermost namespace can ‘shadow’ hide any use of the same name within an outer scope.

Python Local Namespaces.

As a dynamic language, the python local namespace can change during program execution and names are checked for presence in the namespace at run time, however python does pre-process the code within def statements the time the ‘def’ is executed, so checks of names within ‘def’ statements do take place. Consider the following Python code:

>>> a = 7  # define a in the global namespace
>>> def foo(parm):
     a= 7
     print(a)
     del a
     print(a)
>>> foo(2)
7
Traceback (most recent call last):
  File "<pyshell#20>", line 1, in <module>
    foo(2)
  File "<pyshell#18>", line 5, in foo
    print(a)
UnboundLocalError: local variable 'a' referenced before assignment

Note that removing ‘a’ from locals does not result in Python using the ‘globals’ namespace and using that ‘a’. The ‘def’ statement determined what will be local at the time of the executing the ‘def’, at the time of storing the code of the function, in a manner with similarities to compilation at ‘def’ time, but the code stored checks names at runtime, but to increase efficiency, what namespace to search for each name has been predetermined.

With every function having its own locals namespace, then how many namespaces will be needed for a 3 level nested function? The actual answer is one (1), because all names from outer local namspaces are copied into the a single locals() namespace that contains the combination entries required from all levels. Accessing the value defined by the ‘def’ returns an object with code, and has a dictionary of all data required for the code to be run as a closure. Consider the following example, which outputs the locals() for three nested levels:

other = 3
def foo(parm1,parm2):
    def inside():
        def inside2():
           nonlocal foo_a1
           global other
           e=15
           in_c
           print(f"inside2 {locals()} oth: {other}")
        in_b=11
        in_c =12
        parm2
        foo_a1
        print(f"inside {locals()})
        inside2()
        return inside2
    foo_a1= 7
    foo_a2 = 8
    print("fooy",locals())
    return inside()

# output
>>> res foo(1,2)
foo {'parm1': 1, 'inside': <function foo.<locals>.inside at 0x00000157BF1B0AE8>, 'foo_a2': 8, 'foo_a1': 7, 'parm2': 2}
inside {'inside2': <function foo.<locals>.inside.<locals>.inside2 at 0x00000157BF1B0B70>, 'in_b': 11, 'in_c': 12, 'foo_a1': 7, 'parm2': 2}
inside2 {'e': 15, 'foo_a1': 7, 'in_c': 12} oth: 3
>>> res()
inside2 {'e': 15, 'foo_a1': 7, 'in_c': 12} oth: 3

Note how each locals() contains all “accessed” values, from all levels enclosing the current scope, but does not contain other values(). So the ‘inside2’ locals contains in_c but not in_b from the outer ‘inside’ scope, because of the reference to ‘in_c’ in the code. Entries to locals are from:

• ‘global’ statements
• ‘nonlocal’ statements
• parameters
• names to the left of assignment, not already in present in locals

In the previous code example, the return statement of ‘inside’ function returns the value of ‘inside2’ identifier – which is the object holding the code. The return statement does not run inside2- just returns the callable function referenced. The foo function returns the result of ‘inside’, which is the callable function ‘inside2’. So the result of ‘foo’ can be called, and still prints the all the values of that were created inside the nested code. By saving all the values in ‘locals’, inside2 created a closure with access to the same identifiers, even when those indentifiers have gone out of scope.

Also consider the following code:

other = 3
e = 4
def inner2():
   if False:
      global other
      e=15
   # e
   print(f"inside2 {locals()} oth: {other}")  

Even though ‘global’ cannot be executed as it is inside ‘False’ if block, the value of ‘other’ still appears in ‘locals’. Further, if the statement ‘# e’ is uncommented, rather than accessing the global ‘e’, an error that “local variable ‘e’ has not been referenced before assignment’ will result. This means that ‘if’ or other logic plays no role in what identifiers are defined within a ‘def’. The code is scanned at def time, without the code being run, to find assignments, local and nonlocal definitions.

Kotlin Local Namespaces.

Kotlin Local Namespaces have a completely different concept of scope to Python local namespaces. While Python local namespaces are ‘def’ or function based, Kotlin namespaces are entirely ‘block’ based. If a function has no ‘block’ it has no local namespace, and if it has three ‘blocks’ then it will have three namespaces, while all Python functions have a single local namespace for the entire ‘def’/function.

Here is an example with two blocks, each with a ‘foo’ in their namespace. The outer namespace can be accessed from within, but using the same name will hide the outer name.

fun abc() = "abc" // no block so no local namespace is possible
fun main() { // now in a block can have names
    val foo = 1
    val bar = 3
    if (true){ // another block, another name space
        val foo = bar  // another foo...hides the outer foo
    }
    println("foo $foo") // shows the outer foo was not changed
}

Labels do allow getting around ‘shadowing’ or one name hiding another, but while this illustrates well how namespaces for blocks work, it makes little sense in code to have conflicting names in the first place, and even less to then use labels to resolve them, so this is not recommended for real programs.

fun main(){

    mainblock@ if(true){
        val foo = 1
        if (true) {
            val foo = mainblock@foo + 1
            println("innerfoo $foo")

        }
        println("foo $foo")
    }
}
// printout
innerfoo 2
foo 1

Class Namespaces

(brief intro only at this time, more content to follow, please ask if you need it now.)

In Kotlin, the code of a class is constrained within a block, and as with any block, the block wrapping a class defines a namespace. All methods within the block have access to the names of the the class itself. This contrasts to Python code, where methods within a class do NOT have access to a class namespace directly, an instance reference (by convention called ‘self’ to then access the object namespace.

Python Module Namespaces

(brief intro only at this time, more content to follow, please ask if you need it now.)

Each file of a python program or package is its own global namespace. For packages, the __init__ file within the package allow importing names from the modules of the package into the package namespace. See imports for more.

Kotlin Package Namespaces

(brief intro only…at this time, content to follow, please ask if you need it now.)

Every package defines its own namespace. All code compiled within a project has direct access to all globals within the same package, but other names must be imported into the package namespace in order to be accessed.