Summary of Python underscore handing

This post summarizes the handling of underscores in Python.

In Python, underscores are given before and after functions and methods, and also they are used as variables. It is a customary usage or a system constraint, but sometimes I forget and wonder what it is, so I will summarize it as a reminder.

Underscore position

Underscores are used in a variety of positions. The following is a list of positions where underscores are conventionally or systematically used.

Let’s go through them in order.

Underscore only

Underscore only is customary, and is used to assign values that are not needed (or temporary variables).

For example, In case you don’t need to access the index in a for statement like the following.

for _ in range(47):
    print('Hello, world')

And it is used as a destination for assigning unneeded values and return values.

temp_list = [0, 1, 2, 3]
a, b, _, _ = temp_list
print(a)
# 0
print(b)
# 1
print(_)
# 3

# When only the x-axis is needed in a function to get 3D coordinates
x, _, _ = get_coordinate(area)

One underscore at the beginning

There are the following patterns that are given a single underscore at the beginning.

• Variables and methods in a class: Conventional implications
• function: May be subject to systematic constraints

Adding a single underscore to the beginning of variables and methods in a class

When variables or methods in a class are prefixed with a single underscore, the customary meaning is that it is referenced and used only within the class.

In other words, it means that the variables or methods are not intended to be accessed from outside the class. However, since it is a customary meaning, access itself is possible, so it is just a form of intention (warning).

class Test:
    def __init__(self):
        # The _foo variable is not intended to be called directly from outside the class.
        self._foo = 0

    # The bar method is also intended to be called directly from outside the class.
    def bar()
        self._foo += 1
        return self._foo
    # The _baz method is not intended to be called directly from outside the class.
    def _baz()
        self._foo = 0
        return self._foo

t =  Test()
# Although we don't want it to be accessed conventionally, can be accessed from outside the class.
print(t._foo)
# 0

# It is intended to be accessed from outside the class  and can be accessed from outside the class.
print(t.bar())
# 1

# Although we don't want it to be accessed conventionally, can be accessed from outside the class.
print(t._baz())
# 0

adding a single underscore to the beginning of a function

When a function is prefixed with a single underscore, it is subject to systematic restrictions. Function that start with an underscore will not be loaded when imported with wildcards from a module that contains functions

# test_module.py

def foo():
    return 0
def _bar():
    return 1

from test_module import *

foo() # Imported from test_module
_bar() # _ is at the beginning and will not be imported

However, wildcard importing is not recommended because it makes it impossible to know what you are importing. So you may not encounter this case.

One underscore at the end

If the variable name you want to use is the same as a Python reserved keyword (class, del, etc.), you can avoid name collisions by adding a single underscore at the end. This is another customary use of the underscore.

def foo(class_):
    print(class_)

The reserved keywords for python can be found below.

import keyword
print(keyword.kwlist)

# ['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif',  'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']

Two underscore at the beginning

The target of the first two underscores is a variables or methods in a class. In the above case, name mangling will be applied to the target.

Name mangling

Name mangling means that for variables and methods with a leading __ in a class, the name is converted to "_class name" + variable name (or method name). As a result, unlike the case of one underscore at the beginning, the defined variable or method name will not be accessible from outside the class.

class Test():
    def __init__(self, name):
        self.name = name
        self._name = name
        self.__name = name

t = Test('foo')

# Accessible from outside the class
a = t.name

# Although we don't want it to be accessed conventionally, can be accessed from outside the class
b = t._name

# Cannot be accessed from outside the class (does not exist)
c = t.__name

# Names after name mangling can be accessed from outside the class.
d = t._Test__name

In other words, when name mangling is implemented, it behaves like a private variable and method, albeit pseudo. Of course, from within the class, you can access the variable by its name before the name mangling.

Notes on name mangling

Name mangling is useful, but there is one thing you have to be careful about. It is the behavior when inheriting.

Inheritance behavior of variables with name mangling applied

An error occurs when you try to access a variable with name mangling applied in the base class in the derived class, as shown here. This occurs because the string ‘Alice’ is assigned to _Parent__name in the base class, but the derived class refers to _Child__name.

class Parent():
    def __init__(self):
        self.__name = 'Alice'

# Inherit from Parent class
class Child(Parent):
    def print_name():
        print(self.__name)

c = Child()
c.print_name()
# AttributeError: 'Child' object has no attribute '_Child__name'

Inheritance behavior of methods with name mangling applied

The following code outputs the title name retrieved by the __get_title method with the print_title method. It overrides __get_title in the Child class, so it expects the method to be called after the override.

However, the __get_title method has name mangling applied, and the __get_title referenced by print(self.__get_title()) in the Parent class is actually _Parent__get_title. This is not the expected behavior.

class Parent():
    def print_title(self):
        print(self.__get_title()) # Outputting the return value of _Parent__get_title

    def __get_title(self): #  _Parent__get_title is defined
        return "Alice's Adventures in Wonderland"

# Inherit from Parent class
class Child(Parent):
    def __get_title(self): # _Child__get_title is defined
        return "Through the Looking-Glass, and What Alice Found There"

c = Child()
The following should output "Through the Looking-Glass, and What Alice Found There", but it actually outputs the Parent class "Alice's Adventures in Wonderland".
c.print_title()
# Through the Looking-Glass, and What Alice Found There

Two underscores each at the beginning and end

Two underscores each at the beginning and end, indicate a magic method for special use in python. It is not subject to name mangling.

There are __init__, __len__, and so on, but you should not define or override them without understanding their use, since they already have a defined use for the object.

The dir function will tell you the attributes of the object, so you can see what magic methods are available for each object.

foo = 1
dir(foo)
# ['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

One or more underscores along the way

Use this to make numbers easier to read, or to connect words in function or method names.

# Separate each three digits for easy viewing
value = 100_000_000
print(value, type(value))
# 100000000, <class 'int'>

#  Connect the words in the function name with _
def get_final_params():
    pass

Conclusion

In this post, I summarized the handling of underscores in Python. I often forget how to handle underscores, so I’ll be reviewing them periodically to get better at using them.