Unpacking & Arguments in Python

Photo by Kevin Ku on Unsplash

Python has clean and concise coding style, sometimes we call it ‘pythonic’. Among those pythonic coding, unpacking is a powerful and extremely common technique which can be seen everywhere in Python, sometimes you have not even noticed. In this article, we will discuss several topics to cover the usage of unpacking, and then look into how we pass arguments to a function in Python.

Basic Prerequisites

Before we enter the topic, here are some concepts we need to be very clear with. If you are comfortable with those concepts, please jump into the next section directly.

Iterable:

The targets of unpacking are iterable objects. An interable refers to an object can be iterated (e.g. use in a for loop), including List, Tuple, String, Set, and Dictionary in Python.

Essentially, custom classes with implementation of __iter__() and __next__()methods are also iterable objects.

Assignment in Python:

Assigning a value to a variable, essentially is creating a variable pointing to the object stored in memory, or, the memory address. Thus in Python a variable is neither passed by value, nor by reference, to be accurate, is passed by object, which differs in mechanism depending on object mutability.

LHS / RHS:

Literally means left-hand side and right-hand side of an expression. In the context of value assignment, we assign the value from RHS to variables on the LHS.

Mutability:

Each object stored in memory has a datatype, and changing the data inside an object means modifying its internal state.

The mutable, including list, set, dictionary, literally means datatype whose internal state can be changed. This means changing a mutable object will be conducted in-place, the variable is still pointing to the same object.

On the other hand, immutable includes all types of numbers (Integer, Float, etc), Tuple, String, Frozen Set and other special objects such as None , bool . The internal state of an immutable object cannot be changed, which means changing an immutable object (e.g. concatenate two strings, += a number) will create a new object, then let the variable point to the new object.

Tuple:

Tuples are iterable, have order, very similar with lists, while they are immutable, which means the elements inside a tuple cannot be deleted, inserted, or replaced. Typically we use parenthesis to create a tuple, while in many cases we may use a tuple without parenthesis (will see some examples in the next section).

Parameter and Arguments:

Parameter refers to the abstract variable when defining the function, you can assign default value to those parameters. While an argument refers to actual and concrete value we passed in when calling the function, or could be considered as an instance of the parameter.

Alright, time to jump into unpacking first.

Variable Assignment

As discussed above, unpacking can only be applied to an iterable object, which will split packed values into individual elements. First let us see some examples in the case of variable assignment.

Notes:

• Parallel assignment is a good example of unpacking, essentially it is unpacking a tuple.
• A very pythonic technique on swapping value, is essentially an application of unpacking, notice the evaluation order.
• The unpacking can be applied to a nested iterable object as well.
• As sets are unordered, unpacking from sets will lead to inconsistent results (however as far as I tested on 3.7, 3.8 and 3.9, the unpack assignment returns consistent order).
• Dictionaries did not have order, while from Python 3.7, the order will be kept in standard dictionaries. Thus it is also possible to unpack a dictionary.

Asterisk for Unpacking

Hope those examples warmed you up. There is another very common use on unpacking by making use of asterisk operators, either single * or double **. They have similar functions, both collect or unpack elements in an iterable, while double asterisk is specific for key value pairs, or say a dictionary.

Let us see some examples of LHS unpacking first:

Notes:

• The slicing cannot be used in set and dictionary.
• Double asterisk cannot be used in LHS.
• You can only have one asterisk in case of LHS unpacking.
• Usage in LHS is collecting elements.

Let us compare with RHS usage:

Notes:

• Both single and double asterisk can be used in RHS.
• Functionally equivalent to merging lists and dictionaries.
• Items with same key in multiple dicts, the later one will override the value.
• Usage in RHS is unpacking elements.

Unpacking Function Arguments

Now we have seen couples of examples. I believe you already have a grasp on unpacking. In this section we will focus on another important usage, that is unpacking the function arguments.

In Python we have 2 different ways to pass arguments, pass by position or by keyword/name. To be clear, here we are discussing 2 different ways of passing arguments, NOT types of arguments. We will discuss further about types in the next section.

We use single or double asterisk to collect arguments in a function, and the arguments tuple or dictionary can be unpacked inside the function scope (recall the LHS/RHS usage above). Let us see some examples:

Notes:

• Single * to collect arbitrary number of positional arguments.
• Double ** to collect arbitrary number of keyword arguments.
• A positional argument can be either passed by position or by name/keyword(check next section for detail discussion).
• In case of passing by name, position/order does not matter anymore.
• As a general form: positional -> collect positional -> keyword-only -> collect keyword-only

However, there are several constraints when using them together which we need to be careful, let us look at some more examples:

Notes:

• * without any parameter name means stop accepting positional argument, only named/keyword arguments can follow behind.
• As *args will collect all(or rest) positional arguments, only named/keyword arguments can follow behind *args.
• Similarly, as double **kwargs collect all/rest keyword arguments, no arguments are allowed after **kwargs.
• No matter there is * or not, if you passed an argument by keyword, the following arguments must also be passed with keywords.
• Default value can be both assigned to positional and keyword arguments, be aware that a default value does NOT determine if the argument must be passed by keyword or not. Do not be distracted by the default value.

Now as a summary, compare two ways of passing arguments:

Positional:

• May or may not have default values (in common)
• Use *args to collect and exhaust remaining positional arguments
• Arguments collected by *args can be unpacked inside the function
• Use single * to indicate the end of positional arguments
• Positional arguments are not allowed after * or *args

Keyword:

• May or may not have default values (in common)
• Placed AFTER positional arguments have been exhausted, either after a * or *args
• Only keyword arguments are allowed after a keyword argument
• Use **kwargs to collect and exhaust remaining keyword arguments
• Arguments collected by **kwargs can be unpacked inside the function
• No arguments are allowed after **kwargs

Last Piece: Inspect Argument Types

So far we have discussed about 2 different ways of passing arguments to a function. As you may guess, if we think about the argument TYPES in a logical way, there should be 3 different types of arguments based on those constraints we discussed above:

1. Positional-or-Keyword arguments: can be passed in both ways
2. Keyword-only argument: can only passed by keyword
3. Positional-only argument: can be passed only by position, emm… are we missing something?

You may say, well that looks reasonable, but how could it be positional-only? Does that even exist?

So, let us dive even deeper on this topic. Now we want to check how Python defines the types of parameters in a function, there is a built-in module inspect can help us.

Notes:

• A Signature object represents the call signature of a function and its return annotation. For each parameter accepted by the function it stores a Parameter object in its parameters collection.
• The Parameter object has a kind attribute which tells the type of the arguments, or how can we pass arguments
• Positional-only arguments, DOES exist! Similar with the placeholder of *, we use / to represent that arguments before the slash can only be passed by position, while after slash it could be either positional or keyword.
• In Python, it defines 5 types of parameters, or say how argument values are bound to the parameter: POSITIONAL_ONLY, POSITIONAL_OR_KEYWORD, VAR_POSITIONAL, KEYWORD_ONLY, VAR_KEYWORD.

Now we can say: in Python we have 2 different WAYS to pass an argument, and there are 3 TYPES of arguments depending on how we can pass the argument.

Bonus: Pitfall of Default Value Assignment

Assigning a default value to a parameter will be helpful sometime, while there is a pitfall we need to care about.

A default argument will be assign when the function is created by the def keyword, which means as long as you do not reassign the default value, no matter where and how many times the function is called, the default argument value will not change.

Let us see an example:

Imaging when we need to set a logger to log some message and the datetime, if we assign the default value as current time in the parameter, the assignment will be only executed once in this program lifetime, which will lead to same datetime of different message.

A common solution is to set default as None, and check inside the function, which guarantees the variable will be reassigned every time when the function is invoked.

Similar pitfall may occur when we assign a mutable object as default, as when we make changes to the mutable object, the changed state will persist, every time when the function is invoked, the variable of the mutable object is pointing to the same object in memory.

However, we may also make use of this characteristic of mutable, think about this example:

Notes:

• Use cache dict to store values (check Dynamic Programming if you feel not familiar with this).
• As the dictionary type is mutable, changes in cache will be saved.
• Only when we hit a new value not existing in cache, will run the print function.

Summary

In this article we looked into the unpacking and assignment in Python, from some daily techniques such as parallel assignment, swapping value, to the complex function arguments unpacking, with some bonus on default value setting in function arguments.

As we discussed, there are several different applications of unpacking in python, essentially they are doing the same thing, that is either collect individual elements into a tuple or dictionary, or split packed values of an interable into individual elements.

The function arguments unpacking has several constrains and may look confusing, in the end I hope this article helps you to clarify the ways of passing arguments and types of arguments, and how they related to each other based on all kinds of constraints.

Happy coding! Cheers!