April 8, 2017
9 min. read
This post is part of the Python Tips series.
In my current role, I don’t do much mentoring to other programmers at a language specific level. Most of my job is isolated somewhat and I’m the only serious Python programmer at the company. I can offer insight in architecture, DB design and other pieces relative to our Android, iOS, and Web applications, but not Python.
So I decided to capture some of my thoughts about what I would tell a new Python programmer, to help them progress into a successful Pythonista faster. I will be adding posts to this series, rather than a large article, so I can get them coming out as I think of them.
Python 2 vs 3
As of writing this, there are two current Python versions: 2.7 and 3.6. Python 3 is intentionally backwards incompatible with Python 2. There are many warts that existed in Python 2. Unicode was painful. Floor division was confusing. Other things were just messy. I’m still not convinced that the method used for the updates was the best, but it is where we are now.
The last version of Python 2 is 2.7. Python 2 was arguably better than Python 3 for a few years. Combined with lack of library support for some time and you get the fairly long conversion time for programmers to switch between Python 2 and 3.
The only reason to be using Python 2 these days is if your code base is large enough that conversion could not be done yet or your software conversion is waiting on vendor support for 3. There are a few industries whose Python support is lacking 3, but most are on their way. Python 3.6 is a superior version of Python than 2.7.
I do not worry about Python 2 support at all with libraries I write. It is dead. Support is ending 2020. We need to move on.
For beginners, start using Python 3.6. Just do it. There are many articles on the differences between 2 and 3, but the future is 3.
With anything that has been around as long as Python, you will find far too much historical information online. This is more frustrating when articles have no date reference to understand “freshness”. You may find tutorials that are Python 2 related. Knowing the differences between 2 and 3 allows you to do these in Python 3. Hopefully this will become less and less.
Some of the basics
Slices
Most Python programmers quickly learn slices. But I see many that don’t understand the 3 part of a slice. It is often taught just with start and end, and the step is not mentioned.
# All three arguments.
slicable[start:end:step]
# Example list
my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9]
# Start begins slice inclusively (index is included)
my_list[0:] # > [1, 2, 3, 4, 5, 6, 7, 8, 9]
my_list[5:] # > [6, 7, 8, 9]
my_list[14:] # > [] (start off end of list, makes empty list)
# End stops slice before index
my_list[1:5] # > [2, 3, 4, 5]
my_list[4:200] # > [5, 6, 7, 8, 9] (only as long as list, same as missing end)
# If start is left off, begin from 0
my_list[:5] # > [1, 2, 3, 4, 5]
# If both left off, it is a copy
my_list[:] # > [1, 2, 3, 4, 5, 6, 7, 8, 9]
# We can use step with our without start and end value.
my_list[1:5:1] # > [2, 3, 4, 5] (1 is default, so this is dumb)
my_list[1:5:-1] # > [] (we can't step backwards from 1 to 5)
my_list[5:1:-1] # > [6, 5, 4, 3] (Notice we didn't reverse [1:5:1], because start is inclusive, not end)
my_list[::-1] # > [9, 8, 7, 6, 5, 4, 3, 2, 1] (simple reverse)
my_list[::3} # > [1, 4, 7] (Step every 3)This is not just useful on lists and tuples. You can use on a string as well. Try this with many of the examples above.
'Joe Sacher'[::-1] # > 'rehcaS eoJ'Comprehensions
If you find you are building up lists, dictionaries, or sets with a loop, you are usually doing it wrong. When you understand how comprehensions work, they are just like a loop, but better performance.
A comprehension is of the format:
[output expression] [variable] [input sequences] [optional conditions]
You will see how this works as we create some.
List Comprehension
We define a list with square brackets, such as [1, 2, 3, 4, 5]. What if you wanted to make a list of the squares of 1-9? Well you could do something like:
my_list = []
for i in range(1,10):
my_list.append(i*i)
# my_list = [1, 4, 9, 16, 25, 36, 49, 64, 81]That isn’t terrible, but takes a little looking to see what you are doing. Just like creating a list, you use square brackets for a list comprehension. For everything but the simplest comprehensions, I like breaking parts onto separate lines. I think it makes them quicker to see what is going on. I’ll do that here, even though these are simple enough for a single liner.
In this first example, we have the output exprress on line 1 and the variable and input sequence on line 2.
my_list = [i*i
for i in range(1,10)]
# my_list = [1, 4, 9, 16, 25, 36, 49, 64, 81]When you format it like this, it almost reads like a loop. But the first thing you see is what is being generated, then how. Now if we want to only do this for odd values in range, we need to use the conditional capability on line 3.
my_list = [i*i
for i in range(1,10)
if i % 2 == 1]
# my_list = [1, 9, 25, 49, 81]It is possible to nest multiple comprehensions. If they are simple enough, this is fine. However, at a certain point, go ahead and loop or use multiple steps for list creation with possibly multiple comprehensions. The speed gain by being overly complicated is a waste of time if it is too hard to understand. Always remember that you will read code much more often than write it. Clever is not always a good thing. Often it is bad.
Set Comprehension
A set comprehension is the same as a list comprehension, just using curly braces instead of square braces. If you are unfamiliar with a set, this is an unordered collection distinct items. You can keep adding the same value to it, but it will only exist once. Membership check is inexpensive, and math operations common to manipulating sets are possible.
We used range in list comprehensions, but it is common to start with a list and use a comprehension to create a different one. The other reason for using a list, is that I want to show the set retaining only one copy of distinct items.
my_set = {i for i in [1,2,4,5,3,5,3,2,4,5,1,2,3]}
# my_set = {1, 2, 3, 4, 5} (note order might not be same)This is a little dumb, because we would have the same thing by wrapping the list in a set() call. So lets generate a set of the square of members of a list less than 5.
my_set = {i*i
for i in [1,2,4,5,3,5,3,2,4,5,1,2,3]
if i < 5}
# my_set = {16, 1, 4, 5} (see what I said about order?)It was just coincidence that the first came out order for me in Python 3.6.
Dictionary Comprehension
A dict comprehension is a little more complex. It uses curly braces like a set, but the output expression in a key: value style is what determines between the two. Instead of just having one value, you can modify and process both the key and value. Variable assignment can use tuple unpacking in any comprehension. It is most likely used here.
If we want to take an existing dictionary and change it somehow, then we would get at both the key and value with dict.items(). Say we want to take a dictionary and square the value.
d = {'a': 1, 'b': 4, 'c': 2, 'd': 7}
new_d = {key: value*value
for (key, value) in d.items()}
# new_d = {'a': 1, 'c': 4, 'b': 16, 'd': 49}Conditional Output
So far we have processed the full input or filtered with a condition before processing the value. I over simplified comprehensions for the output. It is possible to include logic about what the output should be in the comprehension.
Lets do another arbitrary but simple example. For some reason we want to take a list and output a 0 if the value is less than 5 otherwise a 1.
my_list = [1, 6, 10, 3, 8, 2, 7, 20000, 3]
filtered = [0 if i < 5 else 1
for i in my_list]
# filtered = [0, 1, 1, 0, 1, 0, 1, 1, 0]enumerate
Coming from some languages, it can take a new Python programmer time to think about consuming items in a for loop. Python’s for works more like a for each in some languages. Even when it is doing a loop with a counter, it is consuming a generator of range(). This can take some adjustment to write loops simply and Pythonic.
If you find yourself writing code that looks anything like this:
i = 0
for name in names:
print("Name #{} is {}".format(i, name)
i += 1Or like this:
for i in range(len(names)):
print("Name #{} is {}".format(i, names[i]))Stop. You are working far too hard and making those reading your code work hard trying to figure out what is going on. You are adding code where you can introduce errors for no good reason. Enumerate is used to maintain a counter for you.
for (i, name) in enumerate(names):
print("Name #{} is {}".format(i, name)This is much easier to read and immediately know what is going on. There is no forgetting to increment a counter or get the range wrong.
Final Ideas
I’m getting pretty long on this post, so I’ll wrap this up with one thought:
Simplify.
When you start to write code, it may be 20 lines to get something done. As you start to understand how Python best practices work, you will eliminate that loop with a comprehension or enumeration. You will use .get for a dictionary, so you can specify a value if the item doesn’t exist. This makes your code shorter, better, and harder to get wrong.
Good software engineering is about making things as simple as possible, but no more. Work on this as you write and refactor your code. If you have to write 4 times as many comments as code to explain what you are doing, you are being too clever. Stop it.
Hopefully this was helpful to you. I will be posting shorter Python specific tips in this series, with one or two subjects each.
Part 1 of 9 in the Python Tips series.