gsweene2 / garrett-learns-python Goto Github PK

I still don't reaaaallly know how garbage collection works in python, but what I do know is that generators are the language's great defence against memory explosion.

What is the difference between the following two collections?

>>> list_1 = range(1,6)
>>> list_2 = [1,2,3,4,5]

Well a good first thing might be to see what happens if we print them out

>>> list_2
[1,2,3,4,5]

Okay that was pretty obvious

>>> list_1
range(1,6)
>>> list(list_1)
[1,2,3,4,5]

WTF WHY?

range is a super common python generator you'll see but they are much more ubiquitous and useful than just looping through integers.

Generators most importantly do not hold an entire collection in memory before looping (as opposed to iterators like lists) they move one item at a time and they vanish after you've processed all the items in the collection. This is super useful for things like IO when you want to load elements one at a time, do some stuff, but not create a lot of overhead.

But before we get there we can begin with some silly examples

All we need to do to create generators is use the yield key word in place of a return statement. There is another way to initialize generators using brackets but I leave that to your googling.

#stupid example
def dumb_generator(limit=50):
  for i in range(1,limit):
    yield i

This will also illustrate how the python key word next can increment through generators one at a time.

>>> dumb_generator(limit=100)
<generator object dumb_generator at 0x000001ACDADB5CC8>
>>> for i in dumb_generator(limit=5):
...  print(i)
1
2
3
4
>>> dumb_dumb = dumb_generator(limit=5)
>>> next(dumb_dumb)
1
>>> next(dumb_dumb)
2
>>> for i in dumb_dumb:
...  print(i)
3
4
>>>

I think you get the idea

This is actually what python does by default when it opens files to be more memory efficient

with open(my_file,'r') as f:
  for line in f:
    print(line)

Our f in context manager is actually a generator

At the risk of insulting any reader's intelligence, as I am a lowly python programmer, there isn't anything difficult about loops in python, but there are some nifty time saving tricks that aren't available in other languages.

As you probably already know python is awesome at looping through collections without doing anything things like using counter variables. It can loop through pretty much any collection of objects just using an iterator or generator and the famous for i in Iterator syntax. But it gets so much better than that.

Tip 1: use enumerate

How many times have you thought to yourself wouldn't it be great if I could get the collection object AND the integer indexer?

>>> my_list = ["a","b",lambda x: print(x),1,2,3] # why you'd have a list of mixed types like this who knows
>>> for i,item in enumerate(my_list):
...  print(i,item)
0 a
1 b
2 <function <lambda> at 0x000001ACDADA64C8>
3 1
4 2
5 3

I use this all the time.

Tip 2: the wonders of zip

You probably know this but python provides a way to loop through as many objects as you want concurrently provided the collections are of equal length

>>> first_names=["garrett","john","gavin","daniel"]
>>> last_names=["sweeney","sweeney","sweeney","luftspring"]
>>> for first,last in zip(first_names,last_names):
... print(" effing ".join([first,last]))
garrett effing sweeney
john effing sweeney
gavin effing sweeney
daniel effing luftspring

You can go crazy with zip and python is so nice it lets you name the iterators inside the loop so you don't lose track of what's inside your various collections.

Tip 3: getting nothing from your loops

This is a bit of an edge case, but sometimes we want to return absolutely nothing from the collection but still loop through it. Especially when we are trying to make sense of #2

For example lets say we encounter a wild generator x but we know nothing about it. How might we discover it's length without having to brute force pull the whole thing into memory by passing it to the list() constructor? We can do it oh can we do it.

>>> x = (i for i in range(1,50)) #suspend your disbelief
>>> x
<generator object <genexpr> at 0x000001ACDAD4D348>
>>> sum([1 for _ in x])
49

I found this as a work around for some really nasty ms powerpoint python API. It was really hard to figure out table dimensions so I had to use this trick to build (x,y) dimensions in my class constructor.

Python can't be a real language because it doesn't have a switch or case statement

• some C programmer probably

Python actually does have a nifty way of handling switch style statements using dictionaries.

>>> cases = {"case_1": do_something, "case_2": do_something_else, "case_3": stop_doing_the_thing}
>>> cases.get("case_1") #you can pass params from inside a class to your dictionaries to simulate switch statements

This brings up another important and nice feature of good python- using default values when looking up values in dictionaries

Say I want to check a bunch of values but have a common way of error handling without bugging out my whole program. A very good choice is to supply a default option when you lookup a value like so...

>>> some_dictionary = {'a':1,'b':2,'c':3,'ERROR':"Not in this alphabet my friend"}
>>> some_dictionary.get('a','ERROR')
1
>>> some_dictionary.get('xref','ERROR')
"Not in this alphabet my friend"

Another lesser known thing is that you can supply lambda functions as your default lookup value.

Python also has a collection in the standard library called a DefaultDict which is perfect for people who don't like supplying the default key every time they make a call to the underlying dictionary.

Happy snake charming...