Iterators,Generators,List comprehension and Python

     Last time in my post I discussed about how you can use python in functional programming style. Python offers iterators, generators and list comprehension which serve many useful purposes when writing simple code which require production of sequences, series etc.

So, what is an iterator?

 According to Wikipedia an iterator is similar in behavior to a database cursor. It allows the user to traverse a container which contains the value, without having the liberty to perform any iterations on it.  Though the interface and semantics of a given iterator are fixed, iterators are often implemented in terms of the structures underlying a container implementation and are often tightly coupled to the container to enable the operational semantics of the iterator. 

Iterators act like pointers which reference a value and point to the next value in the list. The basic purpose of an iterator is to allow access to a user to a value without the knowledge of its container. This gives the container to hold the value in any way it wants and the iterator assumes it as a list or sequence. Python has many objects that help implement iterators like lists, dictionary, tuples, files. 

The use of iterators saves a lot of resource. In cases you only wish to get results to a certain limit, iterators save a lot of memory as they do not require to load the complete result set to memory. They help in making a cleaner code, and work fine with infinite sequences.

In the code given below is a simple iterator for a counter.

import itertools

counter = itertools.count(10)

for i in counter:

   print i

   if i == 15:

     break 

 gives - 10,11,12,13,14,15 

 Another simple example using lists.

rand = []
i = 0
for item in range(10):
        i = i + 4
        rand.append(i)
print rand

a = iter(rand)
print a.next()
print a.next()  gives the output -

[4, 8, 12, 16, 20, 24, 28, 32, 36, 40]
4
8
>>> a.next()
12
>>> a.next()
16
>>> a.next()
20

What are generators?

 A generator is nothing but a function which is able to control the iterations, stop at a point save the context and resume from where it stopped last. A generator can generate a sequence of numbers and yields the result one at a time. Since generators return one value at a time they take up less memory and behave similar to an iterator. Generators are usually evoked in a loop. A simple Fibonacci series generator code to explain the idea of generators:-

def fibo():
        a = 0
        yield a
        b = 1
        yield b
        while 1:
                c = a + b
                a, b = b, c
                yield c
d = fibo()

This code continuously produces a element in the Fibonacci series, till any limit.

 Generators can be of many types like recursive generators, generating sequences, zero crossing detectors etc. An example for a range generator is given below:-

for i in range(100000):

        i = i + 1

        print i

What is list comprehension?

    A list comprehension as defined by python.org is a concise way of creating lists. We  are to define operations that are to be applied to each element in the list. It simplifies the code, where we have to use multiple statements to get desired output to a more shorter form. Eg:-

for x in range(10):

          squares.append(x*x)

>>>squares => [1,4,9,16,25,36,49,64,81]

while this could be done in a single statement as:-

>>>squares = [x*x for x in range(10)] 

In python it is also allowed to have nested list comprehensions.

matrix = [ [1,2,3,4],

                  [5,6,7,8],

                  [9,10,11,12]

               ]

[[row[i] for row in matrix] for i in range(4)] 

List allows supports many built-in functions which allows us to use it in place of any data structure like stacks, queues, tree etc. Eg:-

>>>stack = [1,2,3]

>>>stack.append(6) 

>>>stack => [1,2,3,6]

>>>stack.append(5)

>>>stack => [1,2,3,6,5]

>>>stack.pop()

>>> stack => [1,2,3,6]

In the pop() if we do not mention a value in the list by default it removes the last one inserted, giving the idea same as in stack. Lists find most use in functional programming aspect of python. The functions like map, filter and reduce are used with ease with lists. Eg:-

>>>map(cube, range(1,11))

>>>[1,8,27,64,125,216,343,512,729,1000]

>>>reduce(add, [1,2,3,4])

>>> 10

These are a few easy techniques for generating sequences and iterations using the functions provided by iterators, generators and lists, which help make a cleaner and concise code.