––– Day 7 –––
––– Generator –––
So, the term generator is used to refer to two different (but related) things:
• A generator function is a function which has a `yield` statement instead of `return`
• A generator iterator is the object created by a generator function
/1
––– Generator –––
So, the term generator is used to refer to two different (but related) things:
• A generator function is a function which has a `yield` statement instead of `return`
• A generator iterator is the object created by a generator function
/1
You can have several generator iterators created from the same generator function
But, before talking about generator functions, let’s look at another way you can create a generator object
/2
But, before talking about generator functions, let’s look at another way you can create a generator object
/2
Let’s start with a list comprehension:
>>> numbers = [number * 2 for number in range(10)]
>>> numbers
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
>>> type(numbers)
<class 'list'>
/3
>>> numbers = [number * 2 for number in range(10)]
>>> numbers
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
>>> type(numbers)
<class 'list'>
/3
![Let’s start with a list comprehension:
>>> numbers = [number * 2 for number in range(10)]...](https://pbs.twimg.com/media/Fpga8MXaUAITX8h.jpg)
This creates a list (which is a sequence, and therefore a collection, and therefore an iterable, and also a container!)
/4
/4
But, if you replace the square brackets with parentheses ( ), you don’t get a tuple. Instead you get a generator object:
>>> numbers = (number * 2 for number in range(10))
>>> numbers
<generator object <genexpr> at 0x106eb60c0>
>>> type(numbers)
<class 'generator'>
/5
>>> numbers = (number * 2 for number in range(10))
>>> numbers
<generator object <genexpr> at 0x106eb60c0>
>>> type(numbers)
<class 'generator'>
/5
A generator* is an iterator
Therefore, it doesn’t hold any of its own data. Instead, it generates the data when needed, one item at a time
(* I'll use the term 'generator' to refer to a 'generator iterator' from now on, unless I specifically write 'generator function')
/6
Therefore, it doesn’t hold any of its own data. Instead, it generates the data when needed, one item at a time
(* I'll use the term 'generator' to refer to a 'generator iterator' from now on, unless I specifically write 'generator function')
/6
Since generators are iterators, you can access the next item in a generator using the built-in function `next()`:
/7
/7
>>> next(numbers)
0
>>> next(numbers)
2
>>> next(numbers)
4
>>> next(numbers)
6
>>> next(numbers)
8
>>> next(numbers)
10
>>> next(numbers)
12
>>> next(numbers)
14
>>> next(numbers)
16
>>> next(numbers)
18
>>> next(numbers)
Traceback (most recent call last):
...
StopIteration
/8
0
>>> next(numbers)
2
>>> next(numbers)
4
>>> next(numbers)
6
>>> next(numbers)
8
>>> next(numbers)
10
>>> next(numbers)
12
>>> next(numbers)
14
>>> next(numbers)
16
>>> next(numbers)
18
>>> next(numbers)
Traceback (most recent call last):
...
StopIteration
/8
Each value was generated when it was needed using the generator expression you wrote when you created `numbers`
**The values are not stored in memory**
Once a value is created, the generator moves on to the next value without holding on to the previous one
/9
**The values are not stored in memory**
Once a value is created, the generator moves on to the next value without holding on to the previous one
/9
Once a generator is exhausted, it can no longer generate any values
Generators, like iterators (since they _are_ iterators), are disposable
You can only use them once!
/10
Generators, like iterators (since they _are_ iterators), are disposable
You can only use them once!
/10
You can recreate a generator by defining a generator function:
>>> def create_numbers():
... for number in range(10):
... yield number * 2
...
>>> numbers = create_numbers()
>>> numbers
<generator object create_numbers at 0x1056f85f0>
/11
>>> def create_numbers():
... for number in range(10):
... yield number * 2
...
>>> numbers = create_numbers()
>>> numbers
<generator object create_numbers at 0x1056f85f0>
/11
When you call the generator function, you create a generator object similar to the one you created with the comprehension earlier
Unlike standard functions, when you call a generator function to create a generator object, the function is in a _paused state_
/12
Unlike standard functions, when you call a generator function to create a generator object, the function is in a _paused state_
/12
Each time you use `next()`, the code within the function is executed until you reach the `yield` statement
When an item is yielded, the function pauses again
Each time `next()` is called, the function resumes until it reaches a `yield` statement again
/13
When an item is yielded, the function pauses again
Each time `next()` is called, the function resumes until it reaches a `yield` statement again
/13
Once the function reaches its end, a `StopIteration` exception is raised
You can use `next()` to access the first few items using this generator object:
>>> next(numbers)
0
>>> next(numbers)
2
>>> next(numbers)
4
/14
You can use `next()` to access the first few items using this generator object:
>>> next(numbers)
0
>>> next(numbers)
2
>>> next(numbers)
4
/14
The first time you called `next(numbers)`, the function started the `for` loop and yielded the first value which is `0 * 2`
The function paused at this stage
The second call of `next(numbers)` resumed the function
/15
The function paused at this stage
The second call of `next(numbers)` resumed the function
/15
As the function had paused within a `for` loop, this resumed by moving to the second iteration of the `for` loop and yielded `1 * 2`
In the code above, you stop at the third value, which is `2 * 2`
/16
In the code above, you stop at the third value, which is `2 * 2`
/16
The generator is now paused at the end of the third iteration in the `for` loop
Generator objects are also iterable. So you can use a generator in a `for` loop
Let’s try to loop through the generator `numbers` in the same REPL/Console session as the code above:
/17
Generator objects are also iterable. So you can use a generator in a `for` loop
Let’s try to loop through the generator `numbers` in the same REPL/Console session as the code above:
/17
>>> for item in numbers:
... print(item)
...
6
8
10
12
14
16
18
/18
... print(item)
...
6
8
10
12
14
16
18
/18
It works
You can loop through a generator object
However, note that the first value printed using this `for` loop is `6`, which is `3 * 2`
/19
You can loop through a generator object
However, note that the first value printed using this `for` loop is `6`, which is `3 * 2`
/19
That’s because you had already used up the first three values from the generator
The generator was paused at the end of the third iteration of the `for` loop in the generator function
/20
The generator was paused at the end of the third iteration of the `for` loop in the generator function
/20
You can create several generator objects from the same generator function. These will be independent of each other:
>>> numbers = create_numbers()
>>> more_numbers = create_numbers()
>>> next(numbers)
0
>>> next(numbers)
2
>>> next(numbers)
4
>>> next(more_numbers)
0
/21
>>> numbers = create_numbers()
>>> more_numbers = create_numbers()
>>> next(numbers)
0
>>> next(numbers)
2
>>> next(numbers)
4
>>> next(more_numbers)
0
/21
Even though you used up the first three values from `numbers` in the first three calls to `next()`, when you used `next()` on `more_numbers` for the first time, you got the first value `0`
/22
/22
A generator is an iterator. This also makes it an iterable
However, it doesn’t have a size and it’s not a container since it doesn’t contain any of its own data
Instead, it creates the data as and when needed
/23
However, it doesn’t have a size and it’s not a container since it doesn’t contain any of its own data
Instead, it creates the data as and when needed
/23
––– Etymology Corner –––
We’ve already discussed the word ending “-ator” when discussing “iterator”
It’s something which performs an action
The rest of the word comes from the Latin 'generare' which means “to produce”
/24
We’ve already discussed the word ending “-ator” when discussing “iterator”
It’s something which performs an action
The rest of the word comes from the Latin 'generare' which means “to produce”
/24
The root 'gene-' has older origins and means “give birth”
Therefore a generator produces or gives birth to an item, one at a time!
/25
Therefore a generator produces or gives birth to an item, one at a time!
/25
––– The End –––
This brings us to the end of the series on terms used to categorise different objects that hold or deal with several items on data
I'll finish off with the diagram I shared a couple of days ago…
/26
This brings us to the end of the series on terms used to categorise different objects that hold or deal with several items on data
I'll finish off with the diagram I shared a couple of days ago…
/26
Loading suggestions...