# Chapter 11: Iterators and Generators

Iterators and generators are advanced Python features that allow for efficient looping and data generation. They enable you to handle large datasets and compute values lazily, improving performance and memory usage.

#### Iterators

An iterator is an object that implements the `__iter__()` and `__next__()` methods. It represents a sequence of values, returning one value at a time when iterated.

**Example of an Iterator:**

```python
# Create an iterator
numbers = iter([1, 2, 3, 4, 5])

# Access elements using next()
print(next(numbers))  # Output: 1
print(next(numbers))  # Output: 2

# Iterate over the rest using a loop
for num in numbers:
    print(num)  # Output: 3, 4, 5
```

**Key Methods:**

| **Method**   | **Description**                                                  |
| ------------ | ---------------------------------------------------------------- |
| `__iter__()` | Returns the iterator object itself.                              |
| `__next__()` | Returns the next item in the sequence or raises `StopIteration`. |

#### Custom Iterators

You can create custom iterators by defining a class that implements the `__iter__()` and `__next__()` methods.

**Example:**

```python
class Counter:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.end:
            raise StopIteration
        self.current += 1
        return self.current - 1

# Using the custom iterator
counter = Counter(1, 5)
for num in counter:
    print(num)  # Output: 1, 2, 3, 4, 5
```

#### Generators

A generator is a special type of iterator created using functions and the `yield` keyword. Generators are used to lazily produce values one at a time, as needed.

**Creating a Generator:**

```python
def generate_numbers(n):
    for i in range(1, n + 1):
        yield i

# Using the generator
for num in generate_numbers(5):
    print(num)  # Output: 1, 2, 3, 4, 5
```

**Key Differences Between Generators and Iterators:**

1. Generators are defined using functions and the `yield` keyword.
2. Generators automatically implement the `__iter__()` and `__next__()` methods.

#### Generator Expressions

Generator expressions are similar to list comprehensions but produce values lazily.

**Example:**

```python
# Generator expression
squares = (x ** 2 for x in range(1, 6))

# Iterate over the generator
for square in squares:
    print(square)  # Output: 1, 4, 9, 16, 25
```

#### Advantages of Generators

1. **Memory Efficiency**: Generate values on the fly instead of storing them in memory.
2. **Lazy Evaluation**: Values are computed only when needed.
3. **Readable Code**: Easier to implement compared to custom iterators.

#### Practical Use Cases

1. **Reading Large Files**: Process a file line by line without loading the entire file into memory.

   ```python
   def read_large_file(filename):
       with open(filename, "r") as file:
           for line in file:
               yield line.strip()

   for line in read_large_file("example.txt"):
       print(line)
   ```
2. **Infinite Sequences**: Generate an unbounded sequence of values.

   ```python
   def infinite_numbers(start):
       while True:
           yield start
           start += 1

   for num in infinite_numbers(1):
       if num > 5:
           break
       print(num)  # Output: 1, 2, 3, 4, 5
   ```
3. **Efficient Data Pipelines**: Process data streams incrementally.

#### Exercises

**Exercise 1:**

Write a custom iterator that generates the Fibonacci sequence up to a given number `n`.

**Solution:**

```python
class Fibonacci:
    def __init__(self, n):
        self.n = n
        self.a, self.b = 0, 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.a > self.n:
            raise StopIteration
        result = self.a
        self.a, self.b = self.b, self.a + self.b
        return result

# Using the iterator
for num in Fibonacci(10):
    print(num)  # Output: 0, 1, 1, 2, 3, 5, 8
```

**Exercise 2:**

Write a generator function to yield even numbers up to a given number `n`.

**Solution:**

```python
def even_numbers(n):
    for i in range(2, n + 1, 2):
        yield i

# Using the generator
for num in even_numbers(10):
    print(num)  # Output: 2, 4, 6, 8, 10
```

**Exercise 3:**

Use a generator expression to create a sequence of cubes for numbers 1 through 5.

**Solution:**

```python
cubes = (x ** 3 for x in range(1, 6))
for cube in cubes:
    print(cube)  # Output: 1, 8, 27, 64, 125
```

#### Best Practices

1. Use generators for processing large datasets or streams to save memory.
2. Combine generator expressions with functions like `sum()` and `max()` for concise calculations.
3. Prefer `yield` over returning large lists when possible.
4. Test custom iterators to handle edge cases, such as empty sequences.

In the next chapter, we will explore decorators and context managers, which are powerful tools for enhancing and managing Python functions and resources.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://py.d19.in/chapter-11-iterators-and-generators.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.