Chapter 18: Automation with Python

Automation with Python enables repetitive tasks to be performed efficiently, saving time and reducing human error. Common use cases include file handling, web scraping, and system monitoring.

---

File and Directory Automation

Python’s os and shutil modules facilitate file and directory manipulation.

File Management

import os

# Create a new file
with open("example.txt", "w") as file:
    file.write("Hello, Automation!")

# Rename a file
os.rename("example.txt", "new_example.txt")

# Delete a file
os.remove("new_example.txt")

Directory Management

# Create a directory
os.mkdir("new_folder")

# List contents of a directory
print(os.listdir("."))

# Remove a directory
os.rmdir("new_folder")

Copying and Moving Files

import shutil

# Copy a file
shutil.copy("source.txt", "destination.txt")

# Move a file
shutil.move("file.txt", "new_folder/file.txt")

---

Web Scraping

Web scraping extracts data from websites. Python’s requests and BeautifulSoup libraries are commonly used for this purpose.

Installing Dependencies

pip install requests beautifulsoup4

Example: Scraping Website Data

import requests
from bs4 import BeautifulSoup

# Send a GET request
url = "https://example.com"
response = requests.get(url)

# Parse HTML content
soup = BeautifulSoup(response.text, "html.parser")

# Extract specific elements
titles = soup.find_all("h1")
for title in titles:
    print(title.text)

Handling Dynamic Content

For websites with dynamic content, use selenium.

pip install selenium

from selenium import webdriver
from selenium.webdriver.common.by import By

# Setup WebDriver
driver = webdriver.Chrome()
driver.get("https://example.com")

# Extract dynamic content
elements = driver.find_elements(By.TAG_NAME, "h1")
for element in elements:
    print(element.text)

driver.quit()

---

System Monitoring

System monitoring tasks, such as tracking resource usage or automating backups, can be achieved using Python.

Example: Monitoring CPU and Memory Usage

import psutil

# Get CPU usage
print("CPU Usage:", psutil.cpu_percent(), "%")

# Get memory usage
memory = psutil.virtual_memory()
print("Memory Usage:", memory.percent, "%")

Scheduling Tasks

The schedule library is used to run tasks at specific intervals.

pip install schedule

import schedule
import time

def job():
    print("Running scheduled task...")

# Schedule the task\schedule.every(10).seconds.do(job)

while True:
    schedule.run_pending()
    time.sleep(1)

---

Automating Emails

Automating emails can be achieved using the smtplib library.

Example: Sending an Email

import smtplib
from email.mime.text import MIMEText

# Email details
sender = "[email protected]"
recipient = "[email protected]"
subject = "Automated Email"
body = "Hello, this is an automated email."

# Create MIMEText object
msg = MIMEText(body)
msg["Subject"] = subject
msg["From"] = sender
msg["To"] = recipient

# Send email
with smtplib.SMTP("smtp.example.com", 587) as server:
    server.starttls()
    server.login(sender, "your_password")
    server.send_message(msg)

---

Exercises

Exercise 1: Automate File Cleanup

Write a script to delete all .tmp files in a directory.

Solution:

import os

for file in os.listdir("."):
    if file.endswith(".tmp"):
        os.remove(file)
        print(f"Deleted: {file}")

---

Exercise 2: Scrape and Save Data

Scrape titles from a webpage and save them to a file.

Solution:

url = "https://example.com"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

with open("titles.txt", "w") as file:
    for title in soup.find_all("h1"):
        file.write(title.text + "\n")

---

Exercise 3: Monitor Disk Usage

Write a script to monitor and alert if disk usage exceeds 80%.

Solution:

import psutil

disk = psutil.disk_usage("/")
if disk.percent > 80:
    print("Disk usage alert!", disk.percent, "%")

---

Best Practices

1. Use libraries suited to the task for efficiency.

2. Avoid hardcoding sensitive information like passwords; use environment variables instead.

3. Test automation scripts in a controlled environment before deploying.

4. Implement error handling to manage unexpected scenarios.

5. Document scripts for future reference and maintenance.

In the next chapter, we will explore Python’s role in cloud and DevOps, including writing deployment scripts and working with cloud SDKs.