Python Logging Basics

1. From print() to Basic Logging

At its simplest, logging can seem like a glorified print() statement. However, it offers far more power and control, which is essential for a backend system. With print(), you have no control over the message's importance, no automatic timestamp, and no easy way to turn messages on or off. Logging solves all these problems.

The logging module provides five standard levels of severity, allowing you to categorize your messages:

• DEBUG: Detailed information, typically of interest only when diagnosing problems.
• INFO: Confirmation that things are working as expected.
• WARNING: An indication that something unexpected happened, but the software is still working as expected.
• ERROR: Due to a more serious problem, the software has not been able to perform some function.
• CRITICAL: A very serious error, indicating that the program itself may be unable to continue running.

By default, Python's logger will only show messages of WARNING level and above. To see everything, we need to configure it.

Corey Schafer's video provides an excellent introduction to these basic concepts. You'll see how to set the logging level, redirect logs to a file, and customize the message format.

Let's consolidate what you've just seen. The logging.basicConfig() function is a quick way to set up logging for a simple script. Here is a configuration that includes all the key elements:

import logging

logging.basicConfig(
    level=logging.DEBUG,
    filename='app.log',
    filemode='w', # 'w' for write (overwrite), 'a' for append
    format='%(asctime)s - %(levelname)s - %(message)s'
)

logging.debug("This is a debug message.")
logging.info("Application is running smoothly.")
logging.warning("The disk is 90% full.")
logging.error("Failed to connect to the database.")
logging.critical("Catastrophic failure. Shutting down.")

If you run this code, it won't print anything to the console. Instead, it will create a file named app.log with formatted, timestamped entries for all the messages.

2. Logging Exceptions

This is where logging connects directly with our previous lesson. When you catch an exception in a try...except block, you should almost always log it. This provides a permanent record of the failure, complete with the context of what went wrong.

The logging module makes this extremely easy. When you're inside an except block, you can use logging.exception() or pass exc_info=True to another logging function.

Here's an example that combines our knowledge from the last lesson with today's topic:

import logging




# Basic configuration
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

def divide(x, y):
    try:
        result = x / y
        logging.info(f"Division successful: {x}/{y} = {result}")
        return result
    except ZeroDivisionError:



        # logging.exception() automatically logs at ERROR level and includes the stack trace.
        logging.exception("Attempted to divide by zero.")
        return None

divide(10, 2)
divide(10, 0)

Running this script will produce the following output:

2023-10-27 10:30:00,123 - root - INFO - Division successful: 10/2 = 5.0
2023-10-27 10:30:00,124 - root - ERROR - Attempted to divide by zero.
Traceback (most recent call last):
  File "<stdin>", line 3, in divide
    result = x / y
ZeroDivisionError: division by zero

This is an incredibly powerful debugging tool. You get your custom message and the full technical traceback that led to the error.

3. The Professional Setup: Loggers, Handlers, and Formatters

While basicConfig() is great for single-file scripts, it has a major flaw: it configures the root logger. In a larger application with multiple modules (like a Django project), different parts of the code might try to configure the root logger, leading to unpredictable behavior where one module's settings overwrite another's.

The professional solution is to create named loggers for different parts of your application and configure them with specific Handlers and Formatters.

• Logger: The object you interact with in your code (logger.info(), logger.error(), etc.). You get one using logging.getLogger(__name__). The __name__ convention automatically gives the logger a name corresponding to the Python module it's in.
• Handler: Determines where the log messages are sent. For example, a StreamHandler sends logs to the console, and a FileHandler sends them to a file. A single logger can have multiple handlers.
• Formatter: Controls the string format of the log messages. You can have different formatters for different handlers (e.g., a simple format for the console and a detailed one for the file).

This architecture gives you immense flexibility.

Corey Schafer's advanced logging video is the best resource to understand why this modular setup is necessary and how to implement it.

Logger and Handler Level Interaction

A key concept is the interaction between the logger's level and its handlers' levels.

1. A message is first checked against the logger's level. If the message's severity is too low (e.g., a DEBUG message on a logger set to INFO), it is discarded immediately.
2. If it passes the logger's check, the message is then passed to all of the logger's handlers.
3. Each handler then checks the message against its own level. If the message is too low for a specific handler, that handler ignores it.

This hierarchy is powerful. A common pattern is to set the main logger's level very low (e.g., DEBUG) and then control what gets outputted by setting more restrictive levels on the handlers. For example:

• A StreamHandler (console) set to INFO for general progress messages during development.
• A FileHandler set to WARNING to create a production log file that only contains important issues.

Hands-On Task: Configure a Modular Logger

Now, let's practice this professional setup. Create a new Python file and implement the following:

1. Get a logger for the current module using logging.getLogger(__name__).
2. Set the logger's level to logging.DEBUG.
3. Create a StreamHandler for console output. Set its level to logging.INFO.
4. Create a FileHandler to write to a file named app_errors.log. Set its level to logging.ERROR.
5. Create a Formatter with the format %(asctime)s - %(name)s - %(levelname)s - %(message)s.
6. Set this formatter on both the StreamHandler and the FileHandler.
7. Add both handlers to your logger.
8. Write five log messages, one for each level (DEBUG, INFO, WARNING, ERROR, CRITICAL).

When you run your script, you should see the INFO, WARNING, ERROR, and CRITICAL messages on your console, but only the ERROR and CRITICAL messages should appear in the app_errors.log file.

import logging




# 1. Get a logger
# The __name__ variable will be '__main__' when this file is run directly.
logger = logging.getLogger(__name__)




# 2. Set logger's level to the lowest possible, to pass all messages to handlers.
logger.setLevel(logging.DEBUG)




# 3. Create a StreamHandler for console output
console_handler = logging.StreamHandler()



# Set its level to INFO to show general messages
console_handler.setLevel(logging.INFO) 




# 4. Create a FileHandler for file output
file_handler = logging.FileHandler('app_errors.log')



# Set its level to ERROR to only log serious problems
file_handler.setLevel(logging.ERROR)




# 5. Create a Formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')




# 6. Set the formatter for both handlers
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)




# 7. Add both handlers to the logger
logger.addHandler(console_handler)
logger.addHandler(file_handler)




# 8. Log messages
logger.debug("This is a detailed debug message. It won't be seen on console or file.")
logger.info("The application started.") # Will appear on console
logger.warning("Configuration file is missing, using defaults.") # Will appear on console
logger.error("Could not establish a connection to the database.") # Will appear on console AND file
logger.critical("The entire system is on fire.") # Will appear on console AND file

Conclusion

You've now covered one of the most practical and essential skills for a backend developer. Effective logging transforms your application from a black box into a transparent system that tells you exactly what it's doing.

Key Takeaways:

• Logging over print(): Provides levels, formatting, timestamps, and destination control.
• basicConfig(): Quick and easy for simple scripts but problematic for larger projects.
• Professional Structure: Use named loggers (getLogger), Handlers (destinations), and Formatters (layouts) for modular, conflict-free logging.
• Hierarchical Levels: The logger acts as a primary filter, and each handler can apply a secondary, more specific filter.
• Log Exceptions: Always use logging.exception() within an except block to capture the full error context.

This skill will be indispensable as we move forward. In our next lesson, we will begin our journey into automated testing with pytest. You'll see that logging can even be a valuable tool in testing, allowing you to assert that certain events were logged as expected during a test run.