When working with resources in Python, we often perform repetitive tasks such as setting up and closing processes, initializing something, opening and allocating resources, or closing it to free up all the resources. However, we face a memory leak problem when the program retains the resources forever, even if they are no longer required. It compromises the system’s valuable resources. Therefore, all these tasks were automated using the context manager. A context manager makes it easier to allocate free resources when working with files, setting up database connections, or managing network resources. It is important because efficient allocation and deallocation will prevent resource wastage and maximize the use of available resources.
In this article, we discuss in detail how to achieve the maximum use of resources through context manager, its significance, and working with other Python modules.
Understanding Context Manager
1. Explanation Of What Context Manager Is And Why It Is Useful
The context manager allows us to properly manage resources so that we can specify exactly what we want to set up and tear down when working with certain objects, such as file handling or database connections. It sets up a temporary runtime context for you and destroys the context after all operations are completed. In Python, context manager play a vital role in resource management by providing a convenient way to acquire and release resources within the context. It is widely performed using the "with" keyword. It makes sure that the application will not undergo crashes by properly handling resources.
2. Introduction To The "with" Statement As The Primary Way To Use Context Manager In Python
The "with" statement is a built-in module of Python that is used to streamline the management of resources. It assures that the required external resources are properly handled and cleaned up after use, even if an exception occurs during execution. It is favored mostly when dealing with files, network or database connections, or any other kind of resource that needs to be properly handled.
3. Overview Of The "contextlib" Module For Creating Context Manager
As we explore the realm of context manager, it is imperative to grasp the significance of the "contextlib" module. This is a fundamental module that provides utilities for creating and working with the context management function and "with" statements by providing more control over resource management. It is a part of the Python standard library and offers various functions to work with the context manager. For instance, it includes the "contextmanager" decorator that allows you to define the context manager using generators. Also, the "closing()" function creates the context manager of the specified object and closes it when it leaves the context to handle resources properly. Thus, it is a handy Python module that allows a concise way of dealing with a context manager or "with" statements.
Using Context Manager With The "with" Statement
1. Demonstration of Using Built-in Context Manager With The "with" Statement
The "with" statement contains many built-in context manager functions that wrap the execution of the block of code within the specified context. The first is the open function, which is used to perform file I/O operations. By using the context manager in the program, we ensure that the allocated resources are released, and the file is closed correctly. This will help us reduce the deadlock condition and resource wastage. You can perform this by opening the file using the "with" keyword.
file_path = "Context.txt" #provides file location
with open(file_path, 'r') as file: #context manager having file path and its opening mode
file_content = file.read() #this will read the file and save the file into file_content object
print("The Content of file is:")
print(file_content)
In this example, the “with” statement is used to open a file named "Context.txt". The file is automatically closed after it is read, even if an exception occurs during the reading process. We do not have to explicitly mention close() because the “with” statement will close the file properly and release the resources, thereby preventing any potential resource leak.
2. Context Manager Protocol And The __enter__() And __exit__() Methods
Python has defined some context management protocols that dictate how an object can act as a context manager by implementing the __enter__() and __exit__() methods. Any class, function, or object that implements these methods according to the protocols would act as a context manager. These methods are defined as follows:
- __enter__(): It is called by the "with" statement to set up any resources or state required by the context manager to function.
- __exit__(): It is called right before leaving the "with" statement to handle the cleanup operations.
When the "with" statement executes, the expression within it is evaluated and creates the context manager (a resulting value e.g. object of the file), and Python calls the __enter__() methods to assign the return value to the variable given by "as". Then, the __exist__() method is called on the context manager object to release the allocated resources.
Remember that the double underscore methods are special methods that are not called but triggered. They are internally set to run at specific times or after certain events.
Example Of Implementing A Custom Context Manager Using A Class
Let's understand the __enter__() and __exit__() methods by implementing a custom context manager using a class.
class File:
def __init__(self, filename, mode): #initilize the class object with file name and mode
self.filename = filename
self.mode = mode
def __enter__(self): #initilize the __enter__ method for opening the file
print(f"Opening {self.filename} .......")
self.file = open(self.filename, self.mode)
return self.file
def __exit__(self, exc_type, exc_value, traceback): # initilize the __exit__ method for closing the file
print(f"Closing {self.filename} ......")
self.file.close()
#using the custom context manager
with File('Context.txt', 'r') as file:
content = file.read()
print(content)
print("Done!")
Output
Context Manager With The "contextlib" Module
1. "contextlib" Module And Its Utilities For Creating Context Manager
The "contextlib" module provides utilities for common tasks involving the Python "with" statement. It provides a decorator "contextmanager" that allows us a simple way to manage resources and perform setup/teardown operations. This module provides a convenient way to deal with context manager. It is implemented according to the context manager protocols. This is useful when dealing with operations that require resource management, including file handling and database connections for acquiring and releasing locks.
2. Illustration Of Using The "contextmanager" Decorator To Create Context Manager As Generator Function
The "contextmanager" is the decorator of the "contextlib" module, which turns the generator function into a context manager. It is defined using the "@" keyword. It contains the yield value that acts as the required resource. The block of code before the yield value is the setup block to enter the "with" statement, while the code after the yield value is called the teardown block to clean up the resources. Let's understand this with the help of the file opening and reading the Python example.
from contextlib import contextmanager #import library
@contextmanager #decorator on generator function making it context manager
def file(filename, mode): #before yielding, it acts as the enter function
print("This is the implicit ENTER block")
my_file = open(filename, mode)
yield my_file #yielding the with statement to open the file
print("This is the implicit EXIT block") #after yielding, it acts as exit function
my_file.close()
with file("Context.txt", 'r') as file_content: #operating for opening the file
content = file_content.read()
print(content)
print("I'm in WITH block")
pass
Output
3. Example Of Creating A Context Manager For Timer Context Manager Using The "contextmanager" Decorator
Let's take another example of using the 'contextmanager' decorator for implementing timer code execution of context manager. The timer context manager is going to tell us how long a bunch of statements took to execute that were within the 'with' statement.
from contextlib import contextmanager
from datetime import datetime
from time import sleep
@contextmanager #context manager decorator
def timer(): #function to calculate the time taken by each line in with statement
start = datetime.now()
yield
print(f"Time Taken {(datetime.now() - start).seconds} second")
def timer_test(): #for testing the function
with timer():
sleep(2)
timer_test() #calling the function
Output
Explanation: In the above code, I use three libraries: the first is for creating a context manager, the second is for obtaining the current datetime for calculation, and the last is for making the process pause for 2 seconds. This means that this code measures the time taken by the “with” statement code. Therefore, to test this, we paused the execution of the “with” statement for 2 s.
Handling Exceptions With Context Manager
1. How Context Manager Handle Exceptions And Ensure Resource Cleanup
Python streamlines exception handling using the context manager and the "with" statement. We know that exceptions are errors that deviate the program from its normal flow. We need exception handling to handle them gracefully. However, the "with" statement in Python performs exception handling by following the context manager protocols. This ensures that the required setup and teardown process is properly managed even if an exception occurs. It assures the proper management of program resources and executes the necessary "cleanup" actions, regardless of the occurrence of an exception.
2. Illustration Of Using Context Manager To Safely Manage Resources In The Presence Of Exceptions
By utilizing the context manager in the program, we can handle exceptions beforehand while coding the application as well as manage the resources. It is important to use the "with" statement when working with file operations to handle potential exceptions gracefully related to FileNotFound or FileExist errors. So, let's discuss the example of a CSV file where an exception can occur and cause the program to crash and abruptly stop.
import csv #importing the csv module
file_path = "Sample-CSV.csv" #provide location of file in the same directory
try:
#use the context manager for closing the file
with open(file_path, 'r', encoding='utf-8') as file: #here, the file open operation is performed with read mode
csv_content = csv.reader(file) #read the file using csvreader function and save it into variable
for record in csv_content: #iterate through the rows
print(record) #print the record
except FileNotFoundError as e: #if file not found
print("Provide correct File Path")
except UnicodeDecodeError as e: #if incorrect encoding scheme used to decode
print("Cannot decode the file")
else:
print("No exception found")
finally:
print("END of With statement")
Output
3. Example Of Implementing Exception Handling Logic In A Custom Context Manager
Instead of using the "with" statement for exception handling, we can also create our custom-based context manager that manages the resources and gracefully handles the exception. So, let's create a simple custom context manager.
class FileWriter:
def __init__(self, filename, mode): #initilize the class object with file name and mode
self.filename = filename
self.mode = mode
def __enter__(self): #initilize the __enter__ method for opening the file
print(f"Opening {self.filename} .......")
self.file = open(self.filename, self.mode)
return self.file
def __exit__(self, exc_type, exc_value, traceback): # initilize the __exit__ method for closing the file
print(f"Closing {self.filename} ......")
self.file.close()
if exc_type is not None:
print(f"An Error Occured: {exc_value}") #Intentionally raised exception.
return True
#using the custom context manager
with FileWriter('Context.txt', 'w') as file:
file.write(f"Lots of help: Many tools to make different kinds of programs.")
raise Exception("Test exception - this will be suppressed.")
Output
'exc_type' is the exception class, such as AttributeError and IndexError. 'exc_value' is the exception instance with the message. 'traceback' is the full traceback of the error.
In the above code, the __enter__() method opens the file name "Context.txt" in writing mode. It returns the file object that would be used to operate within the "with" block. After that, the __exit__() method is called, which ensures that the resources are closed and not left open. Then, it checks for the exception occurrence at this line "exc_type is not None" which verifies whether the exception type is passed to the exit method of the context manager. Here, I am explicitly raising the exception to explain this in a better way. Then, upon obtaining the exception information, it attempts to surpass the exception with "return True" to manage the resources in case the exception occurs.
Be cautious while raising exceptions within context manager when working in real-world scenarios, as it can impact the control flow and execution of the program.
Q: Why are context manager used in Python
A: Context manager is used to handle setup and teardown actions in a convenient manner hoping that all the resources are properly managed and released regardless of exception occurrence.
Q: How do you create a context manager using classes?
A: You can create the custom context manager by implementing the context manager protocols which include two methods: __enter__() and __exit__().
Q: How to implement context manager using contextlib library?
A:The "contextmanager" decorator is used to implement the function as a context manager, just like the generator function.
Wrapping Up
In conclusion, we have seen how the context manager provides a concise and structured approach to managing resources within the "with" statement. It promotes the allocation and deallocation of resources in a well-organized manner. In addition, to try-finally block the clean-up task, we can efficiently manage the resources using the Python built-in feature i.e. "with" statement. We start with the basic concept, including the predefined way of using it, and move toward the custom-based context manager, which we can create using the class and the contextlib library. We also learned how to use it to deal with exceptions and abnormal behavior in the program. As a junior developer, understanding and grasping this concept will level up you're coding and resource management skills. I hope this guide has given you a basic knowledge of the context manager. Thank You for reading this guide. Happy Coding!
Additonal Resources