Opening & Reading Files Handling in Python

Opening & Reading Files Handling in Python

posted 12 min read

We need to understand the programming concept of file handling for storing and reading incoming data from multiple sources in the file. Practically every programming language has incorporated this concept, but I will discuss it in Python. We use the concept of file handling to store the data permanently and temporarily in our machine. However, python provides us with various built-in operations to handle the data into files consistently. So, in this article, I will cover the details of handling different file formats using file-handling operations while gracefully ensuring exception handling and some techniques for optimizing the performance of file-handling operations.

Handling Different File Types

File handling has a crucial role in all programming languages. We can handle different formats of files using the file-handling methods provided by Python. When handling different types of files, use the open() function to access the file. So, let's delve into the details of handling distinct file formats like text, CSV, and JSON files.

Text files

1. Reading Plain Text Files

To read and access the plain text file, open the file using open() function with the mode of reading 'r', also gave the relative path to the file location as an argument.

file_path = "Content.txt" #provide location of file in the same directory   
with open(file_path, 'r') as file:#here, the file open operation is performed with read mode    
    content = file.read() #reads the content from the file and stores it into the variable 
    print(content) #prints the file content

Output

2. Handling Encoding and Decoding

When your program fetches the data from different sources and each source follows a different encoding scheme, then handling text encoding and decoding schemes is necessary. You simply have to open the file with the reading mode. With the open() function, write the encoding scheme as parameter. This will ensure the encoding and decoding of the file content properly.

file_path = "Content.txt" #provide location of file in the same directory   
with open(file_path, 'r', encoding='utf-8') as file:#here, the file open operation is performed with read mode    
    content = file.read() #reads the content from the file and stores it into the variable 
    decode = content.encode().decode() #explicitely decode the encoded content
    print(decode) #prints the file content
Caution: Be mindful while using the proper encoding schemes in the software program to ensure proper text interpretation.

Output

CSV files

1. Using the 'csv' module for Parsing CSV files

Python provides us with the CSV module for performing the file handling operations in CSV format. To perform the operations, you must import the library of CSV. Then, use the open() function to open the file and provide the path of the file with the mode of reading 'r' as we are performing reading operations. After opening the file, use a csv.reader() function to parse the CSV data from the file by passing the file as an argument.

import csv #importing the csv module
file_path = "Sample-CSV.csv" #provide location of file in the same directory        
with open(file_path, 'r') as file: #here, the file open operation is performed with read mode    
    csv_content = csv.reader(file) #read the file using csvreader function and save it into variable
    for record in csv_content: #iterate through the rows
        print(record) #print the record 

Output

2. Reading CSV files with Custom Delimiters and dialects

You can also read the data of the CSV file by specifying the custom delimiters and dialects. Use the CSV module with csv.reader() function. After that, use the delimiter as an argument in the csv.reader() function with its value. Alternatively, you can also write the delimiter that you want to replace. Also, using the next(), you can skip the header while reading the file.

import csv #importing the csv module
file_path = "Sample-CSV.csv" #provide location of file in the same directory        
with open(file_path, 'r', newline = '') as file: #here, the file open operation is performed with read mode   
    record_csv = csv.reader(file, delimiter=';') #this will create the data with custom settings
    first_row = next(record_csv) #skip the header of the CSV file
    for rows in record_csv:
        print(rows)

Output

JSON Files

1. Reading JSON files using the 'json' module

You can also deal with JSON files by using file-handling operations. You must know about the ways of reading data from a JSON file. So, start by importing the JSON module into the program. Then, using the json.load() function, you read the data that is already available in the file.

import json  
file_path = "sample3.json" #provide location of file in the same directory    
with open(file_path, 'r') as file: #here, the file open operation is performed with write mode  
   content = json.load(file) #this will load the stored data from file and saved it into content variable 
   print(content)

Output

2. Handling nested JSON structures

For handling the nested JSON structure, firstly, parse the JSON string into the Python object, then load the data into the file. After that, you can read the data with specifying keys and indices to access and navigate into the file.

import json  
data = '''{ 
     "name": "Chris", 
     "age": 23, 
     "address": { 
         "city": "New York", 
         "country": "America" 
         }, 
    "friends":  
    [ 
        { 
            "name": "Emily", 
            "hobbies": [ "biking", "music", "gaming" ] 
        } 
    ] 
}''' 
encode_Data = json.loads(data) #this will parse the data and stores in the variable  
#accessing the root key 
name = encode_Data["name"] 
age = encode_Data["age"] 

Output

Error Handling and Resource Management

It's important to handle exceptions beforehand while coding the application, such as using try-except-finally blocks when working with file operations to handle potential exceptions gracefully related to FileNotFound or FileExist errors. So, let's discuss the example where an exception can occur and cause the program to crash and abruptly stop.

1. Handling File not found Errors

When you attempt to open the file using the open() function, the file path you have provided for locality is incorrect. So, when Python tries to open the file, and it does not exist at that location, then the compiler raises an exception to FileNotFound error.

file_path = "Content.txt" #provide location of file in the same directory   
try:
    with open(file_path, 'r') as file:#here, the file open operation is performed with read mode    
        content = file.read() #reads the content from file and stores it into the variable 
        print(content) #prints the file content
except FileNotFoundError as e: #if file not found  
  print("Provide correct File Path")   
else:  
  print("No exception found")  
finally:   
  print("Reading operation performed successfully")

Output

2. Using Context Managers for File Handling

By utilizing the context manager in the program, we ensure that the allocated resources are released, and the file is closed correctly. It will help us to reduce the deadlock condition and resource wastage. You can perform it by opening the file using the with keyword.

import csv #importing the csv module
file_path = "Sample-CSV.csv" #provide location of file in the same directory        
try:
    #use the context manager for closing the file
    with open(file_path, 'r', encoding='utf-8') as file: #here, the file open operation is performed with read mode    
       csv_content = csv.reader(file) #read the file using csvreader function and save it into variable
       for record in csv_content: #iterate through the rows
         print(record) #print the record 
except FileNotFoundError as e: #if file not found  
  print("Provide correct File Path") 
except UnicodeDecodeError as e: #if incorrect encoding scheme used to decode  
  print("Cannot decode the file")   
else:  
  print("No exception found")  
finally:   
  print("Reading operation performed successfully") 
Note: When we open the file, the os provides the file handler object to read and write into the file. Once the task is complete, closing the file is recommended.

Output

Performance Optimization and Best Practices

While performing opening and reading operations, it is essential to consider several performance factors when dealing with large amounts of data. I will discuss performance considerations and optimization techniques to enhance the work quality, along with best practices for writing clean and maintainable code.

Performance Considerations for File Handling

  1. Buffering: When you call the open() function with read mode, Python will read the data more than the user requested and store it in the internal buffer. After completely reading the specified buffer, it will again load the data into the buffer. This buffered mechanism will reduce the overhead of input-output operations for accessing the file. Reading and writing the large buffer into the file reduces the need for multiple reading/writing operations. Also, python has the specialty to save the buffered output by default.
  2. Reading Large Files: Reading and writing a large amount of data at once is better than reading each line separately. It reduces the time complexity and improves performance.
  3. Modes Impact: When you open the file, the mode used to perform the task can impact the performance. For example, the appended mode will take longer than the write mode as the pointer needs to be shifted to the end of the file.
  4. Closing Files: When reading is complete, it's better to close the file. This will ensure the release of the allocated resources and stored data permanently to disk.
  5. Asynchronous input/output Operation: Try to utilize asynchronous file input/output operation when you are dealing with large files because it allows the program to perform another task while waiting for the file operations to be completed.

Techniques for Efficient File Writing

  1. When performing file handling, it's better to open the file once, perform all necessary reading and writing, and then close it, rather than opening and closing the file repeatedly.
  2. Always use the right mode while dealing with files. For example, if you need to add data to an existing file, use append mode; otherwise, use write mode to overwrite the file.
  3. Reading and writing a bunch of data to the file is better than doing one piece at a time. It means adding larger chunks to perform the writing/reading operation efficiently.
  4. For storing written data, faster storage drives like SDD provide better storage than the traditional HDD.

Best Practices for Writing Clean and Maintainable Code

  1. Always write code that is easy to understand. The name of the variables and functions should describe their purpose.
  2. Always incorporate as many comments as possible where necessary.
  3. Follow the convention for writing the clean code defined by the PEP 8 style guide to make a consistent code style.
  4. Always incorporate error handling in your program. It will help you to deal with the unexpected event beforehand and prevent the program from crashing.
  5. Try to write small modules that perform one specific functionality instead of writing the large modules.

Challenges and Considerations

1. Dealing with Large Files and Memory Constraints

When you're dealing with larger files, it is essential not to load the whole file into the memory at once because managing the memory constraints becomes a critical issue to prevent the system from crashing and reduce the performance issues. It is better to incorporate the techniques for dealing with large files i.e. memory mapping, processing in chunks, and buffering that help to mitigate these issues.

2. Handling Platform-Specific file Operations

Each operating system has its own file-handling operations and requirements. When you're performing file handling operations, adapt the platform-specific API where possible, and utilize the libraries and modules like “sys.platform” that abstract away the platform-specific differences. So, adapting these file handling codes will ensure compatibility across different platforms and is an essential consideration.

3. Strategies for handling file Permissions and Security

The permission for accessing and manipulating the files is related to the operating system file system. Ensuring the proper permission setting is mostly related to who can open, read, write, or execute the file, which needs to be set by OS command. Permission settings are necessary when you are working in a multiuser environment because they ensure that the file content cannot get truncated. So, the best strategies are to implement encryption techniques while accessing the file and give permission to the person who is authorized to work on the file.

FAQs Q: When FileExist error occur?
A: When you are trying to open the file with the 'x' mode which creates the new file before performing the file handling operation, but upon existing it throws the error.
Q: Can I also open and read the data in an XML file?
A: Yes, you can read the XML data by opening the XML file, just like we open the other file formats.
Q: Can I read the entire content of the file in a single string?
A: Yes, you can read the entire content by using the read() function in Python

Wrapping Up

In the end, we discussed the file handling opening and reading operation concerning various file formats. We also understand the need for exception handling when performing file handling. Exception handling is necessary for every software program because it ensures application safety and stops the application from showing abrupt behavior once errors occur. Also, we need to understand some performance optimization techniques that help us to efficiently improve the flow of working with files. I hope this guide is helpful for you when you are dealing with file operations. Also, if you have any questions or concerns, then feel free to ask and give feedback. Thank you for following this guide. Happy Coding!


If you read this far, tweet to the author to show them you care. Tweet a Thanks

More Posts

Mastering Reading and Writing CSV Files in Python

Abdul Daim - Mar 27, 2024

Python Opening & Reading Files

Abdul Daim - Mar 8, 2024

Writing to Files In Python

Abdul Daim - Apr 12, 2024

Python Writing to Files

Abdul Daim - Mar 19, 2024

Working With JSON File in Python

Abdul Daim - Mar 18, 2024
chevron_left