part of Course 131 Data Munging Tips and Tricks

This is a list of snippets for reading and writing data files in Python. It's not encyclopedic, but it's a convenient collection of the things I google regularly.

Read a text file

with open("filename.txt", "rt") as f:
    data_lines = f.readlines()

Save a Python object

import pickle as pkl
with open("filename.pkl", "wb") as f:
    pkl.dump(object, f) 

Load a Python object

import pickle as pkl
with open("filename.pkl", "rb") as f:
    object = pkl.load(f)

Load an image as a Numpy array

import numpy as np
from PIL import Image
img = np.asarray(Image.open("image_filename.jpg"))

Save a Matplotlib image

import matplotlib.pyplot as plt
# Create image in Figure named fig
fig.savefig("filename.png", dpi=300)

Load a csv as a Python list of dictionaries

import csv
with open("data.csv") as csv_file:
    list_of_dicts = list(csv.DictReader(csv_file))

Load a csv as a Pandas DataFrame

import pandas as pd
df = pd.read_csv("data.csv")

Download a csv from a URL

import urllib.request
response = urllib.request.urlopen(csv_url)
rows = response.read().decode("utf-8").split("\n")

Convert a JSON string to Python dictionary and back

import json
json_dict = json.loads(json_formatted_string)
json_formatted_string = json.dumps(json_dict)