Image Data with Numpy and Matplotlib

Setup

Import Libraries

import numpy as np
import matplotlib.pyplot as plt

from os import makedirs, mkdir
from PIL import Image
from urllib.request import urlopen
from pathlib import Path
from io import BytesIO
import owncloud

Download Data

Path('data').mkdir(exist_ok=True, parents=True)

owncloud.Client.from_public_link('https://uni-bonn.sciebo.de/s/CjGGdYrPn2LakdA').get_file('/', 'data/cells.jpg')

owncloud.Client.from_public_link('https://uni-bonn.sciebo.de/s/xCYyyrkzYm9ka8N').get_file('/', 'data/intestine.jpg')

owncloud.Client.from_public_link('https://uni-bonn.sciebo.de/s/2piM6YrbFqAx2iw').get_file('/', 'data/montypython.jpg')

owncloud.Client.from_public_link('https://uni-bonn.sciebo.de/s/BZPomc42zGDzs9a').get_file('/', 'data/skull.png')

True

Section 1: Working with Images

Image data is stored as a 3D matrix, storing the brightness of each pixel along 3 coordinates:

0. Which row the pixel is in (between 0 and the height of the image)
1. Which column the pixel is in (between 0 and the width of the image)
2. What color channel the pixel is in (red, green, blue, and sometimes alpha)

White pixels usually have the highest brightness values, and black pixels the lowest.

If you zoom in enough, you can see the square grid. For example, as we zoom in on this heart we start to see the square pixels that make it up:

This means that visualizing any image in Python can be done by “plotting” a matrix!
Let’s load an image and visualize it onscreen using Matplotlib, a plotting library. Working with images generally uses these 3 functions:

Exercises

Example: Read and Plot the Monty Python image in the data folder.

im = plt.imread('data/montypython.jpg')
plt.imshow(im)

Exercise: Read and plot the mouse intestine image in the data folder:

Solution

im = plt.imread('data/intestine.jpg')
plt.imshow(im);

Exercise: Read and plot the “Cells” image in the data folder.

Solution

im = plt.imread('data/cells.jpg')
plt.imshow(im);

Example: Using the Cells image, index only the first 50 rows of the image and plot it, then save it as “top_part.png”. (This is called “cropping” an image)

Solution

cropped = im[:50, :]
plt.imshow(cropped);
plt.imsave("top_part.png", cropped)

Exercise: Crop and Plot only the left cluster of cells, then save the image as “left_cluster.png”.

Solution

cropped = im[:, :180]
plt.imshow(cropped);
plt.imsave('left_cluster.png', cropped)

Exercise: Crop and Plot only the right cluster of cells, then save the image as “right_cluster.png”.

Solution

cropped = im[15:90, 200:]
plt.imshow(cropped);
plt.imsave("right_cluster.png", cropped)

Section 2: Analyzing Image Data: Examining Each Channel of a Color Image

Fluorescent imaging is made so that each color channel contains data from a different source. Matrices that represent color images put this data into its own axis. In matplotlib, the color channel is the third axis, with the data ordered RGB channels.

For the section on separating channels, here’s a markdown table summarizing the methods used along with one-line explanations:

Each of the exercises below use a two-photon image of a mouse intestine. In this image, the red channels label actin filaments, the green label cell nuclei, and the blue label goblet cells.

Exercises

Example: Plot only the red channel, using the “gray” colormap.

image = plt.imread('data/intestine.jpg')
red_channel = image[:, :, 0]
plt.imshow(red_channel, cmap='gray');
plt.colorbar();

Exercise: Plot only the green channel, using the “gray” colormap.

Solution

image = plt.imread('data/intestine.jpg')
green_chan = image[:, :, 1]
plt.imshow(green_chan, cmap='gray');
plt.colorbar();

Exercise: Plot only the blue channel, using the “gray” colormap.

Solution

image = plt.imread('data/intestine.jpg')
blue_chan = image[:, :, 2]
plt.imshow(blue_chan, cmap='gray');
plt.colorbar();

Exercise: Plot only the blue channel, using the “viridis” colormap.

Solution

image = plt.imread('data/intestine.jpg')
blue_chan = image[:, :, 2]
plt.imshow(blue_chan, cmap='viridis');
plt.colorbar();

Exercise: Plot only the green channel, using the “magma” colormap.

Solution

image = plt.imread('data/intestine.jpg')
green_chan = image[:, :, 1]
plt.imshow(green_chan, cmap='magma');
plt.colorbar();

Exercise: Matplotlib has dozens of colormaps, all good for different purposes. Let’s try some out! Using the reference here, replace the cmap value in the code below to plot the green channel’s data using three different colormaps:

Solution

image = plt.imread('data/intestine.jpg')
green_chan = image[:, :, 1]
fig = plt.figure(figsize=(100, 30))

# Left subplot
plt.subplot(1, 3, 1)
plt.imshow(green_chan, cmap='inferno')
plt.axis('off')

# Middle subplot
plt.subplot(1, 3, 2)
plt.imshow(green_chan, cmap='plasma')
plt.axis('off')  # hide the axis number labels (just show the image)

# Right subplot
plt.subplot(1, 3, 3)
plt.imshow(green_chan, cmap='viridis');
plt.axis('off')

(-0.5, 713.5, 466.5, -0.5)

Section 3: Aggregating Across Axes

Almost all of the Numpy aggregation functions have an axis option, which lets you limit the operation to just that axis.

For example, to get the mean of all columns:

>>> array = np.arange(12).reshape(3, 4)
>>> array.mean(axis=0)
array([4., 5., 6., 7.])

And the mean of the rows:

>>> array.mean(axis=1)
array([1.5, 5.5, 9.5])

Notice that the number of dimensions goes down by default whenever you aggregate across the axis. If you’d like to keep the dimensions the same, you can also use the keepdims=True option:

>>> array.mean(axis=1, keepdims=True)
array([[1.5],
       [5.5],
       [9.5]])

Try it out for yourself, with the provided array data:

Exercises

np.random.seed(42)
data = np.random.randint(0, 10, size=(5, 3)) * [1, 10, 100]
data

array([[  6,  30, 700],
       [  4,  60, 900],
       [  2,  60, 700],
       [  4,  30, 700],
       [  7,  20, 500]])

Example: What is the mean of each column?

a = data.mean(axis=0, keepdims=False)
a

array([  4.6,  40. , 700. ])

Exercise: What is the standard deviation of each row?

data.std(axis=1)

array([321.64714967, 409.81730347, 316.25727923, 322.14420511,
       229.3996997 ])

Exercise: What is the maximum of each column?

data.max(axis=0)

array([  7,  60, 900])

Exercise: What is the mean of each column’s median?

np.median(data, axis=0).mean()

244.66666666666666

Exercise: What is the standard deviation of all the numbers in the matrix?

np.std(data)

328.17397824934255

Exercise: What is the maximum of each row?

data.max(axis=1)

array([700, 900, 700, 700, 500])

Section 4: Analyzing Data with Histograms and Line Plots

Exercises

Example: How does the average green brightness level change in the image as you look left-to-right in it?

image = plt.imread('data/intestine.jpg')
av_green_brightness = image[:, :, 1].mean(axis=0)
plt.plot(av_green_brightness, 'g');

Exercise: How does the average red brightness level change in the image as you look left-to-right in it?

Solution

image = plt.imread('data/intestine.jpg')
red_brightness = image[:, :, 0].mean(axis = 0)
plt.plot(red_brightness, 'r')

Exercise: Plot the average brightness of each “intenstine” image color’s columns in the same figure, showing how the image is mostly green on the left of the image, mostly blue in the middle of the image, and mostly red on the right of the image.

Solution

image = plt.imread('data/intestine.jpg')
plt.plot(image[:, :, 0].mean(axis=0), 'r');
plt.plot(image[:, :, 1].mean(axis=0), 'g');
plt.plot(image[:, :, 2].mean(axis=0), 'b');

Example: What is the distribution of green brightness levels in the image?

image = plt.imread('data/intestine.jpg')
green_chan = image[:, :, 1]
plt.hist(green_chan.flatten(), range=(0, 256), bins=128);

Exercise: What is the distribution of red brightness levels in the image?

image = plt.imread('data/intestine.jpg')
red_chan = image[:, :, 0]
plt.hist(red_chan.flatten(), range=(0, 256), bins=128);

Exercise: What is the distribution of blue brightness levels in this image?

image = plt.imread('data/intestine.jpg')
blue_chan = image[:, :, 2]
plt.hist(blue_chan.flatten(), range=(0, 256), bins=256);

Section 5: (Extra Challenge): Swapping around Color Channels

Exercise: Make a new image from intestine.jpg, where all the color channels have swapped with each other (e.g. red is now green, green is now blue, blue is now red).

Solution

im = plt.imread('data/intestine.jpg')
im2 = np.zeros_like(image)
im2[:, :, 0] = im[:, :, 2]
im2[:, :, 1] = im[:, :, 0]
im2[:, :, 2] = im[:, :, 1]

plt.figure()
plt.subplot(1, 2, 1)
plt.axis('off')
plt.imshow(im);
plt.subplot(1, 2, 2)
plt.imshow(im2);
plt.axis('off')

(-0.5, 713.5, 466.5, -0.5)