Project 1: Classifying Natural Images

This project is designed to test your current knowledge on applying a CNN to the natural images dataset on Kaggle. This dataset contains 6,899 images from 8 distinct classes to include airplane, car, cat, dog, flower, fruit, motorbike and person.

Your goal is to develop a CNN model to accurately classify new images. Using only the knowledge you’ve gained thus far, and repurposing code from previous modules, you should be able to obtain an accuracy of approximately 90% or higher.

Good luck!

Package Requirements

Depending on your approach you may need to load more libraries.

library(keras)

Part 1: Data Preparation

We have already downloaded and organized the images into train, validation, and test directories.

# define the directories:
image_dir <- here::here("materials", "data", "natural_images")
train_dir <- file.path(image_dir, "train")
valid_dir <- file.path(image_dir, "validation")
test_dir <- file.path(image_dir, "test")

As previously mentioned, there are 8 total classes, each with fairly proportional number of train, validation, and test images:

classes <- list.files(train_dir)
total_train <- 0
total_valid <- 0
total_test <- 0

for (class in classes) {
  # how many images in each class
  n_train <- length(list.files(file.path(train_dir, class)))
  n_valid <- length(list.files(file.path(valid_dir, class)))
  n_test <- length(list.files(file.path(test_dir, class)))
  
  cat(crayon::underline(crayon::red(class)), ": ", 
      "train (", n_train, "), ", 
      "valid (", n_valid, "), ", 
      "test (", n_test, ")", "\n", sep = "")
  
  # tally up totals
  total_train <- total_train + n_train
  total_valid <- total_valid + n_valid
  total_test <- total_test + n_test
}

airplane: train (436), valid (145), test (146)
car: train (580), valid (193), test (195)
cat: train (531), valid (177), test (177)
dog: train (421), valid (140), test (141)
flower: train (505), valid (168), test (170)
fruit: train (600), valid (200), test (200)
motorbike: train (472), valid (157), test (159)
person: train (591), valid (197), test (198)

cat("\n", "total training images: ", total_train, "\n",
    "total validation images: ", total_valid, "\n",
    "total test images: ", total_test, sep = "")

total training images: 4136
total validation images: 1377
total test images: 1386

Let’s check out the first image from each class:

op <- par(mfrow = c(2, 4), mar = c(0.5, 0.2, 1, 0.2))
for (class in classes) {
  image_path <- list.files(file.path(train_dir, class), full.names = TRUE)[[1]]
  plot(as.raster(jpeg::readJPEG(image_path)))
  title(main = class)
}
       
par(op)

Part 2: Modeling

There are two approaches you could take to model this data:

1. End-to-end trained CNN with your own custom convolutional layer (reference the 04-computer-vision-CNNs/02-cats-vs-dogs.Rmd file). This solution requires less code but takes 1-2 hours without GPUs. Using the same model structure in this notebook should net you about 89-90% accuracy.
2. Apply a pre-trained model (reference the 04-computer-vision-CNNs/03-transfer- learning.Rmd file). This solution is much quicker to train but requires more code. Using the feature extraction model structure in this notebook should net you about 98% accuracy.

Leverage your neighbors’ knowledge and here are some things to think about:

• Architecture & compile steps:
  • This is a multi-class classification problem so be sure to pick the activation and loss function that aligns to this type of problem.
  • You will likely not have time to test out multiple models and tune hyperparameters so you can assume that the default learning rate or just slightly smaller will work sufficiently (lr = 1e-3 or 1e-4).
• Image data generators
  • The images are traditional RGB images with pixel values of 0-255
  • Resizing the images to 150x150 is sufficient
  • Be sure to set class_mode = "categorical" in the flow_images_from_directory function.
  • Batch size of 32 is sufficient
  • Use the train_dir, valid_dir, and test_dir directories above
• Model fitting
  • 50 epochs will be more than enough and…
  • Be sure to use an early stopping callback to speed up training time
  • Using a callback to automatically reduce the learning rate will improve performance

Do the best you can leveraging code from the Cats vs. Dogs and Transfer Learning notebooks. Much of this code will transfer over one-for-one. If you run into problems it is ok to peak at the solution notebook but try to make this your last resort.