How Luigi Introduced Me To Artificial Intelligence: Intro to Computer Vision
Memory Match — Super Mario Bros
I was 6 years old when I got my first Nintendo DS. Not a DSI (yes the one with a camera). And not a 3DS (also with a camera). My favorite game on my DS was Super Mario Bros’ Memory Match.
Here’s How Luigi Explains the Game:
● “Pick two matching cards to make them disappear. Clear all the cards to win. If you make three mistakes, Luigi Wins.”
Let me preface something really quick: I was horrible at this game. Luigi stole all of my money from losing over and over. I didn’t remember which cards belonged where or how they looked. So once I picked up a new card, I’d mistake it for another which led to many, many losses.
Hundreds of rounds later, I began to familiarize myself with the cards’ features, like color and shape, which helped me distinguish them. Only then was I able to get my money back.
This game is similar to many applications of Computer Vision, a subcategory of Artificial Intelligence.
Computer Vision IRL
Google defines Computer Vision as:
“a field of artificial intelligence that trains computers to interpret and understand the visual world. Using digital images from cameras and videos and deep learning models, machines can accurately identify and classify objects — and then react to what they “see.”” — sas.com
This explains how I gradually got better at identifying the cards; I constantly analyzed new cards and their features which made it easier for me to match them.
But obviously, I am NOT a computer. Though they do function a similar way…
So…How Does it Work?
First, let’s look at some kitties!
Storing Data
At the highest level, a computer would examine data by:
1. Analyzing a dataset of images and the different features that they present including but not limited to: color, shape, orientation, distance, shadows, etc.
2. Storing this data in a list (shown in the image above).
3. After extracting this data, it compares it to the data of other images in your dataset. 4. Then, it determines whether it is, or is not that image. For example, “Cat” or “Not Cat”
Cat vs Not Cat
Training
To train our model, we use data that we know is correctly labeled — we call this our training set. This can train our model to label those specific examples using supervised learning.
● Other types of machine learning include unsupervised learning (attempt to make inferences about unlabelled data) and reinforcement learning (based on human psychology).
We measure our model’s performance during training by looking at how accurate its predictions are for the examples we have seen in the training set or the training error.
Fitting Your Data
Instead of memorizing the correct labels for the specific examples in our training set, we find patterns in the data that allow us to generalize.
● Overfitting: When we fit too specifically to the data we’ve seen (memorization) ● Underfitting: When the model doesn’t do well on the data we have seen ● Good Fit: If the model understands the patterns in our data and can generalize
Evaluation
We can use a Confusion Matrix to determine the accuracy of our model.
Columbia University’s AI4ALL: Confusian Matrix
● Precision: Probability that a positive prediction is correct, TP / (TP + FP) ● Recall (or Sensitivity): Probability that an actual positive outcome is predicted correctly, TP / (TP + FN)
● Specificity: Probability that an actual negative outcome is predicted correctly, TN / (TN + FP)
● F1 Score: Combination of precision and recall, (2 * Precision * Recall) / (Precision + Recall)
● Accuracy: (TP + FP) / all
Conclusion
Right now, we are on the cusp of solving the uncertainties of medical care. With Computer Vision, we are able to use technology to decrease the margin of error in detecting skin cancer and eye diseases, like Glaucoma. As more solutions are being discovered using Computer Vision and other branches of Artificial Intelligence, the faster we are able to supply treatment to the ill. We are only at the beginning and could use all hands on deck to do this!
Abstract
Computer Vision is a subcategory of Artificial Intelligence that deals with computers extracting information from digital items and videos to gain a high-level understanding of what is being presented. First, the computer will debunk a wide range of images in a dataset and extract their information in the form of 1s and 0s. Then it will store this data into a list, and compare it with other items to depict whether or not these images are presenting the same object. This is called training your dataset. Finally, to evaluate, we use a confusion matrix that calculates the probability of false positives, false negatives, true positives, and true negatives to determine the
accuracy of our data conclusions. This allows us to generalize our data and group them in order of what images are the same and what are different.
Sources:
1. “Memory Match (Super Mario Series).” Super Mario Wiki,
www.mariowiki.com/Memory_Match_(Super_Mario_series).
2. Mihajlovic, Ilija. “Everything You Ever Wanted To Know About Computer Vision. Here's A Look Why It's So Awesome.” Medium, Towards Data Science, 9 Feb. 2020, towardsdatascience.com/everything-you-ever-wanted-to-know-about-computer-vision-he res-a-look-why-it-s-so-awesome-e8a58dfb641e.
3. Sarkar, Dipanjan (DJ). “A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning.” Medium, Towards Data Science, 17 Nov. 2018,
towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real world-applications-in-deep-learning-212bf3b2f27a.
4. “Introduction to Computer Vision, Part 1.” Machine Learning,
machinelearning.technicacuriosa.com/introduction-to-computer-vision-part-1/ 5. I have permission from Columbia University's AI4ALL Program to use this image.