Computer Vision

This project contains multiple functionalities to manipulating and writing image and frames (video). By writing functions to the program, many effects can be applied to images. Users can easily choose what type of filters to apply to by pressing the corresponding keystrokes. The enviornment that I use is Windows OS and the IDE is Visual Studio Code. I will demonstrate and illustrate each functionalities in below content. To see video version results and the extension task of the project, please scroll all the way down to Video Processing (Project 1 Video Capture.mp4).

Image Display: imgDisplay.cpp contains two functions that allow users to terminate the program or to display images.

*By default, the first image will be automaticlly displayed when the program is started.

Press “q”, “Q”, or “ESC” to close the current program.
Press “x” or “X” to open the second image.

Basics : vidDisplay.cpp contains mutiple different functions that allow users to perform different basic tasks.

*By default, the camera will be automaticlly opened when the program is started. Failure to open the camera would result termination.

1. Press “q”, “Q”, or “ESC” to close the current program.

2. Press “s” or “S” to save the current frame (savedCaptures.jpg) under the current folder.

3. Press “Enter” to repeat the menu.

Image Processing: vidDisplay.cpp contains multiple different functions that allow users to apply filters to the current capture.

Press “g” to apply greyscale filter:

this is a built-in OpenCV function allow user to apply grayscale filter to the current frame.

2. Press “h” to apply customized grayscale filter:

this function takes the original frame and convert it to grayscale.

3. Press “b” to apply Gaussian Blur filter:

this function takes the original frame and blurs the image 1×5 both vertically and horizontally by accesing pixels and color channels.

4. Press “x” to apply Sobel X filter:

this function applys Sobel X to the original colored image and convert to a Vec3s (signed short) format.

5. Press “y” to apply Sobel Y filter:

this function applys Sobel Y to the original colored image and conver to a Vec3s (signed short) format.

6. Press “m” to apply gradient magnitude from Sobel X and Sobel Y:

this function generates a gradient magnitude image by calculating the Euclidean distance from Sobel X and Sobel Y.

7a. Press “i” to blur and to quantize the image:

this function first applys a 5×5 Gaussian blur to the image and then set a default level of “15” to calculate the size of a bucket by accesing each pixel and color channel.

7b. Press “w” to blur and to quantize the image with a customized level:

this function applys 5×5 Gaussian blur to the image and then set a customized level (in this case, we have set the level to 10 by inputing cin level) value to calculate the size of a bucket by accesing each pixel and color channel.

8a. Press “c” to cartoonize the image:

this function first calculate the gradient magnite then blur and quantize the image with a default level value of “15”. Finally takes the blured and quantized image to outline the gradient magnitude. When each of the gradient maginitude’s color channel is bigger larger than the default value of “15”, we set the color channel to black and draw the outline of the blured and quantized image.

8b. Press “e” to cartoonize the image with customized level and threshold.

this function first calculate the gradient magnite then blur and quantize the image with a customized level value (in this case, we have set the level to 10 by inputing cin). Finally takes the blured and quantized image to outline the gradient magnitude. When each of the gradient maginitude’s color channel is bigger larger than the customized value (in this case, we have set the threshold to 10 by inputing cin threshold), we set the color channel to black and draw the outline of the blured and quantized image.

9. Press “r” to alter the brightness by customized value.

this function changes the brightness of the image by giving inputs. In this case, we have decressed the brighness of the original frame by 50. We first input “0” as negetive and then we input 50 value to the image.

Extension: Press “o” or “O” to preform a Blue Line Challenge

=>>>>>(my bicipes took too much steroids)

Both of the images are after effect result my cat moves too much..). The idea of the extension is from a Tiktok filter “Blue line filter”. As the blue line graduatly moving down and the upper content became still. I implemented this filter by first drawing a blue line and creating a temp frame with black background. As the blue line moving down, if the temp frame color is zero, the result would be same as the current frame; otherwise the result would be same as the temp frame.

To see the result, please check the mp4 file (timestamp: 01:54) in Video Processing (Project 1 Video Capture.mp4).

Video Processing: vidDisplay.cpp contains multiple different functions that allow users to apply filters to the cam frames.

*Because of the complicity of capturing videos, and the results are similar to applying filters to still images. Please check the below mp4 file to view all filters including the extension (Blue Line challenge).

All video processing are in vidDisplay.cpp. You can easily operate all filters to videos by pressing:

“G” for grayscale filter to video;

“H” for customized grayscale filter to video;

“B” for Gaussian blur filter to video;

“X” for Sobel X filter to video;

“Y” for Sobel Y filter to video;

“M” for gradient magnitude filter to video;

“I” for blur and quantize filters to video;

“W” for customized level blur and quantize filters to video;

“C” for cartonized filter to video;

“E” for customized level/threshold cartoonized filters to video;

“R” for brightness change to video.

“O” for Blue Line Challenge.

Project-1-Video-Capture.mp4

Project Reflection:

This project is very challenging for me because I have zero fundemental in Computer Vision, OpenCV, and very limited knowledge in C++. The first part of the assignment, to setup the enviornment took me an entire afternoon to config and to run the program. As I worked deeper on the tasks, the harder it gets. I had a lot of issues with data conversion throughout the project. I think It is very important to ask for helps. Dr. Bruce and the Piazza discussions provided me many helps as I working on the project, which I believe asking for helps is one of the most important things I learned.

In the begining of the project, the biggest challenge that I had was to do with data conversion. I had so much issues with showing the result and turns out I didn’t convert Vec3s (CV_16SC3) to Vec3b (CV_8UC3) since cv::imshow() only works with CV_8UC3. I spent tons of time on working with data conversion and I think the result is good because now I understand the concepts of how image is processed after accesing pixels and color channels.

I also had some struggles with cartoonization and turns out it was because I only assigned values to non-black pixels but never had a situation for anything other than 0s (basiclly I only had if statment but no else statment).

Almost when I finished the 9th tasks, cartoonization, I started working on filters on videos. And I spent hours on figuring out how to apply filters to frames(video). The way I approched was to make another for loop inside of the main for loop. I am not very sure if this is the most efficient way to do it but I think it works fine.

Overall I think the purpose of project is very important because it makes us to be familar with thje mechanism of manipulating image and frames. It is a very comprehensive project for the basic conncept.

Acknowledgement:

Thanks to Dr. Bruce Maxwell and the Piazza posts and discussions.

OpenCV tutorial, makefile tutorial, Cmake tutorial, and Cuda-Chen on Github: https://github.com/Cuda-Chen/opencv-config-with-vscode

Data conversion documents from Stackoverflow and OpenCV.

Tiktok Blue Line Challenge.

Project 1: Real-time filtering

Feb 1, 2021

Sida Zhang