Audiovisual Analysis

Audiovisual Analysis: Trump On Bilibili

August 2023

This project was based on previous findings that videos containing misinformation exhibited statistically significant discrepancies in both visual and audio quality when compared to reputable news sources. Extending on these findings, this project aimed to understand if there were similar correlations with positive or negative representations of Donald Trump on Chinese social media site Bilibili. The visual and audio attributes measured were brightness, warm or cool color dominance, entropy, audio tempo (BPM), and loudness.

An image of Donald Trump, who looks angry.

Face Analysis output. Reads: Is Trump: True. Dominant emotion: angry.

I began by scraping the data from previously collected Bilibili videos. Using the open source computer vision package OpenCV, I created a Python program that would extract video frames and audio data and process them using object-oriented programming. For each frame, the program would calculate the previously mentioned attributes and write them into a CSV file. For the next phase of this project, I will be identifying faces and analyzing their emotions using DeepFace AI.

Table with a sample of the output of the data scraping. The columns are: file name, dominant color,
brightness, entropy, tempo (BPM), loudness, and processing time (mins)

Reflections:
This project was another opportunity to learn new skills. I was given 3 research papers to learn the methodology for collecting audiovisual data, and then I had to apply it right away! This project was not without its frustrations - the first version of my Python program took over 12 hours to analyze a 30-minute video! Not ideal. Thankfully, I was able to greatly cut down the processing times in further iterations, as you can see in the table! I really enjoyed the challenge this project presented.