Measuring Film Genres

A few months ago, my friends got into an argument over whether Avengers: Infinity War is a fantasy or a science-fiction film. IMDb lists the movie under both genres, evolving the debate into whether the film is more fantasy or more sci-fi.

IMDb will tell you what genre any movie falls under, but not to what extent. That is, we can agree Inception and Star Trek are both sci-fi films, but who’s to say which is more sci-fi? And no, not which film is more futuristic or extraterrestrial - rather, which film better represents science-fiction as a genre and more strongly captures the elements commonly seen throughout the category. If you were to define “sci-fi” to a young child, which film would be the better example?

Up until now, there’s been no quantifying movie genre classifications. A movie is either a fantasy, or it isn’t. A movie is either a horror, or it isn’t. To get my friends to shut up about Avengers, that needed to change.

Ultimately, Avengers: Infinity War is much more sci-fi than it is fantasy. Having that knowledge, let’s explore the rest of Hollywood’s most popular hits; with data on 2,166 other films, there’s a lot of interesting genre compositions to see (more about the methodology below). Using the distance formula, we can also see which films have similar scores.

Interested in watching a movie that’s the perfect combination of all your favorite genres? Use the sliders below to find what best matches what you’re looking for. What would your favorite movie resemble with half the action? Or twice the fantasy? Remember that each genre is a part of a whole, so your total across all seven groups cannot exceed 100%.

Methodology

When reviewing a movie on IMDb, users can submit plot keywords to give the film a high-level description. I scraped these words for any movie with at least 50,000 user reviews and 150 submitted keywords. This resulted in data on 2,167 movies - only including films listed in the website’s action, comedy, drama, fantasy, horror, sci-fi, or thriller sections.

I then determined how often each keyword was used to describe films in each genre category (excluding keywords which were used to describe less than 2% of movies in the dataset, which removed overly-specific tags like “klingon” or “hogwarts”).

Most Common Words Used to Describe Each Genre

This ultimately did little to showcase unique features of genres and to distinguish them from one another, as most movie tropes are popular enough to span across all categories. “Murder,” for example, was used to describe more action movies than any other keyword, but it’s also a common occurrence in dramas, horrors, sci-fis, and thrillers. To uniquely describe “action,” we need the keywords which often describe action films but rarely describe films in other genres.

Most Genre-Specific Keywords

The most genre-specific keywords were determined by calculating the relative usage of each keyword for each genre. This helps evaluate how unique a keyword is to any genre. For example, “underwater scene” is used to describe action films 1.69x more often than other genres, sci-fi 1.43x more often, fantasy 1.42x more often, and all other genres less than the average usage. Meaning if we know a film has an underwater scene, we can presume it is likely more of an action, sci-fi, or fantasy than a horror, drama, or comedy. See how other keywords break down:

Finally, any movie’s genre scores was calculated using its keywords’ relative usage scores and summation. For example, any movie containing the “underwater scene” keyword had its action, sci-fi, and fantasy scores slightly incremented. This same process would be applied to any keyword for which relative usage scores were calculated. You can check out the data and .ipynb on GitHub.

Footnotes

1. For the purposes of this project, I did not include romance as a genre.
• I initially did include romance, only to find it correlated far too strongly with drama. This meant the data often failed to distinguish between the two genres, leading to lower relative usage scores and cannibalizing each other’s final genre scores. A similar phenomenon happened between action and adventure when adventure was included as its own genre.
• Romance is far more binary than action, fantasy, horror, etc. There is either a love story or there isn’t, making it pretty black and white. Compare this to action: a film can have a lot of action, absolutely no action, or anything in between. Romance is usually either there or it isn’t, with little room in the middle. Therefore, the romance genre doesn’t lend itself to gradient measurement very well.
2. The radar charts were created using a library written by Nadieh Bremer and updated by Matthieu Viry.
3. The sliders were created using a library written by John Walley.
4. The “Most __” Films were based on movies’ absolute genre scores used to calculate genre percentages, rather than the actual percentages themselves. This way, a film is not punished for scoring highly in more than one genres - such as RED 2, which has relatively lower action and thriller percentages because it has above average absolute scores in both categories. Scores are controlled for number of reviews received on IMDb, ensuring more popular films did not have an advantage over others.