Mythological names are on the rise; Pokémon, not so much.

 (Click to enlarge graphs)

As you can see, mythological boys' names were pretty negligible until the mid-1990s, after which they've had quite an explosion, with boys named Phoenix, Odin and Ares leading the pack. Girls having mythological names was more common than boys in the past, but they've increased as well, and the composition of the names has changed dramatically. In 1940, Minerva and Vesta were the most popular (a virgin Greek warrior goddess and a virgin Roman goddess of the hearth ... I'll let you draw your own conclusions from this). Now it's Isis, unfortunately for those who watch the news from Iraq these days.

Categorizing baby names is not straightforward; there's a judgment call involved. You can cast a broad net, and accept those called Amon, which is coincidentally both an Egyptian God's name and a Hebrew name, which pretty much makes the list meaningless because it's dominated by such names. Or you try to judge through semi-quantitative methods whether a name would, by a reasonable person in American society, be thought of as mythological.

So I limited the mythologies to Greek, Norse, Egyptian and Roman, because they're the most well-known mythologies in this culture. I had to pass on Celtic, because so many of their names are both mythological and common (like Brigid or Dylan). I used nameberry.com's database of name origin's to see whether names were mythological in origin or shared the name with other, more popular, traditions.

The graphs start at 1940 because even though the Social Security Administration publishes them back to 1880, the data is extremely unrepresentative in the early years.

The list of names I started with and which I eliminated at each step are in my GitHub repo, along with an IPython notebook of the code I used to analyze the data and make the graphs.


Gotta catch 'em all... okay, a few of 'em.



Were people named after Pokémon? Obviously, the reverse happens, since there's a Pokémon named Casey. These graphs are more jagged because the y axis is less than 10% that of the mythological names charts, and some names are rising above and dipping below the dataset's minimum of five babies in a given year.

It appears that only the boys' name Yadon appeared post-Pokémon; rather surprisingly, Lizardo appears once in 1970 before reappearing once in 2010. This dataset is riddled with errors, however, especially before digitial data entry, so it's quite possibly apocryphal.

Girls named Eevee, Amaura, Kimon and Kameil only appeared post-Pokémon; Abra has been around since the mid-'50s and enjoyed a brief surge around the time the Steve Miller Band's "Abracadabra" was playing on the radio. I remember those days; the lyric "I wanna reach out and grab ya" was pretty racy for Top 40 back then.

How often does a given letter follow another in English?

The following chart is an interactive heat map of the probability that, given the letter on the vertical axis in an English word, the next letter will be the letter on the horizontal axis.
Conditional probabilities can give you a headache; that's why the Monty Hall problem is so difficult. The best way to grasp it is by example. Look at the darkest point, for QU. This shows that, GIVEN Q, the next letter is U 98.7% of the time. Similarly, the dark spot on the bottom left shows that GIVEN Y, the most probable event is that there is NO letter following it (signified by "_"), i.e. it's at the end of a word.

The second graph below shows the reverse probabilities, e.g. given U, what's the probability that a Q precedes it, and given we're at the end of a word, what's the probability that the last letter is Y. Trust me, I know, it requires a little mental agility, I've been working on this for a week and I still get mixed up. If you can understand why, in the top graph, the horizontal rows add up to 100% but the vertical columns don't, you've totally got it.

A while back, I posted a series of charts about letter positions in English words. Nathan Yau of FlowingData was kind enough to write about it, and he suggested I look at letter proximity.

The source data is the COHA corpus of Historical American English; each word was analyzed and weighted as to their frequency (so the "th" in "the" influenced the probability of H following T way more than the "th" in theremin.)

Here's a GitGub repo with the code used to produce the data; after experimentation, both Plotly and Bokeh had serious drawbacks when it came to presenting heatmaps of this sort (which will presumably be addressed by later releases), so I went with Tableau Public, took about 20 minutes tops. Note that with this app, you can click things and hide things and have all kinds of fun.

Here's the graph of the probabilities of letters preceding, not following, one another. There are also static graphic versions at the very end. Enjoy!


Static versions (Click to enlarge):

   

Buck naked to butt naked, arms to anus, 19th century iPhones and other Google Ngram oddities

I've posted a couple times about the Google Books Ngrams Viewer before:
This data set is a rich vein for data mining. Plus it's almost completely uncurated, so it's a good target for data spelunking (that's my own idiolect for testing the boundaries of a dataset, to see what false conclusions it can appear to support). However, it's slow going because the metadata alone is really, really enormous (I've only got a fraction of it, and it's more than 3 terabytes). But as I peruse, I've come across some items of interest, totally non-systematically:

1. Butt Naked appears to be taking the place of Buck Naked
The etymology of the phrase "buck naked" is shrouded in mystery; some even think it's a Bowdlerization, and "butt naked" was the original term. But it's clear that in this corpus, anyway, "butt naked" is becoming more and more popular. I hypothesize that it's an example of elision (the k sound followed by the n sound is difficult to say, whereas the t becomes a glottal stop and rolls right off the... er, glottis.) Plus it does make a certain semantic sense: if you're naked, one can see your butt, no?
    Speaking of naked (my searches for this word seem to have influenced my Google AdWords profile, so I'm getting much racier suggestions online), this was a little surprising to me:

Does this mean we're getting more prurient, and less willing to discuss the absence of clothes? Probably not. The Google Books corpus is heavily weighted with 'Library Bias'; it reflects the contents of books it was able to scan in the mid-2000s. I believe a higher proportion of 19th-century books in the corpus are biblical or scientific compared to later books, and use the word less ashamedly.

2. OCR sometimes misreads 'arms' as 'anus'

I didn't come up with this observation of the fallible nature of Optical Character Recognition, but I haven't seen any Ngrams of it. This story got wide media coverage in May 2014, when someone noticed some old romance novels in Google Books contained phrases like this (click to enlarge):

Most of these Google Books examples are difficult to find individually in Google Ngrams viewer (but they're there, you just have to dig), because the exact search phrase has to appear more than 40 times in a year to be listed in their metadata. I first became aware of this risible phenomenon in 2009 thanks to this blog post, but it didn't get much traction at the time. So it goes.

3. 19th century iPhone?

I'm reasonably certain U.S. President Martin Van Buren didn't have an iPhone in the 1830s. Anachronisms in this data set sometimes come from documents being assigned the wrong pulibcation year, but that bias usually works in the opposite direction: A book writien in 1848 is reprinted in 1964, so it shows up in the database as a later year. In general, it's been my experience that modern terms in the past come from OCR errors (sometimes, as in this case, errors in word boundaries; there's a species of snake and a character in the Aeneid called Tisiphone that sometimes is rendered "tis iphone"... and then there are errors that are much less understandable, such as the following:
The OCR thinks "There" is "iPhone", with proper trademark capitalization? I posit there's a non-random error responsible for that.

I'll leave you with a few other 19th-century anachronisms (click to enlarge):




Sex ratio is the clearest indicator of bias in the baby names dataset


I've written before about how the U.S. Social Security baby names dataset, despite being trotted out by plenty of commercial websites aimed at partents, needs to be taken with a grain of salt, and a whole shaker of salt before the 1930s. This is just about the clearest graphical demonstration I've come up with.

It's impossible to quantify race ratio for the dataset, but since only certain occupations were allowed at first, and they excluded most of the occupations that were available for black men and women (for example, day labor and domestic work), it's safe to say the database is severely unbalanced in that regard as well.

Despite having an extensive work history in biology, I never knew that more male babies are born than female babies, a univeral phenomenon across the world (exacerbated by sex-selective abortions in some regions, unfortunately).

I've updated my previous Tableau Public storyboard on the limitations of the Social Security dataset to include this tidbit.

Full movies in 60 seconds: 70 animated gifs of films

Here are 70 animated gifs I've made over the past year, condensing full movies into about 60 seconds. I'll just list the first batch below, then I'll talk a little about the process, then I'll list the rest. Please note that these are about 10 MB each, so they can take a while to load.
Star Wars
    Star Wars Episode I: The Phantom Menace (1999) animated full movie gif
    Star Wars Episode II: Attack of the Clones (2002) animated full movie gif
    Star Wars Episode III: Revenge of the Sith (2005) animated full movie gif
    Star Wars (1977) animated full movie gif
    The Empire Strikes Back (1980) animated full movie gif
    Return of the Jedi (1983) animated full movie gif
    All Star Wars movies in 60 seconds animated full movie gif
    Star Wars Holiday Special (1978) animated full movie gif

First obvious question: why? I did these for the Reddit forum /r/FullMovieGifs. Why do (some) people like them? Because it gives a nice feeling to be quickly reminded of a favourite movie, and also it can reveal things not obvious in the full movie. For example, Paul Thomas Anderson likes really long takes where the camera slowly, slowly zooms in; this is way more evident when you watch the gif of Magnolia, below.

This was not just a matter of taking one frame every 10-15 seconds; I did my best to choose specific frames in order to give the best possible summary. I tried to linger on the most important and/or memorable scenes, and go quickly through ones that were less important or were a wasted frame (such as most establishing shots, where they show the building exterior before they show the characters in a room in a building; it's a good visual shorthand but useless in a gif), or were disorienting (long dialogue scenes that flip quickly back and forth between two people's faces are way more kinetic and disorienting in gif form, so they had to be cut down and the frames carefully chosen.

Here's an example of the frame rate of the Star Wars, Ep. IV (which I call Star Wars, 1977 above, 'cause I'm old and stubborn). You can see it's not very uniform, and if you click to enlarge you'll see I lingered on the most iconic scenes.


The Lord of the Rings
    The Lord of the Rings: The Fellowship of the Ring (2001) animated full movie gif
    The Lord of the Rings: The Two Towers (2002) animated full movie gif
    The Lord of the Rings: The Return of the King (2003) animated full movie gif
Star Trek
    Star Trek II: The Wrath of Khan (1982) animated full movie gif
    Star Trek Into Darkness (2013) animated full movie gif
    Star Trek (2009) animated full movie gif
    Star Trek (2009) [lens flares only] animated full movie gif
TV Specials
    A Charlie Brown Christmas (1965) animated full movie gif
    Dr. Seuss' How the Grinch Stole Christmas! (1966) animated full movie gif
    Rudolph the Red-Nosed Reindeer (1964) animated full movie gif
Sci-Fi
    A Trip to the Moon (1902) animated full movie gif
    2001: A Space Odyssey (1968) animated full movie gif
    Planet of the Apes (1968) animated full movie gif
    Alien (1979) animated full movie gif
    Blade Runner (Final Cut) (1982) animated full movie gif
    The Terminator (1984) animated full movie gif
    Terminator 2: Judgment Day (1991) animated full movie gif
    Contact (1997) animated full movie gif
    Predator (1987) animated full movie gif
    Robocop (1987) animated full movie gif
    28 Days Later (2002) animated full movie gif
    The Fountain (2006) animated full movie gif
    Inception (2010) animated full movie gif
Cornetto Trilogy
    Shaun of the Dead (2004) animated full movie gif
    Hot Fuzz (2007) animated full movie gif
    The World's End (2013) animated full movie gif
Anime
    Summer Wars (2009) animated full movie gif
Comedy
    Monty Python and the Holy Grail (1975) animated full movie gif
    The Rocky Horror Picture Show (1975) animated full movie gif
    Say Anything... (1989) animated full movie gif
    Ferris Bueller's Day Off (1986) animated full movie gif
    The Princess Bride (1987) animated full movie gif
    Groundhog Day (1993) animated full movie gif
    Fargo (1996) animated full movie gif
Drama/Action
    Citizen Kane (1941) animated full movie gif
    Casablanca (1942) animated full movie gif
    Rashomon (1950) animated full movie gif
    The GodFather, Part II (1974) animated full movie gif
    Barry Lyndon (1975) animated full movie gif
    Jaws (1975) animated full movie gif
    One Flew Over the Cuckoo's Nest (1975) animated full movie gif
    Apocalypse Now (1979) animated full movie gif
    Raiders of the Lost Ark (1981) animated full movie gif
    Scarface (1983) animated full movie gif
    The Silence of the Lambs (1991) animated full movie gif
    Reservoir Dogs (1992) animated full movie gif
    Schindler's List (1993) animated full movie gif
    The Shawshank Redemption (1994) animated full movie gif
    The Usual Suspects (1994) animated full movie gif
    Trainspotting (1996) animated full movie gif
    The Big Lebowski (1998) animated full movie gif
    Magnolia (1999) animated full movie gif
    Memento (2000) animated full movie gif
    Memento (2000) in chronological order animated full movie gif
    Ocean's Eleven (2001) animated full movie gif
    Y Tu Mama Tambien (2001) [NSFW] animated full movie gif
    Eternal Sunshine of the Spotless Mind (2004) animated full movie gif
    The Prestige (2006) animated full movie gif
    300 (2006) animated full movie gif
    No Country for Old Men (2007) animated full movie gif
    There Will Be Blood (2007) animated full movie gif
    Scott Pilgrim vs. the World (2010) animated full movie gif