Jonathan Dobres

I work with lots of data. Not what you’d call “big data”, at least not technically, but maybe “biggish”. More than enough that Excel would crash just trying to open the dataset, assuming you were foolish enough to try. The amount of data is voluminous enough, and the relationship between the raw data and what you’re trying to analyze complex enough, that you need pretty decent data management chops to even access it correctly. But let’s say you have accessed it correctly. Now you can proceed to perform your analysis, make data visaulizations, and be a sorcerer of the digital age. Right?

Wrong. You left out the most important step: getting your data into the right format, making sure each data point has all the right labels in the right places to allow you to proceed to the real science. Cleaning data—that is, pulling in the unprocessed data, transforming it, rearranging it, relabeling it, discarding garbage, and otherwise getting it into a format that will play nicely with your analysis tools—is easily 90% of the job.

Let me give you an example.

This is a plot of the hours of daylight (sunset time subtracted from sunrise time) that Boston, Massachusetts received throughout 2014. I got the data from the US Naval Observatory after reading this post about the merits of Daylight Savings Time. Request a data file for any location in the US, and you’ll find it looks like this (scroll the box rightward to see the whole thing):

             o  ,    o  ,                                BOSTON, MASSACHUSETTS                         Astronomical Applications Dept.
Location: W071 05, N42 19                          Rise and Set for the Sun for 2014                   U. S. Naval Observatory        
                                                                                                       Washington, DC  20392-5420     
                                                         Eastern Standard Time                                                        
                                                                                                                                      
                                                                                                                                      
       Jan.       Feb.       Mar.       Apr.       May        June       July       Aug.       Sept.      Oct.       Nov.       Dec.  
Day Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set  Rise  Set
     h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m   h m  h m
01  0713 1623  0658 1659  0620 1734  0527 1810  0440 1844  0410 1914  0411 1925  0437 1904  0510 1818  0541 1726  0618 1638  0654 1613
02  0714 1624  0657 1700  0618 1735  0525 1811  0438 1845  0410 1915  0412 1925  0438 1903  0511 1817  0543 1724  0619 1637  0655 1613
03  0714 1624  0656 1701  0616 1737  0524 1812  0437 1846  0409 1916  0413 1924  0439 1901  0512 1815  0544 1722  0620 1635  0656 1612
04  0714 1625  0655 1702  0615 1738  0522 1814  0436 1847  0409 1917  0413 1924  0440 1900  0513 1813  0545 1721  0621 1634  0657 1612
05  0713 1626  0653 1704  0613 1739  0520 1815  0435 1848  0409 1917  0414 1924  0441 1859  0514 1811  0546 1719  0623 1633  0658 1612
06  0713 1627  0652 1705  0612 1740  0518 1816  0433 1849  0408 1918  0414 1924  0442 1858  0515 1810  0547 1717  0624 1632  0659 1612
07  0713 1628  0651 1706  0610 1741  0517 1817  0432 1850  0408 1919  0415 1923  0443 1856  0516 1808  0548 1716  0625 1631  0700 1612
08  0713 1629  0650 1708  0608 1743  0515 1818  0431 1852  0408 1919  0416 1923  0444 1855  0517 1806  0549 1714  0626 1629  0701 1612
09  0713 1630  0649 1709  0607 1744  0513 1819  0430 1853  0408 1920  0416 1922  0445 1854  0518 1805  0550 1712  0628 1628  0702 1612
10  0713 1632  0647 1710  0605 1745  0512 1820  0428 1854  0407 1920  0417 1922  0446 1852  0519 1803  0551 1711  0629 1627  0702 1612
11  0712 1633  0646 1712  0603 1746  0510 1821  0427 1855  0407 1921  0418 1921  0447 1851  0520 1801  0553 1709  0630 1626  0703 1612
12  0712 1634  0645 1713  0601 1747  0508 1823  0426 1856  0407 1921  0419 1921  0448 1850  0521 1759  0554 1707  0631 1625  0704 1612
13  0712 1635  0644 1714  0600 1749  0507 1824  0425 1857  0407 1922  0419 1920  0450 1848  0522 1758  0555 1706  0633 1624  0705 1612
14  0711 1636  0642 1715  0558 1750  0505 1825  0424 1858  0407 1922  0420 1920  0451 1847  0523 1756  0556 1704  0634 1623  0706 1612
15  0711 1637  0641 1717  0556 1751  0504 1826  0423 1859  0407 1923  0421 1919  0452 1845  0524 1754  0557 1702  0635 1622  0706 1613
16  0710 1638  0639 1718  0555 1752  0502 1827  0422 1900  0407 1923  0422 1919  0453 1844  0525 1752  0558 1701  0636 1622  0707 1613
17  0710 1640  0638 1719  0553 1753  0500 1828  0421 1901  0407 1923  0423 1918  0454 1842  0526 1751  0559 1659  0637 1621  0708 1613
18  0709 1641  0637 1721  0551 1754  0459 1829  0420 1902  0407 1924  0424 1917  0455 1841  0527 1749  0601 1658  0639 1620  0708 1614
19  0709 1642  0635 1722  0549 1755  0457 1830  0419 1903  0407 1924  0424 1916  0456 1839  0529 1747  0602 1656  0640 1619  0709 1614
20  0708 1643  0634 1723  0548 1757  0456 1832  0418 1904  0408 1924  0425 1916  0457 1838  0530 1745  0603 1655  0641 1618  0709 1614
21  0707 1644  0632 1724  0546 1758  0454 1833  0418 1905  0408 1925  0426 1915  0458 1836  0531 1743  0604 1653  0642 1618  0710 1615
22  0707 1646  0631 1726  0544 1759  0453 1834  0417 1906  0408 1925  0427 1914  0459 1835  0532 1742  0605 1652  0644 1617  0711 1615
23  0706 1647  0629 1727  0543 1800  0451 1835  0416 1907  0408 1925  0428 1913  0500 1833  0533 1740  0607 1650  0645 1617  0711 1616
24  0705 1648  0628 1728  0541 1801  0450 1836  0415 1908  0409 1925  0429 1912  0501 1832  0534 1738  0608 1649  0646 1616  0711 1617
25  0704 1650  0626 1729  0539 1802  0448 1837  0414 1909  0409 1925  0430 1911  0502 1830  0535 1736  0609 1647  0647 1615  0712 1617
26  0704 1651  0624 1731  0537 1803  0447 1838  0414 1910  0409 1925  0431 1910  0503 1828  0536 1735  0610 1646  0648 1615  0712 1618
27  0703 1652  0623 1732  0536 1805  0445 1839  0413 1910  0410 1925  0432 1909  0504 1827  0537 1733  0611 1644  0649 1614  0712 1619
28  0702 1653  0621 1733  0534 1806  0444 1841  0412 1911  0410 1925  0433 1908  0505 1825  0538 1731  0613 1643  0650 1614  0713 1619
29  0701 1655             0532 1807  0443 1842  0412 1912  0411 1925  0434 1907  0506 1823  0539 1729  0614 1642  0652 1614  0713 1620
30  0700 1656             0530 1808  0441 1843  0411 1913  0411 1925  0435 1906  0507 1822  0540 1728  0615 1640  0653 1613  0713 1621
31  0659 1657             0529 1809             0411 1914             0436 1905  0509 1820             0616 1639             0713 1622

                                             Add one hour for daylight time, if and when in use.

Getting that plot out of this data turns out to be a little tricky, and most of the trick is in the import and cleanup phases. Right now, the data are arranged such that the day of the month is on the rows, while the month, hour, minute, and sunrise/sunset label are on the columns. This is often called “wide” data, which is easy to look at, but usually hard to work with. Our goal is to create a “long” dataset in which each row holds a single timestamp corresponding to one day’s sunrise or sunset (essentially, two rows per day). I’m going to show you how to do it using R. You’ll also need the following R packages: reshape2, plyr, and lubridate.

First things first, we need to import the data, ideally so that each meaningful number (the hours and minutes for each day of the year) ends up in a neat column. While the double-nested headers are unfortunate (hour and minute are nested within sunrise/sunset, which are nested within month), at least the data follow a nice fixed-width format, with each column ending after a predictable number of characters. R happens to have a handy read.fwf function, which is specialized for reading in these types of files.

data.raw <- read.fwf(
  file='Boston Daylight Data 2014.txt', 
  skip=9, 
  nrows=31,
  colClasses='numeric', 
  strip.white=T,
  widths=c(2, rep(c(4, 2, 3, 2), 12))
);

The read.fwf command accomplishes a lot, so I’ve spread its arguments out over several lines. I’m telling the function to read in the file, skip its first nine rows (none of which contain data), read exactly the next 31, make sure to import all the columns as numbers (not text strings), strip out any extra whitespace, and lastly, how many characters wide each column should be. This produces a dataset that looks like this (I’m cutting out a lot of the data, but there are a total of 49 columns and 31 rows):

 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20
  1  7 13 16 23  6 58 16 59   6  20  17  34   5  27  18  10   4  40  18
  2  7 14 16 24  6 57 17  0   6  18  17  35   5  25  18  11   4  38  18
  3  7 14 16 24  6 56 17  1   6  16  17  37   5  24  18  12   4  37  18
  4  7 14 16 25  6 55 17  2   6  15  17  38   5  22  18  14   4  36  18
  5  7 13 16 26  6 53 17  4   6  13  17  39   5  20  18  15   4  35  18
  6  7 13 16 27  6 52 17  5   6  12  17  40   5  18  18  16   4  33  18

Now we just need to name the columns:

colnames(data.raw) <- c('day', paste(rep(month.abb, each=4), rep(c('rise', 'set'), each=2), c('hour', 'minute')));

This is a somewhat confusing use of the paste function, but basically I’m creating a vector of names: the first one is “day”, followed by names that follow the convention “Month rise/set hour/minute” (for example, “Jan rise hour”). Creating the labels at this stage saves us the trouble of having to extract them later.¹ Our next step is to melt the dataset.

data.daylight <- melt(data.raw, id.vars='day');

By default, melt wants to reduce a dataset to just two columns: “variable” and “value” (“variable” becomes a column containing the dataset’s former column names, and “value” stores their corresponding values). The columns specified in id.vars are preserved in the new data frame, and are not melted into the “variable” column. So now our dataset looks like this:

 day      variable value
   1 Jan rise hour     7
   2 Jan rise hour     7
   3 Jan rise hour     7
   4 Jan rise hour     7
   5 Jan rise hour     7
   6 Jan rise hour     7

Now I want to take my “variable” column and split it into three new columns: month, event (sunrise/sunset), and time (hour/minute). This is easily done with colsplit. Note that I’m combining it with cbind, so that I can attach the new columns to my dataset without creating a temporary variable.

data.daylight <- cbind(data.daylight, colsplit(data.daylight$variable, ' ', c('month', 'event', 'time')));

Which makes the data look like this:

 day      variable value month event time
   1 Jan rise hour     7   Jan  rise hour
   2 Jan rise hour     7   Jan  rise hour
   3 Jan rise hour     7   Jan  rise hour
   4 Jan rise hour     7   Jan  rise hour
   5 Jan rise hour     7   Jan  rise hour
   6 Jan rise hour     7   Jan  rise hour

We’re nearly there. All that’s left is to get each event’s hour and minute into the same row. As near as I can tell, there’s no better way to do it than with the handy dcast function. With this function, I’m saying that “month”, “day”, and “event” should define the rows, while the different values stored in “time” should form new columns.

data.daylight <- dcast(data.daylight, month + day + event ~ time);

 month day event hour minute
   Apr   1  rise    5     27
   Apr   1   set   18     10
   Apr   2  rise    5     25
   Apr   2   set   18     11
   Apr   3  rise    5     24
   Apr   3   set   18     12

From importing the data to this use of dcast, I’ve only written five lines of code. Now would be a great time to scroll back up and remember how the data looked originally. I’ll wait.

And that’s what I call “the first 90%”. The data are now in a highly flexible “long” format, and can be used with ease. For example, say we wanted to a) convert the “month” and “day” columns into proper Date data, which will make plotting much easier, and b) calculate the minute of the day at which the sunrise/sunset event occurred. Enter mutate, the easiest way to do this kind of transformation (with a call to lubridate’s ymd function to turn strings of numbers into Dates):

data.daylight <- mutate(data.daylight,
                      date=ymd(paste('2014', month, day)),
                      event.minute=hour * 60 + minute);

 month day event hour minute       date event.minute
   Apr   1  rise    5     27 2014-04-01          327
   Apr   1   set   18     10 2014-04-01         1090
   Apr   2  rise    5     25 2014-04-02          325
   Apr   2   set   18     11 2014-04-02         1091
   Apr   3  rise    5     24 2014-04-03          324
   Apr   3   set   18     12 2014-04-03         1092

Think about how tedious and error-prone it would have been to create the equivalent of the “date” and “event.minute” columns with the data as originally formatted. But now we’re getting into what I call “the other 90%”, which is another story for another time.

Number 8: Human Nature / The Family of Blood

Some might say that including two-part stories in a top ten list is cheating. I would tell those people to go write their own top ten list.

“Human Nature” was originally a Seventh Doctor story, if you can believe it. It started life as a New Adventures novel published way back in 1995. These novels filled the gap during the “wilderness years” between the old show and the new, and many of modern Who’s most influential voices—your Stevens Moffat, Marks Gatiss, and Russells T. Davieseses—cut their teeth writing these semi-canonical stories. They often explored darker, more complicated, more internal subject matter, and “Human Nature”, written by Paul Cornell (and eventually adapted by him for these 2007 episodes), is widely considered to be one of the best.¹

It’s not hard to imagine Sylvester McCoy at the center of “Human Nature”. His Doctor was by far the most professorial of the eleven we’ve seen so far, so much so that his companion usually called him “Professor”, rather than “Doctor” (much to his chagrin). The Seventh Doctor’s attitude was defined by a kind of winking propriety that would have been a natural fit for a period piece like this one. McCoy is easily the most talented actor to portray the Doctor pre-reboot, and he did so in its worst days, repeatedly polishing nonsensical garbage into something almost enjoyable. I would have loved to see him tackle a legitimately great story like this one, especially since the John Smith persona seems to have grown directly from the Seventh Doctor: pompous, condescending, nerdy, but ultimately a force for good, and not afraid of the occasional daydream.

Though the characterization really does seem made for McCoy, David Tennant is a great actor in his own right (Broadchurch, anyone?), and he does a terrific job with this very challenging material. Smith is the answer to the fan-fickish question, “If the Doctor were human, what sort of human would he be?” The answer we get is a surprisingly nuanced mixed bag. Smith is smaller, softer, less grand, and undeniably marked by his time. He casually gives permission for one student to beat another as a disciplinary measure and repeatedly dismisses Martha as his social inferior. Yet he is capable of great emotion, finding a love in Joan Redfern (another great guest role, courtesy of Jessica Hynes). There is also an undeniable streak of heroism in him, whether saving a baby from an imminently collapsing piano (really?), doing his duty to protect his charges, or ultimately, giving his life to restore the Doctor. As Redfern says to the Doctor, “He was braver than you, in the end.”

Both the Doctor and the Family of Blood are masquerading as humans, but their reactions to their disguises couldn’t be more different. The Family of Blood, particularly Son of Mine, view the whole enterprise as an ugly necessity to be cast off as quickly as possible. Harry Lloyd² plays Son of Mine completely unhinged, and yet it works. His knowledge of the human race’s immediate future makes all their small-minded posturing hilarious to him, and nearly every word he says to the humans is uttered as an acidic mockery. Meanwhile, John Smith clings to his humanity, desperately trying to find a way around the contents of the pocket watch and his inevitable transformation.³ “Why can’t I be John Smith,” he asks through tears at one point, “isn’t he a good man?”

The climax of the story comes at the very end, not when the Doctor confronts the Family (once he’s restored, they’re easily dispatched), but when he returns to Redfern. It’s the opposite of a regeneration in every possible way. Same body, different person. Not a chaotic new beginning, but a willful ending:

JOAN: Could you change back?

DOCTOR: Yes.

JOAN: Will you?

DOCTOR: No.

Tennant’s delivery in this final scene is quiet and flat, and all the more devastating for it. He can’t love Joan, he’s not interested in trying, and he wants to get out of 1913 before he causes more damage. “Come with me,” he offers, almost off-handedly. “We could start again–I’d like that! We could try, at least.” The offer is perfunctory. He knows she won’t accept, and she knows that whatever love John Smith felt for her has been lost in the wake of the Doctor. She is now a war widow twice over. She comes to her own conclusions about the man in front of her, asking him, “If the Doctor had never visited us, if he’d never chosen this place—on a whim—would anybody here have died?” The Doctor’s only reply is silence; they both know the answer. Finally, Joan dismisses him with a simple, “You can go.”⁴

This short, quiet scene lays bare some of the darkest aspects of the Doctor’s character. Thrilling as his adventures may be, they rather often get innocent people killed, as illustrated so brutally here (and hinted at throughout much of the Davies era in episodes like “Rose” and “Love and Monsters”). The Doctor, as Tom Baker once famously intoned, “walks in eternity”, a being far greater than any one particular place and time. He is mythic (just look at how he punishes the Family), a larger than life figure who, tellingly, will never shrink himself down for the love of one woman on one planet in one tiny corner of the vast universe.⁵ He is a force for good, but far from a perfect one, and not one that we can ever truly understand. By skillfully illustrating the Doctor’s enormity as a tragic contrast with the unlived life of John Smith, “Human Nature” / “The Family of Blood” earns the #8 spot.

An Aside: Martha Jones

I couldn’t quite work it into the main write-up, but it’s worth discussing how this story portrays Martha, my favorite companion. On the one hand, these episodes come at the nadir of her one-sided crush on the Doctor. She’s sacrificed everything to keep him safe: her profession, her family, her social status, in short, for a couple of months in 1913, she gives up her life. Yet she can only lament her love for him. “You had to go and fall in love with a human, and it wasn’t me!” she cries, pathetically, to a video image. On the other hand, some of my absolute favorite Martha moments happen in this story. It is she who holds off the Family in a Mexican standoff at the mid-story climax. And it takes her less than a minute to suspect and then confirm that her friend has been possessed by the Family (keep this in mind the next time you watch Rose talk to her obviously plastic boyfriend in “Rose”).⁶ In this episode she was very much the action hero, with John Smith filling the role of the weepy damsel, and it was great.

Number 9: Vincent and the Doctor

I’m as surprised as you are, really. “Vincent and the Doctor”, coming toward the end of Matt Smith’s first season, is among the most polarizing episodes in Who history. Some love it for daring to wear its heart on its sleeve and largely succeeding, while others deride it for its overwhelming sentimentality and shaky plot.

I was in the latter camp at first, as the episode does poorly as a piece of straight science fiction. The Krafayis is an unconvincing monster that, for basically no reason, is portrayed as a genuine threat against a Doctor who has faced Silurians, Daleks, Weeping Angels, and other assorted alien armies in this season alone. As gimmicks go, “it’s invisible and very violent” is a rather unimaginative concept, and it’s not like we can empathize with a creature that does little more than some implied thrashing (the CGI is a real limitation here) and angry roaring. Smith is given very little to do other than waste time, and he plays most of the episode in full tilt spastic teenager mode. While the previous three episodes in this season—“Amy’s Choice”, “The Hungry Earth”, and “Cold Blood”—managed to weave together their monsters of the week with their moral quandaries, the Krafayis feels like an afterthought. You can summarize the action beats of this episode as “Vincent Van Gogh fights an invisible space chicken.”¹

However, this episode is a personal favorite of a friend of mine, who forced me to sit through it again. I don’t know what happened between the first and second viewings (spoiler alert: you’re about to find out), but it was like watching a completely different episode. That second chance revealed three things that make “Vincent and the Doctor” worthy of the #9 spot.

First, the episode harkens back to Doctor Who’s roots as a sort-of-educational program. The series’ very first story, “An Unearthly Child”, found the Doctor and his companions in the paleolithic era, helping cavemen rediscover the art of fire, while the second story, “The Daleks”, featured the Doctor expounding on the particulars of static electricity to explain how Daleks move. In much the same way, we are educated on the life of Van Gogh in the academic sense (courtesy of Bill Nighy, in an uncredited role as the Van Gogh exhibit’s curator) and in the personal, as we get a glimpse of the artist’s daily struggles and inner torment.

Second, this episode is chock full of wonderful performances. Granted, Smith isn’t given much to work with, but Tony Curran does an amazing job as Van Gogh, keeping his appearance memorable without descending into caricature. It’s easily one of the best guest roles in the entire series.² Nighy is terrific in his small role, as he tells the audience why Van Gogh is considered to be one of history’s greatest artists, and manages to sell every word. Karen Gillan, for her part, has a chemistry with Tony Curran that she never really developed in all her time with Arthur Darvill. Amy Pond usually comes off as an overly aggressive combination of sassy/sexy/pixie, but here Gillan dials it back enough to affect genuine charm. Amy becomes increasingly concerned for Van Gogh’s wellbeing as the story progresses, and really seems to want to make his life better than history had left it. Where Gillan usually places Amy at some remove from the historical figures she meets (“Oi! Churchill!”), here she seems emotionally invested. You can almost believe that she might have stayed behind to become Mrs. Van Gogh.

Third, this episode really lays on the shlock,³ and yet somehow doesn’t collapse under all of that emotional weight. This is the key thing to understand about the episode: the Krafayis really is an afterthought, and it’s all about Van Gogh. More specifically, it’s trying to answer the question, “Why art?” The show makes the case that Van Gogh was especially perceptive (he’s the only person in the world who can see the Krafayis, and he senses Amy’s sadness over an event that she herself cannot remember), and that this enabled him to paint things in a way that no person before him had ever mastered. “Vincent and the Doctor” lays out, pretty explicitly, what made Van Gogh’s art so true and resonant for the ages. To review: this episode explains art, for God’s sake, and does so successfully!

Then there’s Van Gogh’s trip into the future, where he learns, in no uncertain terms, that his life’s work, the source of so much humiliation and anguish, was worth it after all. This is pretty big philosophical territory. How many of us have grappled with that question ourselves? Is what I’m doing important? Does it matter? Will it matter? Here, Doctor Who was brave enough to imagine what would happen if someone worthy of the question found out the answer. The consequences are decidedly Who-ish; a few small tweaks to the timeline, but things stay mostly the same. The Doctor and Amy gave Van Gogh a moment of beauty (after all, what is art if not that?), but it wasn’t enough to vanquish all his demons or prevent the inevitable. As the Doctor says, putting perhaps too fine a point on it, “every life is a pile of good things and bad things…the good things don’t always soften the bad things, but vice versa, the bad things don’t necessarily spoil the good things or make them unimportant. And we definitely added to his pile of good things.”

So, what changed between my first and second viewings? The first time around, I wasn’t willing to see the episode on its own terms. It wanted to tell a story from the heart, a weird yarn that begins when the Doctor spots a monster in a painting and ends when we learn why art matters. I just wasn’t in the mood for it, couldn’t tune in on the emotional frequency the episode asks of the audience. But that is exactly the type of show Doctor Who can be, if you’re willing to let it. Bear in mind that this episode came right after “Cold Blood”, in which the Doctor faced off against a civilization of intelligent lizards (and racism) and lost Amy’s fiancé to a crack in the universe. That the show could successfully shift gears to a big-hearted flight of fancy like “Vincent and the Doctor” is a testament to Doctor Who’s flexibility and nerve as a storytelling vehicle. What other show could have possibly pulled off an episode that tackles these types of artistic and philosophical aspirations? And that’s why “Vincent and the Doctor” makes the #9 spot.

It’s been a bad year for Doctor Who, no question. It’s hard to view the seventh season as anything other than disappointing, with its boring, do-nothing episodes, nonsensical melodramas, and huge buildups that went nowhere. But any show that’s been on for half a century is going to wax and wane, and hope springs eternal for the stalwart Doctor Who fan, especially with a new Doctor on the way.

With the news that the eighth season of Doctor Who will premiere in August, I thought it might be fun to write a rundown of my personal picks for the ten best episodes of the reboot. Any such list is subjective, of course, and you’re free to disagree with me. Bearing that in mind, off we go to #10 on my list: “The End of the World”.

Number 10: The End of the World

Doctor Who didn’t exactly leave television on the best terms in 1989. Its final few years on air were marked by infighting at the BBC and borderline incompetent creative decisions. The final story, “Survival”, is a boring piece of nothing about a race of cheetah-people who reside in a parallel universe, and also the Master is there. It was a hasty and ignominious swan song for a program that had once been seen as innovative, experimental, and captivating.

Russell T. Davies certainly had his work cut out for him when he set out to reboot the show in 2005, a task made even harder by the decision to treat it as continuous with the previous twenty-six seasons of material. Davies made a great start with the opening story, “Rose”. In fact, “Rose” very nearly made my top ten list, but it feels more like an episode of Davies’ eventual spin-off, Torchwood, with its big explosions in the middle of heavily populated areas, a street level view of fantastical events, and a climactic set piece that doesn’t quite deliver on what it’s promising.

Instead, it’s the reboot’s second episode, “The End of the World”, that perfectly bridges the old and the new. In many ways, it feels like an episode straight out of the old series. There are a dozen monsters in rubber suits, some hokey musical cues, an obvious villain (spoiler alert: it’s the character with the most lines, after the Doctor and Rose), and a cinch ending that amounts to the Doctor deciding it’s time for him to win.

At the same time, these old-school sci-fi tropes coexist with some decidedly new elements. The Doctor comes to the year five billion on a lark, literally just to prove that he can. Rose, however, is overwhelmed, at first by the strangeness of the future, and then by the sudden realization that some lunatic in a leather jacket just invited her into his van, and she hopped in without giving it a lot of thought.

Rose’s anxiety comes to a head in my favorite scene, which is worth reading in full:

ROSE: Where are you from?

DOCTOR: All over the place.

ROSE: They [the aliens she’s met] all speak English.

DOCTOR: No, you just hear English. It’s a gift of the TARDIS. The telepathic field, gets inside your brain and translates.

ROSE: It’s inside my brain?

DOCTOR: Well, in a good way.

ROSE: Your machine gets inside my head. It gets inside and it changes my mind, and you didn’t even ask?

DOCTOR: I didn’t think about it like that!

ROSE: No, you were too busy thinking up cheap shots about the Deep South! Who are you, then, Doctor? What are you called? What sort of alien are you?

DOCTOR: I’m just the Doctor.

ROSE: From what planet?

DOCTOR: Well, it’s not as if you’ll know where it is!

ROSE: Where are you from?

DOCTOR: What does it matter!

ROSE: Tell me who you are!

DOCTOR: This is who I am, right here, right now, all right? All that counts is here and now, and this is me.

It’s especially interesting to watch this scene in light of Eccleston’s successors in the role. Faced with a distraught companion, David Tennant’s Doctor would have winked and charmed, and Matt Smith would have fumbled and distracted, but Eccleston’s Doctor gets angry. The manic show-off who took Rose on a field trip to the year five billion is really just a cover for the damaged, bitter refugee lurking underneath. In fact, the Doctor didn’t take Rose to the year five billion. He took her to see the final destruction of her world, which happens to be in the year five billion. This says more about the Doctor’s character and the Time War, only barely hinted at here, than any grandiose monologue ever could.¹ The scene also exposes what a tremendous force Eccleston brought to the role, and really makes me miss him.

The moment hangs in the air unresolved. Rose drops the issue without ever getting an apology or an explanation of the Doctor’s motives. The Doctor does, however, upgrade her cellphone (how quaint!) thus allowing her to do what everyone wants to do when they’re scared: call Mom. It’s a touching interlude that anchors Rose and cools the tension from moments before. It’s also another signature of the reboot, which, unlike the old show, often moves the story along via emotional beats instead of a series of narrative events.

The villain of the piece, the Lady Cassandra O’Brien.Δ17, a.k.a. the Last Human, also provides a deftly balanced mix of old and new. Her villainy is broad, obvious, and laughable. But more than just an old-school vamping egomaniac, the Last Human is an elitist and a racist, proudly clarifying that she is “the last pure human”.

The message, which Cassandra makes thuddingly obvious (in true Old Who fashion), is that racism and classism are bad. This theme is also conveyed much more subtlely by Raffalo, the pleasant blue plumber who must ask Rose for permission to speak before actually doing so. The Doctor also expresses this theme in a bit of off-handed dialogue, where he explains that “the great and the good are gathering to watch the planet burn,” and that by the great and good, he means “the rich.”

“End of the World” represents not just a re-imagining of Doctor Who, (as “Rose” does), but a maturation. The show retains its silly rubber suits,² its fantastical settings,³ and its Doctor’s sense of smug superiority. But this new Doctor also carries an anger, even a fatalism, not seen in his predecesssors. Eccleston grins and giggles in the face of the Last Human’s grotesque appearance, showing concern only as Rose becomes more uncomfortable. When Cassandra finally faces the music, the Doctor simply says, “Everything has its time and everything dies,” a statement that also applies to the Earth, and certainly, his own people. The new show operates on an emotional frequency that the old show almost never tapped. The Doctor opens the episode by musing on the human race’s improbable, incredible survival, and closes it by reflecting on the destruction of his own world, and the inevitably of the Earth doing the same, regardless of whether humanity survives. This is big territory for the show to handle in its second episode, and, as would become the hallmark of the Davies era, it does so in a way that satisfies the heart and leaves the mind thirsting for the next adventure.

Have you ever owned something—a pizza cutter, let’s say—and you thought to yourself, “I know I don’t use it often, but it’s probably worth keeping around for later?” So you put the pizza cutter in the kitchen drawer, and you sort of forget about it. Sort of, but not quite. It’s never entirely out of your thoughts, but you just can’t think of a good reason to pull it out. And then when you finally have a reason to use it, you realize that maybe this pizza cutter, which is shaped like the starship Enterprise, incidentally, isn’t exactly appropriate to your needs.

That’s what happened with me and this website. I left it dormant for so long that by the time I started thinking about it again, I realized that it needed more than a new coat of paint.

Total Realignment

My website has been silent for the last two years not because I’ve been bored, but because my job has kept me very busy. It’s also made me very productive, to the point that I began to reimagine the website as a showcase for my professional output and what I increasingly think of as the areas of my expertise. The time has come, then, to transition the site from a glorified blog to a professional portfolio, plus blog.¹

It’s common knowledge that most academic/scientist personal sites are rarely updated, poorly maintained jokes. When I set out to redesign my own site, I had to think carefully about what the ideal “personal academic website” might look like. What problems does such a site need to solve?

Above all else, the site should rapidly communicate who I am and what I do. I address this with the front page, which is designed to function as a kind of business card. Want to know who I am in ten seconds or less? Read the tweet-length blurb and then look at the pretty picture. Have a full minute to kill? Scroll down. Bored at work? You can click to dive deeper, which will take you to my brief biography, a description of my research interests,² a nicely formatted list of my publications, or this very blog.³

The visual overhaul of the site reflects new priorities. If I’m going to present myself as an expert on human factors and design issues, I’d better be able to walk the walk, right? I designed and coded the site myself, as I have done since the 90s. The redesign also includes a responsive stylesheet for mobile devices, so check it out on your smartphone or make your browser window suitably tiny. Following the recommendations of the talented and knowledgable Hawke Bassignani, body text is set in the serious-but-not-too-serious Merriweather, while Open Sans is used for headings and navigation. Lastly, the site is Retina-ready, using high resolution graphics and font-based icon sets (courtesy of Ico Moon) wherever possible.

A New Foundation

Longtime visitors (all five of you) might have noticed that the site feels a little leaner. That’s because I’ve rebuilt the whole thing with Jekyll, ending a nearly decade-long love/hate relationship with WordPress. Over the years, WordPress has grown to become the user-friendly front end to about a quarter of the web, and while that’s been a boon for most users, it has also made WordPress’s internals extremely difficult to understand. Developing a proper WordPress theme from scratch is a full-time job, and even simply deactivating the pieces that I don’t need in existing themes is difficult and fraught with peril.

Jekyll, on the other hand, makes it relatively easy to do things like store my site’s front page content in a way that makes sense, or create a custom template without weeding through three dozen esoteric PHP calls. At the end of the day, it’s a simpler system. It carries a lot of other benefits as well; no security holes to patch, no comment spam to manage, easy, human-readable data storage, and I can write my posts in Markdown, which is simply a joy to use.⁴

We will, however, be permanently closing comments. Jekyll doesn’t do comments. I suppose I could use a service like Disqus to fill the gap, but I’d just be trading PHP overhead for Javascript overhead. Though I have enjoyed reading comments over the years, I can’t say that I’ll miss them. If you really want to comment on something I’ve written, feel free to send me an email, or click one of the tasteful social networking icons that adorn the individual article pages.

Writing is Fundamental

While mulling over the details of this grand redesign, I did briefly entertain the notion of ditching the blog⁵ altogether. But if I did that, then why have a site at all? Why not just fold my online presence into LinkedIn and ResearchGate, and call it a day?

I’ve been writing online for a very, very long time. My earliest online writing—that I can find, at any rate—dates from 1997, when I was just fourteen years old.⁶ Writing for my own personal enjoyment hasn’t led to fame, riches, or a book deal (yet), but it has helped me in countless other ways. Our thoughts are a chaotic tangle of overlapping concepts, and writing helps us put them in order. Writing has, without question, made me better at my job. Writing has encouraged me to seek out varied sources of inspiration to stay fresh. Writing has kept my mind flexible. Writing has helped me figure out who I am, and what I want to do.

So I will continue to write online. The writing will be more focused on things like design, data visualization, and matters of general nerdery, but there will always be room for the personal and a bit of pop culture. I write because I want to write, not because I feel professionally obligated. I can’t make promises about what I’ll write about or how frequently I’ll write, but I will write.

Off We Go

Lots of people enjoy making things. I do, too. And this website, from its look to its content to its code, is something I made, and will continue to make with each new update. And that’s it. Welcome to my new pizza cutter website. I hope you enjoy reading it as much as I do making it.

I promised myself that there would be no new major purchases until my dissertation is complete. And then there was a sudden and somewhat unexpected inflow of extra cash, and well, you know how it goes. I am now the proud owner of a shiny new Sceptre LCDTV (42”, 1080p, and more HDMI inputs than I know what to do with). I’m very pleased with it, especially given the bargain basement price. Let’s hear it for no-name brands that deliver on their fundamentals.¹

There is, however, one tiny problem. A very tiny problem. The TV has one “stuck” pixel. In most practical viewing conditions I’ll never, ever notice it; it’s one malfunctioning pixel in a field of over two million. It’s only visible when the screen is completely black, and even then, I’ve got to be looking for it. It’s no big deal, and not worth the hassle of boxing the whole thing up and sending it back to Newegg.

Still, it’s there. And though it may not materially impact my viewing experience, I will always know.There are various ways to cure a stuck pixel, from physically “massaging” the area to flashing images on the monitor to try to “scrub” it out. The effectiveness of these methods varies from device to device, which is why I am absolutely galled—galled, I tell you—to find that some people are actually charging for image-flashing software. Software which, let me emphasize, may not even work for your particular pixel problem.

So here’s LCD Scrub, a “pixel scrubbing” program that I’m releasing for free. It was built in Processing (easily, I might add), and the source code is included. Here are the details:

• Start the application and simply move your mouse over it to display a handy help pop-up.
• LCD Scrub can display several solid-color screens: black, white, red, green, and blue. These are handy for checking your screen for any stuck pixels. If a pixel appears black on the white screen setting, that pixel is likely “dead” entirely.
• The “color cycle” mode will rapidly flash between black, white, and random colors. This is the mode that will hopefully “unstick” your pixels. At the least, it will remove any burn-in from your screen. Warning: could totally cause a seizure.
• The speed of the color cycle mode can be adjusted with the up and down arrow keys.
• Zip file includes versions for Mac, Windows, and Linux. If you don’t have a convenient way to hook up your computer directly to your TV, I’ve also included a plain video of the “color cycle” setting. Stream away.

I’d recommend running the color cycle mode (or looping the video file) for an hour or two to see if that fixes your stuck pixel. I make no guarantees about the effectiveness of this software, nor will I be held responsible in the highly unlikely event that the software damages your screen. LCD Scrub has so far failed to unstick my pixel, but then again, I haven’t run it for a decent length of time (my screensaver kicked in). My pixel may come unstuck simply through repeated use of the TV. I’ve seen it happen. In any case, here’s hoping this little tool helps you out.