Robert Siegel was interviewing Caterina Fake, co-founder of Flickr and now chief product officer for an Internet "taste-profiling" service called Hunch.com, on NPR's "All Things Considered" the other day. It's a software-driven inference engine of the sort you see all over the web these days that provides you with "if you like this, then we suggest you'll like this" recommendations. Netflix has one, Amazon has one, iTunes' relatively new "Genius" feature is one.
Here's what Siegel said:
I must say, it remains a big question for me as to whether one can effectively quantify taste. I'll give you an example. I, and it turned out, a contemporary of mine, we're both baby boomers and both of us quit Netflix at precisely the same time for precisely the same reason, completely independent of each other, which was that after watching and liking "Garden State" a great deal, we got the message, if you like that movie, you'll like "Harold & Kumar Go to White Castle."
And somewhere in the first half hour of this movie I realized that someone had assumed I liked movies about doper teenagers in New Jersey and that was what made it. And you could write the algorithm easily. It just didnt get to the quality or the kind of movie at all. I was watching teenage entertainment.
"Yes, Mr. Siegel," I shouted in my car, "because that's what you told it you wanted! Garbage in - garbage out, dummy!" (Yes, I became Dr. Steve Brule for an instant at the end of that exclamation.) I was momentarily troubled not so much because "Harold and Kumar Go to White Castle" is infinitely smarter, funnier and more mature than "Garden State" (though that is obvious). And I'm not accusing Mr. Siegel of being one of those ubiquitous morons who doesn't understand what a point of comparison is. After all, he provided three of them -- "doper," "teenagers," "New Jersey" -- in his remarks.
No, what really bugged me was that he took it personally, without taking personal responsibility for the result. (I was happy he used the word "algorithm," though.) I'm as disgusted by the idiotic anthropomorphizing of computers as I am of the anthropomorphizing of animals. Algorithms do not read minds. If Siegel didn't like "Harold and Kumar," he should have given it a one-star rating on Netflix. That would have increased the chances that he'd get better recommendations next time. (Jeez, most of these services even tell you why the software is serving up a recommendation: "Based on your interest in...")
Siegel did not say how many movies he'd actually rated at Netflix, or if he'd filled out the multi-leveled (and, I feel, rather silly) "taste preferences" questionnaire about what moods/qualities/genres he favors. Oh, and I'm not saying that the questionnaire itself is "silly." I can see how it would be useful. I just mean that I feel somewhat silly rating such qualities as "Mind-bending" and "Understated" on a scale of 1 to 3.
There are two main ways these applications work: 1) by comparing the things you've said you like/don't like with the tastes of other people who have similar likes and dislikes and recommending other things you may have in common; and/or 2) by comparing individual traits or genres (doper, teenage, New Jersey) -- or, as I sometimes see served up on my Netflix homepage: "Dark Foreign Dramas from the 1970s" or "Scary Critically-acclaimed Polish Comedies."
The recommendations can only be as good as the amount of information you've contributed to help, of course. I checked and I've rated about 1,200 movies on Netflix -- which is really easy to do thanks to a cute little Java script. You just drag your cursor over the stars (or the "not interested" button) and click. Netflix does a pretty good job of predicting how I'll feel about movies (it even predicts what star rating I'm likely to award) -- and when I notice a movie I've seen that I haven't rated yet, I quickly let it know what I think to confirm an accurate prediction or correct a faulty one.
Netflix uses the other method as well -- which means that Siegel and his boomer friend may very well have each other to blame for that shocking "Harold and Kumar" recommendation -- and they were not acting "independently" at all. Supposedly, they have similar tastes. Maybe they both liked not only "Garden State" but "Slumdog Millionaire," "Donny Darko," "Gran Torino," "Brokeback Mountain," "Sixteen Candles," and "A Serious Man" -- and they hated "Crash," "Irreversible" and "Mulholland Dr." And some other people who gave those movies similar ratings also liked "Harold and Kumar Go to White Castle." In that case, they are likely to get a recommendation for "Harold and Kumar," because of a simple statistical correlation. It's nothing personal, Mr. Siegel. It's just business...
30 Comments
My only issue with Netflix's system (from which I've managed to discover a whole lot of films I'd never even heard of) is that it seems to lump all non-English films into the "Foreign" category, meaning because I liked Inside, I would of course also like Amelie. Of course, I personally like BOTH those movies, but not at all because they both happen to be in French.
I think they use the "Foreign" category (you can also express a preference for films from certain countries -- let's hear it for Polish cinema!) to screen out subtitled films for those who don't want to read them, and to help people who are actually looking for films from another country/culture (or maybe learning a language) to find them.
I've been using Netflix Instant for close to 2 months and can't believe I waited so long to join.I love it.
I enjoy rating films,reading the reviews and i've found their recommendations usually correct. It's also fun to see what rating they think i'll give.
The stoner buddy films have been underated for far too long. I find movies like Harlod and Kumar, the Big Lebowski, and Pineapple Express to be far more creative than most of the movies that are nominated for Oscars.
The service that Netflix provides is the renting of movies from your home. Why would it recommending "teenage entertainment" be cause for quitting the service? Or was this just a way to trumpet superior (questionably superior, that is) taste. I think a point was made about quantifying taste that wasn't intended by the interviewer.
The beautiful thing about the recommendations is that it might provide those with an open mind (and comprehension of the device that is providing the recommendations) with opportunities to screen films that they otherwise wouldn't think of (as well as old favorites) and if enough information is given as input... well, interesting results will be the result.
But it goes without saying that an open mind capable of discerning and screening input will benefit from any recommendation... if just for the extra info. I have a feeling that might be why Netlix allows you to click "Not Interested".
Siegel's comment almost provoked a laugh from me. To take a single inaccurate recommendation as proof that a system is flawed... especially a system based on statistical likelihood and trial-and-error aggregation of data... is beyond silly. It's like firing an employee because they asked you for feedback on a minor assignment.
Than again, it's easy for me to recognize these inane remarks from other people... but it makes me wonder sometimes: how often do I make equally silly, sloppily-reasoned statements, without even noticing it?
Let's back up: Siegel quit Netflix specifically because it gave him one bad recommendation?
I had no idea people even paid much attention to the recommendations. Or, if they did, I assumed they at least used their heads before robotically adding them to their queue. Because it takes about one second -- maybe after seeing the cover art or just reading the title -- to realize that Harold and Kumar Go to White Castle is a very different experience than Garden State.
So much about what he said is bafflingly stupid.
I heard the same NPR piece, but didn't have nearly the same reaction you did. I guess that after years of hearing Siegel do interviews I've just come to assume that his incredulity is somewhat tongue-in-cheek and part of his shtick. Another thing to consider is that it was likely just a very planned example for the audience right down to Siegel's tone of incredulity. What I mean is, could that piece of communication have been designed for those listeners who don't understand the tech at all and who would actually take such an "offense" (renting a movie based on Netflix recommendation only to find they hated it) personally. That's what I assumed. I must have a low estimation of people on the whole (okay, I definitely do) because I think a lot of Netflix users would misunderstand the tech, their ability to control it, and be annoyed at "bad recommendations." I just figured that Siegel's anecdote wasn't based on reality (or at least his real emotions) so much as it was used to connect with the listeners who would think and behave that way. Of course that means I'm giving Siegel a whole ton of credit while admitting I'm admittingly giving the public at large very little... even though Siegel was the one who behaved that way. Okay. I guess I've just successfully made a bad argument. But, there you have it! =)
You may be right -- that he's playing a role as an audience surrogate. But if that's so (and I have been a fan of his for a long time), I wish he wouldn't assume we're that dumb! He could have just asked: "Why do I sometimes get 'bad' recommendations from Netflix, like... ?"
I find it kinda fascinating that he took that to be the main purpose of netflix, to the degree that when that featured failed him, then the whole site had. It's a movie delivery system. It's like burning your blockbuster card because the clerk recommended Jackass to you, as opposed to the myriad other reasons not to use blockbuster.
I've been a Netflix member for about 6 years and have rated just under 2500 movies. At this point, the recommendation engine is pretty good, but I don't even pretend like it's going to know my tastes perfectly. I agree, I think Robert Siegel made the mistake of thinking Netflix's system is near infallible. Even our best machines/systems are a product of our imperfect selves...
I do the same thing when learning how to program. I consciously realize if something goes wrong it is my fault. It still irritates me and I find myself criticizing the computer.
What I've come to do with Netflix is not rate a film based on how much I like it, but by how much I appreciate it.
I didn't like "Un Chien Andalou" but I gave it a good rating because I'd like to be recommended more stuff like it, which I might have a better chance of enjoying! I gave all of Robert Bresson's stuff five stars, even though he has a few films that I flatly don't enjoy (sprinkled in with his numerous masterpieces), because I'd like to be recommended more stuff that's similar to even his worst films!
Likewise, I enjoyed the Harry Potter movies well enough I suppose, but gave all but the Cuaron-directed film (the third one) one star since I don't really want to be recommended anything like them! I know where to find Harry Potter knock-offs, thank you. And I know where to find Garden State knock-offs too, which is why I'll avoid "It's Kind of a Funny Story".
By far the best recommendation engine that I have used so far is the one that Criticker uses. Not only does it do a good job guessing movies that I'll like but it actually assigns a point value on (a scale of 1 to 100) which usually ends up being within a couple of points of what I actually end up rating the movie (on a scale of 1 to 100).
Unfortunately not too many people rate movies on a 100 point scale like I do. But everyone has their own way of rating movies and their system seems to adapt to that.
Jim is smart and interesting as per usual. But I'm still not convinced that I should spend any extra energy trying to explain my soul to Netflix in a few star clicks. I have zero faith that it will improve my recommendations. Because, frankly, I'm interested in everything. There's not a single category of films--at least the way they so bluntly group and name them--that I'm not interested in. And there's really no accounting for the quality or the taste of if-you-liked-that-then-you'll-like-this comparisons. I suppose I object in principle to this kind of self-chosen narrow-casting. I mean the Internet is kind of ghettoized enough as it is. I don't want to help sites like Netflix or Amazon to think they understand what I want to see better. Because sometimes I don't even know myself. I like the wide-open option of not being pigeon-holed by some electronic screen I can't even see. I use sites like this one and a dozen others to find fodder for my queue. I only need Netflix to send/stream it to me. This is also why I still enthusiastically support my neighborhood mom-and-pop video store. Take Jim's example above about foreign films, for instance. Well, until I started watching Apichatpong Weerasethakul's films, I would have clicked to display a total lack of interest in, say, films from Thailand. But I would have been wrong. And I would have been limiting my future self's pop-up choices. I suppose this is all pretty silly anyway because I really shouldn't be randomly browsing Netflix for recs and I've only ever been that awfully procrastinating a couple of times in the past few years. The best reason of all not to rate movies--save precious click seconds toward watching one. Or getting your work done, which I will now go do.
Good point. But I have to say I feel great giving low ratings to films I really hate ("No! Don't give me any more like THAT!"), and high ratings to films I love ("Yes, please. Let's have more of these in the world!"). I don't know that it improves my recommendations -- but, like you, I rarely watch anything based on Netflix recommendations. I have hundreds of movies already in my DVD and Instant queues. But I don't take the recommendations personally, either. It's just a tool you can use or not use -- and not favoring something (like Thai films) won't prevent you from looking them up as you normally would.
Note: I'm not a witch. (And I'm not Christine O'Donnell.) I would not try to persuade people to explain their souls to a technology.
I'm quitting this web site because you think Harold and Kumar is smarter and funnier than Garden State.
I'm kidding, of course (although I am somewhat disturbed by that claim... it's not that Garden State is particularly smart or funny, it's just that Harold and Kumar is incredibly stupid and not funny). I just find it ridiculous that Siegel would quite Netflix because of that!
I mean, really, who uses Netflix for the recommendation system? Heck, I'm surprised there are people who even use the recommendation system at all. Anyone who makes their movie selections based on the recommendations from a computer program (as opposed to, say, recommendations from actual, living human beings) gets what is coming to them.
warren,
- Because, frankly, I'm interested in everything
- And there's really no accounting for the quality or the taste of if-you-liked-that-then-you'll-like-this comparisons
- Because sometimes I don't even know myself
- I would have clicked to display a total lack of interest in, say, films from Thailand. But I would have been wrong.
your argument is very confused. your objections to the netflix recommendation service are actually reasons why it should be useful to you.
The netflix algorithm isnt trying to pigeon hole you or label you or strip you of your individuality....it is simply attempting to identify patterns.....patterns that may turn out to be very useful to you. In its purest form it wouldnt care about genre, language, or even the fact that it is tracking movies. You are a number, each movie is a number; and all of the movies that you have ever enjoyed in your life is a collection of numbers.
Like it or not, somewhere in the world there is another Warren Oates. That is, someone with a collection of numbers that looks a lot like yours. They too may fancy their tastes to be so eclectic to be unique and unquantifiable, but that flies in the face of math.
And perhaps what differentiates their collection of numbers from yours is the absense of those numbers that represent films by Apichatpong Weerasethakul....it is this difference that an algorithm would identify, therefore facilitating your like minded match's introduction to Thai cinema.
Milk,
Your comments are ostensibly true, but I doubt that any of us know the level of sophistication involved in the NetFlix algorithm. You make it seem as though the system is very refined, capable of searching a database for one person with nearly-identical taste and making suggestions based on that.
This could be true, hypothetically, but I don't think this is the level NetFlix works on. It seems to be a relatively straight-forward, mass-market-ish-y recommendation system. I think, in the case of Garden State and Harold and Kumar, that Siegel is absolutely right, in that the system used a broad set of identifiers (dopers, teenagers, New Jersey) to find a "similar" film. In this case, at least, it doesn't look like NetFlix used any sort of advanced statistical comparison model to find people with similar taste. In short, although your comments might be true (one day, with a different system), they don't seem to be in this particular instance.
This post was way too long, as I only intended to point out that Siegel seems absolutely correct in his assumption of how the system works (although his decision to quit NetFlix because of the recommendation is odd, to say the least).
Also, the fact that he and a peer quit NetFlix on the same day for the same reason seems to me to be a wink at the audience. Isn't the two of them quitting akin to NetFlix's recommendation system using Warren's and Nearly-Warren's tastes to decide what films to suggest? I wish I could have spit that out a bit more clearly, but I am tired.
The discussion about people not understanding technology nor statistical models reminds me of the greatest blog ever, at www.firejoemorgan.com. Fans of baseball, statistics, snarky articles, hilarity, and knocks on stupid journalists should check it out.
I wonder if he would disown a trusted friend who happened to reccommend a movie that he did not like. Its essentially the same thing. Heck, I've been recommended films on netflix I loved that I wouldn't have watched otherwise.
You have my full support on the Garden State argument. There's more humor, passion and honesty in Harold and Kumar then Zack Braff could ever muster in his pinky.
Just thought I'd share a quick story. A while back I got the Six Feet Under series from Netflix. I thought it was decent. Then my recommendations from Netflix really changed. I started getting a lot more "alternative" movies than I was used to, alternative movies with covers where two dudes often grinned goofily at each other over crazily-fonted titles like "Telling the Family." I realized after a while what was going on: Netflix thought I was gay. I rushed to rate movies like Predator and Die Harder, then wondered if that wouldn't actually increase the new recommendations. Then for fifteen intense minutes I tried to outsmart the algorithm by attempting to nail down exactly what a solidly male heterosexual movie would look like. Then I said "fuck it" and gave up.
Jim,
Your response to Seigel's reaction is actually more alarming. You yelled in your car? For a person that seems to be constantly confronted with unreasonable people and the arguments they give, (via the media, I mean) you seem to have odd reactions to them. As for Seigel, some people have exaggerated responses to what they perceive to be a stupid operation from software. As for you, some people have exaggerated responses to what they perceive to be a stupid person (in this case, stupid because the person seems unwilling to see their direct connection to a recommended movie).
Netflix has just started here in Canada, so I haven't even joined it yet, but I've got to say that Amazon's recommendations are uncannily accurate, and takes very few purchases before it seems to be spot on. My wallet hurts from browsing there. iTunes Genius works fine within your own internal library, but the Genius Recommendations in the iTunes store are ridiculously slow at finding good matches, and find odd matches based on what you "like." The same music I rate on Amazon give me completely different recommendations than those on iTunes. I'm surprised they don't have a similar algorithm.
I love Netflix. It saves me so much money and has a ton more selection than when I used to go to Blockbuster. I pretty much ignore all the recommendations. First off, I have enough movies I want to see on my own without being told by an algorithm what to watch. Second, I only rate movies I watch on Netflix. It don't rate ever movie I've ever seen. That would take a long time. Third, I'm looking at some of their suggestions. While many are accurate, their reasonings are strange. Take for instance Outlaw Josey Wales, a good, if a bit overrated Eastwood Western. It's suggested because I liked three other movies.
1. Unforgiven-they are both highly praised Eastwood Westerns. I think Unforgiven is a deeper film that Wales, but I that's not the point.
2. National Lampoon's Animal House-Beyond John Vernon and that they were both movies from the late 70's that went against the establishment (although I can't think of a pro-establishment movie from the late 70's), I can't think of any other similarities.
3. Patton-I guess you could say they are both war movies, but I can't think of any other similarities.
When I click on Wales, it tells me it is also being recommended because I enjoyed Spartacus and Bridge on the River Kwai. Beyond war themes in both and the slavery in Spartacus, these two movie have almost nothing in common with Wales. It s I would probably give it 3.8 stars. That's probably a little high. I say 3.3 or 3.4.
So ... Siegel was upset that Netflix's preference prediction algorithm is not an accurate mind-reading device? So upset, in fact, that he quit the service?
This man has lofty expectations. I wonder if he storms out of restaurants when the waitress fails to accurately guess what he wants.
...Yeah, I dunno, "Garden State" has Ron Leibman, Ian Holm, Jean Smart, Jim Parsons, Denis O'Hare and Michael Weston, all great. Peter Sarsgaard by himself is better than anything in "H&K."
I loved Garden State. I thought it was an extremely sweet, insightful film. I would be curious as to why some people don't like it.
I guess you could say it wants or tries to be profound, like a lot of "Just live your life!" type movies. And there's a lot of "quirky" characters and set design. But the excessive idiosyncracies add up to a sort of convincing reality for me. I have a job where I enter a lot of people's homes, and, yeah, people are weird. Anyway, I look at it as a nice romantic comedy, with a lot of great character roles (Ron Liebman!).
This sounds totally cheesy but I'm gonna say it anyway: Peter Sarsgaard perfectly embodies the fragile dignity of the working class dissatisfied with their lives. I seriously feel that way. He's so ballin' in that movie.
I had a bad experience with NetFlix's recommendation system. I didn't quit NetFlix over it; I just stopped putting so much stock into its algorithm.
(Actually, a bigger part of why I stopped caring about its recommendations is that it gives me so few--120, as compared to thousands that the rest of my family gets. I'm stingy with 1-stars and 5-stars, so I think that has a lot to do with how the program works.)
I also get confused by its taste recommendations. They do give a better idea of what I'm looking for, but it's so vague sometimes. For example, on the Watch Instantly, the top two movies on its page are "The Girl with the Dragon Tattoo" and "The Scorpion King." I have no idea how either one is particularly "mind-bending;" I was under the impression it meant confusing or thoughtful. I'm also confused by the vagueness of the term "Strong Female Leads"--it gives me a lot of dull-sounding romcoms, whereas I was expecting it to be feminist or maybe girls in action movies. Apparently, it just means that there is, in fact, a girl as the main character.
The most puzzling thing of all, however, is why somebody would give a high rating to a movie as bland and cookie-cutter-indie as "Garden State." I haven't seen "Harold and Kumar," but I expect it's a much more interesting, unusual movie.
Not that I'm about to cancel my subscription, but I just had "Caligula" recommended based on my interest in "The Madness of King George". I'd say that's a better example than the one Robert Siegel used.
Leave a comment