on television pundits and experts more generally - Tilted Forum Project Discussion Community

roachboy · 03-26-2009, 01:36 PM

Quote:

Op-Ed Columnist
Learning How to Think
By NICHOLAS D. KRISTOF

Ever wonder how financial experts could lead the world over the economic cliff?

One explanation is that so-called experts turn out to be, in many situations, a stunningly poor source of expertise. There’s evidence that what matters in making a sound forecast or decision isn’t so much knowledge or experience as good judgment — or, to be more precise, the way a person’s mind works.

More on that in a moment. First, let’s acknowledge that even very smart people allow themselves to be buffaloed by an apparent “expert” on occasion.

The best example of the awe that an “expert” inspires is the “Dr. Fox effect.” It’s named for a pioneering series of psychology experiments in which an actor was paid to give a meaningless presentation to professional educators.

The actor was introduced as “Dr. Myron L. Fox” (no such real person existed) and was described as an eminent authority on the application of mathematics to human behavior. He then delivered a lecture on “mathematical game theory as applied to physician education” — except that by design it had no point and was completely devoid of substance. However, it was warmly delivered and full of jokes and interesting neologisms.

Afterward, those in attendance were given questionnaires and asked to rate “Dr. Fox.” They were mostly impressed. “Excellent presentation, enjoyed listening,” wrote one. Another protested: “Too intellectual a presentation.”

A different study illustrated the genuflection to “experts” another way. It found that a president who goes on television to make a case moves public opinion only negligibly, by less than a percentage point. But experts who are trotted out on television can move public opinion by more than 3 percentage points, because they seem to be reliable or impartial authorities.

But do experts actually get it right themselves?

The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.

The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.

“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.

Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites. People who shouted — like, yes, Jim Cramer!

Mr. Tetlock called experts such as these the “hedgehogs,” after a famous distinction by the late Sir Isaiah Berlin (my favorite philosopher) between hedgehogs and foxes. Hedgehogs tend to have a focused worldview, an ideological leaning, strong convictions; foxes are more cautious, more centrist, more likely to adjust their views, more pragmatic, more prone to self-doubt, more inclined to see complexity and nuance. And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right.

This was the distinction that mattered most among the forecasters, not whether they had expertise. Over all, the foxes did significantly better, both in areas they knew well and in areas they didn’t.

Other studies have confirmed the general sense that expertise is overrated. In one experiment, clinical psychologists did no better than their secretaries in their diagnoses. In another, a white rat in a maze repeatedly beat groups of Yale undergraduates in understanding the optimal way to get food dropped in the maze. The students overanalyzed and saw patterns that didn’t exist, so they were beaten by the rodent.

The marketplace of ideas for now doesn’t clear out bad pundits and bad ideas partly because there’s no accountability. We trumpet our successes and ignore failures — or else attempt to explain that the failure doesn’t count because the situation changed or that we were basically right but the timing was off.

For example, I boast about having warned in 2002 and 2003 that Iraq would be a violent mess after we invaded. But I tend to make excuses for my own incorrect forecast in early 2007 that the troop “surge” would fail.

So what about a system to evaluate us prognosticators? Professor Tetlock suggests that various foundations might try to create a “trans-ideological Consumer Reports for punditry,” monitoring and evaluating the records of various experts and pundits as a public service. I agree: Hold us accountable!

http://www.nytimes.com/2009/03/26/op...ristof.html?em

ever wonder about these "experts" or "pundits" that you encounter in the press with these little bylines underneath their names that identify without situating some affiliation or another, about whom you or i know nothing, whose work we in all probability have never seen and will never see, but whose opinions, for some reason, folk take seriously?

this edito from the ny times asks the same kind of question, which of course are more difficult for an editorial writer to pose in an editorial column because, well, he has seen the enemy and he is them. kinda.

who the hell are these people?
how do they get air time or column space?
more importantly, why do any of us take them seriously?
what is it about a claim to "expertise" that prompts you to consider the interpretations of information that they are routinely brought in to provide any more compelling than interpretations you might make of that same information?

how do you sort out who's definitely an idiot from who might not be when confronted with these "experts"?

i wonder the extent to which--on television in particular---expertise is a matter of appearance and situation and byline---in the way that those old commercials for preparation h were built around the assumption that a guy wearing a lab coat and standing in front of "reassuringly brown) bookcases knew alot about such medication...

Craven Morehead · 03-26-2009, 03:00 PM

Quote:

how do you sort out who's definitly an idiot from who might not be when confronted with these "experts"?

Really good question. Most of the time, I get the feeling the TV 'experts' are chosen more for their superior presentation skills rather than their depth of knowledge.

Nimetic · 03-28-2009, 06:57 PM

Question everything and everyone. Including yourself.

The difficult bit is remember to be suspicious of your own beliefs.

Cynthetiq · 03-28-2009, 07:29 PM

I was have been asked to be the resident expert for the following situations:

Corporate re: Seminar Technology expert: Color printers cost of ownership versus laser printers
Person who did the speaking engagement was paid $450, the person who did it had less knowledge than I did and he collected the $450.
News re: Websites: Technology expert on camera for the Heaven's Gate suicide
I didn't want to be associated with the news company. They guy calling me wanted to sleep with me
Virus: Technology expert on the proliferation of virus in MS products.

I didn't want to be associated with the news company. They guy calling me wanted to sleep with me
Gameshow - Who Wants to be a Millionaire: Technology question verification that there is only one possible single answer.
I wanted to do this one, and I wanted to sleep with the hot chick that was the researcher

I declined all of them except the gameshow. Everything else I felt I was not qualified to be an expert.

dippin · 03-28-2009, 07:53 PM

Just like you have fast food, you have these "fast thinkers."

Prepackaged, preconceived knowledge in small doses that satisfies but isn't really good for you.

As as example, look at the current mess: why bring someone in who might actually point out the systemic aspect of the crisis, and how there are hard choices ahead of us regarding priorities and so on? Just bring someone in who will say that everything is fine, everyone is a victim, and that the solution is to hang the undesirables, be them politicians, illegal aliens, CEOs, etc. by the balls.

Point being, beware of anyone who portrays reality or current issues as simple.

03-28-2009, 07:29 PM	#4 (permalink)
Cynthetiq Tilted Cat Head Administrator Location: Manhattan, NY	I was have been asked to be the resident expert for the following situations: Corporate re: Seminar Technology expert: Color printers cost of ownership versus laser printers Person who did the speaking engagement was paid $450, the person who did it had less knowledge than I did and he collected the $450. News re: Websites: Technology expert on camera for the Heaven's Gate suicide I didn't want to be associated with the news company. They guy calling me wanted to sleep with me Virus: Technology expert on the proliferation of virus in MS products. I didn't want to be associated with the news company. They guy calling me wanted to sleep with me Gameshow - Who Wants to be a Millionaire: Technology question verification that there is only one possible single answer. I wanted to do this one, and I wanted to sleep with the hot chick that was the researcher I declined all of them except the gameshow. Everything else I felt I was not qualified to be an expert. __________________ I don't care if you are black, white, purple, green, Chinese, Japanese, Korean, hippie, cop, bum, admin, user, English, Irish, French, Catholic, Protestant, Jewish, Buddhist, Muslim, indian, cowboy, tall, short, fat, skinny, emo, punk, mod, rocker, straight, gay, lesbian, jock, nerd, geek, Democrat, Republican, Libertarian, Independent, driver, pedestrian, or bicyclist, either you're an asshole or you're not. What's the best nation in the world? DO-NATION!

03-28-2009, 06:57 PM	#3 (permalink)
Nimetic Junkie Location: Melbourne, Australia	Question everything and everyone. Including yourself. The difficult bit is remember to be suspicious of your own beliefs.

03-28-2009, 07:53 PM	#5 (permalink)
dippin Crazy, indeed Location: the ether	Just like you have fast food, you have these "fast thinkers." Prepackaged, preconceived knowledge in small doses that satisfies but isn't really good for you. As as example, look at the current mess: why bring someone in who might actually point out the systemic aspect of the crisis, and how there are hard choices ahead of us regarding priorities and so on? Just bring someone in who will say that everything is fine, everyone is a victim, and that the solution is to hang the undesirables, be them politicians, illegal aliens, CEOs, etc. by the balls. Point being, beware of anyone who portrays reality or current issues as simple.