Saturday, May 03, 2008

A Nice, Clear Critique of Hypothesis Testing

Students at times come to my office to tell me that their professors in research methods courses emphasize the importance of null hypotheses and hypothesis testing. It always brings to mind the last words Gary Chamberlain said in his econometrics class: "I didn't mention hypothesis testing, because I don't like it. And don't ever use those damned stars." [I am, of course, paraphrasing, as I took the class 25 years ago].

In any event, I ran across this nice critique of hypothesis testing. The point of statistics, after all, should be parameter estimation and prediction, not some arbitrary test of statistical significance. In large samples, statistically significant parameters can be unimportant; in small samples, statistically insignificant parameters can simply mean that we don't really know what is going on.

Much of the confusion underlying how medical information is conveyed in the media arises from the hypothesis testing tradition: the media infer that if x is a statistically significant predictor of y, then x must be important in determining y. This is not necessarily so.

5 comments:

Austin Kelly said...

I've always found academics strangely biased towards hypotheses testing as opposed to decision theory, even though I was always taught by my 'metrics profs to distinguish between 'importance' and 'statistical significance.'

My two favorite personal experiences are from work I did on capitalization of above market financing and down payment assistance. I noticed that the literature on capitalized financing invariably tested the null hypothesis of 'beta=0' even though theory implied 'beta=1.' I beleive everyone wanted to make the easy case 'hey we got a significant coefficient' rather than address the issue of 'hey we got a different coefficient than theory implies.' In the down payment assistance case GAO found that in a small sample there was about an 85% probability that these things cost FHA money, and Stephen Fuller responded to the study by indicating that we haven't proven that it was the case at 95% probability. I looked at the critique and thought to myself "there's an 85% chance that they are losing money and they shouldn't do anything because it isn't 95%???? - what does decision theory say about that?" Anyway, later with an expanded dataset I did find a greater than 95% probability, so I guess that can finally be put to bed.

Phyliss said...

Interesting to know.

Anonymous said...

^^Thanks!!

婚前徵信婚姻感情大陸抓姦外遇抓姦法律諮詢家暴婚前徵信尋人感情挽回大陸抓姦離婚工商徵信婚前徵信外遇抓姦感情挽回尋人大陸抓姦離婚家暴工商徵信法律諮詢跟蹤工商徵信婚前徵信感情挽回外遇抓姦法律諮詢家暴尋人大陸抓姦離婚大陸抓姦外遇尋人家暴工商徵信法律諮詢家暴感情挽回大陸抓姦外遇婚前徵信離婚尋人工商徵信外遇抓姦法律諮詢家暴婚前徵信大陸抓姦尋人感情挽回外遇抓姦婚前徵信感情挽回尋人大陸抓姦工商徵信法律諮詢離婚家暴工商徵信外遇抓姦法律諮詢家暴婚前徵信尋人感情挽回大陸抓姦離婚婚前徵信工商徵信外遇抓姦尋人離婚家暴大陸抓姦感情挽回法律諮詢離婚感情挽回婚前徵信外遇抓姦家暴尋人工商徵信外遇抓姦法律諮詢家暴婚前徵信尋人感情挽回">徵大陸抓姦離婚婚前徵信工商徵信外遇抓姦尋人離婚家暴大陸抓姦感情挽回法律諮詢

polo shirts said...

History of polo ralph lauren. Polo fashions had its humble beginnings in 1968 when tie salesman Ralph Lauren gave it a kick start. By 1969 he had a boutique polo ralph lauren factory stores within the Manhattan department store Bloomingdale's. ... Brands and luxury standard. Since Ralph Lauren's first brand, Polo Ralph Lauren, was launched, the company has expanded to include a variety of luxury brands such as Polo Golf, Polo Denim, Polo Sport. You can buy cheap Ralph Lauren Clothing at Ralph Lauren outlet.Also We provide polo shirts
Ralph Lauren polo shirt, 50% OFF! polo ralph lauren outlet online is your best choice!In 2006, polo ralph lauren outlet became the first designer in Wimbledon's 133-year history to create official uniforms for the tournament. As part of this year's event, which starts next week, polo ralph lauren sale will introduces the first ... determination to maintain and enhance the values for which our two brands are famous throughout the world. The rugby ralph lauren brand brings to Wimbledon the look of timeless elegance, drawing on our rich history and traditions

gaohui said...

You will not have ed hardy clothes to worry about any Christian audigier discomfort, and best of all; most are ed hardy shoes designed to fit your body as the ed hardy outlet little bundle of joy grows allowing ed hardy Bikini you to wear the maternity clothing more than ed hardy hats once.The development of holiday maternity ed hardy swimsuits clothing has coincided with the changing attitudes of ed hardy clothing women regarding their bodies. Today's ed hardy glasses holiday maternity fashions are ed hardy trendy, stylish and sexy, fitting today's ed hardy iphone cases fast-paced lifestyle Now, women ed hardy dresses are proud to show off their baby bump while wearing ed hardy Jackets striking and classy maternity clothing.