How to lie with drug statistics
by John Horgan
The New Republic
April 1, 1990
In the war on drugs J. Michael Walsh is one of the administration's most valuable officers. His mission is
to implement -- and, more important, to justify -- programs in which the urine of working people is searched for signs of illegal drugs. Walsh designed the drug-testing program for federal employees mandated by Ronald Reagan several years ago and is now advising business leaders on how to test their workers. He has argued in favor of testing before Congress and federal judges, on national radio and TV shows, and in countless other public forums.
Walsh's basic message is this: drug users, from crack addicts to weekend marijuana smokers, make less productive workers than non-users. So employers are justified in using drug tests -- which cannot distinguish between chronic abuse on the job and occasional use at home -- to root out all users from the work force.
Other officials say the same thing, but Walsh's title gives his words extra weight. He is director of the Division of Applied Research and the Office of Workplace Initiatives at the National Institute on Drug Abuse (NIDA), the chief federal drug research agency. The evidence he marshals is passed on to drug czar William Bennett and is widely disseminated, with the federal government's seal of approval. This role, and Walsh's status as a scientist, oblige him to uphold high standards of objectivity and competence. More than any other person, he is responsible for the intellectual honesty of the whole Reagan-Bush drive for workplace testing. A look at some of the central claims he has made in support of that drive reveals that he has not met his responsibility.
Two years ago Walsh testified in federal court that the "cost of drug abuse to U.S. industry" was nearly $50 billion a year, according to "conservative estimates." This claim is a staple of anti-drug rhetoric. It is frequently quoted without qualification by the media, and last year President Bush rounded it upward to "anywhere from $60 billion to $100 billion."
Here's how the figure was derived. In 1982 NIDA surveyed 3,700 households around the country. The Research Triangle Institute (RTI), a NIDA contractor in North Carolina, then analyzed the data and found that the household income of adults who had ever smoked marijuana daily for a month (or at least twenty out of thirty days) was twenty-eight percent less than the income of those who hadn't. The RTI analysts called this difference "reduced productivity due to daily marijuana use." They calculated the total "loss," when extrapolated to the general population, at $26 billion. Adding the estimated costs of drug-related crimes, accidents, and medical care produced a grand total of $47 billion for "costs to society of drug abuse."
Several things are wrong here, but the most glaring is the simpleminded conclusion that marijuana smoking caused the lower incomes with which it was associated. There are many respects in which the behavior of lower-income people differs statistically from that of upper-income people, but this mere correlation never establishes causality. It is quite probable, for example, that people who would admit to watching "Wheel of Fortune" every night have lower incomes than those who do not. Should we then conclude that television game shows decrease productivity? Or, by similar logic, should we conclude that Thunderbird wine hurts productivity but Chivas Regal scotch helps it?
You may be wondering why the RTI study used the awkward variable of "marijuana-use-daily-for-a-month-ever" as an indicator of drug abuse. Good question.
Actually, the RTI researchers had information on current use (at least once in the last thirty days) of drugs --including cocaine, heroin; amphetamines and LSD as well as marijuana--but they could find no connection to decreased income. If Walsh really accepts the logic linking a single month-long marijuana binge to decreased productivity, he must also conclude that current use does not decrease productivity. But he shows no signs of doing so.
Walsh now admits that the RTI study "is based on assumptions that need additional validation." One wonders why he hasn't passed that information on to Bush, Bennett, representatives of the drug-testing industry, and others who continue to treat the "cost of drug abuse to industry" as a scientifically established fact.
Anyway, Walsh says he has other evidence demonstrating the urgent need for workplace testing. He and another NIDA official edited and a published a 340-page collection of studies called: Drugs in the Workplace. When I interviewed Walsh recently, he drew my attention to three studies in particular --one done by the Navy and two by electric-power utilities-- that he said showed that drug users make poor employees.
Let's start with the Navy study. The primary subjects were 500 recruits who tested positive for marijuana in 1985 and were admitted anyway (those who tested positive for any other illegal drug were rejected). Two-and-a-half years later the Navy had discharged forty-three percent of those who had tested positive and only nineteen percent of those who had tested negative during the same recruitment period. Sounds like former potheads make lousy soldiers, right? Walsh even suggested to me that the lingering effects of pre-service marijuana use may have been responsible. "We do know that it stays in the system for a long time," he noted.
But hold on. Look at the difference between the positives and the negatives. The positives were about twice as likely to be black and to lack a high school diploma. Why not blame poor education or cultural factors correlated with race for their high discharge rate? (In fact, civil rights groups have complained that the Navy uses testing to keep blacks out of the service.)
Actually, there is an even simpler explanation. One of the conditions for being admitted to the Navy after testing positive for marijuana was agreeing to submit to rigorous "surveillance" thereafter, including unusually frequent drug tests (all naval personnel undergo three such tests a year). A third of the positives were discharged because they failed another drug test.
There are two problems here. One is that these sailors had no documented loss of productivity and exhibited no misbehavior except for drug use. So the natural conclusion isn't that people who have used drugs are more likely to be deficient workers (the only conclusion that would justify workplace testing), but rather that people who have used drugs are more likely than others to use drugs in the future. But even this conclusion is undermined by the second problem with the study: these sailors, facing more frequent testing than sailors in the control group, stood a better chance of getting caught using drugs.
If one discounts the sailors discharged for failing subsequent drug tests the difference in the discharge rates between the positives and the negatives shrinks from twenty-four to ten percent. Other forms of extra surveillance could easily account for that difference. Obviously, the more closely you watch someone, the more likely you are to see him misbehaving, especially if you think he is a troublemaker to begin with.
Another study cited by Walsh, conducted by the Utah Power and Light Company, makes the Navy experiment look like a paragon of scientific rigor. The Utah study compared the work history of employees who tested positive for drugs with a control group of employees whose ages and jobs were similar. The data showed a "significant difference between drug users and non-users in terms of being involved in accidents, being absent from work, and overutilization of health benefits," Walsh told me.
When one reads the study two flaws quickly stand out: there were only twelve positives in all (eleven for marijuana and one for cocaine), an absurdly small sample, and the control group was never tested for drugs. The study's conclusion could be rejected on these grounds alone.
But there is an even bigger problem. Eight of the twelve "drug abusers" (to use the Utah researchers' term) were tested because they were in accidents, and some were injured and needed time off to recuperate. Of the four remaining positives, two were tested for other performance-related problems and two because they had enrolled in a substance-abuse program. High absenteeism almost invariably precedes--and precipitates--both performance-related testing and submission to a substance-abuse program. Moreover, all employees who undergo a test are suspended until the results come back, which usually takes three or four days.
Incredibly, the Utah researchers included in their calculations the accidents and absenteeism directly associated with the testing of the twelve subjects. By this logic, you could link any trait to accidents and absenteeism. Round up employees who have been in an accident or have been absent a lot, test them for, say, type 0 blood, and send them home for a few more days of absenteeism. Then compare the accident and absenteeism rates within the type 0 group with those for a "control" group with no particular history of accidents or absenteeism -- and whose blood type, in keeping with the Utah methodology, wouldn't even be tested; it would just be assumed not to be type 0. Surprise, surprise: type Os had more accidents and missed work more often than people whose blood wasn't tested. Better get rid of everyone with type 0 blood.
And what about the "overutilization of health benefits" that Walsh had mentioned? He apparently misspoke. In the introduction to the NIDA monograph he calls the health benefits data "inconclusive." In fact, the positives consumed almost fifty percent less in health benefits than the control group. If the positives had used fifty percent more, would Walsh have found that "inconclusive"?
The other utility study cited by Walsh, which was done at the Georgia Power, Company, also focused on employees tested "for cause." But the Georgia researchers used a different control group: the 116 people who came up positive were compared with 713 who passed the test. This comparison, ostensibly fairer than that in the Utah study, found that the positives missed about five more days of work per year than the negatives.
But even the methodology that yielded this modest finding is flawed. The authors of the study note, with refreshing candor, that "the primary subject for the database is the problem employee." Indeed, the negatives and positives all missed work much more often than the company average. It's kind of like testing burglars for marijuana and concluding that --because those who tested positive had robbed a few more houses than those who did not-- marijuana makes ordinary people more likely to steal.
Moreover, couldn't alcohol, which is often consumed in conjunction with illegal drugs, contribute to the higher absenteeism of the positives? None of the employees was tested for alcohol. Of course, tests wouldn't prove much anyway, since alcohol is detectable in body fluids for only six hours or so, whereas cocaine persists for two or three days and marijuana for up to a month. That may be one reason that this and some other studies fail to consider the possibility that alcohol abuse is sometimes responsible for misbehavior attributed to illegal drugs.
When I asked Walsh about the flaws in these studies, he replied: "You could make the same argument about any database in higher research, such as cancer research, or mental health research." This is an insult to every scientist in those fields, none of the studies that Walsh touts has passed peer review (or is likely to), whereas thousands of papers on cancer and mental health have passed this minimal standard.
Walsh sometimes sounds quite reasonable and "scientific," especially when he talks about the mechanics of testing. He emphasizes that when a drug test comes up positive it should be confirmed by a more sophisticated analysis (a good idea, since poppy seeds, ibuprofen, and cold pills can trigger false alarms for heroin, marijuana, and amphetamines, respectively). He also stresses the importance of maintaining high laboratory standards and confidentiality.
But whenever he ventures beyond the mechanics of testing and tries to justify the enterprise itself, he shows an odd disregard for facts. For example, he frequently suggests that drug abuse is rampant not just in specific sectors of society, but broadly and pervasively. "The problem of drug abuse is so widespread in America," he declared in a speech last year, "that every company must assume that its employees will eventually be faced with a substance abuse-problem of their own, of a family member, of a co-worker, or of a friend." He always neglects to mention that NIDA'S own statistics show that the use of illegal drugs-marijuana, cocaine, PCP, the whole lot--has been declining sharply for years. Severe cocaine addiction (daily use) has increased, but primarily among the unemployed, who are beyond the reach of workplace testing.
I asked Walsh whether he thought that, given these data, his efforts to promote workplace testing might be misplaced. "I think drug abuse has gone down because of these workplace programs," he replied. I pointed out--again according to NIDA's data--that the decline began in 1979, well before testing had caught on. Then Walsh asserted that, whether or not drug use has declined among workers and whether or not severe addiction occurs largely among the unemployed, there are still an estimated 10 million working people who are using illicit drugs, and they represent a "much bigger problem than the few hard core...... The concept is to eliminate illegal drug use," he said, "not just to focus on those who are addicted."
This sentiment, of course, lies at the core of all of Walsh's uninformed claims. He adheres to the zero-tolerance line, which doesn't discriminate between use and abuse, between a secretary smoking marijuana on weekends and an AIDS-ridden prostitute smoking $100 worth of crack a day. From this perspective, drug testing--which also fails to discriminate between casual and chronic user--makes sense.
But then, Walsh does believe that even occasional marijuana use can have devastating effects. "I think we have reached the point where the involvement of marijuana in accidents exceeds that of alcohol," he said. Where is his proof for this dramatic statement? He doesn't have any, but "it's one of the things in my research program we're trying to do right now." Note the procedure: Walsh reaches his conclusion first, then sets out to prove it.
As a scientist, Walsh is probably in over his head. He has testified to having "over twenty years of experience (fifteen years in the laboratory) in research on the physiological and behavioral effects of psychoactive drugs." Well, sort of. Before joining NIDA in 1980, Walsh spent fourteen years at the Naval Medical Research Institute studying what might be called underwater altered states. For his Ph.D., which he received from American University in 1973, he studied nitrogen narcosis ("rapture of the deep"). Typical of his pre-NIDA publications is a paper about what happens to mice when they are given morphine under "hyperbalic abnormally high] pressure" and an article called "Should Divers Take Drugs?" published in something called Faceplate.
On the other hand, if you view Walsh not as a scientist but as a propagandist, you have to admit he has done his job well. A decade ago virtually no companies had testing programs. Now a majority --including such bastions of liberalism as The New York Times-- test employees, job applicants, or both.Walsh can't take all the credit, but he certainly has done his part. Perhaps that is why he remains so strangely sanguine when confronted with all the inconsistencies in his logic. He has already won the day. "Drug testing," he says with pride, "is here to stay."
Walsh's basic message is this: drug users, from crack addicts to weekend marijuana smokers, make less productive workers than non-users. So employers are justified in using drug tests -- which cannot distinguish between chronic abuse on the job and occasional use at home -- to root out all users from the work force.
Other officials say the same thing, but Walsh's title gives his words extra weight. He is director of the Division of Applied Research and the Office of Workplace Initiatives at the National Institute on Drug Abuse (NIDA), the chief federal drug research agency. The evidence he marshals is passed on to drug czar William Bennett and is widely disseminated, with the federal government's seal of approval. This role, and Walsh's status as a scientist, oblige him to uphold high standards of objectivity and competence. More than any other person, he is responsible for the intellectual honesty of the whole Reagan-Bush drive for workplace testing. A look at some of the central claims he has made in support of that drive reveals that he has not met his responsibility.
Two years ago Walsh testified in federal court that the "cost of drug abuse to U.S. industry" was nearly $50 billion a year, according to "conservative estimates." This claim is a staple of anti-drug rhetoric. It is frequently quoted without qualification by the media, and last year President Bush rounded it upward to "anywhere from $60 billion to $100 billion."
Here's how the figure was derived. In 1982 NIDA surveyed 3,700 households around the country. The Research Triangle Institute (RTI), a NIDA contractor in North Carolina, then analyzed the data and found that the household income of adults who had ever smoked marijuana daily for a month (or at least twenty out of thirty days) was twenty-eight percent less than the income of those who hadn't. The RTI analysts called this difference "reduced productivity due to daily marijuana use." They calculated the total "loss," when extrapolated to the general population, at $26 billion. Adding the estimated costs of drug-related crimes, accidents, and medical care produced a grand total of $47 billion for "costs to society of drug abuse."
Several things are wrong here, but the most glaring is the simpleminded conclusion that marijuana smoking caused the lower incomes with which it was associated. There are many respects in which the behavior of lower-income people differs statistically from that of upper-income people, but this mere correlation never establishes causality. It is quite probable, for example, that people who would admit to watching "Wheel of Fortune" every night have lower incomes than those who do not. Should we then conclude that television game shows decrease productivity? Or, by similar logic, should we conclude that Thunderbird wine hurts productivity but Chivas Regal scotch helps it?
You may be wondering why the RTI study used the awkward variable of "marijuana-use-daily-for-a-month-ever" as an indicator of drug abuse. Good question.
Actually, the RTI researchers had information on current use (at least once in the last thirty days) of drugs --including cocaine, heroin; amphetamines and LSD as well as marijuana--but they could find no connection to decreased income. If Walsh really accepts the logic linking a single month-long marijuana binge to decreased productivity, he must also conclude that current use does not decrease productivity. But he shows no signs of doing so.
Walsh now admits that the RTI study "is based on assumptions that need additional validation." One wonders why he hasn't passed that information on to Bush, Bennett, representatives of the drug-testing industry, and others who continue to treat the "cost of drug abuse to industry" as a scientifically established fact.
Anyway, Walsh says he has other evidence demonstrating the urgent need for workplace testing. He and another NIDA official edited and a published a 340-page collection of studies called: Drugs in the Workplace. When I interviewed Walsh recently, he drew my attention to three studies in particular --one done by the Navy and two by electric-power utilities-- that he said showed that drug users make poor employees.
Let's start with the Navy study. The primary subjects were 500 recruits who tested positive for marijuana in 1985 and were admitted anyway (those who tested positive for any other illegal drug were rejected). Two-and-a-half years later the Navy had discharged forty-three percent of those who had tested positive and only nineteen percent of those who had tested negative during the same recruitment period. Sounds like former potheads make lousy soldiers, right? Walsh even suggested to me that the lingering effects of pre-service marijuana use may have been responsible. "We do know that it stays in the system for a long time," he noted.
But hold on. Look at the difference between the positives and the negatives. The positives were about twice as likely to be black and to lack a high school diploma. Why not blame poor education or cultural factors correlated with race for their high discharge rate? (In fact, civil rights groups have complained that the Navy uses testing to keep blacks out of the service.)
Actually, there is an even simpler explanation. One of the conditions for being admitted to the Navy after testing positive for marijuana was agreeing to submit to rigorous "surveillance" thereafter, including unusually frequent drug tests (all naval personnel undergo three such tests a year). A third of the positives were discharged because they failed another drug test.
There are two problems here. One is that these sailors had no documented loss of productivity and exhibited no misbehavior except for drug use. So the natural conclusion isn't that people who have used drugs are more likely to be deficient workers (the only conclusion that would justify workplace testing), but rather that people who have used drugs are more likely than others to use drugs in the future. But even this conclusion is undermined by the second problem with the study: these sailors, facing more frequent testing than sailors in the control group, stood a better chance of getting caught using drugs.
If one discounts the sailors discharged for failing subsequent drug tests the difference in the discharge rates between the positives and the negatives shrinks from twenty-four to ten percent. Other forms of extra surveillance could easily account for that difference. Obviously, the more closely you watch someone, the more likely you are to see him misbehaving, especially if you think he is a troublemaker to begin with.
Another study cited by Walsh, conducted by the Utah Power and Light Company, makes the Navy experiment look like a paragon of scientific rigor. The Utah study compared the work history of employees who tested positive for drugs with a control group of employees whose ages and jobs were similar. The data showed a "significant difference between drug users and non-users in terms of being involved in accidents, being absent from work, and overutilization of health benefits," Walsh told me.
When one reads the study two flaws quickly stand out: there were only twelve positives in all (eleven for marijuana and one for cocaine), an absurdly small sample, and the control group was never tested for drugs. The study's conclusion could be rejected on these grounds alone.
But there is an even bigger problem. Eight of the twelve "drug abusers" (to use the Utah researchers' term) were tested because they were in accidents, and some were injured and needed time off to recuperate. Of the four remaining positives, two were tested for other performance-related problems and two because they had enrolled in a substance-abuse program. High absenteeism almost invariably precedes--and precipitates--both performance-related testing and submission to a substance-abuse program. Moreover, all employees who undergo a test are suspended until the results come back, which usually takes three or four days.
Incredibly, the Utah researchers included in their calculations the accidents and absenteeism directly associated with the testing of the twelve subjects. By this logic, you could link any trait to accidents and absenteeism. Round up employees who have been in an accident or have been absent a lot, test them for, say, type 0 blood, and send them home for a few more days of absenteeism. Then compare the accident and absenteeism rates within the type 0 group with those for a "control" group with no particular history of accidents or absenteeism -- and whose blood type, in keeping with the Utah methodology, wouldn't even be tested; it would just be assumed not to be type 0. Surprise, surprise: type Os had more accidents and missed work more often than people whose blood wasn't tested. Better get rid of everyone with type 0 blood.
And what about the "overutilization of health benefits" that Walsh had mentioned? He apparently misspoke. In the introduction to the NIDA monograph he calls the health benefits data "inconclusive." In fact, the positives consumed almost fifty percent less in health benefits than the control group. If the positives had used fifty percent more, would Walsh have found that "inconclusive"?
The other utility study cited by Walsh, which was done at the Georgia Power, Company, also focused on employees tested "for cause." But the Georgia researchers used a different control group: the 116 people who came up positive were compared with 713 who passed the test. This comparison, ostensibly fairer than that in the Utah study, found that the positives missed about five more days of work per year than the negatives.
But even the methodology that yielded this modest finding is flawed. The authors of the study note, with refreshing candor, that "the primary subject for the database is the problem employee." Indeed, the negatives and positives all missed work much more often than the company average. It's kind of like testing burglars for marijuana and concluding that --because those who tested positive had robbed a few more houses than those who did not-- marijuana makes ordinary people more likely to steal.
Moreover, couldn't alcohol, which is often consumed in conjunction with illegal drugs, contribute to the higher absenteeism of the positives? None of the employees was tested for alcohol. Of course, tests wouldn't prove much anyway, since alcohol is detectable in body fluids for only six hours or so, whereas cocaine persists for two or three days and marijuana for up to a month. That may be one reason that this and some other studies fail to consider the possibility that alcohol abuse is sometimes responsible for misbehavior attributed to illegal drugs.
When I asked Walsh about the flaws in these studies, he replied: "You could make the same argument about any database in higher research, such as cancer research, or mental health research." This is an insult to every scientist in those fields, none of the studies that Walsh touts has passed peer review (or is likely to), whereas thousands of papers on cancer and mental health have passed this minimal standard.
Walsh sometimes sounds quite reasonable and "scientific," especially when he talks about the mechanics of testing. He emphasizes that when a drug test comes up positive it should be confirmed by a more sophisticated analysis (a good idea, since poppy seeds, ibuprofen, and cold pills can trigger false alarms for heroin, marijuana, and amphetamines, respectively). He also stresses the importance of maintaining high laboratory standards and confidentiality.
But whenever he ventures beyond the mechanics of testing and tries to justify the enterprise itself, he shows an odd disregard for facts. For example, he frequently suggests that drug abuse is rampant not just in specific sectors of society, but broadly and pervasively. "The problem of drug abuse is so widespread in America," he declared in a speech last year, "that every company must assume that its employees will eventually be faced with a substance abuse-problem of their own, of a family member, of a co-worker, or of a friend." He always neglects to mention that NIDA'S own statistics show that the use of illegal drugs-marijuana, cocaine, PCP, the whole lot--has been declining sharply for years. Severe cocaine addiction (daily use) has increased, but primarily among the unemployed, who are beyond the reach of workplace testing.
I asked Walsh whether he thought that, given these data, his efforts to promote workplace testing might be misplaced. "I think drug abuse has gone down because of these workplace programs," he replied. I pointed out--again according to NIDA's data--that the decline began in 1979, well before testing had caught on. Then Walsh asserted that, whether or not drug use has declined among workers and whether or not severe addiction occurs largely among the unemployed, there are still an estimated 10 million working people who are using illicit drugs, and they represent a "much bigger problem than the few hard core...... The concept is to eliminate illegal drug use," he said, "not just to focus on those who are addicted."
This sentiment, of course, lies at the core of all of Walsh's uninformed claims. He adheres to the zero-tolerance line, which doesn't discriminate between use and abuse, between a secretary smoking marijuana on weekends and an AIDS-ridden prostitute smoking $100 worth of crack a day. From this perspective, drug testing--which also fails to discriminate between casual and chronic user--makes sense.
But then, Walsh does believe that even occasional marijuana use can have devastating effects. "I think we have reached the point where the involvement of marijuana in accidents exceeds that of alcohol," he said. Where is his proof for this dramatic statement? He doesn't have any, but "it's one of the things in my research program we're trying to do right now." Note the procedure: Walsh reaches his conclusion first, then sets out to prove it.
As a scientist, Walsh is probably in over his head. He has testified to having "over twenty years of experience (fifteen years in the laboratory) in research on the physiological and behavioral effects of psychoactive drugs." Well, sort of. Before joining NIDA in 1980, Walsh spent fourteen years at the Naval Medical Research Institute studying what might be called underwater altered states. For his Ph.D., which he received from American University in 1973, he studied nitrogen narcosis ("rapture of the deep"). Typical of his pre-NIDA publications is a paper about what happens to mice when they are given morphine under "hyperbalic abnormally high] pressure" and an article called "Should Divers Take Drugs?" published in something called Faceplate.
On the other hand, if you view Walsh not as a scientist but as a propagandist, you have to admit he has done his job well. A decade ago virtually no companies had testing programs. Now a majority --including such bastions of liberalism as The New York Times-- test employees, job applicants, or both.Walsh can't take all the credit, but he certainly has done his part. Perhaps that is why he remains so strangely sanguine when confronted with all the inconsistencies in his logic. He has already won the day. "Drug testing," he says with pride, "is here to stay."