Countdown To CAOmageddon: Flaw #26 - Professional Error

One of the boldest examples of regulatory ignorance that I've ever seen appeared in our County on February 4, 2011. That is the date of a letter, sent out under Erik Stockdale's signature, that contains an explanation of "risk" by Dr. Tom Hruby. The explanation is, quite simply, wrong.

In short, Dr. Hruby says that "risk" is related to uncertainty surrounding poor decision making. He goes on to equate levels of "risk" to the legal concepts of "beyond a reasonable doubt," "clear and convincing proof," and a "preponderance of evidence." Wow! That is such confused testimony about the meaning of "risk" that it brings to mind that book Disorder In The Court which recounts actual exchanges between lawyers and witnesses in court.

Attorney: She had three children, right?
Witness: Yes.
Attorney: How many were boys?
Witness: None.
Attorney: Were there any girls?
Witness: Your Honor, I think I need a different Attorney. Can I get a new Attorney?

Likewise, I think we need a different risk expert. Can we can get a new risk expert? Dr. Hruby is describing "error" not "risk," and they are as different from one another as boys are from girls.

So what is "error" anyway? Error is when we reach a conclusion different from something's true value, and the probability of drawing the wrong conclusion is uncertainty (i.e., range of uncertainty and confidence intervals).  Think of it with respect to fire alarms. We want a fire alarm to go off when there is a fire, but stay quiet when there isn't a fire. However, if the alarm goes off when there is no fire, that's a false alarm ... and that's bad. In the language of "error," false alarms are called "Type 1" errors aka "false positives." The opposite kind of error occurs if the alarm stays quiet when there really is a fire. That's a failed alarm, and that's bad too. Failed alarms are called "Type 2" errors or "false negatives." In science, as with fire alarms, we want to limit uncertainty by controlling the probability of Type 1 and Type 2 errors. Neither uncertainty nor error is risk.

Hruby has mistaken "risk" for the probability of "failed/false alarms." Why should we care? We care because his deeply flawed reasoning about "risk" has found its way verbatim into our CAOs. Here's an excerpt from our proposed wetland CAO.

The approach to sizing wetland buffers taken in this ordinance is intended to be a medium risk alternative based on the premise that there is clear and convincing proof, and a high probability that the buffers will protect wetland and associated fish and wildlife habitat conservation area functions and values. The standard for evidence needed to meet this criterion is less than that needed for “beyond a reasonable doubt,” but higher than that needed for a “preponderance of the evidence.” The probability that the buffers will not be adequate is relatively low; between 5% and 50%.

Ack! Wrong, wrong, wrong. No, no, no! Just this one paragraph has so many holes in it that pages and pages of technical criticism could be written on this alone. Suffice to say, though, that Hurby's explanation has nothing to do with ecological risk. Furthermore, the eco-hypochondriacs out there are not interested in controlling for the uncertainty of false positives. They are more than willing to accept an unlimited number of false positives to avoid any probability of a false negative. That's not risk management. That's just uncontrolled (Type 1) error. That's a failure to manage uncertainty.

All this talk of "error" reminds me of another passage from Disorder In The Court:

Attorney: Doctor, before you performed the autopsy, did you check for a pulse?
Witness: No.
Attorney: Did you check for blood pressure?
Witness: No.
Attorney: Did you check for breathing?
Witness: No.
Attorney: So, then it is possible that the patient was alive when you began the autopsy?
Witness: No .
Attorney: How can you be so sure, Doctor?
Witness: Because his brain was sitting on my desk in a jar.
Attorney: I see, but could the patient have still been alive, nevertheless?
Witness: Yes, it is possible that he could have been alive and practicing law.

Other possible occupations might include working for Ecology or drafting our County's CAOs.