From @helios.physics.utoronto.ca:LISS@FNALD.FNAL.GOV Fri Oct 1 17:14:48 1993 Received: from helios.physics.utoronto.ca by cepheid.physics.utoronto.ca with SMTP id AA15562; Fri, 1 Oct 93 17:14:48 -0400 Received: from FNALJ.FNAL.GOV ([131.225.108.4]) by helios.physics.utoronto.ca with SMTP id <1697>; Fri, 1 Oct 1993 17:14:36 -0400 Date: Fri, 1 Oct 1993 17:14:18 -0400 From: LISS@FNALD.FNAL.GOV To: pekka@physics.utoronto.ca Message-Id: <931001161418.2040918a@FNALD.FNAL.GOV> Subject: Almost forgot Status: R From: FNALD::BRANDENBURG "George - (617)495-2824" 1-OCT-1993 12:12:21.64 To: ALVIN,HUTH,LISS,CLAUDIOC CC: BRANDENBURG Subj: Am I losing my marbles... For fun I count the number of marbles in my pocket looking for a special new kind of marble. My prediction based on much study is that on the average there should be 5.5 normal marbles in my pocket, and that the normal marbles are monchromatic and come in three different colors. I reach in my pocket and find 12 marbles. Much to my surprise three of the marbles are dichromatic so I also have 15 color occurances in my hand. I decide to estimate the statistical significance of my result if I assume that there are only normal marbles and not special ones. My dilemma is whether I should find the degree to which an expectation of 5.5 marbles is consistent with the observation of 12 marbles, OR the consistency of an expectation of 5.5 color occurances with an observation of 15 color occurances. The probability of 5.5 marbles fluctuating to 12 (or more) is 1.5% while the probability that 5.5 color occurances fluctuates to 15 (or more) is only 0.06%, so my choice has a major bearing on whether or not I may have found special marbles. Although the presence of dichromatic marbles already seems like a good sign I have found special marbles, I nonetheless conclude that their very existence negates the use of simple Poisson statistics in the case of color occurances. If the color occurances can come in correlated pairs, then it is certainly improper to find the significance for a predicted number of individual uncorrelated color occurances to fluctuate to a larger number, unless I fold in a model for the correlations. Therefore on the basis of the number of marbles observed I conclude that my observation was remarkable, but that I haven't demonstated the existence of special marbles. Having seen the dichromatic marbles I also redo my prediction about the expected number of normal marbles. I discover that in fact only 5.0 normal marbles are to be expected, but that dichromatic normal marbles do exist, and there should be 0.5 of them on average included in the 5.0 normal marbles in my pocket. In other words I should expect 5.0 normal marbles with 5.5 color occurances. I now redo my estimate and find that the probability that 5 expected marbles will result in 12 (or more) is 0.6%, which is still less than three standard deviations. Finally I decide to use all of the information at hand. So I define a a new variable which characterizes the marbles, namely the fraction of the time that a marble is dichromatic. My prediction is this variable on the average should have the value of 0.1. My observation was that for the 12 marbles in my pocket the variable had the value 0.25. So now my task is to estimate the probability for both observations to occur, namely that I observed 12 marbles instead of 5 and that their dichromatic fraction was 0.25 instead of 0.1. I consider combining these observations into a chisq... (to be continued)