From @helios.physics.utoronto.ca:LISS@FNALD.FNAL.GOV  Tue Sep 28 14:59:56 1993
Received: from helios.physics.utoronto.ca by cepheid.physics.utoronto.ca with SMTP id AA04102; Tue, 28 Sep 93 14:59:56 -0400
Received: from FNALI.FNAL.GOV ([131.225.108.3]) by helios.physics.utoronto.ca with SMTP id <1651>; Tue, 28 Sep 1993 14:59:52 -0400
Date: 	Tue, 28 Sep 1993 14:59:33 -0400
From: LISS@FNALD.FNAL.GOV
To: pekka@physics.utoronto.ca
Message-Id: <930928135933.23a098ec@FNALD.FNAL.GOV>
Subject: Combined probability
Status: RO

From:	WHCDF::WINER        27-SEP-1993 22:55:06.54
To:	@PROB_DIST.DIS
CC:	
Subj:	Combining Probabilities


	On adding results from more than one experiment.
					Hughes, Winer, Tipton, Watts

	Lately there has been some confusion on how to combine probabilities
from more than one experiment into an overall probability.  We propose
using the product probability (as many before us have suggested).  
We define the  product probability P_tot=P1*P2*P3,
where P1, P2 and P3 are the three probabilities defined by the sum of the
the poisson distribution for observing N1 or more events when expecting 
a background of X1.   For our case, we have 
SVX, X1=1.9+/- 0.2 and N1 is 6., and P1 is 0.0142 (there is a typo in the 
PRL, sorry) for SLT, N2=7,X2=3.0, and the DiLeptons, N=3, and 
X3=0.61+/-0.13.  We get P_tot for the dileptons*SLT*SVX= 6.4E-5.

	It is slightly counter-intuitive, but the probability that in the
absence of any top we observe a product probability of P_tot or less
is not the value of P_tot.  Take the following example:  Consider 
three experiments which are completely consistent with background:  30 on a 
background of 30, 10 on a background of 10, and 5 on a background of 5,
for example.  All these will have P near 0.5 so 
let's assume P1=P2=P3=0.5.  P_tot would be 1/8.  Obviously the probability
that we would observe the above outcome of the three experiments without
top is not 1/8, but something much larger than 1/8, near 0.5

	The solution to this problem is easier to visualize in two dimensions,
with only two experiments yielding P1 and P2.  In the absence of top, 
the two distributions for P1 and P2 should be flat from 0. to 1.  Consider a
square with Y axis P2 and x axis P1.  Each point in this square is
equally likely to be populated by a set of experiments in the
absence of top.  P1*P2 defines the area of a rectangle
in the lower left hand corner.  However if you want to answer the question:
"How often will we get a product probability (in the absence of top) less than
P_tot?",  the answer is not P1*P2.  We must find the area which satisfies
P1*P2<P_tot.  

	          p2=1,p1=P_tot
		    |	
		    V
		____*______________________________________
		|   . 					  |
		|					  |
		|   .* 					  |
		|					  |
		|   .					  |
		|	*				  |
		|   .					  |
		|					  |
		|   .		*			  |
		|					  |
		|   .			*		  |
		|				*	  |
		|   .					  * P2=ptot,P1=1
		| 					  |
		___________________________________________


This is gotten by integrating the area under the function
P2=P_tot/P1 from P_tot to 1, and adding it to the area to the left of the
dotted line, which is equal to P_tot.  
This area we call the "Renormalized P" and is

		Renormalized P = P_tot (1- Log(P_tot) ).

With three variables the answer is

 Renormalized P = P_tot (1- Log(P_tot) + ( ( (log(P_tot) )**2 )/2 ) ).
(This is the formula the jet probability guys use to remormalize their
probabilities.)
	For our three results P_tot is 6.4e-5 and Renormalized P is 0.0037,
a factor of 57 times more likely.  The origin of the log(P_tot) correction 
factors is that we could have gotten a very large P1 and a very small P2 
such that their product was less than P_tot.  One might argue that if we 
observed 0 SVX tags on a background of 2.0 and thus got a P1 of 1.0, we 
would not be publishing these three papers now.  Thus the degradation of 
P_tot to the Renormalized P is too conservative.  We agree with this reasoning.
In hind-sight it is hard to know what one would have done under certain
hypothetical outcomes, however it seems likely 
that if any of the experiments observed no excess over background (ie, negative
top) we would not be publishing.  Thus a natural cutoff in calculating the
area under the curve is to not allow any P1, P2 or P3 to have a 
value larger than 0.5.  Thus the above limits of
integration change and the correction factor is smaller.
If one redoes the above calculus with an arbitrary cutoff of "f", then the
expression is:

 Renormalized P = P_tot (1- Log(P_tot/F**2) + ( (log(P_tot/f**2)**2)/2 ) ).

	If we choose f=0.5,  Renormalized P =0.00235, or a factor of 44 times
as likely.
	In fact, the above calculus is valid for counting experiments with
large statistics, giving a smooth curve in P1, P2, P3 space of P_tot/P1.  
This is not the case for us, where we have large discreet jumps between
observing N and N+1 events on an given expected background.  Thus we have 
used the above logic, but not the above formulas to do the more correct 
calculation.   We calculate the poisson probabilities for our three 
experiments for n=0 to infinity. (it turns out that going to 200 is complete 
overkill and what we used in the code).  The triple sum
over product probabilities P1(i)*P2(j)*P3(k) will be 1.0 for i, j,k summed from
0 to infinity.  We sum up the  product probability for each case with
P1*P2*P3 less than P_tot, and P1, P2 and P3 all less than f, which we take to be
0.5.  We get a result of 5.4E-4, or .5%, which we think is our overall 
likelihood that we would see these three results in the absence of top.

	We have tested our code in some simple limiting cases, but we still 
would like to push it a little bit to gain some more confidence.
We also know how to feed in correlations in background to correct for this, 
once those correlations are understood.
	One of us will give a talk this week at top (if we are encouraged
to do so) to try to make this clear, especially if this note has failed 
to do so.  Please send comments.
				Richard, Brian, Paul and Gordon.