python - How do I calculate Pr(model|data) in Bayesian inference with extremely small numbers? -

i'm doing bayesian inference (manually, using grid search) in python. want calculate probability of each model given data. problem can calculate 'evidence' in log, otherwise 0. so, though between 0-1, can't results for:

pr(data|model1) / (pr(data|model1) + pr(data|model2))   

since each term 0 in non-log form.

any ideas?

thanks

let logpr1 , logpr2 log(data|model1) , log(data|model2), respectively, , suppose

in [57]: logpr1 = -802  in [58]: logpr2 = -800 

if try express probabilities (not logarithms of probabilities), 0:

in [59]: np.exp(logpr2) out[59]: 0.0 

now want compute

log(pr(data|model1) / (pr(data|model1) + pr(data|model2))), 

which can write as

log(pr(data|model1)) - log(pr(data|model1) + pr(data|model2)). 

for last term, can use function numpy.logaddexp (which essential tip in answer; see scipy.misc.logsumexp). calculation is:

in [60]: logp = logpr1 - np.logaddexp(logpr1, logpr2)  in [61]: logp out[61]: -2.1269280110429918 

in case, number not small. in fact, can express plain probability:

in [62]: np.exp(logp) out[62]: 0.11920292202211526