The Art Of Probability PDF

Preface

Every field of knowledge has its subject matter and its methods, along with a style for handling them. The field of Probability has a great deal of the Art component in it-not only is the subject matter rather different from that of other fields, but at present the techniques are not well organized into systematic methods. As a result each problem has to be "looked at in the right way" to make it easy to solve. Thus in probability theory there is a great deal of art in setting up the model, in solving the problem, and in applying the results back to the real world actions that will follow. It is necessary to include some of this art in any textbook that tries to prepare the reader to use probability in the real world of science and engineering rather than merely admire it as an abstract discipline and a branch of mathematics.

It is widely agreed that art is best taught through concrete examples. Especially in teaching probability it is necessary to work many problems. Since the answers are already known the purpose of the Examples and Exercises cannot be to "get the answer" but to illustrate the methods and style of thinking. Hence the Examples in the text should be studied for the methods and style as well as for the results; also they often have educational value. Thus in solving the Exercises style should be considered as part of their purpose.

It is not enough to merely give solutions to problems in probability. If the art is to be communicated to the reader then the initial approach-which is so vital in this field-must be carefully discussed. From where, for example, do the initial probabilities come? Only in this way can the reader learn this art-and most mathematically oriented text book simply ignore the source of the probabilities!

What is probability? I asked myself this question many years ago, and found that various authors gave different answers. I found that there were several main schools of thought with many variations. First, there were the frequentists who believe that probability is the limiting ratio of the successes divided by the total number of trials (each time repeating essentially the same situation). Since I have a scientific-engineering background, this approach, when examined in detail, seemed to me to be non-operational and furthermore excluded important and interesting situations

Second, there are those who think that there is a probability to be attached to a single, unique event without regard to repetitions. Via the law of large numbers they deduce the frequency approach as something that is likely, but not sure.

I also found that there were flagrant omissions in all the books; whole areas of current use of probability, such as quantum mechanics, were completely ignored! Few authors cared to even mention them.

Third, I found, for example, that the highly respected probabilist di Finetti [dF, p. x] wrote at the opening of his two volume treatise,

Probability Does Not Exist.

Fourth, I found that mathematicians tend to simply postulate a Borel family of sets with suitable properties (often called a u-algebra or a Borel field) for the sample space of events and assign a measure over the field which is the corresponding probability. But the main problems in using probability theory are the choice of the sample space of events and the assigment of probability to the events! Some highly respected authors like Feller [F, p. x] and Kac [K, p. 24] were opposed to this measure theoretic mathematical approach to probability; both loudly proclaimed that probability is not a branch of measure theory, yet both in their turn seemed to me to adopt a formal mathematical approach, as if probability were merely a branch of mathematics and not an independent field.

Fifth, I also found that there is a large assortment of personal probabilists, the most prominent being the Bayesians, at least in volume of noise. Just what the various kinds of Bayesians are proclaiming is not always clear to me, and at least one said that while nothing new and testable is produced still it is "the proper way to think about probability".

Finally, there were some authors who were very subjective about probability, seeming to say that each person had their own probabilities and there need be little relationship between their beliefs; possibly true in some situations but hardly scientific.

When I looked at the early history of probability I found the seeds of most of these views; apparently very little has been settled in all these years.

I also found that there were flagrant omissions in all the books; whole areas of current use of probability, such as quantum mechanics, were completely ignored! Few authors cared to even mention them.

What was I to make of all this? Following an observation of Disraeli, I decided to find out what I myself believed by the simple process of writing a book that would be, to me at least, somewhat more believable than what had I found. Of necessity, being application oriented, it would have a lot more philosophy than most textbooks that postulate a probability model and then present the techniques without regard to understanding when, how, and where to use the techniques, or why the particular postulates are assumed. Indeed, most mathematicians blandly assume that probability is a branch of mathematics without ever doubting this assumption. Such books tend to discourage the reader from making new applications outside the accepted areas-yet it is hard to believe that the range of applications of probability has been anywhere near exhausted. It seems to me that the philosophy of probability is not a topic to be avoided in a first course, but rather, in view of the dangers of the misapplication of the· theory (which are many), it is an essential part.

Initially I wanted to provide some organization and structure for the various methods for solving probability problems instead of merely giving the usual presentation of problems and their solution by any method, selected almost at random, that would work. My success has been of limited extent, but I feel that I have taken a few steps in that direction.

I further decided that it would be necessary to build up my intuition about problems so that: (1) many of the false results that are so easily obtained would be noticed, and (2) that even if I could not solve a problem still I might have a feeling for the size and nature of the answer.

Finally, as a sometime engineer, I well know that few things are known exactly, and that many times probabilities used in the final result are based on estimates, hence the robustness (sensitivity) of the results with respect to the small changes in the initially assumed probabilites and their interrelationships must be investigated carefully. Again, this is a much neglected part of probability theory, though clearly it is essential for serious applications.

The availability of computers, even programmable hand held ones, greatly affects probability theory. First, one can easily evaluate formulas that would have taxed hand computation some decades ago. Second, often the simulation of probability problems is now very practical [see Chapter 10]; not that one can get answers accurate to many decimal places, but that a crude simulation can reveal a missed factor of 2, the wrong sign on a term, and other gross errors in the formula that purports to be the answer. These simulations can also provide some intuition as to why the result is what it is, and even at times suggest how to solve the problem analytically. Simulation can, of course, give insight to problems we cannot otherwise solve.

The fact that probability theory is increasingly being used to make important decisions is a further incentive for examining the theory carefully. I have witnessed very important decisions being made that were based on probability, and as a citizen I have to endure the consequences of similar decisions made in Washington D.C. and elsewhere; hence I feel that anything that can clarify and improve the quality of the application of probability theory will be of great benefit to our society.

It is generally recognized that it is dangerous to apply any part of science without understanding what is behind the theory. This is especially true in the field of probability since in practice there is not a single agreed upon model of probability, but rather there are many widely different models of varying degrees of relevance and reliability. Thus the philosophy behind probability should not be neglected by presenting a nice set of postulates and then going forward; even the simplest applications of probability can involve the underlying philosophy. The frequently made claim that while the various foundational philosophies of probability may be different, still the subsequent technique (formalism) is always the same, is flagrantly false! This book gives numerous examples illustrating this·fact. One can only wonder why people make this claim; the reason is probably the desire to escape the hard thinking on the foundations and to get to protection of the formalism of mathematics. Furthermore, the interpretation of the results may be quite difficult and require careful thinking about the underlying model of probability that was assumed.

This book is the result. It is one man's opinion using a rather more scientific (as opposed to mathematical) approach to probability than is usual. It is hardly perfect, leaves a lot open for further work, and omits most of subjective probability as not being scientific enough to justify many actions in the real world based on it. Not only are many scientific theories and problems based on probability, but many engineering tasks, such as the launching of space vehicles, depend on probability estimates. Perhaps most important, many political, biological, medical, and social decisions involve probability in an essential way. One would like to believe that these decisions are based on sound principles and not on personal prejudices, politics and propaganda.

In order to strengthen the reader's intuitions for making probability judgements, I have included a reasonable number of tables of results. These tables are well worth careful study to understand why the numbers are the way they are. I have also examined various results to show how they agree, or disagree, with common experience. In my opinion this is a necessary part of any course in probability, since in normal living one has a very limited exposure to the varieties of peculiar results that can occur.

The material has therefore been carefully presented in a pedagogical manner, including deliberate repetitions, for the benefit of the beginner, and not in the logical order for the benefit of the professor who already understands probability.

But one's efforts are limited, and it occurred to me that others might want to examine, criticize, change, and advance further the problem of what probability is, hence what I found is presented here for their consideration. I doubt that in the future there will be any single, widely accepted model of probability that is useful for all applications; hence the need for multiple approaches to the topic, and in time new ones not yet discovered!

I am greatly indebted to Professor Roger Pinkham of Stevens Institute, Hoboken, for endless patience and guidance while I tried to learn probability theory, as well as for a large supply of Examples to illustrate various points. I am, of course, solely responsible for the contents of this book and he cannot be held responsible for my opinions and errors; he did his best!

I am also indebted to Professor Bruce MacLennan of the University of Tennesee, Knoxville, for many stylistic improvements and suggestions for a dearer presentation; again he is not responsible for the final product. Professor Don Gaver has been of help in numerous discussions.

Author: Richard W. Hamming