I found the discussion useful; since it prompted me to design a type of spherical ball for use in such drawings where each of 366 such balls would have exactly the same number of atoms, and be mechanically almost identical to such an extent, that no mechanical measurement coud distinguish between them, since every atom on the surface of each of the 366 balls, would occupy identical locations in a three dimensional lattice; yet the balls are uniquely identifiable; and differ in total mass by less than about one part in 10^25. I won’t be able to calculate the mass spread between the balls till I go to work tomorrow, and get to my reference books.

Evidently my Mathematics degree led me down the garden path when it comes to elementary statistics theory.

Quite true.

By bullheadedness I refer to your insistence that the dates have no inherent relation to each other and that somehow sampling without replacement from an ordered list is single event. It’s a well understood problem and amenable to analyses such as the poker hand test for randomness of shuffle (see Knuth) and similar tests.

As for the rest: there’s little point in treading old ground. I suggest at least looking at the references I’ve posted previously .

I also suggest tabling this discussion as it seems to have stagnated.

And we are talking about a single event.

Is a 100 meter dash a single event; or do you want to claim that each step must be considered as a separate event.

So unlike HR, I DO claim that a single coin toss is the same as a single draft lottery; and it has no statistical significance at all.

And moreover, it could land on its edge; the laws of physics do not preclude a coin from landing and remaining on its edge..

The only thing about the calendar sequential draw is that it happens to be a sequence of unique symbols which somebody just might recognise.

Besides, any ocntention that a given draw is biassed, and therefore unfair, can only be proven by conducting a huge multiplicity of such draws; and that destroys the premise that there was only one such event. (before someone chose to declare it biassed).

I’m mildly amused by DAV’s contention that I am being “bullheaded”.

We have that problem in the California Legislature.; The Democrats keep tying to illegally raise taxes to keep funding their gravy train projects that keep the non-taxpaying californians voting them into office. The Republican point out that we have a recession, and raising taxes will simply move more jobs and industries out of califonia, so that will never balance the budget, and the only solution is to stop government spending; which is what the taxpaying Californians have to do when their budget doesn’t balance.

So the Democrats blame the republicans for lack of “bipartisanship”, yet when Republican “cross the aisle”, as that old fool John McClain does all the time, and tried to sell it as a Presidential strategy, the Democrats simply take the gratuity, and then respond with; “now all we want is the land next to our land.”.

Bullheadedness, is nothing more than two views of a problem that are not commensurate with each other.

Back at the turn of the 1960s; when I was a junior faculty lecturer in the Physics Dept of my Alma Mater, our once a month faculty meeting to discuss recent literature, always began with the latest shot in the “Dingle versu Mc Cray” battle on the “clock paradox”. These two chaps argued back and forth in the physics literature as to who got older, or whether anybody got older than the other.
I believe that the feud died out when it became apparent, that each of the two combatants was actually describing a different problem from what the other chap was talking about. The dispute never got resolved; because there really was no dispute; just tow persons talking over each other about two entirely different discussions. The distinction was as I recall quite esoteric; and no I have no recollection of what the two discussions really were about.

George; who fortunately is only a single event in world history.

Surprisingly, I had looked at that exact wikipedia entry you linked to before writing my last post. I’ll go have a look at interval data and ratio data in a bit.

“Since we are dealing with an ordered list, the ordinal nature of dates is of value. Of course, we could have thrown that away but that would mean determining the randomness of the output would be nearly impossible. When dealing with nominal data, I would like to have every data value seen at the very least 10 times. The lottery was only run for 4 years so there is insufficient information for any meaningful test. In addition, we have been discussing the very first lottery where each date has been listed only once. Considering the dates as nominal for this purpose dooms the outcome from the start.”

That put a nice cap on this thread. Thank you again for your patient and thorough explanations. I got a lot more than “ribbon seals taste like chicken” out of this thread by following the thread this far.

“Sorry

You should never apologize for thinking unless sarcasm is intended. Apology not accepted :-)”

Oh, the apology was for going off the topic of sequences. I brought up the “one-off single coin toss” twice before it hit me that it wasn’t a sequence. That was more of a forehead-slapping-Doh! moment of mine.

I think you and I are the last ones still on this thread, particularly since it’s been bumped to the second page, so let’s turn out the lights here and go see what’s new on Page One, eh?

There are four categories of values used in statistics (in ascending order): nominal, ordinal, interval and ratio. The category is an inherent attribute of any variable. Each category (or level) allows certain mathematical operations. Reasonable explanation here:

http://en.wikipedia.org/wiki/Nominal_data

The dates aren’t being assigned ordinal values — it’s an inherent property. As it turn out, dates are actually interval data because the difference between them is meaningful but an examination of the draft lottery problem reveals that differencing is likely an extraneous feature in solving it.

The draft lottery goal can be stated as: produce a randomly ordered list of from an ordered list. This is is almost the goal of any random number generator. The major difference is that a random number generator selects with replacement from the input list while the lottery selected without replacement. IOW: they didn’t want duplicate dates. The only real difference between the selection methods is the resulting distribution because the probabilities change for each successive draw in the latter.

Since we are dealing with an ordered list, the ordinal nature of dates is of value. Of course, we could have thrown that away but that would mean determining the randomness of the output would be nearly impossible. When dealing with nominal data, I would like to have every data value seen at the very least 10 times. The lottery was only run for 4 years so there is insufficient information for any meaningful test. In addition, we have been discussing the very first lottery where each date has been listed only once. Considering the dates as nominal for this purpose dooms the outcome from the start.

Since the goal was to produce a randomly ordered list, it should have been picking the dates at random. Because it’s selecting from an ordered list, we can use the ordinality of dates to see where the selections occur.

I thought about the coin toss and if we flip a coin once and only once, we can’t say anything about any possible bias in the coin. You need at least 2 of something to make a sequence to even begin an analysis and it won’t be a good analysis at that. I don’t think the one-off coin toss has anything to do with this discussion.

That’s correct. Determining the bias of a coin though is applicable to the lottery problem. One of the tricks in statistics to convert the problem at hand into a Well Known Problem with a Well Known Solution. The biased coin test happens to be one of those. We convert to it by asking how many times the second date in a pair is selected above (heads) or below (tails) the first date in the pair and compare the distribution to that of an unbiased coin. To use one the Well Known coin bias solutions (there are many of these), it’s necessary to ensure that the “coin flips” are truly independent because real coin flips are and their independence is an assumption of the coin bias problem solution.

Note that independence between events and variables is not a hard requirement but without it life gets complicated as do the computations. It also means you can’t readily convert to a Well Known Problem/Solution because, invariably, each took the easy way out.

Sorry

You should never apologize for thinking unless sarcasm is intended. Apology not accepted :-)

Thank you very much for your thoughtful consideration of my comments.

Above, you wrote:

“Actually, HR, you’ve come very close to the heart of the matter.

If the drawing were of nominal data (meaning having no particular order), such as zip codes, area codes, city names, etc., then there would be no features to examine and the only evidence for non-randomness would be if some values appeared more (or less) often than if they had been randomly selected from the general population. This would require a sequence many times longer than the number of values to ensure fairness of the test.”

Yes. You’ve got the sticky point nailed down exactly… with a thirty-penny nail. If you view the dates Jan 1 through Dec 31 as ordinal data, then all your (patient and thorough) arguments are valid. If the dates are viewed as nominal data and that some poor schlub got stuck with the symbol of March 22 due to an accident of birth, and won an all-expenses-paid trip to Viet Nam, then we don’t know if the one-off draw was biased. I didn’t see any particular person’s birth date as being any different from any other birth date other than the arbitrary symbol (Dec 28 e.g.), which marked it. That’s how I saw the dates and why I brought up using arbitrary symbols for dates. I saw it as man putting an external meaning on arbitrary dates by putting a ranking (1-366) on them after the drawing. After all, we could stick with the flower, star, ampersand and put them in a line on the table and never map them to the ordinal numbers.

But then (honest question) does it matter if you assign ordinal numbers before or after drawing nominal dates? Are the statistical tests then meaningful or meaningless? Attempting to answer my own question, I suppose it’s just like doing quantitative analysis on qualitative data.

BTW, I thought about the coin toss and if we flip a coin once and only once, we can’t say anything about any possible bias in the coin. You need at least 2 of something to make a sequence to even begin an analysis and it won’t be a good analysis at that. I don’t think the one-off coin toss has anything to do with this discussion. Sorry.

If the drawing were of nominal data (meaning having no particular order), such as zip codes, area codes, city names, etc., then there would be no features to examine and the only evidence for non-randomness would be if some values appeared more (or less) often than if they had been randomly selected from the general population. This would require a sequence many times longer than the number of values to ensure fairness of the test.

Make no mistake though, EVERY data set is ONE sequence. In fact, dividing a large set into smaller sets often conveys little advantage except possibly making computation easier.

The ordering of the dates supplies yet another source of information, which in turn permits shorter sequences to be used because (being clever) the tests treat the data as a sequence of binary digits (two nominal symbols) with lots of values. The trick is to make sure that the digits are obtained independently. The tests essentially say, THIS sequence is consistent with RANDOM or NOT RANDOM, which, in turn, is a statement about the process in which the sequence was obtained. Many tests on random number generators use FAR LESS than the number of values which can be obtained: a 32 bit value has over 4 billion unique states.

Technically less. It’s mostly a device for seeing if the sequence calculations are reasonable. I should get the same answer either way — AND — if I do, then they are essentially equivalent, yes?.

Even then, it’s far more probable that a sequence has certain features than not. This makes sequences with those features more likely than others because it belongs to a special group.

If someone were to fire a gun at a target and they miss entirely is that evidence FOR or AGAINST that person having a good aim? NB: I am NOT saying PROOF! George seems to be saying “Neither” because only one shot was fired.

If you catch a person in a lie is that evidence FOR or AGAINST calling that person a liar?

Likewise, if I obtain a sequence with highly unusual features, extremely inconsistent with randomness is it evidence FOR or AGAINST randomness? Note that I have arrived at how consistent those features will be via TWO separate paths.

If you go back: George wanted to know how one sequence can be looked at askance and another not. I have shown why, more than once. At this point, I think he’s being bullheaded.

I said symbols because instead of using numbers or the unique days of the calendar year, they could have used say, a flower symbol

Yes, that seems George’s hang-up as well. It is quite possible to look at the distribution of hi-lo pairs. This is possible since dates are ordinal data — regardless of the number of symbols used to express them.

—-

Carry on. I’m enjoying the discussion ‘twixt you two and I’ve learned a lot.

Actually, I’ve pretty much reached the end of what I have to say. If one goes back and reads the references from my last post, one will discover that ALL of them are tests on SINGLE sequences. The reason: there is NO difference between a series of short experiments and one very long one.

These tests are the culmination of the works of many people with significant stature in the statistical world. If the arguments of Knuth, Dahlquist, Spearman, Von Mises, et alia aren’t convincing then what chance have I?

At this point, I will only entertain argument that the various tests for randomness don’t apply but every reference I’ve seen says they do.

I’ve been following this with interest but I’m get stuck on the concept of a single experiment never repeated.

If you flip a coin once and never flip it again; you accept the results of that experiment and move on. What can you say about the fairness of the coin? What was the probability of the result.

If you draw-from-a jar-throw-up-in-the-air-shuffle-and-deal-from-a-shoe-or-whatever 366 symbols and do it only once, isn’t the same as a single coin toss? What can we say about biases? What is the probability of any given result?

I said symbols because instead of using numbers or the unique days of the calendar year, they could have used say, a flower symbol for August 13th, a star symbol for March 22nd and so on. Given a drawing of 366 symbols that the human mind does not associate with any sequence, such as numbers or letters of the alphabet or days of the year, most would be willing to accept the one-time result as no more likely than any of the other possible results.

Think again; what is the probability of the numbers 1-2-3 being drawn right from the start of a a drawing from the numbers 1-366 vs. the probability of drawing a star-flower-ampersand right from the start from a pool of 366 symbols? The probabilities are exactly the same if the drawing occurs once and only once. We can’t say much about bias, either, eh?

Carry on. I’m enjoying the discussion ‘twixt you two and I’ve learned a lot.

I refer you to many of the excellent discourses on testing a sequence for random distribution.

Knuth, D.E., 1981, Seminumerical Algorithms, 2nd ed., vol 2 of The Art of Computer Programming, (Addison-Wesley), CH. 3, particularly section 3.5. Discusses generation of random numbers and testing their sequences for randomness.

Dahlquist, G. and Bjorck, A., 1974, Numerical Methods, (Prentice-Hall), ch 11.

From the web:

http://www.fi.muni.cz/~xkrhovj/lectures/2005_PA168_Statistical_Testing_slides.pdf
Particularly, the sections entitled “Frequency (Monobit) Test”, “Runs Test” and “NIST Testing Strategy”

http://www.math.umbc.edu/~rukhin/papers/talk.pdf
The NIST document.

Again, note that I have continuously referred to tests on the ranking of the values instead of the actual values. I also referred to pairs with reversals (hi-lo). This is important as it makes computation of the probabilities tractable. Other features can be used in lieu of reversals. The idea is to convert the problem into a problem of runs of binary values.

The NIST document refers to Von Mises (1964). There are many interesting discussions on the meaning of “probability”. Von Mises’ arguments are often in the foreground.

Sequence delivered whole: if you had all of those sequences in a jar and pulled one out at random what’s the probability that sequence would have at least one numerical reversal? It’s almost certainty. Another way of putting it: what’s the probability that, on the very first try, you would select the one and only one (or one of two depending upon your definition of “reversal”) that was perfect — the proverbial needle from a haystack? Wouldn’t the improbability of it certainly be grounds for suspicion and evidence for need of investigation? You wouldn’t entertain the thought that maybe, just maybe, the process isn’t quite as random as claimed — even for a moment?

What are the probabilities for 2,3, etc. reversals? If you compute those, you will find that you should expect N reversals on the average. Another way of putting it: a sequence of N reversals has a higher probability of being drawn. Getting one more than 2 sigma from that should at least cause raised eyebrows. Sure, it COULD happen but how improbable is it?

Sequence of events: On top of that, the lottery drawing result wasn’t just pulled whole from the jar but was instead the confluence of 365 independent events.(1)(2) Independent, that is, for the purposes of considering reversals. Almost every sequence of events is unique in that its likelihood of repetition decreases as its length increases. I asked you before: how many coin flips would it take to convince you — or at least make you suspicious — the coin is biased (or not)? If the answer is N, why? Isn’t that just a sequence of N flips? Why would that one sequence cause suspicion and another not? Can’t you see it’s no different than examining reversals in the first lottery result? A reversal is almost the equivalent of a coin flip, no?

Yes, it’s true, obtaining the sequence was a one-time event. Obtaining a sequence of coin flip result is also a one-time event. The specific order has nothing special about it but the number of heads vs. tails says a lot about the coin’s fairness and, yes indeed, some of those sequences are VERY far from expectation if the coin is fair.

A reversal is just one feature, other features can also be used.

Another example: You commission an automatic shuffler for a blackjack shoe. In Vegas, they used to use a 7 deck shoe (and many places still do) containing 364 cards — about the same level of complexity as the draft lottery. A shuffler result is also a confluence of events. If on its very first use, it delivers all seven decks in original order, you wouldn’t think maybe, just, maybe, it isn’t shuffling very well? And the fact that it did this even once wouldn’t cause you to forever hold it in suspicion?

(1) 366 really, the last is unnecessary but was performed anyway
(2) For the purpose of reversal consideration you have to use 1/2 of the jars and draw twice from each jar — a minor detail.

What I propose is that at the time of registering for the draft, each registrant is assigned a “draft pool number”. These numbers were randomly picked in another lottery, and as they were picked, each number was assigned to the next calendar date in order. so each person registered, is assigned to a pool nuber which has a one to one mapping to the calendar dates; which mapping is unknown to anyone; and only the master database computer knows the mapping and the draft pool number of any registrant; the registrants themselves do not know their pool number.

then when the actual draft lottery is held, the selection order determines the draft pool num,bers in order of choosing; and nobody knows what calendar date that refers to.

The draft pool numbers are presumably selected in what to an observer looks like a random sequence.; one out of factorial 366. There are at least two sequences that might be of special interest.; besides the actual sequence that would have occurred in the actual draweing held.

One of those apparently random sequence of numbers happens to select the calendfar dates in correct order; or reverse order, or other manipulation. No reason whay any of those sequences would be any different from the actual sequence. Also possible is that the darft pool numbers themselves came up in numerical order.

I say all of those possibilities are equally likely. You evidently believe they are not, presumably even one that doesn’t list anyone’s birthday in calendar order; but lists an unknown sequence of draft pool numbers that came up in order, but which remain unknown to anybody. in the end, some persons were told that their draft number came up, but not the reason why.

If you push the red button, that sets off a nuclear bomb; one would certainly regard that as an event of some significance; but if you only push it once; it has no “Statistical Significance”; however destructive the event might be.

You keep talking about sequences; as in plotting them. The event I described whatever the details of how it was carried out; was ONE SINGLE SEQUENCE, not any kind of distribution of sequences. The calendar order draw differs from any other draw only in being a recognizable sequence.

You suggested tossing 366 pieces of paper with the dates, up in the air, and recording the order they landed. Unless you can prove such a process is somehow biassed in an unfair fashion, the result is exactly the same as pulling the papers one at a time. The complete set of 366 numbers is recorded one at a time, and the number remaining to be pulled or land diminishes as each is recorded, and the only difference in the result, is that some results may be recognizable as a known sequence.

So if the drawing is done by an unbiassed machine in a secret ballot, so nobody knows the result; any of factorial 366 possible different draws would be equally likely to occur in one single such experiment.

But by your logic; the moment the result is made public, the result becomes unfair to some observers; depending on the extent to which the observer recognizes the result as a sequence known to him/er, since its probabliity has mysteriously increased or decresed as the case may be.

There isn’t any distribution to talk about; we have a single point on a graph; and we have no information about any other point on that graph; let alone any distribution.

So how does the probability of occurrence relate to the degree of recognition by an observer, and does the degree of unfairness change with the level of recognition by an observer in the case of multiple observers..

Since you evidently have the exact result of that lottery drawing; perhaps you could rank the calendar dates in order of the degree of unfairness pertaining to any person born on that date. Which calendar date was the most fairly chosen, and which date was the most unfair?

Not that it matters now of course.

Is the distribution of the digits of Pi in base 10 numbers more fair or less fair, than the distribution of the digits in (e); how about Euler’s constant; is it fairly distributed?

Well you totally bamboozled me with that reference to logarithms. I can see why that is an important consideration; just about as easily as I can see your claim, that a calendar sequence result has a different probability from any other result.

Heisenberg told us that trying to observe the result of an experiment; changes the result in a comletely unknown manner (but maybe with a boounded error); so now we can add to that the amazing fact that the likelihood of getting a result depends on the extent to which it is recognizable..

But far be it it from me, to try and stop anyone from believing that.

I actually linked it. Found it within about 10 seconds of googling. Guess you missed that part. Unless I’m mistaken the photo was of the actual drawing. The drawing was public; even televised. Here is again:

http://www.sss.gov/lotter1.htm

I’ll stick with my original premise; that any possible result was equally unlikely whether a recognizable sequence or not, and being a single selection from a very large set; statistics is quite inappropriate to apply to such an event.

Very well but I had hoped to show you otherwise.

BUT WAIT! One last ditch effort :-) Remember the comment I made about the probability of any real variable equalling any specific value you choose being EXACTLY zero yet some values are more probable than others? (It’s really true, you know! Why? Hint: think confidence intervals). Well, I also mentioned that similarly, while any given sequence of dates has an extremely low probability, some sequences are more probable than others. I had hoped the hints would have led you to see why.

Perhaps you are being hung up by the fact that each ball has its own name (a date). If the lottery was conducted by using something similar to a blackjack dealer’s shoe with the order set by shuffling and subsequently revealed then I would agree with you. However, a drawing like the draft lottery is really a sequence of drawings from jars of diminishing content. True, the specific content of each jar depends upon the last drawing, but many distribution properties depend only upon the ranking of the values and not the specific values themselves. All of the hints in my last post were based upon ranking. (BTW: hints #3 and $4 were related as #3 is the equivalent of a coin flip).

If you treat the hints as exercises and actually perform the calculations you will discover that some sequences (such as JAN1-DEC31 in order, its reverse, and other sequences of similar ordered content) are the least probable while others are many orders of magnitude more probable. If you label the sequences by the features they exhibit then plot them they will form a normal distribution centered around what most people would call “random.” Not an accident, either — think confidence intervals. It’s essentially a demonstration of why mathematically chaos is more probable than order. Warning: the calculations are tedious and may require resorting to logarithms.

Ain’t stats fun?

So was there anything statistically suspicious or unfair about the big bang; another event that so far as we know has only happened once ?

Never really thought about it. Guess it really impacted the neighborhood way-back-when. Haven’t heard any complaints though.

——

All,
I neglected to include this in my previous post:

Merry Christmas!

Well DAV, ther aim of the first draft lottery, was to aasign numbers to groups of persons that would be used to psecify in numerical order how they would be selected to go to Viet Nm.

I described a process where the calendar days were numbered in order, and the numbers were drawn presumably in a fair and unbiassed fahion. Nobody has offered a theory of how that process might have been biassed and if so in what manner it could be biassed.

You described a different process, where it was the dates that were (presumably) fairly and unbiassedly drawn, and then numbers were assigned in sequence to the dates drawn. OK I wrote from memory; you evidently have access to the specific data; I will defer to your description of the methodology.

I matters not; the upshot was that the calendar dates were rearranged in a different order; one of factorial 366 such orders that they can be placed in. The method of drawing removed each date from the pool as it was darwn, so that no date could come up twice.

That was the original aim of the exercise, and whether or not they pulled all 366 numbers to complete the sequence I don’t know; but it would not be relevent unless they pulled so few dates, that they ran out of people to send.

In any case, the date sequence numbered as drawn; was if rearranged in calendar order, a quite unrecognizable number. I named an equally likely result that would have been a recognizable number if the calendar dates had come up in order. another equally unlikely but recognizable result, would be if the number was the first 366 digits of Pi in correct order, or the digits of (e) in correct order.

The only distinction between any of those results is that there are many possibilities giving numbers that are recognizable, but other wise no more likely or unlikely, than the unrecognizable result that actually happened.

And as I pointed out, any recognizable number result can be removed by simply recoding all the numbers with a very simple substitution code, itself randomly chosen. No such substitution encoding would alter the sequence in which persons were sent to Viet Nam..

The selection of the full set of 366 dates (or a shorter subset of them) was a single experiment yielding one result out of a maximum of factorial 366 equally likey different results.

If you want to characterize it as 366 different experiments (maximum), then of course the probability of selecting calendar dates is different for each experiment, since the remaining choices constantly diminish. The result is the same; the calendar dates are assigned a unique order of selection out of an extremely large number of such orders, and only one such order was chosen; having no statistical significance at all.

The SSS could have have grouped all of the registered draftees into 366 groups each identified by a unique word selected randomly from a dictionary, or encyclopedia; or even from the bible, and used those groupings rather than calendar date groupings. Such a random name grouping, would not be recognizable; and would have had no influence on the risk that any draftee faced of being drafted.

I’ll stick with my original premise; that any possible result was equally unlikely whether a rcognizable sequence or not, and being a single selection from a very large set; statistics is quite inappopriate to apply to such an event.

So was there anything statistically suspicious or unfair about the big bang; another event that so far as we know has only happened once ?