Wikipedia:Reference desk/Archives/Mathematics/2010 August 6

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

August 6

Empirical maximum likelihood problem

Hello everybody

I am struggling with an optimization problem. I have a stochastic computer model with two parameters which I call (nu,M).

With a fixed value for (nu,M) the model produces two outputs which I call (theta,overl) which seem to have a bivariate Gaussian distribution. The distribution depends on the values of (nu,M). I know the "correct" value for (theta,overl).

I know very little about the model's properties, but I can sample from it; a single sample takes about an hour's computing time.

I want to calculate a maximum likelihood estimate for (nu,M).

I feel sure that this must be a standard problem, but googling "empirical maximum likelihood estimate" comes up unhelpful.

Is my problem a special case of a well-known technique? Robinh (talk) 10:19, 6 August 2010 (UTC)

Bessel's correction

So I'm confused about the proof behind Bessel's correction, and it's driving me insane!

The proof is as follows:

${\begin{aligned}\operatorname {E} (s^{2})&=\operatorname {E} \left(\sum _{i=1}^{n}{\frac {(x_{i}-{\overline {x}})^{2}}{n-1}}\right)\\&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu +\mu -{\overline {x}})^{2}\right)\\&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-2({\overline {x}}-\mu )\sum _{i=1}^{n}(x_{i}-\mu )+\sum _{i=1}^{n}({\overline {x}}-\mu )^{2}\right)\\&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-2({\overline {x}}-\mu )n({\frac {\sum _{i=1}^{n}x_{i}}{n}}-\mu )+n({\overline {x}}-\mu )^{2}\right)\\&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-2n({\overline {x}}-\mu )^{2}+n({\overline {x}}-\mu )^{2}\right)\\&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-n({\overline {x}}-\mu )^{2}\right)\\&={\frac {1}{n-1}}\left(\sum _{i=1}^{n}\operatorname {E} ((x_{i}-\mu )^{2})-n\operatorname {E} (({\overline {x}}-\mu )^{2})\right)\\\end{aligned}}$

So what I'm not following is how did they get from here:

${\begin{aligned}&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-2({\overline {x}}-\mu )\sum _{i=1}^{n}(x_{i}-\mu )+\sum _{i=1}^{n}({\overline {x}}-\mu )^{2}\right)\\\end{aligned}}$

to here:

${\begin{aligned}&={\frac {1}{n-1}}\operatorname {E} \left(\sum _{i=1}^{n}(x_{i}-\mu )^{2}-2({\overline {x}}-\mu )n({\frac {\sum _{i=1}^{n}x_{i}}{n}}-\mu )+n({\overline {x}}-\mu )^{2}\right)\\\end{aligned}}$

??? - 114.76.235.170 (talk) 15:22, 6 August 2010 (UTC)

${\overline {x}}$ and $\mu$ don't depend on i so $\sum _{i=1}^{n}({\overline {x}}-\mu )^{2}=n({\overline {x}}-\mu )^{2},$ and $\sum _{i=1}^{n}(x_{i}-\mu )=\sum _{i=1}^{n}x_{i}-n\mu =n({\frac {\sum _{i=1}^{n}x_{i}}{n}}-\mu )$. In other words, summing n copies of the same thing is the same as multiplication by n. Rckrone (talk) 16:01, 6 August 2010 (UTC)

Sir, you are an absolute legend!!! I can't thank you enough for this explanation :-) - 114.76.235.170 (talk) 22:59, 6 August 2010 (UTC)

Define a Pseudo-topology

Define a pseudo-topology as a pair (X,Σ) consisting of a set X and a collection Σ of subsets of X, called open sets, satisfying the following axioms:

1. The union of open sets is an open set.
2. The intersection of open sets is an open set.
3. X and the empty set ∅ are open sets.

Notice that I no longer require a finite intersection of sets for the result to an open set.

• What novel properties does a pseudo-topology have, as opposed to a standard topology?
• Why was the finiteness condition insisted upon for intersections?

— Fly by Night (talk) 19:18, 6 August 2010 (UTC)

I can't answer the first question, but the second is easy: The prototype for topological spaces is Euclidean space and an infinite intersection of open sets in Euclidean space is not necessarily open. A lot of types of mathematical objects (topological spaces, vector spaces, manifolds, differential manifolds, groups, rings, fields, etc.) are generalisations of very familiar objects (Euclidean space, integers, real numbers, etc.). We try and work out which properties of that familiar object are actually required to get the results we like and those properties become the definition of the generalisation. --Tango (talk) 19:40, 6 August 2010 (UTC)

Tango, the topology that Fly by Night constructed does have a name, and does have some useful properties (but it is certainly not as useful as the standard notion of a topology): It is called an Alexandrov topology. There are some pretty weird constructions in mathematics, I should add, so it should not be too surprising that people have attempted to study these sorts of topologies before. (One of the most "naturally occuring" though weird topologies is the Zariski topology in algebraic geometry, but even this topology is not in general Alexandrov.) Nonetheless, Alexandrov topologies do actually occur naturally sometimes in set-theoretic and point-set topology. PST 00:36, 7 August 2010 (UTC)

Just to exhaust other possibilities of "playing with the axioms", I shall note that the requirements that the entire set X and the empty set be open are partially there to ensure that constant functions are continuous. (But the theory would not change much if you removed this requirement and defined a function to be continuous if and only if the preimage of any open set under the function is either open, all of X, or the empty set.) PST 00:43, 7 August 2010 (UTC)

Thanks for your reply, Tango. It's clear that Euclidean space with a metric topology is an intuitive example of a topology. But I was hoping for something more substantial. For example, are the axioms inconsistent if we drop the finiteness of intersections condition? Etc. — Fly by Night (talk) 20:57, 6 August 2010 (UTC)

You're definitely not going to get any inconsistencies. Just look at the discrete topology on any set - it will satisfy the axioms you listed. It's more likely that these pseudo-topologies turn out to be less interesting/useful/familiar than the standard notion of a topology. You may also be interested by sigma algebra if you've never run into one of those before. J Elliot (talk) 21:01, 6 August 2010 (UTC)

I shall read that now. Thanks for the link. — Fly by Night (talk) 22:10, 6 August 2010 (UTC)

It is worthwhile to note two things about the sigma algebra example: Sigma algebras are closed under countable intersections (and hence also under finite intersections) and sigma algebras are closed under countable unions (and hence also under finite unions). (These can be proven by manipulating the axioms and can be a good exercise if you are not familiar with sigma algebras.)

If you have done measure theory or probability theory, it is easy to appreciate why these axioms are the way they are. The most obvious reason is the same as that given by Tango above: If we allowed uncountable unions of measurable sets to be measurable, accepting that singleton sets in Euclidean spaces are measurable would yield that every subset of Euclidean space is measurable. This would, however, not be very useful since the primary purpose of measurable spaces is to equip them with measures (though there are other purposes, as well) and it is impossible to define a countably additive, translation invariant measure on the set of all subsets of Euclidean space such that the measure of any open cell is its volume in the ordinary sense.

You can define a subadditive measure on all subsets of Euclidean space that has the other properties; the proof of the Riesz representation theorem for positive linear functionals on a space of continuous complex-valued functions with compact support gives such a construction. Also note that if X is a locally compact Hausdorff space in which every open set is sigma compact, then any positive Borel measure on X that is finite on compact sets must necessarily be regular. Hence the ordinary Lebesgue measure on Euclidean space is a Borel measure. (Again, this can be shown directly using the Riesz representation theorem; in fact, the proof of this general statement (at least the one I know; there could be a simpler proof) relies on the Riesz representation theorem.) Up to multiplication by scalars, in fact, this is the only positive, translation invariant, Borel measure on all Borel subsets of Euclidean space that is finite on compact sets. (The proof of this is not too hard.)

However, in ZFC you would have to remove at least one requirement to define a measure on all subsets of Euclidean space as the Vitali set shows. It is a remarkable result due to Robert M. Solovay, that in fact, roughly speaking, it is impossible to construct a nonmeasurable subset of Euclidean space without using the axiom of choice. (The construction of a (nonmeasurable) Vitali set relies on the axiom of choice.) PST 01:35, 7 August 2010 (UTC)

(ec) No, there is nothing inconsistent about them. The trivial and discrete topologies both fit your axioms, just to name two easy examples. Your axioms are more restrictive than the standard ones, so anything satisfying your axioms will satisfy the standard ones. It turns out there are lots of interesting results you can get with just the standard ones, so the standard ones are useful and therefore get used. It is possible there are some results that are only true for spaces satisfying your axioms, in which case yours would be useful too as would also get used. --Tango (talk) 21:03, 6 August 2010 (UTC)

A pseudotopology is essentially the same thing as a complete lattice. Algebraist 00:43, 7 August 2010 (UTC)

See Alexandrov topology. (I mentioned this above, but I think it is more visible when not indented.) PST 07:13, 7 August 2010 (UTC)