Talk:Cross-entropy

Statistics Low‑importance

	This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics articles
Low	This article has been rated as Low-importance on the importance scale.

Mathematics Low‑priority

	Mathematics portal This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.MathematicsWikipedia:WikiProject MathematicsTemplate:WikiProject Mathematicsmathematics articles
Low	This article has been rated as Low-priority on the project's priority scale.

Physics Low‑importance

	Physics portal This article is within the scope of WikiProject Physics, a collaborative effort to improve the coverage of Physics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.PhysicsWikipedia:WikiProject PhysicsTemplate:WikiProject Physicsphysics articles
Low	This article has been rated as Low-importance on the project's importance scale.

Wiki Education Foundation-supported course assignment[edit]

This article is or was the subject of a Wiki Education Foundation-supported course assignment. Further details are available on the course page. Student editor(s): Chrishaack.

Above undated message substituted from Template:Dashboard.wikiedu.org assignment by PrimeBOT (talk) 18:42, 16 January 2022 (UTC)[reply]

Untitled[edit]

This article uses the notation KL(p, q) and also DKL(p || m) when talking about Kullback-Leibler divergence. Are these notations two ways of expressing the same idea? If so, the article may want to indicate this equivalence.

The log-likelihood of the training data for a multinomial model is the same as the cross-entropy of the data. (Elements of Statistical Learning, page 32)

L(theta) = sum (all classes k) I(G=k) log Pr(G=k | X = x)

I guess "I(G=k)" is p and Pr(G=k | X=x) is q here.

Could somebody in the know please add this? Thanks!

WikiProject class rating[edit]

This article was automatically assessed because at least one WikiProject had rated the article as stub, and the rating on other projects was brought up to Stub class. BetacommandBot 09:46, 10 November 2007 (UTC)[reply]

To improve clarity, this stub should be merged into the "Kullback–Leibler divergence" article. "Cross entropy" and "relative entropy" refer to the same quantity in the literature, at least up to a sign convention. DRB (talk) 00:39, 15 January 2010 (UTC)[reply]

Hmmm... fair idea but not a cut-and-dry case. maybe put a merge tag on it, at least. Kevin Baas^talk 14:07, 15 January 2010 (UTC)[reply]

What's missing[edit]

At least one analogy or example would be most helpful. I get no sense at all as to what this article is attempting to describe. Marc W. Abel (talk) 02:24, 7 May 2012 (UTC)[reply]

Why is "cross-entropy" not hyphenated in the title of the page?[edit]

In the text, "cross-entropy" is written with a hyphen every time (as I believe to be correct). In the title of the page, however, it is not hyphenated. Why is this? — Preceding unsigned comment added by Thomas Tvileren (talk • contribs) 12:48, 27 May 2022 (UTC)[reply]

I thought that was odd too. I've moved the page to fix it. — The Anome (talk) 12:23, 7 August 2023 (UTC)[reply]