Talk:Analytic hierarchy process/Archive 5

Support for Theory

An anonymous IP editor has split this off from the Criticisms section. There were no edit summaries. The user has a slightly questionable history. The material is about criticisms of AHP, and there doesn't seem to be any reason to split it off. I propose to put it back in the criticisms section. MathDame (talk) 17:05, 4 January 2009 (UTC)

I agree with MathDame. There's no sense in breaking off the new section. A week has passed with no comment. Put it as it was. Lou Sander (talk) 17:17, 11 January 2009 (UTC)

Support. Combine the two. 74.1.175.146 (talk) 01:29, 15 January 2009 (UTC)

Done.

MathDame (talk) 15:22, 15 January 2009 (UTC)

I agree that this was a good move. Barbfletch (talk) 13:26, 21 December 2009 (UTC)

Summary section

I've redone the first part of the summary section without (I hope) changing its essence. The steps now correspond to those in the cited reference. I removed two claims that were challenged over ten months ago and well over a year ago and never responded to. Lou Sander (talk) 18:24, 11 January 2009 (UTC)

I renamed "Summary" as "Steps in using the process." The section has expanded greatly since the original heading was put in, and I think the new name is a more accurate description of what is actually there. I hope that others agree. Lou Sander (talk) 15:05, 22 January 2009 (UTC)

Decision situations

I added some material about the decision situations in which AHP is often used. It's more than just picking the best alternative. Lou Sander (talk) 17:42, 16 January 2009 (UTC)

Analytic Hierarchy Process/More Hierarchies

I have moved a page that was formerly at Analytic Hierarchy Process/More Hierarchies to Talk:Analytic Hierarchy Process/More Hierarchies. That page was apparently a draft page for proposed addditions to this page, and as such it was a subpage that should have been added to talk rather than article space. This note is jurt to help you find it. - Smerdis of Tlön (talk) 15:51, 26 January 2009 (UTC)

Thanks for handling it in an effective way. Lou Sander (talk) 17:54, 26 January 2009 (UTC)

Criticisms section

I removed a bunch of original research from this section. It was posted by an anonymous editor who seems to have made other inappropriate edits to the article over time. IMHO it doesn't seem to be vandalism, but it looks like somebody is trying to make a point but doesn't understand some basic things about how Wikipedia works. Lou Sander (talk) 00:16, 6 February 2009 (UTC)

I'm new to this but I had to respond to one of the claims in this article. At one point the article claims that the criticisms of AHP have been "settled in favor" of AHP and sites the INFORMS Impact Prize to Saaty as evidence of this. The Impact Prize is certainly no testament to whether a method works. The prize description clearly states "This is not a research award. The awards committee is not judging the quality of a body of work. Instead, emphasis will be placed on evaluating the breadth of the impact of an idea or body of research." See the INFORMS Impact Prize description here http://www.informs.org/index.php?c=116&kat=INFORMS+Impact+Prize

I was surprised to read that anyone would claim this is a "settled" issue. From what I see, INFORMS meetings are still heated arguments about this controversial method of reducing important decisions to a series of subjective pairwise comparisons. The reason why the prize is worded so carefully is because, I think, a prize that would effectively endorse the method as valid would never have gotten a majority vote in INFORMS. Of the 18 different prizes awarded by INFORMS, I would say the Impact prize is among the least like any validation of the theory. The John Von Neumann Theory Prize, on the other, might be used as evidence of general agreement and validation by the members of the validity of a method. If that prize were awarded, I would think that would suffice as evidence that the issue has been settled.

It seems clear that the awarding of this prize isn't any kind of evidence for the claim being made. The more recent citations listed, on the other hand, seem to make the point that the method still has serious issues and more continue to be found. Also, it appears to overempahsize rank reversal, which doesn't strike me as a key issue anymore now that there as at least some mathematical guard against it.

It seems that, at a minimum, any claim that the issue is "settled" should be removed since the citation provided clearly does not support that claim. Just my thoughts. DFLovett (talk) 22:10, 2 March 2009 (UTC)

We keep hearing from people who assert that AHP is "controversial," or "doesn't work," or is "dangerous," or "clearly" has some other problems, or isn't the widely-accepted tool that huge amounts of evidence show that it is. So far, these folks have never offered much to back up their assertions. On the other hand, the AHP tracks at important OR meetings keep growing and growing, the use of AHP is booming, the prizes keep coming, etc. It would be interesting to see a few coherent paragraphs on this page, backed by citations from the 21st century, about AHP's perceived drawbacks. What, specifically, are the problems? Why don't the naysayers seem to be gaining any traction? Or ARE they gaining traction and nobody has pointed it out here? Lou Sander (talk) 22:49, 2 March 2009 (UTC)

You mean evidence other than the published articles this very article cites? You asked for specific problems but the published articles already cited seem to do an adequate job in that regard, especially once the only "evidence" that they were settled (the Impact Award) is refuted. You asked for 21st century (as if the age of the flaw somehow addresses the flaw), but it looks like the Perez citation alone meets that requirement.

This wasn't the point of my comment but it would help if you showed the evidence that shows AHP measurably improves decisions in controlled experiments. There are many articles that are case studies about AHP beling used, but I find none that show that users of AHP beat some other method (or even intuition) in a repeated, well-defined decision making task. If you could find such an article - not just another case study where the users say they were satisfied - then it should certainly be listed and discussed here in at least as much detail as rank reversal. And I think it would satisfy many skeptics, myself included.

Regarding your point about the traction of skeptics, I would be just as curious to see a survey of decision analysis experts or INFORMS members as you. Otherwise, it would seem we are just winging it with personal opinions and anecdotes. All I can say is that in my circles AHP doesn't seem to be taken as a serious alternative to, say, probabilistic models of systems that use empirical measurements and are tested against repeated observation. Perhaps your circles are different. I might be more open to AHP than some of my peers because I think AHP might be relevant for perhaps a narrow set of problems that can only be satisfactorally described as a system of subjective pairwise comparisons. Again, that's why a survey would be great if anyone could find a citation. It would also help settle what appears to be the entirely subjective opinions about what is "undue weight" of various issues on this topic.

But this is not the main point I was making, either. The main point I was making was simply in regards to the use of the Impact Award as evidence that the controversy is settled. The language of the award would justify no such claim that this is some kind of endorsement of the method's validity. Regardless of anything else you agree to about the article or the merits of the method, it seems clear that the award does not "settle" the issue at all and should not be represented as such. Can we at least agree that the Impact Award is not a research award and explicitly is NOT a recognition of the quality of the work (as the award description states)?DFLovett (talk) 00:12, 3 March 2009 (UTC)

From the citation of the INFORMS Impact Award:

"The AHP has revolutionized how we resolve complex decision problems.... The AHP has been applied worldwide to help decision makers in every conceivable decision context across both the public and private sectors, with literally thousands of reported applications.

"The AHP is now an established methodology that is part of the OR curriculum for multi-criteria decision-making in business and engineering schools. Software for its implementation is readily available. The International Society on Multiple Criteria Decision Making awarded the MDCM Gold Medal to Professor Saaty in 2000 "...for the development of the Analytic Hierarchy Process (AHP) and the impact of his lifelong research contributions on several disciplines." Professor Saaty received the 2007 Akao Prize for excellence in Quality Function Deployment (QFD) from the QFD Institute.

"For his seminal work on the Analytic Hierarchy Process, and for its deployment and extraordinary impact, INFORMS is delighted to award the 2008 Impact Prize to Professor Thomas Saaty.

Your personal opinions are not germane. Don't be surprised if people don't respond to them. It doesn't do to merely besmirch the AHP or to present your own criticism of sources or arguments. It doesn't do to cherry-pick quotes from isolated papers (see WP:UNDUE). Try presenting a coherent description of published material showing that there are important criticisms and describing what they are. Good Cop (talk) 02:16, 3 March 2009 (UTC)

I didn't claim personal opinions were germane. If I thought that, I would have just made the changes in the article instead of discussing it. Nor did I cherry pick any quotes from any paper - isolated or otherwise. The discussion Lou Sanders and I were having was simply that, a discussion. In that context opinions are entirely appropriate. In other words, it won't do to simply say "it won't do". I didn't add any NEW published material showing important criticisms because the existing citations seem adequate. All I said was that the Impact Prize is not evidence that those criticisms were addressed and the only source I quoted was the definition of the prize as described on the INFORMS site. Here it is once again (This is verifiable, not an opinion):

"This is not a research award. The awards committee is not judging the quality of a body of work. Instead, emphasis will be placed on evaluating the breadth of the impact of an idea or body of research." See the INFORMS Impact Prize description here http://www.informs.org/index.php?c=116&kat=INFORMS+Impact+Prize

At first, it does seem inconsistent with the language you just cited. But on closer review the award language you cited only shows that it is widely used and on that basis Saaty certainly deserves the reward. Nothing in the language you provided serves to "settle" the specific theoretical issues the cited articles describe, which was my point. The language of the award would seem to specifically exclude using the award as evidence that specific theoretical issues have been addressed.

In other words, my point has nothing to do with whether the method or its critics are right. I was only talking about whether the Impact Award is evidence that those specific issues mentioned in those cited papers were settled - nothing else. But if you want to argue the pros and cons of this method, keep in mind that I'm a skeptic, not a critic and I'm easily swayed by evidence of measurable improvements of decisions in controlled tests. I, for one, am not entirely swayed by the existance of theoretical flaws since its not clear to me how frequently those situations arise in real-world problems.

Of course, Undue Weight rules just don't apply as long as we are in discussions like this, so I don't see how that applies, either. But it is a good idea to at least carefully read the points others make before you respond with something that appears to not address anything the other person actually said. I understand that this is a very heated topic for critics and proponents alike but, as I said, I don't side with either camp. Just read my comments a little more objectively and don't take it as a personal attack. The WP rule that DOES apply to the discussions is the assumption of good intent. Cheers. DFLovett (talk) 03:05, 3 March 2009 (UTC)

I added a reference and did some copyediting to (hopefully) address DFLovett's legitimate criticism that the INFORMS prize alone isn't a strong enough justification that "the debates have been settled in favor of AHP." Lou Sander (talk) 15:29, 13 March 2009 (UTC)

New example

I'm working on replacing the Jones car buying example with something a lot less complex. The Jones stuff will be retained, but moved out of the main article. Please be patient with the changes I'm making to set this up. Lou Sander (talk) 21:28, 24 February 2009 (UTC)

As part of this, I've made changes to the Priorities subsection. They are intended to make it clearer and more precise. At least one reviewer has said they meet this intention. Lou Sander (talk) 09:58, 11 March 2009 (UTC)

Is the math correct on the "Choose the best car" graphic in the "Make the decision" section? The totals column appears to simply be the sum of the numbers in each row. Mollynet (talk) 17:39, 22 April 2009 (UTC)

I think it's fine. The totals ARE the sum of the numbers in each row. Each of those numbers represents that factor's contribution to the overall priority of the alternative. Lou Sander (talk) 21:58, 22 April 2009 (UTC)

But if it's simply the sum of the members in each row, then the weights of the criteria are not included in the calculation, unless I am mistaken. See page 5 of www.booksites.net/download/coyle/student_files/AHP_Technique.pdf where it appears that each element of the row is multiplied by the corresponding Eigenvector element for that criterion, then the products are summed. To me, it looks like the two tables, "Choose the best car" and the so-called VMF vector in the provided link, are in conflict. Mollynet (talk) 11:39, 23 April 2009 (UTC)

Ah, I see the error of my ways. The global priorities are in the table, not local. Maybe this multiplication should be saved until the end to present the process in a more concise way? It is not stated in the article that the global priority is the local priority multiplied by the criterion's priority- it just says something like "the AHP software returns the following global priorities." Mollynet (talk) 20:09, 24 April 2009 (UTC)

Got it! I'm trying REAL hard to come up with a much simpler example, and I think I'm getting close. Your comments will help make it easier for the reader. Thanks. It would be useful to talk with you on the phone. If you're interested, send me email through my user page. Lou Sander (talk) 00:09, 25 April 2009 (UTC)

Rank Reversal, old discussion continued

Some current discussion of rank reversal was included in the old material just moved to Archive 4-07/08 through 12/08. The current material is repeated below so the discussion can continue. Lou Sander (talk) 02:00, 9 November 2009 (UTC)

---

The following material was inappropriately put in the article itself. I moved it here. Lou Sander (talk) 01:08, 1 December 2008 (UTC)

Comment: The above paragraph does not make sense to me. Your original table simply did not represent the number of available dresses of each type. As your example continues, the customer now takes into account the number of available dresses of each type (e.g. relative scarcity of each alternative), and so clearly the decision table should be modified to include this new criterion. With this new criterion in place, priorities can be re-assigned and alternatives (there will be no additional ones) re-evaluated. If now the ranking of alternatives is different, e.g. reverses, this is due to the additional criterion (relative scarcity). In fact, referring to the discovery of new dresses (of existing types) as "additional alternatives" or "copies" makes very little sense to me: a criterion was added, not an alternative. With respect to the subsequent example (the 2004 elections), there are no "copies" involved, rather you discuss the effect of a dominated alternative on the relative rankings of the dominating alternatives. 69.217.120.82 (talk)

• If the material was deemed inappropriate for the article itself, and hence moved here, it should have been addressed here (it has not been), or it should have been formatted/reworded to be suitable for the main article (it has not until now). I am adding this point of contention to the main article, in a more appropriate format. Please don't remove it again, without addressing the main point of contention / confusion. —Preceding unsigned comment added by 69.217.124.169 (talk) 03:07, 4 February 2009 (UTC)

• Agreed. Not only do I agree with the concern of User:69.217.120.82 regarding the error of the statement "there is no doubt", his alternative is perfectly justifiable. He may have presented his alternative in a biased/unprofessional/untraditional way, but his point should be taken and included. -dustingooding —Preceding unsigned comment added by 128.157.160.13 (talk) 20:40, 3 September 2009 (UTC)

I've removed the "dress" example, partly because it's nonsense and partly because it's nonNPOV. It's nonsense because introducing copies changes the value of the things already in the decision. It's nonNPOV because it's promoting a particular perspective. --217.169.11.173 (talk) 11:44, 7 November 2009 (UTC)

Here's the removed text

A simple example will demonstrate the phenomenon of rank reversal:

Consider a pretty girl in a small town. She's having a party next week, and she wants to buy a dress that will impress her guests. She visits the town's only dress store and goes to the rack of party dresses. There are five such dresses, and after long consideration she ranks them by desirability as follows:

Rank	Style	Color	Price
1	Style A	Blue	$109
2	Style A	Green	$109
3	Style B	Red	$119
4	Style C	Yellow	$99
5	Style D	Off-White	$149

Now imagine that she enters the back room and sees the store's entire inventory of dresses. The dresses she has looked at in Styles B, C, and D are the only ones of their kind, but there are four more Style A dresses in green and eight more Style A dresses in blue. In the language of decision science, these dresses are copies of the existing alternatives. In our one-store small town scenario, there's a reasonable chance that one or more party guests would buy and wear one of the copies.

When made aware of these new alternatives, our fashion-conscious girl might rank her choices in a different order. Considering her great embarrassment if a guest were to wear the same dress that she did, she might rank her choices like this:

Rank	Old Rank	Style	Color	Price
1	3	Style B	Red	$119
2	4	Style C	Yellow	$99
3	5	Style D	Off-White	$149
4	2	Style A	Green	$109
5	1	Style A	Blue	$109

Notice that the rankings of the two Style A dresses have reversed (since there are more copies of the blue dress than of the green one). Not only that, but Style A has gone from the most preferred style to the least preferred. Rank reversal has occurred. Axioms of decision theories have been violated. Scholars and researchers can cry "foul," or impugn the method by which the girl has made her choice, but there is no denying that in the world of our example, ranks have been reversed. There is no doubt that the reversal is due to the introduction of additional alternatives that are no different from the existing ones. —Preceding unsigned comment added by 217.169.11.173 (talk) 11:45, 7 November 2009 (UTC)

I've reverted a recent change to the Criticisms section. The reasons are below. A good faith effort to communicate with the reverted editor(s) is further below.

The edit was reverted because:

• It deleted properly cited material. Please do not do that without a strong consensus.
• It replaced the deleted material with original research -- no reference to any source.
• It used weasel words: "some decision theories", "disputed by some".
• The editor justified deleting the dress example with the somewhat uncivil "nonsense".
• The editor alleged POV without being specific about the "perspective" that was objected to.

To the anonymous editors who have made the comments above this one:

• Some of you seem to have good things to contribute to this article. Other editors who watch this article are strongly in favor of that.
• Possibly you aren't familiar with Wikipedia protocols. Feel free to ask for help.
• Please don't think that just because you suggest or demand a change, other people will make it. (Though many editors will make simple changes if you ask them to.)
• It's best for all concerned if you discuss non-minor changes here before you make them. If you are rational and fair, you will find support here, even from those who disagree with you. If you just want to "bash" AHP, you might not find much sympathy.
• If you are an advocate of other decision theories or methods, please do what you can to generate Wikipedia articles on them, since they are pretty poorly represented right now (see MCDA).
• It would REALLY be helpful if you'd get a username and a talk page. Those things facilitate discussion between editors, and without them, we all operate at a disadvantage.
• Please sign your posts. Unsigned posts from anonymous editors somehow seem to have a little less weight than they otherwise might. Good Cop (talk) 05:33, 10 November 2009 (UTC)

You're right, I was lazy not to log in and I apologise (I am the anonymous editor above). However, perhaps you're the one who should ask for help if you don't understand wikipedia - whatever happened to "be bold"? My response to your individual points follow.

• It deleted properly cited material. Please do not do that without a strong consensus.

I did accidentally delete a reference yes, apologies.

• It replaced the deleted material with original research -- no reference to any source.

I didn't add any significant content, so it's hard to see how it could be original research. In fact, the example I deleted was original research - it was created to "demonstrate" a particular argument about rank reversal.

• It used weasel words: "some decision theories", "disputed by some".

No, they were in the original text.

• The editor justified deleting the dress example with the somewhat uncivil "nonsense".

I apologise if I was uncivil, but that's not a reason to revert an edit.

• The editor alleged POV without being specific about the "perspective" that was objected to.

The perspective is on whether that example correctly demonstrates rank reversal. --Khendon (talk) 08:56, 11 November 2009 (UTC)

Khendon: Thanks for identifying yourself. You seem to be 217.169.11.173. Are you also the OTHER editors referred to above: 69.217.120.82 and 128.157.160.13? Both are anons who didn't sign posts, and both seem to have rational material to contribute about rank reversal. Good Cop (talk) 14:42, 12 November 2009 (UTC)

I am 217.169.11.* and not the others. When you say they have rational material to contribute, is that your subtle way of saying I don't? :-) --Khendon (talk) 17:13, 12 November 2009 (UTC)

Not at all. Good Cop (talk) 01:21, 13 November 2009 (UTC)

Moving the example

This article has been tagged as having too much "how-to" content. That's a very good point, in spite of the well-known difficulty of understanding the AHP without working through a detailed example.

I propose to move the not-so-simple "simple example" to a place where it can be linked to, but outside the article itself. The advantage of this is that excessive "how-to" content will be removed from the article, but will still be available to interested readers. Moving the example will also significantly reduce the length of this probably-too-long article. Lou Sander (talk) 22:18, 15 November 2009 (UTC)

Support. Good idea. Go for it. It's essential to keep some sort of link to an example, though. DCLawyer (talk) 15:53, 16 November 2009 (UTC)

Support. I think moving it is a good idea. That example might be a little long though it does an excellent job of taking the reader through the judgment process. I think the Tom, Dick and Harry figure at the top of the article would make a good brief introductory example that illustrates the main points of AHP such as the pairwise comparison matrices and priority vectors.MathDame (talk) 13:38, 17 November 2009 (UTC)

Support. Move it. Keep a link. Article is way too long as it stands. A shorter example would be good. Good Cop (talk) 15:01, 17 November 2009 (UTC)

I've moved most of the example elsewhere, leaving enough to give the general flavor of what is going on. I'll try to rework what's left into something shorter or maybe simpler. The "Tom, Dick, and Harry" illustration at the beginning of the article might be something to use as a complete example in the article. I'll work on it. Lou Sander (talk) 14:06, 29 November 2009 (UTC)