User talk:Dispenser/Dabfix

dabfix to specify if dab term is mentioned?

http://toolserver.org/~dispenser/cgi-bin/dabfix.py?page=ARK lists a lot of people as having the initials ARK, but presumably none of these pages mention ARK.

I'd like to see your lists of suggestions broken into two: those that mention the disambiguation page (Root name), or any redirects to it; and those that don't. There's a much higher chance that a page is a viable disambiguation term if any of the ambiguous terms are mentioned on it. I don't know if this will have a terrible effect on performance. Josh Parris 11:39, 1 February 2012 (UTC)

Actually, it would be helpful if there was also some note as to the existence of the term on articles already listed on a dab page, for clean-up purposes. Josh Parris 22:31, 1 February 2012 (UTC)

dabfix may/can -> refer

dabfix didn't offer {{subst:refer|type=stand}} for

EGM can stand for:

Just thought it ought to, is all. Josh Parris 01:46, 2 February 2012 (UTC)

Fixed It was a regression caused by trying to improve the link matching code. I've come up with a new method and tested against the 35 test cases. — Dispenser 03:22, 9 February 2012 (UTC)

Thanks! Josh Parris 03:30, 9 February 2012 (UTC)

dabfix.py reports Missing hatnote

For Beinn Ghlas (disambiguation) I get:

WARNING: Missing hatnote on [[Beinn Ghlas]]

but Beinn Ghlas has {{otheruses}} on it, so what hatnote is missing? Josh Parris 05:48, 8 February 2012 (UTC)

tools:~dispenser/cgi-bin/dabfix.py/Aubonne (disambiguation) doesn't report this error, yet the hatnote on Aubonne doesn't mention Aubonne (disambiguation); could your logic be back-to-front?Josh Parris 06:26, 8 February 2012 (UTC)

Seems to be, now that {{otheruses}} is on Aubonne, I'm getting the warning. Josh Parris 10:01, 8 February 2012 (UTC)

Fixed Prior to October 2011, {{dablink}} referred to hatnotes that linked to a disambiguation page. Now {{hatnote}} is this and {{Japanese name}}, {{Correct title}}, and others now use different templates. {{Distinguish}} seems to a bit of an odd ball, but I'll let time sort it out. I'll inform User:JaGa so he can fix Dab Challenge. — Dispenser 16:37, 4 April 2012 (UTC)

dabfix.py BLUE LINK search should only be by article-space

https://toolserver.org/~dispenser/cgi-bin/dabfix.py?page=Aubonne_%28disambiguation%29 lists in its BLUE LINK search a number of redlinked articles that have no article-space links - Aubonne (horse) is only linked from userspace, and Aubonne (VD) from project space. They're not candidates for listing on the dab page, because they can't get the corresponding bluelink. Josh Parris 06:20, 8 February 2012 (UTC)

dabfix.py duplicates templates

The Copy and Paste stuff, {{intitle}} and {{Lookfrom}} are offered even if they already appear on the page. Josh Parris 08:41, 8 February 2012 (UTC)

The section will be automatically removed when you leave the page without changing the section name. It's a static addition as I can't remember their names, nothing fancy. I had to add a warning due to inappropriate usage. You can submit a patch to add them dynamically. — Dispenser 06:38, 2 April 2012 (UTC)

Whitehouse and dabfix.py

I was working on Whitehouse and got

* [[Mary Whitehouse]], (1910 – 2001), British christian morality campaigner

which was refactored to

* [[Mary Whitehouse]] (1910–2001), British christian morality campaigner

which is fine. But only Mary got this offer:

== 1 recovered names (All redirects) ==

* [[Mary Whitehouse]] (1910–2001), CBE (born <b>Constance Mary Hutcheson</b>, 13 June 1910 – 23 November 2001) was an English social activist known for her prominent opposition to social liberalism and the mainstream British media, both of which she accused of encouraging a more permissive society

which is odd, given:

1. it's not a redirect
2. we've already got it
3. the other names that we've already got and have been refactored in the same way didn't get this offer.

Josh Parris 11:12, 11 February 2012 (UTC)

disambig and disambiguation

In other news: sometimes I've seen {{disambiguation}} added, but {{disambig}} not removed. This was one such occasion. Josh Parris 11:12, 11 February 2012 (UTC)

I've just see {{disambiguation}} left on the page and {{disambig}} transformed into {{disambiguation}} for https://toolserver.org/~dispenser/cgi-bin/dabfix.py?page=Flash - which leaves multiple dab notices on the page. Josh Parris 14:00, 17 February 2012 (UTC)

Rod has {{disambig|geo|surname}}, which is left on the page and {{disambiguation}} is added. Josh Parris 10:09, 23 February 2012 (UTC)

dabfix Bliss

Included in the suggestions was

== recovered names (2 non-redirects) ==
* [[Cosmic bliss]], a term that originated in Buddhism, but is commonly referred to in various human endeavors, with the meaning of 'bliss': ultimate contentment
* [[Ignorance is bliss (disambiguation)]]

The second suggestion already exists on the page.

In other thoughts: it would be helpful if redlinked terms linked to "What links here" rather than "edit me"... unless one was to create redirects... hmmmm... Josh Parris 04:15, 16 February 2012 (UTC)

dabfix doesn't undertand wiki suffixes

* [[Flash (Chuck)|Flash (''Chuck'']]), how Chuck retrieves data in the TV series ''Chuck''

was transformed into

* [[Flash (Chuck)|Flash (''Chuck'' (Chuck)]]), how Chuck retrieves data in the TV series ''Chuck''

Admittedly, a weird case. Josh Parris 13:50, 17 February 2012 (UTC)

Actually, that's not the problem. The closing ) in the original doesn't get hyperlinked. The lack of ) inside the square brackets must throw the parser out. Josh Parris 13:54, 17 February 2012 (UTC)

How the thing basically works, it copies the link's "(subject qualifier)" to the label side if roots (before the parenthesis) match on each side. It'll blindly copy subject qualifier (e.g. Buffy episode → Buffy the Vampire Slayer) and the human operator (I hope) will catch incorrect cases such as double-redirect bot blunders and syntax problems like your. — Dispenser 05:30, 2 April 2012 (UTC)

From MSM

Converting

* Miami Sound Machine, a musical group formed with [[Gloria_Estefan#1984.E2.80.931988:_The_Miami_Sound_Machine|Gloria Estefan]]

to

* Miami Sound Machine, a musical group formed with [[Miami Sound Machine|Gloria Estefan]]

is very clever. Now you just need

* [[Miami Sound Machine]], a musical group formed with Gloria Estefan

to make it amazing. Josh Parris 04:12, 3 March 2012 (UTC)

Interwiki confusion

I've been sanity checking adding the dispenser tools to simple: and immediately found that tools:~dispenser/cgi-bin/dabfix.py/simple:Mars_(disambiguation) lists, above the diff,

[[Mars (TV series)]] from [[Alan Kuo]] 

but [[Alan Kuo]] hyperlinks to en.wikipedia.org rather than simple.wikipedia.org

So, can't invite others to use it on simple: yet. I have created {{refer}}. What other interwiki information do you need? Dab templates? Preferred dab template? Josh Parris 06:44, 7 March 2012 (UTC)

Fixed I had hard coded the domain since I didn't want to deal with global variables. Anyway, lot of the code is wiki dependent if not language dependent. The obvious stuff is:

• simple:Template:Wiktionarypar, links to non-Simple Wiktionary. It should be renamed to something more obvious.
• Category regex for "No title formatting", "Quote title", and "Italics title". If {{Italic title}} is used in infoboxes then it's less of a worry.
• Date category regular expression, e.g. person lifespan or product introduction.
• Heading rename mapping. On enwiki "Real people"/"Historical persons"/"Persons" are renamed to "People". Additional maps from English to Simple would allow automated conversion.
• A much better system for selecting good Wiktionary links (either language). The tool doesn't have utilities for selecting relevant Wiktionary links. For example: it'll suggest wikt:Wikipedian and wikt:Wikipedians despite the latter as an effective redirect to the former. Keep the run time under a second (no HTTP requests).

— Dispenser 05:09, 8 March 2012 (UTC)

More interwiki problems

https://toolserver.org/~dispenser/cgi-bin/dabfix.py?page=simple:Jupiter_%28disambiguation%29 doesn't work well; the existing page is covered in orphaned redlinks, like to simple:Jupiter (tugboat), and they're not being detected as orphan redlinks (and commented out). Josh Parris 13:08, 7 March 2012 (UTC)

Problem with diacritics

Hi, dabfix is having a problem with pages containing certain diacritics, or pages with redirects from diacritics e.g. Hans Hess or František Donth. Thanks! Tassedethe (talk) 18:25, 10 March 2012 (UTC)

Fixed, actually < 2 days after your posting. — Dispenser 02:50, 2 April 2012 (UTC)

ndash, spaced ndash

tools:~dispenser/cgi-bin/dabfix.py?page=Jacob_Bailey is unhappy. Josh Parris 01:42, 2 April 2012 (UTC)

Added, {{spaced ndash}} isn't even linked from anything page in the Wikipedia namespace. — Dispenser 02:27, 2 April 2012 (UTC)

Yeah, it's new - a more explicit version of {{ndash}}, which is going to become an endash rather than the misleading beast it is today. Josh Parris 02:36, 2 April 2012 (UTC)

Possible failures

Hi Dispenser. These might be a consequence of the toolserver problems but I haven't been able to dabfix these two pages for several days: [1] and [2]. I get a <type 'exceptions.ValueError'>. Other pages fix fine, and even if I see errors they usually clear themselves up. Let me know if you need more informations. Thanks. Tassedethe (talk) 00:48, 24 May 2013 (UTC)

Broken?

Hi Dispenser. I've been unable to use Dabfix for a day or so. It always results in "A problem occurred in a Python script." errors e.g.[3]. Thanks. Tassedethe (talk) 01:44, 10 November 2013 (UTC)

Dabfix hasn't been working for me today. It keeps giving a "<type 'exceptions.ValueError'>" error. Cheers, Ost (talk) 19:00, 7 May 2014 (UTC)

Thanks for the changes on the 14th; seems to be working reliably for me again. —Ost (talk) 17:27, 16 May 2014 (UTC)

Dabfix as of April 2015

Hey Dispenser, I was a huge user of Dabfix before the tool site went under. Do you have any plans to revive this very useful tool? Right now, the script just says "Database connection is down, expect some tools to be limited or non-functioning" and doesn't load any pages (gives the error "A problem occurred in a Python script."). :(

128.84.125.178 (talk) 16:59, 12 April 2015 (UTC)

The script is back up and running. Created a new cursor class to handle the differences between MySQLdb and oursql (Needed rowcount) and now handles Unicode correctly too. I've also removed all %_SearchTerm as they weren't working on Labs. I'll have to replace them with something else. — Dispenser 05:39, 25 September 2015 (UTC)

Python Script error?

Hey everyone,

I was trying to run Dabfix for Herbert earlier but Dabfix keeps saying that there's a python script error. Anyone any ideas?--5 albert square (talk) 02:20, 29 October 2015 (UTC)

Fixed It was trying to scrape the Special page. I've added a fail condition for that. — Dispenser 18:34, 29 October 2015 (UTC)

Tool is tagging for cleanup when you're cleaning up

OK, when I run Dabfix for a random cleanup, I've noticed that it's actually tagging the page that I'm cleaning up for a cleanup. Considering I'm cleaning up the page, should it be tagging it for this? An example of a page where this happened is ABC-TV, fortunately I noticed before I saved the page!--5 albert square (talk) 00:44, 4 January 2016 (UTC)

@Dispenser: not sure if you've seen this, is the tool supposed to do this?--5 albert square (talk) 00:23, 6 January 2016 (UTC)

Presumably if you were observant enough to cleanup all the problems, you wouldn't have missed the template and just pressed the Untag button :-P. Anyway, added a warning if the cleanup template is still there when leaving. — Dispenser 18:52, 27 January 2016 (UTC)

Random cleanup not working?

Hi,

Is the random cleanup working for anyone? I'm clicking on Dabfix then random cleanup and I get the following error message:

Traceback (most recent call last):
  File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 29, in <module>
    import sys; sys.path.append('/user-data/mosdab/');import mosdabbot
ImportError: No module named mosdabbot

/home/dispenser/public_html/cgi-bin/tracebacks/dabfix_fEGTJU.html contains the description of this error

I then tried to run Dabfix on a specific page to see if that was working. In case it matters, the page I picked to use was John Partridge. I clicked on Run Dabfix and that also isn't working, I got the following error message:

<snipped same as above>

I tried logging out and logging back in and no difference.

@Dispenser: is the tool down?--5 albert square (talk) 23:31, 19 January 2016 (UTC)

Fixed I was cleaning the mosdab directory (no need for a slow, crash prone, HTML version) and deleted the many symlinks. Apparently, some of them were still used. I've improved the fail state if that module is not present. — Dispenser 01:00, 20 January 2016 (UTC)

Yep, working for me now. Thanks for that! Sorry about repeating the error message, I could have sworn that it was different! :)--5 albert square (talk) 01:14, 20 January 2016 (UTC)

Python error from Dabix

Hi. When I run dabfix on any page I get the following error:

 Traceback (most recent call last):
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 3269, in <module>
   main()
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2761, in main
   robot.run()
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2647, in run
   self.text = re.sub(ur'^:*[#*]+.*$', self.parseline, self.text, flags=re.M)
 File "/usr/lib/python2.7/re.py", line 151, in sub
   return _compile(pattern, flags).sub(repl, string, count)
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 1221, in parseline
   pline,extra = parseline(self, self.cursor, line, self.prefixes)
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 609, in parseline
   line = re.sub(pattern, repl, line, flags=re.U | re.I)
 File "/usr/lib/python2.7/re.py", line 151, in sub
   return _compile(pattern, flags).sub(repl, string, count)
 File "/usr/lib/python2.7/re.py", line 244, in _compile
   raise error, v # invalid expression
 error: nothing to repeat

Thanks. Tassedethe (talk) 12:41, 20 March 2016 (UTC)

Fixed. While working on a bug yesterday, I extended the year range to ?-2010. I forgot to escape ?. — Dispenser 13:38, 20 March 2016 (UTC)

Parsing error on 111 (disambiguation)

Hi Dispenser. When I try and run dabfix on 111 (disambiguation) I get an error:

 Traceback (most recent call last):
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 3286, in <module>
   main()
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2778, in main
   robot.run()
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2623, in run
   runsection(self.addWiktionary)
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2563, in runsection
   func()
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2109, in addWiktionary
   definitions = self.getdefinitions(maximum=5)
 File "/home/dispenser/public_html/cgi-bin/dabfix.py", line 2073, in getdefinitions
   x = dr['page_len_adj'] - (500 if dr['Cited'] else 0) - (300 if dr['Links'] < 3 else 0) - 200 * dr['Alt_of']
 TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

No problem running on 11 (disambiguation) though. Tassedethe (talk) 23:48, 26 June 2016 (UTC)

Fixed You found one of the 2,494 entries (of 4,733,119 or 0.005%) that didn't use any templates. And where the search didn't find any other terms to use. — Dispenser 21:02, 27 June 2016 (UTC)

Carla (disambiguation)

Hi Dispenser. When I run dabfix on Carla (disambiguation) I get an error. From the end of the message: /home/dispenser/public_html/cgi-bin/tracebacks/dabfix_KeyError_3850_HPEH9h.html contains the description of this error.

Thanks. Tassedethe (talk) 00:00, 23 May 2017 (UTC)

Fixed Wikibase is garbage. This should not be possible: {u'datatype': u'time', u'property': u'P569', u'snaktype': u'somevalue'} That existed on d:Q18684600. I've change the code to handle NULL valued time. — Dispenser 03:22, 23 May 2017 (UTC)

Fune

Hi Dispenser. I ran dabfix on Fune and got an error: /home/dispenser/public_html/cgi-bin/tracebacks/dabfix_NameError_3863_mfkkn1.html contains the description of this error. Thanks! Tassedethe (talk) 23:18, 19 June 2017 (UTC)

Fixed I spelt it description instead of descriptions. — Dispenser 14:26, 22 June 2017 (UTC)

Error with date format in the blue link

Hi Dispenser. If I run dabfix on John Collins then it identifies a blue link:

== BLUE LINK: People ==
*John Collins (died 1795)

But simultaneously tells me "Consider removing John Collins (died 1795) per MOS:DABRL." This is because the actual link is John Collins (d. 1795). I assume the formatting algorithm that is applied to get correct dob/dod is being applied to the link. Thanks. Tassedethe (talk) 00:00, 22 June 2017 (UTC)

Fixed. Your assumption was correct, it was correcting the link from (d. 1795) to (died 1795). — Dispenser 14:21, 22 June 2017 (UTC)

Thanks for both fixes! Tassedethe (talk) 00:31, 23 June 2017 (UTC)

ERROR: simplifyLinks exception: 'NoneType' object has no attribute 'group'

Hi Dispenser. I occasionally get errors running Dabfix but 99% resolve themselves when rerun. This particular page has been repeatedly failing (for at least 5 days):

https://dispenser.info.tm/~dispenser/cgi-bin/dabfix.py?page=Bill_Hanrahan

/home/dispenser/public_html/cgi-bin/tracebacks/dabfix_AttributeError_4030_iN24bD.html contains the description of this error.

Thanks! Tassedethe (talk) 21:33, 20 June 2018 (UTC)

Error: Unable to get extract from <pagenames>'s HTML

Hi Dispenser. The last day or so I'm seeing many more errors of the form "Error: Unable to get extract from <pagenames>'s HTML". For example for the dab page Looking Forward (disambiguation) it couldn't extract from any of the 4 pages: https://dispenser.info.tm/~dispenser/cgi-bin/dabfix.py?page=Looking_Forward_(a) I thought perhaps there some sort of header change, or perhaps Infobox? Thanks! Tassedethe (talk) 18:16, 6 July 2018 (UTC)

Disambiguation pages no longer need short-page monitoring

Note the page history here. Just after my Bot1058 task 6 removed the text added by {{subst:long comment}}, another editor added it right back using Dabfix. – wbm1058 (talk) 18:26, 27 December 2019 (UTC)

Fails to complete: Division by 0 or Table 'wikidatawiki_p.wb_items_per_site' doesn't exist

Dabfix has been giving me an error that is either "Division by 0" or "Table 'wikidatawiki_p.wb_items_per_site' doesn't exist". These errors have been preventing me from getting any results from the tool. —Ost (talk) 20:09, 12 July 2021 (UTC)