User talk:The Transhumanist/AWE.js
![]() |
Source code |
i18n file |
Stylesheet |
Run script |
Settings page |
- This is the workshop support page for the user script AWE.js. Comments and requests concerning the program are most welcome. Please post discussion threads below the section titled Discussions. Thank you. By the way, the various scripts I have written are listed at the bottom of the page.[1]
- This script is functional, and is undergoing further development
This is a fork of JWB.js, and is undergoing extensive commenting on its sourcepage, while I figure out how the thing works.
Script's workshop
[edit]- This is the work area for developing the script and its documentation. Even though this documentation is in user space, feel free to edit it. The talk page portion of this page starts at #Discussions, below.
Description & instruction manual for AWE.js
[edit]AutoWikiEditor is a script that allows users to make semi-automated edits more easily. For general use, it works similarly to the downloadable AutoWikiBrowser, but it requires no executable installation, and can run on any (major) operating system. This script heavily borrows from AWB in design and functionality. For example, it uses AWB's Regex Typo Fixing and User Checkpage. To use this script, you need to be listed on the wiki's AutoWikiBrowser CheckPage (Project:AutoWikiBrowser/CheckPage on your wiki), or have administrator rights on the wiki.
Installation
[edit]To install this script on any wiki, put the following code in your JavaScript file (for example Special:MyPage/common.js, meta:Special:MyPage/global.js, or a skin-specific JS file):
mw.loader.load('//en.wikipedia.org/w/index.php?title=User:The_Transhumanist/AWE.js/load.js&action=raw&ctype=text/javascript');
or for users on en.wikipedia.org itself:
{{subst:iusc|User:The_Transhumanist/AWE.js/load.js}}
Getting started
[edit]After installing the script, simply go to this page to run the script. For users using the script on another wiki, simply go to the page Project:AutoWikiBrowser/Script on the wiki you'd like to start the script on, and have installed the above code on.
The interface of AWE may seem a bit crowded at first, but you'll eventually get used to working with it. This densely packed interface allows for fewer different tabs to be used, which allows you to do more things without having to switch tabs.
The most important part of the script of course is the page list. You can enter any amount of pages in the page list in the bottom left corner of the interface. The page list, along with the edit area and the results window (the top part) are resizable, and for the text fields, will expand over the rest of the AWE interface to prevent it from moving the content aside.
While editing, all inputs will be disabled, so to modify your settings, you first need to press the stop button.
Page list
[edit]In the first tab (Setup), you will find a set of options that allows you to perform certain actions on the list of pages. The Remove duplicates button will automatically filter out any duplicate page names, and the Sort button will sort the page list alphabetically.
Page variables
[edit]You can define page variables in the page list. This can be done by putting a |
after the page name, and following it with the value of the page variable. This variable can then be accessed using $x
in the Replace and With replacement boxes, in both of the Skip when ... boxes, and for sysops, the target page name box.
This can be useful when moving large amounts of pages. Simply prepare a list of original page names with the new page name put after it with a |
in between, and it will automatically fill in the new page name. It can also be used when you need to replace a specific word that's different for each of the pages in your list, or if you need to replace something with a specific word. If no page variable is specified, the page title is used as a page variable.
Pre-parsing
[edit]The pre-parse mode can be used to automatically filter your pagelist to those that meet certain requirements. Simply check the Use pre-parse mode box, and define your skip requirements in the Skip tab. After having finished defining your skip requirements, press start to begin loading each page in the list, and either skipping it when it meets the skip requirements, or moving it to the bottom of the page list to be used later.
AWE will automatically insert a #PRE-PARSE-STOP
flag at the bottom of the list, so that it only goes through the list of pages once. If you want to change where the pre-parsing should stop, simply insert #PRE-PARSE-STOP
on a separate line in the page list below the last page you want to pre-parse (as if it were a page name).
You can press stop any time you like during the pre-parsing. The #PRE-PARSE-STOP
will be left in place then, so that you can simply modify your skip rules, and continue on, or you can press the reset button next to the Use pre-parse mode checkbox to re-parse the whole page list again.
Generating page lists
[edit]To generate a page list, simply click the button Generate in the Setup tab. That will open up a window where you can specify which requirements the page list must meet. Note that generating a page list is limited to 50 consecutive requests at a time. This is to prevent overloading the server. The maximum amount of pages per request depends on which generator is used. For pages in a category or links on a page, the limit is 500 (or 5000 for bots), and for the other generators it is 50 (or 500 for bots).
Each of these generators filters its results based on the selected namespaces in the namespace box to the right of the generator options. You can select multiple namespaces by either holding the Ctrl or Shift keys and clicking, or dragging across the namespaces. If the generated list of pages exceeds the maximum amount of pages, you can use this to filter down the results to get the full list, in several steps.
Simply check the box before the field set's legend to enable the fields and include that list generator when generating the page list.
- Category
- Gets all category members of the entered category.
To exclude either subcategories, files, or regular pages, simply uncheck the respective checkbox to disable it.
- Links to page
- Gets all links to the specified page.
- It can include regular wikilinks, template transclusions, or file usage. These can all be used simultaneously.
- It also has the option to filter for redirects. You can choose whether to include redirects only, to exclude redirects, or to include both redirects and wikilinks.
- Finally, it has the option to include links to the page's redirects. That means that if page A redirects to page B, and page C links to page A, it will still be included in the page list when this checkbox is checked.
- Pages with prefix
- Gets a list of all pages that start with the specified prefix. This can be useful when trying to generate a list of all subpages for a certain page for example.
- Watchlist
- This generator simply fetches all pages in your watchlist.
- Links on page
- Gets a list of all links that are included in the specified page. This can be useful when using a more complex page list generator such as DPL.
Skip options
[edit]Using the Skip tab, you can set several rules for which pages should be skipped automatically.
- Redirects
- You can either select to follow redirects and edit the page the redirect leads to (this will also follow double redirects), or edit the redirects themselves. You can also select to skip redirect pages altogether.
- No changes
- If you select to skip when no changes are made, the page will automatically skip when the defined find&replace rules don't apply any changes to the page.
- Page existence
- You can either select to skip when the page exists, skip when it doesn't exist, or edit both existing and non-existing pages.
- Contents
- You can skip either when the page contains a certain string, or when it doesn't contain that. If the phrase entered in When page contains results in any matches on the page, or if the phrase entered in When page doesn't contain results in no matches, the page will be skipped.
- You can either enter a certain keyphrase that should or should not occur on the page, or you can select to use a regular expression. For more information on regular expressions, see #Using regex.
Editing options
[edit]In the Editing tab, you can specify the basic settings for editing. You can fill in the summary, and whether or not to mark the edit as 'minor'. The checkbox next to the summary input box toggles whether or not to append (via AWE) to your summary automatically. You can also select what to do to your watchlist. You can either select to add or remove every page you edit to or from your watchlist, not to modify your watchlist, or to watch pages based on your preference settings.
While editing, you can also directly add or remove pages to or from your watchlist by pressing the button next to the watchlist dropdown. The button will be labelled with what action it will trigger (either adding or removing).
Every page will automatically display the difference between the stored content, and the content after the find&replace rules are applied. You can also press the Preview button to see the generated output. Some styles may not be loaded in this view though, since AWE uses a page structure different from the normal page contents. This may cause styles that depend on the position in the document not to function anymore.
Users with the bot usergroup will also be able to automatically save their edits. This can be done with a throttle to prevent overloading the server. The number entered in the "every []
sec" field is the amount of seconds between edits. This number can also contain decimals (using a .
as decimal mark).
Replacements
[edit]Perhaps the most important feature of AWE is automated replacements. You can define the rules for replacing in the Editing tab. By default, all replacements are performed globally. That means that specifying that a should be replaced with b, then every single letter a in the document will be replaced with b. When using regular expressions, this can be disabled by removing the g
flag.
Newlines can be inserted in both Regex and regular mode, by putting \n
in the place where you'd like to insert a newline character. You can also insert a backslash by putting \\
in the text box.
Typo fixing
[edit]You can use the AWB typo list to automatically fix any typos that exist on the page. Keep in mind that you must always check if the automatic fixes are correct. These automatic fixes will not be applied to image names, template names and parameters[2], quotes[3], and any text following a colon or asterisk, as well as skipping any rule that also matches a wikilink target. These rules are taken from the rules listing for AWB.
Do note that this uses the typo list on the wiki the script is executed from. If you are using this script on a wiki which has no Project:AutoWikiBrowser/Typos page, typo fixing will not work.
Using regex
[edit]Aside from simple text find&replace rules, you can also define regular expression replacements. Here you have more control over what gets matched, and what doesn't. You can specify the flags you'd like to use in the box next to the Regular Expression checkmark.
In this AWE script, you can also specify the _
(underscore) flag. That will make the expression treat all spaces and underscores equally. That can be useful when dealing with wikilinks, template transclusions, etc.
To perform multiple replacements on the same page, you can use the More repace fields button, which will open up a window where you can add more rules in the same format as the format in the Editing tab. When you press Tab while having your cursor on the last replacement rule, another set of inputs will be added. Alternatively, you can press the Add more fields button.
To test regular expressions (albeit without the _
flag), you can use online IDEs such as http://regex101.com/, http://debuggex.com/, or http://regexr.com/.
Ignoring unparsed content
[edit]Checking this box will cause the replacement rule to be performed only on content outside the following:
- Comments:
<!-- This will be ignored -->
- Code tags: contents of
<nowiki>
,<code>
,<source>
,<syntaxhighlight>
and<pre>
will be ignored - Formatting tags:
<math>
,<gallery>
,<math>
and<timeline>
The content within those tags will be left untouched. This option will also work when not using regular expressions.
Note that these exception rules are not the same as the rules for Regex Typo fixing; elements such as quotes and links will not be ignored with this mode.
Settings
[edit]AWE allows you to store your current settings either on the wiki, or on your own computer. You can also temporarily store them during this session by clicking Store setup. That allows you to go back to the temporarily stored settings during this browser session, so that you can for example re-run AWE on a certain page list. To access them in a later session, you have to save them to the wiki, or download them.
Saving to the wiki stores your settings on Special:MyPage/AWB-settings.js. Settings from this page are also automatically loaded to AWB when it loads, and the settings named "default"
will automatically be applied when it loads. If you've modified your AWE settings on your /AWB-settings.js subpage, you can also refresh them in AWE by clicking the Update button.
You can also download the settings as a JSON file. When saving this file, be sure to specify the file extension .json
, otherwise you won't be able to select it via the Import button. You can import files by either clicking Import and selecting the JSON file, or by dragging a JSON file over the AWE window and releasing it (drag&drop). If the file extension is JSON, it will automatically load the settings from the file.
You can also delete a setup from your list by selecting the setup via the dropdown menu, and pressing Delete. If you accidentally clicked this button, you can easily revert the deletion by pressing Undo in the status bar.
Other actions
[edit]Administrators can also perform other actions on the page. These actions can not be automated, not even on accounts with both bot and sysop rights.
Moving
[edit]When moving pages, the move summary used is taken from the Editing tab. You can select to suppress redirect, which prevents the move from leaving a redirect in the place of the old page's title. You can also select to move all subpages and/or the talk page.
Deleting
[edit]Deleting, like moving uses the summary box for its delete reason.
Protecting
[edit]For protection, you can specify the protection levels for moving and creating separately. When the page doesn't exist, the level defined in the edit protection is instead applied to the create protection. The expiry field allows input in the same way as normal time input does. That means inputs like "1 hour", "5 January 2000", etc. are allowed here.
Logs
[edit]The Log tab stores a history of all actions made in chronological order. The bottom stats bar also keeps track of these actions. The logs also contain links to the edits made, so you can review your edit after making it.
Explanatory notes (source code walk-through)
[edit]This section is for explaining the source code in detail. However, I've decided to put all the notes on the source code page itself, for the time being. It is more convenient to have them there while I'm figuring out the program. See User:The Transhumanist/AWE.js.
General approach
[edit](general approach goes here)
More specifically, starting at the beginning...
Change log for AWE.js
[edit]- (Use hyphens for date separators)
- 2018-02-19
- Forked JWB.js to AWE.js
- Forked JWB.css to AWE.css
- Forked JWB/i18n.js to AWE/i18n.js
- Forked JWB/load.js to AWE/load.js
- Updated load.js to point to AWE - AWE runs
- Removed install instructions (they will be covered on this workshop page)
- Forked JWB.js to AWE.js
Copied AWE (manual) to this workshop page
- 2018-03-14
- Forked RETF.js
- 2018-09-12
- Completed forking...
- Search/replaced "JWB" to "AWE" in AWE.js
- Search/replaced "JWB" to "AWE" in i18n.js
- Search/replaced "JWB" to "AWE" in load.js
- Search/replaced "JWB" to "AWE" in RETF.js
- Search/replaced "JWB" to "AWE" in AWE.css, and updated comments to reflect fork
- Search/replaced "JWB" to "AWE" in AWE.js
- Tested fork - it runs
- Started adding comments to explain the program in detail
- Global namespace
- First if statement
- Moved RETF.js to be subpage of AWE.js
- For support of All pages prefix listing
- Completed forking...
Task list
[edit]- Update source references in load.js
Done
- Update AWE source to access its copy of i18n.js
- Update source references in i18n.js
- Ascertain relation of RETF.js to AWE, if any.
- Update other source references in AWE
- Integrate the load code into the main program (it is currently a separate script)
- Install and test it from this new location
- Re-enable summary box (possibly more) when not busy submitting.
- Split up i18n to separate files per language (in the same way MediaWiki does it)
Bug reports
[edit]Desired/completed features
[edit]- Completed features are marked with
Done
- The purpose of this fork is to develop the program further and possibly in new directions
- Integrate the menu item feature from the load.js script, so that it works from within this script.
- More advanced pagelist-generating options
- Feature to generate page list based on images on a page
- Feature to perform general cleanup (<table> to {|, fullurl-links to wikilinks, removing underscores from wikilinks)
Development notes for AWE.js
[edit]Rough rough talk-through
[edit]Script dependencies
[edit]Discussions
[edit]- This is where the actual talk page starts for AWE.js. Please post your discussion threads below...
- ^
- ^ Because of the non-regular nature of wikitext, this script will recognise any closing
}}
as the end of the outermost template. This means that if a template is nested inside another, and a typo occurs after that nested template, it will not be ignored. - ^ Quotes that are not surrounded by a quotation template (ie. quotes surrounded by plain
""
) can only span a single line to be recognised. If an opening quote is on one line and the quote isn't ended until the next line, it will not be regarded as a quote, and will not be ignored. Please watch for quotes that do not get recognised manually.