From 6ea6c5c69c1d4baaa8ffc8f29f8507085b7a4911 Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:34:05 -0500 Subject: [PATCH 1/6] Show wiki matches in the board search, with an ACP switch and block size A "Wiki (N)" block above the forum results of a keyword search, and an "Also search the Wiki" option in the advanced search, through the core search events and two template events. Settings: on/off and the number of articles in the block. The wiki's search page and the block share one search-results service. Fixes the MySQL native backend's total for titles-only and text-only searches, which counted one row whatever matched. All three backends are listed in the ACP, the unusable ones disabled. The finished search plan is archived and the TODO decisions recorded. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 8 +- adm/style/acp_wiki_search.html | 18 ++ config/services.yml | 31 ++- controller/acp_search.php | 20 +- controller/search.php | 111 +--------- docs/TODO.md | 30 +-- .../archive1.md} | 24 ++- docs/TODO/todo-wiki-markup.md | 19 +- docs/TODO/todo-wiki-templates-repo.md | 13 +- event/search_listener.php | 183 +++++++++++++++++ language/en/acp_wiki.php | 6 + language/en/search.php | 8 + migrations/add_wiki_board_search.php | 32 +++ search/backend/wiki_fulltext_native.php | 4 +- .../search_body_search_options_append.html | 9 + .../event/search_results_header_after.html | 30 +++ wiki/search_results.php | 189 ++++++++++++++++++ 17 files changed, 587 insertions(+), 148 deletions(-) rename docs/TODO/{todo-forum-search-integration.md => archive/archive1.md} (58%) create mode 100644 event/search_listener.php create mode 100644 migrations/add_wiki_board_search.php create mode 100644 styles/prosilver/template/event/search_body_search_options_append.html create mode 100644 styles/prosilver/template/event/search_results_header_after.html create mode 100644 wiki/search_results.php diff --git a/README.md b/README.md index 22215ee..f0d23ae 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ A wiki for phpBB with version history, side-by-side compare, and a moderation qu - Tables, written as HTML in an article: ``, ``, ``, ``, ``, `
`, `
` and ``, with `colspan` and `rowspan` on a cell. BBCode and `[[links]]` work inside cells. Any other attribute (`style`, `onclick`, `class`) is dropped, and any other element (`script`, `iframe`) stays plain text. Like headings, tables work in wiki articles only; in a forum post `` stays plain text. After upgrading, disable and enable the extension so the board rebuilds its cache. - Templates: `{{Name}}`, `{{Name|first|second}}` and `{{Name|key=value}}` in an article include the live version of the article `Template:Name`, so a notice or infobox is written once and edited in one place. See [Templates](#templates). - Categories: `[[Category:Name]]` in an article puts it in a category. The article lists its categories at the bottom, `/wiki/Category:Name` lists the articles in a category, and `/wiki/categories` lists every category. See [Categories](#categories). -- Search: `/wiki/search` and a search box on the overview find articles by their title, description and text, with highlighted snippets. It has its own index, kept current as articles are saved, approved, set offline and deleted, and three backends to choose from in the ACP (native, MySQL fulltext, PostgreSQL fulltext). See [Search](#search). +- Search: `/wiki/search` and a search box on the overview find articles by their title, description and text, with highlighted snippets. It has its own index, kept current as articles are saved, approved, set offline and deleted, and three backends to choose from in the ACP (native, MySQL fulltext, PostgreSQL fulltext). The board's own search can show the matching articles in a block of their own, with an "Also search the Wiki" option. See [Search](#search). - The Wiki link in the board header shows the Wikipedia icon by default. Under **Extensions > Wiki > Settings** in the ACP the icon can be changed to any [Font Awesome](https://fontawesome.com/v4.7.0/icons/) icon name, or left blank for no icon. - A **Start discussion** button on each article creates its discussion topic in a forum chosen under **Extensions > Wiki > Settings** in the ACP, and links the topic to the article. It shows to members who hold the "Can change the discussion topic for an article" permission and may post in that forum, and stays hidden until a forum is chosen. - Separate permissions for viewing, editing, version history, deleting and moderating. @@ -120,7 +120,9 @@ The wiki has its own search, separate from phpBB's search of posts: **Search the - It follows the articles: saving a live article, approving a version, setting an article offline, putting it back and deleting it all update the search at once. - The search needs the permission to read the wiki and phpBB's own "Can use search" permission, and it is off when the board's search is off. -**In the ACP**, under **Extensions > Wiki > Search**, switch the wiki search on or off, set the results per page, and choose the backend: +**In the board's search.** A search for words in phpBB's own search, or from the search box in the header, also looks in the wiki. The matching articles are shown in a **Wiki (N)** block above the forum results, with the title, description, a snippet and a link to all of the matches; the forum results themselves are not changed. The advanced search has an **Also search the Wiki** option, ticked by default, to leave the wiki out. The block shows only on the first page of the results, only for members who may read the wiki, and not for searches by author, within one topic, or the pre-made ones such as active topics. + +**In the ACP**, under **Extensions > Wiki > Search**, switch the wiki search on or off, set the results per page, switch the board search block on or off and set how many articles it lists, and choose the backend: | Backend | For | Notes | | --- | --- | --- | @@ -128,7 +130,7 @@ The wiki has its own search, separate from phpBB's search of posts: **Search the | MySQL fulltext | MySQL | Adds fulltext indexes to the search's text table | | PostgreSQL fulltext | PostgreSQL | Adds text-search indexes to the search's text table | -Only the backends the database can run are offered. After installing, or after switching backend, press **Create index** under **Search index** for the backend you chose; until then the native backend finds only articles saved since it was installed, and the fulltext backends find nothing. The page shows each index's statistics and can delete it. +All three backends are listed; the ones the database cannot run are marked as unavailable and cannot be chosen. After installing, or after switching backend, press **Create index** under **Search index** for the backend you chose; until then the native backend finds only articles saved since it was installed, and the fulltext backends find nothing. The page shows each index's statistics and can delete it. The search keeps a plain-text copy of each live article in its own table (`wiki_search_text`); **Create index** makes that copy again from the articles before it builds the backend's index, so it is also the way to rebuild everything. diff --git a/adm/style/acp_wiki_search.html b/adm/style/acp_wiki_search.html index ea3835c..432c212 100644 --- a/adm/style/acp_wiki_search.html +++ b/adm/style/acp_wiki_search.html @@ -17,6 +17,17 @@

{L_ACP_WIKI_SEARCH}


{L_WIKI_PER_PAGE_SEARCH_EXPLAIN}
+
+

{L_WIKI_SEARCH_BOARD_EXPLAIN}
+
+ + +
+
+
+

{L_WIKI_SEARCH_BOARD_MAX_EXPLAIN}
+
+
{L_SEARCH_TYPE} @@ -59,6 +70,12 @@

{L_ACP_SEARCH_INDEX}

{L_WIKI_SEARCH_INDEX_EXPLAIN}

+ +
+ {L_INDEX_STATS}{L_COLON} {backend.NAME} +

{L_WIKI_SEARCH_BACKEND_UNAVAILABLE_EXPLAIN}

+
+
{backend.S_HIDDEN_FIELDS} @@ -94,6 +111,7 @@

{L_ACP_SEARCH_INDEX}

{S_FORM_TOKEN}
+ diff --git a/config/services.yml b/config/services.yml index 04fc9db..ae3d5c8 100644 --- a/config/services.yml +++ b/config/services.yml @@ -362,12 +362,24 @@ services: - '@phpbbmodders.wiki.search.backend_factory' - '@phpbbmodders.wiki.search_sync' + # runs a keyword search and turns the matches into rows for a template + phpbbmodders.wiki.search_results: + class: phpbbmodders\wiki\wiki\search_results + arguments: + - '@dbal.conn' + - '@controller.helper' + - '@user' + - '@phpbbmodders.wiki.search.backend_factory' + - '%tables.wiki.article%' + - '%tables.wiki.search_text%' + - '%core.root_path%' + - '%core.php_ext%' + phpbbmodders.wiki.search: class: phpbbmodders\wiki\controller\search arguments: - '@auth' - '@config' - - '@dbal.conn' - '@controller.helper' - '@language' - '@pagination' @@ -375,7 +387,20 @@ services: - '@template' - '@user' - '@phpbbmodders.wiki.search.backend_factory' - - '%tables.wiki.article%' - - '%tables.wiki.search_text%' + - '@phpbbmodders.wiki.search_results' - '%core.root_path%' - '%core.php_ext%' + + phpbbmodders.wiki.search_listener: + class: phpbbmodders\wiki\event\search_listener + arguments: + - '@auth' + - '@config' + - '@controller.helper' + - '@language' + - '@request' + - '@template' + - '@phpbbmodders.wiki.search.backend_factory' + - '@phpbbmodders.wiki.search_results' + tags: + - { name: event.listener } diff --git a/controller/acp_search.php b/controller/acp_search.php index 101d06a..2f56f5b 100644 --- a/controller/acp_search.php +++ b/controller/acp_search.php @@ -103,19 +103,23 @@ public function settings($id, $mode) $search_options = ''; foreach ($this->backends as $search) { - // Only the backends this database can use + // All three are listed; the ones this database cannot run are shown but cannot be chosen + $type = $search->get_type(); if ($search->is_available()) { - $type = $search->get_type(); $selected = ($this->config['wiki_search_type'] === $type) ? ' selected="selected"' : ''; $search_options .= ''; } + else + { + $search_options .= ''; + } } $cfg_array = $this->request->is_set('config') ? $this->request->variable('config', array('' => ''), true) : array(); $updated = $this->request->variable('updated', false); - $settings = array('wiki_search' => 'bool', 'wiki_per_page_search' => 'integer:1:9999'); + $settings = array('wiki_search' => 'bool', 'wiki_per_page_search' => 'integer:1:9999', 'wiki_search_board' => 'bool', 'wiki_search_board_max' => 'integer:1:50'); foreach ($settings as $config_name => $var_type) { if (!isset($cfg_array[$config_name])) @@ -189,6 +193,8 @@ public function settings($id, $mode) 'S_SEARCH_TYPES' => $search_options, 'S_WIKI_SEARCH' => (bool) $this->config['wiki_search'], 'WIKI_PER_PAGE_SEARCH' => $this->config['wiki_per_page_search'] ? (int) $this->config['wiki_per_page_search'] : 10, + 'S_WIKI_SEARCH_BOARD' => (bool) $this->config['wiki_search_board'], + 'WIKI_SEARCH_BOARD_MAX' => $this->config['wiki_search_board_max'] ? (int) $this->config['wiki_search_board_max'] : 5, 'S_SETTINGS' => true, 'U_ACTION' => $this->u_action . '&hash=' . generate_link_hash('phpbbmodders_wiki_search'), )); @@ -261,15 +267,21 @@ protected function index_overview() { foreach ($this->backends as $search) { - // A backend this database cannot use has no index to build + // A backend this database cannot use is listed, but has no index to build if (!$search->is_available()) { + $this->template->assign_block_vars('backend', array( + 'NAME' => $search->get_name(), + 'TYPE' => $search->get_type(), + 'S_AVAILABLE' => false, + )); continue; } $this->template->assign_block_vars('backend', array( 'NAME' => $search->get_name(), 'TYPE' => $search->get_type(), + 'S_AVAILABLE' => true, 'S_ACTIVE' => $search->get_type() === $this->config['wiki_search_type'], 'S_HIDDEN_FIELDS' => build_hidden_fields(array('wiki_search_type' => $search->get_type())), 'S_INDEXED' => $search->index_created(), diff --git a/controller/search.php b/controller/search.php index 0050dc5..5309445 100644 --- a/controller/search.php +++ b/controller/search.php @@ -15,21 +15,12 @@ */ class search { - /** Longest stretch of text shown for a result, in characters */ - const SNIPPET_LENGTH = 300; - - /** Most matches counted, to spare the server on a very broad search */ - const MATCH_LIMIT = 1000; - /** @var \phpbb\auth\auth */ protected $auth; /** @var \phpbb\config\config */ protected $config; - /** @var \phpbb\db\driver\driver_interface */ - protected $db; - /** @var \phpbb\controller\helper */ protected $helper; @@ -51,11 +42,8 @@ class search /** @var \phpbbmodders\wiki\search\wiki_search_backend_factory */ protected $factory; - /** @var string */ - protected $article_table; - - /** @var string */ - protected $search_text_table; + /** @var \phpbbmodders\wiki\wiki\search_results */ + protected $search_results; /** @var string */ protected $phpbb_root_path; @@ -66,7 +54,6 @@ class search /** * @param \phpbb\auth\auth $auth * @param \phpbb\config\config $config - * @param \phpbb\db\driver\driver_interface $db * @param \phpbb\controller\helper $helper * @param \phpbb\language\language $language * @param \phpbb\pagination $pagination @@ -74,16 +61,14 @@ class search * @param \phpbb\template\template $template * @param \phpbb\user $user * @param \phpbbmodders\wiki\search\wiki_search_backend_factory $factory - * @param string $article_table - * @param string $search_text_table + * @param \phpbbmodders\wiki\wiki\search_results $search_results * @param string $phpbb_root_path * @param string $php_ext */ - public function __construct(\phpbb\auth\auth $auth, \phpbb\config\config $config, \phpbb\db\driver\driver_interface $db, \phpbb\controller\helper $helper, \phpbb\language\language $language, \phpbb\pagination $pagination, \phpbb\request\request $request, \phpbb\template\template $template, \phpbb\user $user, \phpbbmodders\wiki\search\wiki_search_backend_factory $factory, $article_table, $search_text_table, $phpbb_root_path, $php_ext) + public function __construct(\phpbb\auth\auth $auth, \phpbb\config\config $config, \phpbb\controller\helper $helper, \phpbb\language\language $language, \phpbb\pagination $pagination, \phpbb\request\request $request, \phpbb\template\template $template, \phpbb\user $user, \phpbbmodders\wiki\search\wiki_search_backend_factory $factory, \phpbbmodders\wiki\wiki\search_results $search_results, $phpbb_root_path, $php_ext) { $this->auth = $auth; $this->config = $config; - $this->db = $db; $this->helper = $helper; $this->language = $language; $this->pagination = $pagination; @@ -91,8 +76,7 @@ public function __construct(\phpbb\auth\auth $auth, \phpbb\config\config $config $this->template = $template; $this->user = $user; $this->factory = $factory; - $this->article_table = $article_table; - $this->search_text_table = $search_text_table; + $this->search_results = $search_results; $this->phpbb_root_path = $phpbb_root_path; $this->php_ext = $php_ext; } @@ -124,32 +108,6 @@ protected function check_allowed() } } - /** - * The words to mark in the results, as a regular expression part, from the search words. - * - * @param string $keywords What was typed - * @return array The words, each already safe in a regular expression - */ - protected function highlight_words($keywords) - { - $hilit = phpbb_clean_search_string(str_replace(array('+', '-', '|', '(', ')', '"'), ' ', $keywords)); - $words = array_filter(explode(' ', $hilit), 'strlen'); - - $safe = array(); - foreach ($words as $word) - { - $word = phpbb_clean_search_string($word); - $word = str_replace('\*', '\w*?', preg_quote($word, '#')); - $word = preg_replace('#(^|\s)\\\\w\*\?(\s|$)#', '$1\w+?$2', $word); - if ($word !== '') - { - $safe[] = $word; - } - } - - return $safe; - } - /** * The search page. * @@ -200,8 +158,9 @@ public function main() return $this->helper->render('search.html', $this->user->lang('WIKI_SEARCH')); } - $correct = $search->split_keywords($keywords, $terms); - if (!$correct || !$search->get_search_query()) + $per_page = max(1, (int) $this->config['wiki_per_page_search']); + $found = $this->search_results->find($keywords, $terms, $fields, $sort_key, $sort_dir, $start, $per_page); + if ($found === false) { $common = $search->get_common_words(); $ignored = count($common) ? $this->language->lang('IGNORED_TERMS_EXPLAIN', implode(' ', $common)) . '
' : ''; @@ -209,58 +168,10 @@ public function main() trigger_error($ignored . ($length ? $this->language->lang('NO_KEYWORDS', $this->language->lang('CHARACTERS', (int) $length['min']), $this->language->lang('CHARACTERS', (int) $length['max'])) : '')); } - $per_page = max(1, (int) $this->config['wiki_per_page_search']); - $sort_by_sql = array('t' => 'article_date', 's' => 'article_title'); - $ids = array(); - $total = (int) $search->keyword_search('posts', $fields, $terms, $sort_by_sql, $sort_key, $sort_dir, '0', array(), 0, array(), '', $ids, $start, $per_page); - - $words = $this->highlight_words($keywords); - $pattern = !empty($words) ? '#(?!<.*)(?]*(?:)#isu' : ''; - - if (!empty($ids)) + $total = $found['total']; + foreach ($found['rows'] as $row) { - $sql = 'SELECT s.article_id, s.article_title, s.article_description, s.article_body, s.article_date, s.author, a.article_url, a.article_user_id - FROM ' . $this->search_text_table . ' s - INNER JOIN ' . $this->article_table . ' a ON (a.article_id = s.article_id) - WHERE ' . $this->db->sql_in_set('s.article_id', $ids) . ' - AND a.article_approved = 1'; - $result = $this->db->sql_query($sql); - $rows = array(); - while ($row = $this->db->sql_fetchrow($result)) - { - $rows[(int) $row['article_id']] = $row; - } - $this->db->sql_freeresult($result); - - // In the order the backend gave - foreach ($ids as $id) - { - if (!isset($rows[(int) $id])) - { - continue; - } - - $row = $rows[(int) $id]; - $title = htmlspecialchars($row['article_title'], ENT_COMPAT, 'UTF-8'); - $description = htmlspecialchars($row['article_description'], ENT_COMPAT, 'UTF-8'); - $snippet = ($fields === 'titleonly') ? '' : get_context(htmlspecialchars($row['article_body'], ENT_COMPAT, 'UTF-8'), $words ? array_map('stripslashes', $words) : array(), self::SNIPPET_LENGTH); - if ($pattern !== '') - { - $mark = '$1'; - $title = preg_replace($pattern, $mark, $title); - $description = preg_replace($pattern, $mark, $description); - $snippet = preg_replace($pattern, $mark, $snippet); - } - - $this->template->assign_block_vars('results', array( - 'TITLE' => $title, - 'DESCRIPTION' => $description, - 'SNIPPET' => $snippet, - 'DATE' => $this->user->format_date($row['article_date']), - 'AUTHOR' => htmlspecialchars($row['author'], ENT_COMPAT, 'UTF-8'), - 'U_ARTICLE' => $this->helper->route('phpbbmodders_wiki_article', array('article' => $row['article_url'])), - )); - } + $this->template->assign_block_vars('results', $row); } $u_params = array('keywords' => $keywords); diff --git a/docs/TODO.md b/docs/TODO.md index d97184b..13f3dc2 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -22,10 +22,20 @@ Not built: The URL slug is fixed forever once an article is created — only the title can change. There's no way to rename a page's URL while keeping its version history attached; today that means manually recreating the article under a -new URL and losing the connection to its history. Needs a "move" action -(permission-gated, presumably `u_wiki_set_redirect`-adjacent or its own new -permission) that renames `article_url` across all of an article's versions -and optionally leaves a redirect stub at the old URL. +new URL and losing the connection to its history. Needs a "move" action that +renames `article_url` across all of an article's versions. + +Decided 10/13/2026: + +- It gets its own permission, named like the existing ones (`u_wiki_move`), + rather than reusing `u_wiki_set_redirect`. +- A move carries the whole article with it: every version, the uploaded files + and the watchers. +- The move form has a "leave a redirect at the old address" box, ticked by default. + +Still to decide: what a moderator sees when moving a `Template:` page (articles that use +it by the old name would break), and whether links in other articles are rewritten or left +to the redirect. ## Categories @@ -45,13 +55,6 @@ Not built: - The member list reads every live article that holds a category mark, so it is meant for wikis of moderate size; a membership table would also fix that. -## Wiki search - -Built: the wiki's own search page, index and ACP page (a port of the Knowledge Base's engine; see -the README and [`docs/TODO/todo-forum-search-integration.md`](TODO/todo-forum-search-integration.md)). -Next: a "Wiki (N)" block and an "Also search the Wiki" checkbox in phpBB's own search, with the wiki -results kept apart from the board results. - ## A wiki-style UI, not just reskinned forum templates Every wiki page right now is built from phpBB's forum-listing template @@ -117,8 +120,9 @@ allow-list so no unsafe HTML gets through. Full plan, including the syntax not t A separate CC BY-SA 4.0 repository of templates adapted from Wikipedia's for what this wiki can do, kept out of the GPL-2.0 extension, with an ACP button that -installs them as `Template:` pages. Waits for the template system in the Wiki -markup item above. Plan, licence rules and open questions: +installs them as `Template:` pages. The template system it needs is built. To be named +`phpbbmodders/wiki-templates` under the phpbbmodders organisation (agreed 10/13/2026), and +created on 10/14/2026 once the go-ahead is given. Plan, licence rules and open questions: [`docs/TODO/todo-wiki-templates-repo.md`](TODO/todo-wiki-templates-repo.md). ## Readable version compare diff --git a/docs/TODO/todo-forum-search-integration.md b/docs/TODO/archive/archive1.md similarity index 58% rename from docs/TODO/todo-forum-search-integration.md rename to docs/TODO/archive/archive1.md index 8369eb5..af7ed04 100644 --- a/docs/TODO/todo-forum-search-integration.md +++ b/docs/TODO/archive/archive1.md @@ -1,16 +1,18 @@ -# Wiki search and forum search integration +# Archive of completed TODOs -Status: direction confirmed 10/12/2026. Step 1 is built (the wiki's own search); step 2 is not started. Companion to the Knowledge Base's plan in +## Wiki search and forum search integration (completed 10/13/2026) + +Status: both steps built. Step 1 (the wiki's own search) was merged on 10/13/2026; step 2 (the board search block and option) followed the same day, with settings in the ACP to switch it off and to set how many articles the block lists. Companion to the Knowledge Base's plan in `phpbbmodders/knowledgebase` (`docs/TODO/todo-forum-search-integration.md`). -## Goal +### Goal Wiki articles can be found with a search, and a member who searches the board with phpBB's normal search also sees matching wiki articles beside the forum results ("option 1": grouped results, not one mixed list). Today the wiki has no search at all: its articles are undiscoverable except by browsing the overview page or knowing the URL. -## Step 1 (built): give the wiki a search by porting the Knowledge Base's +### Step 1 (built): give the wiki a search by porting the Knowledge Base's The Knowledge Base already has a complete search: its own copies of phpBB's three search engines (native, MySQL fulltext, PostgreSQL fulltext) behind a small interface and a factory, its own index @@ -29,7 +31,7 @@ page with highlighting. Port that to the wiki, adapted to wiki articles: newer (the wiki since 10/12/2026), so the Knowledge Base code can be ported as it is, with no rewrite for older PHP syntax. -## Step 2: tie it into the board search (option 1) +### Step 2: tie it into the board search (option 1) - Same design as the Knowledge Base: a listener on the core search events (the same on 3.3.x and 4.0) runs the wiki's own search with the keywords of the board search and shows a "Wiki (N)" @@ -40,15 +42,23 @@ page with highlighting. Port that to the wiki, adapted to wiki articles: changed, and a member who may not view any matching article sees no block and no count. - If an extension point is missing, write the request up under `docs/events/` instead of patching core. -## Verify +### Verify Indexing lifecycle (create, edit, approve, delete, restore, rebuild), permissions in results and counts, the block next to forum results, pagination of the post results untouched, the checkbox, non-keyword searches, and the extension disabled; on phpBB 3.3.x and 4.0 with SQLite, MySQL and PostgreSQL. Record what was and was not covered. -## Not in scope +### Not in scope One mixed, paginated list of posts and articles (would need a hidden "shadow" topic per article and a copy of the wiki permissions into a forum; rejected for the risk of leaking restricted text), Sphinx, and changing how phpBB indexes posts. + +### As built and verified (10/13/2026) + +- The block and the "Also search the Wiki" option follow the plan. The option is a ticked box with a hidden marker field, so a search that does not come from the advanced form (the header search box) includes the wiki by default, and an unticked box stays unticked through the result pages. The block shows on the first page of results only, and not for searches by author, within one topic, or the pre-made ones. Nothing was needed from phpBB's core; no `docs/events/` request was written. +- ACP: the wiki's Search page has "Show wiki matches in the board search" (on or off) and "Articles in that block" (1 to 50, default 5), beside the results-per-page setting for the wiki's own search page. All three backends are listed; the ones the database cannot run are shown but cannot be chosen. +- A bug in the port was found and fixed: on MySQL the native backend reported a total of 1 for titles-only and text-only searches, because the Knowledge Base's copy of the count query always counted one row. The wiki now counts as phpBB core does. +- Tested with the browser test (27 checks for the board search, 22 or 23 for the wiki's own search and ACP) on phpBB 3.3.19 with SQLite, MySQL and PostgreSQL, and phpBB 4.0.0-a3-dev with SQLite, MySQL and PostgreSQL, each with the native backend, and MySQL and PostgreSQL also with their fulltext backend. Unit tests: 256 tests, 563 assertions. The code sniffer is clean. +- Not covered locally: the Windows and MSSQL combinations that run in CI, and styles other than prosilver. diff --git a/docs/TODO/todo-wiki-markup.md b/docs/TODO/todo-wiki-markup.md index 768b03a..e863701 100644 --- a/docs/TODO/todo-wiki-markup.md +++ b/docs/TODO/todo-wiki-markup.md @@ -169,14 +169,11 @@ out. Still to do: A set of ready-made templates, adapted from Wikipedia's, is planned as a separate repository: [Starter templates repository](todo-wiki-templates-repo.md). -## Still to decide - -- Template syntax: MediaWiki's `{{Name | key = value}}`, or a BBCode such as - `[template=Name]`, which fits phpBB's editor buttons better. -- Whether infoboxes get a default floating-box style in prosilver, or each - template brings its own HTML table. -- Heading levels: only `==` and `===`, or the full MediaWiki range. -- Anchor names for headings, so `[[Article#Section]]` matches reliably - (spaces, case, non-ASCII titles). -- Whether existing articles need reparsing after the change, using the - extension's reparser (`php bin/phpbbcli.php reparser:reparse`). +## Decided + +- Template syntax is MediaWiki's `{{Name|key=value}}`; there is no `[template=Name]` BBCode. +- All six heading levels are supported, and `[[Article#Section]]` anchors are built. +- Existing articles are reparsed with the extension's reparser + (`php bin/phpbbcli.php reparser:reparse phpbbmodders_wiki_article`). +- Infoboxes get no default floating-box style in prosilver; each template brings its own + table, as the Extension infobox does (10/13/2026). diff --git a/docs/TODO/todo-wiki-templates-repo.md b/docs/TODO/todo-wiki-templates-repo.md index 255b12d..20f3eb5 100644 --- a/docs/TODO/todo-wiki-templates-repo.md +++ b/docs/TODO/todo-wiki-templates-repo.md @@ -3,7 +3,7 @@ A separate repository of ready-made wiki templates, adapted from Wikipedia's for what this wiki can do, so a board can install a useful set instead of writing every template from nothing. It depends on the template system in -[Wiki markup](todo-wiki-markup.md), which is not built yet. +[Wiki markup](todo-wiki-markup.md), which is built. ## Why a separate repository @@ -65,11 +65,14 @@ Only templates that work as plain wiki text with parameters: - TemplateStyles (CSS shipped inside a template). - Anything whose licence notice on the source page says it is not plain CC BY-SA. +## Decided + +- The repository is named `phpbbmodders/wiki-templates` and lives under the `phpbbmodders` + organisation (10/13/2026). +- It is created on 10/14/2026, now that the template system exists, after the go-ahead + (creating a public repository is an outward-facing step). + ## Still to decide -- The repository's name, and whether it lives under the `phpbbmodders` organisation. -- Whether to create it now, with a few hand-written templates, or after the - template system exists so every file can be tested against it. Creating a public - repository is an outward-facing step and needs a go-ahead first. - How the ACP import is packaged: files read from a downloaded copy of the repository, or a release archive the extension fetches. diff --git a/event/search_listener.php b/event/search_listener.php new file mode 100644 index 0000000..e9c9e0e --- /dev/null +++ b/event/search_listener.php @@ -0,0 +1,183 @@ +auth = $auth; + $this->config = $config; + $this->helper = $helper; + $this->language = $language; + $this->request = $request; + $this->template = $template; + $this->factory = $factory; + $this->search_results = $search_results; + } + + public static function getSubscribedEvents() + { + return array( + 'core.search_modify_forum_select_list' => 'add_option', + 'core.search_modify_url_parameters' => 'keep_option', + 'core.search_results_modify_search_title' => 'add_block', + ); + } + + /** + * Whether the board search may bring in the wiki at all: both switches are on and the + * member may read the wiki. + * + * @return bool + */ + protected function enabled() + { + return $this->config['wiki_search'] && $this->config['wiki_search_board'] && $this->auth->acl_get('u_wiki_view'); + } + + /** + * Whether this search asked for the wiki. A search that does not come from the advanced + * form, such as the search box in the header, includes it; the form sends "wiki_form" so + * that an unticked box can be told from a form that does not have one. + * + * @return bool + */ + protected function wanted() + { + return $this->request->variable('wiki_form', 0) ? (bool) $this->request->variable('wiki_also', 0) : true; + } + + /** + * Show the "Also search the Wiki" option on the advanced search form. + * + * @return void + */ + public function add_option() + { + if (!$this->enabled()) + { + return; + } + + $this->language->add_lang('search', 'phpbbmodders/wiki'); + $this->template->assign_var('S_WIKI_BOARD_SEARCH', true); + } + + /** + * Carry an unticked option through the links of the results (pages, search in results). + * + * @param \phpbb\event\data $event + * @return void + */ + public function keep_option($event) + { + if ($this->enabled() && !$this->wanted()) + { + $event['u_search'] = $event['u_search'] . '&wiki_form=1'; + } + } + + /** + * Show the wiki block on the first page of the results of a search for words. + * + * @param \phpbb\event\data $event + * @return void + */ + public function add_block($event) + { + $keywords = (string) $event['keywords']; + if ($keywords === '' || $event['start'] > 0 || !$this->enabled() || !$this->wanted()) + { + return; + } + + // Searches that are not for words in the forums: the pre-made ones, one author's + // posts, and a search inside one topic + if ($this->request->variable('search_id', '') !== '' || $this->request->variable('author', '', true) !== '' || $this->request->variable('author_id', 0) || $this->request->variable('t', 0)) + { + return; + } + + if (!$this->factory->get_active()->index_created()) + { + return; + } + + $terms = ($this->request->variable('terms', 'all') === 'any' && $this->request->variable('add_keywords', '', true) === '') ? 'any' : 'all'; + $fields = $this->request->variable('sf', 'all'); + $fields = in_array($fields, array('titleonly', 'msgonly'), true) ? $fields : 'all'; + + $max = max(1, (int) $this->config['wiki_search_board_max']); + $found = $this->search_results->find($keywords, $terms, $fields, 't', 'd', 0, $max); + if ($found === false || $found['total'] < 1 || empty($found['rows'])) + { + return; + } + + $this->language->add_lang('search', 'phpbbmodders/wiki'); + foreach ($found['rows'] as $row) + { + $this->template->assign_block_vars('wiki_matches', $row); + } + $this->template->assign_vars(array( + 'S_WIKI_BOARD_BLOCK' => true, + 'WIKI_BOARD_TOTAL' => $this->language->lang('WIKI_BOARD_BLOCK_TITLE', (int) $found['total']), + 'WIKI_BOARD_VIEW_ALL' => $this->language->lang('WIKI_BOARD_VIEW_ALL', (int) $found['total']), + 'S_WIKI_BOARD_MORE' => $found['total'] > count($found['rows']), + 'U_WIKI_BOARD_VIEW_ALL' => $this->helper->route('phpbbmodders_wiki_search', array( + 'keywords' => html_entity_decode($keywords, ENT_COMPAT), + 'terms' => $terms, + 'sf' => $fields, + )), + )); + } +} diff --git a/language/en/acp_wiki.php b/language/en/acp_wiki.php index 6360dca..d51228e 100644 --- a/language/en/acp_wiki.php +++ b/language/en/acp_wiki.php @@ -40,6 +40,12 @@ 'WIKI_SEARCH_ENABLE_EXPLAIN' => 'Lets members search the wiki. When off, the search page and the search box are hidden.', 'WIKI_PER_PAGE_SEARCH' => 'Results per page', 'WIKI_PER_PAGE_SEARCH_EXPLAIN' => 'How many articles the wiki search results page shows at a time.', + 'WIKI_SEARCH_BOARD' => 'Show wiki matches in the board search', + 'WIKI_SEARCH_BOARD_EXPLAIN' => 'Adds an “Also search the Wiki” option to the advanced search and shows the wiki articles that match a board search in a block above the forum results. Needs the wiki search to be on.', + 'WIKI_SEARCH_BOARD_MAX' => 'Articles in that block', + 'WIKI_SEARCH_BOARD_MAX_EXPLAIN' => 'The most articles the block lists. A link to all of the matches is shown below them.', + 'WIKI_SEARCH_BACKEND_UNAVAILABLE' => 'not available on this database', + 'WIKI_SEARCH_BACKEND_UNAVAILABLE_EXPLAIN' => 'This backend needs a different database than the one this board runs on, so it cannot be chosen and has no index to build.', 'WIKI_SEARCH_INDEX_EXPLAIN' => 'Build the index of the search backend you use. Until it is built the native backend finds only articles saved since it was installed, and the MySQL and PostgreSQL backends find nothing. Switching backend does not delete the other one’s index.', 'LOG_WIKI_CONFIG_SEARCH' => 'Altered wiki search settings', 'WIKI_DISCUSSION_FORUM_INVALID' => 'The forum you chose does not exist or cannot hold topics.', diff --git a/language/en/search.php b/language/en/search.php index 89def9f..e83c2a8 100644 --- a/language/en/search.php +++ b/language/en/search.php @@ -38,4 +38,12 @@ 1 => 'Found 1 article', 2 => 'Found %d articles', ), + + 'WIKI_BOARD_SEARCH_ALSO' => 'Also search the Wiki', + 'WIKI_BOARD_SEARCH_ALSO_EXPLAIN' => 'Shows the wiki articles that match in a block of their own above the forum results.', + 'WIKI_BOARD_BLOCK_TITLE' => 'Wiki (%d)', + 'WIKI_BOARD_VIEW_ALL' => array( + 1 => 'View the wiki article', + 2 => 'View all %d wiki articles', + ), )); diff --git a/migrations/add_wiki_board_search.php b/migrations/add_wiki_board_search.php new file mode 100644 index 0000000..d4c23ba --- /dev/null +++ b/migrations/add_wiki_board_search.php @@ -0,0 +1,32 @@ +db->sql_query($sql_count); -// $total_results = (int) $this->db->sql_fetchfield('total_results'); - $total_results = count($this->db->sql_fetchrowset($result)); + // A grouped query gives one row per article; without a group it gives the count itself + $total_results = $sql_array['GROUP_BY'] ? count($this->db->sql_fetchrowset($result)) : (int) $this->db->sql_fetchfield('total_results'); $this->db->sql_freeresult($result); if (!$total_results) diff --git a/styles/prosilver/template/event/search_body_search_options_append.html b/styles/prosilver/template/event/search_body_search_options_append.html new file mode 100644 index 0000000..472b027 --- /dev/null +++ b/styles/prosilver/template/event/search_body_search_options_append.html @@ -0,0 +1,9 @@ + +
+

{L_WIKI_BOARD_SEARCH_ALSO_EXPLAIN}
+
+ + +
+
+ diff --git a/styles/prosilver/template/event/search_results_header_after.html b/styles/prosilver/template/event/search_results_header_after.html new file mode 100644 index 0000000..b6a44da --- /dev/null +++ b/styles/prosilver/template/event/search_results_header_after.html @@ -0,0 +1,30 @@ + +
+
+
    +
  • {WIKI_BOARD_TOTAL}
    • +
+ +
+
+ diff --git a/wiki/search_results.php b/wiki/search_results.php new file mode 100644 index 0000000..abc28f3 --- /dev/null +++ b/wiki/search_results.php @@ -0,0 +1,189 @@ +db = $db; + $this->helper = $helper; + $this->user = $user; + $this->factory = $factory; + $this->article_table = $article_table; + $this->search_text_table = $search_text_table; + $this->phpbb_root_path = $phpbb_root_path; + $this->php_ext = $php_ext; + } + + /** + * The words to mark in the results, as parts of a regular expression, from the search words. + * + * @param string $keywords What was typed + * @return array The words, each already safe in a regular expression + */ + public function highlight_words($keywords) + { + include_once($this->phpbb_root_path . 'includes/functions_content.' . $this->php_ext); + + $hilit = phpbb_clean_search_string(str_replace(array('+', '-', '|', '(', ')', '"'), ' ', $keywords)); + $words = array_filter(explode(' ', $hilit), 'strlen'); + + $safe = array(); + foreach ($words as $word) + { + $word = phpbb_clean_search_string($word); + $word = str_replace('\*', '\w*?', preg_quote($word, '#')); + $word = preg_replace('#(^|\s)\\\\w\*\?(\s|$)#', '$1\w+?$2', $word); + if ($word !== '') + { + $safe[] = $word; + } + } + + return $safe; + } + + /** + * Search the live articles. + * + * @param string $keywords What was typed + * @param string $terms 'all' or 'any' + * @param string $fields 'all', 'titleonly' or 'msgonly' + * @param string $sort_key 't' for the last edit, 's' for the title + * @param string $sort_dir 'a' or 'd' + * @param int $start Offset of the first result + * @param int $per_page Most results to return + * @return array|bool False when the words cannot be searched on (all too short or too + * common). Otherwise array('total' => matches, 'rows' => rows) where a row + * has TITLE, DESCRIPTION and SNIPPET (marked, safe to print), DATE, AUTHOR + * and U_ARTICLE. + */ + public function find($keywords, $terms, $fields, $sort_key, $sort_dir, $start, $per_page) + { + $search = $this->factory->get_active(); + if (!$search->split_keywords($keywords, $terms) || !$search->get_search_query()) + { + return false; + } + + $sort_by_sql = array('t' => 'article_date', 's' => 'article_title'); + $ids = array(); + $total = (int) $search->keyword_search('posts', $fields, $terms, $sort_by_sql, $sort_key, $sort_dir, '0', array(), 0, array(), '', $ids, $start, $per_page); + + $rows = array(); + if (!empty($ids)) + { + $rows = $this->build_rows($ids, $keywords, $fields); + } + + return array('total' => $total, 'rows' => $rows); + } + + /** + * @param array $ids Article ids, in the order the backend gave them + * @param string $keywords + * @param string $fields + * @return array The rows, as described for find() + */ + protected function build_rows(array $ids, $keywords, $fields) + { + $words = $this->highlight_words($keywords); + $pattern = !empty($words) ? '#(?!<.*)(?]*(?:)#isu' : ''; + + $sql = 'SELECT s.article_id, s.article_title, s.article_description, s.article_body, s.article_date, s.author, a.article_url, a.article_user_id + FROM ' . $this->search_text_table . ' s + INNER JOIN ' . $this->article_table . ' a ON (a.article_id = s.article_id) + WHERE ' . $this->db->sql_in_set('s.article_id', $ids) . ' + AND a.article_approved = 1'; + $result = $this->db->sql_query($sql); + $found = array(); + while ($row = $this->db->sql_fetchrow($result)) + { + $found[(int) $row['article_id']] = $row; + } + $this->db->sql_freeresult($result); + + $rows = array(); + foreach ($ids as $id) + { + if (!isset($found[(int) $id])) + { + continue; + } + + $row = $found[(int) $id]; + $title = htmlspecialchars($row['article_title'], ENT_COMPAT, 'UTF-8'); + $description = htmlspecialchars($row['article_description'], ENT_COMPAT, 'UTF-8'); + $snippet = ($fields === 'titleonly') ? '' : get_context(htmlspecialchars($row['article_body'], ENT_COMPAT, 'UTF-8'), $words ? array_map('stripslashes', $words) : array(), self::SNIPPET_LENGTH); + if ($pattern !== '') + { + $mark = '$1'; + $title = preg_replace($pattern, $mark, $title); + $description = preg_replace($pattern, $mark, $description); + $snippet = preg_replace($pattern, $mark, $snippet); + } + + $rows[] = array( + 'TITLE' => $title, + 'DESCRIPTION' => $description, + 'SNIPPET' => $snippet, + 'DATE' => $this->user->format_date($row['article_date']), + 'AUTHOR' => htmlspecialchars($row['author'], ENT_COMPAT, 'UTF-8'), + 'U_ARTICLE' => $this->helper->route('phpbbmodders_wiki_article', array('article' => $row['article_url'])), + ); + } + + return $rows; + } +} From 324d22371d87159fa009953b70ea52a4385e9998 Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:34:22 -0500 Subject: [PATCH 2/6] Record the template-page move decision in the TODO Co-Authored-By: Claude Sonnet 5.5 --- docs/TODO.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/TODO.md b/docs/TODO.md index 13f3dc2..a3ee89c 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -32,10 +32,11 @@ Decided 10/13/2026: - A move carries the whole article with it: every version, the uploaded files and the watchers. - The move form has a "leave a redirect at the old address" box, ticked by default. +- Moving a `Template:` page that articles use shows a warning with the number of articles, + and asks for a second confirmation before it goes ahead, because every one of them would + need fixing by hand if no redirect is left. -Still to decide: what a moderator sees when moving a `Template:` page (articles that use -it by the old name would break), and whether links in other articles are rewritten or left -to the redirect. +Still to decide: whether links in other articles are rewritten or left to the redirect. ## Categories From b10d7198f670d247d271fc396b238e77084ae27e Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:35:39 -0500 Subject: [PATCH 3/6] Record that a move leaves links in other articles alone Co-Authored-By: Claude Sonnet 5.5 --- docs/TODO.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/TODO.md b/docs/TODO.md index a3ee89c..048183c 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -35,8 +35,8 @@ Decided 10/13/2026: - Moving a `Template:` page that articles use shows a warning with the number of articles, and asks for a second confirmation before it goes ahead, because every one of them would need fixing by hand if no redirect is left. - -Still to decide: whether links in other articles are rewritten or left to the redirect. +- Links in other articles are not rewritten. The redirect left at the old address covers + them, so it has to stay in place for those links to keep working. ## Categories From 33fac9de02a2f1c9648bf933ac15e93fc99fead7 Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:38:24 -0500 Subject: [PATCH 4/6] Record how the starter templates reach a board Co-Authored-By: Claude Sonnet 5.5 --- docs/TODO.md | 3 ++- docs/TODO/todo-wiki-templates-repo.md | 12 ++++++------ 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/TODO.md b/docs/TODO.md index 048183c..97e743f 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -123,7 +123,8 @@ A separate CC BY-SA 4.0 repository of templates adapted from Wikipedia's for wha this wiki can do, kept out of the GPL-2.0 extension, with an ACP button that installs them as `Template:` pages. The template system it needs is built. To be named `phpbbmodders/wiki-templates` under the phpbbmodders organisation (agreed 10/13/2026), and -created on 10/14/2026 once the go-ahead is given. Plan, licence rules and open questions: +created on 10/14/2026 once the go-ahead is given. Admins upload its zip in the ACP or copy the +files into a folder in `store/`. Plan, licence rules and decisions: [`docs/TODO/todo-wiki-templates-repo.md`](TODO/todo-wiki-templates-repo.md). ## Readable version compare diff --git a/docs/TODO/todo-wiki-templates-repo.md b/docs/TODO/todo-wiki-templates-repo.md index 20f3eb5..b937f20 100644 --- a/docs/TODO/todo-wiki-templates-repo.md +++ b/docs/TODO/todo-wiki-templates-repo.md @@ -43,7 +43,8 @@ out how the licences apply to them. ## Using it from the wiki -An ACP button, "Install starter templates", reads the files and creates one +An ACP button, "Install starter templates", reads the files from the uploaded zip or from the +folder in `store/` and creates one `Template:` page per file. Each created page keeps the header, so the board shows the attribution on the template page itself. Installing is optional and can be repeated; an existing template page is not overwritten without asking. @@ -71,8 +72,7 @@ Only templates that work as plain wiki text with parameters: organisation (10/13/2026). - It is created on 10/14/2026, now that the template system exists, after the go-ahead (creating a public repository is an outward-facing step). - -## Still to decide - -- How the ACP import is packaged: files read from a downloaded copy of the - repository, or a release archive the extension fetches. +- The templates reach a board by hand, with no download by the extension: the admin uploads + the repository's zip in the ACP from the browser, or copies the files by FTP into a folder + in `store/` (folder name to be chosen when it is built). The ACP button then installs from + there (10/13/2026). From 69b9228cd3b21f14a4d70aac5e90fae44f790d07 Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:41:11 -0500 Subject: [PATCH 5/6] Record the edit-conflict decision in the TODO Co-Authored-By: Claude Sonnet 5.5 --- docs/TODO.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/TODO.md b/docs/TODO.md index 97e743f..2c32cd3 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -101,7 +101,11 @@ other happened until they check the pending-approval queue. Even a simple form would help; phpBB's own posting flow has similar double-submission awareness to look at for the pattern. -A stronger option is page locking: opening the edit form locks the +Decided 10/13/2026: build both, and let the admin choose in the ACP between the notice +alone and the lock. When the lock option is on, moderators get a notification that a lock +has occurred. + +The lock works like this: opening the edit form locks the article for that user for a limited time (for example 30 minutes, renewed while they keep editing). Others see who holds the lock and until when, and can't save until it's released or expires. Moderators can break a From 2b8c5fb0398f94f501c62a34d06c3e75cfe7b618 Mon Sep 17 00:00:00 2001 From: William Jacoby Date: Tue, 13 Oct 2026 02:42:16 -0500 Subject: [PATCH 6/6] Record when moderators are notified of an edit lock Co-Authored-By: Claude Sonnet 5.5 --- docs/TODO.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/TODO.md b/docs/TODO.md index 2c32cd3..3035bb0 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -102,8 +102,8 @@ form would help; phpBB's own posting flow has similar double-submission awareness to look at for the pattern. Decided 10/13/2026: build both, and let the admin choose in the ACP between the notice -alone and the lock. When the lock option is on, moderators get a notification that a lock -has occurred. +alone and the lock. When the lock option is on, moderators get a notification when a lock +blocks someone from editing (not each time a lock is placed), so they can break it if needed. The lock works like this: opening the edit form locks the article for that user for a limited time (for example 30 minutes, renewed