Show wiki matches in the board search

README.md

@@ -21,7 +21,7 @@ A wiki for phpBB with version history, side-by-side compare, and a moderation qu
 - Tables, written as HTML in an article: `<table>`, `<caption>`, `<thead>`, `<tbody>`, `<tfoot>`, `<tr>`, `<th>` and `<td>`, with `colspan` and `rowspan` on a cell. BBCode and `[[links]]` work inside cells. Any other attribute (`style`, `onclick`, `class`) is dropped, and any other element (`script`, `iframe`) stays plain text. Like headings, tables work in wiki articles only; in a forum post `<table>` stays plain text. After upgrading, disable and enable the extension so the board rebuilds its cache.
 - Templates: `{{Name}}`, `{{Name|first|second}}` and `{{Name|key=value}}` in an article include the live version of the article `Template:Name`, so a notice or infobox is written once and edited in one place. See [Templates](#templates).
 - Categories: `[[Category:Name]]` in an article puts it in a category. The article lists its categories at the bottom, `/wiki/Category:Name` lists the articles in a category, and `/wiki/categories` lists every category. See [Categories](#categories).
-- Search: `/wiki/search` and a search box on the overview find articles by their title, description and text, with highlighted snippets. It has its own index, kept current as articles are saved, approved, set offline and deleted, and three backends to choose from in the ACP (native, MySQL fulltext, PostgreSQL fulltext). See [Search](#search).
+- Search: `/wiki/search` and a search box on the overview find articles by their title, description and text, with highlighted snippets. It has its own index, kept current as articles are saved, approved, set offline and deleted, and three backends to choose from in the ACP (native, MySQL fulltext, PostgreSQL fulltext). The board's own search can show the matching articles in a block of their own, with an "Also search the Wiki" option. See [Search](#search).
 - The Wiki link in the board header shows the Wikipedia icon by default. Under **Extensions > Wiki > Settings** in the ACP the icon can be changed to any [Font Awesome](https://fontawesome.com/v4.7.0/icons/) icon name, or left blank for no icon.
 - A **Start discussion** button on each article creates its discussion topic in a forum chosen under **Extensions > Wiki > Settings** in the ACP, and links the topic to the article. It shows to members who hold the "Can change the discussion topic for an article" permission and may post in that forum, and stays hidden until a forum is chosen.
 - Separate permissions for viewing, editing, version history, deleting and moderating.
@@ -120,15 +120,17 @@ The wiki has its own search, separate from phpBB's search of posts: **Search the
 - It follows the articles: saving a live article, approving a version, setting an article offline, putting it back and deleting it all update the search at once.
 - The search needs the permission to read the wiki and phpBB's own "Can use search" permission, and it is off when the board's search is off.
 
-**In the ACP**, under **Extensions > Wiki > Search**, switch the wiki search on or off, set the results per page, and choose the backend:
+**In the board's search.** A search for words in phpBB's own search, or from the search box in the header, also looks in the wiki. The matching articles are shown in a **Wiki (N)** block above the forum results, with the title, description, a snippet and a link to all of the matches; the forum results themselves are not changed. The advanced search has an **Also search the Wiki** option, ticked by default, to leave the wiki out. The block shows only on the first page of the results, only for members who may read the wiki, and not for searches by author, within one topic, or the pre-made ones such as active topics.
+
+**In the ACP**, under **Extensions > Wiki > Search**, switch the wiki search on or off, set the results per page, switch the board search block on or off and set how many articles it lists, and choose the backend:
 
 | Backend | For | Notes |
 | --- | --- | --- |
 | Native | Every database, including SQLite | Keeps its own word tables. Words in most articles are treated as common once a wiki has 100 articles or more, as in phpBB's own search |
 | MySQL fulltext | MySQL | Adds fulltext indexes to the search's text table |
 | PostgreSQL fulltext | PostgreSQL | Adds text-search indexes to the search's text table |
 
-Only the backends the database can run are offered. After installing, or after switching backend, press **Create index** under **Search index** for the backend you chose; until then the native backend finds only articles saved since it was installed, and the fulltext backends find nothing. The page shows each index's statistics and can delete it.
+All three backends are listed; the ones the database cannot run are marked as unavailable and cannot be chosen. After installing, or after switching backend, press **Create index** under **Search index** for the backend you chose; until then the native backend finds only articles saved since it was installed, and the fulltext backends find nothing. The page shows each index's statistics and can delete it.
 
 The search keeps a plain-text copy of each live article in its own table (`wiki_search_text`); **Create index** makes that copy again from the articles before it builds the backend's index, so it is also the way to rebuild everything.
 

adm/style/acp_wiki_search.html

@@ -17,6 +17,17 @@ <h1>{L_ACP_WIKI_SEARCH}</h1>
 			<dt><label for="wiki_per_page_search">{L_WIKI_PER_PAGE_SEARCH}{L_COLON}</label><br><span>{L_WIKI_PER_PAGE_SEARCH_EXPLAIN}</span></dt>
 			<dd><input id="wiki_per_page_search" type="number" min="1" max="9999" name="config[wiki_per_page_search]" value="{WIKI_PER_PAGE_SEARCH}"></dd>
 		</dl>
+		<dl>
+			<dt><label for="wiki_search_board1">{L_WIKI_SEARCH_BOARD}{L_COLON}</label><br><span>{L_WIKI_SEARCH_BOARD_EXPLAIN}</span></dt>
+			<dd>
+				<label><input type="radio" class="radio" id="wiki_search_board1" name="config[wiki_search_board]" value="1"<!-- IF S_WIKI_SEARCH_BOARD --> checked="checked"<!-- ENDIF -->> {L_YES}</label>
+				<label><input type="radio" class="radio" name="config[wiki_search_board]" value="0"<!-- IF not S_WIKI_SEARCH_BOARD --> checked="checked"<!-- ENDIF -->> {L_NO}</label>
+			</dd>
+		</dl>
+		<dl>
+			<dt><label for="wiki_search_board_max">{L_WIKI_SEARCH_BOARD_MAX}{L_COLON}</label><br><span>{L_WIKI_SEARCH_BOARD_MAX_EXPLAIN}</span></dt>
+			<dd><input id="wiki_search_board_max" type="number" min="1" max="50" name="config[wiki_search_board_max]" value="{WIKI_SEARCH_BOARD_MAX}"></dd>
+		</dl>
 	</fieldset>
 	<fieldset>
 		<legend>{L_SEARCH_TYPE}</legend>
@@ -59,6 +70,12 @@ <h1>{L_ACP_SEARCH_INDEX}</h1>
 
 	<p>{L_WIKI_SEARCH_INDEX_EXPLAIN}</p>
 	<!-- BEGIN backend -->
+		<!-- IF not backend.S_AVAILABLE -->
+		<fieldset class="tabulated" id="wiki_search_unavailable_{backend.TYPE}">
+			<legend>{L_INDEX_STATS}{L_COLON} {backend.NAME}</legend>
+			<p>{L_WIKI_SEARCH_BACKEND_UNAVAILABLE_EXPLAIN}</p>
+		</fieldset>
+		<!-- ELSE -->
 		<form id="wiki_search_index_{backend.TYPE}" method="post" action="{U_INDEX_ACTION}">
 		<fieldset class="tabulated">
 		{backend.S_HIDDEN_FIELDS}
@@ -94,6 +111,7 @@ <h1>{L_ACP_SEARCH_INDEX}</h1>
 		{S_FORM_TOKEN}
 		</fieldset>
 		</form>
+		<!-- ENDIF -->
 	<!-- END backend -->
 <!-- ENDIF -->
 

config/services.yml

@@ -362,20 +362,45 @@ services:
             - '@phpbbmodders.wiki.search.backend_factory'
             - '@phpbbmodders.wiki.search_sync'
 
+    # runs a keyword search and turns the matches into rows for a template
+    phpbbmodders.wiki.search_results:
+        class: phpbbmodders\wiki\wiki\search_results
+        arguments:
+            - '@dbal.conn'
+            - '@controller.helper'
+            - '@user'
+            - '@phpbbmodders.wiki.search.backend_factory'
+            - '%tables.wiki.article%'
+            - '%tables.wiki.search_text%'
+            - '%core.root_path%'
+            - '%core.php_ext%'
+
     phpbbmodders.wiki.search:
         class: phpbbmodders\wiki\controller\search
         arguments:
             - '@auth'
             - '@config'
-            - '@dbal.conn'
             - '@controller.helper'
             - '@language'
             - '@pagination'
             - '@request'
             - '@template'
             - '@user'
             - '@phpbbmodders.wiki.search.backend_factory'
-            - '%tables.wiki.article%'
-            - '%tables.wiki.search_text%'
+            - '@phpbbmodders.wiki.search_results'
             - '%core.root_path%'
             - '%core.php_ext%'
+
+    phpbbmodders.wiki.search_listener:
+        class: phpbbmodders\wiki\event\search_listener
+        arguments:
+            - '@auth'
+            - '@config'
+            - '@controller.helper'
+            - '@language'
+            - '@request'
+            - '@template'
+            - '@phpbbmodders.wiki.search.backend_factory'
+            - '@phpbbmodders.wiki.search_results'
+        tags:
+            - { name: event.listener }

controller/acp_search.php

@@ -103,19 +103,23 @@ public function settings($id, $mode)
 		$search_options = '';
 		foreach ($this->backends as $search)
 		{
-			// Only the backends this database can use
+			// All three are listed; the ones this database cannot run are shown but cannot be chosen
+			$type = $search->get_type();
 			if ($search->is_available())
 			{
-				$type = $search->get_type();
 				$selected = ($this->config['wiki_search_type'] === $type) ? ' selected="selected"' : '';
 				$search_options .= '<option value="' . $type . '"' . $selected . '>' . $search->get_name() . '</option>';
 			}
+			else
+			{
+				$search_options .= '<option value="' . $type . '" disabled="disabled">' . $search->get_name() . ' (' . $this->language->lang('WIKI_SEARCH_BACKEND_UNAVAILABLE') . ')</option>';
+			}
 		}
 
 		$cfg_array = $this->request->is_set('config') ? $this->request->variable('config', array('' => ''), true) : array();
 		$updated = $this->request->variable('updated', false);
 
-		$settings = array('wiki_search' => 'bool', 'wiki_per_page_search' => 'integer:1:9999');
+		$settings = array('wiki_search' => 'bool', 'wiki_per_page_search' => 'integer:1:9999', 'wiki_search_board' => 'bool', 'wiki_search_board_max' => 'integer:1:50');
 		foreach ($settings as $config_name => $var_type)
 		{
 			if (!isset($cfg_array[$config_name]))
@@ -189,6 +193,8 @@ public function settings($id, $mode)
 			'S_SEARCH_TYPES'		=> $search_options,
 			'S_WIKI_SEARCH'			=> (bool) $this->config['wiki_search'],
 			'WIKI_PER_PAGE_SEARCH'	=> $this->config['wiki_per_page_search'] ? (int) $this->config['wiki_per_page_search'] : 10,
+			'S_WIKI_SEARCH_BOARD'	=> (bool) $this->config['wiki_search_board'],
+			'WIKI_SEARCH_BOARD_MAX'	=> $this->config['wiki_search_board_max'] ? (int) $this->config['wiki_search_board_max'] : 5,
 			'S_SETTINGS'			=> true,
 			'U_ACTION'				=> $this->u_action . '&amp;hash=' . generate_link_hash('phpbbmodders_wiki_search'),
 		));
@@ -261,15 +267,21 @@ protected function index_overview()
 	{
 		foreach ($this->backends as $search)
 		{
-			// A backend this database cannot use has no index to build
+			// A backend this database cannot use is listed, but has no index to build
 			if (!$search->is_available())
 			{
+				$this->template->assign_block_vars('backend', array(
+					'NAME'			=> $search->get_name(),
+					'TYPE'			=> $search->get_type(),
+					'S_AVAILABLE'	=> false,
+				));
 				continue;
 			}
 
 			$this->template->assign_block_vars('backend', array(
 				'NAME'				=> $search->get_name(),
 				'TYPE'				=> $search->get_type(),
+				'S_AVAILABLE'		=> true,
 				'S_ACTIVE'			=> $search->get_type() === $this->config['wiki_search_type'],
 				'S_HIDDEN_FIELDS'	=> build_hidden_fields(array('wiki_search_type' => $search->get_type())),
 				'S_INDEXED'			=> $search->index_created(),

controller/search.php

@@ -15,21 +15,12 @@
  */
 class search
 {
-	/** Longest stretch of text shown for a result, in characters */
-	const SNIPPET_LENGTH = 300;
-
-	/** Most matches counted, to spare the server on a very broad search */
-	const MATCH_LIMIT = 1000;
-
 	/** @var \phpbb\auth\auth */
 	protected $auth;
 
 	/** @var \phpbb\config\config */
 	protected $config;
 
-	/** @var \phpbb\db\driver\driver_interface */
-	protected $db;
-
 	/** @var \phpbb\controller\helper */
 	protected $helper;
 
@@ -51,11 +42,8 @@ class search
 	/** @var \phpbbmodders\wiki\search\wiki_search_backend_factory */
 	protected $factory;
 
-	/** @var string */
-	protected $article_table;
-
-	/** @var string */
-	protected $search_text_table;
+	/** @var \phpbbmodders\wiki\wiki\search_results */
+	protected $search_results;
 
 	/** @var string */
 	protected $phpbb_root_path;
@@ -66,33 +54,29 @@ class search
 	/**
 	 * @param \phpbb\auth\auth											$auth
 	 * @param \phpbb\config\config										$config
-	 * @param \phpbb\db\driver\driver_interface							$db
 	 * @param \phpbb\controller\helper									$helper
 	 * @param \phpbb\language\language									$language
 	 * @param \phpbb\pagination											$pagination
 	 * @param \phpbb\request\request									$request
 	 * @param \phpbb\template\template									$template
 	 * @param \phpbb\user												$user
 	 * @param \phpbbmodders\wiki\search\wiki_search_backend_factory	$factory
-	 * @param string													$article_table
-	 * @param string													$search_text_table
+	 * @param \phpbbmodders\wiki\wiki\search_results					$search_results
 	 * @param string													$phpbb_root_path
 	 * @param string													$php_ext
 	 */
-	public function __construct(\phpbb\auth\auth $auth, \phpbb\config\config $config, \phpbb\db\driver\driver_interface $db, \phpbb\controller\helper $helper, \phpbb\language\language $language, \phpbb\pagination $pagination, \phpbb\request\request $request, \phpbb\template\template $template, \phpbb\user $user, \phpbbmodders\wiki\search\wiki_search_backend_factory $factory, $article_table, $search_text_table, $phpbb_root_path, $php_ext)
+	public function __construct(\phpbb\auth\auth $auth, \phpbb\config\config $config, \phpbb\controller\helper $helper, \phpbb\language\language $language, \phpbb\pagination $pagination, \phpbb\request\request $request, \phpbb\template\template $template, \phpbb\user $user, \phpbbmodders\wiki\search\wiki_search_backend_factory $factory, \phpbbmodders\wiki\wiki\search_results $search_results, $phpbb_root_path, $php_ext)
 	{
 		$this->auth = $auth;
 		$this->config = $config;
-		$this->db = $db;
 		$this->helper = $helper;
 		$this->language = $language;
 		$this->pagination = $pagination;
 		$this->request = $request;
 		$this->template = $template;
 		$this->user = $user;
 		$this->factory = $factory;
-		$this->article_table = $article_table;
-		$this->search_text_table = $search_text_table;
+		$this->search_results = $search_results;
 		$this->phpbb_root_path = $phpbb_root_path;
 		$this->php_ext = $php_ext;
 	}
@@ -124,32 +108,6 @@ protected function check_allowed()
 		}
 	}
 
-	/**
-	 * The words to mark in the results, as a regular expression part, from the search words.
-	 *
-	 * @param string $keywords What was typed
-	 * @return array The words, each already safe in a regular expression
-	 */
-	protected function highlight_words($keywords)
-	{
-		$hilit = phpbb_clean_search_string(str_replace(array('+', '-', '|', '(', ')', '"'), ' ', $keywords));
-		$words = array_filter(explode(' ', $hilit), 'strlen');
-
-		$safe = array();
-		foreach ($words as $word)
-		{
-			$word = phpbb_clean_search_string($word);
-			$word = str_replace('\*', '\w*?', preg_quote($word, '#'));
-			$word = preg_replace('#(^|\s)\\\\w\*\?(\s|$)#', '$1\w+?$2', $word);
-			if ($word !== '')
-			{
-				$safe[] = $word;
-			}
-		}
-
-		return $safe;
-	}
-
 	/**
 	 * The search page.
 	 *
@@ -200,67 +158,20 @@ public function main()
 			return $this->helper->render('search.html', $this->user->lang('WIKI_SEARCH'));
 		}
 
-		$correct = $search->split_keywords($keywords, $terms);
-		if (!$correct || !$search->get_search_query())
+		$per_page = max(1, (int) $this->config['wiki_per_page_search']);
+		$found = $this->search_results->find($keywords, $terms, $fields, $sort_key, $sort_dir, $start, $per_page);
+		if ($found === false)
 		{
 			$common = $search->get_common_words();
 			$ignored = count($common) ? $this->language->lang('IGNORED_TERMS_EXPLAIN', implode(' ', $common)) . '<br>' : '';
 			$length = $search->get_word_length();
 			trigger_error($ignored . ($length ? $this->language->lang('NO_KEYWORDS', $this->language->lang('CHARACTERS', (int) $length['min']), $this->language->lang('CHARACTERS', (int) $length['max'])) : ''));
 		}
 
-		$per_page = max(1, (int) $this->config['wiki_per_page_search']);
-		$sort_by_sql = array('t' => 'article_date', 's' => 'article_title');
-		$ids = array();
-		$total = (int) $search->keyword_search('posts', $fields, $terms, $sort_by_sql, $sort_key, $sort_dir, '0', array(), 0, array(), '', $ids, $start, $per_page);
-
-		$words = $this->highlight_words($keywords);
-		$pattern = !empty($words) ? '#(?!<.*)(?<!\w)(' . implode('|', $words) . ')(?!\w|[^<>]*(?:</s(?:cript|tyle))?>)#isu' : '';
-
-		if (!empty($ids))
+		$total = $found['total'];
+		foreach ($found['rows'] as $row)
 		{
-			$sql = 'SELECT s.article_id, s.article_title, s.article_description, s.article_body, s.article_date, s.author, a.article_url, a.article_user_id
-				FROM ' . $this->search_text_table . ' s
-				INNER JOIN ' . $this->article_table . ' a ON (a.article_id = s.article_id)
-				WHERE ' . $this->db->sql_in_set('s.article_id', $ids) . '
-					AND a.article_approved = 1';
-			$result = $this->db->sql_query($sql);
-			$rows = array();
-			while ($row = $this->db->sql_fetchrow($result))
-			{
-				$rows[(int) $row['article_id']] = $row;
-			}
-			$this->db->sql_freeresult($result);
-
-			// In the order the backend gave
-			foreach ($ids as $id)
-			{
-				if (!isset($rows[(int) $id]))
-				{
-					continue;
-				}
-
-				$row = $rows[(int) $id];
-				$title = htmlspecialchars($row['article_title'], ENT_COMPAT, 'UTF-8');
-				$description = htmlspecialchars($row['article_description'], ENT_COMPAT, 'UTF-8');
-				$snippet = ($fields === 'titleonly') ? '' : get_context(htmlspecialchars($row['article_body'], ENT_COMPAT, 'UTF-8'), $words ? array_map('stripslashes', $words) : array(), self::SNIPPET_LENGTH);
-				if ($pattern !== '')
-				{
-					$mark = '<span class="posthilit">$1</span>';
-					$title = preg_replace($pattern, $mark, $title);
-					$description = preg_replace($pattern, $mark, $description);
-					$snippet = preg_replace($pattern, $mark, $snippet);
-				}
-
-				$this->template->assign_block_vars('results', array(
-					'TITLE'			=> $title,
-					'DESCRIPTION'	=> $description,
-					'SNIPPET'		=> $snippet,
-					'DATE'			=> $this->user->format_date($row['article_date']),
-					'AUTHOR'		=> htmlspecialchars($row['author'], ENT_COMPAT, 'UTF-8'),
-					'U_ARTICLE'		=> $this->helper->route('phpbbmodders_wiki_article', array('article' => $row['article_url'])),
-				));
-			}
+			$this->template->assign_block_vars('results', $row);
 		}
 
 		$u_params = array('keywords' => $keywords);