Skip to content

MDEV-40222 Prevent MTR hang when waiting for wsrep_ready #5312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mariadb-TeemuOllakka wants to merge 1 commit into
base: 10.11
Choose a base branch
from
+31 −6
Open

MDEV-40222 Prevent MTR hang when waiting for wsrep_ready #5312

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 31 additions & 6 deletions mysql-test/mariadb-test-run.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5439,8 +5439,13 @@ ($$)
# Run a query against a server using mysql client. The output of
# the query will be written into outfile.
#
# If $timeout (seconds) is given, the client is not waited for
# indefinitely: a server stuck in an unstable state can accept the
# connection but never answer the query, which would otherwise block forever.
# In that case the client is killed and a non-zero status is returned.
#
sub run_query_output {
my ($mysqld, $query, $outfile)= @_;
my ($mysqld, $query, $outfile, $timeout)= @_;
my $args;

mtr_init_args(\$args);
Expand All @@ -5449,7 +5454,7 @@ sub run_query_output {
mtr_add_arg($args, "--silent");
mtr_add_arg($args, "--execute=%s", $query);

my $res= My::SafeProcess->run
my $proc= My::SafeProcess->new
(
name => "run_query_output -> ".$mysqld->name(),
path => $exe_mysql,
Expand All @@ -5458,7 +5463,15 @@ sub run_query_output {
error => $outfile
);

return $res
# wait_one() returns 1 while the process is still running,
# in which case we kill the hung client.
if ($proc->wait_one($timeout))
{
$proc->kill();
return 1;
}

return $proc->exit_status();
Comment on lines +5466 to +5474

@gemini-code-assist gemini-code-assist Bot Jun 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If $timeout is not passed to run_query_output (which is the case for other callers or if it is omitted), $timeout will be undef. Passing undef to $proc->wait_one($timeout) can cause two issues:\n\n1. It may trigger a Perl warning about an uninitialized value.\n2. If wait_one treats undef as 0 (non-blocking poll), it will return 1 immediately because the process is still running, causing the client to be killed prematurely.\n\nTo prevent this, we should explicitly check if $timeout is defined before passing it to wait_one.

  # wait_one() returns 1 while the process is still running,
  # in which case we kill the hung client.
  if (defined $timeout ? $proc->wait_one($timeout) : $proc->wait_one())
  {
    $proc->kill();
    return 1;
  }

  return $proc->exit_status();

}


Expand All @@ -5476,7 +5489,13 @@ ($$)
my ($tinfo, $mysqld)= @_;

my $sleeptime= 100; # Milliseconds
my $loops= ($opt_start_timeout * 1000) / $sleeptime;

# Bound the whole wait by the server startup timeout. This must be a
# wall-clock deadline rather than a simple loop count: a single query
# against a wedged server can block indefinitely, which would otherwise
# defeat the loop bound and hang MTR until the surrounding suite timeout
# fires.
my $timeout= start_timer($opt_start_timeout);

my $name= $mysqld->name();
my $outfile= "$opt_vardir/tmp/$name.wsrep_ready";
Expand All @@ -5485,11 +5504,17 @@ ($$)
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE VARIABLE_NAME = 'wsrep_ready'";

for (my $loop= 1; $loop <= $loops; $loop++)
while (1)
{
# Cap each query by the time left so a hung client cannot exceed the
# overall startup budget. Integer seconds, and at least 1 (wait_one()
# treats 0 as a non-blocking poll).
my $remaining= int($timeout - time);
last if $remaining <= 0;

# Careful... if MTR runs with option 'verbose' then the
# file contains also SafeProcess verbose output
if (run_query_output($mysqld, $query, $outfile) == 0 &&
if (run_query_output($mysqld, $query, $outfile, $remaining) == 0 &&
mtr_grab_file($outfile) =~ /WSREP_READY\s+ON/)
{
unlink($outfile);
Expand Down