guoxin
/
test-example-projects
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
用于EagleEye3.0 规则集漏报和误报测试的示例项目，项目收集于github和gitee
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
3 Commits
1 Branch
0 Tags
722 MiB
Tag: Branch: Tree: 574f9c1e1d
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '574f9c1e1d'
${ noResults }
test-example-projects/c++_projects/280w-mysql-8.0.18/mysql-test/mysql-test-run.dox

8407 lines
277 KiB
Raw Normal View History Unescape

first commit
3 months ago
########## The MySQL Test Framework ##########
/**
@page PAGE_MYSQL_TEST_RUN The MySQL Test Framework
This manual describes the MySQL test framework, consisting of the
test driver and the test script language.
-----
<h3>Table of Contents</h3>
- @subpage PAGE_PREFACE
- @subpage PAGE_INTRODUCTION
- @subpage PAGE_MYSQLTEST_FRAMEWORK_COMPONENTS
- @subpage PAGE_RUNNING_TESTCASES
- @subpage PAGE_WRITING_TESTCASES
- @subpage PAGE_MYSQLTEST_PROGRAMS
- @subpage PAGE_MYSQLTEST_LANGUAGE_REFERENCE
- @subpage PAGE_UNIT_TESTS
- @subpage PAGE_PLUGINS
*/
###############################################################################
## Preface
/**
@page PAGE_PREFACE Preface
MySQL distributions include a set of test cases and programs for
running them. These tools constitute the MySQL test framework that
provides a means for verifying that MySQL Server and its client
programs operate according to expectations. The test cases consist
mostly of SQL statements, but can also use test language constructs
that control how to run tests and verify their results.
This manual describes the MySQL test framework. It describes the
programs used to run tests and the language used to write test
cases.
*/
###############################################################################
## Introduction to the MySQL Test Framework
/**
@page PAGE_INTRODUCTION Introduction to the MySQL Test Framework
MySQL distributions include a test suite: a set of test cases and
programs for running them. (If you find that the test suite is not
included in your distribution, look for a similar distribution with
<b>-test</b> in the name and install that as well.) These tools
constitute the MySQL test framework that provides a means for
verifying that MySQL Server and its client programs operate
according to expectations. The test cases consist mostly of SQL
statements, but can also use test language constructs that control
how to run tests and verify their results. Distributions also
provide facilities for running unit tests and creating new unit
tests.
This document describes the components of the MySQL test framework,
how the test programs work, and the language used for writing test
cases. It also provides a tutorial for developing test cases and
executing them.
The application that runs the test suite is named @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl". Its location is the
<b>mysql-test</b> directory, which is present both in source and
binary MySQL Server distributions.
On platforms other than Windows, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" is also available through the shortened name
<b>mtr</b> in the same directory, as either a symbolic link or a
copy.
The @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" application
starts MySQL servers, restarts them as necessary when a specific
test case needs different start arguments, and presents the test
result. For each test case, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" invokes the @ref PAGE_MYSQLTEST "mysqltest"
program (also referred to as the “test engine”) to read the test
case file, intepret the test language constructs, and send SQL
statements to the server.
Input for each test case is stored in a file, and the expected
result from running the test is stored in another file. The actual
result is compared to the expected result after running the test.
For a MySQL source distribution, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" is located in the <b>mysql-test</b> directory,
and @ref PAGE_MYSQLTEST "mysqltest" is located in the <b>client</b>
directory. The <b>mysql-test</b> and <b>client</b> directories
are located in the root directory of the distribution.
For a MySQL binary distribution, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" is located in the <b>mysql-test</b> directory,
and @ref PAGE_MYSQLTEST "mysqltest" is located in the same directory
where other client programs such as <b>mysql</b> or
<b>mysqladmin</b> are installed. The locations of the
<b>mysql-test</b> and other directories depend on the layout used
for the distribution format.
Within the <b>mysql-test</b> directory, test case input files and
result files are stored in the <b>t</b> and <b>r</b>
directories, respectively. The input and result files have the same
basename, which is the test name, but have extensions of
<b>.test</b> and <b>.result</b>, respectively. For example, for
a test named “decimal”, the input and result files are
<b>mysql-test/t/decimal.test</b> and <b>mysql-test/r/decimal.result</b>.
Each test file is referred to as one test case, but usually consists
of a sequence of related tests. An unexpected failure of a single
statement in a test case makes the test case fail.
There are several ways a test case can fail:
<ul>
<li>
The @ref PAGE_MYSQLTEST "mysqltest" test engine checks the result
codes from executing each SQL statement in the test input. If the
failure is unexpected, the test case fails.
</li>
<li>
A test case can fail if an error was expected but did not occur
(for example, if an SQL statement succeeded when it should have
failed).
</li>
<li>
The test case can fail by producing incorrect output. As a test
runs, it produces output (the results from <b>SELECT</b>,
<b>SHOW</b>, and other statements). This output is compared
to the expected result found in the <b>mysql-test/r</b> directory
(in a file with a <b>.result</b> suffix). If the expected and
actual results differ, the test case fails. The actual test result
is written to a file in the <b>mysql-test/var/log</b> directory
with a <b>.reject</b> suffix, and the difference between the
<b>.result</b> and <b>.reject</b> files is presented for
evaluation.
</li>
<li>
A test case will fail if the MySQL server dies unexpectedly during
the test. If this happens, the @ref PAGE_MYSQLTEST "mysqltest"
test client will usually also report a failure due to loosing the
connection.
</li>
<li>
Finally, the test case will fail if the error log written by the
MySQL server during the test includes warnings or errors which are
not filtered (suppressed). See @ref PAGE_SUPPRESSING_ERRORS_WARNINGS
for more about suppressing warnings.
</li>
</ul>
This method of checking test results puts some restrictions on how
test cases can be written. For example, the result cannot contain
information that varies from run to run, such as the current time.
However, if the information that varies is unimportant for test
evaluation, there are ways to instruct the test engine to replace
those fields in the output with fixed values.
Because the test cases consist mostly of SQL statements in a text
file, there is no direct support for test cases that are written in
C, Java, or other languages. Such tests are not within the scope of
this test framework. But the framework does support executing your
own scripts and initiating them with your own data. Also, a test
case can execute an external program, so in some respects the test
framework can be extended for uses other than testing SQL
statements. Finally, it is possible to embed small pieces of Perl
code within the test. This can sometimes be used to perform actions
or execute logic which is beyond the capabilities of the test
language or SQL.
*/
###############################################################################
## MySQL Test Framework Components
/**
@page PAGE_MYSQLTEST_FRAMEWORK_COMPONENTS MySQL Test Framework Components
<h3>Table of Contents</h3>
- @subpage PAGE_TEST_FRAMEWORK_SYSTEM_REQUIREMENTS
- @subpage PAGE_REPORT_BUGS
The MySQL test framework consists of programs that run tests, and
directories and files used by those programs.
Test Framework Programs
-----------------------
The MySQL test framework uses several programs:
<ul>
<li>
The @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" Perl script
is the main application used to run the test suite. It invokes
@ref PAGE_MYSQLTEST "mysqltest" to run individual test cases.
</li>
<li>
@ref PAGE_MYSQLTEST "mysqltest" runs test cases.
</li>
<li>
The @ref PAGE_MYSQL_CLIENT_TEST "mysql_client_test" program is
used for testing aspects of the MySQL client API that cannot be
tested using @ref PAGE_MYSQLTEST "mysqltest" and its test
language.
</li>
<li>
The @ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" Perl
script performs stress-testing of the MySQL server.
</li>
<li>
A unit-testing facility is provided so that individual unit test
programs can be created for storage engines and plugins.
</li>
</ul>
Test suite programs can be found in these locations:
<ul>
<li>
For a source distribution, @ref PAGE_MYSQLTEST "mysqltest" is in
the <b>client</b> directory. For a binary distribution, it is in
the MySQL <b>bin</b> directory.
</li>
<li>
For a source distribution, @ref PAGE_MYSQL_CLIENT_TEST
"mysql_client_test" is in the <b>tests</b> directory. For a
binary distribution, it is in the MySQL <b>bin</b> directory.
</li>
<li>
The other programs are located in the <b>mysql-test</b>
directory. For a source distribution, <b>mysql-test</b> is found
under the source tree root. For a binary distribution, the
location of <b>mysql-test</b> depends on the layout used for the
distribution format.
</li>
</ul>
Test Framework Directories and Files
-------------------------------------
The test suite is located in the <tt>mysql-test</tt> directory,
which contains the following components:
<ul>
<li>
The @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" and @ref
PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" programs that
are used for running tests.
</li>
<li>
The <b>t</b> directory contains test case input files. A test
case file might also have additional files associated with it.
</li>
<ul>
<li>
A file name of the form <b><em>test_name</em>.test</b> is a
test case file for a test named <b><em>test_name</em></b>. For
example, <b>subquery.test</b> is the test case file for the
test named <b>subquery</b>.
</li>
<li>
A file name of the form <b><em>test_name</em>-master.opt</b>
provides options to associate with the named test case.
<b>mysql-test-run.pl</b> restarts the server with the options
given in the file if the options are different from those
required for the currently running server.
Note that the <b>-master.opt</b> file is used for the “main”
server of a test, even if no replication is involved.
</li>
<li>
A file name of the form <b><em>test_name</em>-slave.opt</b>
provides slave options.
</li>
<li>
Server options which need to be passed during initialization
of the server can be passed in both
<b><em>test_name</em>-master.opt</b> and
<b><em>test_name</em>-slave.opt</b>. Each bootstrap variable
must be passed as the value of a <b>`--initialize`</b> option so
that @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" can
recognize that the variable should be used during server
initialization, remove the existing data directory, and
initialize a new data directory with the options provided.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will also
start the server afterwards, with the bootstrap options, and
other options specified.
</li>
<li>
A file name of the form <b><em>test_name</em>-client.opt </b>
provides @ref PAGE_MYSQLTEST "mysqltest" client options.
</li>
<li>
A file name of the form <b><em>test_name</em>.cnf</b> contains
additional entries for the config file to be used for this
test.
<ul>
<li>
During a test run, each server is started with a different
<b>defaults-group-suffix</b> value and this suffix value
can be used with a group name to specify server specific
options. For example, section <b>[mysqld.1]</b> is used to
specify options for mysqld server with suffix value
<b>".1"</b>.
</li>
<li>
The <b>[mysqltest]</b> section is used to specify options
for mysqltest client.
</li>
<li>
The <b>[ENV]</b> section is used to specify environment
variables for a test case.
</li>
<li>
To include the options mentioned in another <b>.cnf</b>
file, <b>!include path_to_cnf_file</b> is used. Path to
the included <b>.cnf</b> should either be an absolute path
or a relative path from current location or from
<b>mysql-test</b> directory.
</li>
</ul>
Here is a sample <b>.cnf</b> file.
@verbatim
!include include/default_my.cnf
[mysqld.1]
Options for server mysqld.1
[mysqld.2]
Options for server mysqld.2
[mysqltest]
ps-protocol
...
...
...
[ENV]
SERVER_MYPORT_1= @mysqld.1.port
SERVER_MYPORT_2= @mysqld.2.port
@endverbatim
</li>
<li>
A file name of the form <b><em>test_name</em>.combinations</b>
provides section of options for each test run.
</li>
<li>
A file name of the form <b><em>suite</em>.opt</b> provides
server options to all the tests within the suite it belongs
to.
If a test run is started with more than one server, the
options specified in a <b><em>suite</em>.opt</b> are used by
all of them. It is also possible to pass the server options
needed during the server initialization in the
<b><em>suite</em>.opt</b> file.
The options mentioned in the <b><em>suite</em>.opt</b> file
are overridden by those found in a
<b><em>test_name</em>-master.opt</b> file or in a
<b><em>test_name</em>-slave.opt</b> file.
</li>
<li>
A file name of the form <b><em>test_name</em>-master.sh</b> is
a shell script that will be executed before the server is
started for the named test case. There may also be a
<b><em>test_name</em>-slave.sh</b> which is executed before
the slave server is started.
These files will not work on Windows and may therefore be
replaced with a more portable mechanism in the future.
</li>
<li>
The <b>disabled.def</b> file contains information about
deferred/disabled tests. When a test is failing because of a
bug in the server and you want it to be ignored by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl", list the test in
this file.
The format of a line in the <b>disabled.def</b> file looks
like this, where fields are separated by one or more spaces.
@verbatim
suite_name.test_name [@platform|@!platform] : <BUG|WL>#<XXXX> [<comment>]
@endverbatim
Example:
@verbatim
main.1st @linux : BUG#18980 Test fails randomly
@endverbatim
<b><em>suite_name</em></b> is the suite name.
<b><em>test_name</em></b> is the test case name.
An additional element after the test name and before the
colon you may add a <b>\@<em>platform</em></b> to have the
test disabled only on a specific platform. Basic OS names as
reported by <b>$^O</b> in Perl, or <b>'windows'</b> are
supported. Alternatively, <b>@!<em>platform</i></b> will
disable the test on all except the named platform.
<b>BUG#<em>nnnnn</em></b> or <b>WL#<em>nnnn</em></b> indicates
the bug or the WL related to the test that causes it to fail
(and thus requires it to be disabled).
The <b><em>comment</em></b> text following the
<b>BUG#<em>nnnnn</em></b> or the <b>WL#<em>nnnn</em></b> is
not part of the mandatory syntax for this file,
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will not
actually care what is written here. Length of a
<b><em>comment</em></b> text must not be more than <b>80</b>
characters.
A comment line can be written in the file by beginning the
line with a “<b>#</b>” character.
</li>
</ul>
<li>
The <b>r</b> directory contains test case result files:
<ul>
<li>
A file name of the form <b><em>test_name</em>.result</b> is
the expected result for the named test case. A file
<b>r/<em>test_name</em>.result</b> is the output that
corresponds to the input in the test case file
<b>t/<em>test_name</em>.test</b>.
</li>
<li>
A file name of the form <b><em>test_name</em>.reject</b>
contains output for the named test case if the test fails
due to output mismatch (but not it it fails for other
reasons).
</li>
</ul>
For a test case that succeeds, the <b>.result</b> file
represents both the expected and actual result. For a test case
that fails, the <b>.result</b> file represents the expected
result, and the <b>.reject</b> file represents the actual
result.
If a <b>.reject</b> file is created because a test fails, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" removes the file
later the next time the test succeeds.
When <b>`--check-testcases`</b> option is enabled, a test case
not having its corresponding <b>.result</b> file is marked as
failed by MTR. See @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl".
</li>
<li>
The <b>include</b> directory contains files that are included by
test case files using the <b>source</b> command. These include
files encapsulate operations of varying complexity into a single
file so that you can perform the operations in a single step.
See @ref PAGE_INCLUDE_FILES.
</li>
<li>
The <b>lib</b> directory contains library files used by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl", and database
initialization SQL code.
</li>
<li>
The <b>std_data</b> directory contains data files used by some
of the tests.
</li>
<li>
The <b>var</b> directory is used during test runs for various
kinds of files: log files, temporary files, trace files, Unix
socket files for the servers started during the tests, and so
forth. This directory cannot be shared by simultaneous test
runs.
</li>
<li>
The <b>suite</b> directory contains a set of subdirectories
containing named test suites. Each subdirectory represents a
test suite with the same name.
A test suite directory contains the following components:
<ul>
<li>
The <b>t</b> directory contains test cases.
</li>
<li>
The <b>r</b> directory contains test case result files.
</li>
<li>
The <b>include</b> directory contains files that are
included by the test cases belonging to that suite.
</li>
<li>
A file name of the form <b>combinations</b> provides section
of options for each test run. See @ref
PAGE_CONTROLLING_BINARY_LOG_FORMAT.
</li>
<li>
A file name of the form <b><em>my</em>.cnf</b> contains
additional entries for the config file used by all the tests
within the suite it belongs to.
A suite specific <b><em>my</em>.cnf</b> file is overridden
by a <b><em>test_name</em>.cnf</b> file.
</li>
</ul>
</li>
<li>
The <b>collections</b> directory contains collections of test
runs that are run during integration and release testing of
MySQL. They are not directly useful outside this context, but
need to be part of the source repository and are included for
reference.
</li>
</ul>
Unit test-related files are located in the <b>unittest</b>
directory. Additional files specific to storage engines and plugins
may be present under the subdirectories of the <b>storage</b> or
<b>plugin</b> directories.
Test Execution and Evaluation
-----------------------------
There are a number of targets in the top-level <b>Makefile</b> that
can be used to run sets of tests. <b>make test</b> only runs unit
tests. Other targets run subsets of the tests, or run tests with
specific options for the test programs. Have a look at the
<b>Makefile</b> to see what targets are available.
A “test case” is a single file. The case might contain multiple
individual test commands. If any individual command fails, the
entire test case is considered to fail. Note that “fail” means “does
not produce the expected result.” It does <b><em>not</em></b>
necessarily mean “executes without error,” because some tests are
written precisely to verify that an illegal statement does in fact
produce an error. In such an instance, if the statement executes
successfully without producing the expected error, that is
considered failure of the test.
Test case output (the test result) consists of:
<ul>
<li>
Input SQL statements and their output. Each statement is written
to the result followed by its output. Columns in output
resulting from SQL statements are separated by tab characters.
</li>
<li>
The result from @ref PAGE_MYSQLTEST "mysqltest" commands such as
<b>echo</b> and <b>exec</b>. The commands themselves are not
echoed to the result, only their output.
</li>
</ul>
The <b>disable_query_log</b> and <b>enable_query_log</b> commands
control logging of input SQL statements. The
<b>disable_result_log</b> and <b>enable_result_log</b> commands
control logging of SQL statement results, and warning or error
messages resulting from those statements.
@ref PAGE_MYSQLTEST "mysqltest" reads a test case file from its
standard input by default. The <b>`--test-file`</b> or <b>-x</b>
option can be given to name a test case file explicitly.
@ref PAGE_MYSQLTEST "mysqltest" writes test case output to the
standard output by default. The <b>`--result-file`</b> or <b>-R</b>
option can be used to indicate the location of the result file. That
option, together with the <b>`--record`</b> option, determine how
@ref PAGE_MYSQLTEST "mysqltest" treats the test actual and expected
results for a test case:
<ul>
<li>
If the test produces no results, @ref PAGE_MYSQLTEST "mysqltest"
exits with an error message to that effect, unless
<b>`--result-file`</b> is given and this file is empty.
</li>
<li>
Otherwise, if <b>`--result-file`</b> is not given, @ref
PAGE_MYSQLTEST "mysqltest" sends test results to the standard
output.
</li>
<li>
With <b>`--result-file`</b> but not <b>`--record`</b>, @ref
PAGE_MYSQLTEST "mysqltest" reads the expected results from the
given file and compares them with the actual results. If the
results do not match, @ref PAGE_MYSQLTEST "mysqltest" writes a
<b>.reject</b> file in the log directory and exits with an
error. It will also print out a diff of the expected and actual
result, provided it can find an appropriate
<b>diff</b> tool to use.
</li>
<li>
With both <b>`--result-file`</b> and <b>`--record`</b>, @ref
PAGE_MYSQLTEST "mysqltest" updates the given file by writing the
actual test results to it. The file does not need to pre-exist.
</li>
</ul>
@ref PAGE_MYSQLTEST "mysqltest" itself knows nothing of the <b>t</b>
and <b>r</b> directories under the <b>mysql-test</b> directory. The
use of files in those directories is a convention that is used by
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl", which invokes @ref
PAGE_MYSQLTEST "mysqltest" with the appropriate options for each
test case to tell @ref PAGE_MYSQLTEST "mysqltest" where to read input
and write output.
*/
###############################################################################
## Test Framework System Requirements
/**
@page PAGE_TEST_FRAMEWORK_SYSTEM_REQUIREMENTS Test Framework System Requirements
The @ref PAGE_MYSQLTEST "mysqltest" and @ref PAGE_MYSQL_CLIENT_TEST
"mysql_client_test" programs are written in C++ and are available on
any system where MySQL itself can be compiled, or for which a binary
MySQL distribution is available.
Other parts of the test framework such as @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" are Perl scripts and
should run on systems with Perl installed.
@ref PAGE_MYSQLTEST "mysqltest" uses the <b>diff</b> program to
compare expected and actual test results. If <b>diff</b> is not
found, @ref PAGE_MYSQLTEST "mysqltest" writes an error message and
dumps the entire contents of the <b>.result</b> and
<b>.reject</b> files so that you can try to determine why a test
did not succeed. If your system does not have <b>diff</b>, you may
be able to obtain it from one of these sites:
- http://www.gnu.org/software/diffutils/diffutils.html
- http://gnuwin32.sourceforge.net/packages/diffutils.htm
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" cannot function
properly if started from within a directory whose full path includes
a space character, due to the complexities of handling this
correctly in all the different contexts it will be used.
*/
###############################################################################
## How to Report Bugs from Tests in the MySQL Test Suite
/**
@page PAGE_REPORT_BUGS How to Report Bugs from Tests in the MySQL Test Suite
If test cases from the test suite fail, you should do the following:
<ul>
<li>
Do not file a bug report before you have found out as much as
possible about what when wrong. See the instructions at
https://dev.mysql.com/doc/refman/8.0/en/bug-reports.html.
</li>
<li>
Make sure to include the output of @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl", as well as contents of all <b>.reject</b>
files in the <b>mysql-test/var/log</b> directory, or the (often
much shorter) diff that @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" reported.
</li>
<li>
Check whether an individual test in the test suite also fails
when run on its own:
@verbatim
shell> cd mysql-test
shell> ./mysql-test-run.pl test_name
@endverbatim
If this fails, and you are compiling MySQL yourself, you should
configure MySQL with <b>`--with-debug`</b> and run @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" with the
<b>`--debug`</b> option. If this also fails, open a bug report
and upload the trace file <b>mysql-test/var/tmp/master.trace</b>
to the report, so that we can examine it. For instructions, see
[How to Report Bugs or Problems]
(https://dev.mysql.com/doc/refman/8.0/en/bug-reports.html).
Please remember to also include a full description of your
system, the version of the <b>mysqld</b> binary and how you
compiled it.
</li>
<li>
Run @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" with the
<b>`--force`</b> option to see whether any other tests fail.
</li>
<li>
If you have compiled MySQL yourself, check the MySQL Reference
Manual to see whether there are any platform-specific issues for
your system. There might be configuration workarounds to deal
with the problems that you observe. Also, consider using one of
the binaries we have compiled for you at
https://dev.mysql.com/downloads/. All our standard binaries
should pass the test suite!
</li>
<li>
If you get an error such as <b>Result length mismatch</b> or
<b>Result content mismatch</b> it means that the output of the
test was not an exact match for the expected output. This could
be a bug in MySQL or it could be that your version of
<b>mysqld</b> produces slightly different results under some
circumstances.
This output from @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
should include a diff of the expected and actual result. If
unable to produce a diff, it will instead print out both in
full. In addition, the actual result is put in a file in the
<b>r</b> directory with a <b>.result</b> extension.
</li>
<li>
If a test fails completely, you should check the logs file in
the <b>mysql-test/var/log</b> directory for hints of what went
wrong.
</li>
<li>
If you have compiled MySQL with debugging, you can try to debug
test failures by running @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" with either or both of the <b>`--gdb`</b>
and <b>`--debug`</b> options.
If you have not compiled MySQL for debugging you should probably
do so by specifying the [-DWITH_DEBUG]
(https://dev.mysql.com/doc/refman/8.0/en/source-configuration-options.html#option_cmake_with_debug)
option when you invoke <b>CMake</b>.
</li>
</ul>
*/
###############################################################################
## Running Test Cases
/**
@page PAGE_RUNNING_TESTCASES Running Test Cases
Typically, you run the test suite either from within a source tree
(after MySQL has been built), or on a host where the MySQL server
distribution has been installed. (If you find that the test suite is
not included in your distribution, look for a similar distribution
with <b>-test</b> in the name and install that as well.)
To run tests, your current working directory should be the
<b>mysql-test</b> directory of your source tree or installed
distribution. In a source distribution, <b> mysql-test</b> is
under the root of the source tree. In a binary distribution, the
location of <b>mysql-test</b> depends on the distribution layout.
The program that runs the test suite, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl", will figure out whether you are in a source
tree or an installed directory tree.
To run the test suite, change location into your <b>mysql-test</b>
directory and invoke the @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" script:
@verbatim
shell> cd mysql-test
shell> ./mysql-test-run.pl
@endverbatim
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" accepts options on
the command line. For example:
@verbatim
shell> ./mysql-test-run.pl --force --suite=binlog
@endverbatim
By default, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" exits if
a test case fails. <b>`--force`</b> causes execution to continue
regardless of test case failure.
For a full list of the supported options, see @ref
PAGE_MYSQL_TEST_RUN_PL.
To run one or more specific test cases, name them on the @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" command line. Test case
files have names like <b>t/<em>test_name</em>.test</b>, where
<b><em>test_name</em></b> is the name of the test case, but each name
given on the command line should be the test case name, not the full
test case file name. The following command runs the test case named
<b>rpl_abcd</b>, which has a test file of
<b>t/rpl_abcd.test</b>:
@verbatim
shell> ./mysql-test-run.pl rpl_abcd
@endverbatim
To run a family of test cases for which the names share a common
prefix, use the <b>`--do-test`</b> option:
@verbatim
shell> ./mysql-test-run.pl --do-test=prefix
@endverbatim
For example, the following command runs the events tests (test
cases that have names beginning with <b>events</b>):
@verbatim
shell> ./mysql-test-run.pl --do-test=events
@endverbatim
To run a specific named testsuite with all the test cases in it,
use the <b>`--suite`</b> option:
@verbatim
shell> ./mysql-test-run.pl --suite=suite_name
@endverbatim
For example, the following command runs the replication tests
located in the <b>rpl</b> suite:
@verbatim
shell> ./mysql-test-run.pl --suite=rpl
@endverbatim
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" starts the MySQL
server, sets up the environment for calling the @ref PAGE_MYSQLTEST
"mysqltest" program, and invokes @ref PAGE_MYSQLTEST "mysqltest" to
run the test case. For each test case to be run, @ref PAGE_MYSQLTEST
"mysqltest" handles operations such as reading input from the test
case file, creating server connections, and sending SQL statements
to servers.
The language used in test case files is a mix of commands that the
@ref PAGE_MYSQLTEST "mysqltest" program understands and SQL
statements. Input that @ref PAGE_MYSQLTEST "mysqltest" doesn't
understand is assumed to consist of SQL statements to be sent to the
database server. This makes the test case language familiar to those
that know how to write SQL and powerful enough to add the control
needed to write test cases.
You need not start a MySQL server first before running tests.
Instead, the @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" program
will start the server or servers as needed. Any servers started for
the test run use ports in the range from 13000 by default.
@section SECTION_RUNNING_TESTS_IN_PARALLEL Running Tests in Parallel
It is possible to run more than one instance of @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" simultaneously on the
same machine. Both will by default use server ports from 13000 but
will coordinate used port numbers as well as check for availibility,
to avoid conflicts.
Running several instances from the same <b>mysql-test</b>
directory is possible but problematic. You must the use the
<b>`--vardir`</b> to set different log directories for each
instance. Even so, you can get into trouble becuse they will write
<b>.reject</b> files to the same directories.
It is also possible to have a single @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" run tests in several threads in parallel.
Execution of the tests will be distributed among the threads. This
is achieved using the <b>`--parallel`</b> option, with the number
of threads as argument. The special value <b>auto</b> will ask
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" to pick a value
automatically, based on system information. The parallel option may
also be given using the environment variable <b>MTR_PARALLEL</b>.
@note
Test cases which use <b>not_parallel.inc</b> are run at the end with
a parallel value of <b>1</b> overriding the <b>`--parallel`</b>
option value. Such test cases will be executed at the end of the test
run one at a time. These tests also require
<b>`--non-parallel-test`</b> option to be ON.
*/
###############################################################################
## Writing Test Cases
/**
@page PAGE_WRITING_TESTCASES Writing Test Cases
<h3>Table of Contents</h3>
- @subpage PAGE_QUICK_START
- @subpage PAGE_TESTCASE_CODING_GUIDELINES
- @subpage PAGE_SAMPLE_TESTCASES
- @subpage PAGE_CLEANUP_PREVIOUS_TEST_RUN
- @subpage PAGE_GENERATE_TESTCASE_RESULT_FILE
- @subpage PAGE_CHECK_EXPECTED_ERRORS
- @subpage PAGE_CONTROL_INFORMATION_PRODUCED_TESTCASE
- @subpage PAGE_DEALING_OUTPUT
- @subpage PAGE_PASSING_OPTIONS
- @subpage PAGE_TESTCASE_SPECIFIC_SERVER_OPTIONS
- @subpage PAGE_TESTCASE_SPECIFIC_BOOTSTRAP_OPTIONS
- @subpage PAGE_TESTCASE_SPECIFIC_CLIENT_OPTIONS
- @subpage PAGE_INCLUDE_FILES
- @subpage PAGE_BINARY_LOG_FORMAT
- @subpage PAGE_WRITING_REPLICATION_TESTS
- @subpage PAGE_THREAD_SYNCHRONIZATION
- @subpage PAGE_SUPPRESSING_ERRORS_WARNINGS
- @subpage PAGE_STOPPING_AND_RESTARTING_SERVER_DURING_TEST
- @subpage PAGE_TIPS_WRITING_TESTCASES
Normally, you run the test suite during the development process to
ensure that your changes do not cause existing test cases to break.
You can also write new test cases or add tests to existing cases.
This happens when you fix a bug (so that the bug cannot reappear
later without being detected) or when you add new capabilities to
the server or other MySQL programs.
This chapter provides guidelines for developing new test cases for
the MySQL test framework.
Some definitions:
- One “test file” is one “test case”.
- One “test case” might contain a “test sequence” (that is, a
number of individual tests that are grouped together in the same
test file).
- A “command” is an input test that @ref PAGE_MYSQLTEST "mysqltest"
recognizes and executes itself. A “statement” is an SQL statement
or query that @ref PAGE_MYSQLTEST "mysqltest" sends to the MySQL
server to be executed.
*/
###############################################################################
## Writing a Test Case: Quick Start
/**
@page PAGE_QUICK_START Writing a Test Case: Quick Start
The basic principle of test case evaluation is that output resulting
from running a test case is compared to the expected result. This is
just a <b>diff</b> comparison between the output and an expected-result
file that the test writer provides. This simplistic method of
comparison does not by itself provide any way to handle variation in
the output that may occur when a test is run at different times.
However, the test language provides commands for postprocessing
result output before the comparison occurs. This enables you to
manage certain forms of expected variation.
Use the following procedure to write a new test case. In the
examples, <b><em>test_name</em></b> represents the name of the test
case. It is assumed here that you'll be using a development source
tree, so that when you create a new test case, you can commit the
files associated with it to the source repository for others to use.
<ol>
<li>
Change location to the test directory
<b>mysql-<em>version</em>/mysql-test</b>:
@verbatim
shell> cd mysql-version/mysql-test
@endverbatim
<b>mysql-<em>version</em></b> represents the root directory of
your source tree.
</li>
<li>
Create the test case in a file
<b>t/<em>test_name</em>.test</b>. You can do this with
any text editor. For details of the language used for writing
@ref PAGE_MYSQLTEST "mysqltest" test cases, see @ref
PAGE_MYSQLTEST_LANGUAGE_REFERENCE.
</li>
<li>
Create an empty result file:
@verbatim
shell> touch r/test_name.result
@endverbatim
</li>
<li>
Run the test:
@verbatim
shell> ./mysql-test-run.pl test_name
@endverbatim
</li>
<li>
Assuming that the test case produces output, it should fail
because the output does not match the result file (which is
empty at this point). The failure results in creation of a
reject file named
<b>mysql-test/var/log/<em>test_name</em>.reject</b>.
Examine this file. If the reject file appears to contain the
output that you expect the test case to produce, copy its
content to the result file:
@verbatim
shell> cp mysql-test/var/log/test_name.reject r/test_name.result
@endverbatim
Another way to create the result file is by invoking @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" with the
<b>`--record`</b> option to record the test output in the
result file:
@verbatim
shell> ./mysql-test-run.pl --record test_name
@endverbatim
</li>
<li>
Run the test again. This time it should succeed:
@verbatim
shell> ./mysql-test-run.pl test_name
@endverbatim
You can also run the newly created test case as part of
the entire suite:
@verbatim
shell> ./mysql-test-run.pl
@endverbatim
</li>
</ol>
It is also possible to invoke the @ref PAGE_MYSQLTEST "mysqltest"
program directly. If the test case file refers to environment
variables, you will need to define those variables in your
environment first. For more information about the @ref
PAGE_MYSQLTEST "mysqltest" program, see @ref PAGE_MYSQLTEST.
*/
###############################################################################
## Test Case Coding Guidelines
/**
@page PAGE_TESTCASE_CODING_GUIDELINES Test Case Coding Guidelines
<h3>Table of Contents</h3>
- @subpage PAGE_NAMING_ORGANIZATION_GUIDELINES
- @subpage PAGE_CONTENT_FORMATTING_GUIDELINES
- @subpage PAGE_NAMING_CONVENTIONS
*/
###############################################################################
## File Naming and Organization Guidelines
/**
@page PAGE_NAMING_ORGANIZATION_GUIDELINES File Naming and Organization Guidelines
Test case file names may use alphanumeric characters (<b>A-Z</b>,
<b>a-z</b>, <b>0-9</b>), underscore (<b>'_'</b>) or dash
(<b>'-'</b>), but should not start with underscore or dash. No other
special characters are allowed, if used, then an error is thrown and
the test run is aborted.
Test names have traditionally used lowercase only, and we recommend
continuing this for tests added to the common repository, though
uppercase letters are also supported.
Test cases are located in the <b>mysql-test/t</b> directory. Test
case file names consist of the test name with a <b>.test</b>
suffix. For example, a test named <b>foo</b> should be written in
the file <b>mysql-test/t/foo.test</b>.
In addition to this directory, tests are organized in test suites,
located in subdirectories under the <b>suite</b> directory. For
example, a test named <b>bar</b> under the replication suite
<b>rpl</b> may be stored in the file <b>mysql-
test/suite/rpl/t/bar.test</b>.
In practice, the file would likely be called <b>rpl_bar.test</b>
as tests in a suite usually also have the suite name as a prefix.
This is just a convention from the time when suites were not
supported, and not a requirement for test naming.
One test case file can be a collection of individual tests that
belong together. If one of the tests fails, the entire test case
fails. Although it may be tempting to write each small test into a
single file, that will be too inefficient and makes test runs
unbearably slow. So make the test case files not too big, not too
small.
Each test case (that is, each test file) must be self contained and
independent of other test cases. Do not create or populate a table
in one test case and depend on the table in a later test case. If
you have some common initialization that needs to be done for
multiple test cases, create an include file. That is, create a file
containing the initialization code in the <b>mysql-test/include</b>
directory, and then put a <b>source</b> command in each test case
that requires the code. For example, if several test cases need to
have a given table created and filled with data, put the statements
to do that in a file named <b>mysql-test/include/create_my_table.inc</b>.
The <b>.inc</b> is a convention, not a requirement. Then put the
following command in each test case file that needs the
initialization code:
@verbatim
--source include/create_my_table.inc
@endverbatim
The file name in the <b>source</b> command is relative to the
<b>mysql-test</b> directory. Remember to drop the table at the
end of each test that creates it.
*/
###############################################################################
## Test Case Content-Formatting Guidelines
/**
@page PAGE_CONTENT_FORMATTING_GUIDELINES Test Case Content-Formatting Guidelines
When you write a test case, please keep in mind the following
general guidelines.
There are C/C++ coding guidelines in the MySQL Internals manual;
please apply them when it makes sense to do so: @ref
PAGE_CODING_GUIDELINES.
Other guidelines may be found in this page, which discusses general
principles of test-case writing: MySQL Internals:
[How to Create Good Test Cases]
(https://dev.mysql.com/doc/internals/en/good-tests.html)
The following guidelines are particularly applicable to writing test
cases:
<ul>
<li>
To write a test case file, use any text editor that uses
linefeed (newline) as the end-of-line character.
</li>
<li>
Avoid lines longer than 80 characters unless there is no
other choice.
</li>
<li>
A comment in a test case is started with the "#" character.
@ref PAGE_MYSQL_TEST_INPUT_CONVENTIONS discusses the details
of input syntax for mysqltest test cases.
</li>
<li>
Use spaces, not tabs.
</li>
<li>
Write SQL statements using the same style as our manual:
<ul>
<li>
Use uppercase for SQL keywords.
</li>
<li>
Use lowercase for identifiers (names of objects such as
databases, tables, columns, and so forth).
</li>
</ul>
Ignore this guidline if your intent is to test lettercase
processing for SQL statements, of course.
Use lowercase for @ref PAGE_MYSQLTEST "mysqltest" commands
(<b>echo</b>, <b>sleep</b>, <b>let,</b> and so forth).
You may notice that many existing test cases currently do not
follow the lettercase guideline and contain SQL statements
written entirely in lowercase. Nevertheless, <b><em>please use
the guideline for new tests</em></b>. Lettercase for older tests
can be left as is, unless perhaps you need to revise them
significantly.
</li>
<li>
Break a long SQL statement into multiple lines to make it more
readable. This means that you will need to write it using a
“<b>;</b>” delimiter at the end of the statement rather than
using “<b>`--`</b>” at the beginning because the latter style
works only for single-line statements.
</li>
<li>
Include comments. They save the time of others. In particular:
<ul>
<li>
Please include a header in test files that indicates the
purpose of the test and references the corresponding worklog
task, if any.
</li>
<li>
Comments for a test that is related to a bug report should
include the bug number and title.
</li>
</ul>
Worklog and bug numbers are useful because they enable people
who are interested in additional background related to the test
case to know which worklog entries or bug reports to read.
</li>
</ul>
Example SQL statement, formatted onto multiple lines for readability:
@verbatim
SELECT f1 AS "my_column", f10 ....
FROM mysqltest1.t5
WHERE (f2 BETWEEN 17 AND 25 OR f2 = 61)
AND f3 IN (SELECT ....
FROM mysqltest1.t4
WHERE .....)
ORDER BY ... ;
@endverbatim
Example test file header:
@verbatim
########### suite/funcs_1/t/a_processlist_val_no_prot.test #############
# #
# Testing of values within INFORMATION_SCHEMA.PROCESSLIST #
# #
# The prepared statement variant of this test is #
# suite/funcs_1/t/b_processlist_val_ps.test. #
# #
# There is important documentation within #
# suite/funcs_1/datadict/processlist_val.inc #
# #
# Note(mleich): #
# The name "a_process..." with the unusual prefix "a_" is #
# caused by the fact that this test should run as second test, that #
# means direct after server startup and a_processlist_priv_no_prot. #
# Otherwise the connection IDs within the processlist would differ. #
# #
# Creation: #
# 2007-08-09 mleich Implement this test as part of #
# WL#3982 Test information_schema.processlist #
# #
########################################################################
@endverbatim
Example test reference to bug report:
@verbatim
# Bug #3671 Stored procedure crash if function has "set @variable=param"
@endverbatim
*/
###############################################################################
## Naming Conventions for Database Objects
/**
@page PAGE_NAMING_CONVENTIONS Naming Conventions for Database Objects
It is possible to run test cases against a production server. This
is very unlikely to happen by accident, as @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will start its own server
unless you use the <b>`--extern`</b> Even so, try to write test
cases in a way that reduces the risk that running tests will alter
or destroy important tables, views, or other objects. (<b>DROP
DATABASE</b> statements are particularly dangerous if written using
names that could exist on a customer's machine.) To avoid such
problems, we recommend the following naming conventions:
<ul>
<li>
User names: User names should begin with “mysql” (for example,
<b>mysqluser1</b>, <b>mysqluser2</b>)
</li>
<li>
Database names: Unless you have a special reason not to, use
the default database named <b>test</b> that is already created
for you. For tests that need to operate outside the <b>test</b>
database, database names should contain “test” or begin with
“mysql” (for example, <b>mysqltest1</b>, <b>mysqltest2</b>)
</li>
<li>
Table names: <b>t1</b>, <b>t2</b>, <b>t3</b>, ...
</li>
<li>
View names: <b>v1</b>, <b>v2</b>, <b>v3</b>, ...
</li>
</ul>
For examples of how to name objects, examine the existing test
cases. Of course, you can name columns and other objects inside
tables as you wish.
*/
###############################################################################
## Sample Test Case
/**
@page PAGE_SAMPLE_TESTCASES Sample Test Case
Here is a small sample test case:
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1;
SET @@sql_mode='NO_ENGINE_SUBSTITUTION';
--enable_warnings
SET SQL_WARNINGS=1;
CREATE TABLE t1 (a INT);
INSERT INTO t1 VALUES (1);
INSERT INTO t1 VALUES ("hej");
DROP TABLE t1;
@endverbatim
The first few lines try to clean up from possible earlier runs of
the test case by dropping the <b>t1</b> table. The test case uses
<b>disable_warnings</b> to prevent warnings from being written to
the output because it is not of any interest at this point during
the test to know whether the table <b>t1</b> was there. The value of
<b>sql_mode</b> is changed to <b>NO_ENGINE_SUBSTITUTION</b> to avoid
the error thrown by the second insert statement. After this, the
test case uses <b>enable_warnings</b> so that subsequent warnings
will be written to the output. The test case also enables verbose
warnings in MySQL using the <b>SET SQL_WARNINGS=1;</b> statement.
Next, the test case creates the table <b>t1</b> and tries some
operations. Creating the table and inserting the first row are
operations that should not generate any warnings. The second insert
should generate a warning because it inserts a nonnumeric string
into a numeric column. At the end, it drops the table <b>t1</b>. The
output that results from running the test looks like this:
@verbatim
DROP TABLE IF EXISTS t1;
SET @@sql_mode='NO_ENGINE_SUBSTITUTION';
SET SQL_WARNINGS=1;
CREATE TABLE t1 (a INT);
INSERT INTO t1 VALUES (1);
INSERT INTO t1 VALUES ("hej");
Warnings:
Warning 1366 Incorrect integer value: 'hej' for column 'a' at row 1
DROP TABLE t1;
@endverbatim
Note that the result includes not only the output from SQL
statements, but the statements themselves. Statement logging can be
disabled with the <b>disable_query_log</b> test language command.
There are several options for controlling the amount of output from
running the tests.
If there was a test failure, it will be reported to the screen. You
can see the actual output from the last unsuccessful run of the test
case in the reject file
<b>mysql-test/var/log/<em>test_name</em>.reject</b>.
*/
###############################################################################
## Cleaning Up from a Previous Test Run
/**
@page PAGE_CLEANUP_PREVIOUS_TEST_RUN Cleaning Up from a Previous Test Run
For efficiency, the @ref PAGE_MYSQLTEST "mysqltest" test engine does
not start with a clean new database for running each test case, so a
test case generally starts with a “cleaning up section”. Assume that
a test case will use two tables named <b>t1</b> and <b>t2</b>.
The test case should begin by making sure that any old tables with
those names do not exist:
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1,t2;
--enable_warnings
@endverbatim
The <b>disable_warnings</b> command instructs the test engine not
to log any warnings until an <b>enable_warnings</b> command occurs
or the test case is ended. (MySQL generates a warning if the table
<b>t1</b> or <b>t2</b> does not exist.) Surrounding this part of
the test case with commands to disable and enable warnings makes its
output the same regardless of whether the tables exist before the
test is started. After ensuring that the tables do not exist, we are
free to put in any SQL statements that create and use the tables
<b>t1</b> and <b>t2</b>. The test case should also clean up at
the end of the test by dropping any tables that it creates.
Let's put in some SQL code into this test case:
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1,t2;
--enable_warnings
CREATE TABLE t1 (
Period SMALLINT(4) UNSIGNED ZEROFILL DEFAULT '0000' NOT NULL,
Varor_period SMALLINT(4) UNSIGNED DEFAULT '0' NOT NULL
);
CREATE TABLE t2 (Period SMALLINT);
INSERT INTO t1 VALUES (9410,9412);
INSERT INTO t2 VALUES (9410),(9411),(9412),(9413);
SELECT PERIOD FROM t1;
SELECT * FROM t1;
SELECT t1.* FROM t1;
SELECT * FROM t1 INNER JOIN t2 USING (Period);
DROP TABLE t1, t2;
@endverbatim
If a test case creates other objects such as stored programs or
user accounts, it should take care to also clean those up at the
beginning and end of the test. Temporary files should also be
removed, either at the end or just after they have been used.
*/
###############################################################################
## Generating a Test Case Result File
/**
@page PAGE_GENERATE_TESTCASE_RESULT_FILE Generating a Test Case Result File
The test code we just wrote contains no checks of the result. The
test will report a failure for one of two reasons:
- An individual SQL statement fails with an error
- The overall test case result does not match what was expected
Note that these are the reasons why @ref PAGE_MYSQLTEST "mysqltest"
would fail; if the test is run from @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" the test may fail for additional reasons.
In the first case, @ref PAGE_MYSQLTEST "mysqltest" aborts with an
error. The second case requires that we have a record of the
expected result so that it can be compared with the actual result.
To generate a file that contains the test result, run the test with
the <b>`--record`</b> option, like this:
@verbatim
shell> cd mysql-test
shell> ./mysql-test-run.pl --record foo
@endverbatim
Running the test as shown creates a result file named
<b>mysql-test/r/foo.result</b> that has this content:
@verbatim
DROP TABLE IF EXISTS t1,t2;
CREATE TABLE t1 (
Period SMALLINT(4) UNSIGNED ZEROFILL DEFAULT '0000' NOT NULL,
Varor_period SMALLINT(4) UNSIGNED DEFAULT '0' NOT NULL
);
CREATE TABLE t2 (Period SMALLINT);
INSERT INTO t1 VALUES (9410,9412);
INSERT INTO t2 VALUES (9410),(9411),(9412),(9413);
SELECT period FROM t1;
period
9410
SELECT * FROM t1;
Period Varor_period
9410 9412
SELECT t1.* FROM t1;
Period Varor_period
9410 9412
SELECT * FROM t1 INNER JOIN t2 USING (Period);
Period Varor_period
9410 9412
DROP TABLE t1, t2;
ok
@endverbatim
If we look at this result file, it contains the statements in the
<b>foo.test</b> file together with the output from the
<b>SELECT</b> statements. The output for each statement includes a
row of column headings followed by data rows. Rows have columns
separated by Tab characters.
At this point, you should inspect the result file and determine
whether its contents are as expected. If so, let it be part of your
test case. If the result is not as expected, you have found a
problem, either with the server or the test. Determine the cause of
the problem and fix it. For example, the test might produce output
that varies from run to run. To deal with this, you can postprocess
the output before the comparison occurs. See @ref
PAGE_DEALING_OUTPUT.
*/
###############################################################################
## Checking for Expected Errors
/**
@page PAGE_CHECK_EXPECTED_ERRORS Checking for Expected Errors
A good test suite checks not only that operations succeed as they
ought, but also that they fail as they ought. For example, if a
statement is illegal, the server should reject it with an error
message. The test suite should verify that the statement fails and
that it fails with the proper error message.
The test engine enables you to specify “expected failures.” Let's
say that after we create <b>t1</b>, we try to create it again
without dropping it first:
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1,t2;
--enable_warnings
CREATE TABLE t1 (
Period SMALLINT(4) UNSIGNED ZEROFILL DEFAULT '0000' NOT NULL,
Varor_period SMALLINT(4) UNSIGNED DEFAULT '0' NOT NULL
);
CREATE TABLE t2 (Period SMALLINT);
INSERT INTO t1 VALUES (9410,9412);
INSERT INTO t2 VALUES (9410),(9411),(9412),(9413);
SELECT period FROM t1;
SELECT * FROM t1;
SELECT t1.* FROM t1;
SELECT * FROM t1 INNER JOIN t2 USING (Period);
CREATE TABLE t1 (something SMALLINT(4));
@endverbatim
The result is failure and an error:
@verbatim
At line 21: query 'CREATE TABLE t1 (something SMALLINT(4))'
failed: 1050: Table 't1' already exists
@endverbatim
To handle this error and indicate that indeed we do expect it to
occur, we can put an <b>error</b> command before the second
<b>create table</b> statement. Either of the following commands
test for this particular MySQL error:
@verbatim
--error 1050
--error ER_TABLE_EXISTS_ERROR
@endverbatim
<b>1050</b> is the numeric error code and
<b>ER_TABLE_EXISTS_ERROR</b> is the symbolic name. Symbolic names
are more stable than error numbers because the numbers sometimes
change, particularly for those created during recent development.
For such errors, use of numbers rather than the names in a test case
will require test to be revised should the numbers change.
After we make a change to add an <b>error</b> command before the
<b>CREATE TABLE</b> statement and run the test again, the end of
the result will look like this:
@verbatim
CREATE TABLE t1 (something SMALLINT(4));
ERROR 42S01: Table 't1' already exists
@endverbatim
In this case, the result shows the statement that causes the error,
together with the resulting error message. The fact that @ref
PAGE_MYSQLTEST "mysqltest" does not terminate and that the error
message becomes part of the result indicates that the error was
expected.
You can also test for errors by specifying an SQLSTATE value. For
MySQL error number <b>1050</b>, the corresponding SQLSTATE value
is <b>42S01</b>. To specify an SQLSTATE value in an <b>error</b>
command, use an <b>S</b> prefix:
@verbatim
--error S42S01
@endverbatim
A disadvantage of SQLSTATE values is that sometimes they correspond
to more than one MySQL error code. Using the SQLSTATE value in this
case might not be specific enough (it could let through an error
that you do not actually expect).
If you want to test for multiple errors, the <b>error</b> command
allows multiple arguments, separated by commas. For example:
@verbatim
--error ER_NO_SUCH_TABLE,ER_KEY_NOT_FOUND
@endverbatim
For a list of MySQL error codes, symbolic names, and SQLSTATE values,
see https://dev.mysql.com/doc/refman/8.0/en/error-messages-server.html.
You can also examine the <b>mysqld_error.h</b> and
<b>sql_state.h</b> files in the <b>include</b> directory of a MySQL
source distribution.
It is also possible to use symbolic error names to refer to client
errors:
@verbatim
--error CR_SERVER_GONE_ERROR
@endverbatim
For a list of MySQL client error codes, see
https://dev.mysql.com/doc/refman/8.0/en/error-messages-client.html.
You can also examine the <b>errmsg.h</b> file in the <b>include</b>
directory of a MySQL source distribution.
The built-in variable <b>$mysql_errno</b> contains the numeric
error returned by the most recent SQL statement sent to the server,
or 0 if the statement executed successfully. This may be useful
after statements that may or may not fail, or fail in more than one
way (more than one argument to the <b>error</b> command), in case
you need to perform different actions. Note that this applies to SQL
statements, not to other commands.
There is also a variable <b>$mysql_errname</b> which contains the
symbolic name of the last error. In some cases the symbolic name is
not available; in those cases the variable will contain the string
<b>"<Unknown>"</b>. For new test development we recommend testing
against the name rather than the number, since the number may change
in future versions. If the last statement succeeded
(<b>$mysql_errno</b> is 0), this variable is an empty string.
*/
###############################################################################
## Controlling the Information Produced by a Test Case
/**
@page PAGE_CONTROL_INFORMATION_PRODUCED_TESTCASE Controlling the Information Produced by a Test Case
By default, the @ref PAGE_MYSQLTEST "mysqltest" test engine produces
output only from <b>select</b>, <b>show</b>, and other SQL
statements that you expect to produce output (that is, statements
that create a result set). It also produces output from certain
commands such as <b>echo</b> and <b>exec</b>. @ref
PAGE_MYSQLTEST "mysqltest" can be instructed to be more or less
verbose.
Suppose that we want to include in the result the number of rows
affected by or returned by SQL statements. To do this, add the
following line to the test case file preceding the first
table-creation statement:
@verbatim
--enable_info
@endverbatim
After rerunning the test by invoking @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" with the <b>`--record`</b> option to record
the new result, the result file will contain more information:
@verbatim
DROP TABLE IF EXISTS t1,t2;
CREATE TABLE t1 (
Period SMALLINT(4) UNSIGNED ZEROFILL DEFAULT '0000' NOT NULL,
Varor_period SMALLINT(4) UNSIGNED DEFAULT '0' NOT NULL
);
affected rows: 0
CREATE TABLE t2 (Period SMALLINT);
affected rows: 0
INSERT INTO t1 VALUES (9410,9412);
affected rows: 1
INSERT INTO t2 VALUES (9410),(9411),(9412),(9413);
affected rows: 4
info: Records: 4 Duplicates: 0 Warnings: 0
SELECT period FROM t1;
period
9410
affected rows: 1
SELECT * FROM t1;
Period Varor_period
9410 9412
affected rows: 1
SELECT t1.* FROM t1;
Period Varor_period
9410 9412
affected rows: 1
SELECT * FROM t1 INNER JOIN t2 USING (Period);
Period Varor_period
9410 9412
affected rows: 1
DROP TABLE t1, t2;
affected rows: 0
ok
@endverbatim
To turn off the affected-rows reporting, add this command to the
test case file:
@verbatim
--disable_info
@endverbatim
In general, options can be enabled and disabled for different parts
of the test file. Suppose that we are interested in the internals of
the database as well. We could enable the display of query metadata
using <b>enable_metadata</b>. With this option enabled, the test
output is a bit verbose. However, as mentioned earlier, the option
can be enabled and disabled selectively so that it is enabled only
for those parts of the test case where it interests you to know
more.
If you perform an operation for which you have no interest in seeing
the statements logged to the result, you can disable statement
logging. For example, you might be initializing a table where you
don't really expect a failure, and you are not interested in seeing
the initialization statements in the test result. You can use the
<b>disable_query_log</b> command to temporarily disable recording
of input SQL statements, and enable recording again with
<b>enable_query_log</b>. You can disable the recording of the
output from executing commands using <b>disable_result_log</b> and
enable recording again with <b>enable_result_log</b>.
*/
###############################################################################
## Dealing with Output That Varies Per Test Run
/**
@page PAGE_DEALING_OUTPUT Dealing with Output That Varies Per Test Run
It is best to write each test case so that the result it produces
does not vary for each test run, or according to factors such as the
time of day, differences in how program binaries are compiled, the
operating system, and so forth. For example, if the result contains
the current date and time, the test engine has no way to verify that
the result is correct.
However, sometimes a test result is inherently variable according to
external factors, or perhaps there is a part of a result that you
simply do not care about. @ref PAGE_MYSQLTEST "mysqltest" provides
commands that enable you to postprocess test output into a more
standard format so that output variation across test runs will not
trigger a result mismatch.
One such command is <b>replace_column</b>, which specifies that
you want to replace whatever is in a given column with a string.
This makes the output for that column the same for each test run.
To see how this command works, add the following row after the first
insert in the test case:
@verbatim
INSERT INTO t1 VALUES (DATE_FORMAT(NOW(), '%s'),9999);
@endverbatim
Then record the test result and run the test again:
@verbatim
shell> ./mysql-test-run.pl --record foo
shell> ./mysql-test-run.pl foo
@endverbatim
Most likely, a failure will occur and @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" will display the difference between the expected
result and what we actually got, like this (the header has been
simplified):
@verbatim
CURRENT_TEST: main.foo
--- r/foo.result 2009-11-17 16:22:38
+++ var/log/foo.reject 2009-11-17 16:22:47
@@ -10,15 +10,15 @@
SELECT period FROM t1;
period
9410
-0038
+0047
SELECT * FROM t1;
Period Varor_period
9410 9412
-0038 9999
+0047 9999
SELECT t1.* FROM t1;
Period Varor_period
9410 9412
-0038 9999
+0047 9999
SELECT * FROM t1 INNER JOIN t2 USING (Period);
Period Varor_period
9410 9412
mysqltest: Result content mismatch
@endverbatim
The actual numbers will likely be different for your case, and the
format of the diff may also vary.
If we are not really interested in the first column, one way to
eliminate this mismatch is by using the <b>replace_column</b>
command. The duration of the effect of this command is the next SQL
statement, so we need one before each <b>select</b> statement:
@verbatim
--replace_column 1 SECONDS
SELECT period FROM t1;
--replace_column 1 SECONDS
SELECT * FROM t1;
--replace_column 1 SECONDS
SELECT t1.* FROM t1;
@endverbatim
In the <b>replace_column</b> commands, <b>SECONDS</b> could be
any string. Its only purpose is to map variable output onto a
constant value. If we record the test result again, we will succeed
each time we run the test after that. The result file will look like
this:
@verbatim
DROP TABLE IF EXISTS t1,t2;
CREATE TABLE t1 (
Period SMALLINT(4) UNSIGNED ZEROFILL DEFAULT '0000' NOT NULL,
Varor_period SMALLINT(4) UNSIGNED DEFAULT '0' NOT NULL
);
affected rows: 0
CREATE TABLE t2 (Period SMALLINT);
affected rows: 0
INSERT INTO t1 VALUES (9410,9412);
affected rows: 1
INSERT INTO t1 VALUES (DATE_FORMAT(NOW(), '%s'),9999);
affected rows: 1
INSERT INTO t2 VALUES (9410),(9411),(9412),(9413);
affected rows: 4
info: Records: 4 Duplicates: 0 Warnings: 0
SELECT period FROM t1;
period
SECONDS
SECONDS
affected rows: 2
SELECT * FROM t1;
Period Varor_period
SECONDS 9412
SECONDS 9999
affected rows: 2
SELECT t1.* FROM t1;
Period Varor_period
SECONDS 9412
SECONDS 9999
affected rows: 2
SELECT * FROM t1 INNER JOIN t2 USING (Period);
Period Varor_period
9410 9412
affected rows: 1
DROP TABLE t1, t2;
affected rows: 0
ok
@endverbatim
*/
###############################################################################
## Passing Options from mysql-test-run.pl to mysqld or mysqltest
/**
@page PAGE_PASSING_OPTIONS Passing Options from mysql-test-run.pl to mysqld or mysqltest
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" supports several
options that enable you to pass options to other programs. Each of
these options takes a value consisting of one or more comma-separated
options:
<ul>
<li>
The <b>`--mysqld`</b> option tells @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" to start the <b>mysqld</b> server with the
named option added. More than one such extra option may be
provided. The following command causes <b>`--big-tables`</b>
and <b>`--key_buffer_size`=16384</b> to be passed to
<b>mysqld</b>:
@verbatim
shell> ./mysql-test-run.pl
--mysqld=--big-tables
--mysqld=--key_buffer_size=16384
@endverbatim
Note how <b>`--mysqld`</b> needs to be repeated for each
server option to add. It does not work to add several server
options with one <b>`--mysqld`</b> even if enclosed in
quotation marks, as that will be interpreted as a single server
option (including spaces).
</li>
<li>
The <b>`--combination`</b> option is similar to
<b>`--mysqld`</b>, but behaves differently. @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" executes multiple
test runs, using the options for each instance of
<b>`--combination`</b> in successive runs. The following
command passes <b>`--big-tables`</b> to <b>mysqld</b> for the
first test run, <b>`--innodb`</b> for the second run and
<b>`--innodb-file-per-table`</b> for the last run:
@verbatim
shell> ./mysql-test-run.pl
--combination=--big-tables
--combination=--innodb
--combination=--innodb-file-per-table
@endverbatim
If <b>`--combination`</b> is given only once, it behaves similar
to <b>`--mysqld`</b> option.
For test runs specific to a given test suite, an alternative to
the use of the <b>`--combination`</b> option is to create a
<b>combinations</b> file in the suite directory. The file should
contain a section of options for each test run. For an example,
see @ref PAGE_BINARY_LOG_FORMAT.
<b>combinations</b> file can support <b>bootstrap</b> variables
too. The <b>bootstrap</b> variable has to be passed as a value
to the <b>`initialize`</b> keyword.
For a test specific combinations, an alternative to the use of
the <b>`--combination`</b> option is to create a
<b><em>test_name</em>.combinations</b> file in the test
directory.
</li>
<li>
The <b>`--mysqltest`</b> option is used to pass extra options
to @ref PAGE_MYSQLTEST "mysqltest".
@verbatim
shell> ./mysql-test-run.pl --mysqltest=options
@endverbatim
For an example, see @ref PAGE_MYSQL_TEST_RUN_PL.
</li>
</ul>
*/
###############################################################################
## Specifying Test Case-Specific Server Options
/**
@page PAGE_TESTCASE_SPECIFIC_SERVER_OPTIONS Specifying Test Case-Specific Server Options
Within a test case, many system variables can be set by using
statements such as these:
@verbatim
SET sql_warnings= 1;
SET sql_mode= 'NO_AUTO_VALUE_ON_ZERO';
@endverbatim
But sometimes you need to restart the server to use command-line
options that are specific to a given test case. You can specify
these options in a file named
<b>mysql-test/t/<em>test_name-master</em>.opt</b>. When a
file named <b>t/<em>test_name-master</em>.opt</b> exists,
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" examines it for
extra options that the server needs to be run with when executing
the <em>test_name</em> test case. If no server has yet been
started or the current server is running with different options,
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" restarts the server
with the new options.
You may refer to environment variables in the option file, using the
usual <b><em>$VAR_NAME</em></b> syntax. It is also possible to refer
to optional variables using the syntax <b><em>$?VAR_NAME</em></b>.
This will be replaced with an empty string if the variable is not
set.
As a special case, the option <b>`--skip-core-file`</b> will be
interpreted by @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl",
which will then block the server from producing any core files or
crash dumps during this test. This may be useful for tests that
intentionally crash the server.
Another special case is <b>`--testcase-timeout`=minutes</b> which
can be used to set a different, longer timeout for a particular test
case. The given timeout in minutes will be used for this test if
it's longer than the default.
Files in the <b>mysql-test/t</b> directory with names ending in
<b>-slave.opt</b> are similar, but they are used for slave servers
in replication tests.
Sometimes it's also necessary to execute some external commands
before starting the server, such as creating a directory. If you add
a file named <b>t/<em>test_name-master</em>.sh</b>, it will
be executed by @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
before it starts the server; a similar file may be created for the
slave in replication tests.
Because the <b>.sh</b> file is executed through <b>/bin/sh</b>, it
cannot be used on Windows, and any tests using such a file will
automatically be skipped if you run on Windows. For this reason,
this mechanism may be replaced with a more portable one in some
future release of MySQL.
*/
###############################################################################
## Specifying Test Case-Specific Bootstrap Options
/**
@page PAGE_TESTCASE_SPECIFIC_BOOTSTRAP_OPTIONS Specifying Test Case-Specific Bootstrap Options
If a test has to run with a particular value of a bootstrap variable
such as <b>`--innodb-page-size`</b> or
<b>`--innodb-data-file-path`</b>, the option can be passed on the
command line while running @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl". For example, consider a test that can only run
with <b>`--innodb-page-size`=8k</b>. The test can be run like this:
@verbatim
shell> ./mysql-test-run.pl test_name_8k --initialize=--innodb-page-size=8k
@endverbatim
This will initialize the data directory with 8k page size, and start the
server with that value.
It is also possible to pass bootstrap options in the
<b>master.opt</b> file of the test, so that the test can run with
the specified value of the bootstrap options without using any
command line arguments. The usage is:
@verbatim
--initialize --innodb-page-size=8k
@endverbatim
or
@verbatim
--initialize=--innodb-page-size=8k
@endverbatim
Specifying bootstrap variables in the opt file is the preferred
method.
Each bootstrap variable must be specified as the value of a
<b>`--initialize`</b> option in the opt file to ensure @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" recognizes that the
variable must be used during server initialization. If there are
bootstrap options in the file, the current data directory is
deleted, and initialized again with the options set in the file. The
server is also started with the bootstrap options passed along with
the other options in the opt file.
Similarly, bootstrap options passed in the <b>slave.opt</b> will be
used to reinitialize the slave server in replication scenarios.
Bootstrap options set in the opt file are passed to the server
first, followed by the server options. So, server start up options
have precedence over bootstrap options, regardless of the order in
which they are set in the opt file.
Note that since the bootstrap options are passed in the opt files,
they have precedence over the command line bootstrap options. So,
if a test has a master.opt file containing <b>`--innodb-page-size`=8k</b>,
and while it is being run, <b>`--innodb-page-size`=4k</b> is passed on
the command line, the test will run with <b>8k</b> page size.
Some tests require the server to be restarted multiple times with
different server options, as well different bootstrap options. In
such cases, the existing data directory is deleted inside the test
and a new data directory is initialized with the bootstrap options.
When the server is started with the bootstrap options after this,
the SQL queries will run with the specified options.
@verbatim
--exec $MYSQLD --no-defaults --initialize-insecure
--lc_messages_dir=$MYSQL_SHAREDIR --innodb_page_size=8K
--basedir=$MYSQLD_BASEDIR --datadir=$MYSQL_TMP_DIR/datadir1/data
--init-file=$BOOTSTRAP_SQL --secure-file-priv=""
@endverbatim
*/
###############################################################################
## Specifying Test Case-Specific Client Options
/**
@page PAGE_TESTCASE_SPECIFIC_CLIENT_OPTIONS Specifying Test Case-Specific Client Options
If a test case needs a specific client option, you can specify such
options in a file named <b><em>testname</em>-client.opt</b>. When a
test case has its own <b>-client.opt</b> file, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" examines it for any
additional options that the @ref PAGE_MYSQLTEST "mysqltest" should
run with while executing the test case.
For example, consider a test case <b>example.test</b> needs
<tt>latin1</tt> as default character set for client and connection,
then that option can be specified in
<b><em>example</em>-client.opt</b> file.
@verbatim
--default-character-set=latin1
@endverbatim
You may refer to environment variables in the option file, using the
usual <b><em>$VAR_NAME</em></b> syntax. It is also possible to refer
to optional variables using the syntax <b><em>$?VAR_NAME</em></b>.
This will be replaced with an empty string if the variable is not
set.
*/
###############################################################################
## Using Include Files to Simplify Test Cases
/**
@page PAGE_INCLUDE_FILES Using Include Files to Simplify Test Cases
The <b>include</b> directory contains many files intended for
inclusion into test case files.
These include files serve many purposes, but in general, they
encapsulate operations of varying complexity into single files so
that you can perform each operation in a single step. Include files
are available for operations such as these:
<ul>
<li>
Ensure that a given feature is available. The file checks to
make sure that the feature is available and exits if not.
</li>
<ul>
<li>
Storage engine tests: These files have names of the form
<b>have_<em>engine_name.inc</em></b>, such as
<b>have_archive.inc</b> or <b>have_blackhole.inc</b>. The
<b>MyISAM</b>, <b>MERGE</b>, and <b>MEMORY</b> storage
engines are always supported and need not be checked.
</li>
<li>
Windows specific tests: Include <b>have_windows.inc</b> file
if a test runs only on windows machines.
</li>
<li>
Debugging capabilities: Include the <b>have_debug.inc</b>
file if a test requires that the server was built for
debugging (that is, that the MySQL distribution was configured
with the <b>`--with-debug`</b> option).
</li>
</ul>
<li>
Wait for a condition to become true. Set the
<b>$wait_condition</b> variable to an SQL statement that
selects a value and then include the <b>wait_condition.inc</b>
file. The include file executes the statement in a loop with a
0.1 second sleep between executions until the select value is
nonzero. For example:
@verbatim
--let $wait_condition= SELECT c = 3 FROM t
--source include/wait_condition.inc
@endverbatim
</li>
<li>
Control the binary log format. See @ref PAGE_BINARY_LOG_FORMAT.
</li>
<li>
Control replication slave servers. See @ref
PAGE_WRITING_REPLICATION_TESTS.
</li>
</ul>
You can think of an include file as a rudimentary form of subroutine
that is “called” at the point of inclusion. You can “pass
parameters” by setting variables before including the file and
referring to them within the file. You can “return” values by
setting variables within the file and referring them following
inclusion of the file.
*/
###############################################################################
## Controlling the Binary Log Format Used for Tests
/**
@page PAGE_BINARY_LOG_FORMAT Controlling the Binary Log Format Used for Tests
<h3>Table of Contents</h3>
- @subpage PAGE_CONTROLLING_BINARY_LOG_FORMAT
- @subpage PAGE_SPECIFY_BINARY_LOG_FORMAT
The server can do binary logging using statement-based logging
(SBL), which logs events as statements that produce data changes, or
row-based logging (RBL), which logs events as changes to individual
rows. It also supports mixed logging, which switches between SBL and
RBL automatically as necessary.
The server's global <b>binlog_format</b> system variable indicates
which log format is in effect. It has possible values of
<b>STATEMENT</b>, <b>ROW</b>, and <b>MIXED</b> (not case
sensitive). This system variable can be set at server startup by
specifying <b>`--binlog_format`=<em>value</em></b> on the
command line or in an option file. A user who has the <b>SUPER</b>
privilege can change the log format at runtime. For example:
@verbatim
SET GLOBAL binlog_format = STATEMENT;
@endverbatim
Some tests require a particular binary log format. You can exercise
control over the binary log format in two ways:
- To control the log format that the server uses for an entire
test run, you can pass options to @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" that tell it which format <b>mysqld</b>
should use.
- To verify that a particular log format is in effect for a specific
test case, you can use an appropriate include file that checks
the current format and exits if the format is other than what
is required.
The following sections describe how to use these techniques.
*/
###############################################################################
## Controlling the Binary Log Format Used for an Entire Test Run
/**
@page PAGE_CONTROLLING_BINARY_LOG_FORMAT Controlling the Binary Log Format Used for an Entire Test Run
To specify the binary log format for a test run, you can use the
<b>`--mysqld`</b> or <b>`--combination`</b> option to tell @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" to pass a logging option
to <b>mysqld</b>. For example, the following command runs the tests
from the <b>rpl</b> suite that have names that begin with
<b>rpl_row</b>. The tests are run once with the binary log format
set to <b>STATEMENT</b>:
@verbatim
shell> ./mysql-test-run.pl --suite=rpl --do-test=rpl_row
--mysqld=--binlog_format=statement
@endverbatim
To run tests under multiple log formats, use two or more instances
of the <b>`--combination`</b> option. The following command runs
the same tests as the preceding command, but runs them once with the
binary log format set to <b>ROW</b> and a second time with the
format set to <b>MIXED</b>:
@verbatim
shell> ./mysql-test-run.pl --suite=rpl --do-test=rpl_row
--combination=--binlog_format=row
--combination=--binlog_format=mixed
@endverbatim
The <b>`--combination`</b> option must be given at least twice or
it has no effect.
As an alternative to using the <b>`--combination`</b> option, you
can create a file named <b>combinations</b> in the test suite
directory and list the options that you would specify using
<b>`--combination`</b>, one line per option. For the preceding
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" command, the suite
name is <b>rpl</b>, so you would create a file named
<b>suite/rpl/combinations</b> with these contents:
@verbatim
[row]
--binlog_format=row
[mixed]
--binlog_format=mixed
@endverbatim
Then invoke @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" like this:
@verbatim
shell> ./mysql-test-run.pl --suite=rpl --do-test=row
@endverbatim
Similar to <b>combinations</b> file in the test suite directory, you
can also create a file named <b><em>test_name</em>.combinations</b>
in the test directory and list the options that you would specify
using <b>`--combination`</b>, one line per option.
If there exist both suite level <b>combinations</b> file and a
<b><em>test_name</em>.combinations</b> file, then the
<b><em>test_name</em>.combinations</b> file will take the priority
i.e combinations specified in suite level <b>combinations</b> file
will be ignored.
The format of the <b>combinations</b> file is similar to that of
<b>my.cnf</b> files (section names followed by options for each
section). @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" displays
the section name following the test name when it reports the test
result.
Any <b>`--combination`</b> options specified on the command line
override those found in a <b>combinations</b> file.
The <b>`--combination`</b> option and the <b>combinations</b>
file have different scope. The <b>`--combination`</b> option
applies globally to all tests run by a given invocation of @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl". The
<b>combinations</b> file is placed in a test suite directory and
applies only to tests in that suite. The
<b><em>test_name</em>.combinations</b> file is placed in a test
directory and applies only to <b>test_name</b>.
*/
###############################################################################
## Specifying the Required Binary Log Format for Individual Test Cases
/**
@page PAGE_SPECIFY_BINARY_LOG_FORMAT Specifying the Required Binary Log Format for Individual Test Cases
To specify within a test case that a particular binary log format is
required, include one of the following lines to indicate the format:
@verbatim
--source include/have_binlog_format_row.inc
--source include/have_binlog_format_statement.inc
--source include/have_binlog_format_mixed.inc
@endverbatim
The following files can be used for tests that support two binary
log formats:
@verbatim
--source include/have_binlog_format_mixed_or_row.inc
--source include/have_binlog_format_mixed_or_statement.inc
--source include/have_binlog_format_row_or_statement.inc
@endverbatim
Before @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" runs the test
case, it checks whether the value that it is using for the
<b>binlog_format</b> system variable matches what the test
requires, based on whether the test refers to one of the preceding
include files. If <b>binlog_format</b> does not have an
appropriate value, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
skips the test.
If a test supports all binary log formats, none of the
<b>have_binlog_format_*.inc</b> include files should be used in
the test file. A test that includes no such file is assumed to
support all formats.
*/
###############################################################################
## Writing Replication Tests
/**
@page PAGE_WRITING_REPLICATION_TESTS Writing Replication Tests
If you are writing a replication test case, the test case file
should begin with this command:
@verbatim
--source include/master-slave.inc
@endverbatim
To switch between the master and slave, use these commands:
@verbatim
connection master;
connection slave;
@endverbatim
If you need to do something on an alternative connection, you can
use <b>connection master1;</b> for the master and <b>connection
slave1;</b> for the slave.
To run the master with additional options for your test case, put
them in command-line format in
<b>t/<em>test_name-master</em>.opt</b>. When a file named
<b>t/<em>test_name-master</em>.opt</b> exists, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" examines it for extra
options that the server needs to be run with when executing the
<em>test_name</em> test case. If no server has yet been started
or the current server is running with different options, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" restarts the server with
the new options.
For the slave, similar principles apply, but you should list
additional options in <b>t/<em>test_name-slave</em>.opt</b>.
Several include files are available for use in tests that enable
better control over the behavior of slave server I/O and SQL
threads. The files are located in the <b>include</b> directory and
have names of the form <b>wait_for_slave_*.inc</b>. By using these
files, you can help make replication tests more stable because it
will be more likely that test failures are due to replication
failures, not due to problems with the tests themselves.
The slave-control include files address the issue that it is not
always sufficient to use a <b>START SLAVE</b> or <b>STOP
SLAVE</b> statement by itself: When the statement returns, the
slave may not have reached the desired operational state. For
example, with <b>START SLAVE</b>, the following considerations
apply:
<ul>
<li>
It is not necessary to wait for the SQL thread after <b>START
SLAVE</b> or <b>START SLAVE SQL_THREAD</b> because the thread
will have started by the time statement returns.
</li>
<li>
By contrast, it is necessary to wait for the I/O thread after
<b>START SLAVE</b> or <b>START SLAVE IO_THREAD</b> because
although the thread will have started when the statement returns,
it may not yet have established the connection to the master.
</li>
</ul>
To verify that a slave has reached the desired state, combine the
use of <b>START SLAVE</b> or <b>STOP SLAVE</b> with an
appropriate “wait” include file. The file contains code that waits
until the state has been reached or a timeout occurs. For example,
to verify that both slave threads have started, do this:
@verbatim
START SLAVE
--source include/wait_for_slave_to_start.inc
@endverbatim
Similarly, to stop both slave threads, do this:
@verbatim
STOP SLAVE;
--source include/wait_for_slave_to_stop.inc
@endverbatim
The following list describes the include files that are available
for slave control:
<ul>
<li>
<tt>wait_for_slave_to_start.inc</tt>
Waits for both slave threads (I/O and SQL) to start. Should be
preceded by a <b>START SLAVE</b> statement.
</li>
<li>
<tt>wait_for_slave_to_stop.inc</tt>
Waits for both slave threads (I/O and SQL) to stop. Should be
preceded by a <b>STOP SLAVE</b> statement.
</li>
<li>
<tt>wait_for_slave_sql_to_stop.inc</tt>
Waits for the slave SQL thread to stop. Should be preceded by
a <b>STOP SLAVE SQL_THREAD</b> statement.
</li>
<li>
<tt>wait_for_slave_io_to_stop.inc</tt>
Waits for the slave I/O thread to stop. Should be preceded by
a <b>STOP SLAVE IO_THREAD</b> statement.
</li>
<li>
<tt>wait_for_slave_param.inc</tt>
Waits until <b>SHOW SLAVE STATUS</b> output contains a given
value or a timeout occurs. Before including the file, you should
set the <b>$slave_param</b> variable to the column name to
look for in <b>SHOW SLAVE STATUS</b> output, and
<b>$slave_param_value</b> to the value that you are waiting
for the column to have.
Example:
@verbatim
--let $slave_param= Slave_SQL_Running
--let $slave_param_value= No
--source include/slave_wait_slave_param.inc
@endverbatim
</li>
<li>
<tt>wait_for_slave_sql_error.inc</tt>
Waits until the SQL thread for the current connection has gotten
an error or a timeout occurs. Occurrence of an error is
determined by waiting for the <b>Last_SQL_Errno</b> column of
<b>SHOW SLAVE STATUS</b> output to have a nonzero value.
</li>
</ul>
*/
###############################################################################
## Thread Synchronization in Test Cases
/**
@page PAGE_THREAD_SYNCHRONIZATION Thread Synchronization in Test Cases
The Debug Sync facility allows placement of synchronization points
in the code. They can be activated by statements that set the
<b>debug_sync</b> system variable. An active synchronization point
can emit a signal or wait for a signal to be emitted by another
thread. This waiting times out after 300 seconds by default. The
<b>`--debug-sync-timeout`=<em>N</em></b> option for
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" changes that
timeout to <b><em>N</em></b> seconds. A timeout of zero disables
the facility altogether, so that synchronization points will not
emit or wait for signals, even if activated.
The purpose of the timeout is to avoid a complete lockup in test
cases. If for some reason the expected signal is not emitted by
any thread, the execution of the affected statement will not block
forever. A warning shows up when the timeout happens. That makes a
difference in the test result so that it will not go undetected.
For test cases that require the Debug Sync facility, include the
following line in the test case file:
@verbatim
--source include/have_debug_sync.inc
@endverbatim
For a description of the Debug Sync facility and how to use
synchronization points,
see [MySQL Internals: Test Synchronization]
(https://dev.mysql.com/doc/internals/en/test-synchronization.html).
*/
###############################################################################
## Suppressing Errors and Warning
/**
@page PAGE_SUPPRESSING_ERRORS_WARNINGS Suppressing Errors and Warning
After a test is finished, and if it didn't fail for some other
reason, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will check
the log written by the server(s) during the test for any warnings or
errors. If it finds any, it will print them out and the test will be
reported as failed.
However, many warnings and also a few errors may occur normally
without signifying a real problem. Also, many tests purposefully
perform actions that will provoke warnings. For these not to result
in a test failure, it is possible to specify that certain messages
are to be suppressed.
There is a list of global suppressions; warnings that will always be
suppressed. These are listed in the file
<b>include/mtr_warnings.sql</b>. Any error or warning that
contains one of the listed strings will be ignored. It is not
necessary to include the whole text, and regular expressions can be
used, such as <b>.*</b> to match any substring or <b>[0-9]*</b>
to match any number.
You will rarely need to change or add to this global list, but you
may need to suppress a particular error or warning for a new test
case. A typical case is a test that performs some illegal action on
purpose to test the respone; if this also results in a warning in
the server log, this should not cause this particular test to fail.
To add a suppresion for a warning text like for example <b>The
table 't23456' is full</b> where the number in the table name can
vary, add this line anywhere in the test case file:
@verbatim
call mtr.add_suppression("The table 't[0-9]*' is full");
@endverbatim
This may be put anythere in the test, but we recommend putting it
either near the top, or just after the action that may result in the
warning. If you're adding this line to an existing test, keep in
mind that it will be echoed in the output, so the exact same line
also needs to be added to the corresponding place in the result
file. Alternatively, you may turn off logging like this, and will
then not have to edit the result file:
@verbatim
--disable_query_log
call mtr.add_suppression("The table 't[0-9]*' is full")
--enable_query_log
@endverbatim
It is also possible to intruct @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" to skip the check for errors and warnings
completely, by use of the <b>`--nowarnings`</b> command line
option.
*/
###############################################################################
## Stopping and restarting a Server During a Test
/**
@page PAGE_STOPPING_AND_RESTARTING_SERVER_DURING_TEST Stopping and Restarting a Server During a Test
If a server dies during execution of a test case, this will be
interpreted as a failure. However, there may be cases where you
actually want to stop and possibly restart a server intentionally.
It is possible to let the system know you expect a server to
terminate, and to either wait or have it restarted immediately:
Before you initiate the action that will stop the server, the test
case should write either <b>restart</b> or <b>wait</b> to the
file <b>$MYSQLTEST_VARDIR/tmp/mysqld.1.expect</b>. The easiest way
to do this is using the <b>exec echo</b> construct like in the
following example.
If you write <b>restart</b> into this file, the server will be
immediately restarted. If you write <b>wait</b> instead, the
server will remain down, but can be restarted at a later time by
writing <b>restart</b> to the file.
@verbatim
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server 10
--source include/wait_until_disconnected.inc
# Do something while server is down
--enable_reconnect
--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--source include/wait_until_connected_again.inc
@endverbatim
The file name to write the command to will be
<b>mysqld.2.expect</b> for the slave server in replication tests.
Note that you have to use <b>$MYSQLTEST_VARDIR/tmp</b> for the
directory here; if you use <b>$MYSQL_TMP_DIR</b> instead, it will
not work when running tests in parallel.
For your convenience, there are files which can be used to do
what's shown in the example.
<ul>
<li>
<tt>restart_mysqld.inc</tt>
<b>include/restart_mysqld.inc</b> can be sourced in your
own test case, to shutdown and start the server but
the server is immediately started up again.
@verbatim
# Restart the server
--source include/restart_mysqld.inc
@endverbatim
It is also possible to provide additional command line options
to the restarted server, by writing a line with <b>restart:</b>
(with a colon) followed by one or more options into the expect
file. These extra options will be dropped if the same server
is restarted again, unless they are repeated. A value for
shutdown timeout can also be set with
<b>$shutdown_server_timeout</b>, when this file is used.
@verbatim
# Restart server with extra parameters
--let $shutdown_server_timeout= 10
--let $restart_parameters= "restart: --innodb_autoinc_lock_mode=1"
--source include/restart_mysqld.inc
@endverbatim
</li>
<li>
<tt>kill_and_restart_mysqld.inc</tt>
Similar to <b>include/restart_mysqld.inc</b>,
<b>include/kill_and_restart_mysqld.inc</b> can be used to
restart the server, but setting <b>$shutdown_server_timeout</b>
will have no effect.
@verbatim
# Restart server with extra parameters
--let $restart_paramters= "restart: --innodb_autoinc_lock_mode=0"
--source include/kill_and_restart_mysqld.inc
@endverbatim
</li>
<li>
<tt>shutdown_mysqld.inc and start_mysqld.inc</tt>
If there is something to be done between the server shutdown
and startup, there is a file
<b>include/shutdown_mysqld.inc</b>
which can be used to shutdown the server. A shutdown timeout
value can be set and passed to
<b>include/shutdown_mysqld.inc</b>.
<b>include/start_mysqld.inc</b> can be used to start the
server again after it has been shutdown.
@verbatim
# Timeout of 10 seconds
--let $shutdown_server_timeout= 10
--source include/shutdown_mysqld.inc
# Do something while the server is down
--source include/start_mysqld.inc
@endverbatim
</li>
<li>
<tt>kill_mysqld.inc</tt>
To the kill the server,
<b>include/kill_mysqld.inc</b> can be sourced, which
internally calls <b>--shutdown_server 0</b>. Therefore,
setting a value for shutdown timeout will have no effect.
@verbatim
--source include/kill_mysqld.inc
# Do something while the server is down
--source include/start_mysqld.inc
@endverbatim
</li>
<li>
<tt>wait_until_disconnected.inc</tt>
It is recommended to use <b>include/wait_until_disconnected</b>
in tandem with <b>--shutdown_server</b>. It ensures that all
connections to the server are dropped before proceeding with
the test. This file is sourced internally in
<b>include/shutdown_mysqld.inc</b>,
<b>include/kill_mysqld.inc</b>,
<b>include/kill_and_restart_mysqld.inc</b>,
and <b>include/restart_mysqld.inc</b>.
</li>
<li>
<tt>wait_until_connected_again.inc</tt>
<b>include/wait_until_connected_again.inc</b> is used after
the server is started again, to ensure that the client
connection is restored. It tries to establish the connection
till a timeout occurs. This file is sourced internally in
<b>include/start_mysqld.inc</b>,
<b>include/restart_mysqld.inc</b>,
and <b>include/kill_and_restart_mysqld.inc</b>.
</li>
</ul>
*/
###############################################################################
## Other Tips for Writing Test Cases
/**
@page PAGE_TIPS_WRITING_TESTCASES Other Tips for Writing Test Cases
<ul>
<li>
Writing loops
If you need to do something in a loop, you can use something like this:
@verbatim
let $1= 10;
while ($1)
{
# execute your statements here
dec $1;
}
@endverbatim
</li>
<li>
Pausing between statements
To sleep between statements, use the <b>sleep</b> command. It
supports fractions of a second. For example, <b>sleep 0.2</b>;
sleeps 0.2 seconds.
Try not to use <b>sleep</b> or <b>real_sleep</b> commands
more than necessary. The more of them there are, the slower the
test suite becomes. In some cases, heavy reliance on sleep
operations is an indicator that the logic of a test should be
reconsidered.
</li>
<li>
Commenting the test result
When the output in a result file is not understandable by
inspection, it can be helpful to have the test case write
comments to the result file that provide context. You can use
the <b>echo</b> command for this:
@verbatim
--echo # Comment to explain the following output
@endverbatim
Of course, the same line (without <b>`--echo`</b>) will need
to be put in the corresponding place in the result file, if you
are adding this to an existing test.
</li>
<li>
Sorting result sets
If a test case depends on <b>SELECT</b> output being in a
particular row order, use an <b>ORDER BY</b> clause. Do not
assume that rows will be selected in the same order they are
inserted, particularly for tests that might be run multiple
times under conditions that can change the order, such as with
different storage engines, or with and without indexing.
If you do not want to add <b>ORDER BY</b> to the query, an
alternative is to use the test command <b>sorted_result</b>
which will sort the result of the following statement before
putting it into the result file.
</li>
<li>
Performing file system operations
Avoid using <b>exec</b> or <b>system</b> to execute
operating system commands for file system operations. This used
to be very common, but OS commands tend to be platform specific,
which reduces test portability. @ref PAGE_MYSQLTEST "mysqltest"
now has several commands to perform these operations portably,
so these commands should be used instead: <b>remove_file</b>,
<b>chmod</b>, <b>mkdir</b>, and so forth.
</li>
<li>
Local versus remote storage
Some test cases depend on being run on local storage, and may
fail when run on remote storage such as a network share. For
example, if the test result can be affected by differences
between local and remote file system times, the expected result
might not be obtained. Failure of these test cases under such
circumstances does not indicate an actual malfunction. It is not
generally possible to determine whether tests are being run on
local storage.
</li>
<li>
Skipping consistency check or forcing restart after test
As mentioned before, a test case should leave the server in a
"clean" state for the next test. In cases where this is not
possible or too costly, the test may in stead ask for the
consistency check to be skipped, and the server(s) to be
restarted. This is done by this executing this command at any
time during the test:
@verbatim
call mtr.force_restart();
@endverbatim
This signals to @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
that it should restart the server before the next test, and also
skip the consistency check.
</li>
</ul>
*/
###############################################################################
## MySQL Test Programs
/**
@page PAGE_MYSQLTEST_PROGRAMS MySQL Test Programs
<h3>Table of Contents</h3>
- @subpage PAGE_MYSQLTEST
- @subpage PAGE_MYSQL_CLIENT_TEST
- @subpage PAGE_MYSQL_TEST_RUN_PL
- @subpage PAGE_MYSQL_STRESS_TEST_PL
This chapter describes the test programs that run test cases. For
information about the language used for writing test cases, see @ref
PAGE_MYSQLTEST_LANGUAGE_REFERENCE.
The test suite uses the following programs:
<ul>
<li>
The @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" Perl script
is the main application used to run the MySQL test suite. It
invokes @ref PAGE_MYSQLTEST "mysqltest" to run individual test
cases. (Prior to MySQL 4.1, a similar shell script,
<b>mysql-test-run</b>, can be used instead).
</li>
<li>
@ref PAGE_MYSQLTEST "mysqltest" runs test cases.
</li>
<li>
The @ref PAGE_MYSQL_CLIENT_TEST "mysql_client_test" program is
used for testing aspects of the MySQL client API that cannot be
tested using @ref PAGE_MYSQLTEST "mysqltest" and its test
language.
</li>
<li>
The @ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" Perl
script performs stress-testing of the MySQL server.
</li>
</ul>
*/
###############################################################################
## mysqltest — Program to Run Test Cases
/**
@page PAGE_MYSQLTEST mysqltest — Program to Run Test Cases
The @ref PAGE_MYSQLTEST "mysqltest" program runs a test case against
a MySQL server and optionally compares the output with a result
file. This program reads input written in a special test language.
Typically, you invoke @ref PAGE_MYSQLTEST "mysqltest" using @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" rather than invoking it
directly.
Features of @ref PAGE_MYSQLTEST "mysqltest":
- Can send SQL statements to MySQL servers for execution
- Can execute external shell commands
- Can test whether the result from an SQL statement or shell
command is as expected
- Can connect to one or more standalone <b>mysqld</b> servers
and switch between connections
By default, @ref PAGE_MYSQLTEST "mysqltest" reads the test case on
the standard input. To run @ref PAGE_MYSQLTEST "mysqltest" this way,
you normally invoke it like this:
@verbatim
shell> mysqltest [options] [db_name] < test_file
@endverbatim
You can also name the test case file with a <b>`--test-
file`=<em>file_name</em></b> option.
The exit value from @ref PAGE_MYSQLTEST "mysqltest" is 0 for
success, 1 for failure, and 62 if it skips the test case (for
example, if after checking some preconditions it decides not to
run the test).
@ref PAGE_MYSQLTEST "mysqltest" supports the following options:
<ul>
<li>
<tt>`--help`, -?</tt>
Display a help message and exit.
</li>
<li>
<tt>`--async-client`</tt>
Enable asynchronous communication support in mysql protocol.
</li>
<li>
<tt>`--basedir`=<b>dir_name</b>, -b <b>dir_name</b></tt>
The base directory for tests.
</li>
<li>
<tt>`--character-sets-dir`=<b>path</b></tt>
The directory where character sets are installed.
</li>
<li>
<tt>`--colored-diff`</tt>
Colorize the diff part of the output.
If enabled, @ref PAGE_MYSQLTEST "mysqltest" uses <b>diff</b>
command with <b>`--color`='always'</b> option to print the
colored diff. If the command fails or is not supported, @ref
PAGE_MYSQLTEST "mysqltest" will report an error and abort the
test run.
@note
<b>`--color`</b> option for <b>diff</b> command is available
from <b>GNU diffutils</b> version <b>3.4</b>.
</li>
<li>
<tt>`--compress`, -C</tt>
Compress all information sent between the client and the server
if both support compression.
</li>
<li>
<tt>`--cursor-protocol`</tt>
Use cursors for prepared statements.
</li>
<li>
<tt>`--database`=<b>db_name</b>, -D <b>db_name</b></tt>
The default database to use.
</li>
<li>
<tt>`--debug`[=<b>debug_options</b>], -#[<b>debug_options</b>]</tt>
Write a debugging log if MySQL is built with debugging support.
The default <b><em>debug_options</em></b> value is
<b>'d:t:S:i:O,/tmp/mysqltest.trace'</b>.
</li>
<li>
<tt>`--debug-check`</tt>
Print some debugging information when the program exits.
</li>
<li>
<tt>`--debug-info`</tt>
Print debugging information and memory and CPU usage statistics
when the program exits.
</li>
<li>
<tt>`--default-character-set`=<b>charset_name</b></tt>
Use <b>charset_name</b> as the default character set for the
client and connection. For more information, see [Connection
Character Sets and Collations]
(https://dev.mysql.com/doc/refman/8.0/en/charset-connection.html).
</li>
<li>
<tt>`--explain-protocol`</tt>
Run <b>EXPLAIN EXTENDED</b> on all <b>SELECT</b>,
<b>INSERT</b>, <b>REPLACE</b>, <b>UPDATE</b> and
<b>DELETE</b> queries.
</li>
<li>
<tt>`--host`=<b>host_name</b>, -h <b>host_name</b></tt>
Connect to the MySQL server on the given host.
</li>
<li>
<tt>`--include`=<b>file_name</b>, -i <b>file_name</b></tt>
Include the contents of the given file before processing the
contents of the test file. The included file should have the
same format as other @ref PAGE_MYSQLTEST "mysqltest" test files.
This option has the same effect as putting a <b>`--source`
<em>file_name</em></b> command as the first line of the test
file.
</li>
<li>
<tt>`--json-explain-protocol`</tt>
Run <b>EXPLAIN EXTENDED</b> on all <b>SELECT</b>,
<b>INSERT</b>, <b>REPLACE</b>, <b>UPDATE</b> and
<b>DELETE</b> queries. The <b>json-explain-protocol</b>
</li>
<li>
<tt>`--logdir`=<b>dir_name</b></tt>
The directory to use for log files.
</li>
<li>
<tt>`--mark-progress`</tt>
Write the line number and elapsed time to
<b><em>test_file</em>.progress.</b>
</li>
<li>
<tt>`--max-connect-retries`=<b>num</b></tt>
The maximum number of connection attempts when connecting to
server.
</li>
<li>
<tt>`--max-connections`=<b>num</b></tt>
The maximum number of simultaneous server connections per client
(that is, per test). If not set, the maximum is 128. Minimum
allowed limit is 8, maximum is 5120.
</li>
<li>
<tt>`--no-defaults`</tt>
Do not read default options from any option files. If used, this
must be the first option.
</li>
<li>
<tt>`--plugin-dir`=<b>path</b></tt>
The directory in which to look for plugins. It may be necessary
to specify this option if the <b><em>default_auth</em></b>
argument is used for the <b>connect()</b> command to specify
an authentication plugin but @ref PAGE_MYSQLTEST "mysqltest"
does not find it.
</li>
<li>
<tt>`--password`[=<b>password</b>], -p[<b>password</b>]</tt>
The password to use when connecting to the server. If you use
the short option form (<b>-p</b>), you <b><em>cannot</em></b>
have a space between the option and the password. If you omit
the <b><em>password</em></b> value following the
<b>`--password`</b> or <b>-p</b> option on the command line,
you are prompted for one.
</li>
<li>
<tt>`--port`=<b>port_num</b>, -P <b>port_num</b></tt>
The TCP/IP port number to use for the connection.
</li>
<li>
<tt>`--protocol`={TCP|SOCKET|PIPE|MEMORY}</tt>
Choose the protocol for communication with the server.
<b>SOCKET</b> is default.
</li>
<li>
<tt>`--ps-protocol`</tt>
Use the prepared-statement protocol for communication.
</li>
<li>
<tt>`--quiet`</tt>
Suppress all normal output. This is a synonym for
<b>`--silent`</b>.
</li>
<li>
<tt>`--record`, -r</tt>
Record the output that results from running the test file into
the file named by the <b>`--result-file`</b> option, if that
option is given. It is an error to use this option without also
using <b>`--result-file`</b>.
</li>
<li>
<tt>`--result-file`=<b>file_name</b>, -R <b>file_name</b></tt>
This option specifies the file for test case expected results.
<b>`--result-file`</b>, together with <b>`--record`</b>,
determines how @ref PAGE_MYSQLTEST "mysqltest" treats the test
actual and expected results for a test case:
<ul>
<li>
If the test produces no results, @ref PAGE_MYSQLTEST
"mysqltest" exits with an error message to that effect,
unless <b>`--result-file`</b> is given and the named file
is an empty file.
</li>
<li>
Otherwise, if <b>`--result-file`</b> is not given, @ref
PAGE_MYSQLTEST "mysqltest" sends test results to the
standard output.
</li>
<li>
With <b>`--result-file`</b> but not <b>`--record`</b>,
@ref PAGE_MYSQLTEST "mysqltest" reads the expected results
from the given file and compares them with the actual
results. If the results do not match, @ref PAGE_MYSQLTEST
"mysqltest" writes a <b>.reject</b> file in the log
directory, outputs a diff of the two files, and exits with
an error.
</li>
<li>
With both <b>`--result-file`</b> and <b>`--record`</b>,
@ref PAGE_MYSQLTEST "mysqltest" updates the given file by
writing the actual test results to it.
</li>
</ul>
</li>
<li>
<tt>`--server-public-key-path`=<b>file_name</b></tt>
The path name to a file containing the server RSA public key.
The file must be in PEM format. The public key is used for RSA
encryption of the client password for connections to the server
made using accounts that authenticate with the
<b>sha256_password</b> plugin. This option is ignored for
client accounts that do not authenticate with that plugin. It is
also ignored if password encryption is not needed, as is the
case when the client connects to the server using an SSL
connection.
The server sends the public key to the client as needed, so it
is not necessary to use this option for RSA password encryption
to occur. It is more efficient to do so because then the server
need not send the key.
For additional discussion regarding use of the
<b>sha256_password plugin</b>, including how to get the RSA
public key, see [The SHA-256 Authentication Plugin]
(https://dev.mysql.com/doc/refman/8.0/en/sha256-authentication-plugin.html).
</li>
<li>
<tt>`--silent`, -s</tt>
Suppress all normal output.
</li>
<li>
<tt>`--skip-safemalloc`</tt>
Do not use memory allocation checking.
</li>
<li>
<tt>`--sleep`=<b>num</b>, -T <b>num</b></tt>
Cause all <b>sleep</b> commands in the test case file to sleep
<b><em>num</em></b> seconds. This option does not affect
<b>real_sleep</b> commands.
An option value of 0 can also be used, which effectively
disables <b>sleep</b> commands in the test case.
</li>
<li>
<tt>`--socket`=<b>path</b>, -S <b>path</b></tt>
The socket file to use when connecting to <b>localhost</b>
(which is the default host).
</li>
<li>
<tt>`--sp-protocol`</tt>
Execute DML statements within a stored procedure. For every DML
statement, @ref PAGE_MYSQLTEST "mysqltest" creates and invokes a
stored procedure that executes the statement rather than
executing the statement directly.
</li>
<li>
<tt>`--tail-lines`=<b>n</b></tt>
Specify how many lines of the result to include in the output if
the test fails because an SQL statement fails. The default is 0,
meaning no lines of result printed.
</li>
<li>
<tt>`--test-file`=<b>file_name</b>, -x <b>file_name</b></tt>
Read test input from this file. The default is to read from the
standard input.
</li>
<li>
<tt>`--timer-file`=<b>file_name</b>, -m <b>file_name</b></tt>
If given, the number of millisecond spent running the test will
be written to this file. This is used by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" for its reporting.
</li>
<li>
<tt>`--tls-version`=<b>protocol_list</b></tt>
The protocols permitted by the client for encrypted connections.
The value is a comma-separated list containing one or more of
these protocols: TLSv1, TLSv1.1, TLSv1.2.
</li>
<li>
<tt>`--tmpdir`=<b>dir_name</b>, -t <b>dir_name</b></tt>
The temporary directory where socket files are created.
</li>
<li>
<tt>`--trace-exec`</tt>
If enabled, this option causes @ref PAGE_MYSQLTEST "mysqltest"
to immediately display the output from executed programs to
stdout.
</li>
<li>
<tt>`--user`=<b>user_name</b>, -u <b>user_name</b></tt>
The MySQL user name to use when connecting to the server.
</li>
<li>
<tt>`--verbose, -v`</tt>
Verbose mode. Print out more information about what the program
does.
</li>
<li>
<tt>`--version, -V`</tt>
Display version information and exit.
</li>
<li>
<tt>`--view-protocol`</tt>
Every <b>SELECT</b> statement is wrapped inside a view.
</li>
</ul>
*/
###############################################################################
## mysql_client_test — Test Client API
/**
@page PAGE_MYSQL_CLIENT_TEST mysql_client_test — Test Client API
The @ref PAGE_MYSQL_CLIENT_TEST "mysql_client_test" program is used
for testing aspects of the MySQL client API that cannot be tested
using @ref PAGE_MYSQLTEST "mysqltest" and its test language. @ref
PAGE_MYSQL_CLIENT_TEST "mysql_client_test" is run as part of the
test suite.
The source code for the programs can be found in in
<b>tests/mysql_client_test.c</b> in a source distribution. The
program serves as a good source of examples illustrating how to use
various features of the client API.
@ref PAGE_MYSQL_CLIENT_TEST "mysql_client_test" is used in a test by
the same name in the main tests suite of @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" but may also be run directly. Unlike the other
programs listed here, it does not read an external description of
what tests to run. Instead, all tests are coded into the program,
which is written to cover all aspects of the C language API.
@ref PAGE_MYSQL_CLIENT_TEST "mysql_client_test" supports the
following options:
<ul>
<li>
<tt>`--help`, -?</tt>
Display a help message and exit.
</li>
<li>
<tt>`--basedir`=<b>dir_name</b>, -b <b>dir_name</b></tt>
The base directory for the tests.
</li>
<li>
<tt>`--count`=<b>count</b>, -t <b>count</b></tt>
The number of times to execute the tests.
</li>
<li>
<tt>`--database`=<b>db_name</b>, -D <b>db_name</b></tt>
The database to use.
</li>
<li>
<tt>`--debug`[=<b>debug_options</b>], -#[<b>debug_options</b>]</tt>
Write a debugging log if MySQL is built with debugging support.
The default <b>debug_options</b> value is
<tt>'d:t:o,/tmp/mysql_client_test.trace'</tt>.
</li>
<li>
<tt>`--getopt-ll-test`=<b>option</b>,
-g <b>option</b></tt>
Option to use for testing bugs in the <tt>getopt</tt> library.
</li>
<li>
<tt>`--host`=<b>host_name</b>, -h <b>host_name</b></tt>
Connect to the MySQL server on the given host.
</li>
<li>
<tt>`--password`[=<b>password</b>], -p[<b>password]</b></tt>
The password to use when connecting to the server. If you use
the short option form (<b>-p</b>), you <tt><em>cannot</em></tt>
have a space between the option and the password. If you omit
the <b>password</b> value following the
<b>`--password`</b> or <b>-p</b> option on the command line,
you are prompted for one.
</li>
<li>
<tt>`--port`=<b>port_num</b>, -P <b>port_num</b></tt>
The TCP/IP port number to use for the connection.
</li>
<li>
<tt>`--show-tests`, -T</tt>
Show all test names.
</li>
<li>
<tt>`--silent`, -s</tt>
Be more silent.
</li>
<li>
<tt>`--socket`=<b>path</b>, -S <b>path</b></tt>
The socket file to use when connecting to <b>localhost</b>
(which is the default host).
</li>
<li>
<tt>`--testcase`, -c</tt>
The option is used when called from @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl", so that @ref PAGE_MYSQL_CLIENT_TEST
"mysql_client_test" may optionally behave in a different way
than if called manually, for example by skipping some tests.
Currently, there is no difference in behavior but the option is
included to make this possible.
</li>
<li>
<tt>`--user`=<b>user_name</b>, -u <b>user_name</b></tt>
The MySQL user name to use when connecting to the server.
</li>
<li>
<tt>`--vardir`=<b>dir_name</b>, -v <b>dir_name</b></tt>
The data directory for tests. The default is
<b>mysql-test/var</b>.
</li>
</ul>
*/
###############################################################################
## mysql-test-run.pl — Run MySQL Test Suite
/**
@page PAGE_MYSQL_TEST_RUN_PL mysql-test-run.pl — Run MySQL Test Suite
The @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" Perl script is
the main application used to run the MySQL test suite. It invokes
@ref PAGE_MYSQLTEST "mysqltest" to run individual test cases.
Invoke @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" in the
<b>mysql-test</b> directory like this:
@verbatim
shell> mysql-test-run.pl [options] [test_name] ...
@endverbatim
Each <b>test_name</b> argument names a test case. The test
case file that corresponds to the test name is
<b>t/<em>test_name</em>.test</b>.
For each <b>test_name</b> argument, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" runs the named test case.
With no <b>test_name</b> arguments, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" runs all <b>.test</b>
files in the <b>t</b> subdirectory.
If no suffix is given for the test name, a suffix of <b>.test</b>
is assumed. Any leading path name is ignored. These commands are
equivalent:
@verbatim
shell> mysql-test-run.pl mytest
shell> mysql-test-run.pl mytest.test
shell> mysql-test-run.pl t/mytest.test
@endverbatim
A suite name can be given as part of the test name. That is, the
syntax for naming a test is:
@verbatim
[suite_name.]test_name[.suffix]
@endverbatim
If a suite name is given, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" looks in that suite for the test. The test file
corresponding to a test named <b>suite_name.test_name</b> is
found in
<b>suite/<em>suite_name</em>/t/<em>test_name</em>.test</b>.
There is also an implicit suite name <b>main</b> for the tests in
the top <b>t</b> directory. With no suite name, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" looks in the default list
of suites for a match and runs the test in any suites where it finds
the test. Suppose that the default suite list is <b>main</b>,
<b>binlog</b>, <b>rpl</b>, and that a test <b>mytest.test</b>
exists in the <b>main</b> and <b>rpl</b> suites. With an
argument of <b>mytest</b> or <b>mytest.test</b>, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will run
<b>mytest.test</b> from the <b>main</b> and <b>rpl</b> suites.
To run a family of test cases for which the names share a common
prefix, use the <b>`--do-test`=<em>prefix</em></b> option.
For example, <b>`--do-test`=rpl</b> runs the replication tests
(test cases that have names beginning with <b>rpl</b>).
<b>`--skip-test`</b> has the opposite effect of skipping test
cases for which the names share a common prefix.
The argument for the <b>`--do-test`</b> and <b>`--skip-test`</b>
options also allows more flexible specification of which tests to
perform or skip. If the argument contains a pattern metacharacter
other than a lone period, it is interpreted as a Perl regular
expression and applies to test names that match the pattern. If the
argument contains a lone period or does not contain any pattern
metacharacters, it is interpreted the same way as previously and
matches test names that begin with the argument value. For example,
<b>`--do-test`=testa</b> matches tests that begin with testa,
<b>`--do-test`=main.testa</b> matches tests in the <b>main</b>
test suite that begin with <b>testa</b>, and
<b>`--do-test`=main.*testa</b> matches test names that contain
<b>main</b> followed by <b>testa</b> with anything in between.
In the latter case, the pattern match is not anchored to the
beginning of the test name, so it also matches names such as
<b>xmainytesta</b>.
It is also possible to put a list of test names in a file and have
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" run those tests,
using the option <b>`--do-test-list`=file</b>. The tests should be
listed one per line in the file. A line beginning with <b>#</b>
indicates a comment and is ignored. Similary an empty line in the
file is also ignored.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" also supports a
<b>`--do-suite`</b> option, which is similar to <b>`--do-test`</b>
but permits specifying entire suites of tests to run.
To perform setup prior to running tests, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" needs to invoke <b>mysqld</b> with the
<b>`--initialize`</b> and <b>`--skip-grant-tables`</b> options.
If MySQL was built with the compiler flag
<b>-DDISABLE_GRANT_OPTIONS</b>, then <b>`--initialize`</b>,
<b>`--skip-grant-tables`</b>, and <b>`--init-file`</b> will be
disabled. To handle this, set the <b>MYSQLD_BOOTSTRAP</b>
environment variable to the full path name of a server that has all
options enabled. @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
will use that server to perform setup; it is not used to run the
tests.
The <b>init_file</b> test will fail if <b>`--init-file`</b> is
disabled. This is an expected failure in this case.
To run @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" on Windows,
you'll need either Cygwin or ActiveState Perl to run it. You may
also need to install the modules required by the script. To run the
test script, change location into the <b>mysql-test</b> directory,
set the <b>MTR_VS_CONFIG</b> environment variable to the
configuration you selected earlier (or use the <b>`--vs-config`</b>
option), and invoke @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl".
For example (using <b>Cygwin</b> and the <b>bash</b> shell):
@verbatim
shell> cd mysql-test
shell> export MTR_VS_CONFIG=debug
shell> ./mysqltest-run.pl --force --timer
shell> ./mysqltest-run.pl --force --timer --ps-protocol
@endverbatim
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" uses several
environment variables. Some of them are listed in the following
table. Some of these are set from the outside and used by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl", others are set by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" instead, and may be
referred to in tests.
<table>
<tr>
<th>Variable</th>
<th>Description</th>
</tr>
<tr>
<td><tt>MTR_BUILD_THREAD</tt></td>
<td>
If set, defines which port number range is used for the server
</td>
</tr>
<tr>
<td><tt>MTR_MAX_PARALLEL</tt></td>
<td>
If set, defines maximum number of parallel threads if
<b>`--parallel`=auto</b> is given
</td>
</tr>
<tr>
<td><tt>MTR_MEM</tt></td>
<td>
If set to anything, will run tests with files in "memory"
using tmpfs or ramdisk. Not available on Windows. Same as
<b>`--mem`</b> option
</td>
</tr>
<tr>
<td><tt>MTR_<b>NAME</b>_TIMEOUT</tt></td>
<td>
Setting of a timeout in minutes or seconds, corresponding to
command line option <b>`--name-timeout`</b>. Avaliable timeout
names are <b>TESTCASE</b>, <b>SUITE</b> (both in minutes) and
<b>START</b>, <b>SHUTDOWN</b>, <b>CTEST</b> (all in seconds).
<b>MTR_CTEST_TIMEOUT</b> is for <b>ctest</b> unit tests.
</td>
</tr>
<tr>
<td><tt>MTR_PARALLEL</tt></td>
<td>
If set, defines number of parallel threads executing tests. Same
as <tt>`--parallel`</tt> option
</td>
</tr>
<tr>
<td><tt>MTR_PORT_BASE</tt></td>
<td>
If set, defines which port number range is used for the server
</td>
</tr>
<tr>
<td><tt>MTR_RECORD</tt></td>
<td>
Set to 1 if a MTR run is started with <b>`--record`</b>
option, 0 otherwise.
</td>
</tr>
<tr>
<td><tt>MTR_UNIQUE_IDS_DIR</tt></td>
<td>
The method in which the free ports are allocated is by maintaining
a list under the unique ids directory. In case there are multiple
chroot environments on the same host, then the same set of ports may
get allocated for all environments, because there will be
multiple unique ids directories in different physical locations,
but MTR assumes it is the same directory.
This will lead to a conflict while reserving ports for the
different MTR runs. To avoid this problem, when using chroot
environments, <b>MTR_UNIQUE_IDS_DIR</b> environment variable can
be set to a common location, so that all the MTR processes will
use the same unique ids directory.
Users have to make sure this variable is set to the same path
on all environments, and that this common path is mounted on
all the environments.
<tr>
<td><tt>MYSQL_CONFIG_EDITOR</tt></td>
<td>
Path name to <b>mysql_config_editor</b> binary.
</td>
</tr>
<tr>
<td><tt>MYSQL_TEST</tt></td>
<td>
Path name to @ref PAGE_MYSQLTEST "mysqltest" binary
</td>
</tr>
<tr>
<td><tt>MYSQL_TEST_DIR</tt></td>
<td>
Full path to the <tt>mysql-test</tt> directory where tests
are being run from
</td>
</tr>
<tr>
<td><tt>MYSQL_TEST_LOGIN_FILE</tt></td>
<td>
Path name to login file used by <b>mysql_config_editor</b>.
If not set, the default is <b>$HOME/.mylogin.cnf</b>, or
<b>\%APPDATA%\\MySQL\\\.mylogin.cnf</b> on Windows.
</td>
</tr>
<tr>
<td><tt>MYSQL_TMP_DIR</tt></td>
<td>
Path to temp directory used for temporary files during tests
</td>
</tr>
<tr>
<td><tt>MYSQLD</tt></td>
<td>
Full path to server executable used in tests.
</td>
</tr>
<tr>
<td><tt>MYSQLD_BOOTSTRAP</tt></td>
<td>
Full path name to <b>mysqld</b> that has all options enabled
</td>
</tr>
<tr>
<td><tt>MYSQLD_BOOTSTRAP_CMD</tt></td>
<td>
Full command line used for initial database setup for this
test batch
</td>
</tr>
<tr>
<td><tt>MYSQLD_CMD</tt></td>
<td>
Command line for starting server as used in tests, with the
minimum set of required arguments.
</td>
</tr>
<tr>
<td><tt>MYSQLTEST_VARDIR</tt></td>
<td>
Path name to the <b>var</b> directory that is used for logs,
temporary files, and so forth
</td>
</tr>
<tr>
<td><tt>NUMBER_OF_CPUS</tt></td>
<td>
Defines number of processors.
</td>
</tr>
<tr>
<td><tt>TSAN_OPTIONS</tt></td>
<td>
Path name to a file containing ThreadSanitizer suppressions.
</td>
</tr>
</table>
The variable <b>MTR_PORT_BASE</b> is a more logical replacement
for the original variable <b>MTR_BUILD_THREAD</b>. It gives the
actual port number directly (will be rounded down to a multiple of
10). If you use <b>MTR_BUILD_THREAD</b>, the port number is found
by multiplying this by 10 and adding 10000.
Tests sometimes rely on certain environment variables being defined.
For example, certain tests assume that <b>MYSQL_TEST</b> is
defined so that @ref PAGE_MYSQLTEST "mysqltest" can invoke itself
with <b>exec $MYSQL_TEST</b>.
Other tests may refer to the last three variables listed in the
preceding table, to locate files to read or write. For example,
tests that need to create files will typically put them in
<b>$MYSQL_TMP_DIR/file_name</b>.
The variable <b>$MYSQLD_CMD</b> will include any server options
added with the <b>`--mysqld`</b> option to @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl", but will not include
server options added specifically for the currently running test.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" supports the
options in the following list. An argument of -- tells @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" not to process any
following arguments as options.
<ul>
<li>
<tt>`--help, -h`</tt>
Display a help message and exit.
</li>
<li>
<tt>`--big-test`</tt>
Allow tests marked as "big" to run. Tests can be thus marked by
including the line <b>`--source` include/big_test.inc</b>, and
they will only be run if this option is given, or if the
environment variable <b>BIG_TEST</b> is set to 1.
This is typically done for tests that take very long to run, or
that use very much resources, so that they are not suitable for
running as part of a normal test suite run.
If both <b>`--big-test`</b> and <b>`--only-big-tests`</b>
are given, <b>`--only-big-tests`</b> is ignored.
@note
This option is enabled by default when test cases are specified
on command line.
</li>
<li>
<tt>`--boot-dbx`</tt>
Run the <b>mysqld</b> server used for bootstrapping the database
through the <b>dbx</b> debugger.
</li>
<li>
<tt>`--boot-ddd`</tt>
Run the <b>mysqld</b> server used for bootstrapping the database
through the <b>ddd</b> debugger.
</li>
<li>
<tt>`--boot-gdb`</tt>
Run the <b>mysqld</b> server used for bootstrapping the database
through the <b>gdb</b> debugger.
See also the <b>`--manual-boot-gdb`</b> option.
</li>
<li>
<tt>`--build-thread`=<b>number</b></tt>
Specify a number to calculate port numbers from. The formula is
<b>10 * build_thread + 10000</b>. Instead of a number, it can
be set to <b>auto</b>, which is also the default value, in
which case @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will
allocate a number unique to this host.
The value (number or <b>auto</b>) can also be set with the
<b>MTR_BUILD_THREAD</b> environment variable.
This option is kept for backward compatibility. The more logical
<b>`--port-base`</b> is recommended instead.
</li>
<li>
<tt>`--callgrind`</tt>
Instructs <b>valgrind</b> to use <b>callgrind</b>.
</li>
<li>
<tt>`--charset-for-testdb`=<b>charset_name</b></tt>
Specify the default character set for the test database. The
default value is latin1.
</li>
<li>
<tt>`--check-testcases`</tt>
Check test cases for side effects. This is done by checking the
system state before and after each test case. If there is any
difference, the test case is marked as failed because of it.
Similarly, when <b>`--check-testcases`</b> option is enabled,
MTR does additional check for missing <b>.result</b> file and a
test case not having its corresponding <b>.result</b> file is
marked as failed.
This check is enabled by default. To disable it, use the
<b>`--nocheck-testcases`</b> option.
</li>
<li>
<tt>`--clean-vardir`</tt>
Clean up the <b>var</b> directory with logs and test results
etc. after the test run, but only if there were no test
failures. This option only has effect if also running with
option <b>`--mem`</b>. The intent is to alleviate the problem
of using up memory for test results, in cases where many
different test runs are being done on the same host.
</li>
<li>
<tt>`--client-bindir`=<b>path</b></tt>
The path to the directory where client binaries are located.
</li>
<li>
<tt>`--client-dbx`</tt>
Start @ref PAGE_MYSQLTEST "mysqltest" in the <b>dbx</b>
debugger.
</li>
<li>
<tt>`--client-ddd`</tt>
Start @ref PAGE_MYSQLTEST "mysqltest" in the <b>ddd</b>
debugger.
</li>
<li>
<tt>`--client-debugger`=<b>debugger</b></tt>
Start @ref PAGE_MYSQLTEST "mysqltest" in the named debugger.
</li>
<li>
<tt>`--client-gdb`</tt>
Start @ref PAGE_MYSQLTEST "mysqltest" in the <b>gdb</b> debugger.
</li>
<li>
<tt>`--client-libdir`=<b>path</b></tt>
The path to the directory where client libraries are located.
</li>
<li>
<tt>`--colored-diff`</tt>
Colorize the diff part of the output.
When this option is enabled, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" passes <b>`--colored-diff`</b> option to
@ref PAGE_MYSQLTEST "mysqltest". If colored diff is enabled,
@ref PAGE_MYSQLTEST "mysqltest" uses <b>diff</b> command with
<b>`--color`='always'</b> option to print the colored diff.
@note
<b>`--color`</b> option for <b>diff</b> command is available
from <b>GNU diffutils</b> version <b>3.4</b>.
</li>
<li>
<tt>`--combination`=<b>value</b></tt>
Extra option to pass to <b>mysqld</b>. The value should consist
of a single <b>mysqld</b> option including dashes. This option
is similar to <b>`--mysqld`</b> but has a different effect.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" executes
multiple test runs, using the options for each instance of
<b>`--combination`</b> in successive runs. If
<b>`--combination`</b> is given only once, it has no effect and
behaves similar to <b>`--mysqld`</b> option.
If a test being run, passes the same <b>`--mysqld`</b> option from an
opt file which is also passed from <b>`--combination`</b>, then that
<b>`--combination`</b> run is skipped because opt file always takes
higher precedence over command line options. The test will just run
once using the options provided in the opt file.
For test runs specific to a given test suite, an alternative to
the use of <b>`--combination`</b> is to create a
<b>combinations</b> file in the suite directory. The file
should contain a section of options for each test run. See @ref
PAGE_PASSING_OPTIONS.
For test specific combinations, onc can use
<b><em>test_name</em>.combinations</b> file in the test
directory.
</li>
<li>
<tt>`--comment`=<b>str</b></tt>
Write <b><em>str</em></b> to the output within lines filled
with #, as a form of banner.
</li>
<li>
<tt>`--compress`</tt>
Compress all information sent between the client and the server
if both support compression.
</li>
<li>
<tt>`--cursor-protocol`</tt>
Pass the <b>`--cursor-protocol`</b> option to @ref
PAGE_MYSQLTEST "mysqltest" (implies <b>`--ps-protocol`</b>).
</li>
<li>
<tt>`--dbx`</tt>
Start <b>mysqld</b> in the <b>dbx</b> debugger.
</li>
<li>
<tt>`--ddd`</tt>
Start <b>mysqld</b> in the <b>ddd</b> debugger.
</li>
<li>
<tt>`--debug`</tt>
Dump trace output for all clients and servers.
</li>
<li>
<tt>`--debugger`=<b>debugger</b></tt>
Start <b>mysqld</b> using the named debugger.
</li>
<li>
<tt>`--debug-common`</tt>
This option works similar to <b>`--debug`</b> but turns on
debug only for the debug macro keywords <b>query</b>,
<b>info</b>, <b>error</b>, <b>enter</b>, <b>exit</b>
which are considered the most commonly used.
</li>
<li>
<tt>`--debug-server`</tt>
Runs <b>mysqld.debug</b> (if available) instead of
<b>mysqld</b> as server. If it does find <b>mysqld.debug</b>,
it will search for plugin libraries in a subdirectory
<b>debug</b> under the directory where it's normally located.
This option does not turn on trace output and is independent of
the <b>debug</b> option.
</li>
<li>
<tt>`--debug-sync-timeout`=<b>seconds</b></tt>
Controls whether the Debug Sync facility for testing and
debugging is enabled. The option value is a timeout in seconds.
The default value is 300. A value of 0 disables Debug Sync. The
value of this option also becomes the default timeout for
individual synchronization points.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" passes
<b>`--loose-debug-sync-timeout`=<em>seconds</em></b> to
<b>mysqld</b>. The <b>`--loose`</b> prefix is used so that
<b>mysqld</b> does not fail if Debug Sync is not compiled in.
For information about using the Debug Sync facility for testing,
see @ref PAGE_THREAD_SYNCHRONIZATION.
</li>
<li>
<tt>`--default-myisam`</tt>
Use <b>MyISAM</b> as the default storage engine for all except
<b>InnoDB</b>-specific tests. This option is off by default.
</li>
<li>
<tt>`--defaults-file`=<b>file_name</b></tt>
Use the named file as fixed config file template for all tests.
</li>
<li>
<tt>`--defaults_extra_file`=<b>file_name</b></tt>
Add setting from the named file to all generated configs.
</li>
<li>
<tt>`--do-suite`=<b>prefix or regex</b></tt>
Run all test cases from suites having a name that begins with
the given <b><em>prefix</em></b> value or matches the regular
expression. If the argument matches no existing suite, @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" aborts.
The argument for the <b>`--do-suite`</b> option allows more
flexible specification of which tests to perform. See the
description of the <b>`--do-test`</b> option for details.
In case a suite or a list of suites is specified using the
<b>`--suite`</b> option along with a pattern or regular
expression given through <b>`--do-suite`</b>, only suite(s)
from the list which match the prefix or regular expression
are picked up.
When the <b>`--do-suite`</b> option is specified more than once
on the command line, only the last occurrence is considered.
</li>
<li>
<tt>`--do-test`=<b>prefix or regex</b></tt>
Run all test cases having a name that begins with the given
<b><em>prefix</em></b> value or matches the regular expression.
This option provides a convenient way to run a family of
similarly named tests.
The argument for the <b>`--do-test`</b> option allows more
flexible specification of which tests to perform. If the
argument contains a pattern metacharacter other than a lone
period, it is interpreted as a Perl regular expression and
applies to test names that match the pattern. If the argument
contains a lone period or does not contain any pattern
metacharacters, it is interpreted the same way as previously and
matches test names that begin with the argument value. For
example, <b>`--do-test`=testa</b> matches tests that begin with
<b>testa</b>, <b>`--do-test`=main.testa</b> matches tests in the
<b>main</b> test suite that begin with <b>testa</b>, and
<b>`--do-test`=main.*testa</b> matches test names that contain
<b>main</b> followed by <b>testa</b> with anything in between.
In the latter case, the pattern match is not anchored to the
beginning of the test name, so it also matches names such as
<b>xmainytestz</b>.
</li>
<li>
<tt>`--do-test-list`=<b>file</b></tt>
Run all tests listed in the file <b><em>file</em></b>. In this
file, tests should be listed one per line in any of the
following forms.
@verbatim
<test_name>
<test_name>.test
<suite_name>.<test_name>
<suite_name>.<test_name>.test
path_to_the_test_file
@endverbatim
Path to the test file should either be an absolute path or a
relative path from base directory.
To run a test present in a non-default test suite, either the
test name qualified with the suite name i.e
<b>suite_name.test_name[.test]</b> or a <b>path</b> to the
test file should be mentioned.
A line beginning with <b>#</b> will be ignored and is considered
as a comment. Similarly an empty line in the file is also
ignored.
For example, consider a test list file <b>test_list</b>
containing following tests.
@verbatim
# Main suite test '1st.test'
1st
1st.test
main.1st
main.1st.test
mysql-test/t/1st.test
# GIS suite test 'geohash_functions.test'
gis.geohash_functions
mysql-test/suite/gis/t/geohash_functions.test
# Non-default suite 'test_services'
test_services.test_services
mysql-test/suite/test_services/t/test_services.test
# Internal suite test 'i_main.user_var'
i_main.user_var
internal/mysql-test/suite/i_main/t/user_var.test
@endverbatim
Following command runs the list of tests mentioned in the file.
@verbatim
./mysql-test-run.pl --do-test-list=test_list
@endverbatim
If both <b>`--suite`</b> and <b>`--do-test-list`</b> are given,
and if suite name is <em>not</em> part of the test names listed
in the file, then those tests are searched only in the suites
mentioned in <b>`--suite`</b> option.
</li>
<li>
<tt>`--enable-disabled`</tt>
Ignore any <b>disabled.def</b> file, and run also tests marked
as disbaled. Success or failure of those tests will be reported
the same way as other tests.
@note
This option is enabled by default when test cases are specified
on command line.
</li>
<li>
<tt>`--explain-protocol`</tt>
Run <b>EXPLAIN EXTENDED</b> on all <b>SELECT</b>, <b>INSERT</b>,
<b>REPLACE</b>, <b>UPDATE</b> and <b>DELETE</b> queries.
</li>
<li>
<tt>`--extern` <b>option=value</b></tt>
Use an already running server. The option/value pair is what is
needed by the <b>mysql</b> client to connect to the server. Each
<b>`--extern`</b> can only take one option/value pair as
argument, so it you need more you need to repeat
<b>`--extern`</b> for each of them.
@verbatim
./mysql-test-run.pl --extern socket=var/tmp/mysqld.1.sock main.1st
@endverbatim
During server initialization, @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" creates a default database <b>test</b> which
is used while establishing an initial connection with the
server. When running a test case with <b>--extern</b> option, if
<b>test</b> database doesn't exist on an already running server,
then the test case will fail with following error.
@verbatim
mysqltest: Could not open connection 'default': 1049 Unknown database 'test'
@endverbatim
So if you're running a test case on an already running server
using <b>--extern</b> option, it is necessary to create
<b>test</b> database on the server before running the test case
if it doesn't exist.
Similarly, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl"
creates <b>mtr</b> database during server initialization which
contains tables and procedures used to suppress global or test
specific warnings. If a test case refers to this database and
the database doesn't exist on running server, then the test case
will fail.
@note
If a test case has an <b>.opt</b> file that requires the server
to be restarted with specific options, the file will not be
used. The test case likely will fail as a result.
</li>
<li>
<tt>`--fast`</tt>
Do not perform controlled shutdown when servers need to be
restarted or at the end of the test run. This is equivalent to
using <b>`--shutdown-timeout`=0</b>.
</li>
<li>
<tt>`--force`</tt>
Normally, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" exits
if a test case fails. <b>`--force`</b> causes execution to
continue regardless of test case failure.
</li>
<li>
<tt>`--force-restart`</tt>
Always restart the server(s) between each tast case, whether
it's needed or not. Will also restart between repeated runs of
the same test case. This may be useful e.g. when looking for the
source of a memory leak, as there will only have been one test
run before the server exits.
Enabling this option will cause @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" to restart the server(s) on a newly
intialized data directory.
This option can also be mentioned in
<b><em>test-name</em>-master.opt</b> file for a test case to
restart the server(s) before it runs.
</li>
<li>
<tt>`--gcov`</tt>
Run tests with the <b>gcov</b> test coverage tool.
</li>
<li>
<tt>`--gdb`</tt>
Start <b>mysqld</b> in the <b>gdb</b> debugger.
</li>
<li>
<tt>`--gprof`</tt>
Run tests with the <b>gprof</b> profiling tool.
</li>
<li>
<tt>`--include-ndbcluster`, `--include-ndb`</tt>
Run also tests that need Cluster.
</li>
<li>
<tt>`--initialize`=<b>value</b></tt>
Extra bootstrap options that need be to be passed during
the server initialization process can be passed as a value
to <b>`--initialize`</b>. The server is then started with
the specified value. Only one option may be specified in
<b>value</b>; to specify more than one, use additional
<b>`--initialize`</b> options.
Options passed as a value to <b>`--mysqld`</b> will have
precedence over the options passed to <b>`--initialize`</b>,
regardless of the order in which they passed on the command
line.
</li>
<li>
<tt>`--json-explain-protocol`</tt>
Run <b>EXPLAIN FORMAT=JSON</b> on all <b>SELECT</b>,
<b>INSERT</b>, <b>REPLACE</b>, <b>UPDATE</b> and <b>DELETE</b>
queries.
</li>
<li>
<tt>`--manual-boot-gdb`</tt>
This option is similar to <b>`--boot-gdb`</b> but attaches the
debugger to the server during the bootstrapping process,
permitting the use of a remote debugger.
</li>
<li>
<tt>`--manual-dbx`</tt>
Use a server that has already been started by the user in the
<b>dbx</b> debugger.
</li>
<li>
<tt>`--manual-ddd`</tt>
Use a server that has already been started by the user in the
<b>ddd</b> debugger.
</li>
<li>
<tt>`--manual-debug`</tt>
Use a server that has already been started by the user in a
debugger.
</li>
<li>
<tt>`--manual-gdb`</tt>
Use a server that has already been started by the user in the
<b>gdb</b> debugger.
</li>
<li>
<tt>`--mark-progress`</tt>
Marks progress with timing (in milliseconds) and line number in
<b>var/log/<em>testname</em>.progress</b>.
</li>
<li>
<tt>`--max-connections`=<b>num</b></tt>
The maximum number of simultaneous server connections that may
be used per test. If not set, the maximum is 128. Minimum
allowed limit is 8, maximum is 5120. Corresponds to the same
option for @ref PAGE_MYSQLTEST "mysqltest".
</li>
<li>
<tt>`--max-save-core`=<b>N</b></tt>
Limit the number of core files saved, to avoid filling up disks
in case of a frequently crashing server. Defaults to 5, set to 0
for no limit. May also be set with the environment variable
<tt>MTR_MAX_SAVE_CORE</tt>.
</li>
<li>
<tt>`--max-save-datadir`=<b>N</b></tt>
Limit the number of data directories saved after failed tests,
to avoid filling up disks in case of frequent failures. Defaults
to 20, set to 0 for no limit. May also be set with the
environment variable <b>MTR_MAX_SAVE_DATADIR</b>.
</li>
<li>
<tt>`--max-test-fail`=<b>N</b></tt>
Stop execution after the specified number of tests have failed,
to avoid using up resources (and time) in case of massive
failures. retries are noe counted. Defaults to 10, set to 0 for
no limit. May also be set with the environment variable
<b>MTR_MAX_TEST_FAIL</b>.
</li>
<li>
<tt>`--mem`</tt>
This option is not supported on Windows and on MacOS.
Run the test suite in memory, using tmpfs or ramdisk. This can
decrease test times significantly, in particular if you would
otherwise be running over a remote file system. @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" attempts to find a
suitable location using a built-in list of standard locations
for tmpfs and puts the <b>var</b> directory there. This option
also affects placement of temporary files, which are created in
<b>var/tmp</b>.
The default list includes <b>/dev/shm</b>, <b>/run/shm</b> and
<b>/tmp</b>. You can also enable this option by setting the
environment variable <b>MTR_MEM[=<em>dir_name</em>]</b>. If
<b><em>dir_name</em></b> is given, it is added to the beginning
of the list of locations to search, so it takes precedence over
any built-in locations.
Once you have run tests with <b>`--mem`</b> within a
<b>mysql-test</b> directory, a soflink <b>var</b> will have been
set up to the temporary directory, and this will be re-used the
next time, until the soflink is deleted. Thus, you do not have
to repeat the <b>`--mem`</b> option next time.
</li>
<li>
<tt>`--mysqld`=<b>value</b></tt>
Extra option to pass to <b>mysqld</b>. Only one option may be
specified in <b>value</b>; to specify more than one, use
additional <b>--mysqld</b> options. See @ref PAGE_PASSING_OPTIONS.
</li>
<li>
<tt>`--mysqld-env`=<b>variable</b>=<b>value</b></tt>
Sets (or changes) an environment variable before starting
<b>mysqld</b>. Varibles set in the environment from which you
run @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will
normally also be propagated to <b>mysqld</b>, but there may be
cases where you want a setting just for a single run, or you may
not want the setting to affect other programs. You may use
additional <b>`--mysqld-env`</b> options to set more than one
variable.
</li>
<li>
<tt>`--mysqltest`=<b>options</b></tt>
Extra options to pass to @ref PAGE_MYSQLTEST "mysqltest".
</li>
<li>
<tt>`--ndb-connectstring`=<b>str</b></tt>
Pass <b>`--ndb-connectstring`=<em>str</em></b> to the master
MySQL server. This option also prevents @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" from starting a
cluster. It is assumed that there is already a cluster running
to which the server can connect with the given connectstring.
</li>
<li>
<tt>`--no-skip`</tt>
This option forces all tests to run, ignoring any
<b>`--skip`</b> commands used in the test. This ensures that all
tests are run. An excluded list (<b>excludenoskip.list</b>) is
maintained to track which tests should continue to be skipped.
The <b>`--no-skip`</b> option continues to skip the tests that
are named in the excluded list. The default value of
<b>--no-skip</b> introduced variable is OFF, which implies users
are not forced to run all tests unless the <b>--no-skip</b> is
explicitly used.
@verbatim
shell> mysql-test-run.pl --suite=innodb --no-skip
@endverbatim
</li>
<li>
<tt>`--nocheck-testcases`</tt>
Disable the check for test case side effects. For additional
information, see the description of the
<b>`--check-testcases`</b> option.
</li>
<li>
<tt>`--noreorder`</tt>
Do not reorder tests to reduce number of restarts, but run them
in exactly the order given. If a whole suite is to be run, the
tests are run in alphabetic order, though similiar combinations
will be grouped together. If more than one suite is listed, the
tests are run one suite at a time, in the order listed.
</li>
<li>
<tt>`--notimer`</tt>
Cause @ref PAGE_MYSQLTEST "mysqltest" not to generate a timing
file. The effect of this is that the report from each test case
does not include the timing in milliseconds as it normally does.
</li>
<li>
<tt>`--nounit-tests`</tt>
Do not run unit tests, overriding default behavior or setting of
the <b>MTR_UNIT_TESTS</b> variable.
</li>
<li>
<tt>`--nowarnings`</tt>
Do not look for and report errors and warning in the server logs.
@note
In addition to being passed as a command-line option,
<b>`--nowarnings`</b> can also be passed in the <b>.opt</b> file of
a test case.
</li>
<li>
<tt>`--only-big-tests`</tt>
This option causes only big tests to run. Normal (non-big) tests
are skipped. If both <b>`--big-test`</b> and
<b>`--only-big-tests`</b> are given, <b>`--only-big-tests`</b>
is ignored.
</li>
<li>
<tt>`--opt-trace-protocol`</tt>
Prints the @ref PAGE_OPT_TRACE "optimizer trace" of SQL statements in a test.
Running any MTR test with this option executes the following SQL
statement after every <b>DML statement</b> that it encounters:
@verbatim
SELECT trace FROM information_schema.optimizer_trace /* injected by --opt-trace-protocol */;
@endverbatim
</li>
<li>
<tt>`--parallel`={<b>N</b>|auto}</tt>
Run tests using <b><em>N</em></b> parallel threads. By default,
1 thread is used. Use <b>`--parallel`=auto</b> to set
<b><em>N</em></b> automatically.
Setting the <b>MTR_PARALLEL</b> environment variable to
<b><em>N</em></b> has the same effect as specifying
<b>`--parallel`=<em>N</em></b>.
The <b>MTR_MAX_PARALLEL</b> environment variable, if set,
specifies the maximum number of parallel workers that can be
spawned when the <b>`--parallel`=auto</b> option is specified.
If <b>`--parallel`=auto</b> is not specified,
<b>MTR_MAX_PARALLEL</b> variable has no effect.
</li>
<li>
<tt>`--port-base`=<b>P</b></tt>
Specify base of port numbers to be used; a block of 10 will be
allocated. <b><em>P</em></b> should be divisible by 10; if it is
not, it will be rounded down. If running with more than one
parallel test thread, thread 2 will use the next block of 10 and
so on.
If the port number is given as <b>auto</b>, which is also the
default, @ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will
allocate a number unique to this host. The value may also be
given with the environment variable <b>MTR_PORT_BASE</b>.
<b>`--port-base`</b> was added as a more logical alternative to
<b>`--build-thread`</b>. If both are used, <b>`--port-base`</b>
takes precedence.
</li>
<li>
<tt>`--print-testcases`</tt>
Do not run any tests, but print details about all tests, in the
order they would have been run.
</li>
<li>
<tt>`--ps-protocol`</tt>
Pass the <b>`--ps-protocol`</b> option to @ref PAGE_MYSQLTEST
"mysqltest".
</li>
<li>
<tt>`--quiet`</tt>
Reuse the output buffer and maintain a single line for reporting
successful tests, skipped tests and disabled tests. Failed tests
and the necessary information about the failure will be printed
on a separate line.
When <b>`--quiet`</b> option is enabled, MTR prints only the
test name and result of the test run.
</li>
<li>
<tt>`--record`</tt>
Pass the <b>`--record`</b> option to @ref PAGE_MYSQLTEST
"mysqltest". This option requires a specific test case to be
named on the command line.
</li>
<li>
<tt>`--reorder`</tt>
Reorder tests to minimize the number of server restarts needed.
This is the default behavior. There is no guarantee that a
particular set of tests will always end up in the same order.
</li>
<li>
<tt>`--repeat`=<b>N</b></tt>
Run each test <b><em>N</em></b> number of times, in parallel if
<b>`--parallel`</b> option value is greater than 1.
</li>
<li>
<tt>`--report-features`</tt>
Display the output of <b>SHOW ENGINES</b> and <b>SHOW
VARIABLES</b>. This can be used to verify that binaries are
built with all required features.
</li>
<li>
<tt>`--report-times`</tt>
At the end of the test run, write a summary of how much time was
spent in various phases of execution. If you run with
<b>`--parallel`</b>, the total will exceed the wall clock time
passed, since it will be summed over all threads.
The times reported should only be treated as approximations, and
the exact points where the time is taken may also change between
releases. If the test run is aborted, including if a test fails
and <b>`--force`</b> is not in use, the time report will not
be produced.
</li>
<li>
<tt>`--report-unstable-tests`</tt>
The option can be used to distinguish unstable tests which pass
on at least one retry attempt from hard failing ones. Unstable
tests are reported separately in the end summary in the format:
<b>test_name(num_of_failures/num_of_attempts)</b>.
In case all failures encountered are due to unstable tests, MTR
will print the below warning and exit with a zero status code.
@verbatim
mysql-test-run: WARNING: There are failures due to unstable test cases.
However, the tests are not hard-failing.
@endverbatim
This option has no effect unless <b>`--force`</b> is used and
both <b>`--retry`</b> and <b>`--retry-failure`</b> are set to
values greater than 1.
</li>
<li>
<tt>`--retry`=<b>N</b></tt>
If a test fails, it is retried up to a maximum of
<b><em>N</em></b> runs, but will terminate after 2 failures.
Default is 3, set to 1 or 0 for no retries. This option has no
effect unless <b>`--force`</b> is also used; without it, test
execution will terminate after the first failure.
The <b>`--retry`</b> and <b>`--retry-failure`</b> options do not
affect how many times a test repeated with <b>`--repeat`</b> may
fail in total, as each repetition is considered a new test case,
which may in turn be retried if it fails.
</li>
<li>
<tt>`--retry-failure`=<b>N</b></tt>
Allow a failed and retried test to fail more than the default 2
times before giving it up. Setting it to 0 or 1 effectively
turns off retries
</li>
<li>
<tt>`--non-parallel-test`</tt>
Allow tests marked as "non-parallel" to run. Tests can be thus
marked by including the line
<b>`--source` include/not_parallel.inc</b>, and they will only
be run if this option is enabled.
</li>
<li>
<tt>`--sanitize`</tt>
Scan the server log files for warnings from various sanitizers.
Use of this option assumes that MySQL was configured with
[-DWITH_ASAN]
(https://dev.mysql.com/doc/refman/8.0/en/source-configuration-options.html#option_cmake_with_asan)
or [-DWITH_UBSAN]
(https://dev.mysql.com/doc/refman/8.0/en/source-configuration-options.html#option_cmake_with_ubsan).
The <b>TSAN_OPTIONS</b> environment variable can be set to
specify the path name of a file containing ThreadSanitizer
suppressions.
</li>
<li>
<tt>`--shutdown-timeout`=<b>seconds</b></tt>
Max number of seconds to wait for servers to do controlled
shutdown before killing them. Default is 10.
</li>
<li>
<tt>`--skip-combinations`</tt>
Do not apply combinations; ignore combinations file or option.
</li>
<li>
<tt>`--skip-ndbcluster`, `--skip-ndb`</tt>
Do not start NDB Cluster; skip Cluster test cases. This option
only has effect if you do have NDB, if not it will have no
effect as it cannot run those tests anyway.
</li>
<li>
<tt>`--skip-ndbcluster-slave`, `--skip-ndb-slave`</tt>
Do not start an NDB Cluster slave.
</li>
<li>
<tt>`--skip-rpl`</tt>
Skip replication test cases.
</li>
<li>
<tt>`--skip-test`=<b>regex</b></tt>
Specify a regular expression to be applied to test case names.
Cases with names that match the expression are skipped. tests to
skip.
The argument for the <b>`--skip-test`</b> option allows more
flexible specification of which tests to skip. If the argument
contains a pattern metacharacter other than a lone period, it is
interpreted as a Perl regular expression and applies to test
names that match the pattern. See the description of the
<b>`--do-test`</b> option for details.
</li>
<li>
<tt>`--skip-test-list`=<b>file</b></tt>
Specify a file listing tests that should be skipped (disabled).
The file has the same format as the <b>disabled.def</b> file
listing disabled tests. With this option, disabling can be done
on a case by case basis.
</li>
<li>
<tt>`--skip-*`</tt>
<b>`--skip-*`</b> options not otherwise recognized by @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" are passed to the
master server.
</li>
<li>
<tt>`--sleep`=<b>N</b></tt>
Pass <b>`--sleep`=<em>N</em></b> to @ref PAGE_MYSQLTEST
"mysqltest".
</li>
<li>
<tt>`--sp-protocol`</tt>
Pass the <b>`--sp-protocol`</b> option to @ref PAGE_MYSQLTEST
"mysqltest".
</li>
<li>
<tt>`--start`</tt>
Initialize and start servers with the startup settings for the
specified test case. You can use this option to start a server
to which you can connect later. For example, after building a
source distribution you can start a server and connect to it
with the <b>mysql</b> client like this:
@verbatim
shell> cd mysql-test
shell> ./mysql-test-run.pl --start alias &
shell> ../mysql -S ./var/tmp/master.sock -h localhost -u root
@endverbatim
If no tests are named on the command line, the server(s) will be
started with settings for the first test that would have been
run without the <b>`--start`</b> option.
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will stop once
the server has been started, but will terminate if the server
dies. If killed, it will also shut down the server.
</li>
<li>
<tt>`--start-and-exit`</tt>
This is similar to <b>`--start`</b>, but @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" terminates once the
server has been started, leaving just the server process
running.
</li>
<li>
<tt>`--start-dirty`</tt>
This is similar to <b>`--start`</b>, but will skip the
database initialization phase and assume that database files are
already available. Usually this means you must have run another
test first.
</li>
<li>
<tt>`--start-from`=<b>test_name</b></tt>
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" sorts the list
of names of the test cases to be run, and then begins with
<b><em>test_name</em></b>.
</li>
<li>
<tt>`--strace-client`</tt>
Create <b>strace</b> output for @ref PAGE_MYSQLTEST "mysqltest".
Will produce default <b>strace</b> output as
<b>mysqltest.strace</b>. Note that this will be overwritten
for each new test case, so it's most useful for running only one
test.
The <b>strace-client</b> option is only supported on Linux.
</li>
<li>
<tt>`--strace-server`</tt>
Create <b>strace</b> output for the server. Will produce default
<b>strace</b> output as <b>mysqld.1.strace</b>. Note that this
will be overwritten each time the server is restarted, so it's
most useful for running a single test, or if you want trace from
the first test that fails.
The <b>strace-server</b> option is available on Linux only.
</li>
<li>
<tt>`--stress`=<b>stress options</b></tt>
Start a server, but instead of running a test, run @ref
PAGE_MYSQLTEST "mysql-stress-test.pl" with the supplied
arguments. Arguments needed to communicate with the server will
be automatically provided, the rest should be given as arguments
to this option. Command line options for @ref PAGE_MYSQLTEST
"mysql-stress-test.pl" should be separeted by a comma.
</li>
<li>
<tt>`--suite(s)`=<b>{suite_name|suite_list|suite_set}</b></tt>
Run a suite or a comma separated list of suites or those suites
which fall under the specified suite set. If the option is not
specified explicitly, MTR assumes that it needs to use the
default suite set for the test run.
The suite set's MTR can use are case insensitive and take the
below values:
@verbatim
default - To run the default list of suites.
all - To scan the mysql directory and run all available suites.
non-default - To scan the mysql directory for available suites and
collect only the non-default suites.
@endverbatim
The suite set 'all' is used implicitly if neither a suite nor a
suite list is specified and\n
(i) There are tests given on the MTR command line <b>or</b>\n
(ii) Any of the following options are used: <b>--do-suite</b>,
<b>--do-test</b>, <b>--do-test-list</b>.
Note that suite sets 'all' and 'non-default' do not include ndb test
suites in case ndb cluster is not enabled.
If a suite set is specified in the suite list, MTR does not treat it
as a suite set and considers it as the name of a suite to be run.
</li>
<li>
<tt>`--suite-timeout`=<b>minutes</b></tt>
Specify the maximum test suite runtime in minutes.
</li>
<li>
<tt>`--summary-report`=<b>file_name</b></tt>
Generate a plain text version of the test summary only and write
it to the file named as the option argument. The file is
suitable for sending by email.
</li>
<li>
<tt>`--test-progress`[={0|1}]</tt>
Print the percentage of tests completed. This option is enabled
by default. The argument to <b>`--test-progress`</b> is
optional. To disable it, set the value to <b>0</b>.
For example, consider a test suite <b>sample_test_suite</b>
containing 3 tests.
@verbatim
./mysql-test-run.pl --suite=sample_test_suite [--test-progress]
@endverbatim
Running the suite with <b>`--test-progress`</b> option enabled
will print the percentage of tests completed.
@verbatim
[ 33%] sample_test_suite.1st [ pass ] 1
[ 66%] sample_test_suite.2nd [ pass ] 1
[100%] sample_test_suite.3rd [ pass ] 1
@endverbatim
</li>
<li>
<tt>`--testcase-timeout`=<b>minutes</b></tt>
Specify the maximum test case runtime in minutes.
@note
When a test case is run with <b>valgrind</b> option enabled, the
test case timeout value becomes 10 times the specified value.
For example, running a test case with timeout value
<b>1 minute</b> and <b>valgrind</b> enabled will increase the
timeout value to <b>10 minutes</b>.
</li>
<li>
<tt>`--timediff`</tt>
Adds to each test report for a test case, the total time in
seconds and milliseconds passed since the preceding test ended.
This option can only be used together with
<b>`--timestamp`</b>, and has no effect without it.
</li>
<li>
<tt>`--timer`</tt>
Cause @ref PAGE_MYSQLTEST "mysqltest" to generate a timing file.
The default file is named <b>./var/log/timer</b>.
</li>
<li>
<tt>`--timestamp`</tt>
Prints a timestamp before the test case name in each test report
line, showing when the test ended.
</li>
<li>
<tt>`--tmpdir`=<b>path</b></tt>
The directory where temporary file are stored. The default
location is <b>./var/tmp</b>. The environment variable
<b>MYSQL_TMP_DIR</b> will be set to the path for this directory,
whether it has the default value or has been set explicitly.
This may be referred to in tests.
</li>
<li>
<tt>`--unit-tests`</tt>
Force running of unit tests, overriding default behavior or
setting of the <b>MTR_UNIT_TESTS</b> variable.
</li>
<li>
<tt>`--unit-tests-report`</tt>
Extend the unit test run by also outputting the log from the
test run, independently of whether it succeeded or not. This
option implies <b>`--unit-tests`</b> so it is not necessary to
specify both.
</li>
<li>
<tt>`--user`=<b>user_name</b></tt>
The MySQL user name to use when connecting to the server.
</li>
<li>
<tt>`--user-args`</tt>
Drops all non-essential command line arguments to the
<b>mysqld</b> server, except those supplied with
<b>`--mysqld`</b> arguemnts, if any. Only works in combination
with <b>`--start`</b>, <b>`--start-and-exit`</b> or
<b>`--start-dirty`</b>, and only if no test name is given.
</li>
<li>
<tt>`--valgrind`</tt>
Run @ref PAGE_MYSQLTEST "mysqltest" and <b>mysqld</b> with
<b>valgrind</b>. This and the following <b>`--valgrind`</b>
options require that the executables have been build with
<b>valgrind</b> support.
When the server is run with valgrind, an extra pass over the
server log file(s) will be performed after all tests are run,
and any report with problems that have been reported at server
shutdown will be extracted and printed. The most common warnings
are memory leaks. With each report will also be listed all tests
that were run since previous server restart; one of these is
likely to have caused the problem.
A final "pseudo" test named <b>valgrind_report</b> is added to
the list of tests when the server is run in valgrind. This test
is reported as failed if any such shutdown warnings were
produced by valgrind. Pass or failure of this test is also added
to the total test count reported.
@note
When a test case is run with <b>valgrind</b> option enabled, the
test case timeout value becomes 10 times the specified value.
For example, running a test case with timeout value
<b>1 minute</b> and <b>valgrind</b> enabled will increase the
timeout value to <b>10 minutes</b>.
</li>
<li>
<tt>`--valgrind-clients`</tt>
Run all clients started by <b>.test</b> files with
<b>valgrind</b>. This option requires <b>valgrind 3.9</b> or
later.
</li>
<li>
<tt>`--valgrind-mysqld`</tt>
Run the <b>mysqld</b> server with <b>valgrind</b>.
</li>
<li>
<tt>`--valgrind-mysqltest`</tt>
Run @ref PAGE_MYSQLTEST "mysqltest" with <b>valgrind</b>.
</li>
<li>
<tt>`--valgrind-option`=<b>str</b></tt>
Extra options to pass to <b>valgrind</b>.
</li>
<li>
<tt>`--valgrind-path`=<b>path</b></tt>
Specify the path name to the valgrind executable.
</li>
<li>
<tt>`--vardir`=<b>path</b></tt>
Specify the path where files generated during the test run are
stored. The default location is <b>./var</b>. The environment
variable <b>MYSQLTEST_VARDIR</b> will be set to the path for
this directory, whether it has the default value or has been set
explicitly. This may be referred to in tests.
</li>
<li>
<tt>`--verbose`</tt>
Give more verbose output regarding test execution. Note that the
output generated within each test case is not affected.
</li>
<li>
<tt>`--verbose-restart`</tt>
Write when and why servers are restarted between test cases.
</li>
<li>
<tt>`--view-protocol`</tt>
Pass the <b>`--view-protocol`</b> option to @ref
PAGE_MYSQLTEST "mysqltest".
</li>
<li>
<tt>`--vs-config`=<b>config_val</b></tt>
Specify the configuration used to build MySQL (for example,
<b>--vs-config=debug --vs-config=release</b>). This option
is for Windows only.
</li>
<li>
<tt>`--wait-all`</tt>
If <b>`--start`</b> or <b>`--start-dirty`</b> is used, wait
for all servers to exit before termination. Otherise, it will
terminate if one (of several) servers is restarted.
</li>
<li>
<tt>`--warnings`</tt>
Search the server log for errors or warning after each test and
report any suspicious ones; if any are found, the test will be
marked as failed. This is the default behavior, it may be turned
off with <b>`--nowarnings`</b>.
</li>
<li>
<tt>`--with-ndbcluster-only`</tt>
Run only test cases that have <b>ndb</b> in their name.
</li>
<li>
<tt>`--xml-report`=<b>file_name</b></tt>
Generate an xml file containing result of the test run and write
it to the file named as the option argument. It includes
suite-wise and overall statistics like the number of tests run,
number of failures, number of disabled and skipped tests along
with information about each individual test according to its
outcome.
The xml report's structure can be found below :
@verbatim
<testsuites ... >
<testsuite ... >
<testcase ... />
<testcase ... >
<failure ... > ... </failure>
</testcase>
</testsuite>
</testsuites>
@endverbatim
</li>
</ul>
@note
The hostname resolves to 127.0.0.1 and not to the actual IP address.
*/
###############################################################################
## mysql-stress-test.pl — Server Stress Test Program
/**
@page PAGE_MYSQL_STRESS_TEST_PL mysql-stress-test.pl — Server Stress Test Program
The @ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" Perl
script performs stress-testing of the MySQL server.
@ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" requires
version of Perl that has been built with threads support.
Invoke @ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" like
this:
@verbatim
shell> mysql-stress-test.pl [options]
@endverbatim
@ref PAGE_MYSQL_STRESS_TEST_PL "mysql-stress-test.pl" supports the
following options:
<ul>
<li>
<tt>`--help`</tt>
Display a help message and exit.
</li>
<li>
<tt>`--abort-on-erro`r=<b>N</b></tt>
Causes the program to abort if an error with severity less than
or equal to N was encountered. Set to 1 to abort on any error.
</li>
<li>
<tt>`--check-tests-file`</tt>
Periodically check the file that lists the tests to be run. If
it has been modified, reread the file. This can be useful if you
update the list of tests to be run during a stress test.
</li>
<li>
<tt>`--cleanup`</tt>
Force cleanup of the working directory.
</li>
<li>
<tt>`--log-error-details`</tt>
Log error details in the global error log file.
</li>
<li>
<tt>`--loop-count`=<b>N</b></tt>
In sequential test mode, the number of loops to execute before
exiting.
</li>
<li>
<tt>`--mysqltest`=<b>path</b></tt>
The path name to the @ref PAGE_MYSQLTEST "mysqltest" program.
</li>
<li>
<tt>`--server-database`=<b>db_name</b></tt>
The database to use for the tests. The default is <b>test</b>.
</li>
<li>
<tt>`--server-host`=<b>host_name</b></tt>
The host name of the local host to use for making a TCP/IP
connection to the local server. By default, the connection is
made to <b>localhost</b> using a Unix socket file.
</li>
<li>
<tt>`--server-logs-dir`=<b>path</b></tt>
This option is required. path is the directory where all client
session logs will be stored. Usually this is the shared
directory that is associated with the server used for testing.
</li>
<li>
<tt>`--server-password`=<b>password</b></tt>
The password to use when connecting to the server.
</li>
<li>
<tt>`--server-port`=<b>port_num</b></tt>
The TCP/IP port number to use for connecting to the server.
The default is <b>3306</b>.
</li>
<li>
<tt>`--server-socket`=<b>file_name</b></tt>
For connections to <b>localhost</b>, the Unix socket file to
use, or, on Windows, the name of the named pipe to use. The
default if <b>/tmp/mysql.sock</b>.
</li>
<li>
<tt>`--server-user`=<b>user_name</b></tt>
The MySQL user name to use when connecting to the server. The
default is <b>root</b>.
</li>
<li>
<tt>`--sleep-time`=<b>N</b></tt>
The delay in seconds between test executions.
</li>
<li>
<tt>`--stress-basedir`=<b>path</b></tt>
This option is required. <b>path</b> is the working
directory for the test run. It is used as the temporary location
for result tracking during testing.
</li>
<li>
<tt>`--stress-datadir`=<b>path</b></tt>
The directory of data files to be used during testing. The
default location is the <b>data</b> directory under the
location given by the <b>`--stress-suite-basedir`</b> option.
</li>
<li>
<tt>`--stress-init-file`[=<b>path</b>]</tt>
<b>file_name</b> is the location of the file that
contains the list of tests to be run once to initialize the
database for the testing. If missing, the default file is
<b>stress_init.txt</b> in the test suite directory.
</li>
<li>
<tt>`--stress-mode`=<b>mode</b></tt>
This option indicates the test order in stress-test mode. The
<b>mode</b> value is either <b>random</b> to select tests in
random order or <b>seq</b> to run tests in each thread in the
order specified in the test list file. The default mode is
<b>random</b>.
</li>
<li>
<tt>`--stress-suite-basedir`=<b>path</b></tt>
This option is required. <b>path</b> is the directory
that has the <b>t</b> and <b>r</b> subdirectories containing
the test case and result files. This directory is also the
default location of the <b>stress-test.txt</b> file that
contains the list of tests. (A different location can be
specified with the <b>`--stress-tests-file`</b> option).
</li>
<li>
<tt>`--stress-tests-file`[=<b>file_name</b>]</tt>
Use this option to run the stress tests. <b>file_name</b>
is the location of the file that contains the list of tests. If
<b>file_name</b> is omitted, the default file is
<b>stress-test.txt</b> in the stress suite directory. (See
<b>`--stress-suite-basedir`</b>).
</li>
<li>
<tt>`--suite`=<b>suite_name</b></tt>
Run the named test suite. The default name is <b>main</b> (the
regular test suite located in the mysql-test directory).
</li>
<li>
<tt>`--test-count`=<b>N</b></tt>
The number of tests to execute before exiting.
</li>
<li>
<tt>`--test-duration`=<b>N</b></tt>
The duration of stress testing in seconds.
</li>
<li>
<tt>`--threads`=<b>N</b></tt>
The number of threads. The default is 1.
</li>
<li>
<tt>`--verbose`</tt>
Verbose mode. Print more information about what the program does.
</li>
</ul>
*/
###############################################################################
## mysqltest Language Reference
/**
@page PAGE_MYSQLTEST_LANGUAGE_REFERENCE mysqltest Language Reference
<h3>Table of Contents</h3>
- @subpage PAGE_MYSQL_TEST_INPUT_CONVENTIONS
- @subpage PAGE_MYSQL_TEST_COMMANDS
- @subpage PAGE_MYSQL_TEST_VARIABLES
- @subpage PAGE_MYSQL_TEST_FLOW_CONTROL
- @subpage PAGE_ERROR_HANDLING
This chapter describes the test language implemented by @ref
PAGE_MYSQLTEST "mysqltest". The language allows input to contain a
mix of comments, commands executed by @ref PAGE_MYSQLTEST
"mysqltest" itself, and SQL statements that @ref PAGE_MYSQLTEST
"mysqltest" sends to a MySQL server for execution.
Terminology notes:
------------------
<ul>
<li>
A “command” is an input test that @ref PAGE_MYSQLTEST
"mysqltest" recognizes and executes itself. A “statement” is
an SQL statement or query that @ref PAGE_MYSQLTEST "mysqltest"
sends to the MySQL server to be executed.
</li>
<li>
When @ref PAGE_MYSQLTEST "mysqltest" starts, it opens a
connection it calls <b>default</b> to the MySQL server, using
any connection parameters specified by the command options. (For
a local server, the default user name is <b>root</b>. For an
external server, the default user name is <b>test</b> or the
user specified with the <b>`--user`</b> option.) You can use
the <b>connect</b> command to open other connections, the
<b>connection</b> command to switch between connections, and
the <b>disconnect</b> command to close connections. However,
the capability for switching connections means that the
connection named <b>default</b> need not be the connection in
use at a given time. To avoid ambiguity, this document avoids
the term “default connection”. It uses the term “current
connection” to mean “the connection currently in use”, which
might be different from “the connection named <b>default</b>”.
</li>
</ul>
*/
###############################################################################
## mysqltest Input Conventions
/**
@page PAGE_MYSQL_TEST_INPUT_CONVENTIONS mysqltest Input Conventions
@ref PAGE_MYSQLTEST "mysqltest" reads input lines and processes
them as follows:
<ul>
<li>
“End of line” means a newline (linefeed) character. A carriage
return/linefeed (CRLF) pair also is allowable as as a line
terminator (the carriage return is ignored). Carriage return by
itself is not allowed as a line terminator.
</li>
<li>
A line that begins with "#" as the first nonwhitespace content
is treated as a comment that extends to the end of the line and
is ignored. Example:
@verbatim
# this is a comment
@endverbatim
</li>
<li>
Earlier versions would also allow comments beginning with “`--`”
unless the first word was a valid @ref PAGE_MYSQLTEST
"mysqltest" command, but this has been deprecated and is longer
allowed.
</li>
<li>
Other input is taken as normal command input. The command
extends to the next occurrence of the command delimiter, which
is semicolon (“<b>;</b>”) by default. The delimiter can be
changed with the <b>delimiter</b> command.
If @ref PAGE_MYSQLTEST "mysqltest" recognizes the first word of
the delimiter-terminated command, @ref PAGE_MYSQLTEST
"mysqltest" executes the command itself. Otherwise, @ref
PAGE_MYSQLTEST "mysqltest" assumes that the command is an SQL
statement and sends it to the MySQL server to be executed.
Because the command extends to the delimiter, a given input line
can contain multiple commands, and a given command can span
multiple lines. The ability to write multiple-line statements is
useful for making long statements more readable, such as a
<b>create table</b> statement for a table that has many
columns.
</li>
</ul>
After @ref PAGE_MYSQLTEST "mysqltest" reads a command up to a
delimiter and executes it, input reading restarts following the
delimiter and any remaining input on the line that contains the
delimiter is treated as though it begins on a new line. Consider the
following two input lines:
@verbatim
echo issue a select statement; select 1; echo done
issuing the select statement;
@endverbatim
That input contains two commands and one SQL statement:
@verbatim
echo issue a SELECT statement
SELECT 1;
echo done issuing the SELECT statement
@endverbatim
Similarly, "#" comments can begin on a command line following
a delimiter:
@verbatim
SELECT 'hello'; # select a string value
@endverbatim
On a multiple-line command, "#" or “`--`” at the beginning of the
second or following lines is not special. Thus, the second and third
lines of the following variable-assignment command are not taken as
comments. Instead, the variable <b>$a</b> is set to a value that
contains two linefeed characters:
@verbatim
let $a= This is a variable
# assignment that sets a variable
-- to a multiple-line value;
@endverbatim
`--` commands and normal commands have complementary properties with
regard to how @ref PAGE_MYSQLTEST "mysqltest" reads them:
- A “`--`” command is terminated by a newline, regardless of how
many delimiters it contains.
- A normal command (without “`--`”) is terminated by the delimiter
(semicolon), no matter how many newlines it contains.
@ref PAGE_MYSQLTEST "mysqltest" commands can be written either with
a leading “`--`”) or as normal command input (no leading “`--`”).
Use the command delimiter only in the latter case. Thus, these two
lines are equivalent:
@verbatim
--sleep 2
sleep 2;
@endverbatim
The equivalence is true even for the <b>delimiter</b> command. For
example, to set the delimiter to “<b>//</b>”, either of these
commands work:
@verbatim
--delimiter //
delimiter //;
@endverbatim
To set the delimiter back to “;”, use either of these commands:
@verbatim
--delimiter ;
delimiter ;//
@endverbatim
A potential ambiguity occurs because a command line can contain
either a @ref PAGE_MYSQLTEST "mysqltest" command or an SQL
statement. This has a couple of implications:
- No @ref PAGE_MYSQLTEST "mysqltest" command should be the same
as any keyword that can begin an SQL statement.
- Should extensions to SQL be implemented in the future, it is
possible that a new SQL keyword could be impossible for @ref
PAGE_MYSQLTEST "mysqltest" to recognize as such if that keyword
is already used as a @ref PAGE_MYSQLTEST "mysqltest" command.
Any ambiguity can be resolved by using the “`--`” syntax to force
interpetation as a @ref PAGE_MYSQLTEST "mysqltest" command, or the
<b>query</b> command to force interpretation as SQL.
All file paths used in test commands should use forward slash
\"<b>/</b>\" as the directory separator as in Unix. They will be
automatically converted when needed if the test is run on Windows.
We also recommend putting all temporary or auxiliary files made
during the test under the directory referred to by
<b>$MYSQL_TMP_DIR</b>. Do not put them under fixed full paths like
<b>/tmp</b>. This will help ensuring portability of the test, and
avoiding conflicts with other programs.
<b>$MYSQL_TMP_DIR</b> is equivalent to
<b>$MYSQLTEST_VARDIR/tmp</b> if you are not running with parallel
test threads, but if you run @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl" with <b>`--parallel`</b>, they will be different.
It is therefore best to be consistent and use <b>$MYSQL_TMP_DIR</b>.
Commands named <b>disable_X</b> or <b>enable_X</b>, except
<b>parsing</b>, <b>reconnect</b> and <b>rpl_parse</b>, can take an
optional modifier <b>ONCE</b>. If this is added, the relevant
setting is enabled or disabled only for the next command or
statement, after which it is reverted to whatever it was before.
Note that the word <b>ONCE</b> must be in upper case; this was
chosen in order to make it more visible when reading the test
script.
For example, <b>`--disable_query_log`</b> <b>ONCE</b> will
ensure query log is disabled for the next statement, but will not
affect whether or not query log is enabled for statements following
the next. It is possible to enable/disable more that one property
(e.g. both query log and result log) for a single statement using
the <b>ONCE</b> modifier.
*/
###############################################################################
## mysqltest Commands
/**
@page PAGE_MYSQL_TEST_COMMANDS mysqltest Commands
@ref PAGE_MYSQLTEST "mysqltest" supports the commands described in
this section. Command names are not case sensitive.
Some examples of command use are given, but you can find many more
by searching the test case files in the <b>mysql-test/t</b>
directory.
<ul>
<li>
<tt>append_file <b>file_name</b> [<b>terminator</b>]</tt>
<b>append_file</b> is like <b>write_file</b> except that the
lines up to the terminator are added to the end of the file.
The file is created if it does not exist. The file name argument
is subject to variable substitution.
@verbatim
write_file $MYSQL_TMP_DIR/data01;
line one for the file
line two for the file
EOF
append_file $MYSQL_TMP_DIR/data01;
line three for the file
EOF
@endverbatim
@verbatim
write_file $MYSQL_TMP_DIR/data02 END_OF_FILE;
line one for the file
line two for the file
END_OF_FILE
append_file $MYSQL_TMP_DIR/data02 END_OF_FILE;
line three for the file
END_OF_FILE
@endverbatim
</li>
<li>
<tt>cat_file <b>file_name</b></tt>
<b>cat_file</b> writes the contents of the file to the output.
The file name argument is subject to variable substitution.
@verbatim
cat_file $MYSQL_TMP_DIR/data01;
@endverbatim
</li>
<li>
<tt>change_user [<b>user_name</b>], [<b>password</b>],
[<b>db_name</b>]</tt>
Changes the current user and causes the database specified by
<b><em>db_name</em></b> to become the default database for the
current connection.
@verbatim
change_user root;
--change_user root,,test
@endverbatim
</li>
<li>
<tt>character_set <b>charset_name</b></tt>
Set the default character set to <b><em>charset_name</em></b>.
Initially, the character set is latin1.
@verbatim
character_set utf8;
--character_set sjis
@endverbatim
</li>
<li>
<tt>chmod <b>octal_mode file_name</b></tt>
Change the mode of the given file. The file mode must be given as
a four-digit octal number. The file name argument is subject to
variable substitution, but must evaluate to a literal file name,
not a file name pattern.
@verbatim
chmod 0644 $MYSQL_TMP_DIR/data_xxx01;
@endverbatim
</li>
<li>
<tt>connect (<b>name</b>, <b>host_name</b>,
<b>user_name</b>, <b>password</b>, <b>db_name</b>
[,<b>port_num</b> [,<b>socket</b> [,<b>options</b>
[,<b>default_auth</b> [,<b>compression algorithm</b>,
[,<b>compression level</b>]]]]]])</tt>
Open a connection to the server and make the connection the
current connection.
The arguments to <b>connect</b> are:
<ul>
<li>
<b><em>name</em></b> is the name for the connection (for use
with the <b>connection</b>, <b>disconnect</b>, and
<b>dirty_close</b> commands). This name must not already be
in use by an open connection.
</li>
<li>
<b><em>host_name</em></b> indicates the host where the
server is running.
</li>
<li>
<b><em>user_name</em></b> and <b><em>password</em></b> are
the user name and password of the MySQL account to use.
</li>
<li>
<b><em>db_name</em></b> is the default database to use. As a
special case, *<b>NO-ONE</b>* means that no default database
should be selected. You can also leave
<b><em>db_name</em></b> blank to select no database.
</li>
<li>
<b><em>port_num</em></b>, if given, is the TCP/IP port
number to use for the connection. This parameter can be
given by using a variable.
</li>
<li>
<b><em>socket</em></b>, if given, is the socket file to use
for connections to <b>localhost</b>. This parameter can be
given by using a variable.
</li>
<li>
<b><em>options</em></b> can be one or more of the following
words, separated by spaces:
<ul>
<li>
<tt>CLEARTEXT</tt>: Enable use of the cleartext authentication
plugin.
</li>
<li>
<tt>COMPRESS</tt>: Use the compressed client/server protocol,
if available.
</li>
<li>
<tt>PIPE</tt>: Use the named-pipe connection protocol, if
available.
</li>
<li>
<tt>SHM</tt>: Use the shared-memory connection protocol, if
available.
</li>
<li>
<tt>SOCKET</tt>: Use the socket-file connection protocol.
</li>
<li>
<tt>TCP</tt>: Use the TCP/IP connection protocol.
</li>
</ul>
Passing <b>PIPE</b> or <b>SHM</b> on non-Windows systems
causes an error, and, similarly, passing <b>SOCKET</b> on
Windows systems causes an error.
</li>
<li>
<b><em>default_auth</em></b> is the name of an
authentication plugin. It is passed to the
<b>mysql_options()</b> C API function using the
<b>MYSQL_DEFAULT_AUTH</b> option. If @ref PAGE_MYSQLTEST
"mysqltest" does not find the plugin, use the
<b>--plugin-dir</b> option to specify the directory where
the plugin is located.
</li>
<li>
<b><em>compression algorithm</em></b> is the name of compression
algorithm to be used to compress data transferred between client
server. It is passed to the <b>mysql_options()</b> C API function
using the <b>MYSQL_OPT_COMPRESSION_ALGORITHMS</b> option.
</li>
<li>
<b><em>zstd compression level</em></b> is the extent of compression
to be applied when zstd compression algorithm is used. It is passed to
the <b>mysql_options()</b> C API function using the
<b>MYSQL_OPT_COMPRESSION_ALGORITHMS</b> option.
</li>
<li>
<b><em>compression algorithm</em></b> is the name of compression
algorithm to be used to comrpess data transferred between client
server. It is passed to the <b>mysql_options()</b> C API function
using the <b>MYSQL_OPT_COMPRESSION_ALGORITHMS</b> option.
</li>
<li>
<b><em>compression level</em></b> is the extent of compression to
be applied based on the compression algorithm used. It is passed to
the <b>mysql_options()</b> C API function using the
<b>MYSQL_OPT_COMPRESSION_ALGORITHMS</b> option.
</li>
</ul>
To omit an argument, just leave it blank. For an omitted
argument, @ref PAGE_MYSQLTEST "mysqltest" uses an empty string
for the first five arguments and the <b><em>options</em></b>
argument. For omitted port or socket options, @ref
PAGE_MYSQLTEST "mysqltest" uses the default port or socket.
@verbatim
connect (conn1,localhost,root,,);
connect (conn2,localhost,root,mypass,test);
connect (conn1,127.0.0.1,root,,test,$MASTER_MYPORT);
@endverbatim
The last example assumes that the <b>$MASTER_MYPORT</b> variable
has already been set (perhaps as an environment variable).
If a connection attempt fails initially, @ref PAGE_MYSQLTEST
"mysqltest" retries five times if the abort-on-error setting is
enabled.
</li>
<li>
<tt>connection <b>connection_name</b></tt>
Select <b><em>connection_name</em></b> as the current
connection. To select the connection that @ref PAGE_MYSQLTEST
"mysqltest" opens when it starts, use the name default.
@verbatim
connection master;
connection conn2;
connection default;
@endverbatim
A variable can be used to specify the
<b><em>connection_name</em></b> value.
</li>
<li>
<tt>let $var= convert_error(<b>error</b>)</tt>
This is not a command as such but rather a function that can be
used in <b>let</b> statements. If the argument is a number, it
returns the name of the corresponding error, or <b>\<Unknown\></b>
if no such error exists. If the argument is an error name, it
returns the corresponding number, or fails if the error name is
unknown. If the argument is 0 or an empty string, it returns 0.
The function can also take a variable as argument.
@verbatim
let $errvar1=convert_error(ER_UNKNOWN_ERROR);
let $errvar2=convert_error(1450);
let $errvar3=convert_error($errvar1);
@endverbatim
</li>
<li>
<tt>copy_file <b>from_file to_file</b> [<b>retry</b>]</tt>
Copy the file <b><em>from_file</em></b> to the file
<b><em>to_file</em></b>. The command fails if
<b><em>to_file</em></b> already exists. The file name arguments
are subject to variable substitution.
<b>copy_file</b> can also take an optional argument <b>retry</b>.
If the command fails due to an environmental issue, the command
can be retried for a <b>retry</b> number of times. Each retry
happens after an interval of one second.
@verbatim
copy_file $MYSQL_TMP_DIR/copy1.txt $MYSQL_TMP_DIR/copy2.txt;
copy_file $MYSQL_TMP_DIR/foo.txt $MYSQL_TMP_DIR/foo2.txt 5;
@endverbatim
</li>
<li>
<tt>copy_files_wildcard <b>src_dir_name dst_dir_name</b>
<b>pattern</b> [<b>retry</b>]</tt>
Copy all files that match the pattern in the source directory to
the destination directory. Patterns can use <b>?</b> to
represent any single character, or <b>*</b> for any sequence of
0 or more characters. The <b>.</b> character is treated like any
other. The pattern may not include <b>/</b>. If all the files
need to be copied, the <b>*</b> wildcard can be used.
<b>copy_files_wildcard</b> can also take an optional argument
<b>retry</b>. If the command fails due to an environmental
issue, the command can be retried for a <b>retry</b> number
of times. Each retry happens after an interval of one second.
The command works like this:
<ul>
<li>
Files that match the pattern are copied from the source
directory to the destination directory. Overwriting of files
is permitted.
</li>
<li>
Copying does not apply to directories matching the pattern
or matching files in subdirectories.
</li>
<li>
If the source or destination directory is not present, an
error occurs.
</li>
<li>
If no files match the specified pattern, an error occurs.
</li>
<li>
If the source directory has no files, an error occurs.
</li>
</ul>
@verbatim
copy_files_wildcard $MYSQLTEST_VARDIR/std_data/ $MYSQLTEST_VARDIR/copy1/ *.txt;
copy_files_wildcard $MYSQLTEST_VARDIR/std_data/ $MYSQLTEST_VARDIR/copy1/ *.txt 5;
@endverbatim
</li>
<li>
<tt>dec <b>$var_name</b></tt>
Decrement a numeric variable. If the variable does not have a
numeric value, the result is undefined.
@verbatim
dec $count;
dec $2;
@endverbatim
</li>
<li>
<tt>delimiter <b>str</b></tt>
Set the command delimiter to <b><em>str</em></b>, which may consist
of 1 to 15 characters. The default delimiter is the semicolon
character (“<b>;</b>”).
@verbatim
delimiter /;
--delimiter stop
@endverbatim
This is useful or needed when you want to include long SQL
statements like <b>CREATE PROCEDURE</b> which include semicolon
delimited statements but need to be interpreted as a single
statement by @ref PAGE_MYSQLTEST "mysqltest". If you have set
the delimiter to “<b>/</b>” as in the previous example, you can
set it back to the default like this:
@verbatim
delimiter ;|
@endverbatim
</li>
<li>
<tt>die [<b>message</b>]</tt>
Aborts the test with an error code after printing the given
message as the reason. Suppose that a test file contains the
following line:
@verbatim
die Cannot continue;
@endverbatim
When @ref PAGE_MYSQLTEST "mysqltest" encounters that line, it
produces the following result and exits:
@verbatim
mysqltest: At line 1: Cannot continue
not ok
@endverbatim
</li>
<li>
<tt>diff_files <b>file_name1 file_name2</b></tt>
Compare the two files. The command succeeds if the files are the
same, and fails if they are different or either file does not
exist. The file name arguments are subject to variable
substitution.
</li>
<li>
<tt>dirty_close <b>connection_name</b></tt>
Close the named connection. This is like <b>disconnect</b>
except that it calls <b>vio_delete()</b> before it closes the
connection. If the connection is the current connection, you
should use the <b>connection</b> command to switch to a
different connection before executing further SQL statements.
A variable can be used to specify the
<b><em>connection_name</em></b> value.
</li>
<li>
<tt>disable_abort_on_error</tt>, <tt>enable_abort_on_error</tt>
Disable or enable abort-on-error behavior. This setting is
enabled by default. With this setting enabled, @ref
PAGE_MYSQLTEST "mysqltest" aborts the test when a statement sent
to the server results in an unexpected error, and does not
generate the <b>.reject</b> file. For discussion of reasons why
it can be useful to disable this behavior, see @ref
PAGE_ERROR_HANDLING.
@verbatim
--disable_abort_on_error
--enable_abort_on_error
@endverbatim
</li>
<li>
<tt>disable_connect_log</tt>, <tt>enable_connect_log</tt>
Disable or enable logging of creation or switch of connections.
Connection logging is disabled by default. With this setting
enabled, @ref PAGE_MYSQLTEST "mysqltest" enters lines in the
test results to show when connections are created, switched or
disconnected.
If query logging is turned off using <b>disable_query_log</b>,
connection logging is also turned off, until query log is re-
enabled.
@verbatim
--disable_connect_log
--enable_connect_log
@endverbatim
</li>
<li>
<tt>disable_info</tt>, <tt>enable_info</tt>
Disable or enable additional information about SQL statement
results. Information display is disabled by default. With this
setting enabled, @ref PAGE_MYSQLTEST "mysqltest" displays the
affected-rows count and the output from the <b>mysql_info()</b>
C API function. The “affected-rows” value is “rows selected” for
statements such as <b>SELECT</b> and “rows modified” for
statements that change data.
@verbatim
--disable_info
--enable_info
@endverbatim
</li>
<li>
<tt>disable_metadata</tt>, <tt>enable_metadata</tt>
Disable or enable query metadata display. Metadata display is
disabled by default. With this setting enabled, @ref
PAGE_MYSQLTEST "mysqltest" adds query metadata to the result.
This information consists of the values corresponding to the
members of the <b>MYSQL_FIELD</b> C API data structure, for each
column of the result.
@verbatim
--disable_metadata
--enable_metadata
@endverbatim
</li>
<li>
<tt>disable_ps_protocol</tt>, <tt>enable_ps_protocol</tt>
Disable or enable prepared-statement protocol. This setting is
disabled by default unless the <b>`--ps-protocol`</b> option is
given.
@verbatim
--disable_ps_protocol
--enable_ps_protocol
@endverbatim
</li>
<li>
<tt>disable_query_log</tt>, <tt>enable_query_log</tt>
Disable or enable query logging. This setting is enabled by
default. With this setting enabled, @ref PAGE_MYSQLTEST
"mysqltest" echoes input SQL statements to the test result.
One reason to disable query logging is to reduce the amount of
test output produced, which also makes comparison of actual and
expected results more efficient.
@verbatim
--disable_query_log
--enable_query_log
@endverbatim
</li>
<li>
<tt>disable_reconnect</tt>, <tt>enable_reconnect</tt>
Disable or enable automatic reconnect for dropped connections.
(The default depends on the client library version.) This
command only applies to the current connection.
@verbatim
--disable_reconnect
--enable_reconnect
@endverbatim
</li>
<li>
<tt>disable_result_log</tt>, <tt>enable_result_log</tt>
Disable or enable the result log. This setting is enabled by
default. With this setting enabled, @ref PAGE_MYSQLTEST
"mysqltest" displays query results (and results from commands
such as <b>echo</b> and <b>exec</b>).
@verbatim
--disable_result_log
--enable_result_log
@endverbatim
</li>
<li>
<tt>disable_rpl_parse</tt>, <tt>enable_rpl_parse</tt>
Disable or enable parsing of statements to determine whether
they go to the master or slave. The default is whatever the
default is for the C API library.
@verbatim
--disable_rpl_parse
--enable_rpl_parse
@endverbatim
</li>
<li>
<tt>disable_session_track_info</tt>, <tt>enable_session_track_info</tt>
Disable or enable display of session tracking information.
Session-tracking display disabled by default.
@verbatim
--disable_session_track_info
--enable_session_track_info
@endverbatim
</li>
<li>
<tt>disable_testcase bug_number</tt>, <tt>enable_testcase</tt>
Disable or enable a section from a test case. This setting is
enabled by default. When disabled, @ref PAGE_MYSQLTEST
"mysqltest" ignores everything until <b>enable_testcase</b>.
These commands are useful for disabling a section inside a
<b>.test</b> file that fails due to a known bug, without having
to disable the entire test case.
Bug number is a mandatory argument to <b>disable_testcase</b>
command and it should be in <b>BUG\#XXXX</b> format where keyword
<b>BUG</b> is case-insensitive and <b>XXXX</b> should contain
only digits.
@verbatim
--disable_testcase BUG#XXXX
--enable_testcase
@endverbatim
</li>
<li>
<tt>disable_warnings</tt>, <tt>enable_warnings</tt>
Disable or enable warnings. This setting is enabled by default.
With this setting enabled, @ref PAGE_MYSQLTEST "mysqltest" uses
<b>SHOW WARNINGS</b> to display any warnings produced by SQL
statements.
@verbatim
--disable_warnings
--enable_warnings
@endverbatim
<b>disable_warnings</b> and <b>enable_warnings</b> commands can
take an optional argument specifying one or more comma-separated
list of warnings to be disabled or enabled. Each warning
specified should be in symbolic error name format.
@verbatim
--disable_warnings ER_BAD_TABLE_ERROR,ER_YES
DROP TABLE IF EXISTS t1;
--enable_warnings ER_BAD_TABLE_ERROR,ER_YES
@endverbatim
Or
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings ER_BAD_TABLE_ERROR,ER_YES
DROP TABLE IF EXISTS t1;
--disable_warnings ER_BAD_TABLE_ERROR,ER_YES
--enable_warnings
@endverbatim
<b>disable_warnings</b> and <b>enable_warnings</b> commands can
also take a second optional argument, <b>"ONCE"</b>, which when
specified will disable or enable the warnings for next
statement only.
@verbatim
--disable_warnings ER_BAD_TABLE_ERROR ONCE
DROP TABLE IF EXISTS t1;
@endverbatim
Or
@verbatim
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings ER_BAD_TABLE_ERROR ONCE
DROP TABLE IF EXISTS t1;
--enable_warnings
@endverbatim
</li>
@note
At the end of a test, all the disabled warnings must be enabled,
else @ref PAGE_MYSQLTEST "mysqltest" will throw an error and
abort the test run.
<li>
<tt>disconnect <b>connection_name</b></tt>
Close the named connection. If the connection is the current
connection, you should use the <b>connection</b> command to
switch to a different connection before executing further SQL
statements.
@verbatim
disconnect conn2;
disconnect slave;
@endverbatim
</li>
<li>
<tt>echo <b>text</b></tt>
Echo the text to the test result. References to variables within
the text are replaced with the corresponding values. The text
does not need to be enclosed in quotation marks; if it is, the
quotation marks will be included in the output.
@verbatim
--echo Another sql_mode test
echo should return only 1 row;
@endverbatim
</li>
<li>
<tt>end</tt>
End an <b>if</b> or <b>while</b> block. If there is no such
block open, @ref PAGE_MYSQLTEST "mysqltest" exits with an error.
See @ref PAGE_MYSQL_TEST_FLOW_CONTROL, for information on
flow-control constructs.
@ref PAGE_MYSQLTEST "mysqltest" considers <b>}</b> and end the
same: Both end the current block.
</li>
<li>
<tt>end_timer</tt>
Stop the timer. By default, the timer does not stop until just
before @ref PAGE_MYSQLTEST "mysqltest" exits.
</li>
<li>
<tt>error <b>error_code</b> [, <b>error_code</b>] ...</tt>
Specify one or more comma-separated error values that the next
command is expected to return. Each <b><em>error_code</em></b>
value is a MySQL-specific error number or an <b>SQLSTATE</b>
value. (These are the kinds of values returned by the
<b>mysql_errno()</b> and <b>mysql_sqlstate()</b> C API
functions, respectively.)
If you specify an <b>SQLSTATE</b> value, it should begin with an
<b>S</b> to enable @ref PAGE_MYSQLTEST "mysqltest" to
distinguish it from a MySQL error number. For example, the error
number 1050 and the SQLSTATE value <b>42S01</b> are equivalent,
so the following commands specify the same expected error:
@verbatim
--error 1050
--error S42S01
@endverbatim
<b>SQLSTATE</b> values should be five characters long and may
contain only digits and uppercase letters.
It is also possible to use the symbolic error name from
<b>mysqld_error.h</b>:
@verbatim
--error ER_TABLE_EXISTS_ERROR
@endverbatim
It is also possible to use symbolic error names from
<b>errmsg.h</b> to refer to client errors:
@verbatim
--error CR_SERVER_GONE_ERROR
@endverbatim
Finally, you can assign either a numerical code or a symbolic
error name to a variable and refer to that in the <b>error</b>
command. Numbers, symbolic names and variables may be freely
mixed.
If a statement fails with an error that has not been specified
as expected by means of a <b>error</b> command, @ref
PAGE_MYSQLTEST "mysqltest" aborts and reports the error message
returned by the MySQL server.
If a statement fails with an error that has been specified as
expected by means of a <b>error</b> command, @ref PAGE_MYSQLTEST
"mysqltest" does not abort. Instead, it continues and writes a
message to the result output.
<ul>
<li>
If an <b>error</b> command is given with a single error
value and the statement fails with that error, @ref
PAGE_MYSQLTEST "mysqltest" reports the error message
returned by the MySQL server.
Input:
@verbatim
--error S42S02
DROP TABLE t;
@endverbatim
@ref PAGE_MYSQLTEST "mysqltest" reports:
@verbatim
ERROR 42S02: Unknown table 't'
@endverbatim
</li>
<li>
If an <b>error</b> command is given with multiple error
values and the statement fails with any of those errors,
@ref PAGE_MYSQLTEST "mysqltest" reports a generic message.
(This is true even if the error values are all the same, a
fact that can be used if you want a message that does not
contain varying information such as table names.)
Input:
@verbatim
--error S41S01,S42S02
DROP TABLE t;
@endverbatim
@ref PAGE_MYSQLTEST "mysqltest" reports:
@verbatim
Got one of the listed errors
@endverbatim
</li>
</ul>
An error value of <b>0</b> or <b>S00000</b> means “no error,”
so using either for an <b>error</b> command is the same as saying
explicitly, “no error is expected, the statement must succeed.”.
To indicate that you expect success or a given error or errors,
specify <b>0</b> or <b>S00000</b> first in the error list. If
you put the no-error value later in the list, the test will abort
if the statement is successful. That is, the following two commands
have different effects: The second form literally means the next
command may fail with error code 0, (rather than succeed) which
in practice never happens:
@verbatim
--error 0,1051
--error 1051,0
@endverbatim
You can use <b>error</b> to specify shell status values for
testing the value of shell commands executed using the
<b>exec</b> command. This does not apply to <b>system</b>, for
which the command status is ignored.
If you use <b>error</b> in combination with <b>send</b> and
<b>reap</b>, the <b>error</b> should be used just before the
<b>reap</b>, as this is the command that actually gives the
result and the potential error.
Variables may also be used as arguments to the <b>error</b>
command; these may contain a number (including 0), an SQLSTATE
or a symbolic error name. Variables and constant values may be
freely combined.
</li>
<li>
<tt>eval <b>statement</b></tt>
Evaluate the statement by replacing references to variables
within the text with the corresponding values. Then send the
resulting statement to the server to be executed. Use
“<b>\$</b>” to specify a literal “<b>$</b>” character.
The advantage of using <b>eval <em>statement</em></b> versus just
<b><em>statement</em></b> is that <b>eval</b> provides variable
expansion.
@verbatim
eval USE $DB;
eval CHANGE MASTER TO MASTER_PORT=$SLAVE_MYPORT;
eval PREPARE STMT1 FROM "$my_stmt";
@endverbatim
</li>
<li>
<tt>exec <b>command</b> [<b>arg</b>] ...</tt>
Execute the shell command using the <b>popen()</b> library call.
References to variables within the command are replaced with the
corresponding values. Use “<b>\\$</b>” to specify a literal
“<b>$</b>” character.
On Cygwin, the command is executed from <b>cmd.exe</b>, so
commands such as <b>rm</b> cannot be executed with <b>exec</b>.
Use <b>system</b> instead.
@verbatim
--exec $MYSQL_DUMP --xml --skip-create test
--exec rm $MYSQLTEST_VARDIR/tmp/t1
exec $MYSQL_SHOW test -v -v;
@endverbatim
@note
<b>exec</b> or <b>system</b> are sometimes used to perform
file system operations, but the command for doing so tend to be
operating system specific, which reduces test portability. @ref
PAGE_MYSQLTEST "mysqltest" now has several commands to perform
these operations portably, so they should be used instead:
<b>remove_file</b>, <b>chmod</b>, <b>mkdir</b>, and so forth.
</li>
<li>
<tt>exec_in_background <b>command</b> [<b>arg</b>] ...</tt>
Execute a shell command using the <b>popen()</b> library call in
the background. When a command is executed using
<b>exec_in_background</b>, @ref PAGE_MYSQLTEST "mysqltest" does
not wait for it to finish, nor attempt to read the output of it.
@verbatim
--exec_in_background $MYSQL_DUMP --xml --skip-create test
--exec_in_background rm $MYSQLTEST_VARDIR/tmp/t1
@endverbatim
</li>
<li>
<tt>execw <b>command</b> [<b>arg</b>] ...</tt>
This is a variant of the <b>exec</b> command which is needed on
Windows if the command line contains non-ASCII characters.
Otherwise it works exactly the same. On platforms other than
Windows there is no difference, but on Windows it uses a
different version of the <b>popen()</b> library call. So if your
command line contains non-ASCII characters, it is recommended to
use <b>execw</b> instead of <b>exec</b>.
</li>
<li>
<tt>exit</tt>
Terminate the test case. This is considered a “normal
termination.” That is, using <b>exit</b> does not result in
evaluation of the test case as having failed. It is not
necessary to use <b>exit</b> at the end of a test case, as the
test case will terminate normally when reaching the end without
failure.
</li>
<li>
<tt>expr <b>$var_name</b>= <b>operand1 operator operand2</b></tt>
Evaluate an expression and assign the result to a variable. The
result is also the return value of the expr command itself.
@verbatim
--let $val1= 10
--let $var2= 20
--expr $res= $var1 + $var2
--echo $res
@endverbatim
<b>operand1</b> and <b>operand2</b> must be valid variables.
expr supports these mathematical operators:
@verbatim
+ Addition
- Subtraction
* Multiplication
/ Division
% Modulo
@endverbatim
expr supports these logical operators:
@verbatim
&& Logical AND
|| Logical OR
@endverbatim
expr supports these bitwise operators:
@verbatim
& Binary AND
| Binary OR
^ Binary XOR
<< Binary left shift
>> Binary right shift
@endverbatim
Operations are subject to these conditions:
- Operations that do not support noninteger operands truncate
such operands to integer values.
- If the result is an infinite value, expr returns the inf
keyword.
- Division by 0 results in an infinite value.
</li>
<li>
<tt>file_exists <b>file_name</b> [<b>retry</b>]</tt>
<b>file_exists</b> succeeds if the named file exists and fails
otherwise. The file name argument is subject to variable
substitution.
<b>file_exists</b> can also take an optional argument
<b>retry</b>. If the command fails due to an environmental
issue, the command can be retried for a <b>retry</b> number
of times. Each retry happens after an interval of one second.
@verbatim
file_exists /etc/passwd;
file_exists /etc/passwd 5;
@endverbatim
</li>
<li>
<tt>force-cpdir <b>src_dir_name dst_dir_name</b></tt>
Copies the source directory, <b><em>src_dir_name</em></b>, to
the destination directory, <b><em>dst_dir_name</em></b>. The
copy operation is recursive, so it copies subdirectories.
Returns 0 for success and 1 for failure.
@verbatim
--force-cpdir testdir testdir2
@endverbatim
If the source directory does not exist, an error occurs.
If the destination directory does not exist, @ref PAGE_MYSQLTEST
"mysqltest" creates it before copying the source directory.
</li>
<li>
<tt>force-rmdir <b>dir_name</b></tt>
Remove a directory named <b><em>dir_name</em></b>. Returns 0 for
success and 1 for failure.
@verbatim
--force-rmdir testdir
@endverbatim
<b>force-rmdir</b> removes the directory as well as its
contents, if any, unlike <b>rmdir</b>, which fails if the
directory to be removed contains any files or directories.
</li>
<li>
<tt>horizontal_results</tt>
Set the default query result display format to horizontal.
Initially, the default is to display results horizontally.
@verbatim
--horizontal_results
@endverbatim
</li>
<li>
<tt>if (<b>expr</b>)</tt>
Begin an if block, which continues until an <b>end</b> or
<b>}</b> line. @ref PAGE_MYSQLTEST "mysqltest" executes the
block if the expression is non-zero. There is no provision for
<b>else</b> with <b>if</b>. See @ref
PAGE_MYSQL_TEST_FLOW_CONTROL, for further information about
<b>if</b> statements.
@verbatim
let $counter= 0;
if ($counter)
{
echo Counter is not 0;
}
if (!$counter)
{
echo Counter is 0;
}
@endverbatim
</li>
<li>
<tt>inc <b>$var_name</b></tt>
Increment a numeric variable. If the variable does not have a
numeric value, the result is undefined.
@verbatim
--inc $i;
inc $3;
@endverbatim
</li>
<li>
<tt>let <b>$var_name</b> = <b>value</b></tt>
<tt>let <b>$var_name</b> = query_get_value(<b>query</b>,
<b>col_name</b>,
<b>row_num</b>)</tt>
Assign a value to a variable. The variable name cannot contain
whitespace or the “<b>=</b>” character. Except for the one-digit
<b>$0</b> to <b>$9</b>, it cannot begin with a number. @ref
PAGE_MYSQLTEST "mysqltest" aborts with an error if the value is
erroneous.
References to variables within <b><em>value</em></b> are replaced
with their corresponding values.
If the <b>let</b> command is specified as a normal command (that
is, not beginning with “<b>--</b>”), <b><em>value</em></b>
includes everything up to the command delimiter, and thus can
span multiple lines.
@verbatim
--let $1= 0
let $count= 10;
@endverbatim
When assigning a literal string to a variable, no quoting is
required even if the string contains spaces. If the string does
include quotation marks, they will be trated like any other
characters and be included in the string value. This is
important to be aware of when using the variable in an SQL
statement.
The result from executing a query can be assigned to a variable
by enclosing the query within backtick (“`”) characters:
@verbatim
let $q= `SELECT VERSION()`;
@endverbatim
The <b>let</b> command can set environment variables, not just
@ref PAGE_MYSQLTEST "mysqltest" test language variables. To
assign a value to an environment variable rather than a test
language variable, just omit the dollar sign:
@verbatim
let $mysqltest_variable= foo;
let ENV_VARIABLE= bar;
@endverbatim
This is useful in interaction with external tools. In
particular, when using the <b>perl</b> command, the Perl code
cannot access test language variables, but it can access
environment variables. For example, the following statement can
access the <b>ENV_VARIABLE</b> value:
@verbatim
print $ENV{'ENV_VARIABLE'};
@endverbatim
The <b>let</b> syntax also allows the retrieval of a value from
a query result set produced by a statement such as <b>SELECT</b>
or <b>SHOW</b>. See the description of <b>query_get_value()</b>
for more information.
</li>
<li>
<tt>mkdir <b>dir_name</b></tt>
Create a directory named <b><em>dir_name</em></b>. Returns 0 for
success and 1 for failure.
@verbatim
--mkdir testdir
@endverbatim
</li>
<li>
<tt>list_files <b>dir_name</b> [<b>pattern</b>]</tt>
<b>list_files</b> lists the files in the named directory. If a
pattern is given, lists only file(s) matching the pattern, which
may contain wild cards.
@verbatim
--list_files $MYSQLD_DATADIR/test t1*
@endverbatim
</li>
<li>
<tt>list_files_append_file <b>file_name dir_name</b>
[<b>pattern</b>]</tt>
<b>list_files_append_file</b> works like <b>list_files</b>,
but rather than outputting the file list, it is appended to the
file named in the first argument. If the file does not exist, it
is created.
@verbatim
--list_files_append_file $MYSQL_TMP_DIR/filelist $MYSQL_TMP_DIR/testdir *.txt;
@endverbatim
</li>
<li>
<tt>list_files_write_file <b>file_name dir_name</b>
[<b>pattern</b>]</tt>
<b>list_files_write_file</b> works like
<b>list_files_append_file</b>, but creates a new file to write
the file list to. If the file already exists, it will be
replaced.
@verbatim
--list_files_write_file $MYSQL_TMP_DIR/filelist $MYSQL_TMP_DIR/testdir *.txt;
@endverbatim
</li>
<li>
<tt>lowercase_result</tt>
Output from the following SQL statement will be converted to
lowercase. This is sometimes needed to ensure consistent result
across different platforms. If this is combined with one of the
<b>replace</b> commands or with <b>sorted_result</b>, both will
take effect on the output, with conversion to lowercase being
applied first.
@verbatim
--lowercase_result
@endverbatim
</li>
<li>
<tt>move_file <b>from_name to_name</b> [<b>retry</b>]</tt>
<b>move_file</b> renames <b><em>from_name</em></b> to
<b><em>to_name</em></b>. The file name arguments are subject to
variable substitution, but must evaluate to a literal file name,
not a file name pattern.
<b>move_file</b> can also take an optional argument <b>retry</b>.
If the command fails due to an environmental issue, the command
can be retried for a <b>retry</b> number of times. Each retry
happens after an interval of one second.
@verbatim
move_file $MYSQL_TMP_DIR/data01 $MYSQL_TMP_DIR/test.out;
move_file $MYSQL_TMP_DIR/data01 $MYSQL_TMP_DIR/test.out 5;
@endverbatim
</li>
<li>
<tt>output <b>file_name</b></tt>
Direct output from the next SQL statement to the named file
rather than to the test output. If the file already exists, it
will be overwritten. Only the next SQL statement will have its
output redirected.
@verbatim
output $MYSQL_TMP_DIR/out-file
@endverbatim
</li>
<li>
<tt>perl [<b>terminator</b>]</tt>
Use Perl to execute the following lines of the test file. The
lines end when a line containing the terminator is encountered.
The default terminator is <b>EOF</b>, but a different terminator
can be provided.
@verbatim
perl;
print "This is a test\n";
EOF
@endverbatim
@verbatim
perl END_OF_FILE;
print "This is another test\n";
END_OF_FILE
@endverbatim
</li>
<li>
<tt>ping</tt>
Ping the server. This executes the <b>mysql_ping()</b> C API
function. The function result is discarded. The effect is that
if the connection has dropped and reconnect is enabled, pinging
the server causes a reconnect.
</li>
<li>
<tt>query [<b>statement</b>]</tt>
Send the statement to the server to be executed. The
<b>query</b> command can be used to force @ref PAGE_MYSQLTEST
"mysqltest" to send a statement to the server even if it begins
with a keyword that is a @ref PAGE_MYSQLTEST "mysqltest"
command.
</li>
<li>
<tt>query_get_value(<b>query</b>, <b>col_name</b>, <b>row_num</b>)</tt>
The <b>query_get_value()</b> function can be used only on the
right hand side of a variable assigment in a <b>let</b>
statement.
<b>query_get_value()</b> enables retrieval of a value from a
query result set produced by a statement such as <b>SELECT</b>
or <b>SHOW</b>. The first argument indicates the query to
execute. The second and third arguments indicate the column name
and row number that specify which value to extract from the
result set. The column name is case sensitive. Row numbers begin
with 1. The arguments can be given literally or supplied using
variables.
Suppose that the test file contains this input:
@verbatim
CREATE TABLE t1(a INT, b VARCHAR(255), c DATETIME);
SHOW COLUMNS FROM t1;
let $value= query_get_value(SHOW COLUMNS FROM t1, Type, 1);
echo $value;
@endverbatim
The result will be:
@verbatim
CREATE TABLE t1(a INT, b VARCHAR(255), c DATETIME);
SHOW COLUMNS FROM t1;
Field Type Null Key Default Extra
a int(11) YES NULL
b varchar(255) YES NULL
c datetime YES NULL
int(11)
@endverbatim
If the query fails, an error message occurs and the test fails.
</li>
<li>
<tt>query_horizontal <b>statement</b></tt>
Execute the statement and display its result horizontally.
@verbatim
query_horizontal SELECT PI();
@endverbatim
</li>
<li>
<tt>query_vertical <b>statement</b></tt>
Execute the statement and display its result vertically.
@verbatim
query_vertical SELECT PI();
@endverbatim
</li>
<li>
<tt>real_sleep <b>num</b></tt>
Sleep <b><em>num</em></b> seconds. <b><em>num</em></b> can have
a fractional part. Unlike the <b>sleep</b> command,
<b>real_sleep</b> is not affected by the
<b>`--sleep command-line`</b> option.
@verbatim
--real_sleep 10
real_sleep 5;
@endverbatim
Try not to use <b>sleep</b> or <b>real_sleep</b> commands more
than necessary. The more of them there are, the slower the test
suite becomes.
</li>
<li>
<tt>reap</tt>
Receive the result of the statement sent with the <b>send</b>
command within the current session. You should not use
<b>reap</b> unless a statement has been sent with <b>send</b>,
and you should not use <b>send</b> again if there is an
outstanding <b>send</b> that has not been processed with
<b>reap</b>.
</li>
<li>
<tt>remove_file <b>file_name</b> [<b>retry</b>]</tt>
<b>remove_file</b> removes the file. It fails with an error if
the file does not exist. The file name argument is subject to
variable substitution, but must evaluate to a literal file name,
not a file name pattern.
<b>remove_file</b> also takes an optional argument <b>retry</b>.
If the command fails due to an environmental issue, the command
can be retried for a <b>retry</b> number of times. Each retry
happens after an interval of one second.
@verbatim
remove_file $MYSQL_TMP_DIR/data01;
remove_file $MYSQL_TMP_DIR/data01 5
@endverbatim
</li>
<li>
<tt>remove_files_wildcard <b>dir_name</b> <b>pattern</b> [<b>retry</b>]</tt>
Remove all files in the named directory that match the pattern.
Removal does not apply to directories matching the pattern or
matching files in subdirectories. Patterns can use <b>?</b> to
represent any single character, or <b>*</b> for any sequence of
0 or more characters. The <b>.</b> character is treated like any
other. The pattern may not include <b>/</b>. If all the files
need to be removed, the <b>*</b> wildcard can be used.
<b>remove_files_wildcard</b> can also take an optional argument
<b>retry</b>. If the command fails due to an environmental
issue, the command can be retried for a <b>retry</b> number of
times. Each retry happens after an interval of one second.
@verbatim
remove_files_wildcard $MYSQL_TMP_DIR file*.txt;
remove_files_wildcard $MYSQL_TMP_DIR file*.txt 5;
@endverbatim
</li>
<li>
<tt>replace_column <b>col_num</b> <b>value</b>
[<b>col_num value</b>] ...</tt>
Replace strings in the output from the next statement. The value
in <b><em>col_num</em></b> is replaced by the corresponding
<b><em>value</em></b>. There can be more than one
<b><em>col_num/value</em></b> pair. Column numbers start with 1.
A replacement value can be double-quoted. (Use “<b>\"</b>” to
specify a double quote within a replacement string.) Variables
can be used in a replacement value if it is not double-quoted.
If mixed <b>replace_<em>xxx</em></b> commands are given, only the
final one applies.
@note
Although <b>replace_regex</b> and <b>replace_result</b> affect
the output from <b>exec</b>, <b>replace_column</b> does not
because <b>exec</b> output is not necessarily columnar.
@verbatim
--replace_column 9 #
replace_column 1 b 2 d;
@endverbatim
</li>
<li>
<tt>replace_numeric_round <b>precision</b></tt>
Rounds off floating point numbers. <b>precision</b> is the
number of digits after the decimal point to which the result will
be rounded off to. It can only be a number between
0 and 16.
If the precision is 0, then the value is rounded off to the
nearest whole number. Trailing zeroes after the decimal point are
removed from the numbers.
Numbers which are <b>greater than <em>1e10</em></b> or <b> lesser
than <em> -1e10 </em></b> are represented using the
exponential notation after they are rounded off.
@verbatim
--replace_numeric_round 6
SELECT 12379.79459775333;
12379.794598
replace_numeric_round 0;
SELECT 876.87302;
877
@endverbatim
</li>
<li>
<tt>replace_regex <b>/pattern/replacement/</b>[i] ...</tt>
In the output from the next statement, find strings within
columns of the result set that match <b><em>pattern</em></b> (a
regular expression) and replace them with
<b><em>replacement</em></b>. Each instance of a string in a
column that matches the pattern is replaced. Matching is case
sensitive by default. Specify the optional <b>i</b> modifier to
cause matching to be case insensitive.
The syntax for allowable patterns is same as the <b>std::ECMAScript</b>
syntax. The pattern can contain parentheses to mark substrings
matched by parts of the pattern. These substrings can be referenced in
the replacement string: An instance of <b><em>N</em></b> in the
replacement string causes insertion of the <b><em>N</em>-th</b> substring
matched by the pattern. For example, the following command matches
<b>strawberry</b> and replaces it with <b>raspberry</b> and
<b>strawberry</b>:
@verbatim
--replace_regex /(strawberry)/raspberry and \1/
@endverbatim
Multiple <b><em>pattern/replacement</em></b> pairs may be given.
The following command replaces instances of <b>A</b> with
<b>C</b> (the first pattern replaces <b>A</b> with <b>B</b>, the
second replaces <b>B</b> with <b>C</b>):
@verbatim
--replace_regex /A/B/ /B/C/
@endverbatim
If a given pattern is not found, no error occurs and the input
is unchanged.
</li>
<li>
<tt>replace_result <b>from_val to_val</b>
[<b>from_val to_val</b>] ...</tt>
Replace strings in the result. Each occurrence of
<b><em>from_val</em></b> is replaced by the corresponding
<b><em>to_val</em></b>. There can be more than
<b><em>from_val/to_val</em></b> pair. Arguments can be quoted
with single quotation marks or double quotation marks. Variable
references within the arguments are expanded before replacement
occurs. Values are matched literally. To use patterns, use the
<b>replace_regex</b> command.
@verbatim
--replace_result 1024 MAX_KEY_LENGTH 3072 MAX_KEY_LENGTH
replace_result $MASTER_MYPORT MASTER_PORT;
@endverbatim
</li>
<li>
<tt>reset_connection</tt>
Reset the connection state by calling [mysql_reset_connection()]
(https://dev.mysql.com/doc/refman/8.0/en/mysql-reset-connection.html).
</li>
<li>
<tt>result_format <b>version</b></tt>
Set the format to the specified version, which is either 1 for
the current, default behavior, or to 2 which is an extended
alternative format. The setting is in effect until it is changed
or until the end of the test.
In format version 2, empty lines and indentation in the test
file are preserveded in the result. Also, comments indicated by
a double \## are copied verbatim to the result. Comments using a
single \# are not copied. Format version 2 makes it easier for
humans to read the result output, but at the cost of somewhat
larger files due to the white space and comments.
@verbatim
--result_format 2
@endverbatim
</li>
<li>
<tt>rmdir <b>dir_name</b></tt>
Remove a directory named <b><em>dir_name</em></b>. Returns 0 for
success and 1 for failure.
@verbatim
--rmdir testdir
@endverbatim
<b>rmdir</b> fails if the directory to be removed contains any
files or directories. To remove the directory as well as its
contents, if any, use <b>force-rmdir</b>.
</li>
<li>
<tt>save_master_pos</tt>
For a master replication server, save the current binary log
file name and position. These values can be used for subsequent
<b>sync_with_master</b> or <b>sync_slave_with_master</b>
commands.
</li>
<li>
<tt>send [<b>statement</b>]</tt>
Send a statement to the server but do not wait for the result.
The result must be received with the <b>reap</b> command. You
cannot execute another SQL statement on the same connection
between <b>send</b> and <b>reap</b>.
If <b><em>statement</em></b> is omitted, the <b>send</b> command
applies to the next statement executed. This means that
<b>send</b> can be used on a line by itself before a statement.
Thus, this command:
@verbatim
send SELECT 1;
@endverbatim
Is equivalent to these commands:
@verbatim
send;
SELECT 1;
@endverbatim
</li>
<li>
<tt>send_eval [<b>statement</b>]</tt>
Evaluate the command, then send it to the server. This is a
combination of the <b>send</b> and <b>eval</b> commands, giving
the functionality of both. After variable replacement has been
done, it behaves like the <b>send</b> command.
If <b><em>statement</em></b> is omitted, the <b>send_eval</b>
command applies to the next statement executed. This means that
<b>send_eval</b> can be used on a line by itself before a
statement. Thus, this command:
@verbatim
--send_eval $my_stmt
@endverbatim
</li>
Is equivalent to these commands:
@verbatim
--send_eval
$my_stmt;
@endverbatim
</li>
<li>
<tt>send_quit <b>connection</b></tt>
Sends a <b>COM_QUIT</b> command to the server on the named
connection.
@verbatim
send_quit con;
@endverbatim
</li>
<li>
<tt>send_shutdown</tt>
Sends a shutdown command to the server but does not wait for it
to complete the shutdown. Test execution proceeds as soon as the
shutdown command is sent.
</li>
<li>
<tt>shutdown_server [<b>timeout</b>]</tt>
Stops the server. This command waits for the server to shut down
by monitoring its process ID (PID) file. If the server's process
ID file is not gone after <b><em>timeout</em></b> seconds, the
process will be killed. If <b><em>timeout</em></b> is omitted,
the default is 60 seconds.
@verbatim
shutdown_server;
shutdown_server 30;
@endverbatim
</li>
<li>
<tt>skip [<b>message</b>]</tt>
Skips the rest of the test file after printing the given message
as the reason. This can be used after checking a condition that
must be satisfied, as a way of performing an exit that displays
a reason. References to variables within the message are
replaced with the corresponding values. Suppose that the test
file <b>mytest.test</b> has these contents:
@verbatim
let $var= 0;
if (!$var)
{
skip value of \$var is '$var', skipping test;
}
echo This command is never reached;
@endverbatim
Executing @ref PAGE_MYSQLTEST "mysqltest" -x mytest.test yields
these results:
@verbatim
The test './mytest' is not supported by this installation
Detected in file ./mytest at line 4
reason: value of $var is '0', skipping test
@endverbatim
If the test is run from @ref PAGE_MYSQL_TEST_RUN_PL
"mysql-test-run.pl", you will instead see the test result as
<b>[ skipped ]</b> followed by the message.
@verbatim
main.mytest [ skipped ] value of $var is '0', skipping test
@endverbatim
</li>
<li>
<tt>sleep <b>num</b></tt>
Sleep <b><em>num</em></b> seconds. <b><em>num</em></b> can have
a fractional part. If the <b>`--sleep`</b> command-line option
was given, the option value overrides the value given in the
<b>sleep</b> command. For example, if @ref PAGE_MYSQLTEST
"mysqltest" is started with <b>`--sleep`=10</b>, the command
<b>sleep 15</b> sleeps 10 seconds, not 15.
@verbatim
--sleep 10
sleep 0.5;
@endverbatim
Try not to use <b>sleep</b> or <b>real_sleep</b> commands more
than necessary. The more of them there are, the slower the test
suite becomes.
</li>
<li>
<tt>sorted_result</tt>
Sort the output from the next statement if it produces a result
set. <b>sorted_result</b> is applied just before displaying the
result, after any other result modifiers that might have been
specified, such as <b>replace_result</b> or
<b>replace_column</b>. If the next statement produces no result
set, <b>sorted_result</b> has no effect because there is nothing
to sort.
@verbatim
sorted_result;
SELECT 2 AS "my_col" UNION SELECT 1;
let $my_stmt=SELECT 2 AS "my_col" UNION SELECT 1;
--sorted_result
eval $my_stmt;
--sorted_result
--replace_column 1 #
SELECT '1' AS "my_col1",2 AS "my_col2"
UNION
SELECT '2',1;
@endverbatim
<b>sorted_result</b> sorts the entire result of the next query.
If this involves constructs such as <b>UNION</b>, stored
procedures, or multi-statements, the output will be in a fixed
order, but all the results will be sorted together and might
appear somewhat strange.
The purpose of the <b>sorted_result</b> command is to produce
output with a deterministic order for a given set of result
rows. It is possible to use <b>ORDER BY</b> to sort query
results, but that can sometimes present its own problems. For
example, if the optimizer is being investigated for some bug,
<b>ORDER BY</b> might order the result but return an incorrect
set of rows. <b>sorted_result</b> can be used to produce sorted
output even in the absence of <b>ORDER BY</b>.
<b>sorted_result</b> is useful for eliminating differences
between test runs that may otherwise be difficult to compensate
for. Results without <b>ORDER BY</b> are not guaranteed to be
returned in any given order, so the result for a given query
might differ between test runs. For example, the order might
vary between different server versions, so a result file created
by one server might fail when compared to the result created by
another server. The same is true for different storage engines.
<b>sorted_result</b> eliminates these order differences by
producing a deterministic row order.
Other ways to eliminate differences from results without use of
<b>sorted_result</b> include:
- Remove columns from the select list to reduce variability in
the output
- Use aggregate functions such as <b>AVG()</b> on all columns
of the select list
- Use <b>ORDER BY</b>
The use of aggregate functions or <b>ORDER BY</b> may also have
the advantage of exposing other bugs by introducing additional
stress on the server. The choice of whether to use
<b>sorted_result</b> or <b>ORDER BY</b> (or perhaps both) may be
dictated by whether you are trying to expose bugs, or avoid
having them affect results. This means that care should be taken
with <b>sorted_result</b> because it has the potential of hiding
server bugs that result in true problems with result order.
</li>
<li>
<tt>partially_sorted_result <b>start_column</b></tt>
Similarly to <b>sorted_result</b>, but assumes that the first
<b>start_column</b> columns are already sorted, sorting only
on the remaining ones. This is useful if you wish to verify
a partial ordering property, e.g. that <b>SELECT a,b,c ORDER BY
a</b>. In this case, you could use <b>partially_sorted_result 1</b>
to sort only on b and c, giving deterministic results without
masking bugs where sorting on a column would not work correctly.
<tt>sorted_result</tt> is functionally equal to
<tt>partially_sorted_result 0</tt>.
</li>
<li>
<tt>source <b>file_name</b></tt>
Read test input from the named file.
If you find that several test case files contain a common
section of commands (for example, statements that create a
standard set of tables), you can put those commands in another
file and those test cases that need the file can include it by
means of a <b>source <em>file_name</em></b> command. This
enables you to write the code just once rather than in multiple
test cases.
Normally, the file name in the <b>source</b> command is relative
to the <b>mysql-test</b> directory because @ref PAGE_MYSQLTEST
"mysqltest" usually is invoked in that directory.
A sourced file can use <b>source</b> to read other files, but
take care to avoid a loop. The maximum nesting level is 16.
@verbatim
--source include/have_csv.inc
source include/varchar.inc;
@endverbatim
The file name can include variable references. Variables are
expanded including any quotation marks in the values, so
normally the values should not include quotation marks. Suppose
that <b>/tmp/junk</b> contains this line:
@verbatim
SELECT 'I am a query';
@endverbatim
The following example shows one way in which variable references
could be used to specify the file name:
@verbatim
let $dir= /tmp;
let $file= junk;
source $dir/$file;
@endverbatim
</li>
<li>
<tt>start_timer</tt>
Restart the timer, overriding any timer start that occurred
earlier. By default, the timer starts when @ref PAGE_MYSQLTEST
"mysqltest" begins execution.
</li>
<li>
<tt>sync_slave_with_master [<b>connection_name</b>]</tt>
Executing this command is equivalent to executing the following
commands:
@verbatim
save_master_pos;
connection connection_name;
sync_with_master 0;
@endverbatim
If <b><em>connection_name</em></b> is not specified, the
connection named <b>slave</b> is used.
The effect is to save the replication coordinates (binary log
file name and position) for the server on the current connection
(which is assumed to be a master replication server), and then
switch to a slave server and wait until it catches up with the
saved coordinates. Note that this command implicitly changes the
current connection.
A variable can be used to specify the
<b><em>connection_name</em></b> value.
</li>
<li>
<tt>sync_with_master <b>offset</b></tt>
For a slave replication server, wait until it has caught up with
the master. The position to synchronize to is the position saved
by the most recent <b>save_master_pos</b> command plus offset.
To use this command, <b>save_master_pos</b> must have been
executed at some point earlier in the test case to cause @ref
PAGE_MYSQLTEST "mysqltest" to save the master's replication
coordinates.
</li>
<li>
<tt>vertical_results</tt>
Set the default query result display format to vertical.
Initially, the default is to display results horizontally.
@verbatim
--vertical_results
@endverbatim
</li>
<li>
<tt>wait_for_slave_to_stop</tt>
Poll the current connection, which is assumed to be a connection
to a slave replication server, by executing <b>SHOW STATUS LIKE
'Slave_running'</b> statements until the result is <b>OFF</b>.
For information about alternative means of slave server control,
see @ref PAGE_WRITING_REPLICATION_TESTS.
</li>
<li>
<tt>while (<b>expr</b>)</tt>
Begin a <b>while</b> loop block, which continues until an
<b>end</b> line. @ref PAGE_MYSQLTEST "mysqltest" executes the
block repeatedly as long as the expression is true (non-zero).
See @ref PAGE_MYSQL_TEST_FLOW_CONTROL, for further information
about while statements.
Make sure that the loop includes some exit condition that
eventually occurs. This can be done by writing
<b><em>expr</em></b> so that it becomes false at some point.
@verbatim
let $i=5;
while ($i)
{
echo $i;
dec $i;
}
@endverbatim
</li>
<li>
<tt>write_file <b>file_name</b> [<b>terminator</b>]</tt>
Write the following lines of the test file to the given file,
until a line containing the terminator is encountered. The
default terminator is <tt>EOF</tt>, but a different terminator
can be provided. The file name argument is subject to variable
substitution. An error occurs if the file already exists.
@verbatim
write_file $MYSQL_TMP_DIR/data01;
line one for the file
line two for the file
EOF
@endverbatim
@verbatim
write_file $MYSQL_TMP_DIR/data02 END_OF_FILE;
line one for the file
line two for the file
END_OF_FILE
@endverbatim
</li>
</ul>
*/
###############################################################################
## mysqltest Variables
/**
@page PAGE_MYSQL_TEST_VARIABLES mysqltest Variables
You can define variables and refer to their values. You can also
refer to environment variables, and there is a built-in variable
that contains the result of the most recent SQL statement.
To define a variable, use the <b>let</b> command. Examples:
@verbatim
let $a= 14;
let $b= this is a string;
--let $a= 14
--let $b= this is a string
@endverbatim
The variable name cannot contain whitespace or the “=” character.
If a variable has a numeric value, you can increment or decrement
the value:
@verbatim
inc $a;
dec $a;
--inc $a
--dec $a
@endverbatim
<b>inc</b> and <b>dec</b> are commonly used in <b>while</b>
loops to modify the value of a counter variable that controls loop
execution.
The result from executing a query can be assigned to a variable by
enclosing the query within backtick (“`”) characters:
@verbatim
let $q= `select version()`;
@endverbatim
References to variables can occur in the <b>echo</b>,
<b>eval</b>, <b>exec</b>, and <b>system</b> commands. Variable
references are replaced by their values. A nonquery value assigned
to a variable in a <b>let</b> command can also refer to variables.
Variable references that occur within
\`<b><em>query</em></b>\` are expanded before the query is
sent to the server for execution.
You can refer to environment variables. For example, this command
displays the value of the <b>$PATH</b> variable from the
environment:
@verbatim
--echo $PATH
@endverbatim
<b>$mysql_errno</b> is a built-in variable that contains the
numeric error returned by the most recent SQL statement sent to the
server, or 0 if the statement executed successfully.
<b>$mysql_errno</b> has a value of −1 if no statement has yet been
sent.
<b>$mysql_errname</b> similarly contains the symbolic name of the
last error, or an empty string if there was no error.
@ref PAGE_MYSQLTEST "mysqltest" first checks @ref PAGE_MYSQLTEST
"mysqltest" variables and then environment variables. @ref
PAGE_MYSQLTEST "mysqltest" variable names are not case sensitive.
Environment variable names are case sensitive.
*/
###############################################################################
## mysqltest Flow Control Constructs
/**
@page PAGE_MYSQL_TEST_FLOW_CONTROL mysqltest Flow Control Constructs
The syntax for <b>if</b> block looks like this:
@verbatim
if (expr)
{
command list
}
@endverbatim
The syntax for <b>while</b> block looks like this:
@verbatim
while (expr)
{
command list
}
@endverbatim
An expression result is true if nonzero, false if zero. If the
expression begins with <b>!</b>, the sense of the test is reversed.
If the expression is a string that does not begin with a numeric
digit (possibly preceeded by a plus or minus sign), it evaluates as
true if non-empty. Any white space is ignored in this case, so a
string consisting of only white space is false.
There is no provision for <b>else</b> with <b>if</b>.
For a <b>while</b> loop, make sure that the loop includes some
exit condition that eventually occurs. This can be done by writing
<b><em>expr</em></b> so that it becomes false at some point.
The allowable syntax for <b><em>expr</em></b> is
<b>$<em>var_name</em></b>, <b>!$<em>var_name</em></b>,
a string or integer, or \`<b><em>query</em></b>\`.
The expression can also be a simple comparison, where the left hand
side must be a variable, and the right hand side can be any type
valid for the single expression except the negated variable. The
supported operators are ==, !=, <, <=, > and >=. Only the first two
may be used if the right hand side does not evaluate to an integer.
With ==, strings must match exactly.
If you use a string on the right hand side of the comparison, it
does not have to be quoted even if it contains spaces. It may
optionally be enclosed in single or double quotation marks which
will then be stripped off before comparison. This is in contrast to
<b>let</b> statements, where quoting is not stripped.
Examples of the expression syntax with comparisons (only the header
shown):
@verbatim
while ($counter<5) ...
if ($value == 'No such row') ...
if ($slave_count != $master_count) ...
@endverbatim
The opening <b>{</b> (curly brace) must be separated from the
preceding <b>)</b> (right parenthesis) by whitespace, such as a
space or a line break.
Variable references that occur within
\`<b><em>query</em></b>\` are expanded before the query is
sent to the server for execution.
*/
###############################################################################
## Error Handling
/**
@page PAGE_ERROR_HANDLING Error Handling
If an expected error is specified and that error occurs, @ref
PAGE_MYSQLTEST "mysqltest" continues reading input. If the command
is successful or a different error occurs, @ref PAGE_MYSQLTEST
"mysqltest" aborts.
If no expected error is specified, @ref PAGE_MYSQLTEST "mysqltest"
aborts unless the command is successful. (It is implicit that you
expect <b>$mysql_errno</b> to be 0.)
By default, @ref PAGE_MYSQLTEST "mysqltest" aborts for certain
conditions:
<ul>
<li>
A statement that fails when it should have succeeded. The
following statement should succeed if table <b>t</b> exists;
@verbatim
SELECT * FROM t;
@endverbatim
</li>
<li>
A statement that fails with an error different from that
specified:
@verbatim
--error 1
SELECT * FROM no_such_table;
@endverbatim
</li>
<li>
A statement that succeeds when an error was expected:
@verbatim
--error 1
SELECT 'a string';
@endverbatim
</li>
</ul>
You can disable the abort for errors of the first type by using the
<b>disable_abort_on_error</b> command. In this case, when errors
occur for statements that should succeed, @ref PAGE_MYSQLTEST
"mysqltest" continues processing intput.
<b>disable_abort_on_error</b> does <b>not</b> cause @ref
PAGE_MYSQLTEST "mysqltest" to ignore errors for the other two types,
where you explicitly state which error you expect. This behavior is
intentional. The rationale is that if you use the <b>error</b>
command to specify an expected error, it is assumed that the test is
sufficiently well characterized that only the specified error is
accceptable.
If you do not use the <b>error</b> command, it is assumed that you
might not know which error to expect or that it might be difficult
to characterize all possible errors that could occur. In this case,
<b>disable_abort_on_error</b> is useful for causing @ref
PAGE_MYSQLTEST mysqltest to continue processing input. This can be
helpful in the following circumstances:
<ul>
<li>
During test case development, it is useful to process all input
even if errors occur so that you can see all errors at once, such
as those that occur due to typographical or syntax errors.
Otherwise, you can see and fix only one scripting problem at a
time.
</li>
<li>
Within a file that is included with a <b>source</b> command by
several different test cases, errors might vary depending on the
processing environment that is set up prior to the <b>source</b>
command.
</li>
<li>
Tests that follow a given statement that can fail are independent
of that statement and do not depend on its result.
</li>
</ul>
*/
###############################################################################
## Creating and Executing Unit Tests
/**
@page PAGE_UNIT_TESTS Creating and Executing Unit Tests
<h3>Table of Contents</h3>
- @subpage PAGE_UNIT_TESTS_TAP
- @subpage PAGE_UNIT_TESTS_GOOGLE
- @subpage PAGE_UNIT_TESTS_MAIN_TEST_RUNS
Storage engines and plugins can have unit tests to test their
components. The top-level <b>Makefile</b> target
<b>test-unit</b> runs all unit tests: It scans the storage engine
and plugin directories, recursively, and executes all executable
files having a name that ends with <b>-t</b>.
The following sections describe MySQL unit testing using TAP and
the Google Test framework.
*/
###############################################################################
## Unit Testing Using TAP
/**
@page PAGE_UNIT_TESTS_TAP Unit Testing Using TAP
The unit-testing facility based on the Test Anything Protocol (TAP)
is mainly used when developing Perl and PHP modules. To write unit
tests for C/C++ code, MySQL has developed a library for generating
TAP output from C/C++ files. Each unit test is written as a separate
source file that is compiled to produce an executable. For the unit
test to be recognized as a unit test, the executable file has to be
of the format <b>mytext-t</b>. For example, you can create a
source file named <b>mytest-t.c</b> that compiles to produce an
executable <b>mytest-t</b>. The executable will be found and run
when you execute <b>make test</b> or <b>make test-unit</b> in the
distribution top-level directory.
Example unit tests can be found in the <b>unittest/examples</b>
directory of a MySQL source distribution. The code for the MyTAP
protocol is located in the <b>unittest/mytap</b> directory.
Each unit test file should be stored in a storage engine or plugin
directory (<b>storage/<em>engine_name</em></b> or
<b>plugin/<em>plugin_name</em></b>), or one of its
subdirectories. A reasonable convention is to create a
<b>unittest</b> subdirectory under the storage engine or plugin
directory and create unit test files in <b>unittest</b>.
*/
###############################################################################
## Unit Testing Using the Google Test Framework
/**
@page PAGE_UNIT_TESTS_GOOGLE Unit Testing Using the Google Test Framework
The Google Test unit-testing framework is available in MySQL source
trees and distributions. Google Test, like MyTAP, provides a unit-
testing framework, but Google Test provides richer functionality,
such as:
- A rich set of predicates
- User-defined predicates and assertions
- Automatic test registration
- Nice error reporting when a predicate fails (with line number, expected
and actual values, and additional comments)
- Test fixtures, and setup/teardown logic
- Death tests
- Disabled tests
- Test filtering and shuffling
Google Test runs on many platforms. Some functionality is missing on
some platforms (such as death tests and parameterized tests), so
those features should not be used.
This section provides notes about using Google Test within the
context of MySQL development. For general-purpose information about
Google Test, see these resources:
- Main Google Test page: http://code.google.com/p/googletest
- Primer: http://code.google.com/p/googletest/wiki/GoogleTestPrimer
- Advanced guide:
http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide
- Google presentation:
http://docs.google.com/present/view?id=dfsbxvm5_0f5s4pvf9
@section SECTION_INSTALL_GOOGLE_TEST Installing Google Test and Running Unit Tests
MySQL sources do not include Google Test. To install it so that you
can use it, use one of these approaches:
<ul>
<li>
<b><em>Install Google Test in individual source
trees</em></b>. Use the <b>-DENABLE_DOWNLOADS=1</b>
configuration option. This causes <b>CMake</b> to download
Google Test and install it in your source tree for you.
</li>
<li>
<b><em>Install a single instance of Google Test</em></b>.
MySQL requires Google Test 1.6 or higher.
<ul>
<li>
To install from a tarball, download
http://googlemock.googlecode.com/files/gmock-1.6.0.zip.
</li>
<li>
To download Google Test from the Subversion repository,
use <b>svn checkout http://googletest.googlecode.com/svn/trunk/
googletest-read-only</b>
</li>
<li>
If a Google Test package is available for your operating
system, you can install it using the package manager. For
example, you might be able to use <b>apt-get</b> for Debian
Linux.
</li>
</ul>
When Google Test has been installed, set the
<b>GTEST_PREFIX</b> environment variable appropriately for
your command interpreter. For example, use this command line for
<b>bash</b>:
@verbatim
GTEST_PREFIX=/path/to/your/install;
export GTEST_PREFIX
@endverbatim
</li>
</ul>
Installing Google Test in individual source trees is the recommended
method. The single-instance installation method can be used only if
all MySQL builds on a machine take place in the same environment
(same operating system, same compiler), for reasons discussed at
https://groups.google.com/group/googletestframework/browse_thread/thread/668eff1cebf5309d?pli=1.
At configuration time, <b>CMake</b> looks for <b>gtest.h</b>. The
build process compiles all Google Test-based unit tests or ignores
them, depending on whether <b>gtest.h</b> is found.
After the build has completed, to run the tests, change location
into the <b>unittest/gunit</b> directory and execute the
<b>ctest</b> command. It need not be invoked with any options, but
you can run it with the <b>`--help`</b> option to see what options
are available.
Another way to run the unit tests is with this command:
@verbatim
make test-unit
@endverbatim
For internal MySQL testing, PushBuild has been extended to install
Google Test as a separate “package”. All trees named
<b>mysql-trunk.*</b> are set up to depend on this package.
@section SECTION_WRITING_UNIT_TESTS Writing Unit Tests for Google Test
The Google Test unit test files are located in the
<b>unittest/gunit</b> directory. You can look at these files to
see how tests are written. Here are some examples:
- <tt>sql_list-t.cc</tt>: A simple test of some list classes
- <tt>mdl-t.cc</tt>: Some tests of metadata locking (MDL),
including testing of lock acquisition from multiple threads
- <tt>mdl_mytap-t.cc</tt>: The same tests as <b>mdl-t.cc</b>, but
written for MyTAP, to illustrate some features of Google Test
Most MyTAP-based tests are likely easily converted to Google Test.
However, there might be low-level tests that absolutely must be run
on every platform, and thus require MyTAP.
As currently implemented, the Google Test unit-test programs produce
TAP output rather than the “plain” alternative. This can be disabled
by using the <b>`--disable-tap-output`</b> command-line option
when running a test executable.
To see what options are available, run a test executable with the
<b>`--help`</b> option.
*/
###############################################################################
## Unit Tests Added to Main Test Runs
/**
@page PAGE_UNIT_TESTS_MAIN_TEST_RUNS Unit Tests Added to Main Test Runs
@ref PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl" will also run unit
tests at the end of full test runs, when being run from within a
build directory. It depends on the unit tests having been built and
defined in a file <b>CTestTestfile.cmake</b> in the top level build
directory. Those will normally be there after a build using
<b>CMake</b>, but will not be in a binary package.
The unit tests are run simply by executing <b>ctest</b> with no
arguments from the top level build directory. The result will be
shown as a single test at the end, named <b>unit_tests</b> which
passes if and only if all unit tests pass. A summary of the result
will be printed, including the name of any failed unit tests. The
set of unit tests will be counted as one test (either passed or
failed) in the overall test summary.
Unit tests will by default be run only if you have not specified any
specific tests or suites on the command line for @ref
PAGE_MYSQL_TEST_RUN_PL "mysql-test-run.pl". This can be overridden
by setting the environment variable <b>MTR_UNIT_TESTS</b> to 0 or 1.
This in turn can be overriden by a command line argument
<b>`--unit-tests`</b> or <b>`--nounit-tests`</b>.
If the file <b>CTestTestfile.cmake</b> and the <b>ctest</b>
command are not both available, unit tests will be silently skipped,
unless you have used the command line option <b>`--unit-tests`</b>.
*/
###############################################################################
## Plugins for Testing Plugin Services
/**
@page PAGE_PLUGINS Plugins for Testing Plugin Services
MySQL server plugins have access to server “services”, as described in
[MySQL Services for Plugins]
(https://dev.mysql.com/doc/refman/8.0/en/plugin-services.html).
MySQL distributions include plugins that demonstrate how to test
plugin service APIs:
<ul>
<li>
The <b>test_framework</b> plugin is a bare bones plugin that
shows the minimum required framework for service testing.
</li>
<li>
The <b>test_services</b> plugin demonstrates how to test the
<b>my_plugin_log_service</b> service in unthreaded context.
</li>
<li>
The <b>test_services_threaded</b> plugin is like
<b>test_services</b>, but for threaded context.
</li>
</ul>
The source code for the plugins is located in the
<b>plugin/test_services</b> directory of MySQL source
distributions. The <b>README</b> file in that directory contains
instructions for running the test cases available for the
<b>test_services</b> and <b>test_services_threaded</b> plugins.
@note
The test plugins in <b>plugin/test_services</b> are daemon plugins
(see [Daemon Plugins]
(https://dev.mysql.com/doc/refman/8.0/en/plugin-types.html#daemon-plugin-type)).
For an example of a nondaemon service-testing plugin plugin, see the
<b>test_security_context.cc</b> file in the <b>plugin/audit_null</b>
directory. This file creates an audit plugin for testing the
<b>security_context</b> service.
Use the following procedure to create a new service-testing plugin
based on one of those provided in the <b>plugin/test_services</b>
directory. Assume that you want to create a new plugin named
<b>test_myservice</b> (or <b>test_myservice_threaded</b> to test
in threaded context).
<ol>
<li>
Select a source file to use as a basis for the new plugin:
<ul>
<li>
To begin with a bare bones plugin, copy <b>test_framework.cc</b>
to <b>test_myservice.cc</b>.
</li>
<li>
To begin with a plugin that already includes code for running tests
in unthreaded context, copy <b>test_services.cc</b> to
<b>test_myservice.cc</b>.
</li>
<li>
To begin with a plugin that already includes code for running tests
in threaded context, copy <b>test_services_threaded.cc</b> to
<b>test_myservice_threaded.cc</b>.
</li>
</ul>
</li>
<li>
There is a plugin descriptor near the end of the new source file. Modify
this descriptor appropriately for your plugin. Change the <b>name</b>,
<b>author</b>, and <b>descr</b> members that indicate the plugin
name and author and provide a description. For example, if you copied
<b>test_framework.cc</b>, those members look like this:
@verbatim
"test_framework",
"Horst Hunger",
"Test framework",
@endverbatim
Change them to something like this:
@verbatim
"test_myservice",
"Your Name Here",
"Test My Service",
@endverbatim
</li>
<li>
Modify your source file appropriately for the service to be tested:
<ul>
<li>
If you copied <b>test_framework.cc</b>, your file has no
tests initially and is set up for unthreaded context. In
this case, add code to the
<b>test_services_plugin_init()</b> function. This code
should invoke the service to be tested.
</li>
<li>
If you copied <b>test_services.cc</b> or
<b>test_services_threaded.cc</b>, the file contains tests
for the <b>my_plugin_log_service</b> service in unthreaded or
threaded contexts. Replace or modify those tests with code
for your own tests.
those tests with code for your own tests.
</li>
</ul>
</li>
</ol>
Compiling your plugin creates a plugin library file, which you
should install in the directory named by the
[plugin_dir]
https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_plugin_dir)
system variable. The file base name is the same as that of the
source file. The file name suffix differs per platform (for example,
<b>.so</b> for Unix and Unix-like systems, <b>.dll</b> for
Windows).
To install or unintall your plugin at server startup, use the
[`--plugin-load`]
(https://dev.mysql.com/doc/refman/8.0/en/server-options.html#option_mysqld_plugin-load) or
[`--plugin-load-add`]
(https://dev.mysql.com/doc/refman/8.0/en/server-options.html#option_mysqld_plugin-load-add)
option. For example, you can use these lines in an option file
(adjust the file name as necessary):
@verbatim
[mysqld]
plugin-load-add=test_myservice.so
@endverbatim
To install or uninstall the plugin at runtime, use these statements
(adjust the plugin name and file name as necessary):
@verbatim
INSTALL PLUGIN test_myservice SONAME 'test_myservice.so';
UNINSTALL PLUGIN test_myservice;
@endverbatim
For addition information about plugin loading, see
[Installing and Uninstalling Plugins]
(https://dev.mysql.com/doc/refman/8.0/en/server-plugin-loading.html).
For information about creating and running test cases for your new
plugin, adapt the instructions in the <b>README</b> file in the
<b>plugin/test_services</b> directory. Test cases for the
<b>test_services</b> and <b>test_services_threaded</b> plugins
are located in <b>mysql-test/suite/test_services</b>.
*/
###############################################################################