Copyright (c) 2015, 2018, Oracle and/or its affiliates. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License, version 2.0, as published by the Free Software Foundation. This program is also distributed with certain software (including but not limited to OpenSSL) that is licensed under separate terms, as designated in a particular file or component or in included license documentation. The authors of MySQL hereby grant you an additional permission to link the program and your derivative works with the separately licensed software that they have included with MySQL. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License, version 2.0, for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ Testing plugin services ----------------------- Plugin services are usually APIs provided by the mysqld server. Theses APIs can be tested with unit test frameworks like "googletest" by developers. In QA it is preferable to test plugin services closer to reality with the help of plugins due to following reasons: - Integrating plugin, plugin services, clients and SQL layer for testing interaction between all. - Having examples for users close to reality (as described in user manual). - Using our (qa) test tools. Implementing a test plugin -------------------------- Depending on the specification of a plugin service, the test plugin must have a special plugin type or even tested with all plugin types. For the frame work to implement test plugins, the plugin type "daemon" has be chosen. It will be loaded (started) with the sql statement "INSTALL PLUGIN" and unloaded (stopped) by calling "UNINSTALL PLUGIN". There is no other communication between server and daemon unless it will be explicitly implemented, e.g. with the help of an API or service. INSTALL and UNINSTALL will be executed by a mtr test. The mtr test may of course contain other sql statements to test the interaction between plugin (service) and SQL layer. The plugin shall write its test results into one or more files. These files can then be written into the result file of the mtr test with the normal mtr commands like "cat_file". With the help of the log_message service the test plugin can easily write messages into "mysqld.1.err". These messages can be made visible in the result file or checked as follows: Load this log into a table with 2 columns (date, info). The column "date" is ignored as not deterministic. The column "info" will be selected by removing "\r" to be compatible to windows (The err log file on windows contains CR,LF as newline). Select the messages so they will be visible in the mtr result file. A perl snippet can be used for that purpose, too. In a plugin status and system (sql) variables can be specified. With status variables the state of the test execution can be reported to the mtr test calling the test plugin. That might be useful to e.g. give a ready message for a test case or the whole plugin. Especially when using threads in a test plugin such status variables are useful to synchronize execution of plugin and mtr test as both are running in parallel. System variables can be used to e.g. control test execution. But, that is very rudimentary as can be used to configure the plugin as its start. Test plugins ------------ There exist test plugins of type "daemon", which can be taken as examples or frameworks and calling mtr tests: - "plugin/test_services" for the test plugin written in C++, - "mysql-test/suite/test_services" for the corresponding mtr tests. Test plugins: - "test_services.cc" showing how to implement a test plugin for simple plugin services like "LogPluginErr" used to write messages to the error log. For each of the services there is a function implemented containing the test cases. Instead of using the standard I/O of C++ it is much more comfortable to used "my_open", "my_write", etc. as it covers all platforms. "LogPluginErr" writes into "mysqld.1.err". The (selected) contents of these files can be seen in the result file of "test_services.test". "test_services.test" didn't create a thread. The effect is that the test (mysqld) is waiting until the plugin completes its work. A waiting condition to synchronize execution of plugin and mtr test is not necessary. - "test_service_threaded.cc" runs the same tests as "test_services.cc", but in a thread. This needs synchronization of plugin and mtr test as both are running in parallel. The synchronization will be done by waiting for the plugin (exactly the thread in the plugin) to be finished. If the thread is needing a very long time then the mtr test may run in a timeout. - "test_framework.cc" is a rudimentary implementation of a daemon plugin, which can be take to create a new one. Of course copying and modifying/extending one of the other plugins to create a new test plugin can be recommended depending on the need of threads. Using threads has the advantage of limiting test execution with a timeout. That may be interesting for long running plugins, testing complex APIs. Plugins needing a short run time may run without threads. To create the plugins and run the test do the following: - modify "CmakeLists.txt" if you want to add a new test plugin, - rebuild ALL or only make test_services by running make in the test_services directory, - in mysql-test: "./mtr --suite=test_services test_services" or "./mtr --suite=test_services test_services_threaded" If a test case in plugin "test_services" shall be switched off run the test as follows: ./mtr --mysqld=--loose-test-services-with-log-message=0 t/test_services.test That shows a simple way of control test execution with system variables. Steps to create a new test plugin --------------------------------- The following is an example of steps to create a test plugin for a plugin service relying on the daemon type: - Create test_.cc file under mysql/plugin/test_plugins deciding if threading is needed or not. - Put the test cases for plugin service to be tested in the init function (preferred) and the clean up in the deinit function. - Add new plugin to plugin.def to make it available to mtr. - Create one or more MTR tests. - MTR test loads plugin and execute the test cases. - After uninstall of the plugin put the test results into the result file of the mtr test. - Create a reference result file.