How MySQL Configuration Impacts the Performance of Web Applications
JAN 26, 2022 • WRITTEN BY ROMAN AGABEKOV
MySQL Configuration tuning is an important component of database management implemented by database professionals and administrators. It aims to configure the database to suit its hardware and workload. But beyond the database management sphere, the usefulness of MySQL Configuration tuning is largely ignored.
We hypothesize that MySQL tuning can significantly affect the performance of web apps. If we can showcase the value of MySQL tuning, we believe that enterprises and organizations may be keen to incorporate this practice on a larger scale.
We hypothesize that MySQL tuning can significantly affect the performance of web apps. If we can showcase the value of MySQL tuning, we believe that enterprises and organizations may be keen to incorporate this practice on a larger scale.
How to Improve Application Performance
Improving application performance with tuning is best achieved with a comprehensive approach that addresses the following areas:
- Server Resources – CPU, Memory, Storage
- Software Configurations – Linux, Nginx, Php…
- Database Management System (DBMS) Configurations – MySQL, PostgreSQL
- Optimize database scheme and change indexes
- Optimize applications – Code, Queries, Architecture…
Testing Approach
Our testing procedure for each web app lets us compare the app's performance before and after configuration using seeded data. By running the test with the default configuration first, we gain valuable control results to compare the tuned configuration against.
We used the following process to prepare and test each application:
We published JMeter tests, MySQL Status, and MySQL Variables during tests on GitHub.
We used the following process to prepare and test each application:
- Deploy Application.
- Seed database with data.
- Prepare test for JMeter.
- Run test for 10 minutes – Ran JMeter test using the Blazemeter performance testing platform.
- Tune MariaDB configuration – After default configuration testing, our setup remained the same, but MariaDB was tuned for workload, server resources, and database size.
- Re-run test – Repeated the JMeter test using Blazemeter for the tuned configuration.
We published JMeter tests, MySQL Status, and MySQL Variables during tests on GitHub.
What metrics we looked at?
The metrics we looked at during this research are:
We collected CPU Utilization and Queries per second metrics to compare the workload.
- Response Time ( Latency ) is the time between sending the request and processing it on the server side to the time the client receives the first byte. It is the important metric that gives you insight into server performance.
- Queries per second is a metric that measures how many queries the database server executes per second.
- CPU Utilization.
We collected CPU Utilization and Queries per second metrics to compare the workload.
Laravel Aimeos
Aimeos Laravel is a popular e-commerce web app framework for creating online shops, marketplaces, and B2B apps. With Aimeos, users can create API-first eCommerce shops for Laravel that can scale to support over 1 billion items. It's available in over 30 languages and has over 300,000 installs.
Testing Setup
To test Aimeos, we started the test with ten users, but we had to decrease the number of users because we couldn't finish the test with the default configuration.
We seeded the database with 500 Mb data.
Our test duration was 10 minutes.
We used:
AWS EC2 instance c5.xlarge with installed Debian 11 as the operating system, Apache as a web server, MariaDB 10.5 set to the default configuration with database size 500 MB.
We seeded the database with 500 Mb data.
Our test duration was 10 minutes.
We used:
AWS EC2 instance c5.xlarge with installed Debian 11 as the operating system, Apache as a web server, MariaDB 10.5 set to the default configuration with database size 500 MB.
MySQL Configuration
The configuration used for Aimeos Laravel is as follows:
query_cache_type=1
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=805306368
max_heap_table_size=16777216
tmp_table_size=16777216
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=201326592
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=805306368
max_heap_table_size=16777216
tmp_table_size=16777216
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=201326592
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
Testing Results
The Aimeos Laravel testing results showcased dramatic performance improvements between the default and tuned configurations.
The optimization of MySQL resulted in a significant improvement in the average server Response Time, which was reduced from 1.4 seconds to under 800 milliseconds.
Response Time ( Latency ) fell by 42% and average CPU utilization by 86%, while Queries per second increased by an incredible 291%, from 12 to 35 queries per second.
The graph of the results is available below:
The optimization of MySQL resulted in a significant improvement in the average server Response Time, which was reduced from 1.4 seconds to under 800 milliseconds.
Response Time ( Latency ) fell by 42% and average CPU utilization by 86%, while Queries per second increased by an incredible 291%, from 12 to 35 queries per second.
The graph of the results is available below:
Latency Aimeos Tuned MySQL Configuration vs Default
CPU Utilization (%), Aimeos Tuned MySQL Configuration vs Default
Queries Per Seconds, Aimeos Tuned MySQL Configuration vs Default
Community Contributors
We teamed up with Laravel developers Gevorg Mkrtchyan and Sergey Sinitsa from Initlab company to investigate this line of questioning and are very grateful for their expertise.
Sergey deployed Aimeos, and Gevorg prepared code for seeding the database.
Sergey deployed Aimeos, and Gevorg prepared code for seeding the database.
Drupal
Drupal is a free and open-source content management system (CMS) written in the PHP programming language. It is used to build websites and web applications of all sizes, from small personal blogs to large corporate and government sites.
Drupal Commerce Kickstart is a distribution of the Drupal, specifically tailored for e-commerce websites. It provides a pre-configured set of modules and features that make it easy to create a fully functional online store.
Drupal Commerce Kickstart is a distribution of the Drupal, specifically tailored for e-commerce websites. It provides a pre-configured set of modules and features that make it easy to create a fully functional online store.
Testing Setup
To test Drupal Commerce Kickstart, we tested with 20 users and prepared the test with around 20 page views and cart actions.
We seeded the two databases with 1 Gb and 3 Gb data.
Our test duration was 10 minutes.
We used:
AWS EC2 instance c5.xlarge with Debian 11 as the operating system, nginx and php-fpm as a web server, MariaDB 10.5 set to the default configuration with database size 1GB AND 3GB.
By repeating the tests with databases of two different sizes, we can see how tuned configurations perform at different scales.
We seeded the two databases with 1 Gb and 3 Gb data.
Our test duration was 10 minutes.
We used:
AWS EC2 instance c5.xlarge with Debian 11 as the operating system, nginx and php-fpm as a web server, MariaDB 10.5 set to the default configuration with database size 1GB AND 3GB.
By repeating the tests with databases of two different sizes, we can see how tuned configurations perform at different scales.
MySQL Configuration
query_cache_type=1
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=1342177280
max_heap_table_size=25165824
tmp_table_size=25165824
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=335544320
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=1342177280
max_heap_table_size=25165824
tmp_table_size=25165824
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=335544320
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
query_cache_type=1
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=4026531840
max_heap_table_size=16777216
tmp_table_size=16777216
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=1006632960
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
query_cache_size=134217728
query_cache_limit=16777216
query_cache_min_res_unit=4096
thread_cache_size=0
key_buffer_size=8388608
max_allowed_packet=1073741824
sort_buffer_size=2097152
read_rnd_buffer_size=262144
bulk_insert_buffer_size=8388608
myisam_sort_buffer_size=8388608
innodb_buffer_pool_chunk_size=134217728
innodb_buffer_pool_size=4026531840
max_heap_table_size=16777216
tmp_table_size=16777216
join_buffer_size=8388608
max_connections=151
table_open_cache=2048
table_definition_cache=1408
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=1006632960
innodb_log_buffer_size=16777216
innodb_write_io_threads=4
innodb_read_io_threads=4
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_thread_concurrency=0
innodb_purge_threads=4
optimizer_search_depth=0
thread_handling=pool-of-threads
thread_pool_size=2
Testing Results
Because Drupal underwent two rounds of testing with different-sized databases, we’ve separated the results into two sections so it’s easy to review and track the results of each round:
1GB Drupal Commerce Database
The Drupal Commerce Kickstart application showed marked improvements in latency and CPU utilization when comparing the default configuration to the tuned configuration.
The optimization of MySQL significantly reduced the average server Response Time, from 150 milliseconds to 60 milliseconds.
Response Time ( Latency ) fell 63% and remained highly stable with the tuned configuration. CPU utilization was reduced by a dramatic 63%. Queries per second increased by a smaller factor, at only 2%, from 692 to 707 queries.
The graph of the results is available below:
The optimization of MySQL significantly reduced the average server Response Time, from 150 milliseconds to 60 milliseconds.
Response Time ( Latency ) fell 63% and remained highly stable with the tuned configuration. CPU utilization was reduced by a dramatic 63%. Queries per second increased by a smaller factor, at only 2%, from 692 to 707 queries.
The graph of the results is available below:
Latency, Drupal 1GB Tuned MySQL Configuration vs Default
CPU Utilization (%), Drupal 1GB Tuned MySQL Configuration vs Default
Queries Per Second, Drupal 1GB Tuned MySQL Configuration vs Default
3GB Drupal Commerce Database
The Drupal Commerce Kickstart application showed even better improvements in latency and CPU utilization for the 3GB database when comparing the default configuration to the tuned configuration.
The optimization of MySQL led to a substantial decrease in the average server Response Time, from 8 seconds to less than 200 milliseconds.
Response Time (Latency) fell by 97% and remained highly stable with the tuned configuration. CPU Utilization was also reduced by 73%.
And while Queries per second only increased by 2% with the 1GB database, we observed a 268% increase with the tuned configuration for the 3GB database, from 760 to 2040 queries per second.
The graph of the results is available below:
The optimization of MySQL led to a substantial decrease in the average server Response Time, from 8 seconds to less than 200 milliseconds.
Response Time (Latency) fell by 97% and remained highly stable with the tuned configuration. CPU Utilization was also reduced by 73%.
And while Queries per second only increased by 2% with the 1GB database, we observed a 268% increase with the tuned configuration for the 3GB database, from 760 to 2040 queries per second.
The graph of the results is available below:
Latency, Drupal 3GB Tuned MySQL Configuration vs Default
CPU Utilization (%), Drupal 3GB Tuned MySQL Configuration vs Default
Queries Per Second, Drupal 3GB Tuned MySQL Configuration vs Default
Community Contributors
We collaborated with Gevorg Mkrtchyan, a Drupal developer from Initlab, to explore this topic and appreciate their expertise. Gevorg set up and prepare the code for populating the database.
Conclusion
Our testing procedure, using Aimeos Laravel and Drupal Commerce Kickstart, showed dramatic improvements in Response Time (Latency), CPU Utilization, and Queries per second after configuring the database server configuration.
Responce Time (Latency) dropped between 42-97%, while CPU Utilization fell between 63-86%. Queries per second increased in every case but ranged widely between 2% for Drupal 1GB and 268% and 291%, respectively, for Drupal 3GB and Aimeos Laravel 500MB.
Responce Time (Latency) dropped between 42-97%, while CPU Utilization fell between 63-86%. Queries per second increased in every case but ranged widely between 2% for Drupal 1GB and 268% and 291%, respectively, for Drupal 3GB and Aimeos Laravel 500MB.
In conclusion, MySQL tuning is an essential aspect of database management that can have a significant impact on the performance of web applications. Poorly performing web applications can lead to increased page load times, slow request handling, and a poor user experience, which can negatively affect SEO and sales. By optimizing the performance of web apps with MySQL tuning, enterprises and organizations can increase sales, pageviews, conversion rates, and SEO rankings.
With this research, we hope to showcase the value of MySQL tuning as a means to improve the performance of web applications and encourage developers to consider this practice when optimizing the performance of their apps.
Using tools like Releem, databases can be quickly and easily configured for optimal performance, reducing the burden on software development teams.
With this research, we hope to showcase the value of MySQL tuning as a means to improve the performance of web applications and encourage developers to consider this practice when optimizing the performance of their apps.
Using tools like Releem, databases can be quickly and easily configured for optimal performance, reducing the burden on software development teams.
WRITTEN BY ROMAN AGABEKOV
