Sometimes, the MySQL Optimizer chooses a wrong plan, and a query that should execute in less than 0.1 second ends-up running for 12 minutes ! This is not a new problem: bugs about this can be traced back to 2014, and a blog post on the subject dates of 2015. But even if this is old news, because this problem recently came to my attention, it is a problem worth writing on.
This is an improved
When Index Condition Pushdown is used, you cannot trust rows examined as reported by the slow query log (nor in some Performance Schema tables) and you cannot trust the global status Innodb_rows_read (nor the InnoDB Metrics dml_reads). These can lead to an incorrect query complexity analysis, to a mistaken estimation of the total load of a MySQL server, and to a lot of wasted time. As we go
Query planning is hard # Have you ever wondered what goes on behind the scenes when you execute a SQL query? What steps are taken to access your data? In this article, I'll talk about the history of Vitess's V3 query planner, why we created a new query planner, and the development of the new Gen4 query planner. Vitess is a horizontally scalable database solution which means that a single table can be spread out across multiple database instances.
Originally posted at Andres's blog. Traditional query optimizing is mostly about two things: first, in which order and from where to access data, and then how to then combine it. You have probably seen the tree shapes execution plans that are produced from query planning. I’ll use an example from the MySQL docs, using FORMAT=TREE which was introduced in MySQL 8.0: mysql>EXPLAINFORMAT=TREE->SELECT*->FROMt1->JOINt2->ON(t1.c1=t2.c1ANDt1.c2<t2.c2)->JOINt3->ON(t2.c1=t3.c1)\G***************************1.row***************************EXPLAIN:->Innerhashjoin(t3.c1=t1.c1)(cost=1.05rows=1)->Tablescanont3(cost=0.35rows=1)->Hash->Filter:(t1.c2<t2.c2)(cost=0.70rows=1)->Innerhashjoin(t2.c1=t1.c1)(cost=0.70rows=1)->Tablescanont2(cost=0.35rows=1)->Hash->Tablescanont1(cost=0.35rows=1)Here we can see that the MySQL optimizer thinks the best plan is to start reading from t1 using a table scan.
MySQL introduced optimizer hints in version 5.7 and greatly
extended the feature in MySQL 8. One thing that has been missing
though is the ability to specify index hints using the syntax of
optimizer hints. This has been improved of in MySQL 8.0.20 with
the introduction of index-level optimizer hints for the
FORCE and IGNORE versions of the index
hints. This blog will look at the new index hint syntax.
Warning
Do not add index hints – neither using the old or new style –
unless you really need them. When you add index hints, you limit
the options of the optimizer which can prevent the optimizer
obtaining the optimal query plan as new optimizer improvements
are implemented or the data changes.
On …
I will try to make it short and clear: if you are writing SQL
queries with “NOT IN” like
SELECT … WHERE x NOT IN (SELECT y FROM …)
you have to be sure to first understand what happens when “x” or
“y” are NULL: it might not be what you want!…
Facebook Twitter LinkedIn
In MySQL 8.0.1, we introduced support for recursive common table expressions (CTE). There are quite a few blog entries showcasing the feature, starting from this one, and there is also a complete documentation. Today, I would like to present a solution to a problem which nearly everybody meets when writing queries with recursive CTE’s: when infinite recursion happens, how to debug ?…
Facebook Twitter LinkedIn
In MySQL 8.0.17, we made an observation in the well-known TPC-H benchmark for one particular query. The query was executing 20% faster than in MySQL 8.0.16. This improvement is because of the “antijoin” optimization which I implemented. Here is its short mention in the release notes:
“The optimizer now transforms a WHERE condition having NOT IN (subquery), NOT EXISTS (subquery), IN (subquery) IS NOT TRUE, or EXISTS (subquery) IS NOT TRUE internally into an antijoin, thus removing the subquery.”…
Facebook Twitter LinkedIn
For a long time, the only algorithm for executing a join in MySQL has been variations of the nested loop algorithm. With the release of MySQL 8.0.18, the server can now execute joins using hash join. This blog post will have a look at how it works, when it is used, and how it compares to the old join algorithms in MySQL in terms of performance.…
Facebook Twitter LinkedIn
MySQL 8.0.18 was just released, and it contains a brand new feature to analyze and understand how queries are executed: EXPLAIN ANALYZE.
What is it?
EXPLAIN ANALYZE is a profiling tool for your queries that will show you where MySQL spends time on your query and why.…
Facebook Twitter LinkedIn