Planet MySQL Planet MySQL: Meta Deutsch Español Français Italiano 日本語 Русский Português 中文
10 Newer Entries Showing entries 31 to 40 of 184 10 Older Entries

Displaying posts with tag: mongodb (reset)

Why aren't you using X, version 2
+1 Vote Up -0Vote Down
Sometimes I get asked why am I not using product X where X is anything but MySQL. The products that are suggested change over time and the value of X very much depends on the person asking the question. An ex-manager from my days at Oracle told me that Oracle would be better and developers from the SQL Server team told me the same. For those keeping score there was a social network that ran SQL Server and they were kind of enough to explain why.

Too often this is an assertion rather than a question and it would be more clear to say "I think you should be using X". A better question would be "Why are you using MySQL". This is the burden we carry for running MySQL at scale, but I am not in search of

  [Read more...]
MongoDB, TokuMX and InnoDB for concurrent inserts
+0 Vote Up -0Vote Down
I used the insert benchmark with concurrent insert threads to understand performance limits in MongoDB, TokuMX and InnoDB. The database started empty and eventually was much larger than RAM. The benchmark requires many random writes for secondary index maintenance for an update-in-place b-tree used by MongoDB and InnoDB. The test server has fast flash storage. The work per transaction for this test is inserting 1000 documents/rows where each document/row is small (100 bytes) and has 3 secondary indexes to maintain. The test used 10 client connections to run these transactions concurrently and each client uses a separate collection/table. The performance summaries listed below are based on the context for this test -- fast storage, insert heavy with secondary index maintenance. My conclusion from running many insert benchmark tests is that I don't want to load big databases with  [Read more...]
MongoDB, TokuMX and InnoDB for disk IO-bound, read-only point queries
+0 Vote Up -0Vote Down
This repeats a test that was done on pure-flash servers. The goals are to determine whether the DBMS can efficiently use the IO capacity of a pure-disk server.  The primary metrics are the QPS that the DBMS can sustain and the ratio of disk reads per query. The summary is that a clustered primary key index makes TokuMX and InnoDB much more IO efficient for PK lookups on IO-bound workloads.

TokuMX and InnoDB get much more QPS than MongoDB from the same IO capacity for this workload. TokuMX and InnoDB have a clustered primary index. There is at most 1 disk read per query assuming all non-leaf nodes from the index are in memory and all leaf nodes are not in memory. With MongoDB the primary key index is not clustered so there can be a disk read for the leaf node of

  [Read more...]
Notes on the storage stack
+0 Vote Up -0Vote Down
If you want high performance and quality of service from a DBMS then you need the same from the OS. The MySQL/Postgres/MongoDB crowd doesn't always speak with the Linux crowd. On the bright side there is a good collection of experts from the Linux side of things at my employer and we have begun speaking. There were several long threads on the PG hackers lists about PG+Linux and this lead to a meeting at the LSFMM summit. I am very happy these groups met. We have a lot to learn from each other. DBMS people can explain our IO patterns and get motivated to write DBMS workload simulators (like innosim,   [Read more...]
Insert benchmark on disks, part 2
+1 Vote Up -0Vote Down
I ran more insert benchmark tests for InnoDB on pure disk servers. The previous results with a lot more detail are here. My goal in this case was to use better configuration options for InnoDB on disk and to understand the impact of innodb_flush_neighbors. With the better settings InnoDB sustains a much higher insert rate.

The first problem in the previous tests was that I used a few settings that are better for flash than disk with InnoDB so I increased innodb_write_io_threads from 4 to 32, reduced

  [Read more...]
Insert benchmark for flash, part 2
+1 Vote Up -0Vote Down
I repeated a few of the long running tests for the insert benchmark using flash storage. My goals were to test at least one new configuration, repeat a few tests to confirm the configuration was what I claimed it was and to confirm the impact of doing fsync-on-commit during this test. In this test the write operation adds 1000 small documents and the redo log write is not small. From casual observation I did not see a big impact from doing fsync-on-commit (or a big benefit from not doing it) but that is the point of this post. From the results here the impact from fsync-on-commit still appears to be minor. But remember that this is specific to one workload for which many KB of data are written to the redo log or journal per commit and for which the eventual bottleneck is random  [Read more...]
TokuMX, MongoDB and InnoDB on IO-bound point queries
+1 Vote Up -0Vote Down
I used sysbench to understand whether TokuMX, MongoDB and InnoDB can use most of the IOPs provided by a fast flash device and whether their use is efficient. The workload query fetches one document/row by primary key. This is a very simple workload but helps me to understand how disk read requests are processed. The primary metric is the QPS that can be sustained when the database is much larger than RAM. A secondary metric is the number of disk reads per query during the test. Efficiency, not doing too many reads per query, matters when you want to support many concurrent users.

tl;dr - new database engines are usually worse on multi-core than old database engines. I know there are exceptions to this rule for both new engines, like WiredTiger and RethinkDB, and old engines that won't be

  [Read more...]
on io scheduling again
+1 Vote Up -0Vote Down

Most of database engines have to deal with underlying layers – operating systems, device drivers, firmware and physical devices, albeit different camps choose different methods.
In MySQL world people believe that InnoDB should be handling all the memory management and physical storage operations – maximized buffer pool space, adaptive/fuzzy flushing, crash recovery getting faster, etc. That can result in lots of efficiency wins, as managing everything with data problem in mind allows to tune for efficiency and performance.

Other storage systems (though I hear it from engineers on different types of problems too) like PostgreSQL or MongoDB consider OS to be much smarter and let it do caching or buffering. Which means that in top Postgres expert presentations you will hear much more about operating systems than in MySQL talks. This results in OS knowledge attrition in MySQL world (all you


  [Read more...]
TokuMX, MongoDB and InnoDB versus the insert benchmark with disks
+0 Vote Up -0Vote Down
I used the insert benchmark on servers that use disks in my quest to learn more about MongoDB internals. The insert benchmark is interesting for a few reasons. First while inserting a lot of data isn't something I do all of the time it is something for which performance matters some of the time. Second it subjects secondary indexes to fragmentation and excessive fragmentation leads to wasted IO and wasted disk space. Finally it allows for useful optimizations including write-optimized algorithms (fractal tree via TokuMX, LSM vis RocksDB and WiredTiger) or the InnoDB insert buffer. Hopefully I can move onto other workloads after this week.  [Read more...]
Redo logs in MongoDB and InnoDB
+0 Vote Up -0Vote Down
Both MongoDB and InnoDB support ACID. For MongoDB this is limited to single document changes while InnoDB extends that to multi-statement and possibly long-lived transactions. My goal in this post is to explain how the MongoDB journal is implemented and used to support ACID. Hopefully this will help to understand performance. I include comparisons to InnoDB.

What is ACID?

There are a few interesting constraints on the support for ACID with MongoDB. It uses a per-database reader-writer lock. When a write is in progress all other uses of that database (writes & reads) are blocked. Reads can be done concurrent with other reads but block writes. The manual states that the lock is
  [Read more...]
10 Newer Entries Showing entries 31 to 40 of 184 10 Older Entries

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.