Q: What is the best readahead size?
Perhaps I agree with Dr. Stonebraker. This is my answer which might not be the correct answer. My reasons for O_DIRECT are performance, quality of service (QoS) and manageability and performance might get too much attention. I don't dislike Linux but the VM, buffered IO, readahead and page cache are there for all Linux use cases. They must be general purpose. Complex system software like a DBMS isn't general purpose and can do its own thing when needed. Also, I appreciate that kernel developers have done a lot to make Linux better for a DBMS. One of the perks at FB was easy access to many kernel developers.
Most of my web-scale MySQL/InnoDB experience is with O_DIRECT. While InnoDB can use buffered IO we always chose O_DIRECT. Eventually, RocksDB …
10 Older Entries »
Q: What is the best readahead size?
I am excited to start a new job next week working on performance
at MongoDB. I have been a fan of the people and product for years
and I look forward to contributing from the inside. The reasons I
have been a fan include the rate at which the product has
improved, WiredTiger and their contribution to MongoRocks.
I look forward to learning the modern performance analysis tool chain courtesy of Brendan Gregg. His BPF book should be ready soon and there is much content on his web site. When I had to understand off-cpu stalls from IO and mutex contention there wasn't much available 10 years ago, thus PMP was born. While it served me well it is time to move on.
I will continue to blog including performance comparisons between database …
Reducing response time variance is important for some workloads.
This post explains sources of variance for workloads with high
write rates when the index structure is an LSM or a B-Tree. I
previously wrote about this in my post on durability debt.
- For a given write rate stalls are more likely with a B-Tree than an LSM
- Many RocksDB write stalls can be avoided via configuration
- Write stalls with a B-Tree are smaller but more frequent versus an LSM
- Write stalls are more likely when the redo log isn't forced on commit
- The worst case difference between an LSM and B-Tree is larger when the working set isn't cached
- Life is easier but more expensive when the working set …
I am wary of papers with performance results for too many
products.Too many means including results from
systems for which you lack expertise. Wary means I
have less faith in the comparison even when the ideas in the
paper are awesome. I have expertise in MySQL, MongoDB, RocksDB,
WiredTiger and InnoDB but even for them I have made and
acknowledged ridiculous mistakes.
Database configuration is too hard. There are too many options, most of them aren't significant and the approach is bottom-up. I an expert on this -- in addition to years of tuning I have added more than a few options to RocksDB and MySQL.
This post was motivated by PostgreSQL. I want to run the insert benchmark for it and need a good configuration. I have nothing against PG with the exception of a few too many why …
I miss a proper database related newsletter for busy people. There’s so much happening in the space, from tech, to licensing, and even usage. Anyway, quick tab sweep.
Paul Vallée (of Pythian fame) has been working on Tehama for sometime, and now he gets to do it full time as a PE firm, bought control of Pythian’s services business. Pythian has more than 350 employees, and 250 customers, and raised capital before. More at Ottawa’s Pythian spins out software platform Tehama.
Percona has …[Read more]
We are fueled and energized by the inquisitiveness and eagerness of Database Administrators as a community, reminding us of conducting the next edition of Mydbops Database Meetup. The regular attendees are now looking forward to this quarterly meetup organized by Mydbops, for the benefit of its participants, with the latest hands-on knowledge being shared by the practitioners themselves.
This time we are back in to Diamond District as the venue for this edition of 5th Mydbops Database Meetup, even though, the place of the meeting will be in a different Tower within the same campus at Gojek Tech, Diamond District, 4th Floor, Tower ‘B’, HAL Road, Bangalore – 560 008.
Mydbops Database Meetup
Let us meet together at Mydbops Database Conference. This …[Read more]
Since three years of our entity’s progression, Mydbops IT Solutions, Bangalore, have been an active participant of Open Source India (OSI) conference, conducted by Open Source For You Magazine which is part of EFY Group, New Delhi.
This annual event is an attempt by the team at Open Source For You magazine in partnership with India’s Open Source community and industry—to bring various stake-holders under one roof. The mission of OSI is simple—to accelerate development and deployment of Open Source in India, and beyond.
This year our floor presence has been shifted to the Entry Bay itself, contrasting from our previous presence at the sides of an Auditorium. This position is alongside the IT bigwig like, Microsoft. This development, show cases strides made in the past …[Read more]
Percona Live is always a great opportunity to learn from the best and brightest in the open-source database community. This time, Percona Live Europe is being held at the Hilton Amsterdam Airport Schiphol, The Netherlands from September 30 to October 2, 2019.
Pythian will be present, as has been the case for the past few years, with some of our technical experts speaking on a variety of subjects and technologies.
Feel free to ping any of us during the conference breaks or community events, as we’ll be happy to answer any questions you have about anything open source, cloud, or data-related.
Here are some sessions you won’t want to miss:
Wednesday, October 2
9:00 AM – …[Read more]
Percona announces the release of Percona Toolkit 3.1.0 on September 13, 2019.
Percona Toolkit is a collection of advanced open-source command-line tools, developed and used by the Percona technical staff, that are engineered to perform a variety of MySQL®, MongoDB®, PostgreSQL® and system tasks that are too difficult or complex to perform manually. With over 1,000,000 downloads, Percona Toolkit supports Percona Server for MySQL, MySQL, MariaDB, PostgreSQL, Percona Server for MongoDB, and MongoDB.
Percona Toolkit, like all Percona software, is free and open source. You can download packages …[Read more]
This is a review of Adapting TPC-C Benchmark to Measure Performance of
Multi-Document Transactions in MongoDB which was published in
VLDB 2019. I appreciate that MongoDB and Asya Kamsky took the
time to get this published. That can be a weekend and nights
project when in industry. I also appreciate that this not a
benchmarketing effort. The purpose wasn't to overstate
performance. The purpose was to show how to get good performance
on a TPC-C like workload with MongoDB and realistic hardware and
configurations. I hope for a similar effort on MongoDB with
- Work was done to reduce write-write conflicts which will be more likely given the extra commit latency from using w:majority writeConcern on a 3-node cluster. That …
10 Older Entries »