• Suicide
  • having no backups
  • depending on slaves for backup
  • keeping backups on same SAN
  • having a single DBA - Frank didn't like this one at all
  • not keeping binlogs
• Restoring from backup
  • how much time?
  • uncompressed backup ready to mount?
  • separate network for recovery?
• In Fotolog, 1TB of data was severely hit.
  • first problem: backup was highly compressed (tar.gz)
  • uncompressing took hours
  • so keep uncompressed backups (at least last N days)
  • it should be mountable, rather than transferable
• Frank going over recovery modes at …

I had a chance to visit the Kickfire booth after the keynotes and before the first presentation. They gave me a kicking t-shirt, followed by a presentation on the newly announced Kickfire appliance (now in beta, shipping in Fall 2008). Here are some notes I jotted down:

• von Neumann bottleneck
• SQL chip (SQC), packs the power of 10s of conventional CPUs
• Query parallelization on the chip
• On-chip memory - 64GB. No registers - no von Neumann bottleneck
• Beats the performance of a given 3 server, 32 CPU, 130TB box (1TB of actual data - space is used for distributing IO)
• SQC uses column-store, compression, intelligent indexing
• SQL Chip, PCI connection, plugs into a Linux server
  • SQL execution
  • Memory management
  • Loader acceleration
• KDB (Kickfire storage engine), plugs into MySQL …

• Baron Schwartz presents
• only works for SELECTs
• nobody dares admit if they've never seen EXPLAIN
• MySQL actually executes the query
• at each JOIN, instead of executing the query, it fills the EXPLAIN result set
• everything is a JOIN (even SELECT 1)
• Columns in EXPLAIN
  • id: which SELECT the row belongs to
  • select_type
    • simple
    • subquery
    • derived
    • union
    • union result
  • table: the table accessed or its alias
  • type:
    • join
    • range
    • …
  • possible_keys: which indexes looked useful to the …

• Robin Schumacher
• gives overview of MySQL products
• MySQL Enterprise
• MySQL 5.1 announced
  • table/index partitioning -> great for data warehouses, range, cache, key, list, composite, subpartitioning. Partition pruning. Response time greatly improved with proper partitioning.
  • row-based/hybrid replication -> safer and smarter
  • disk-based cluster -> supports bigger DBs
  • built-in job scheduler -> simplified task management
  • problem SQL identification -> easier troubleshooting. Dynamic query tracing is now available, no need to trace things in slow query logs.
  • faster full-text search -> 500% increase in some cases
  • 5.1.24RC available for the conference
• MySQL 6.0
  …

• Speaker: Mikael Ronstrom, PhD, the creator of the Cluster engine
• Explains the cluster structure
• Aspects of performance
  • Response times
  • Throughput
  • Low variation of response times
• Improving performance
  • use low level API (NDB API), expensive, hard
  • use new features in MySQL Cluster Carrier Grade Edition 6.3 (currently 6.3.13), more on this later
  • proper partitioning of tables, minimize communication
  • use of hardware
• NDB API is a C++ record access API
  • supports sending parallel record operations within the same transaction or in different transactions
  • asynchronous and synchronous
  • NDB kernel is programmed entirely asynchronously
  …

Even though I didn’t go to MySQL conf this year (really sad about this), this week is gonna be most active in the community so I decided to do some community stuff too Today I’ve released version 0.3 of our innodb recovery toolkit. Now it became much faster, stable and accurate. At this moment it is possible to recover almost any table from corrupted/deleted tablespace without so much effort as it was before. Here is a short changes list (since 0.1 announced here):

• More MySQL data types added: DECIMAL (both old and new), DATE, TIME
• CHAR data type handling improved in table definitions generator
• Indexes filtering added to page_parser
• 64-bit stat() support added to all tools
• Linux has no isnumber() function so we define our own …

As per the Platinum Partners List PrimeBase Technologies has made a key achievement. I think this is a great list to make for the company. There are only 18 Platinum partners, and PrimeBase Technologies are the third Storage Engine provider and the second first open source storage engine company.

The News Release quotes “As a key milestone in our relationship with both the MySQL Community and MySQL, this partnership strengthens our commitment and involvement in working with the MySQL database server starting with PBXT and following with additional product developments.”

Update Thanks to Ken Jacobs, I need make a correct, we are not the first “open …

Tolven Inc an Open Source Healthcare Solutions provider recently carried out a benchmark using PostgreSQL on Solaris 10 using ZFS on Sun Fire X4600.  The Benchmark report is now available on their website.

The size of the  single database instance using PostgreSQL 8.2.6 (32-bit) peaked at 474GB with the largest table having more than 500 million rows. More information is in the report.

Tolven's  setup guide for developers which includes information about PostgreSQL  setup is also available on their website.

Were there too many "my"'s in that title? Anyway… this week's MySQL conference is promising to be really busy and exciting. I can't wait to finally be there and experience it in all its glory. Thanks to the O'Reilly personal conference planner and scheduler and the advice of my fellow conference goers, I was able to easily (not really) pick out the speeches I am most interested in attending.

Here goes (my pass doesn't include Monday ):

Tuesday

8:30am Tuesday, 04/15/2008

State of MySQL

…

Kevin Murphy just posted Tip of the Day ? max_connections and I thought it might be nice to know what people actually set this to so I’ve created a Survey “What’s your max_connections” (totally anonymous).

For the record, while a consultant the record for a production system was 9,500. Yes you read right, and I might add the client was running a 32-bit OS as well, just to boot.
Apparently they had also tested they could get 9,500 connections, problem is they didn’t do anything with the connections, so didn’t see any per thread memory usage, and I did not get confirmation of how the actually system memory usage was used during the test.