Showing entries 1 to 10 of 1206
10 Older Entries »
Displaying posts with tag: Databases (reset)
Insert Random Data into Tables Using mysql_random_data_load

In this short blogpost, we’ll show you how to use the mysql_random_data_load tool to insert random data into tables. This is a great aide in testing when you have empty tables and need them to be populated with data. We’ve all done it manually (the INSERT INTO … VALUES … way), but that is surely a waste of time. It can add up to a lot if you need to test tables with many columns, or even worse, with foreign keys.

Before saying anything else, we would like to mention that this tool is still under development, and that it will insert random data into tables. Be aware if running it in a production environment!

mysql_random_data_load is a tool created by …

[Read more]
Case in Point: A Year of Customer Experience with Percona

In 2017 we have welcomed many new customers into the Percona fold. It’s always interesting to find out what challenges the Percona team helped them to address and how they chose their relationship with Percona. As unbiased champions of open source database software, our consultancy, support and managed services staff apply their expertise across a wide range of technologies. Here are just a few stories from the past year.

Scaling applications on Amazon RDS the right way

Specializing in on-demand transportation services, Grab needed a high-availability, high performing database engine to serve their rapidly growing application. Grab operates in over 30 densely populated …

[Read more]
LDAP with auth_pam and Python to authenticate against MySQL

If that title looks familiar, it is because a few months ago I posted about LDAP with auth_pam and PHP to authenticate against MySQL.

The good news is that recompiling the connector for Python is a lot easier than for PHP. With PHP, the complexity was due to there being one monolithic package to recompile. The bad news is that there is a slight hitch with Python.

Skip down to the hitch and how to compile MySQLdb for use with auth_pam plugin.

As a quick reminder, here is a repeat of the background:

There are two plugins that can be used. From the documentation, the two plugins are:

  • Full PAM plugin called auth_pam. This plugin uses …
[Read more]
This Week in Data with Colin Charles 19: Percona Live Tickets, Call for Papers and FOSDEM

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

The Percona Live Call For Papers closes on December 22, but why aren’t you submitting already? Don’t wait till the last minute! Look at our broad scope as well. Worth noting that the best prices for tickets are available now until January 7.

FOSDEM is happening in Brussels, Belgium (like it has for the past many years). All I can say is that the schedule is out, and it was very hard to make a choice! See the …

[Read more]
How Scary is Enabling Semi-Sync Replication?

Semi-sync Replication is a plugin available for mysql which allows you to create more durable replication topologies.  For instance you can ensure that in the event of a master crash that at least one of your replicas has all transaction currently written to the master so that when you promote, you know you're not missing any data.

That's a huge simplification.

What's the downside?  Write speed.  If a transaction on your master have to wait until a replica acknowledges it has that transaction, then there is going to be some delay.  Not only that, but your network latency between the two points matters a lot.  If you want greater durability, the cost is performance.

It's important to note that the master doesn't wait until the replica actually runs the transaction on the …

[Read more]
On the future of Data Warehousing. Interview with Jacque Istok and Mike Waas

” Open source software comes with a promise, and that promise is not about looking at the code, rather it’s about avoiding vendor lock-in.” –Jacque Istok.

” The cloud has out-paced the data center by far and we should expect to see the entire database market being replatformed into the cloud within the next 5-10 years.” –Mike Waas.

I have interviewed Jacque Istok, Head of Data Technical Field for Pivotal, and Mike Waas, founder and CEO Datometry.
Main topics of the interview are: the future of Data Warehousing, how are open source and the Cloud affecting the Data Warehouse market, and Datometry Hyper-Q and Pivotal Greenplum.


Q1. What is the future of Data Warehouses?

Jacque Istok: I believe that what we’re seeing …

[Read more]
Lesson 09: Managing Users and Privileges in MySQL

Notes/errata/updates for Chapter 9:
See the official book errata at – Chapter 9 includes pages 297 – 350.

In the fourth paragraph of this chapter, starting with “Most applications don’t need superuser privileges for day-to-day activities” they give you some reasons why you want to create users without the SUPER privilege. There are better reasons than the book gives, which are at the MySQL Manual page for the SUPER privilege.

In the section “Creating and Using New Users” (p. 300) they say “There’s no limit on password length, but we recommend using eight or fewer characters because this avoids problems with system libraries on some platforms.” You should ignore this, this book …

[Read more]
MySQL 5.7 InnoDB Temporary Tablespace – but why?

So, recently we had a runaway query eat up all sorts of temporary table space on our machines. Several machines had several terabytes in their ibtmp1 file after this happened. So I set out to find out more about why the InnoDB temporary tablespace is used, why it is better than using regular files, which was what was used prior to MySQL 5.7, and how to make sure that runaway queries do not end up filling up disk space.

Unfortunately, the manual does not go into why ibtmp1 is better than one file per temporary query, which disappears once the query ends. There are a few sections to look into:

Temporary Table Undo Logs – has one paragraph that states that these are the undo logs for temporary tablespaces. Given that these are undo logs, my guess is that this makes MySQL more crash-safe. But that is just a …

[Read more]
Lesson 07: Advanced MySQL Querying

Notes/errata/updates for Chapter 7:
See the official book errata at – Chapter 7 includes pages 223 – 275.

Supplemental blog post – ORDER BY NULL – read the blog post and the comments!

GROUP BY and HAVING examples – Supplemental blog post. The example of HAVING in the text shows a use case where HAVING is the same function as WHERE. This blog posts shows examples of HAVING that you cannot do any other way.

In the section called “The GROUP BY clause”, on pages 231-232, the book says:
“you can count any column in a group, and you’ll get the same answer, so COUNT(artist_name) is the same as …

[Read more]
Identifying MySQL SSL communication using ngrep

Prior to MySQL 5.7 client communications with a MySQL instance were unencrypted by default. This plaintext capability allowed for various tools including pt-query-digest to analyze TCP/IP traffic. Starting with MySQL 5.7 SSL is enabled by default and will be used in client communications if SSL is detected on the server.

We can detect and confirm this easily with the ngrep command.

Default client connection traffic (5.6)

On a MySQL 5.6 server we monitor the default MySQL port traffic.

mysql56$ sudo ngrep -d eth1 -wi -P ' ' -W single -l port 3306
interface: eth1 (
filter: ( port 3306 ) and ((ip || ip6) || (vlan && (ip || ip6)))

We connect to this server using TCP/IP.

host$ mysql -uexternal -p -h192.168.42.16
mysql56> select 'unencrypted';

We can observe the communication to and from the server (in this example is …

[Read more]
Showing entries 1 to 10 of 1206
10 Older Entries »