One of the key features of MySQL Group Replication is its distributed recovery mechanism. Whenever a new member joins a server group, making use of this plugin component, it reaches to a suitable donor and fetches the data that it misses up until the point it is declared online.…
What is headless web browsing?
It's using a web-browser like application to do automated
fetching and analysis of web pages, without a human user
present.This is different from simply fetching HTML content via
HTTP; headless web browsers typically also load images, process
Javascript code, CSS and layout the page content (albeit in an
invisible way).
The developer can then use scripting (usually Javascript) to
examine the page as it is laid out in memory, as if in a "real"
web browser, to look at the style of text, etc.
We could even use OCR to look for text within images shown in the
page. Why?
- More effectively analyse the content of pages. Lots of pages nowadays contain a huge amount of "boiler plate" uninteresting text, often in HTML elements without semantic meaning (e.g. DIV). Only by using CSS (and sometimes Javascript) are we able to have a computer see the page as a human would
- Generation …
Introduction
This document outlines best practices for loading data into MySQL very quickly. While this is not a comprehensive list of loading methods and configuration, it is a good starting point.
MySQL Configuration
Assuming you are loading into InnoDB tables (and you should probably be doing so), you will want to ensure that MySQL is properly performance tuned for loading large amounts of data. Out of the box MySQL configuration is rarely sufficient for performance with MySQL. It is essential that the InnoDB settings, in particular, be set properly.
First of all, consider the InnoDB Buffer Pool. If you are doing a one-time load, it may be a good idea to configure this as large as possible. In fact, we sometimes set this to approximately 90% of the available RAM on the system for the load. This can then be dropped to between 70 and 80% for …
[Read more]I am attending Percona Live Amsterdam 2015 on September 21-23, 2015.
I will be on stage three times:
- September 22nd at 2:10pm Pivot tables: analytics in pure SQL
- September 23rd at 2:10pm The future of replication is today: new features in practice
- September 23rd at 5:10pm as host and speaker in the Lightning talks
My first talk is a topic that has ben among my favorites for long time: I published an article about it in 2001, and several …
[Read more]In order to simplify the configuration of MySQL for standalone nodes, clusters, and replication configurations, we decided it would be easiest to maintain a common my.cnf file. We have to admit, the idea wasn’t ours; we picked the idea up from www.fromdual.com and thought it was such a great idea, we decided to implement it as well.
Below is our version of a standardized my.cnf implementing
several of our best practices. We hope it will be of
benefit to you.
######################################################################################################## # my.cnf (Configuration file for MySQL) # # Provided by Itchy Ninja Software to implement general best practices for MySQL regardless of server # type. We chose a single file instead of maintaining multiple versions of the configuration file. # Based upon http://www.fromdual.com/mysql-configuration-file-sample # # Sections are included for Percona XTRADB Cluster …[Read more]
Recently, we experimented with active/passive clustering of MySQL via Corosync and Pacemaker. While many choose to use DRBD, our requirement was to simply have storage fail over to the other node. In the real world, this would likely be a SAN. For our POC, we chose to use NFS.
This document will provide a high level overview of a redundant and highly-available clustering solution for MySQL in an active/passive clustering configuration. While there are many ways to make MySQL highly available each has its strengths and weaknesses. These will be explained herein.
Components of the Solution
This solution requires only freely available Open Source
components. Each is depicted in the diagram below:
Two server nodes (virtual servers or physical servers)
Virtual IP (VIP)
Shared Storage such as a SAN with filesystem which can be
mounted on either node
…
The Brainiac Corner is a format where we talk with some of the smartest minds in the system, database, devops, and IT world. If you have opinions on pirates, or anything else related, please don’t hesitate to contact us.
Katherine Daniels is an operations engineer at Etsy who got her start in programming with TI-80 calculators back when Geocities was still cool, and these days has opinions on things like ‘the cloud’ and ‘devops’. She lives in Brooklyn with a perfectly reasonable number of cats, and in her spare time can often be found homebrewing, rock-climbing, or hand crafting knitted server koozies for the data center.
How did you get from stork to brainiac (i.e. what do you do today and how did you get there)?
I started out playing around with computers and …
[Read more]We manage hundreds of servers and have a need to add and remove DBAs, application developers, and so on regularly. Doing this manually is just beyond the scope of what we can realistically manage. Since we are already using LDAP, we wanted to find a way to integrate it with MySQL.
After reading many, many articles and getting diverse opinions as to whether or not it was even possible, we decided to give it a try. From what we read, it appears that MySQL Enterprise edition does exactly what we want; unfortunately, we don’t have an Enterprise support contract ruling that out as an option. Knowing that Percona tends to mimic many of the Enterprise features, we decided to pursue doing this with Percona Server. Further reading showed this could be possible with the PAM plugin which would then authenticate via LDAP. A little indirect, but seemed like our only solution.
Since MySQL does not provide a community version of its LDAP …
[Read more]
With the imminent release of ProxySQL as stable version, and its
presentation at PerconaLive Amsterdam , I decide to write few
blog posts to describe how to install, configure and use it
without digging too much through documentation.
Moreover, practical examples are often easier to
understand.
This tutorial is a long one, although none of the topics are
discussed in details to avoid making it even longer (or
boring).
For this tutorial, we will use a small setup with 4 nodes:
- node1 (192.168.124.101) , mysql master
- node2 (192.168.124.102) , mysql slave
- node3 (192.168.124.103) , mysql slave
- app (192.168.124.104) , application server where proxysql will
also run
Replication setup
Without going into …
Several reports we’re published in the news about how easy it is to access data stored in some NoSQL systems, including MongoDB. This is not surprising because security was rather relaxed in earlier versions of MongoDB . This post lists some of the common vulnerabilities in MongoDB and Percona TokuMX.
Network Security
One key point is to ensure that the bind_ip
setting
is correctly adjusted: in MongoDB 2.4 and Percona TokuMX, it is
not set which means that the server will listen to all available
network interfaces. If proper firewall rules (iptables, Security
Groups in AWS, …) are not in place, your dataset could easily be
queried from anywhere in the world!
In MongoDB 2.6+, bind_ip
is set by default to
127.0.0.1 in the official .deb and .rpm packages. This is great
from a security point of view, but remember that you’ll still
have to adjust the setting if the application servers are not …