What is headless web browsing?
It's using a web-browser like application to do automated fetching and analysis of web pages, without a human user present.This is different from simply fetching HTML content via HTTP; headless web browsers typically also load images, process Javascript code, CSS and layout the page content (albeit in an invisible way). 
The developer can then use scripting (usually Javascript) to examine the page as it is laid out in memory, as if in a "real" web browser, to look at the style of text, etc.
We could even use OCR to look for text within images shown in the page. Why?

• More effectively analyse the content of pages. Lots of pages nowadays contain a huge amount of "boiler plate" uninteresting text, often in HTML elements without semantic meaning (e.g. DIV). Only by using CSS (and sometimes Javascript) are we able to have a computer see the page as a human would
• Generation …

Introduction

This document outlines best practices for loading data into MySQL very quickly.  While this is not a comprehensive list of loading methods and configuration, it is a good starting point.

MySQL Configuration

Assuming you are loading into InnoDB tables (and you should probably be doing so), you will want to ensure that MySQL is properly performance tuned for loading large amounts of data.  Out of the box MySQL configuration is rarely sufficient for performance with MySQL.  It is essential that the InnoDB settings, in particular, be set properly.

First of all, consider the InnoDB Buffer Pool.  If you are doing a one-time load, it may be a good idea to configure this as large as possible.  In fact, we sometimes set this to approximately 90% of the available RAM on the system for the load.  This can then be dropped to between 70 and 80% for …

I am attending Percona Live Amsterdam 2015 on September 21-23, 2015.

I will be on stage three times:

• September 22nd at 2:10pm Pivot tables: analytics in pure SQL
• September 23rd at 2:10pm The future of replication is today: new features in practice
• September 23rd at 5:10pm as host and speaker in the Lightning talks

My first talk is a topic that has ben among my favorites for long time: I published an article about it in 2001, and several …

In order to simplify the configuration of MySQL for standalone nodes, clusters, and replication configurations, we decided it would be easiest to maintain a common my.cnf file.  We have to admit, the idea wasn’t ours; we picked the idea up from www.fromdual.com and thought it was such a great idea, we decided to implement it as well.

Below is our version of a standardized my.cnf implementing several of our best practices.  We hope it will be of benefit to you.

########################################################################################################
# my.cnf (Configuration file for MySQL)
#
# Provided by Itchy Ninja Software to implement general best practices for MySQL regardless of server 
# type.  We chose a single file instead of maintaining multiple versions of the configuration file.
# Based upon http://www.fromdual.com/mysql-configuration-file-sample 
#
# Sections are included for Percona XTRADB Cluster …

Recently, we experimented with active/passive clustering of MySQL via Corosync and Pacemaker.  While many choose to use DRBD, our requirement was to simply have storage fail over to the other node.  In the real world, this would likely be a SAN.  For our POC, we chose to use NFS.

This document will provide a high level overview of a redundant and highly-available clustering solution for MySQL in an active/passive clustering configuration. While there are many ways to make MySQL highly available each has its strengths and weaknesses. These will be explained herein.

Components of the Solution
This solution requires only freely available Open Source components. Each is depicted in the diagram below:

 Two server nodes (virtual servers or physical servers)
 Virtual IP (VIP)
 Shared Storage such as a SAN with filesystem which can be mounted on either node
 …

The Brainiac Corner is a format where we talk with some of the smartest minds in the system, database, devops, and IT world. If you have opinions on pirates, or anything else related, please don’t hesitate to contact us. 

Katherine Daniels is an operations engineer at Etsy who got her start in programming with TI-80 calculators back when Geocities was still cool, and these days has opinions on things like ‘the cloud’ and ‘devops’. She lives in Brooklyn with a perfectly reasonable number of cats, and in her spare time can often be found homebrewing, rock-climbing, or hand crafting knitted server koozies for the data center.

How did you get from stork to brainiac (i.e. what do you do today and how did you get there)?

I started out playing around with computers and …

We manage hundreds of servers and have a need to add and remove DBAs, application developers, and so on regularly. Doing this manually is just beyond the scope of what we can realistically manage. Since we are already using LDAP, we wanted to find a way to integrate it with MySQL.

After reading many, many articles and getting diverse opinions as to whether or not it was even possible, we decided to give it a try. From what we read, it appears that MySQL Enterprise edition does exactly what we want; unfortunately, we don’t have an Enterprise support contract ruling that out as an option. Knowing that Percona tends to mimic many of the Enterprise features, we decided to pursue doing this with Percona Server. Further reading showed this could be possible with the PAM plugin which would then authenticate via LDAP. A little indirect, but seemed like our only solution.

Since MySQL does not provide a community version of its LDAP …

With the imminent release of ProxySQL as stable version, and its presentation at PerconaLive Amsterdam , I decide to write few blog posts to describe how to install, configure and use it without digging too much through documentation.
Moreover, practical examples are often easier to understand.

This tutorial is a long one, although none of the topics are discussed in details to avoid making it even longer (or boring).

For this tutorial, we will use a small setup with 4 nodes:
- node1 (192.168.124.101) , mysql master
- node2 (192.168.124.102) , mysql slave
- node3 (192.168.124.103) , mysql slave
- app (192.168.124.104) , application server where proxysql will also run


Replication setup
Without going into …

Several reports we’re published in the news about how easy it is to access data stored in some NoSQL systems, including MongoDB. This is not surprising because security was rather relaxed in earlier versions of MongoDB . This post lists some of the common vulnerabilities in MongoDB and Percona TokuMX.

Network Security

One key point is to ensure that the bind_ip setting is correctly adjusted: in MongoDB 2.4 and Percona TokuMX, it is not set which means that the server will listen to all available network interfaces. If proper firewall rules (iptables, Security Groups in AWS, …) are not in place, your dataset could easily be queried from anywhere in the world!

In MongoDB 2.6+, bind_ip is set by default to 127.0.0.1 in the official .deb and .rpm packages. This is great from a security point of view, but remember that you’ll still have to adjust the setting if the application servers are not …

Thanks to all who attended my webinar last week on Writing Application Code for MySQL High Availability.  This blog is for me to address the extra questions I didn’t have time to answer on the stream.

What do you think about using Galera Cluster but writing to a single Node with LVS ?

Whatever HA strategy you like that can present  a layer 3 or layer 4 to your application tier is fine.  A lot of people using PXC us it in a single-writer (master/slave) kind of way.

Is there any way we can determine slave lag and then decide to use weather master or slave? for e.g. instead of using query to find if data is available in slave then …. use if lag_time < xyz?

One of my main points was that this is usually more expensive to implement inside your application code than it …