Showing entries 31 to 40 of 304
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: Python (reset)
Reading RBR binary logs with pt-query-digest

For purposes of auditing anything that goes on our servers we're looking to parse the binary logs of all servers (masters), as with "Anemomaster". With Row Based Replication this is problematic since pt-query-digest does not support parsing RBR binary logs (true for 2.2.12, latest at this time).

I've written a simple script that translates RBR logs to SBR-like logs, with a little bit of cheating. My interest is that pt-query-digest is able to capture and count the queries, nothing else. By doing some minimal text manipulation on the binary log I'm able to now feed it to pt-query-digest which seems to be happy.

The script of course does not parse the binary log …

[Read more]
Django with time zone support and MySQL

This is yet another story of Django web-framework with time zone support and pain dealing with python datetimes and MySQL on the backend. In other words, offset-naive vs offset-aware datetimes.

Shortly, more about the problem. After reading the official documentation about the time zones, it makes clear that in order to reflect python datetime in the necessary time zone you need to make it tz-aware first and than show in that time zone.

Here is the first issue: tz-aware in what time zone? MySQL stores timestamps in UTC and converts for storage/retrieval from/to the current time zone. By default, the current time zone is the server’s time, can be changed on MySQL globally, per connection etc. So it becomes not obvious what was tz of the value initially before stored in UTC. If you …

[Read more]
Understanding reservations, concurrency, and locking in Nova

Imagine that two colleagues, Alice and Bob, issue a command to launch a new virtual machine at approximately the same moment in time. Both Alice’s and Bob’s virtual machines must be given an IP address within the range of IP addresses granted to their project. Let’s say that range is 192.168.20.0/28, which would allow for a total of 16 IP addresses for virtual machines [1]. At some point during the launch sequence of these instances, Nova must assign one of those addresses to each virtual machine.

How do we prevent Nova from assigning the same IP address to both virtual machines?

In this blog post, I’ll try to answer the above question and shed some light on issues that have come to light about the way in which OpenStack projects currently solve (and sometimes fail) to address this issue.

Demonstrating the …

[Read more]
Popular Programming Languages

First of all, Happy New Year!

IEEE Spectrum published a ranking of the most popular programming languages. Computational journalist Nick Diakopoulos wrote the article. While it may surprise some, I wasn’t surprised to find SQL in the top ten.

Nick weighted and combined 12 metrics from 10 sources (including IEEE Xplore, Google, and GitHub) to rank the most popular programming languages.

  • Compiled programming languages (Java [#1], C [#2], C++ [#3], C# [#4], Objective-C [#16])
  • Interpreted programming languages (Python [#5], JavaScript [#6], PHP [#7], Ruby [#8], Perl [#11], HTML [#12])
  • Data languages (SQL [#9], MATLAB …
[Read more]
Does Python MySQLdb Make You Wait?

Recently while writing a data loading application for a customer, we’ve come into a problematic situation with the Python MySQLdb module that can be installed with base RHEL repository or DVD. As a little background, this application uses an HA architecture where a Virtual IP can be assigned to different servers during a failover scenario. However, during failover, as long as the VIPs are not re-assigned, the application could hang waiting for query response. A quick strace reveals something like:

sendto(5, "W\0\0\0\3SELECT * FROM dataloader.bo"..., 91, 0, NULL, 0) = 91
recvfrom(5,

Further test reveals that this problem will not timeout until after about 15 minutes which matches the default tcp_retries2 value on the system (see man 7 tcp):

[root@node1 ~]# cat /proc/sys/net/ipv4/tcp_retries2
15

It turns out that this version of MySQLdb module has client net read and write timeouts set to 0 allowing the TCP …

[Read more]
High-Availability at MySQL Central

This year’s MySQL Central at Oracle Open World was an exhilarating experience. In contrast to the previous year’s MySQL Connect events, MySQL have now got their own Central at the main Oracle Open World. In the previous years, we were always short on time and trying to get a lot of sessions into just two days was just to much. This time I could both present sessions, attend sessions by other users, and also to talk to people in the MySQL community: something that I really enjoy and also find very valuable to see where we should be heading.

This year, the “MySQL Fabric Team” representation on MySQL Central was me and Narayanan Venkateswaran, which is heading the sharding solution in MySQL Fabric. Together with the conference, we also released MySQL Fabric 1.5.2 as the GA release of MySQL Fabric 1.5 containing a few new features:

  • Server …
[Read more]
Connector/Python 2.1 with C Extension using Connector/C

In time for Oracle OpenWorld 2014, we released Connector/Python 2.0. We also released a labs release Connector/Python 2.1 and we have a new feature: a C Extension which uses Connector/C.

This C Extension is an optional, an alternative to the pure Python MySQL Client protocol implementation. One of the reasons to implement it was to improve performance in some situations, for example, when huge result sets are returned. Pure Python is still default, if C Extension is not available.

The following post will get your through downloading and installing the MySQL Connector/Python 2.1.0 labs release.

Requirements

  • Windows users out of luck; the labs release only compiles on Linux, OSX …
[Read more]
Connector/Python 2.1 with C Extension using Connector/C

In time for Oracle OpenWorld 2014, we released Connector/Python 2.0. We also released a labs release Connector/Python 2.1 and we have a new feature: a C Extension which uses Connector/C.

This C Extension is an optional, an alternative to the pure Python MySQL Client protocol implementation. One of the reasons to implement it was to improve performance in some situations, for example, when huge result sets are returned. Pure Python is still default, if C Extension is not available.

The following post will get your through downloading and installing the MySQL Connector/Python 2.1.0 labs release.

Requirements

  • Windows users out of luck; the labs release only compiles on Linux, OSX and other …
[Read more]
MySQL Connector/Python 2.0.1 an 1.2.3 on PyPI

Both MySQL Connector/Python 2.0.1 and 1.2.3 are now available through the Python Package Index or PyPI. Leaving the previous version available is probably a good idea as you can always go back if needed.

Note that we still have no files hosted at PyPI.

MySQL Connector/Python 2.0.1 an 1.2.3 on PyPI

Both MySQL Connector/Python 2.0.1 and 1.2.3 are now available through the Python Package Index or PyPI. Leaving the previous version available is probably a good idea as you can always go back if needed.

Note that we still have no files hosted at PyPI.

Showing entries 31 to 40 of 304
« 10 Newer Entries | 10 Older Entries »