MySQL window functions can be used to calculate daily averages or moving averages for a 24h time window relatively easily. In an earlier article

basic window functions were already discussed. In this article, we want to see how we can get daily buckets and moving averages.

A sample program is available, as usual, on GitHub

.

Getting sample data

We will be working with a data table named data, with three columns: a sensor id, a measurement datetime d and a metric value m that was sampled at that time. The table definition looks like this: …

A recommendation we often give to our customers is along the lines of “archive old data” to reduce your database size. There is a tradeoff between keeping all our data online and archiving part of it to cold storage.

There could also be legal requirements to keep certain data online, or you might want to query old data occasionally without having to go through the hassle of restoring an old backup.

In this post, we will explore a very useful feature of AWS RDS/Aurora that allows us to export data to an S3 bucket and run SQL queries directly against it.

Archiving Data to S3

Let’s start by describing the steps we need to take to put our data into an S3 bucket in the required format, which is called Apache Parquet.

Amazon states the Parquet format is up to 2x faster to export and consumes up to 6x less storage in S3, compared to other text formats.

1. Create a snapshot of the database (or …

Codership is pleased to announce a new release of Galera Manager. This brings the installer to version 1.5 and the actual Galera Manager GUI to 1.4.0.

Users will notice some usability improvements like the cluster name being added to the authorized_keys generated file, while there are benefits around gmd generating a new key for the updated installation. There is also a port checker, so if ports are used and unavailable, the user will be informed and Galera Manager will refuse to install. In addition, on Debian/Ubuntu where apt/dpkg supports interactivity, we have now ensured that all steps will remain non-interactive.

  We now have native support to deploy MariaDB Server 10.6 From the Galera Manager GUI standpoint, we have ensured that job monitoring and charts have better performance. One user experience enhancement for monitored or self-hosted clusters? Galera Manager now checks to ensure it has access to …

I recently learned how to use MySQL to generate MySQL statements for several admin-type Data Definition Language (DDL) SQL queries I needed to execute on the server. For many DBA’s this is likely nothing new and in fact, is a well-used tool in their kit. However, I mostly focus on developer-related tasks, so it is a fresh-in-my-mind learning experience. Continue reading for more information…

Some backstory for context

I’ve recently been porting over an existing vanilla PHP application to CodeIgniter 4. One of the new features is bootstrapping all of the necessary MySQL CREATE TABLE statements dynamically based on information parsed from a .fxl file (which is close kin to a .xml file structure-wise).

Each CREATE TABLE statement is complete with FOREIGN KEY constraint clauses to ensure data integrity between linking …

Introduction In this article, I’m going to show you the RevoGain software architecture. RevoGain is a web application that helps Revolut users calculate their stocks and crypto trading gains, and I started building it at the beginning of September 2021. Hype vs. Reality Have you ever been to a software conference and felt disconnected from all those hyped technologies that were presented so beautifully on stage? I certainly had. Whenever I came back from a software conference and looked at my project codebase, I could see a large gap between what’s being... Read More

The post RevoGain Software Architecture appeared first on Vlad Mihalcea.

The Japanese supercomputer has a mission: Help solve the world's biggest problems. The cloud is essential to making that goal reality.

Here’s how SailGP collects, analyzes, and streams racing metrics in real time to help crews deftly navigate their F50 vessels and connect with fans.

Here’s how SailGP collects, analyzes, and streams racing metrics in real time to help crews deftly navigate their F50 vessels and connect with fans.

If you ever had to deal with performance and/or disk space issues related to temporary tables, I bet you eventually found yourself puzzled. There are many possible scenarios depending on the type of temporary table, settings, and MySQL version used. We have observed a pretty long evolution in that matter due to a couple of reasons. One of them was the need to completely eliminate the need to use the deprecated MyISAM engine, and at the same time introduce more performant and reliable alternatives. Another set of improvements was required related to InnoDB, where it was necessary to lower the overhead of temporary tables using that engine.

For that reason, I decided to gather them in a sort of summary which may help to troubleshoot their usage. Due to vast changes between major MySQL releases, I divided the article by them.

…

Hey friends over the weekend (Friday actually) I’ve published another issue of my weekly newsletter, OpenLamp.tech. Continue reading for more information…

This issue is full of great curated reads as well as a specific question I’ve asked readers and the community. Looking for some suggestions on planned migrations for upgrades to my WordPress site.

The Newsletter for PHP and MySQL Developers

In this weeks newsletter, there’s information on:

Here’s the full issue for you to read. You can always subscribe (it’s free) and have the weekly issue of OpenLamp.tech delivered directly to your inbox. Just think, no more waiting for me to post it here each Monday.

Visit the sign-up form below to …