For testing purposes, especially if you’re working on a project that uses any database technology to store information, you may need data to try out your project. In that case, you have two options:

• Find a good dataset (Kaggle) or,
• Use a library like Faker

Through this blog post, you will learn how to generate test data for MySQL using Faker.

Requirements Dependencies

Make sure all the dependencies are installed before creating the Python script that will generate the data for your project.

You can create a requirements.txt file with the following content:

pandas
sqlalchemy
PyMySQL
tqdm
faker

Once you have created this file, run the following command:

pip install -r requirements.txt

Or if you’re using Anaconda, create an environment.yml file:

name: percona
dependencies:
  - python=3.10
  - pandas
  - …

After installing and configuring MySQL 8.0.30, I installed the Python connector. During that process on AlmaLinux, there were several changes since I last installed the Python’s mysql module. Here are the step-by-step instructions after installing and configuring MySQL Server (blog for those steps).

Using the MySQL Connector/Python X DevAPI Reference, you must install the pip utility before you install the library. You install the pip library as a sudoer user with the following command:

sudo yum install -y pip

Then, using the pip utility as a sudoer user install the mysql-connector-python module with the following command:

sudo pip install mysql-connector-python

Please note that this …

Usually, I receive a lot of emails, and sometimes I read them on my phone and then… I forgot about them.. (shame on me).

On my Linux desktop, I used to use Get Things Gnome for a long time, due to the declining appeal of the project and the end of the extension for Thunderbird, I found it less and less useful.

I was then looking for a solution to have my todolist accessible from everywhere and that I could manage it myself, not hosted somewhere.

I found a very nice, fast and practical project that was easy to deploy and was using MySQL as backend: myTinyTodo.

However, I was missing the possibility to easily create a new task from an email (and especially on my phone).

This is why I decided to write a script that would perform exactly what I was looking for and integrated …

Are they really tricks or simply basic techniques combined to create a solution. Before writing these mechanics for using native MySQL to create a compound JSON object, let me point out that the easiest way to get one is to use the MySQL Node.js library, as shown recently in my “Is SQL Programming” blog post.

Moving data from a relational model output to a JSON structure isn’t as simple as a delimited list of columns in a SQL query. Let’s look at it in stages based on the MySQL Server 12.18.2 Functions that create JSON values.

Here’s how you return single row as a JSON object, which is quite straightforward:

SELECT JSON_OBJECT('first_name',c.first_name,'last_name',c.last_name) AS json_result
FROM   contact c
WHERE …

I could have also called this article When Pandas meet Dolphins in MySQL Shell.

Some time ago, I wrote a post related on how to collect initial relevant data when trying to seek help for MySQL.

Since then, the MySQL Shell engineering team implemented another powerful native utility that collect all the essential information and more and store them in a single zip file.

This zip archive contains TSV and YAML files that, for example, the MySQL Support Team could use to solve your eventual issue.

For more information regarding MySQL Shell’s util.debug.collectDiagnostics(), I encourage you to check the manual.

This tool is …

When I went over my example of using the WITH clause to solve how to use a series of literal values in data sets, some students got it right away and some didn’t. The original post showed how to solve a problem where one value in the data set is returned in the SELECT-list and two values are used as the minimum and maximum values with a BETWEEN operator. It used three approaches with literal values:

• A list of Python dictionaries that require you to filter the return set from the database through a range loop and if statement that mimics a SQL BETWEEN operator.
• A WITH clause that accepts the literals as bind variables to filter the query results inside the query.
• A table design that holds the literals values that an analyst might use for reporting.

It was the last example that required elaboration. I explained you might build a web form that uses a table, and the table could allow a …

As an example for my class on the usefulness of Common Table Expressions (CTEs), I created three examples with Python. They extend an exercise in Chapter 9 on subqueries from Learning SQL by Alan Beaulieu. All of the examples work with the sakila sample database.

These bullets describe the examples:

1. Uses local variables and a range for loop and if statement that uses the variables to evaluate and add an element to the derived table (or query result set) from MySQL.
2. Uses a CTE with substitution variables from the Python program, which eliminates the need to evaluate and add an element to the query result set because the query does that.
3. Uses a table to hold the variables necessary to evaluate and add the element to the query result set.

This is the first Python program:

# Import the library.
import sys
import …

When you use MySQL Shell with extra Python plugins (like these available on GitHub), sometimes, you could need extra Python modules.

Some of my plugins require requests and prettytable for example.

MySQL 8.0.26 uses embedded Python 3.9 and if your system doesn’t have that version (like Oracle Linux 8), you won’t be able to install the missing module.

So how could we install the missing modules ? The easiest method is to use PIP, but if you don’t have PIP for Python 3.9 installed on the system, this will be more complicated…. not really !

This how to install PIP in MySQL Shell:

wget https://bootstrap.pypa.io/get-pip.py
mysqlsh --py -f get-pip.py 

And now you can use PIP to install extra modules:

mysqlsh --pym pip …

    Pandas is a Python software library for data analysis and manipulation.  It allows you to import data from CSV files, SQL, and Excel.  And it is open source.

    Pandas works well with the MySQL Shell.  After installing Pandas with pip install pandas on my Fedora test system, I was able to load Pandas.  The documentation is excellent at pandas.pydata.org   and I was quickly able to get one of their examples working from within mysqlsh.

Did I mention SQL as a source of data? Reading from MySQL is very easy.  

…

Sometimes clients want that the information they collected from the user should be encrypted and stored in a database. Data encryption and…

Continue reading on Thinkdiff »

…