Showing entries 1 to 5
Displaying posts with tag: LOAD DATA INFILE (reset)
MySQL 5.7, utf8mb4 and the load data infile

In this post, I’ll discuss how MySQL 5.7 handles UTF8MB4 and the load data infile.

Many of my clients have told me that they do not like using the LOAD DATA INFILE statement and prefer to manually parse and load the data. The main reason they do it is issues with the character sets, specifically UTF8MB4 and the load data infile. This was surprising to me as nowadays everyone uses UTF8. MySQL 5.7 (as well as 5.6) has full support for UTF8MB4, which should fix any remaining issues (i.e., you can now load new emoji, like ).

Last week I was investigating an interesting case where we were loading data and got the following error:

mysql -e 'select version()'
+-----------+
| version() |
+-----------+
| 5.7.12    |
+-----------+
$ mysql -vvv testdb < load_data.sql
ERROR 1300 (HY000) at line 1: Invalid …
[Read more]
Shard-Query loader gets a facelift and now Amazon S3 support too

Shard-Query (source) now supports the MySQL “LOAD DATA INFILE” command.

When you use LOAD DATA LOCAL INFILE a single threaded load from the current process will be performed.  You can specify a path to a file anywhere readable by the PHP script.  This allows loading without using the Gearman workers and without using a shared filesystem.

If you do not specify LOCAL, then the Gearman based loader is used.  You must not specify a path to the file when you omit the LOCAL keyword.  This is because the shared path will the pre-pended to the filename automatically.  The shared path must be a shared or network filesystem (NFS,CIFS,etc) and the files to be loaded must be placed on the shared filesystem for the Gearman based loader to work.  This is because workers may run on multiple nodes and …

[Read more]
MariaDB world record price per row 0.0000005$ on a single DELL R710

Don't look at an industry benchmark here, it's a real client story.

200 Billion records in a month and it should be transactional but not durable.

For regular workload we use LOAD DATA INFILE into partitioned InnoDB, but here we have estimated 15TB of RAID storage. This is a lot of disks and it can't no more stay inside a single server internal storage.

MariaDB 5.5 come with TokuDB storage engine for compression, but is it possible in the time frame impose by the workload?

We start benchmarking 380G of raw input data files,  6 Billion rows.

First let's check the compression with the dataset.


Great job my TokuDB 1/5, without tuning a single parameter other than durability! …

[Read more]
Load Delimited Data – CSV to MySQL, Excel to MySQL Database

Loading csv to MySQL or any delimited data files to MySQL database is a very common task frequently questioned about and almost every-time LOAD DATA INFILE come into rescue. In this article I will explain you about all different scenarios and explain you how to get the data loaded directly from csv to MySQL database […]

MySQL 5.1 - LOAD DATA INFILE

Today to debug one of the issue related to InnoDB where the INSERT INTO .. SELECT simply bails with an error as “ERROR 1105 (HY000): Unknown error”; I simply started loading the data onto 5.1 server as it was consistent in 4.1 version (even though it was noted as fixed). So, LOAD DATA INFILE suddenly pops with an error as ..

[Copy to clipboard][-]View Code

1
2
3
4
5
6
7
8
9


mysql> LOAD DATA LOCAL INFILE 'venudata-all.txt' INTO TABLE DOCUMENT_DESC FIELDS TERMINATED BY ',';
ERROR 1148 (42000): The used command IS NOT allowed with this MySQL version
mysql> SHOW variables LIKE '%infile%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile …
[Read more]
Showing entries 1 to 5