Showing entries 1 to 2
Displaying posts with tag: cjk (reset)
InnoDB 전문 검색 : N-gram Parser

기본 InnoDB 전문 검색(Full Text) 파서는 공백이나 단어 분리자가 토큰인 라틴 기반 언어들에서는 이상적이지만 개별 단어에 대한 고정된 구분자가 없는 중국어, 일본어, 한국어(CJK)같은 언어들에서는 각 단어는 여러개의 문자들의 조합으로 이루어집니다. 그래서 이경우엔 단어 토큰들을 처리할 수 있는 다른 방법이 필요합니다.

우리는 CJK에서 사용할 수 있는 n-gram 파서를 제공하기 위한 새로운 플러그블 전문 파서(pluggable full-text parser)를 MySQL 5.7.6 에서 제공할 수 …

[Read more]
FULLTEXT and Asian Languages with MySQL 5.0

I have seen a few people now ask about using MySQL's FULLTEXT indexing with asian languages such as Chinese, Japanese and Korean (herein referred to as CJK.), however, there doesn't seem to be a good centralised article that covers it.

The information is out there, I just don't think it has been well presented yet.

As I have recently done a bunch of research on this topic for a customer, I figured it might be a good opportunity to make my debut in the MySQL blogosphere.

So here we go...

I'll open by saying that attempting to use FULLTEXT with CJK text in MySQL 5.0 will be unsuccessful.

From the CJK FAQ in the MySQL manual:

"For FULLTEXT searches, we need to know where words begin and end. With Western languages, this is rarely a problem because most (if not all) of these use an easy-to-identify word boundary — the space character. However, this is not …

[Read more]
Showing entries 1 to 2