Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 30 of 34 Next 4 Older Entries

Displaying posts with tag: search (reset)

InnoDB Full-Text Search Tutorial
+7 Vote Up -0Vote Down

The InnoDB full-text search capability is an exciting feature. The full-text search itself is generally useful to have in an RDBMS. If an application is using all InnoDB tables except for one that is used for full-text searches, now that last table can be switched to InnoDB. If putting the full-text data in a MyISAM table led to scalability problems, duplication, or a less-than-ideal schema design, now those issues can be addressed.

In this post, I’ll take you through some of the basics of setting up and querying an InnoDB FULLTEXT search index. I’ll leave the scalability and performance aspects to Jimmy’s and Vinay’s blog posts, and just use some toy-sized data for

  [Read more...]
DuckDuckGo and Web of Trust have a trust metric partnership
+0 Vote Up -3Vote Down

There are two search engines that have promise that made themselves public in 2010: DuckDuckGo and Blekko. DuckDuckGo has active search spam removal, you can access it via secure HTTP (HTTPS), and is a search engine that also relies on crowd sourced data.

Web of Trust has reputation ratings of over millions of websites, and has an active community of about 15 million users now. Best of all, there’s no bots doing these ratings, but community members (trust metrics are crowd sourced).

DuckDuckGo and Web of Trust have a

  [Read more...]
Sphinx & MySQL: facts and misconceptions
+6 Vote Up -0Vote Down

Sphinx search is a full text search engine, commonly used with MySQL.

There are some misconceptions about Sphinx and its usage. Following is a list of some of Sphinx’ properties, hoping to answer some common questions.

  • Sphinx is not part of MySQL/Oracle.
  • It is a standalone server; an external application to MySQL.
  • Actually, it is not MySQL specific. It can work with other RDBMS: PostgreSQL, MS SQL Server.
  • And, although described as “free open-source SQL full-text search engine”, it is not SQL-specific: Sphinx can read documents from XML.
  • It is often described as “full text search for InnoDB”. This description is misleading. Sphinx indexes text; be it from any storage engine or external source. It solves, in a way, the issue of “FULLTEXT is
  [Read more...]
TaskFreak! v0.6.2 – Alter Search Plugin
+0 Vote Up -1Vote Down

Background Knowledge


The Search Plugin for TaskFreak! created by DaDaemon and xdu v0.0.1 (March 26, 2007) was designed to create a simple, quick search capability of the tasks title and description. As well it only searched through he current task view (tasks visible at the time) and tasks that are not completed. For some this was not what was desired and would rather have the Search Plugin search through all tasks weather completed or not and as well search through the comments of tasks along with the title and description. I’ll show you how this is done using Searcher, bchristie and davidlmansfield instructions posted on the TaskFreak! Forums.

Solution – Add the Ability to  [Read more...]

MySQL University: Practical Full-Text Search in MySQL
Employee +0 Vote Up -0Vote Down

This Thursday (December 3rd, 16:00 UTC – note the different time), Bill Karwin will talk about Practical Full-Text Search in MySQL. He'll introduce and compare five different approaches of full-text search with MySQL, using built-in functionality as well as third-party tools. It's interesting to see how vastly performance can vary, depending on which tools you use for which purposes.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to.


  [Read more...]
MySQL University: Practical Full-Text Search in MySQL
Employee +0 Vote Up -0Vote Down

This Thursday (December 3rd, 16:00 UTC – note the different time), Bill Karwin will talk about Practical Full-Text Search in MySQL. He'll introduce and compare five different approaches of full-text search with MySQL, using built-in functionality as well as third-party tools. It's interesting to see how vastly performance can vary, depending on which tools you use for which purposes.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't


  [Read more...]
MySQL University: Practical Full-Text Search in MySQL
Employee +0 Vote Up -0Vote Down

This Thursday (December 3rd, 16:00 UTC – note the different time), Bill Karwin will talk about Practical Full-Text Search in MySQL. He'll introduce and compare five different approaches of full-text search with MySQL, using built-in functionality as well as third-party tools. It's interesting to see how vastly performance can vary, depending on which tools you use for which purposes.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't


  [Read more...]
Comparison Between Solr And Sphinx Search Servers (Solr Vs Sphinx – Fight!)
+2 Vote Up -0Vote Down

In the past few weeks I've been implementing advanced search at Plaxo, working quite closely with Solr enterprise search server. Today, I saw this relatively detailed comparison between Solr and its main competitor Sphinx (full credit goes to StackOverflow user mausch who had been using Solr for the past 2 years). For those still confused, Solr and Sphinx are similar to MySQL FULLTEXT search, or for those even more confused, think Google (yeah, this is a bit of a stretch, I know).

Similarities

  • Both Solr and Sphinx satisfy all of your requirements. They're fast and designed to index and search large bodies of data efficiently.
  • Both
  [Read more...]
Handy MySQL documentation indexes
+1 Vote Up -0Vote Down

I just discovered today in the MySQL 5.1 Reference Manual a handy set of additional indexes in the System Navigation section.

  • Index
  • Standard Index
  • C Function Index
  • Command Index
  • Function Index
  • INFORMATION_SCHEMA Index
  • Transaction Isolation Level Index
  • JOIN Types Index
  • Operator Index
  • Option Index
  • Privileges Index
  • SQL Modes Index
  • Status Variable Index
  • Statement/Syntax Index
  • System Variable Index

Perhaps they have been around for some time and I’ve not noticed, but there are much better then searching when you know the content type as per the index list on what you are searching for.

Four short links: 7 August 2009
+1 Vote Up -0Vote Down

  • Defragging the Stimulus -- each [recovery] site has its own silo of data, and no site is complete. What we need is a unified point of access to all sources of information: firsthand reports from Recovery.gov and state portals, commentary from StimulusWatch and MetaCarta, and more. Suggests that Recovery.gov should be the hub for this presently-decentralised pile of recovery data.
  • Memetracker -- site accompanying the research written up by the New York Times as Researchers at Cornell, using powerful computers and clever algorithms, studied the news cycle by looking for repeated phrases and tracking their appearances on 1.6
  •   [Read more...]
    MySQL Librarian: Capturing Community Insights
    Employee +1 Vote Up -0Vote Down

    In the MySQL Community team, our charter is to serve the MySQL community — new and old MySQL users alike. One of the ways we do this is by facilitating information exchange between community members, where the new can learn from the old.

    And there’s been lots of that information exchange going on, such as over mailing lists (in the early days the dominant vehicle), forums, and Planet MySQL.

    One problem with this information exchange has been its ephemeral nature. The same questions pop up for many new users, and should they for some reason not be amongst the issues solved in the MySQL documentation, chances are you’ll have to

      [Read more...]
    Search the planetary archives, and tag your blog entries
    Employee +3 Vote Up -0Vote Down

    A particular blog entry usually feels relevant and topical when fresh, at least to the author. So let’s say a blog entry even carries some non-zero long-term value. How do you find it after a while? And more importantly, how will your readers find your blog entry?

    Descriptive subjects go a long way. But your readers may be searching for “development model” when your header says “release plan”. And even if you anticipate the search words used by your readers, you can only pick one wording for your header.

    Full-text search also helps. There’s now a brand new Search field in the top left corner of Planet MySQL. Chances are you’ll

      [Read more...]
    MYSQL Planet now with tags and search
    Employee +4 Vote Up -0Vote Down

    All this started during a long drive from Charlottesville to Washington, back in November 2008, when I and Dups discussed the status of MySQL Community web presence.

    We agreed that we needed to enhance the usefulness of the tools for the community, and MySQL Planet was the first candidate for change. Externally, you have noticed very little until now. First, a login, then the voting system, the Buzz, the Italian, Japanese, and Russian aggregators, an improved treatment for group blogs, and finally the Tags and Searching.

    Behind the scenes, there is much more. Dups has been

      [Read more...]
    Tags and Searching on PlanetMySQL
    Employee +5 Vote Up -0Vote Down

    Well, let me cut right to the chase. You can finally search the archives of PlanetMySQL. Yes, just go to the sidebar, enter your query and off you go. Yes, it is using a MySQL fulltext search index, and why not, we are MySQL after all ;) The search is happening in Boolean Mode so all the operators and expectations for doing such a search will help you find whatever you are looking for.

    But Wait...! There's a bit more than that.

    You might notice under each post title there's a new row of words.

    We've been importing the tags you place on your posts for years, we just never did anything with them. So what can you do?

    * You can see how people tagged their





      [Read more...]
    MYSQL Planet now with tags and search
    Employee +0 Vote Up -0Vote Down

    All this started during a long drive from Charlottesville to Washington, back in November 2008, when I and Dups discussed the status of MySQL Community web presence.

    We agreed that we needed to enhance the usefulness of the tools for the community, and MySQL Planet was the first candidate for change. Externally, you have noticed very little until now. First, a login, then the voting system, the Buzz, the Italian, Japanese, and Russian aggregators, an improved treatment for group blogs, and finally the Tags and Searching.

    Behind the scenes, there is much more. Dups has been

      [Read more...]
    MYSQL Planet now with tags and search
    Employee +0 Vote Up -0Vote Down

    All this started during a long drive from Charlottesville to Washington, back in November 2008, when I and Dups discussed the status of MySQL Community web presence.

    We agreed that we needed to enhance the usefulness of the tools for the community, and MySQL Planet was the first candidate for change. Externally, you have noticed very little until now. First, a login, then the voting system, the Buzz, the Italian, Japanese, and Russian aggregators, an improved treatment for group blogs, and finally the Tags and Searching.

    Behind the scenes, there is much more. Dups has

      [Read more...]
    Comment Search
    +0 Vote Up -0Vote Down

    You guys are generating an amazing amount of feedback on your blogs. Matt mentioned in the April Wrap-Up that there were 8.6 million comments! Comments are flying in every second of the day.

    And have you ever had one of those blog posts that was good, but the real action was in the comments? The blog post is only half the story, it’s the feedback from everyone else that fills in the rest. To make it easier to find the second half of these stories we’ve added comment search to WordPress.com search.

    Select the comments options from the WordPress.com search page and we’ll hunt through the millions of comments that have been added to WordPress.com blogs to find what you are looking


      [Read more...]
    Buffered Binary Logs…
    +0 Vote Up -0Vote Down

    One of the things that has always bothered me about replication is that the binary logs are written to disk and then read from disk.

    There is are two threads which are for the most part, unaware of each other.

    One thread reads the remote binary logs, and the other writes them to disk.

    While the Linux page buffer CAN work to buffer these logs, the first write will cause additional disk load.

    One strategy, which could seriously boost performance in some situations, would be to pre-read say 10-50MB of data and just keep it in memory.

    If a slave is catching up, it could have GIGABYTES of binary log data from the master. It would then write this to disk. These reads would then NOT come from cache.

    Simply using a small buffer could solve this problem.

    One HACK would be to use a ram drive or tmpfs for logs. I assume that the log thread will block if

      [Read more...]
    2/3 myisam_suggest: an AutoComplete tool for MySQL fulltext indices
    +0 Vote Up -0Vote Down
    As I’ve written in my previous post “1/3 Implementing an AutoSuggest feature using MySQL fulltext indices”, it’s possible to use the MySQL/MyISAM full-text index to extract search words for an AutoSuggest feature with great performance (because the index tree is used actually). This tool, called myisam_suggest, is my first implementation of this. Download Here: myisam_suggest.c [...]
    Tuning Search In Drupal 5
    +0 Vote Up -0Vote Down

    In previous search benchmarks, I utilized random content generated with Drupal's devel module. In these latest benchmarks, I used an actual sanitized copy of the Drupal.org community website database, with email addresses and passwords removed. The first tests were intended to confirm that Xapian continues to perform well with large amounts of actual data. Additional tests were performed to measure the effect of various MySQL tunings and configurations. The following data was derived from several hundred benchmarks run on an Amazon AWS instance over the past week using the SearchBench module.

    These tests confirm that Xapian continues to offer better search performance than Drupal's core search module. Contrary to popular belief, the data also shows that using the InnoDB storage engine for search tables significantly outperforms using the MyISAM storage engine for search tables,

      [Read more...]
    Spinn3r Hiring Senior Systems Administrator
    +0 Vote Up -0Vote Down

    Spinn3r is hiring for an experienced Senior Systems Administrator with solid Linux and MySQL skills and a passion for building scalable and high performance infrastructure.

    About Spinn3r:

    Spinn3r is a licensed weblog crawler used by search engines, weblog analytic companies, and generally anyone who needs access to high quality weblog data.

    We crawl the entire blogosphere in realtime, remove spam, rank, and classifying blogs, and provide this information to our customers.

    Spinn3r is rare in the startup world in that we’re actually profitable. We’ve proven our business model which gives us a significant advantage in future


      [Read more...]
    MySQL Full Text Search by Alex Rubin
    +0 Vote Up -0Vote Down

    Download the PDF: http://www.mysqlfulltextsearch.com/full_text.pdf

    Default search by relevance, default sort is by relevance

    Boolean search is also popular. cats AND dogs. No default sorting, so you need to order the results yourself

    Phrase search

    MySQL Full Text Index, only available with MyISAM, and it supports natural language and Boolean search. ft_min_word_len - 4 characters per word by default is indexed. Frequency based ranking, doesn’t count distance between words

    SELECT * FROM articles WHERE MATCH (title,body) AGAINST (’database’ IN NATURAL LANGUAGE MODE);

    For Boolean, you use AGAINST (’cat AND dog’ IN BOOLEAN MODE).

    n-gram fulltext plugin for CJK languages are available as plugins

    DRBD and MySQL FullText

      [Read more...]
    MySQL: ?SOUNDS LIKE? vs. Full-Text search
    +0 Vote Up -0Vote Down
    A friend of mine asked me: I’m hoping you can help me out with something — I’m trying to optimize a search feature. Since it uses a MySQL database, the search already uses the LIKE statement to get matches for a search query, we might be needing something more flexible. I found mention on MySQL’s website [...]
    O?Reilly Open Source Conference Day Two
    +0 Vote Up -0Vote Down
    A
    O?Reilly Open Source Conference Day One
    +0 Vote Up -0Vote Down
    A
    Five months with MySQL Cluster
    +0 Vote Up -0Vote Down

    So, the whole world changed at dealnews when Yahoo! linked us. We realized that our current infrastructure was not scaling very well. We had to make a change.

    The Problem

    Even though we were using all sorts of cool techniques, the server architecture was really still just a bunch of web servers all serving the same content. In addition to that, our existing systems as the time used a pull method. When a request came in, memcache was checked, if the data was not there, it was fetched from our main MySQL server. So, when there is no data in the cache or when it expires, this was very bad. Like when Yahoo! hit us. Some cache item would expire and 60,000 users would hit a page and each page would try and create the cache item.

      [Read more...]
    Xapian Search Backend Revisited
    +0 Vote Up -0Vote Down

    I wrote previously about looking for a more powerful search solution, and I mentioned that Xapian wasn’t quite so convenient in indexing my data. I then chose to experiment with sphinx a little more, and proceeded to create a number of search engines and indexed a number of data sources in order to decide which direction to go. Unfortunately, while sphinx was convenient and still provides an excellent backend for basic search indexes, I’m revisiting Xapian once again based on it’s more-than-anticipated flexibility. I was brief in my explanation of Xapian however, and didn’t mention some of the more important and powerful aspects of it.

    Xapian provides an API

    Xapian is primarily an API for search indexing/data retrieval. They do provide a handy utility called Omega (available here) for

      [Read more...]
    Sphinx Fulltext Search Engine Part III (continued)
    +0 Vote Up -0Vote Down

    I’m finally taking the time to continue this series =)

    Lyrics Grep

    For testing purposes, I went ahead and scraped about 60,000 song lyrics off of a number of sites and developed a simple search engine for them. The script that did the scraping is pretty nasty (to handle equally nasty HTML that I had to parse through), so I’m going to refrain from posting that script and save myself some embarrassment. Make a pot of coffee, sit down, and write one yourself (or something else that’s similar enough).

    Database Schema

    I created a new database called lyricsgrep with the

      [Read more...]
    Sphinx Fulltext Search Engine Part II (continued)
    +0 Vote Up -0Vote Down

    Note: Part I is located at the page that describes part one

    Disclaimer
    Just a minor clarification to anyone that was confused: I am currently experiencing Sphinx for the first time. Everything I’m writing about is new to me as well, for the most part. So far, I’m drooling over some of it’s capabilities; I may come back in a month and rip it a new ass hole.

    Back to Configuration… (not really, this is the bitching section)

    In preparation for my previous post about Sphinx, I had originally played with a number of configuration options, and even encountered a couple of issues that caused my confused butt to have to debug a number of things, and even recompile


      [Read more...]
    Sphinx Fulltext Search Engine
    +0 Vote Up -0Vote Down

    I’ve been looking for a good solution to manage full text search for large data chunks (2GB - 100GB). I’ve written a couple of solutions using Xapian with limited success, but unfortunately I haven’t been satisfied with it overall. Performance was good, but there were a number of issues with flexibility that have me ultimately looking for another solution.

    At my usual day job, the topic was brought up and I mentioned Xapian and Lucene as solutions, however we’re looking to stay away from Java as it’s not currently in our architecture, and as I stated before: Xapian doesn’t quite have the capabilities I’m looking for to handle even my own systems. Someone brought up Sphinx as something that was being looked into, and I jumped into the typical research

      [Read more...]
    Showing entries 1 to 30 of 34 Next 4 Older Entries

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.