DaveM talks about Ingo’s new SMP lock validator for linux kernel
A note reminding me to go take a look and see what can be ripped out and placed into various bits of MySQL and NDB. Ideally, of course, it could be turned into a LD_PRELOAD for pthread mutexes.
Anybody who wants to look deeper into it before I wake up again is welcome to (and tell me what they find)
In what I laughingly call “spare time” I started hacking on
ha_file.cc, otherwise known as the FILE storage engine. My idea
is relatively simple, I want to be able to store and access my
photos from MySQL. I also want the storage to be relatively
efficient and have the raw image files on disk, not tied up too
much in any different format (my file system is pretty good at
storing multi-megabyte files thank you very much) - it also
doesn’t require any fancy things to re-use space when I delete
things. I should also be able to (efficiently) directly serve the
images out of a web server (satisfying the efficiency itch). You
could also use something like DMF to migrate old rows off to
tape.
So, I started some hacking and some designing and have a working
design and a nearly basically working read/write implementation.
I’ll share the code when it does, in fact, actually work (by
“work” i mean reads and writes basic rows).
I’ve …
At the MySQL User's Conference in April, there were a number of new startups coming to market. One market that I think could be huge is for open source EAI (Enterprise Application Integration) software. I worked in the EAI space for a number of years, helping take Active Software public in 1999 as VP of Marketing and then staying on with webMethods when they acquired Active. The EAI market had huge growth during that time, fueled by the need for new internet systems to integrate with legacy applications. The need still exists today, but the growth has leveled off as traditional EAI solutions have been too complex and expensive for most CIOs.
I think there's a great opportunity for someone to come in and simplify EAI with a solution that works for 90% of the needs out of the box, with the ability for the most demanding customers to customize it to their own needs. This is exactly the type of middleware that could …
[Read more]
My blog link at PlanetMySQL.org is still not updated so here is
list of posts me and Vadim are written during last couple of
weeks.
Group Commit and XA Some more benchmark
results for MySQL 5.0 performance regression because of broken
group commit.
Speedup your LAMP stack with LigHTTPD The
article is about Optimization of Web Server part in LAMP stack in
general and how you can use lighttpd to speed things up in
particular.
How Web Services world affect LAMP Stack
Article discusses some things which become different with Web
Services application. Kind of followup to …
There are many questions how InnoDB allocates memory. I'll try to
give some explanation about the memory allocation at
startup.
Some important constants:
NBLOCKS=count of block in innodb_buffer_pool =
innodb_buffer_pool_size / 16384
OS_THREADS= if (innodb_buffer_pool_size >=
1000Mb) = 50000
else if (innodb_buffer_pool_size >= 8Mb) = 10000
else = 1000 (it's true for *nixes, for Windows there is a bit
another calculation for OS_THREADS)
So InnoDB uses:
Everybody knows that valgrind is great.
Well, I was observing a problem in some MySQL code, it looked like we were writing over some memory that we weren’t meant to be (as the structure hadn’t been initialised yet). But, seeing as this was memory that had been allocated off a MEM_ROOT (one of our memory allocators), valgrind wasn’t gonig to spit out anything.
This is because this bit of memory had already been allocated and subsequently “freed”, but then reallocated. The “free”ing overwrites the memory with garbage (which is what the MEM_ROOT code does) so that you should see crashes (and a pattern) when you do something bad.
The traditional way to troubleshoot this in to modify your memory allocator so that it just calls malloc() and free() all the time (and valgrind will trap them). We have some code in there to do that too. However, this requires …
[Read more]
My good friend Zak Greant spotted a news item at Techworld, where the European
Commission appears to have changed its position on software
patents.
Have a read, it's interesting and I hope that it really does
indicate a new direction from the commission and not just one
loose statement. If it is for real, that's absolutely great, but
let's not be complacent and think "that's it". The pro-swpat
lobbyists certainly won't give up, they have strong interests and
stacks of cash driving them.
Of course it's not helpful to preach to the converted, but
encouragement to continue on this path seems like a good idea.
I'll happily give politicians kudos and encouragement when they
do good stuff!
Also, if you haven't yet …
So what do I do on Memorial Day Weekend?
Well in theory I did not work.
Friday I went to go to see a SIFF, Seattle Internal Film
Festival, movie in the evening. The fact that I can not remember
what it was, and I think I gave it three stars means that I will
not be recommending it to anyone. I have eight movies to see in
three weeks (of which in the three weeks I will go to Boston,
Korea, and probably California). Got home from the movie and
started to port Everydevel to Apache 2.2 and MySQL 5.0. No one
has bothered to do either and I am getting tired of continuing to
set it up on old application stacks. The Apache part took a minor
amount of reading just because of the changes in mod_perl2 from
mod_perl. It was minor and mostly annoying stuff. I also setup
Apache to just use pre-fork since I am personally not sure that I
consider a threaded version of Apache with mod_perl to be safe.
Watched Fire Fly while coding. …
Did you know that you can access tables on handler/engine level? I didn?t until I read the new training material for our new upcoming ?Performance Tuning? course. So, if you didn?t know the HANDLER command, book the course! There?s much more that you might now know yet.
Read http://dev.mysql.com/doc/refman/5.0/en/handler.html for more background.
root@localhost [world]> HANDLER City OPEN; Query OK, 0 rows affected (0.10 sec)
root@localhost [world]> HANDLER City READ FIRST; +----+-------+---------+----------+------------+ | Id | Name | Country | District | Population | +----+-------+---------+----------+------------+ | 1 | Kabul | AFG | Kabol | 1780000 | +----+-------+---------+----------+------------+ 1 …
[Read more]We had discussion today which involved benchmarks of Join speed for MyISAM and Innodb storage engines for CPU bound workload, this is when data size is small enough to fit in memory and so buffer pool.
I tested very simple table, having with about 20.000 rows in it on 32bit Linux. The columns "id" "i" and "c" were populated with same integers so we can allow the same job to be done using different kinds of columns - primary key, integer indexed column and indexed char column. The query is also trivial - the point was to make sure it is not index covered query so it reads the rows and it does not return many rows. I varied the join clause to be id, i and C columns appropriately.
PLAIN TEXT SQL:
