I recently attended an interesting talk by Brion Vibber, CTO of WikiMedia Foundation, a non-profit organisation that runs the infrastructure for Wikipedia. He described how his team of 7 engineers manages the Wikipedia site that gets on an average of 7 billion page views per month. The highlights from the talk are listed below that included the architecture of the site infrastructure to scale up to the traffic that is received. They are ranked amongst the Top 10 sites in terms of traffic.

The site runs on the LAMP stack and you know what that is:

• Linux
• Apache
• MySQL from Sun
• Perl/PHP/Python/Pwhatever :-)
  

WikiMedia runs the site on about 400 x86 servers. Of those, about 250 run the webservers and the remaining run MySQL database. Recently they acquired the …

MySQL is the defacto database of choice for most WebScale and Cloud Computing deployments.

Every day you go to website like Facebook, CraigsList , eBay, Google, PriceGrabber, Yahoo!, and Zappos, you are touching a page that  uses MySQL.
MySQL's popularity is due in large part to its flexibility. MySQL supports over 20 platforms and scales to handle terabytes of data. And, because MySQL is open source, it can be customized to an application's unique specifications. This flexibility has two-fold benefits for ISVs: MySQL is better able to address their applications' specific needs and it won't impose restrictions on their future development.

Through the Sun Partner Advantage Program(SPA) , ISVs can now leverage Sun's entire portfolio of offerings - including MySQL. The SPA Program connects ISVs with free or deeply discounted technology offerings as …

Zembly team has deployed a new application on Zembly called myPicks U.S. Election 2008.

Use this to voice your opinion on the campaign issues being debated in the U.S. Presidential Election - wherever you are in the world.

You can launch the game from: http://zembly.com/mypicksus/

While the frontend is running on Network.Com, the backend including the MySQL server is running on EC2 and data resides on S3

The game is available as a Facebook and MySpace App.

Some cool mashup. They use Dapper to convert some Election related URLs into a API and feed it into the Zembly code/widgets.

Good CloudComputing stuff, check it out!

Seriously kids, what’s with the lack of scalability? I’ve never seen CNN or the NYTimes go down on “trimmed” versions.

Is it a question of bandwidth? Is it lack of hardware?

Take for example, Malaysiakini (the first alternative news source in Malaysia, with a subscription model built around it). It runs FreeBSD, uses PostgreSQL, and has a CMS on top of it (so almost a LAMP stack right there). There’s even use of Squid for caching. Yet there’s lacking load balancing? This is where the cloud can come into play, when there’s high traffic.

Next up, …

On August 22 2008, after reading a post by Lewis Cunningham, I did for the first time a quick evaluation of iCloud, an Internet Operating System by Xcerion.

From a User prospective, an O.S. is a set of applications and a kernel - i.e. the basis on which other applications run. An Internet O.S. is an O.S. that runs in the cloud. You have to image a Desktop with its own icons and a “start” toolbar - running on your browser.

When you double-click on an icon the application linked by that icon will be opened. You can move the window, resize it, maximize it and minimize it - all within your browser.

If you maximize you browser (F11 if you use Internet Explorer) an external …

As part of a project of Technocation, Inc I took a whole bunch of videos at OSCon 2008. The conference was about a month ago, and about 2 weeks ago I’d finished processing and uploading all the videos, but it was only today where I had the 5-6 hours I needed to finish posting all the video, and making this matrix of video.

The video may not be the quality that the O’Reilly folks took and put up on blip tv’s OSCon site, but all the videos here are freely downloadable or playable in your browser.

(more…)

Brian Aker was at the Sun booth today, in a premier slot, where there must have been about 50-60 people, huddled around, to listen to him talk about Drizzle. The project motivations, what’s behind it, what its not aimed to be, and so much more. Check the video out (21 minutes long)!

The Birds of a Feather (BoF) session in the night, was well attended, and there was lots of large discussion on what’s next. I think the important message to take away is that Drizzle doesn’t aim to be MySQL, and there are no plans to “merge” things back (fixes where the code-base is shared though, might make sense). Its also important that the design is for the future, i.e. multi-core machines. It was great to see Brian say that this …

Last week I wrote about whether Google’s potential acquisitions might be stifled by its focus on its own infrastructure software projects but noted that by releasing App Engine the company was encouraging a wider ecosystem of applications based on its platform.

What I didn’t discuss at the time was the potential risk of application vendors finding themselves locked-in to the App Engine platform. Of course Amazon also has this issue, the potential impact of which was …

“Funny how Amazon doesn't use S3 to store any assets for amazon.com”tweet by @gruber

Amazon's S3 suffered a major outage today knocking many websites offline. S3 outage started at approximately 12:00 PM EST and the last time I checked at 11:11PM EST, Smugmug, a popular photo hosting site that extensively uses S3, was still down.

- S3 down for more than 7 hours
- S3 outage, 7 hours and counting
- S3 down again
- Amazon failure downs …

We made some big announcements this week at our annual developer forums, CommunityOne and JavaOne. I thought I'd highlight a couple in particular.

We announced the first commercial release of OpenSolaris - targeting high speed developers and development teams (not consumers...). OpenSolaris focuses on developers wanting to be freed from proprietary software models, who see innovation and automation in operating systems as a source of competitive advantage.

If Solaris 10, OpenSolaris's older brother, is for IT departments prioritizing carrier grade stability over rapid innovation, OpenSolaris targets the exact opposite - developers, from high performance computing to social networking, that prioritize a constantly refreshing repository filled with community innovations (and ZFS-based automated rollback) over an unchanging qualification …