Join me for a webinar where I discuss how the recent changes and trends in big data management effect the enterprise.  This event is sponsored by Red Rock and RockSolid.

Overview:

It is an exciting and interesting time to be involved in data. More change of influence has occurred in the database management in the last 18 months than has occurred in the last 18 years. New technologies such as NoSQL & Hadoop and radical redesigns of existing technologies, like NewSQL , will change dramatically how we manage data moving forward. 

These technologies bring with them possibilities both in terms of the scale of data

  [Read more...]

Often I think about challenges that organizations face with “Big Data”.  While Big Data is a generic and over used term, what I am really referring to is an organizations ability to disseminate, understand and ultimately benefit from increasing volumes of data.  It is almost without question that in the future customers will be won/lost, competitive advantage will be gained/forfeited and businesses will succeed/fail based on their ability to leverage their data assets.

It may be surprising what I think are the near term challenges.  Largely I don’t think these are purely technical.  There are enough wheels in motion now to almost guarantee that data accessibility will continue to improve at pace in-line with the increase in data volume.  Sure, there will continue to be lots of interesting innovation with technology, but

  [Read more...]

Conor O'Mahony over at IBM wrote a good post on a favorite topic of mine “The Future of the NoSQL, SQL, and RDBMS Markets”.  If this is of interest to you then I suggest you read his original post.  I replied in the comments but thought I would also repost my reply here.

-----------------------------------------------------------------------------------------------

Hi Connor, I wish it was as simple as SQL & RDBMS is good for this and NoSQL is good for that.  For me at least, the waters are much muddier than that.

The benefit of SQL & RDBMS is

  [Read more...]

My friends over at IA Ventures are looking both for an Analyst and for an Associate to their team.  If Big Data, New York and start-ups is in your blood then I can’t think of a better VC to be involved in. 

From the IA blog:

"IA Ventures funds early-stage Big Data companies creating competitive advantage through data and we’re looking for two start-up junkies to join our team – one full-time associate / community manager and one full time analyst. Because there are only four of us (we’re a start-up ourselves, in fact), we’ll need you to help us investigate companies, learn about industries, develop investment theses, perform internal operations, organize

  [Read more...]

It is a constant, yet interesting debate in the world of big data.  What scales best?  OldSQL, NoSQL, NewSQL?

I have a longer post coming on this soon.  But for now, let me make the following comments.  Generally, most data technologies can be made to scale - somehow.  Scaling up tends not to be too much of an issue, scaling out is where the difficulties begin.  Yet, most data technologies can be scaled in one form or another to meet a data challenge even if the result isn’t pretty. 

What is best?  Well that comes down to the resulting complexity, cost, performance and other trade-offs.  Trade-offs are key as there are almost always significant concessions to be made as you scale up.

A recent example of mine, I was looking at scalability aspects of MySQL.  In particular, MySQL Cluster

  [Read more...]

Well as predicted, with Aster Data recently being picked up by Teradata most of the key new generation MPP distributed analytics vendors have been acquired (Aster Data, Vertica, Netezza & Greenplum).  This had to happen and was expected to happen.  The MPP Analytics startup “revolution” is over and these technologies will now be integrated into the mainstream.

So what’s next?  As we now, if you are a massive multi-national software company it is a lot less risky to incrementally innovate and leave the development of “game changing” technologies to startups that can be acquired after

  [Read more...]

“NoSQL”, for lack of better name, is a generic term that describes any data management system that does not use SQL as a query interface.  Generally this means any data management system that is non-relational, but the term also has also been stretched as far to include the boundaries of what constitutes a data management system at all (such as Hadoop).

Early on (a couple of years back in NoSQL time) when the term was coined I think the positioning was much more aggressive, but more recently this has been softened so now NoSQL is commonly quoted as meaning of “Not only SQL” or “next generation databases” (whatever that means).  The common message you get now is something along the lines of NoSQL systems are

  [Read more...]

With IBM intending to acquire Netezza the predicted consolidation in the distributed analytics market is well underway.  Recent deals include EMC/Greenplum Teradata/Kickfire and now IBM/Netezza.  A good breakdown of this deal is on Curt’s blog.  There is still more to go of course with one of the crown jewels, Vertica, still ripe for the picking. 

What this indicates is that MPP analytics has moved from the innovative edge into the mainstream market and now the more risk

  [Read more...]

I will be at VLDB 2010 next week.  If anyone on this blog is attending and wants to catch up to discuss start ups and innovation in DB, NoSQL, Big Data etc drop me a line and I will try to meet up.

Image by Aranda\Lasch via Flickr

One of my favorite terms at the moment is “Big Data”.  While all terms are by nature subjective, in this post I will try and explain what Big Data means to me.

So what is Big Data?

Big Data is the “modern scale” at which we are defining or data usage challenges.  Big Data begins at the point where need to seriously start thinking about the technologies used to drive our information needs.

While Big Data as a term seems to refer to volume this isn’t the case.  Many existing technologies have little problem physically handling large volumes (TB or PB) of data.  Instead the Big Data



  [Read more...]

Image via Wikipedia

Oracle has published their promises which have reportedly gone a long way to appeasing the EU, so the likely outcome is the takeover of Sun will be approved in January.

My own personal opinion has been the anti-competitive stance really didn’t hold much water.  Reading Oracle’s promises, none appear very extreme (largely agreeing to maintain the status quo) which would lead you to question why it has taken so long to sort out.  But importantly for getting this resolved they are a concession by Oracle and a win for the EU.



  [Read more...]

That’s right — get your free 10-day trial! All the information I know is here:

http://bit.ly/37E9ld

But the basics are: No access to Rough Cuts or Downloads, for new subscribers only. It’s one of those “sign up and if you do not cancel after 10 days, we bill you” — and at $42.99 a month, that’s not a mistake you want to make. Must sign up by Nov. 24th.

To sign up now: https://ssl.safaribooksonline.com/tryitfree

I was asked to send this information along, so I am…Now’s your chance to skim High Performance MySQL, among other high quality books!

Image by Nathan Lanier via Flickr

<< Back from Blogging Hiatus - Update 2

Ingres

No specific announcements from Ingres other than I think the VectorWise stuff is progressing well.

To me Ingres is a bit of a dark horse.  They are open source and doing reasonable revenues.  And they are active in the enterprise market (something MySQL hasn’t really achieved).  But they remain largely

  [Read more...]

Back from Hiatus - Summary Update 1

GoodData

GoodData has launched and they are providing a cloud based analytics platform for use in integration with online apps.  Starting with some initial focus on SalesForce data, but working hard on expanding the list of ISV’s who choose to provide their customers analytics via GoodData.

GoodData was started by “good guy” Czech serial entrepreneur Roman Stanek (NetBeans) and has just raised funds from Andressen Horowitz and appointed Time O’Reilly to the board.  GoodData is interesting because it is simple, accessible and available on demand.  Still early days


  [Read more...]

Image via Wikipedia

FYI - the thoughts here have been gathered from conversations with several individuals, including an interesting conversation yesterday.  As these conversations were off the record I won’t name names here but thanks to those people.

I love open source software and I am a big supporter of many companies that produce open source offerings.  Here I am not going to debate if Oracle acquiring MySQL will be better for MySQL or not as that has been done to

  [Read more...]

Here is a summary of the key discussions I have had over the last month.  Keep in mind, I’m no analyst.  This is largely opinion based on various conversations I have had with the relevant companies (for analyst insight see Curt Monash).

KickFire

I think Kickfire has been doing it a little tough lately.  The difficulties in a startup launching a hardware appliance (and associated logistics) combined with being too focused on the MySQL customer base has impacted the growth of this interesting start up.  But they aren’t taking it lying down and have adjusted the strategy and have added a new appliance to the range.  Kickfire now seems to have a stronger focus

  [Read more...]

Image by plαdys via Flickr

The following is a list of interesting DBMS related links for the week:

• Ellison's Impatience Over Sun
• Intel squeezes one million IOPS from desktop
• Is the RDBMS doomed (yada yada yada) ?
• EDS

  [Read more...]

Image by Snooch2TheNooch via Flickr

I was speaking with Michael Stonebraker this morning.  I mentioned that lately many have been referencing comments he has made over the last couple of years.  And I also mentioned that many had interpreted them as he was implying the RDBMS is “doomed”.  Mike has been saying the same thing for years, but the current NoSQL movement seems to have picked up on this and highlighting one of the RDBMS's own

  [Read more...]

Image via Wikipedia

There will be plenty of detailed coverage on Exadata V2 so I won’t attempt to replicate that.  However I do have a couple of initial thoughts which I would like to share.  For those who missed it, Oracle has just announced Exadata V2 (which is their pre-built “machine”).  Exadata V1 was built using HP equipment, Exadata V2 is using Sun.  The main addition to Exadata V2 seems to be an extra tier in the memory hierarchy, a flash cache.  Oracle is very quick to point out this is not flash

  [Read more...]

I was fortunate enough to speak with Marcin Zukowski earlier about VectorWise.  If you missed it, VectorWise came out of stealth mode a day or two ago.  The have announced a joint partnership with Ingres and essentially are claiming impressive analytic RDBMS performance gains on conventional hardware.

To start with, a key message that I think needs to be communicated here is that this is not a product announcement.  Ingres and VectorWise have announced a partnership in which they of course plan to build products together, today those products are still in the works.

VectorWise is a spin out of

  [Read more...]

I spoke to Daniel Abadi this morning about his HadoopDB announcement that came out a couple of days back.  I am sure this has been a busy time for Daniel and his team over in Yale as HadoopDB has been getting a lot of interest which I am sure will continue to build.

Some notes from our discussion:

• HadoopDB is primarily focused on high scalability and the required availability at scale.  Daniel questions current MPP’s ability to truly scale past 100 nodes whereas Hadoop has real examples on 3000+ nodes.
• HadoopDB like many MPP analytical database platforms

  [Read more...]

Image by weboo via Flickr

A while ago, about 16 years ago now, I had a desktop computer.  It wasn’t a PC.  It was an Acorn.  It had an ARM processor in it.  Despite the rest of the world starting going crazy for the new Pentium chip, the Acorn with its ARM processor could run rings about it in terms of computing power.  And it was simple and easy to use, I used to write applications in assembly code for it (and it didn't have a fan!).

Not too long after that Acorn went under, Arm was already off on

  [Read more...]

Forrester put out its Enterprise Data Management Q2 2009 report a few days ago, you can buy it from Forrester but it also seems to now be available for free from Microsoft here.  I don’t actively seek out these reports as they usually just re-enforce common knowledge (this one was no exception), however as it turned up I managed to find some time on the weekend for a quick read through.

Few surprises in this report, but some key mentions are:

• DBMS market expected to grow 8% annually
• IBM, Microsoft & Oracle own 88% of the DBMS market (by revenue)
• Current market estimated at $27 billion, $32 billion by 2013
• IBM,

  [Read more...]

(yeah, this company is going to have to get used to the Austin Powers references.)

Groovy Corp put out a press release last night that starts the official launch of their SQL Switch relational database platform.

I have been speaking with Groovy for a few months, and while the press release is a bit noisy there is actually some interesting stuff in it.

First, an overview

• They are an in memory RDBMS
• They have worked with Intel to architect from the ground up for large multi processor concurrency
• Initially they are launching as a multi-core appliance
• They claim

  [Read more...]

Recently on a number of sites the benefits for and against have been debated with, on occasion, these conversations falling into abuse being thrown in both directions.

From a pure technical perspective, the TPC benchmarks make little sense and are probably not relevant to 99% of organizations looking to implement a database technology.  But as a tool for generating visibility, debate and improved public awareness of a vendors technology they still have an impact. 

This is marketing, pure and simple.  Having a great TPC result is akin to an author having a great review on Amazon.  Doesn’t mean it is relevant for you but if faced with a stack of titles you haven’t yet read you’ll probably look more closely at the ones you’ve heard

  [Read more...]

Image via Wikipedia

BI/Data Warehousing is an easier market to enter for new database platform vendors.  This is for a few reasons.  Firstly, most BI deployments are custom built projects for each organization.  This means the ability to pick and choose various layers of the stack is much greater. 

Secondly, BI/DW projects success/failure metrics are often tied to database platform driven properties – performance, scalability, load times etc.  The ability to stray outside any existing database platform “standards” to choose a platform that better meets key

  [Read more...]

Image by RaghuP via Flickr

It is important to realize that there is less diversity in the enterprise OLTP market than at any point in the last 20 years.  Essentially this market has been boiled down to Oracle, SQL Server & DB2 (with few isolated exceptions).   Most new deployments are typically using one of the first two options.  The lack of diversity has created a stalemate or chicken &

  [Read more...]

I have been speaking with a lot or new database vendors over the last 12 months and this has prompted me to revisit a post I wrote mid last year.  The basic premise of this post is that your strategy, and the group of people you’re selling to, largely depends on the market sector you are focusing on (Enterprise OLTP, BI/DW, Cloud & Web 2.0).

A database platform by itself is a largely pointless piece of software.  The only way value is produced from a database platform is through the applications that interact with it.  Therefore the only way to be a successful database platform is by making others successful and motivated to use your platform.

Ok, so as a database platform vendor how do you enter this market then? Well there are a few strategies.  Due to the length of this article I have broken it up into Enterprise OLTP, Enterprise Data Warehousing and Cloud & Web 2.0

Oh my. This is just immensely funny & sad at the same time - Amusing Database Videos http://www.bigdatabaselist.com/wiki/Amusing_Database_Videos

Image via Wikipedia

The NoSQL event has triggered a bit of a hard time for the RDBMS the last week.  I won’t add any commentary as this follows what I have been talking about for a while, but here are some of the links.  Most notable is Michael Stonebraker’s post on the ACM site.

• The End of a DBMS Era (Might be Upon Us)

  [Read more...]