Showing entries 61 to 70 of 118
« 10 Newer Entries | 10 Older Entries »
Displaying posts with tag: Database Management (reset)
Top 3 Questions From Clients

1. This page or area of the website is very slow, why?

There are a lot of components that make up modern internet websites, and a lot of places to get stuck in the mud.  Website performance starts with the browser, what caching it is doing, their bandwidth to your server, what the webserver is doing (caching or not and how), if the webserver has sufficient memory, and then what the application code is doing and lastly how it is interacting with the backend database.

With all this complexity, it's no wonder so many sites struggle.  Typically these types of analysis start with some load testing, to stress test your setup, so you can watch for leaks.  Then some tools are applied to the webserver tier, and the database tier to see where the bottleneck lies.  It may be in the network …

[Read more]
Specialty Technology Consultant – New York Scalability Consultant – MySQL & EC2 Scalability

Amazon EC2 and cloud computing offer great promise for startups to ramp up their online presence quickly.  Navigate those challenges with an strong partner.  We bring 20 years experience to the table with each new client.

  • Scaling Web Applications
  • MySQL High Availability in Amazon EC2
  • Amazon Multi-AZ Deployments
  • Amazon RDS Deployments
  • Migrating to Amazon EC2
  • Migrating to MySQL
  • Managing Backups and Disaster Recovery in the Cloud
  • Horizontal Scalability of MySQL on EC2
  • Horizontal Scalability on Cloud Hosted Servers
  • Evaluating Cloud Providers
  • Evaluating MySQL Distributions and Platforms
  • Strong Customer Facing Skills
  • Integrate Directly with Development Team
  • Agile …
[Read more]
Reply to The Future of the NoSQL, SQL, and RDBMS Markets

Conor O'Mahony over at IBM wrote a good post on a favorite topic of mine “The Future of the NoSQL, SQL, and RDBMS Markets”.  If this is of interest to you then I suggest you read his original post.  I replied in the comments but thought I would also repost my reply here.

-----------------------------------------------------------------------------------------------

Hi Connor, I wish it was as simple as SQL & RDBMS is good for this and NoSQL is good for that.  For me at least, the waters are much muddier than that.

The benefit of SQL & RDBMS is that its general purpose nature has meant it can be applied to a lot of problems, and because of its …

[Read more]
IA Ventures - Jobs shout out

My friends over at IA Ventures are looking both for an Analyst and for an Associate to their team.  If Big Data, New York and start-ups is in your blood then I can’t think of a better VC to be involved in. 

From the IA blog:

"IA Ventures funds early-stage Big Data companies creating competitive advantage through data and we’re looking for two start-up junkies to join our team – one full-time associate / community manager and one full time analyst. Because there are only four of us (we’re a start-up ourselves, in fact), we’ll need you to help us investigate companies, learn about industries, develop investment theses, perform internal operations, organize community events, and work with portfolio companies—basically, you can take on as much …

[Read more]
Realtime Data Pipelines

In life there are really two major types of data analytics.  Firstly, we don’t know what we want to know – so we need analytics to tell us what is interesting.  This is broadly called discovery.  Secondly, we already know what we want to know – we just need analytics to tell us this information, often repeatedly and as quickly as possible.  This is called anything from reporting or dashboarding through more general data transformation and so on.

Typically we are using the same techniques to achieve this.  We shove lots of data into a repository of some from (SQL, MPP SQL, NoSQL, HDFS etc) then run queries/ jobs/ processes across that data to retrieve the information we care about.  

Now this makes sense for data discovery.  If we don’t know what we want to know, having lots of data in a big pile that we can slice and dice in interesting ways is good.   But when we already know what …

[Read more]
What Scales Best?

It is a constant, yet interesting debate in the world of big data.  What scales best?  OldSQL, NoSQL, NewSQL?

I have a longer post coming on this soon.  But for now, let me make the following comments.  Generally, most data technologies can be made to scale - somehow.  Scaling up tends not to be too much of an issue, scaling out is where the difficulties begin.  Yet, most data technologies can be scaled in one form or another to meet a data challenge even if the result isn’t pretty. 

What is best?  Well that comes down to the resulting complexity, cost, performance and other trade-offs.  Trade-offs are key as there are almost always significant concessions to be made as you scale up.

A recent example of mine, I was looking at scalability aspects of MySQL.  In particular, MySQL Cluster.  It is …

[Read more]
MySQL Database Monitoring Best Practices

The MySQL database is a crucial part of a wide variety of products, particularly web applications. Naturally, it is very important to monitor the health status of MySQL.  However, there is constant disagreement on which of the many MySQL status variables provide the best overview on MySQL health status and indicate that something is not right with a server.

It certainly depends on what your application does – tuning read performance is different than optimizing write operations and everything changes when you have a cluster. The average user can use small subset of variables while advanced user want to get more detailed picture of the situation. So there cannot be one set of “magic variables” to quietly optimize every situation. However, it is possible to have a more-or-less optimal set of metrics that will allow to get a “good enough” notion about the general health status of MySQL Server.

The new white paper “ …

[Read more]
101 Tips to MySQL Tuning and Optimization

 

MySQL is a powerful open-source database.  With more and more database driven applications, people have been pushing MySQL to its limits.  Here are 101 tips for tuning and optimizing your MySQL install.  Some tips are specific to the environment they are installed on, but the concepts are universal.   I have divided them up into several categories to help you with getting the most out of MySQL:

 

MySQL Server Hardware and OS Tuning:

1. Have enough physical memory to load your entire InnoDB file into memory – InnoDB is much faster when the file can be accessed in memory rather than from disk.
2. Avoid Swap at all costs …

[Read more]
Who/What to acquire next

Well as predicted, with Aster Data recently being picked up by Teradata most of the key new generation MPP distributed analytics vendors have been acquired (Aster Data, Vertica, Netezza & Greenplum).  This had to happen and was expected to happen.  The MPP Analytics startup “revolution” is over and these technologies will now be integrated into the mainstream.

So what’s next?  As we now, if you are a massive multi-national software company it is a lot less risky to incrementally innovate and leave the development of “game changing” technologies to startups that can be acquired after they prove both the tech and the market.  So what follows MPP? …

[Read more]
What’s hot in Big Data startups?

There are so, so many big data platforms in play at the moment it can be confusing for developers to know where to start.  For startups it used to be simple, MySQL, but dust clouds were created when all the NoSQL platforms started to crash the party 18 months or so ago.  But I do see the dust begin to settle and we are starting to see some market “leaders” appear.  A very unscientific approach is to list the technologies I hear about in the “big data startup” world on a daily basis.  These are, in no particular order:

  • MySQL - yes it is still very much hanging in there despite the Oracle acquisition.  MySQL has been helped by technologies such as AWS RDS and Xeround making it more digestible for big data startups who want to minimize operational overheads.
[Read more]
Showing entries 61 to 70 of 118
« 10 Newer Entries | 10 Older Entries »