Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 27

Displaying posts with tag: Networks (reset)

MySQL Monitoring With Cacti Using Percona Monitoring Plugins (1-minute resolution)
+0 Vote Up -0Vote Down

Today, just like many times before, I needed to configure a monitoring server for MySQL using Cacti and awesome Percona Monitoring Templates. The only difference was that this time I wanted to get it to run with 1 min resolution (using ganglia and graphite, both with 10 sec resolution, for all the rest of our monitoring in Swiftype really spoiled me!). And that’s where the usual pain in the ass Cacti configuration gets really amplified by the million things you need to change to make it work. So, this is a short checklist post for those who need to configure a Cacti server with 1 minute resolution and setup Percona Monitoring Plugins on it.

Configuring Cacti for 60-seconds polling

First of all, we

  [Read more...]
Softlayer Cloud: a Scary Story of One Bad Service
+3 Vote Up -0Vote Down

Disclaimer: the information in this post is the author’s personal opinion and is not the opinion or policy of his employer.


It was spring 2010 when we decided that even though Softlayer‘s server provisioning system is really great and it takes only a few hours to get a new server when we need it, it is still too long sometimes. We wanted to be able to scale up when needed and do it faster. It was especially critical because we were working hard on bringing up Facebook integration to our site and that project could have dramatically changed our application servers cloud capacity requirements.

What buzzword comes to your mind when we talk about scaling up really fast, sometimes within minutes, not hours or days? Exactly – cloud computing! So, after some initial testing and playing around

  [Read more...]
Scribd is Hiring (I’m Looking for an Operations Engineer to Join My Team)
+0 Vote Up -0Vote Down

Scribd is a top 100 site on the web and one of the largest sites built using Ruby on Rails. As one of the first rails sites to reach scale, we’ve built a lot of infrastructure and solved a lot of challenges to get Scribd to where it is today. We actively try to push the envelope and have contributed substantial work back to the open source community.

Scribd has an agile, startup culture and an unusually close working relationship between engineering and ops. You’ll regularly find cross-over work at Scribd, with ops people writing application-layer code and engineers figuring out operations-level problems. We think we’re able to make that work because of the uniquely talented people we have on the team.

To allow us to keep scaling, we’re now looking to add a

  [Read more...]
Nginx-Fu: X-Accel-Redirect From Remote Servers
+0 Vote Up -0Vote Down

We use nginx and its features a lot in Scribd. Many times in the last year we needed some pretty interesting, but not supported feature – we wanted nginx X-Accel-Redirect functionality to work with remote URLs. Out of the box nginx supports this functionality for local URIs only. In this short post I want to explain how did we make nginx serve remote content via X-Accel-Redirect.

First of all, here is why you may need this feature. Let’s imagine you have a file storage on Amazon S3 where you store tons of content. And you have an application where you have some content downloading functionality that you want to be available for logged-in/paying/premium users and/or you want to keep track of downloads your

  [Read more...]
Advanced Squid Caching in Scribd: Cache Invalidation Techniques
+0 Vote Up -0Vote Down

Having a reverse-proxy web cache as one of the major infrastructure elements brings many benefits for large web applications: it reduces your application servers load, reduces average response times on your site, etc. But there is one problem every developer experiences when works with such a cache – cached content invalidation.

It is a complex problem that usually consists of two smaller ones: individual cache elements invalidation (you need to keep an eye on your data changes and invalidate cached pages when related data changes) and full cache purges (sometimes your site layout or page templates change and you need to purge all the cached pages to make sure users will get new visual elements of layout changes). In this post I’d like to look at

  [Read more...]
Enabling IPv6 Support in nginx
+0 Vote Up -0Vote Down

This is going to be a really short post, but for someone it could save an hour of life.

So, you’ve nothing to do and you’ve decided to play around with IPv6 or maybe you’re happened to be an administrator of a web service that needs to support IPv6 connectivity and you need to make your nginx server work nicely with this protocol.

First thing you need to do is to enable IPv6 in nginx by recompiling it with --with-ipv6 configure option and reinstalling it. If you use some pre-built package, check if your nginx already has this key enabled by running nginx -V.

The results should have --with-ipv6 option in configure arguments:

1
2
3
4
5
[root@node ~]# nginx -V
nginx version: nginx/0.7.64





  [Read more...]
Advanced Squid Caching in Scribd: Hardware + Software Used
+0 Vote Up -0Vote Down

After the previous post in this caching related series I’ve received many questions on hardware and software configuration of our servers so in this post I’ll describe our server’s configs and the motivation behind those configs.

Hardware Configuration

Since in our setup Squid server uses one-process model (with an asynchronous requests processing) there was no point in ordering multi-core CPUs for our boxes and since we have a lots of pages on the site and the cache is pretty huge all the servers ended up being highly I/O bound. Considering these facts we’ve decided to use the following hardware specs for the servers:

CPU: One pretty cheap dual-core Intel Xeon 5148 (no need in multiple cores or really high frequencies –

  [Read more...]
Social Networks for MySQL Community
Employee +0 Vote Up -1Vote Down
One of the things about working with MySQL Community is trying to reach you all. If you are reading this then we have reached you.  Talking about community is frankly about where our community exists, where you chat and feel comfortable. To that end we've created a poll about where you might be intrested in hearing *from* us. Please, do let us know which social network you are interested in here from us!

There is no ulterior motive, the poll is anonymous, but it gives us some crucial information on where you want to interact with us, or where we might begin to ask the questions.

Check out the poll.
Network Management Data Reduction and Smoothing -- A MySQL Webinar
Employee +1 Vote Up -0Vote Down

ScienceLogic embeds MySQL (http://www.mysql.com/" target="_blank) in its EM7 network management appliances. An installation of EM7 can perform over half a billion database queries daily, storing massive amounts of data for both real-time and trended performance reporting.

Michael McFadden, senior software architect with ScienceLogic, will discuss all this in an upcoming MySQL webinar (http://mysql.com/news-and-events/web-seminars/display-361.html).

Network Management Data Reduction and Smoothing -- A MySQL Webinar
Employee +0 Vote Up -0Vote Down

ScienceLogic embeds MySQL (http://www.mysql.com/" target="_blank) in its EM7 network management appliances. An installation of EM7 can perform over half a billion database queries daily, storing massive amounts of data for both real-time and trended performance reporting.

Michael McFadden, senior software architect with ScienceLogic, will discuss all this in an upcoming MySQL webinar (http://mysql.com/news-and-events/web-seminars/display-361.html).

Network Management Data Reduction and Smoothing -- A MySQL Webinar
Employee +0 Vote Up -0Vote Down

ScienceLogic embeds MySQL (http://www.mysql.com/" target="_blank) in its EM7 network management appliances. An installation of EM7 can perform over half a billion database queries daily, storing massive amounts of data for both real-time and trended performance reporting.

Michael McFadden, senior software architect with ScienceLogic, will discuss all this in an upcoming MySQL webinar (http://mysql.com/news-and-events/web-seminars/display-361.html).

Follow us on Twitter
Employee +0 Vote Up -0Vote Down

If you are a twitter user, you may like to know that there are many people from MySQL who are regular twitters.

For a start, you may want to follow these ones:

People

  • kajarno Kaj Arnö
  • datacharmer Giuseppe Maxia
  • bytebot Colin Charles
  • LenzGr Lenz Grimmer
  • dups Dups Wijayawardhana
  • Non-people

  • mysqlconf MySQL Users Conference, which is going to be used during the conference to collect feedback for the panel keynotes.
  • planetmysql Planet MySQL feeds
  •   [Read more...]
    Follow us on Twitter
    Employee +0 Vote Up -0Vote Down

    If you are a twitter user, you may like to know that there are many people from MySQL who are regular twitters.

    For a start, you may want to follow these ones:

    People

  • kajarno Kaj Arnö
  • datacharmer Giuseppe Maxia
  • bytebot Colin Charles
  • LenzGr Lenz Grimmer
  • dups Dups Wijayawardhana
  • Non-people

  • mysqlconf MySQL Users Conference, which is going to be used during the conference to collect feedback for the panel keynotes.
  • planetmysql Planet MySQL feeds
  •   [Read more...]
    Follow us on Twitter
    Employee +0 Vote Up -0Vote Down

    If you are a twitter user, you may like to know that there are many people from MySQL who are regular twitters.

    For a start, you may want to follow these ones:

    People

  • kajarno Kaj Arnö
  • datacharmer Giuseppe Maxia
  • bytebot Colin Charles
  • LenzGr Lenz Grimmer
  • dups Dups Wijayawardhana
  • Non-people

  • mysqlconf MySQL Users Conference, which is going to be used during the conference to collect feedback for the panel keynotes.
  • planetmysql Planet MySQL feeds
  •   [Read more...]
    Using SSH tunnel connection as a SOCKS5 proxy
    +0 Vote Up -0Vote Down

    Month ago I was on a vacation and as usual even though our hotel provided us with an internet connection on a pretty decent speeds, I wasn’t able to work there because they’ve banned all tcp ports but some major ones (like 80, 21, etc) and I needed to be able to use ssh, mysql, IMs and other non-web software.

    After a short research I’ve found a pretty simple to set up and easy to use approach to such a connection problems I’d like to describe here.

    First, you’ll need someone (or you can do it before leaving home) to start an ssh daemon on port 80 on one of your servers. I use one of my Slicehub slices for this to permanently have an ability to use it. You can do it like this (if it is a temporary solution):

    1
    # `which sshd` -p 80

    Notice: this `which sshd` was used because on some OSes sshd does not want to start w/o an


      [Read more...]
    Lighttpd Book from Packt – Great Thanksgiving Present
    +0 Vote Up -0Vote Down

    Many people know me as a nginx web server evangelist. But as (IMHO) any professional I think that it is really rewarding to know as much as possible about all the tools available on the market so every time you need to make a decision on some technical issue, you’d consider all pros and cons based on my own knowledge.

    This is why when I received an email from Packt company asking if I’d like to read and review their book on Lighttpd I decided to give it a shot (I usually do not review any books because I do not always have enough time to read a book thoroughly to be able to write a review). So, here are my impressions from this book.

    First, when I received the book, I was in doubt: how such a small book could cover so flexible and multi-purpose piece of software

      [Read more...]
    Advanced Squid Caching for Rails Applications: Preface
    +0 Vote Up -0Vote Down

    Since the day one when I joined Scribd, I was thinking about the fact that 90+% of our traffic is going to the document view pages, which is a single action in our documents controller. I was wondering how could we improve this action responsiveness and make our users happier.

    Few times I was creating a git branches and hacking this action trying to implement some sort of page-level caching to make things faster. But all the time results weren’t as good as I’d like them to be. So, branches were sitting there and waiting for a better idea.

    Few months ago my good friend has joined Scribd and we’ve started thinking on this problem together. As the result of our brainstorming we’ve managed to figure out what were the problems

      [Read more...]
    Bounces-handler Released
    +0 Vote Up -0Vote Down

    Today I’ve managed to finish initial version of our bounces-handler package we use for mailing-related stuff in Scribd.

    Bounces-handler package is a simple set of scripts to automatically process email bounces and ISP‘s feedback loops emails, maintain your mailing blacklists and a Rails plugin to use those blacklists in your RoR applications.

    This piece of software has been developed as a part of more global work on mailing quality improvement in Scribd.com, but it was one of the most critical steps after setting up reverse DNS records, DKIM and SPF.

    The package itself consists of two parts:

    • Perl scripts to process incoming email:
      • bounces processor — could be
      [Read more...]
    Fwd: Scorching hot Startup Needs Scalability Sorcerer and Optimization Freak
    +0 Vote Up -0Vote Down

    Question: Do you think you have what it takes to take a service from a few hundred thousand users to tens of millions of users in 1 year flat? If you do read on and perhaps become the next beloved scalability rockstar of our age.

    We are looking for a data charmer. A mysql magician. A code hack. A funny man. A mad man. A passionate man. Or perhaps a woman who does all these things and more.

    Here’s what you gotta do:

    • Pro-active and reactive performance analysis, monitoring and general database plumbing of all leaky issues.
    • Work with others on the team to help maintain/improve and support the infrastructure for a high traffic, high growth site
    • Optimize and tune the database day to day
    • Algorithmic bent. Develop algos to quicken search times,
      [Read more...]
    Puppet - Admin’s Best Friend
    +0 Vote Up -0Vote Down

    If you’ve ever worked in companies with 5-10+ servers and it was your responsibility to install new boxes, change some configuration files and install new software on many boxes you definitely know how painful this work is. Every time you need to change something on 3-5-100 boxes, you go there and make those changes. Most experienced of us used some weird scripts to perform some task on many boxes or used some stuff like dsh. Even with those tricks I’d never wish this work to anyone.

    While I was working in Galt, I’ve asked our junior admin to check out puppet and try to use it on our servers. After a week of screaming he’s managed to install and

      [Read more...]
    Small Tip: How to set up two interface Xen machine
    +0 Vote Up -0Vote Down

    This will be one of those posts I’d like to publish primarily to be able to coma back later and check it out instead of reading docs again

    So, we have a server with two (or more) network interfaces are we need to be able to use more than one interface in our VDS machines. How do we set it up?

    (more…)

    MySQL Master-Master Replication Manager 1.0 Released
    +0 Vote Up -0Vote Down

    It’s been a long time since we’ve started this project and it is time to make a checkpoint. So, I’ve decided to release final 1.0 version and make 1.X branch stable while all serious development with deep architectural changes will be done 2.X branch (trunk at this moment).

    Changes from previous release:

    • Perl semaphores implementation caused huge memory leaks (mmmd_mod).
    • Now we do not send any commands to hard offline hosts with dead TCP/IP stack to prevent mointoring problems for other hosts.
    • Removed legacy StartSlave method from agent code which caused problems on some Perl versions
    • Added a few fixes to prevent non-exclusive roles
      [Read more...]
    MMM Release 1.0-pre4
    +0 Vote Up -0Vote Down

    New alpha release 1.0-pre4 of the MySQL Master-Master Replication Manager. This release has lots of major fixes and I’m glad to announce first sponsored port of mmm to non-linux platform - it has been ported to Solaris 10. So, here are our changes in this version:

    • Real checks timeouts - I’ve found and fixed lots of problems in checks timeout code and now if you specified in your mmm_mon.conf, that some check should timeout in 5 sec, it would timeout correctly on all supported platforms.
    • External third-party tools using - On all supported non-linux platforms mmm will use system binaries for fping and arp_ping so porting to another platforms would be much easier.
    • Agent
      [Read more...]
    MySQL Conference 2007: First Impressions and Findings
    +0 Vote Up -0Vote Down

    So, first two days of mysqlconf’07 are finished now. What can I say is that without any doubts: It worth it! If you’re working on some high traffic projects, some high-loaded database driven systems, etc, you definitely should attend such conferences - you’d never be able to get such big amounts of information from the best people in the industry as you can get here.

    I’ve been attending mostly practical MySQL scale-out sessions and BOFs and I’ve got really controversial impressions. No - everybody was great, controversial thing is my own level - I never before was so sure that I know nothing at all! I see people here for which most of new things for me (especially in MySQL scaling) are pretty obvious and it is hard to keep myself from some kind of self-beating because I don’t know these things.

    Interesting

      [Read more...]
    SOS!
    +0 Vote Up -0Vote Down

    This blog turned 1 year old last month and I think all of my readers were glad to read it. I’ve never asked for help and I offered help to people who needed it. But today I’m forced to ask my readers for help because it is the first time in my long practice when I really don’t know how to solve my problem.

    (more…)

    MySQL Master-Master replication manager released
    +0 Vote Up -0Vote Down

    So, I’s been a long time since I wrote my last post here. Lots of work and almost no new interesting technologies in my work caused such delay. But today I’m proud to announce release of really interesting project: MySQL Master-Master replication manager - set of flexible scripts for management different MySQL deployment schemes with master-master replication involved.

    More information about this software could be found in detailed announce in Peter Zaitsev blog (actually this software was created by me for his company’s client) or at project page. All your questions and

      [Read more...]
    Looking For Optimal Solution: Ruby On Rails and Mongrel
    +0 Vote Up -0Vote Down

    This article is part of “Looking For Optimal Solution” series, devoted to testing various Ruby On Rails deployment schemes and doing some simple benchmarks on these schemes. General idea of testing is to find subset of most optimal RoR deployment schemes for different situations.

    This small article is about Rails+Mongrel setup and its performance. List of other tested deployment schemes, description of testing methodology and, of course, all benchmark results you can find on “Ruby On Rails Benchmark Summary and Findings” page.

    (more…)

    Showing entries 1 to 27

    Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

    Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.