Showing entries 1 to 6
Displaying posts with tag: big-data (reset)
Druid @ Airbnb Data Platform

How Druid enables analytics at Airbnb Realtime and batch analytics at Airbnb and the role Druid plays in our analytics system architecture

By Pala Muthiah and Jinyang Li

Wikipedia: A druid was a member of the high-ranking professional class in ancient Celtic cultures. It is believed that druids perform secret rituals in forests, not unlike this unique Experience on Airbnb!

Airbnb serves millions of guests and hosts in our community. Every second, their activities on, such as searching, booking, and messaging, generate a huge amount of data we anonymize and use to improve the community’s experience on our platform.

The Data Platform Team at Airbnb strives to leverage this …

[Read more]
Metacat: Making Big Data Discoverable and Meaningful at Netflix

by Ajoy Majumdar, Zhen Li

Most large companies have numerous data sources with different data formats and large data volumes. These data stores are accessed and analyzed by many people throughout the enterprise. At Netflix, our data warehouse consists of a large number of data sets stored in Amazon S3 (via Hive), Druid, Elasticsearch, Redshift, Snowflake and MySql. Our platform supports Spark, Presto, Pig, and Hive for consuming, processing and producing data sets. Given the diverse set of data sources, and to make sure our data platform can interoperate across these data sets as one “single” data warehouse, we built Metacat. In this blog, we will discuss our motivations in building Metacat, a …

[Read more]
Scaling Airbnb’s Experimentation Platform

At Airbnb, we are constantly iterating on the user experience and product features. This can include changes to the look and feel of the website or native apps, optimizations for our smart pricing and search ranking algorithms, or even targeting the right content and timing for our email campaigns. For the majority of this work, we leverage our internal A/B Testing platform, the Experimentation Reporting Framework (ERF), to validate our hypotheses and quantify the impact of our work. Read about the basics of ERF and our philosophy on interpreting experiments.

Introduced in 2014, ERF began as a tool that allowed users to define metrics in a common configuration language, and then compute and surface …

[Read more]
Tracking the Money — Scaling Financial Reporting at Airbnb

At Airbnb, the Payments team is responsible for everything related to moving money in Airbnb’s global marketplace. We build technology that powers Airbnb’s massive daily transaction volume to collect payments from guests and distribute payouts to hosts. Our goal is to make the payment experience on Airbnb delightful, magical, and intuitive.

Historically, the payments team’s focus was to implement new features, currencies, and payment methods to make payments local in a global business. Our sphere has grown to include compliance (sales taxes, earnings taxes, licenses, and more) as well as reconciliation and financial accounting according to generally accepted accounting principles.

Currently, Airbnb’s payment and financial accounting system is a complex ecosystem that transacts in 191 countries, with …

[Read more]
Superset: Scaling Data Access and Visual Insights at Airbnb

By Alanna Scott, Bogdan Kyryliuk, Eli Brumbaugh, Jeff Feng, Max Beauchemin and Vera Liu


At Airbnb, one of our fundamental beliefs is that data access should be democratized to empower every employee in order to make data informed decisions. We believe that grounding decisions with quantitative insights from data, together with qualitative insights (e.g. in-person user research) result in the best possible business decisions. This applies to all parts of the organization, whether it is about deciding to launch a new product …

[Read more]
Nebula as a Storage Platform to Build Airbnb’s Search Backends

by Charles He, Soumyadip Banerjee, Tao Tao, & Krishna Puttaswamy

Image Credit: NASA/JPL (Raghvendra Sahai). This image of the Red Rectangle was produced from data in the Hubble Space Telescope (HST) archive, taken with the Wide Field Planetary Camera 2 onboard the HST as part of Guest Observer program 7297.

Last year Airbnb grew to a point that a scalable and distributed storage …

[Read more]
Showing entries 1 to 6