Scalability at YouTube by J.J. Behrens and Mike Solomon of Google

Presenters: J.J. Behrens and Mike Solomon of Google

PyCon 2012 presentation page: https://us.pycon.org/2012/schedule/presentation/315/

Slides: ???

Video: https://www.youtube.com/watch?v=G-lGCC4KKok&list=PLBC82890EA0228306&index=22&feature=plpp_video

Video running time: 38:43

First few minutes of the video are marketing fluff.

(10:16) The talk really starts.

Briefly

  • scalability
  • efficiency
  • productivity

People often confuse scalabity and efficiency.

Scalable...systems

(12:32)

  • simple - “If I could give you one word about scalable systems, I think the word would be simple
  • solve the problem at hand
  • product of evolution

The Tao of Youtube

“choose the simplest solution with the loosest guarantees that are practical”

Scalable...techniques

(14:51)

  • divide and conquer - e.g.: independent web servers, database sharding, partition the problem, simple and loose connections
  • approximate correctness - cheat and let people see their own comments rather than requiring immediate consistency
  • expert knob twiddling - consistency, durability
  • jitter - introduce more randomness, because surprisingly, things tend to stack up - example: cache expirations - if all caches expires at the same time => thundering herd. Add some random variance.
  • cheat

Jitter puts entropy back into the system.

Scalable...components

(21:06)

  • well defined inputs
  • well defined dependencies
  • frequently end up behind an RPC
  • leverage local optimizations

Scalable...humans?

(23:12)

  • communication through code/data/schema
  • RPC as a means of sanity

Efficiency

(24:26)

  • uncorrelated with scalability
  • focus on algorithms first
  • learn to measure - use representative samples

Efficient libraries

(26:40)

  • wiseguy - YouTube’s fast CGI container
  • pycurl
  • spitfire - YouTube’s templating system; similar to Cheetah
  • serialization formats* - Don’t use pickle

Efficient tools

(29:46)

Productivity

(32:12)

  • philosophy vs. doctrine
  • more conventions - less documentation
  • more effective collaboration
  • more diffusion of responsibility

Zen proverb

(33:54)

“Do not seek to follow in the footsteps of masters; seek what they sought”

Code Bonsai

(34:22)

  • simple
  • pragmatic
  • elegant
  • orthogonal
  • composable

Questions?

(35:00)