Roundup ‘13 - Devops and Culture Change Fully formatted shownotes can always be found at http://javaposse.com

Extra thanks to D.J. Hagberg for the excellent and complete show notes.

  • Netflix "Freedom & Responsibility" - Every team can organize as they choose

  • "Firemen" who get paged when system goes down. Sets best practices

  • "Site Reliability Team" - sets best practices: deployment, monitoring, ...

  • "Cloud Performance Team" - sets best practices for cloud performance

  • Groovy

  • Python

  • Standarization: set up practices/tools such that choosing the "Standard" makes life easier

  • Regulatory constraints (Sarbanes-Oxley, PCI DSS)

  • Communication with Devops: Build, Deploy, Release

  • Documentation: in many cases, auditors are happy with scripts

  • tl;dr

  • Adrian Cockcroft - InfoQ, YouTube, @adrianco

  • Opportunities for Ops to deploy systems, devs to assist & learn from Ops

  • monitoringsucks

  • Uptime and Availabitility - metrics

  • Wake up the developer at 3am when they break things. Devs need to understand Operational Risk of changes

  • PCI Compliance mitigation ideas

. Separate out the SOX/PCI regulated parts of application, pared down to most minimum possible

. Put those parts under separate accounts, machines, audit trails, etc.

  • Metrics: what do you use to measure? Customer Happiness, Money, Loyalty? (Netflix: "movie start metric")

  • Adobe Cloud Services

  • Quicken cloud

  • Twitter - measure sentiment (for consumer-facing/high-profile services)

  • Automation: bite-size, recoverable pieces

  • Eiffel design-by-contract concept: preconditions/postconditions

  • Redefining the job of operations

  • Executive sponsorship

  • QA does the deployment through DevOps-provided tools

  • Post-Mortem, military AAR, Incident Review, Agile Retrospective: have thresholds for Big Meetings vs small outages/rollbacks

  • Metric: time to get new feature to the customer

  • Self-directed improvement: Deming/Lean improvement cycles - Kaizen

  • Visible Ops Handbook - Gene Kim

  • The Phoenix Project (book) - Gene Kim

  • business Case for improvements? What was cost of the last outage?

  • Chase Manhattan - 3 day outage in 2010

  • Every business in the future will be a software business

  • Move developers into Ops teams - Google engineers get extra "hazard" pay/leather jacks - DevOps Culture Hacks

  • Practices brought back from Ops to Dev

. JMX: Measure/Instrument all the things (esp. consumer facing)

. Vagrant/Opscode Chef/Puppet stack into Dev environment - exercise deployment on every Dev change.

  • "Foreign Exchange" (hostage exchange?) program

  • "Don't make customers modify XML Files"

  • Static Config to Files. Stays the same for release.

  • Dynamic Config - looked up from service - Archaius - Netflix

  • Feature Switches/Feature Flags - Etsy

. Enable/disable features & settings at runtime

. Hierarchical/tagged/scoped configurations

. Use config changes for rollback

Thanks * Libsyn.com - http://www.libsyn.com - for hosting and bandwidth * Feedburner.com - http://www.feedburner.com - for feed redirect * Kirsty Doherty, Amy Ehmann for Java Posse artwork * Theme Music:

  • Opening - "Java" the parody song Copyright 1997 Broken Records and Marjorie Music Publ. (BMI),

  • written and performed by Loose Bruce Kerr of the Dr. Demento Show and Sun Microsystems attorney.

  • Based on the WWI popular song, "Ja-da." Ukelele style on the recording taught to Bruce by his dad.
  • Re-produced with kind permission from "Loose" Bruce Kerr - http://loosebrucekerr.libsyn.com http://www.youtube.com/watch?v=TAX0gJt-aZg

  • Closing - Juan Carlos Jimenez - In the House (Intro No. 1)

  • To contact us:

  • Visit our homepage - http://javaposse.com

  • Post on our Google Group - http://groups.google.com/group/javaposse
  • Pose a question on our Google Moderator group - http://tinyurl.com/q4javaposse
  • Call us with questions and feedback - (408) 465-4626
  • Or send us email - javaposse@gmail.com

The Java Posse consists of Tor Norbye, Carl Quinn, Chet Haase and Dick Wall

Products Discussed In This Episode