Or, What Made That Talk Good?
As I learn programming and try to become part of the software community, I go to a lot of meetups. Meetups are awesome, and I will put together a blog post on meetups later, but first there’s one meetup I’d like to talk about: an awesome talk hosted by the DC .NET users meetup. I was excited the whole time, learned a lot, and and made some friends. That evening definitely made me a better developer, and definitely re-energized me a bit. Today, I’ll talk about two things: What I learned at this talk, and why this talk was so good.
What I Learned
Steps for optimizing a web application:
- Know your goals before you do anything else: A quick definition of goals would be a 95% percentile for response times given specific load. Example: I want 95% of requests hitting my server to be handled in less than 200ms, when there are approximately 125k requests per day. If we don’t start with a concrete number, we will make pointless optimizations, or even make things worse.
- Measure your progress. Look at you application’s logs and response times. Are you there already? Are you slower than you thought? If you’re done, stop. You’ve met the business’s requirements and you don’t want to spend much time throwing money after a goal when you’re already there.
- Cache things. One concrete example: You put your logged-in users’ usernames in the top right corner. Awesome. Do you hit the DB every page load for that? Applications ask the same questions over and over. If the answers are already in memory, the application will save performance. Caching data that rarely changes is the safest, cheapest way to improve performance. As always, if you’re done, stop. As we go down this list, the steps get more costly and riskier.
- Distribute work. Can you load balance the site between several different servers? This step is really the classic “scale out” model of building up an application. There are lots of things written about this step. Kendall’s main suggestions are: don’t do this unless you need it, and think about sticky sessions. One of the big takeaways from the talk is that scaling out with more web servers won’t help unless the problem is that requests are waiting on web servers. If your actual problem is that you get too much data from the DB too often, go to 3.
- Defer work. Kendall calls this, “async,” but I’m a C# developer and I kept thinking about they keyword instead of the concept. What Kendall means is “decoupling operations so that the critical path is not blocked.” Use queues for long running processes, perform expensive computations and retrievals ahead of time, etc. This step is the hardest to get right, and is a last resort.
- Remember: Consistency is your limiting factor. In other words, the requirement that users visiting the site from different places at the same time see the same site limits the extent to which we can cache, because cache could be stale, it limits the extent to which we can distribute load, because the different instances might have different state, and it limits the extent to which we can defer work, because some work might finish before others. One of Kendall’s main points here is that most people overestimate the extent to which consistency is a business requirement. What if, when Amazon employees update the summary of a book, it changes after 8 minutes on the East Coast but it takes 12 minutes to change in Colorado? Who cares! Having the artificial requirement that two users see the description change at exactly the same time is silly, and puts a lot of strain on the app to implement.
That is a quick outline of Kendall’s talk. But now, the second part of the post:
What Made this Talk Good?
I am not a web application architect, but after a talk that lasted less than two hours, I was able to write down a great deal of practical, applicable advice. How did Kendall do that?
He began the talk with a problem: You want to build a web application that remains responsive at around 125K page loads per day, with a 95th percentile latency of 200 milliseconds. Boom. That is super specific. You could put that performance requirement in a work authorization without changing a word.
Specific context is important because it gives me a map for remembering things. Why are we caching the username? Oh yeah, because database calls are slow and we need to respond in 200 milliseconds. Ok, that makes sense. Why don’t we spend $10K on giant servers? Because we only need to respond every 200 milliseconds, not every 20.
The reason that Kendall’s talk was so memorable is because I was never left wondering why he was saying what he said. Why is he talking about cache? Response time. Why is he talking about network latency? Response time. There were no orphans in this presentation. That is, there were no paragraphs, slides, or statistics that were just up there for some reason, that I couldn’t connect to the surrounding context. Everything connected back to the goal.
I think that leads us to a rule of presentation:
Tell your audience why they care, and repeatedly connect everything back to this reason.
The second rule of a good presentation is also related to context: You must follow the Known-New Contract. (I learned this rule from my dad. I don’t have an authoritative online source to cite, but some quick googling reveals that many university writing classes teach this as well.)
The Known-New Contract basically says that you must progress from things people know about to things that are new in order for them to follow your reasoning. If you start with things that are new to people, they don’t have any context and lose focus, so you must start with the known. If you stay in the known, people aren’t going to learn a lot, and they might not come back, so you need to continue to the new. Start with the known, continue with the new, repeat.
This simple rule applies recursively to the order of information within a talk. For example, Kendall begins his talk with building a website, and having concrete performance requirements. His audience was a room full of developers, so this was “known.” He then quickly moved on to the things we didn’t know (see above). The rule also applies at the sentence level. (Did you see that? I started with ‘the rule,’ which we’ve been talking about for a while, so it is known, and then told you something new about it.) In a talk, this means that, in addition to using the rule to organize your sentences, you can use the rule to organize subtopics: “The goal is to build a performant web app. The three ways of improving performance are caching, distribution of load, and asynchrony. The first type of caching is to cache rarely changing data in memory. One strategy for caching rarely changing data in memory is to cache a dataset from the DB when the application starts, and simply drop the entire cache if any element would change. The advantage of this caching strategy….” Do you see the repeated structure? Set opening context, give fact A about context, give fact B about fact A, etc. Very often, when you read information that “doesn’t flow” or “is hard to follow,” what you’re really noticing is a violation of the Known-New Contract.
So what makes a good meetup talk? Context, both given for the entire talk, and given for each sub-element within the talk, recursively down to the sentence level.
Please check out @kendallmiller and his company Gibraltar Software, and thank him for letting me use his talk on my blog. And the next time you’re at a talk that’s going really well (or really badly) as yourself: What’s the context? Are they following the Known-Knew Contract?
Till next week, happy learning,