Reading - JavaScript: The Good Parts

I finished reading "JavaScript: The Good Parts" the other day. I read it cover to cover, in a linear fashion and I would say it has been highly rewarding. Now, when I say that, "I finished it," what I really mean is that I am finished building a mental index of the material, with a lightweight context that will help in JavaScript development. A few parts of the language, inheritance for example, have such subtle specifics that it will take a bit more practice before I can say that I have remastered the language and no longer need to reference the book. The best reason to read this particular text is the in-depth analysis of the languages features and suggested reasons to or not to use each.

Spoils of Success and Artificial Availability: SOA Anti-Patterns

I'm afraid. I'm afraid that I've been working hard to solve the wrong problems. Its not just me. I'm afraid that a ton of engineers have been working on solving the wrong problems. I mean, they feel so right. Over the last few weeks I've learned a few things that I had previously either assumed I understood or had never thought about long enough.

Artificial limiting factors

We build things because we want people to use them. We work very hard to write the software, buy hardware, market our product, evangelize adoption, assist with migrations, and provide support. And then, sometimes, when we've succeeded, we start to forget how hard we worked. We start to see growth or increased adoption or unexpected use as an enemy. As an engineer, you start to think something along the lines of, "Well our customers shouldn't be using this so much." Or maybe something like, "This is an abuse of the system." That is when it happens, and you start thinking, "How could we build a system that protects us from our customers?" Building systems that enable you to meter, throttle, or rate limit your service are an admission of defeat in all but the most extreme circumstances. With very few exceptions, if you find yourself reaching for one of these solutions, you are considering the wrong problem. Beyond the obvious poor experience you are creating, you're "protecting" your systems from innovation.

Pride

Take pride in your work. If you built a system that is so successful, that people want to use it, make it even better. Scaling issues are the best issues to have. Take those challenges head on. Partition, shard, pre-compute and cache your way to a solution. Make sacrifices to do more work, but deliver more consistent customer experiences. Raise rate limits, reduce customer expenses, and deliver an experience that customers trust.

Scarcity, Exclusivity and Adoption

When the thing that you're gaiting is the adoption of your service or software, be aware of the sweet spot. This practice is common and a reasonable mechanism for protecting QoS for customers of a new operation during scaling phases. But there is a sweet spot. Grow as rapidly as you can handle, and be loose with viral personal referral growth. Referral invites bring more committed customers than marketed or media references. Aim to grow exponentially. If it takes too long for interested parties to get in, the likelihood that they will be satisfied with your product is slim.

Actual limiting factors

If you must limit use, consider a business solution like tiered pricing. In these cases, the limits are set by the customer. They know how much they want to spend, and in B2B scenarios, you'll find that your customers are happy to make some monetary sacrifices.

An Example: Wireless Networks

Lets consider the wireless network example. Over the last decade or so, with the advent of the smart phone and mobile computing, wireless networks have experienced this huge story arch that takes their character from innovative liberators all the way to hideous tyrants in the eyes of their customers. In the early days, market forces pushed reasonably priced unlimited data plans. We were all free. Customers could go anywhere and access our data. The iPhone and subsequent Android boom enabled us to consume more and more of this data, and we fell in love. Then something odd happened. The networks realized that they were having a difficult time keeping up with demand for bandwidth. They didn't just want more money. I mean, we were hooked. They could have simply raised their prices if they wanted more money. Instead, they started instituting data caps. Beyond which the cost of data is so high, that nobody would reasonably want to pay for it. The effect of such a policy is behavior correction. They are so spoiled by demand, that they see no reason that they should improve their service. They would rather create a poor customer experience, and that is what they have done. This problem has only become worse since the adoption of 4G. Many phones do not simply prefer WIFI over wireless, they prefer the connection that will deliver data the fastest. In many areas, the winning network is wireless. The result is unsuspecting customers with massive phone bills at the end of the month.

Artificial Availability

I want to talk about failure modes. Regardless of how far your service or software is from a customer facing edge, you likely built it with the intention that it should be maximally accessible and available. Furthermore, your service or software likely has some service or data source dependencies. These two facets are really at odds. As cliché as it is to point a finger, dependencies are a real bitch for your availability metrics. If one goes, so does your service. So, what do we do? Well, sometimes we put forth a huge effort to try to make the chain stronger than its weakest link. Some might try to move to an asynchronous processing approach for to service data mutation requests. Others might just indefinitely retry internal service requests indefinitely, or until the client goes away. But both of these coping mechanisms actually do more to damage the system than good. This is a case where we are solving the wrong problem.

Lying to a client is bad

Advertising a service call as a synchronous call, and then handling it asynchronously internally is lying. The reason is simple: asynchronous processing is shit without a reliable feedback channel. Fake-synchronous processing and internal backlogging in order to handle dependency outages will destroy customer expectations and forms the foundation for unstable systems that fail big. This is all about managing customer expectations. Systems fail, it is unavoidable. Your customers will be much more appreciative if you clearly let them know when it happens, and how long you expect the outage to be rather than pretending it doesn't or never did happen. If your software is worth using in the first place, they will try again. Especially where expectations have been clearly managed. Take the outage. Communicate. If you're going to put effort into improving the system, put that effort towards improving the accessibility and availability of your dependencies.

What do we do instead?

What you should do really depends on the software. On accessor type operations, delivering cached results might be an acceptable option. Maybe you could even leverage more available but eventually consistent data stores. On mutation operations, just do everyone a favor and fail fast. The faster the better. If your database is down, don't bother it with continued customer polling. Short-circuit those requests until service has been restored. Get creative, but make sure you're addressing the real problems.

The Right Problems

The right problems are the ones that impact your customer first. The right solutions improve the experience for your customer first. These solutions also have the benefit of being the most innovative.

Tired of WordPress

Last night I took a good look at my blog that I had not touched in quite some time. It consisted of a few solid posts, a few not-so-solid posts, and a huge collection of half finished posts. It was a WordPress blog. Like most WordPress blogs, I had a few themes and plugins installed. It was nothing really noteworthy.

This blog was really just a security hole.

I've patched this system more often that I have written entries. To run WordPress, I run a server (Linux), a web server (Apache), and a database (MySQL). That is a bunch of upkeep, and I just don't have time for it anymore. So, I thought, why bother with all of this overhead? I've had one comment on this blog, and more importantly, I loath comments in general. I do not need a blog capable of providing highly dynamic content.

Inspired, I started doing something I hadn't done in almost two years. I started hacking up some raw HTML and CSS. "Just a basic page." I thought. "I just want to focus on the content." So, in about an hour of tweaking font sizes, and setting up a few S3 buckets to house my content, my new site had form.

I've been meaning to actually use a few new technologies recently, so I figured this was a good opportunity to pick up Node.

The next afternoon after work, I hacked together a JSON data file to house my entries and a quick Node.js script using Mustache.js to render static pages and upload those to S3.

Now, rather than running all of those systems, my blog consists of generated static files on S3.

Advantages:
  • there is no software to patch
  • the system will always perform well
  • it is super cheap
  • highly cacheable content