Introduction To Low Latency Programming: Clarify Program Scope
This post originally appears as a chapter in my new book: ‘Introduction To Low Latency Programming’, a short and approachable entry into the subject. Available now for purchase on Amazon.
I would like to begin this chapter by referencing a quote: “Measure twice, cut once.” While this statement is almost a platitude, it can be a useful reminder for careful, deliberate planning before taking action. I think taking this idea to heart is very important when it comes to low latency programming. The earlier in the development process you include low latency objectives, the better your software design will be and the better the eventual outcomes. Just as with good software engineering in general, we start by clarifying the program scope. Let’s ask ourselves two questions:
- What does the program need to do?
- What does low latency mean in the context of this program?
Take a few minutes to ponder the two questions before I unpack their relevance in the following paragraphs. This is a good learning practice in general as it helps prepare your mind for ingesting new information.
Functional Requirements
Let’s think about the first question: What does the program need to do?
Firstly, I want to go over a semantic detail. I use the word program above, but the question applies to any level of abstraction; from high-level to low-level, from distributed system to a single function. Apply the inquiry to whatever you are trying to build or optimize for low latency.
Secondly, notice the emphasis on the word ‘need’. You are not asking yourself about wants or nice-to-haves. You are strictly thinking about what the functional unit cannot exist without. While this sounds like an obvious question, it actually helps us transition to a different mindset than latency-agnostic programming. ‘Everything’ that is not necessary should be potentially be removed. Let me list some examples to illuminate what I mean. Don’t use a data structure that maintains a sorted order if it is not necessary. Don’t log unless it is necessary. Don’t make a copy of data unless it is necessary. Don’t publish the data to an archive unless it is necessary. Don’t use a library that does more than you need. You must adopt a ruthless mindset when it comes to ripping out functionality and execution steps. Everything must justify its existence in your systems and code.
Sometimes, asking this question ‘hard enough’ will reach back and influence your functional business requirements. If the operations necessary for enabling a feature are deemed ‘too slow’ in the best case, perhaps the business requirements are flawed and must be changed. For most domains, there is a two-way relationship between low-latency abilities and business requirements, as lower latency can potentially open up new business opportunities/functionality, and business requirements can sometimes help push for new latency standards that were once thought impossible. There is a delicate balance to be found and a healthy relationship between the two is important for overall success.
Defining Low Latency Metrics
And now let’s shift our attention to the second question: What does low latency mean in the context of this program?
Answering this question will help us define metrics and goals. When defining your metrics you must identify relevant operations and their respective start and end points. For example, if you are writing a web server this could be the ‘request to response’ duration. Or if you are optimizing a single function, it is the ‘call to return’ duration.
As for defining goals, there are two different options. Either you define a strict, absolute time duration requirement for the relevant operations or simply give yourself a mandate to ‘be as fast as possible’. The former has a strict cut-off, duration wise, as to what is acceptable. The latter simply means you are aiming to reduce the duration between start to end time points as much as you can. While being ‘as fast as possible’ is always a good goal to have, the specific acceptable durations can be crucial from a business perspective and can help provide the push necessary to truly focus on lower latency.
Preparing the metrics and their related goals are incredibly important.
If the goal is a strict absolute time duration requirement it will give us an opportunity to analyze and determine whether it is even possible to fulfill; preventing wasted development time. For example: a business requirement requires parsing and processing a large incoming JSON message in 50 microseconds. You know that the absolute fastest parsing of a JSON message of that size on your target host/architecture takes 55 microseconds. Taking these facts into consideration, you will know it is simply an impossible ask.
Additionally, it helps us to define measurable objectives for project success. This is important in any endeavor, programming or otherwise. If you don’t have a way to measure your deliverables, you simply won’t know if you are succeeded or failing. Having these measurable objectives will also be useful during development time to track your progress and inform you if your program design is working.
Chapter Summary
- Clarify your functional requirements. Be extra aggressive when throwing out unnecessary features.
- Understand the two-way relationship between low latency objectives and business requirements.
- Define your low latency metrics. Use these throughout development and completion to measure project success.
