Real-Time vs Batch
A recent article from Chip Huyen’s personal blog goes into what some in the tech community refer to as Real-time Machine Learning. In the article, he identifies two levels of real-time machine learning as (1) online prediction and (2) online learning. The article is well-written with great clarity starting with the concept of real-time. By real-time, he means seconds for online predictions and minutes for online learning.
We emphasize real-time, in general, because, as human beings, we know speed or what the tech community calls latency is very important. We as customers generally prefer sites that are faster(less latency) and successful businesses that utilize e-commerce know this quite well. They know the more latency their sites have, the more revenue they are going to lose. The more sophisticated companies go so far as to actually measure how much revenue they are losing due to a measured amount of latency.
We contrast real-time with batch. Batch, in this context, means we pre-compute predictions ahead of time, store them, and retrieve them when needed. This is acceptable when there are a finite number of possible outcomes. We both know, this is usually not the case.
Use Cases
If you’re like me you like examples and use cases.
For online predictions, Chip gives use cases of what online prediction is not. We know when we’re on TripAdvisor (NASDAQ:TRIP) we have to filter on a metropolitan area to get meaningful results. A company like TripAdvisor probably made the calculation it’s not worth the investment to not have to do that when searching for places of interest.
For online learning, he gives the use case of TikTok. Within minutes, TikTok can learn your preferences and suggest videos you’re likely to want to watch next. TikTok is fairly new, so you may have to create a personal account and “play” around with it and get a sense of how it works.
I encourage you to read the full article, and I definitely can’t wait to see more stuff like this from him.
References
Machine learning is going real-time. Machine learning is going real-time (huyenchip.com). Last accessed: 12/31/2020.
