Storage Latency and Congestion: The metrics you need to track for your cloud

Earlier this week StorPool did a webinar on the topic “Latency: the #1 Metric for your Cloud”. Boyan Krosnov, StorPool’s CPO, shared some interesting insights regarding the crucial metrics of any cloud. And why latency is the most important one? And, of course, how to reach the lowest latency possible for your cloud. Keep reading, if you have missed our live webinar

Every transaction processing system has specific characteristics. Whether it’s a web server, a database, a storage system, or pretty much any other system, it has the same characteristics – at a low number of transactions per second, you get low latency. Respectively, if you increase the number of transactions, the latency increases as well. This can be very easily illustrated in the graph below.

However, in an ideal system, which, of course, does not exist in reality, the increase of the operations per second doesn’t lead to any change in the latency. The interesting thing here is that, if you go beyond 4 tasks on 4 CPU cores, for example with 6 tasks, you get them processed for 1.5 sec, instead of 1 sec. That is where the linear property behavior of the system, breaks and works in a completely different way.

Elastic mode vs. congested mode

Most transaction processing systems have two very different from each other, operational modes. We have called them elastic mode and congested mode, which are separated by a saturation point. In other words, up to a certain number of load, you request more from the system and you get more from it in return. But once you’ve passed the saturation point, you request more from the system, but you just get more latency back.

Ideally, you would want to run your operations in the elastic mode. In that part of the curve, the operations are delivered in reasonable time and the latency is low. The lowest cost per delivered resource can be achieved at the very end of the elastic mode (90% load), right before the separation point. When you pass the separation point, on the other hand, the loading time increases tenfold.

An interesting performance characteristic also is the system throughput, showing how high is the saturation point and how many transactions per second can the system process in a mode. It is usually expressed in transactions per seconds, opened pages per seconds, etc.

Calculating latency

Many companies that offer public cloud, such as AWS, Digital Ocean, Google, etc. sell disks with a particular number of IOPS in their specification. However, as we already made clear IOPS is just one of the many performance characteristics. But on the above mentioned companies’ websites, you can just see listed the IOPS and the throughput, which are not nearly as important as the latency. Still, there is an easy way for it to be calculated. And you can see the difference in the numbers for yourself.

At StorPool we performed a test comparing the application performance in different public clouds – Dream Host, Digital Ocean, Amazon, and eApps (StorPool’s customer). In order to make the comparison as fair as possible, we have used the same amount of memory, CPUs, the same methodology, and the database. Dream Host and Digital Ocean, both based on CEPH, are not able to serve 1000 TPS. The reason behind this is the high latency. AWS, on the other hand, can happily sell such service at a reasonable price.

Best current practice (BCP)

We also measured StorPool’s BCP, which is our recommendation of how a new public or private clouds should be built, in terms of their storage, networking, infrastructure, etc. For example, if on Amazon EBS, as shown above, you are able to do 2000 TPS, on the StorPool BCP – the number will be up to 7500 TSP. The only two things that we changed here, was that with StorPool we used an NVMe SSD and we changed the 10 GbE network with 25 GbE.

Тo watch the full webinar recording, check out our YouTube channel.

If you have any questions feel free to contact us at info@storpool.slm.dev