Prof. Dr. Cesare Pautasso

Impact of API Rate Limit on Reliability of Microservices- Based Architectures

Amine El Malki, Uwe Zdun, Cesare Pautasso

16th International Conference on Service-Oriented System Engineering (SOSE 2022), San Francisco, USA

August 2022

Abstract

Many API patterns and best practices have been developed around microservices-based architectures, such as Rate Limiting and Circuit Breaking, to increase quality properties such as reliability, availability, scalability, and performance. Even though estimates on such properties would be beneficial, especially during the early design of such architectures, the real impact of the patterns on these properties has not been rigorously studied yet. This paper focuses on API Rate Limit and its impact on reliability properties from the perspective of API clients. We present an analytical model that considers specific workload configurations and predefined rate limits and then accurately predicts the success and failure rates of the back-end services. The model also presents a method for adaptively fine-tuning rate limits. We performed two extensive data experiments to validate the model and measured Rate Limiting impacts, firstly on a private cloud to minimize latency and other biases, and secondly on the Google Cloud Platform to test our model in a realistic cloud environment. In both experiments, we observed a low percentage of prediction errors. Thus, we conclude that our model can provide distributed system engineers and architects with insights into an acceptable value for the rate limits to choose for a given workload. Very few works empirically studied the impact of Rate Limit or similar API-related patterns on reliability.

Download

Citation

Bibtex

@inproceedings{2022:sose:apiace,
	author = {Amine El Malki and Uwe Zdun and Cesare Pautasso},
	title = {Impact of API Rate Limit on Reliability of Microservices- Based Architectures},
	booktitle = {16th International Conference on Service-Oriented System Engineering (SOSE 2022)},
	year = {2022},
	month = {August},
	publisher = {IEEE},
	address = {San Francisco, USA},
	abstract = {Many API patterns and best practices have been
developed around microservices-based architectures, such as Rate
Limiting and Circuit Breaking, to increase quality properties
such as reliability, availability, scalability, and performance.
Even though estimates on such properties would be beneficial,
especially during the early design of such architectures, the real
impact of the patterns on these properties has not been rigorously
studied yet. This paper focuses on API Rate Limit and its impact
on reliability properties from the perspective of API clients. We
present an analytical model that considers specific workload
configurations and predefined rate limits and then accurately
predicts the success and failure rates of the back-end services.
The model also presents a method for adaptively fine-tuning
rate limits. We performed two extensive data experiments to
validate the model and measured Rate Limiting impacts, firstly
on a private cloud to minimize latency and other biases, and
secondly on the Google Cloud Platform to test our model in a
realistic cloud environment. In both experiments, we observed a
low percentage of prediction errors. Thus, we conclude that our
model can provide distributed system engineers and architects
with insights into an acceptable value for the rate limits to
choose for a given workload. Very few works empirically studied
the impact of Rate Limit or similar API-related patterns on
reliability.},
	keywords = {API Analytics}
}