TensorFlow Serving is a flexible, high-performance serving system for machine learning models ... You can use some parts individually (e.g. batch scheduling) and/or extend it to serve new use cases.