Types of inference · Question 10 of 12
Which properties does Amazon SageMaker AI serverless inference have?
2 answers are correct. Medium
Show answer
Answer
- A After idle periods a cold start delays the reply
- D No instance costs arise without requests
Explanation
Serverless endpoints scale automatically down to zero and are billed by compute time, which suits irregular traffic. The trade-off is cold starts after idle periods. GPUs are not supported for serverless inference, and the response comes back directly, not via S3 as with the asynchronous option.