FaaSTest
Powered by Nuweba

Who’s the Faastest?

The newest test results from serverless provider for speed and performance

229
m/s
41
m/s
381
m/s
*Score is mean
IncreasingLoadLvl1
Latest test result: 10/20/2019 10:31:00
Choose a Test
IncreasingLoadLvl1
GO
Test Config
GroupCreated with Sketch.
Request Delay
0m/s
Concurrency Limit
0
Duration
0
Test Type
GroupCreated with Sketch.
RequestsForTimeGraph
Test Description
Gradually invoke more concurrent short runtime functions over time with a 1s delay between hits and benchmark their invocation overhead.
Stack Description
A simple stack containing sleep functions with diffrent configs. The functions sleep for a user specified amount of time and return their execution duration and whether they were warm started or not.
Cold Start Particles
FAQCold Start Particles
01.
What is the purpose of this site?
FaaStest is a website that presents FaaSbenchmark’s results. Users can choose and define which test results they want to examine.
02.
What is FaaSbenchmark?
FaaSbenchmark is an open-sourced framework written in Go to accurately benchmark FaaS platforms' performance.
03.
How often do tests run?
The tests usually run once a week, or when there is a major update to one of the platforms.
Besides that, we run the batch once a new test/provider/stack is added.
04.
Where do tests run?
For each provider, we have a CI that runs the tests in the cloud provider’s network, and we use powerful instances in order to make sure that the VM’s performance does not affect the results.
AWS Lambda - us-east-1, c5.4xlarge
Azure Functions - eastus, Standard_D3_v2
Google Cloud Functions - us-central1-a, n1-standard-4
05.
How is the podium score calculated?
The Score is calculated by averaging all the requests in the test.
Note that at the moment we don't take into account failed invocations.
06.
What does the black graph line represent (test graph)?
Test config is the parameters that define the behavior of the tests, some are passed to httpbench and some are just for faasbenchmark.
07.
Why are there levels for some of the tests?
Levels define different loads on the FaaS platforms. Level 1 is the lightest load.
08.
What is the test config?
A test config is the parameters that define the behavior of the tests, some are passed to httpbench and some are just for faasbenchmark.
09.
Are these benchmarks only for cold start?
FaaSbenchmark is not just about cold starts*,
rather it’s a generic framework to benchmark a variety of different scenarios, such as:
- Third-party tools overhead, i.e: security and visibility/observability
- Function composition overhead
- Cloud services performance using FaaS
- DynamoDB / Microsoft Azure Cosmos DB / Google Cloud Datastore
- S3 / Blob Storage / Google
- Cloud Storage Triggers latency/overhead
- Mem / CPU / IO / Network performance comparison
And much more!
10.
Which library is used to benchmark http?
We use a Golang package called httpbench, which we wrote for fine tuned http/s benchmark with synchronization across requests and detailed trace.

More...