📘

About this guide

In this guide, you will learn about the rate limits applied to the realtime or batch transactions against a Connector.

Demyst's Data APIs do not have a rate limit set for any calls. Any interactions with a Demyst endpoints will not be rate limited and the Demyst Platform will scale horizontally to meet increasing demand. At the upstream vendor level, some sources require implementing a rate limits. In these scenarios, Demyst configures the rate limits for these vendors directly.

Vendor rate limits in APIs

Two possible behaviours for the vendor rate limits.

• In the scenario where Demyst ingested the full vendor dataset, (i.e. Connectors that start with hosted_) do not have any rate limit. Demyst typically ingests a full dataset when the performance provided by a vendor does not meet client expectations, or when a vendor does not expose appropriate egress patterns.
• Connectors that do not start with hosted_ will be regulated by the upstream rate limit. So you might see Demyst propagate a rate_limit_exceeded error if you hit the upstream rate limit of any. If you need to know the rate limits, please ask your account representative and you will be informed about the upstream rate limit that has been set.

Vendor rate limits in batches

Demyst's batch capability has a concurrency limit of 40 records per second.

• Similar to above, hosted Connectors have a concurrency limit of 40 records per second for every batch files that you upload.
• The other Connectors have their concurrency and rate limit set when we integrate them. So the batch calls will be spaced out as per the limits to append data to the input file. Apart from being slow, you should not see any other error. If you do, drop an email to [email protected].