Misunderstanding of why message history exists
Unfortunately we find that a lot of new users to Ably misunderstand what message history is designed for and wanting longer storage is in fact an architecture anti-pattern. We'll explain why we think so now.
The message history feature was designed to allow clients to easily catch up on published messages on a channel after being disconnected. Currently, if a connection is abruptly terminated, the client libraries will automatically reconnect. If they reconnect within two minutes, then the client handles resuming of the connection and catching up on all messages automatically without any intervention from the application using that client library. Find out more how connection state recovery works.
If however the connection cannot be resumed and thus continuity on the channels is not possible, Ably provides a history API that allows a client to request messages published previously on that channel. The history feature stores messages and indexes them based on the channel they were published on, and the time and Ably serial number. This provides an efficient way to request all history from any point in time or attachment on a channel by using the serial number.
The misunderstanding generally comes in when customers think that the message history can replace the role of a database for longer term storage. Whilst technically there is no restriction within the Ably service to store messages indefinitely, the APIs and functionality we provide compared to a database typically make the history API unsuitable. For example, if you were building a chat app, you would typically need the following features:
- Retrieve all messages since I last had the app open
Ably's history API can do this - Find all messages published by an author
Ably's history API cannot do this because messages are not indexed by the publisher, clientId, or any other attribute other than serial, time and channel. - Perform a full text search on all messages in a channel
Ably's history API cannot do this because messages are not indexed by the publisher, clientId, or any other attribute other than serial, time and channel. - Perform a relational query by grouping messages in a channel by thread
Ably's history API cannot do this because messages are not indexed by the publisher, clientId, or any other attribute other than serial, time and channel.
Typically, for more complex message retrieval, we recommend customers consider the use of a database which is designed to satisfy complex data queries. Please see a blog post we wrote describing a design pattern that we recommend customers follow to consume data from our Pub/Sub platform and persist that to a more suitable database storage solution.
Related reading: