Offset vs Cursor-based Pagination: Choosing The Right Pagination Strategy

Offset vs Cursor-based Pagination: Choosing The Right Pagination Strategy

posted 4 min read

Whether you’re building API endpoints or designing sleek interfaces, selecting the right pagination strategy can significantly influence user experience and system performance. By the end of this article, you will understand the two types of pagination strategies and know when to choose one over the other.

What is Pagination & Why Does It Matter?

Pagination is the process of dividing large datasets into smaller, manageable chunks or pages to improve performance and usability. Without pagination:

  • Frontend apps would struggle with rendering thousands of records at once.

  • Backend APIs would waste resources fetching unnecessary data.

Two Types Of Pagination Strategy

During the course of my career, I have seen engineers use the wrong pagination approach when implementing features for their app. Understanding the two types of pagination strategies can make a significant difference in both the performance and user experience of software applications.

Offset-Based Pagination

Offset-based pagination is a commonly used pagination pattern where the frontend specifies a page number and a page size (how many items per page).

Here is a sample React component that makes a request to an API that is based on offset pagination

function ProductList() {
  const [products, setProducts] = useState([]);
  const [page, setPage] = useState(1);

  const fetchProducts = async () => {
    const response = await fetch(`/api/products?page=${page}&limit=10`);
    const data = await response.json();
    setProducts(data);
  };

  return (
    <div>
      {products.map(product => <div key={product.id}>{product.name}</div>)}
      <button onClick={() => setPage(p => p - 1)} disabled={page === 1}>
        Previous
      </button>
      <button onClick={() => setPage(p => p + 1)}>Next</button>
    </div>
  );
}

A sample sql query to implement the pagination could look like this:

SELECT * FROM products LIMIT 10 OFFSET 20;

Pros of offset-based pagination

  • It is easy to implement on the backend and also easy to implement a
    frontend component for interacting with the paginated data.
  • Users can conveniently jump to any specific page.

Cons of offset-based pagination

Performance issues with large datasets:

While it is possible for the frontend to request a specific page number, the database still has to scan all the previous rows in the table until it reaches the exact page number and retrieve the specified page size. This leads to a significant performance bottle-neck for very large datasets. Let me make this clearer. Imagine a dataset of 500,000 records. To request the last 20 records, the frontend will make a GET request similar to this:

GET /api/items?page=25000&limit=20
   
// OR using the offset parameter
   
GET /api/items?offset=499980&limit=20

This means, the database must scan all 500,000 rows (even though the frontend only wants 20), skip the first 499,980 records (wasting CPU, I/O, and memory) and finally return the last 20. This leads to slow queries (higher latencies) and wasted resources (imagine 1000 users hitting page 25,000 simultaneously!)

Inconsistent results with real-time data

Offset-based pagination is not suitable for real-time data such as social media feeds because it can lead to duplicate and inconsistent results. This is why social media apps such as Facebook, Instagram, Twitter, etc., use infinite scroll (which is based on cursor pagination) for loading more data to the UI.

Cursor-based pagination

Cursor-based pagination uses a unique identifier known as a cursor or pointer as a reference point to fetch the next set of results. That is, each time the frontend makes an API call, in addition to sending back data, the API would send a unique identifier that the frontend will pass in the next request. Then the API would use that identifier as a reference point for sending the next set of data.

Below is a sample React component that consumes an API that is based on cursor pagination:

function CommentsFeed() {
  const [comments, setComments] = useState([]);
  const [cursor, setCursor] = useState(null);

  const fetchComments = async () => {
    const url = `/api/comments?limit=10${cursor ? `&cursor=${cursor}` : ''}`;
    const response = await fetch(url);
    const { data, nextCursor } = await response.json();
    setComments(prev => [...prev, ...data]);
    setCursor(nextCursor);
  };

  return (
    <div>
      {comments.map(comment => <div key={comment.id}>{comment.content}</div>)}
      {cursor && <button onClick={fetchComments}>Load More</button>}
    </div>
  );
}

Pros of cursor-based pagination

  • Efficient for large datasets since the database uses the unique
    identifier to retrieve the next set of data without having to scan
    previous records.
  • Suitable for real-time data: Cursor based pagination does not lead to
    duplicate or inconsistent results when applied on datasets that
    update rapidly.

Cons of cursor-based pagination

  • Users cannot jump to specific pages.
  • It is slighltly more complex to implement.

Comparison of Offset vs. Cursor

Feature Offset Pagination Cursor Pagination
Performance Slow with large offsets Fast, even with big data
Random Access ✅ Yes ❌ No (sequential only)
Real-time Data ❌ Pages may shift ✅ Stable & consistent
Implementation Simple Requires cursor tracking
Use Case E-commerce, dashboards Social media, activity logs

Choosing the right pagination strategy depends on the context and requirements of your features. With the above insights, you're ready to paginate with precision—empowering your application to be both user-friendly and performant.

If you read this far, tweet to the author to show them you care. Tweet a Thanks

Great breakdown of pagination strategies! The explanation of how offset-based pagination struggles with large datasets really highlights why performance matters. One question—have you ever encountered a hybrid approach where both offset and cursor-based pagination are used together in a system? If so, how was it implemented, and did it solve any of the downsides of each method?

Great question, James.

Yes, I have worked on a system where both pagination strategies are used.

The choice of which pagination to use was based on the peculiarities of the features in the system.
For example, the product I work on has a table on the admin dashboard that lists all the employees in a company.
The API that returns the list of employees implements offset-based pagination because the list of employees in a company does not change rapidly and the number of employees our clients have ranges from a few hundred to a few thousand. Hence both the size of the employees records and the absence of frequent updates to the employees data made offset-based pagination a good fit.

However, the same system has a chat feature where employees in a company can interact and network with each other. The real-time nature of chat messages and the ever-growing size of chat messages necessitated the use of cursor-based pagination on the backend and infinite scroll on the frontend.

More Posts

Choosing the Right Solana Wallet: Phantom vs. Solflare vs. Sollet

adewumi israel - Jan 20

Non-functional requirements are the pillars that determine whether your application will thrive or fall apart

Segun Adedigba - Feb 24

NeoPopup - The Modern 3D Popup Module

Mayank Chawdhari - Jan 7

Python on the web - High cost of synchronous uWSGI

wewake - Apr 30, 2024

Learn the basics about how REST APIs work

SuperheroJT - Feb 23
chevron_left