Pagination made easy with Django Rest Framework
While building any application, the need for fetching a small subset of a large result-set derived from querying a database table is a common occurrence.
In most cases a user would want quick access to the most recent entries in the result-set, while the older entries are only fetched when the user scrolls up/down or selects the next page of the table.
Pagination is therefore core to any application but there are several ways to implement it using Django. You could go with the Paginator class that comes out of the box if all you need is the bare minimum, but if you are looking for a more feature rich solution, Django Rest Framework is the way to go.
How pagination should work
Any type of pagination can be implemented by following a simple API response pattern. An API must return these 4 details.
- A count of the number of entries in the complete result-set.
- A list of entries which represents one page of the result-set.
- A URL that can be used to fetch the next page, if the next page exists.
- A URL that can be used to fetch the previous page if the previous page exists.
Previously I demonstrated how CRUD applications can be built using DRF with an apple notes like application as a reference and I believe the same example can be used to demonstrate pagination.
Let's say that there are 10 notes in the database, but I only want to look at the latest 2 notes at a time, the API response for the note listing API should look like this
This can be achieved using DRF with 2 lines of code.
Paginating an API response
The quickest way is to set up pagination globally and apply it on all the APIs where a query set is returned as a response.
First make sure you have Django Rest Framework installed in your application, then add these lines of code to settings.py
<p> CODE: https://gist.github.com/sankalpjonn/764a728457b8c3fec4d6b5b1115765f0.js</p>
That's it, we're done! You will now notice that all the APIs that return a query set as a response have automatically become paginated. If this solves your problem then great, but if you want to dig deeper into the rich solutions offered by DRF, please keep reading.
Overriding pagination for each API view
If you would like to override individual views with their own pagination settings, this can be done by writing a custom paginator which inherits from any of the existing paginators that DRF provides.
You can then include a pagination_class in your API view as shown below.
<p> CODE: https://gist.github.com/sankalpjonn/c398b821e2e46419733e21f919e902a5.js</p>
To create a custom pagination class, first create a paginations.py file in your app folder and create a class that inherits from an existing DRF paginator.
<p> CODE: https://gist.github.com/sankalpjonn/4a7ace453e708a1d332a555e997f0177.js</p>
The parameters given in this custom paginator class will enable you to provide a page_size query parameter to determine the size of each page. If this parameter exists, it will override the default page size of 2. The page number itself would be indicated using the query parameter p.
Therefore, calling the URL http://localhost:8000/note/all?p=2&page_size=3 will now result in this response:
This is because of setting a page size of 3 and fetching the second page in the result-set.
Types of paginations offered by DRF
DRF offers 3 types of paginations that you can choose from based on your use case. You can also set any of these types as the default pagination class in settings.py
- Page number pagination
- Limit and offset pagination
- Cursor pagination
Page number pagination
This type of pagination is to used to fetch any arbitrary page from the result-set. How it should be used has already been demonstrated in the above example.
Limit and offset pagination
Simulates a database query where you retrieve part of a result-set from a table by providing a limit and an offset. Limit will constrain the maximum rows returned in the query while offset represents the starting position of the rows with reference to the complete size of the result-set.
<p> CODE: https://gist.github.com/sankalpjonn/89e39507bb1a0f1a5e39c168e75a8c14.js</p>
The limit_query_param and offset_query_param will indicate the names of the query params to be used for limit and offset. You can set a default_limit if no query parameter is provided and a max_limit to restrict the max number of rows returned in a page.
If this paginator class is used, the response returned by the URL http://localhost:8000/note/all?l=3&o=3 would be something like this.
This is because we have specified the limit as 3 which indicates that the page size is 3 and an offset of 3 means that the starting point of this page should be after the first 3 rows in the result-set.
Cursor pagination
Gives you a database cursor of the exact page you are looking at and therefore it becomes very efficient when querying a large dataset, but the limitation is that the ordering cannot be changed and must be unique.
To order the results, one must use a field that is set once during creation and is relatively unique. When this type of pagination is used, we cannot fetch an arbitrary page and can only move forward or backwards in a result-set.
Also, this pagination will not return a count of the entries because there is no query being run on the entire result-set.
<p> CODE: https://gist.github.com/sankalpjonn/74588fe42c44b4b4543dcf748e9e5bd2.js </p>
custom_query_param indicates what query parameter to use to represent a cursor and ordering indicates the field based on which pagination will be applied. Since this field has to have a fixed order and the values must be unique, I have chosen the primary key in descending as the ordering.
This is what the response returned by this type of pagination looks like:
Clicking on the next URL returns
Custom pagination
If you prefer to have the count, previous link and next link in the HTTP headers while the data only contains the list of results, you can make these customisations by overriding the get_paginated_response method in the pagination class that you inherit from any of the above 3 mentioned types of paginations.
<p> CODE: https://gist.github.com/sankalpjonn/e49f95c67fe935952d28cd820c518815.js</p>
You will now see the links in headers and only the actual data in the response
Closing notes
It takes a very low amount of code to paginate any list API using Django Rest Framework and therefore I would highly recommend using it over the default paginator class that Django provides out of the box.
DRF itself inherits from the default paginator class so you can save yourself some time.