Paginated Web services - Fluid Topics - 3.4 - Reference Guides

Fluid Topics API Reference Guide

Product
Fluid Topics
Fluid Topics Version
3.4
Category
Reference Guides
Language
English
Target Audience
public

The ./api/khub/documents and ./api/khub/maps web services are paginated. To paginate a web service, add the following queryString to the endpoint:

${webservice_path}?page=$PAGE_NUMBER&per_page=$NUMBER_OF_RESULTS_PER_PAGE

The following example provides a more concrete idea of how to paginate a web service:

./api/khub/documents?page=1&per_page=20

The following optional query parameters can be set in the queryString:

  • page: the page number, starting at 1.
  • per_page: the number of items per page. In the example provided above, the web service returns 20 items on each page until there is nothing to return.

To crawl all content, it is possible to query the web service multiple times with a decent page size (10 < per_page < 100), and increment the page number as many times as necessary.

The last page may contain fewer results than the number in the query parameter because the end of the list has been reached.

Once an empty result list is returned, all content has been crawled.

For the sake of compatibility, if the page and per_page query parameters are not set manually, the page=1&per_page=-1 values are set automatically by default to return all results on a single page.

It is strongly recommended to use pagination, as performance suffers when this web service is used without pagination for portals with a significant amount of data.

Output examples

The following web service without pagination:

./api/khub/documents

returns the following result:

[
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "1.png",
title: "1",
mimeType: "image/png",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "2.mp4",
title: "2",
mimeType: "video/mp4",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
...
{
id: "VHxDrn6h7bmvvXDpFqSrhw",
filename: "20.jpg",
title: "20",
mimeType: "image/jpeg",
documentApiEndpoint: "/api/khub/documents/VHxDrn6h7bmvvXDpFqSrhw",
metadata: [ ]
},
{
id: "Ve0yD1R8wcjT0x66pDZwgA",
filename: "21.TIF",
title: "21",
mimeType: "image/tiff",
documentApiEndpoint: "/api/khub/documents/Ve0yD1R8wcjT0x66pDZwgA",
metadata: [ ]
}
]

Using pagination, the same web service can be used to return 20 results per page.

The following web service aims to retrieve results for the first page:

./api/khub/documents?page=1&per_page=20

And returns the following result:

[
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "1.png",
title: "1",
mimeType: "image/png",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "2.mp4",
title: "2",
mimeType: "video/mp4",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
...
{
id: "VHxDrn6h7bmvvXDpFqSrhw",
filename: "20.jpg",
title: "20",
mimeType: "image/jpeg",
documentApiEndpoint: "/api/khub/documents/VHxDrn6h7bmvvXDpFqSrhw",
metadata: [ ]
}
]

The following web service aims to retrieve results for the second page:

./api/khub/documents?page=2&per_page=20

[
{
id: "Ve0yD1R8wcjT0x66pDZwgA",
filename: "21.TIF",
title: "21",
mimeType: "image/tiff",
documentApiEndpoint: "/api/khub/documents/Ve0yD1R8wcjT0x66pDZwgA",
metadata: [ ]
}
]

As there are only 21 items on the portal, the web service returns only 1 item on the second page.

The following web service aims to retrieve results for the third page:

./api/khub/documents?page=3&per_page=20

And there is nothing to return anymore, as displayed in the following result:

[ ]

All content has thus been crawled.