Paginated Web services - Fluid Topics - 3.6

Use Fluid Topics Web API

Product
Fluid Topics
FT_Version
3.6
Category
Technical Notes
language
English

The ./api/khub/documents and ./api/khub/maps web services are paginated. To paginate a web service, add the following queryString to the web service:

${webservice_path}?page=$PAGE_NUMBER&per_page=$NUMBER_OF_RESULTS_PER_PAGE

As in the following example:

./api/khub/documents?page=1&per_page=20

The following query parameters can be set in the queryString:

  • page: the page number, starting at 1
  • per_page: the number of items per page. Here, the web service returns 20 items on each page until there is nothing to return.

To crawl the whole content, one should query the web service multiple times with a decent page size (10 < per_page < 100), and increment the page number.

The last page may contain less elements than the required number if you have reached the end of the list. You must not assume that you will have the exact number of replies and therefore verify the list.

Once an empty result list is returned, all content has been crawled.

Compatibility is preserved, if queries are unchanged, results are unchanged: it still returns all results (page=1&per_page=-1).

It is strongly recommended using pagination, as there is a real performance risk if this web service is used without pagination parameters on large data volumes.

Output examples

The following web service without pagination:

./api/khub/documents

returns the following result:

[
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "1.png",
title: "1",
mimeType: "image/png",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "2.mp4",
title: "2",
mimeType: "video/mp4",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
...
{
id: "VHxDrn6h7bmvvXDpFqSrhw",
filename: "20.jpg",
title: "20",
mimeType: "image/jpeg",
documentApiEndpoint: "/api/khub/documents/VHxDrn6h7bmvvXDpFqSrhw",
metadata: [ ]
},
{
id: "Ve0yD1R8wcjT0x66pDZwgA",
filename: "21.TIF",
title: "21",
mimeType: "image/tiff",
documentApiEndpoint: "/api/khub/documents/Ve0yD1R8wcjT0x66pDZwgA",
metadata: [ ]
}
]

Using pagination, the same web service can be used to return 20 results per page.

The following web service aims to retrieve results for the first page:

./api/khub/documents?page=1&per_page=20

And returns the following result:

[
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "1.png",
title: "1",
mimeType: "image/png",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
{
id: "lahoGt~FmU1Oh4LEKZ5PRA",
filename: "2.mp4",
title: "2",
mimeType: "video/mp4",
documentApiEndpoint: "/api/khub/documents/lahoGt~FmU1Oh4LEKZ5PRA",
metadata: [ ]
},
...
{
id: "VHxDrn6h7bmvvXDpFqSrhw",
filename: "20.jpg",
title: "20",
mimeType: "image/jpeg",
documentApiEndpoint: "/api/khub/documents/VHxDrn6h7bmvvXDpFqSrhw",
metadata: [ ]
}
]

The following web service aims to retrieve results for the second page:

./api/khub/documents?page=2&per_page=20

[
{
id: "Ve0yD1R8wcjT0x66pDZwgA",
filename: "21.TIF",
title: "21",
mimeType: "image/tiff",
documentApiEndpoint: "/api/khub/documents/Ve0yD1R8wcjT0x66pDZwgA",
metadata: [ ]
}
]

As there are only 21 items on the portal, the web service returns only 1 item on the second page.

The following web service aims to retrieve results for the third page:

./api/khub/documents?page=3&per_page=20

And there is nothing to return anymore, as displayed in the following result:

[ ]

All content has thus been crawled.