Pagination related quries for Purview API

Question

Pagination related quries for Purview API

Balamurugan, Susinthika 20

Hello Team,

I am trying to understand the pagination behavior mentioned in the Microsoft Purview API documentation. As per the documentation, the pagination keys skipToken and nextLink are expected to be present in the response when retrieving paginated results.

However, after enabling the setup and invoking the endpoints, I am currently not seeing these keys in the API responses.

Could you please help clarify the following:

Are there any specific configurations or parameters that need to be enabled in order to receive the skipToken or nextLink fields in the response?

Or are these fields returned only when the result set exceeds the default page size?

For reference, I am testing the following endpoints:

List Data Sources https://learn.microsoft.com/en-us/rest/api/purview/scanningdataplane/data-sources/list?view=rest-purview-scanningdataplane-2023-09-01&tabs=HTTP

List Scans by Data Source https://learn.microsoft.com/en-us/rest/api/purview/scanningdataplane/scans/list-by-data-source?view=rest-purview-scanningdataplane-2023-09-01&tabs=HTTP

List Scan History https://learn.microsoft.com/en-us/rest/api/purview/scanningdataplane/scan-result/list-scan-history?view=rest-purview-scanningdataplane-2023-09-01&tabs=HTTP

List Policies https://learn.microsoft.com/en-us/rest/api/purview/purview-unified-catalog/policies/list?view=rest-purview-purview-unified-catalog-2025-09-15-preview&tabs=HTTP

Any clarification on how pagination works for these APIs would be greatly appreciated.

Thank you in advance for your help.

0 comments

2 answers

Your answer

Answer 1

For the Microsoft Purview REST APIs, pagination fields are only returned when the service actually needs to paginate the result set. There is no separate configuration switch to “turn on” skipToken or nextLink.

Key points based on the documented patterns:

Pagination is conditional on result size
For list-style APIs, pagination metadata is emitted only when the response cannot return all items in a single page:
- If the total number of items is less than or equal to the default page size, the response is returned as a single page and no pagination token/next link is included.
- When the result set exceeds the default page size, the service returns a pagination mechanism (for example, a nextLink or token) that the client must follow to retrieve subsequent pages.
This is consistent with other Microsoft services:
- Microsoft Graph list APIs return an @odata.nextLink only when there are more pages to fetch and may use $skip or cursor-based tokens depending on the resource.
- Learn Platform APIs use a nextLink field only when there is another page of results to retrieve.
- Some OData-based services use $skiptoken for server-driven paging; the token appears only when the server decides to paginate.
Token/nextLink is opaque and request-driven
When pagination is triggered, the service returns:
- A cursor/URL (for example, nextLink) that already encodes the necessary state (such as a token or skip value).
- Clients should not construct these tokens manually; instead, they should:
  1. Inspect the response for nextLink (or equivalent).
  2. If present, call that URL as-is to get the next page.
  3. Repeat until no nextLink is returned.
Purview Data Map APIs and pagination
The Purview Data Map APIs show the same pattern:
- For large result sets (for example, glossaries, terms, categories, or assigned entities), the documentation explicitly recommends using limit and offset query parameters to page through data when a single call would be too large or may time out.
- When the dataset is small enough, a single call returns all results and no additional pagination metadata is needed.
- When the dataset is large, using limit/offset or related pagination parameters is required to avoid timeouts and to retrieve data page by page.
Why skipToken/nextLink may not appear in current tests
For the specific endpoints being tested (List Data Sources, List Scans by Data Source, List Scan History, List Policies):
- If the environment currently has only a small number of data sources, scans, scan history entries, or policies, the responses will fit within the default page size.
- In that case, the service returns a single page and does not include skipToken or nextLink, which matches the documented behavior of other Microsoft APIs that use server-driven paging.
How to validate pagination behavior
To observe pagination in practice:
- Ensure the relevant collection is large enough (for example, many data sources, scans, or policies) so that the service must paginate.
- Call the list endpoint without specifying a very large page size (or using the default).
- Inspect the response for a pagination field (nextLink or similar).
- Follow that link to retrieve subsequent pages until it is no longer present.

In summary, no extra configuration is required. skipToken/nextLink (or equivalent pagination metadata) are returned only when the result set exceeds what the service returns in a single page, which is determined by the default or requested page size.

References:

Balamurugan, Susinthika 20 Reputation points

2026-03-10T13:22:12.3866667+00:00

Could you please specify what is the default page size for these endpoints?

Answer 2

Smaran Thoomu 34,795 Microsoft External Staff Moderator

Hi @Balamurugan, Susinthika
Thanks for your question.

In Microsoft Purview APIs, pagination fields like skipToken or nextLink are returned only when the response contains more items than can be returned in a single page. If the total number of results is small, the API will return all data in one response, and you will not see these pagination fields.

There is no specific configuration or setting that needs to be enabled to get these fields. Pagination is handled automatically by the service.

Regarding the default page size, this is not always fixed or publicly documented for all Purview endpoints, and it may vary depending on the API and service behavior. Because of this, it is generally recommended to rely on the presence of nextLink / skipToken in the response rather than assuming a specific page size.

If you would like to test pagination behavior, you can try:

Using query parameters such as $top (if supported by that endpoint)
Testing in an environment with a larger number of items

If you are still not seeing pagination when you expect large results, please share a sample request and response (sanitized), and we can review further.

Hope this helps clarify.

Thanks.

Balamurugan, Susinthika 20 Reputation points

2026-03-11T13:50:47.5633333+00:00

Thank you for your response.
Smaran Thoomu 34,795 Reputation points Microsoft External Staff Moderator

2026-03-12T02:57:55.9833333+00:00

Balamurugan, Susinthika Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Balamurugan, Susinthika 20

Hi @Smaran Thoomu

We have generated additional scan run data for the below endpoint (100+ records). However, we are still not receiving pagination keys (e.g., nextLink) in the response.

We have attached the request and response for your reference. Could you please provide insights on this endpoint, specifically regarding its default page size and pagination behavior? This will help us proceed further with validating pagination.

Request Endpoint: https://<domain>.purview.azure.com/scan/datasources/<datasourcename>/scans/<scanname>/runs?api-version=2023-09-01

**Response:

This contains a one sample record and the total count of records from this endpoint.**

        {
            "discoveryExecutionDetails": {
                "discoveryStartTime": "2026-03-10T06:40:11.4817265Z",
                "discoveryEndTime": "2026-03-10T06:44:29.8952577Z",
                "status": "Succeeded",
                "statistics": {
                    "assets": {
                        "discovered": 72,
                        "classified": 1
                    }
                },
                "isErrorLogAvailable": false
            },
            "ingestionExecutionDetails": {
                "ingestionStartTime": "2026-03-10T06:41:29.6976853Z",
                "ingestionEndTime": "2026-03-10T06:44:36.2151666Z",
                "status": "Succeeded",
                "statistics": {
                    "assets": {
                        "succeeded": 72,
                        "failed": 0,
                        "processing": 0
                    },
                    "relationships": {
                        "succeeded": 0,
                        "failed": 0,
                        "processing": 0
                    }
                },
                "estimatedTimeRemainingInSeconds": null,
                "isErrorLogAvailable": false,
                "lastUpdatedAt": "2026-03-20T07:12:32.6500378+00:00"
            },
            "parentId": null,
            "id": "df404def-14e9-4a7e-822f-5959aafe29d5",
            "ingestionJobId": "df404def-14e9-4a7e-822f-5959aafe29d5",
            "resourceId": null,
            "status": "Succeeded",
            "webScanResults": null,
            "diagnostics": {
                "notifications": [],
                "exceptionCountMap": {}
            },
            "startTime": "2026-03-10T06:40:07.3197594Z",
            "endTime": "2026-03-10T06:44:36.2151666Z",
            "scanRulesetVersion": 3,
            "scanRulesetType": "System",
            "scanScopeType": "L3",
            "scanLevelType": "Full",
            "errorMessage": null,
            "error": null,
            "runType": "Manual",
            "dataSourceType": "AmazonS3"
        }
    ],
    "count": 107

Thank you for your support.

Smaran Thoomu 34,795 Reputation points Microsoft External Staff Moderator

2026-03-23T14:16:49.57+00:00
Balamurugan, Susinthika Thank you for sharing the request and response details.

For the scan runs endpoint, the behavior is slightly different from other list APIs. Even when the total count is high (for example 100+ records), the API may not return pagination fields such as nextLink or skipToken.

This endpoint is designed to return:

The total record count

A subset of execution details per request

In many cases, it does not expose standard server-driven pagination metadata like other Purview APIs.

So what you are seeing (single record + count = 107) is expected behavior for this specific endpoint.

What you do next

If you need to retrieve historical scan run data programmatically, we recommend:

Calling the endpoint multiple times with supported filtering parameters (such as time range, if applicable)

Verifying whether your scenario requires scan result APIs instead of scan run summary APIs

Also note that the default page size for this endpoint is not publicly documented and may vary based on service optimization.

If needed, we can check internally and confirm the exact paging mechanism used by this API.
Balamurugan, Susinthika 20 Reputation points

2026-03-23T17:05:18.31+00:00

Hi @Smaran Thoomu
Thank you for your response.

In our scenario, we need to retrieve the run history for a specific scan, along with the corresponding data source on which it was executed. For this purpose, we are using the following endpoint:

https://<domain>.purview.azure.com/scan/datasources/<datasourcename>/scans/<scanname>/runs?api-version=2023-09-01

It would be very helpful if you could provide more clarity on the pagination behavior of this endpoint—specifically, how pagination is triggered and any details regarding the default page size. This will help us validate whether pagination is functioning correctly based on our implementation.

Thank you for your support.
Smaran Thoomu 34,795 Reputation points Microsoft External Staff Moderator

2026-03-24T06:31:27.4466667+00:00
Balamurugan, Susinthika Thanks for the additional clarification - this helps understand your scenario better.

For the scan runs endpoint, pagination does not follow the same pattern as other Purview list APIs.
Even when the total number of runs is high, the API may return:

A subset of run details

Along with the overall count

This endpoint is designed to provide execution summary information, and the service internally controls how much data is returned per response. Because of this:

Pagination is not always exposed via nextLink or skipToken

The response size is determined by service-side optimization rather than a fixed public page size

Seeing a single run record along with a higher count can be expected behavior

How pagination is triggered here

Pagination for this endpoint is typically triggered based on:

Service response limits

Internal performance thresholds

Query scope (for example time-based filtering)

It is not strictly triggered only by total record count.

Recommended approach for your scenario

Since you need complete scan run history:

Use time-range filtering or incremental retrieval logic

Avoid relying on nextLink for this specific API

If detailed per-run data is required, validate whether scan result APIs better match your requirement

If helpful, we can check internally and confirm the exact paging mechanism and expected retrieval pattern for this endpoint.

Thanks.
Balamurugan, Susinthika 20 Reputation points

2026-03-25T09:27:35.3766667+00:00
Hi @Smaran Thoomu

Thank you for your insights.

As per your suggestion, we are currently using the Scan Result API based on the following documentation: https://learn.microsoft.com/en-us/rest/api/purview/scanningdataplane/scan-result/list-scan-history?view=rest-purview-scanningdataplane-2023-09-01&tabs=HTTP

API endpoint used: https://<domain>.purview.azure.com/scan/datasources/<datasourcename>/scans/<scanname>/runs?api-version=2023-09-01

Based on your earlier recommendations:

Use time-range filtering or incremental retrieval logic

Avoid relying on nextLink for this specific API

Validate whether Scan Result APIs better match the requirement for detailed per-run data

We have aligned our approach accordingly. However, we are currently unable to find any time-range filtering parameters (such as start time or end time filters) in the above API documentation.

Could you please help us with:

Whether time-range filtering is supported for this API

If supported, kindly share the relevant documentation or parameters to achieve this

This will help us proceed with implementing the correct incremental data retrieval strategy.

Thanks in advance for your support.
Smaran Thoomu 34,795 Reputation points Microsoft External Staff Moderator

2026-03-27T16:51:46.0333333+00:00
@Smaran Thoomu Thanks for checking on this.

For the scan runs / scan result API, time-based filters (like start time or end time) are not currently supported as request parameters. That’s why you’re not seeing anything in the documentation.

Also, this particular API behaves a bit differently compared to other Purview APIs:

It may return only a subset of records along with the total count

It doesn’t always expose pagination fields like nextLink or skipToken

The number of records returned is controlled by the service internally

So what you’re seeing (few records + higher count) is expected.

For your use case (retrieving full run history), the practical approach is to handle this on your side:

Call the API periodically

Track the last processed run (based on runId or timestamp)

Filter out already processed records in your logic

At the moment, there isn’t a built-in way to control pagination or apply time filters for this endpoint.
Balamurugan, Susinthika 20 Reputation points

2026-03-31T06:41:27.8966667+00:00

Thank you for your clarification.
Smaran Thoomu 34,795 Reputation points Microsoft External Staff Moderator

2026-03-31T07:01:14.05+00:00

@Smaran Thoomu Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Pagination related quries for Purview API

2 answers

Your answer