Pre-release — content may change.

Concepts

Pagination

Efficiently working with large result sets

Pagination allows you to retrieve large datasets in manageable chunks, improving performance and reducing response times.

How Pagination Works

Wicket uses page-based pagination with two parameters:

  • page[number]: The page number to retrieve (1-indexed)
  • page[size]: Number of records per page

Basic Example

GET /people?page[number]=1&page[size]=25

This retrieves the first 25 people.

Default Behavior

  • Default page size: 25 records
  • Maximum page size: 2000 records (some endpoints may have lower limits)
  • Pagination enforcement: Most collection endpoints enforce pagination automatically

As of March 2024, the /organizations endpoint enforces pagination by default with a maximum page size of 500 and a default of 25 records. Some existing tenants may have access to larger page sizes.

Pagination Parameters

Page Number

Specify which page to retrieve:

# First page (default)
GET /people?page[number]=1&page[size]=10

# Second page (records 11-20)
GET /people?page[number]=2&page[size]=10

# Third page (records 21-30)
GET /people?page[number]=3&page[size]=10

Page Size

Control how many records per page:

# Small pages (useful for UI pagination)
GET /people?page[size]=10

# Medium pages (balanced)
GET /people?page[size]=50

# Large pages (bulk operations)
GET /people?page[size]=100

In most cases, smaller page sizes (≤ 100) should be used as these can be load balanced more efficiently. The maximum page size is 2000, but some endpoints have lower limits.

Response Metadata

The response includes navigation links:

{
  "data": [...],
  "links": {
    "self": "https://tenant-api.wicketcloud.com/people?page[number]=2&page[size]=25",
    "first": "https://tenant-api.wicketcloud.com/people?page[number]=1&page[size]=25",
    "prev": "https://tenant-api.wicketcloud.com/people?page[number]=1&page[size]=25",
    "next": "https://tenant-api.wicketcloud.com/people?page[number]=3&page[size]=25",
    "last": "https://tenant-api.wicketcloud.com/people?page[number]=10&page[size]=25"
  },
  "meta": {
    "total": 237
  }
}

Link descriptions:

LinkDescription
selfCurrent page URL
firstFirst page
prevPrevious page (omitted on first page)
nextNext page (omitted on last page)
lastLast page

Total Count

The meta.total field shows the total number of records:

{
  "meta": {
    "total": 237
  }
}

Use this to calculate total pages:

const totalPages = Math.ceil(response.meta.total / pageSize);

Pagination with Sorting

Pagination respects sort parameters:

GET /people?sort=-created_at&page[number]=1&page[size]=25

This returns the first 25 people, sorted by creation date (newest first).

Pagination with Filtering

Combine pagination with filters:

GET /people?filter[status_eq]=Active&sort=family_name&page[number]=1&page[size]=50

This returns the first 50 active people, sorted by last name.

Iterating Through Pages

Sequential Iteration

Fetch all pages sequentially:

async function fetchAllPeople() {
  const allPeople = [];
  let currentPage = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(
      `${apiUrl}/people?page[number]=${currentPage}&page[size]=100`,
      {
        headers: {
          'Authorization': `Bearer ${token}`,
          'Content-Type': 'application/vnd.api+json'
        }
      }
    );

    const data = await response.json();
    allPeople.push(...data.data);

    // Check if there's a next page
    hasMore = !!data.links.next;
    currentPage++;
  }

  return allPeople;
}
def fetch_all_people
  all_people = []
  current_page = 1
  has_more = true

  while has_more
    uri = URI("#{api_url}/people?page[number]=#{current_page}&page[size]=100")
    request = Net::HTTP::Get.new(uri)
    request['Authorization'] = "Bearer #{token}"
    request['Content-Type'] = 'application/vnd.api+json'

    response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
      http.request(request)
    end

    data = JSON.parse(response.body)
    all_people.concat(data['data'])

    # Check if there's a next page
    has_more = data['links']&.key?('next')
    current_page += 1
  end

  all_people
end

Follow the next link from the response:

async function fetchAllPeople() {
  const allPeople = [];
  let nextUrl = `${apiUrl}/people?page[size]=100`;

  while (nextUrl) {
    const response = await fetch(nextUrl, {
      headers: {
        'Authorization': `Bearer ${token}`,
        'Content-Type': 'application/vnd.api+json'
      }
    });

    const data = await response.json();
    allPeople.push(...data.data);

    // Use the next link from the response
    nextUrl = data.links.next;
  }

  return allPeople;
}

Best Practices

1. Choose Appropriate Page Sizes

Small pages (10-25):

  • UI pagination
  • Real-time updates
  • Quick initial loads

Medium pages (50-100):

  • Balanced performance
  • Most use cases
  • Recommended for load balancing

Large pages (100-500):

  • Bulk operations
  • Data export
  • Background processing

2. Respect Rate Limits

When paginating through large datasets:

async function fetchAllPeopleWithDelay() {
  const allPeople = [];
  let currentPage = 1;
  let hasMore = true;

  while (hasMore) {
    const data = await fetchPage(currentPage);
    allPeople.push(...data.data);

    hasMore = !!data.links.next;
    currentPage++;

    // Add delay to respect rate limits
    if (hasMore) {
      await new Promise(resolve => setTimeout(resolve, 200));
    }
  }

  return allPeople;
}

See Rate Limits for more details.

3. Cache Pages

Cache pages to avoid redundant requests:

const pageCache = new Map();

async function fetchPageCached(pageNumber, pageSize = 25) {
  const cacheKey = `${pageNumber}-${pageSize}`;

  if (pageCache.has(cacheKey)) {
    return pageCache.get(cacheKey);
  }

  const response = await fetch(
    `${apiUrl}/people?page[number]=${pageNumber}&page[size]=${pageSize}`,
    { headers }
  );

  const data = await response.json();
  pageCache.set(cacheKey, data);

  return data;
}

4. Use Filters to Reduce Total Pages

Instead of paginating through everything:

# ✅ Good: Filter first, then paginate
GET /people?filter[updated_at_gteq]=2024-01-01&page[size]=100

# ❌ Avoid: Paginate through everything
GET /people?page[size]=100

5. Combine with Sparse Fieldsets

Reduce payload size by requesting only needed fields:

GET /people?fields[people]=uuid,given_name,family_name&page[size]=100

Handling Edge Cases

Empty Results

When no results match your query:

{
  "data": [],
  "meta": {
    "total": 0
  }
}

Last Page

The last page may have fewer records than page[size]:

// Request page size of 25, but only 7 records remain
{
  "data": [/* 7 records */],
  "links": {
    "self": "...",
    "first": "...",
    "prev": "..."
    // No 'next' link
  },
  "meta": {
    "total": 57  // 2 full pages + 7 records
  }
}

Out of Range Page Number

Requesting a page beyond the last page returns an empty result:

# If there are only 3 pages, requesting page 10:
GET /people?page[number]=10&page[size]=25
{
  "data": [],
  "links": {
    "self": "...",
    "first": "...",
    "last": "..."
  },
  "meta": {
    "total": 57
  }
}

Performance Optimization

1. Incremental Sync Pattern

For keeping data in sync, use timestamp filtering with pagination:

async function syncPeople(lastSyncTime) {
  const updatedPeople = [];
  let currentPage = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(
      `${apiUrl}/people?filter[updated_at_gteq]=${lastSyncTime}&sort=updated_at&page[number]=${currentPage}&page[size]=100`,
      { headers }
    );

    const data = await response.json();
    updatedPeople.push(...data.data);

    hasMore = !!data.links.next;
    currentPage++;
  }

  return updatedPeople;
}

// Usage
const lastSync = '2024-01-15T12:00:00Z';
const updates = await syncPeople(lastSync);

2. Parallel Page Fetching

For faster bulk operations, fetch multiple pages in parallel (respect rate limits):

async function fetchPeopleParallel(totalPages, pageSize = 100) {
  const pageNumbers = Array.from({ length: totalPages }, (_, i) => i + 1);

  // Fetch pages in batches to avoid overwhelming the API
  const batchSize = 5;
  const allPeople = [];

  for (let i = 0; i < pageNumbers.length; i += batchSize) {
    const batch = pageNumbers.slice(i, i + batchSize);

    const promises = batch.map(pageNum =>
      fetch(
        `${apiUrl}/people?page[number]=${pageNum}&page[size]=${pageSize}`,
        { headers }
      ).then(r => r.json())
    );

    const results = await Promise.all(promises);
    results.forEach(data => allPeople.push(...data.data));

    // Delay between batches
    if (i + batchSize < pageNumbers.length) {
      await new Promise(resolve => setTimeout(resolve, 1000));
    }
  }

  return allPeople;
}

Be cautious with parallel requests and monitor your rate limit headers to avoid 429 responses.

Pagination Strategies

Strategy 1: Load on Demand

For UI applications, load pages as the user navigates:

// User clicks "Next Page"
function loadNextPage() {
  currentPage++;
  fetchPage(currentPage, 25);
}

Strategy 2: Fetch All (Small Datasets)

For small datasets (< 1000 records), fetch all at once:

const allPeople = await fetchAllPeople();

Strategy 3: Incremental Loading

Load more as needed (infinite scroll):

async function loadMore() {
  if (!hasMore) return;

  const nextPage = await fetchPage(currentPage + 1);
  appendToUI(nextPage.data);

  currentPage++;
  hasMore = !!nextPage.links.next;
}

Strategy 4: Background Sync

For background processes, paginate with delays:

async function backgroundSync() {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const data = await fetchPage(page, 100);
    await processData(data.data);

    hasMore = !!data.links.next;
    page++;

    // Respect rate limits
    await delay(500);
  }
}

On this page