FetchListV2.make

Overview

When you need to process large volumes of data from list methods of REST API version 2 in parts (chunks), use FetchListV2.make().

This method implements a fast algorithm for iterating over large data sets without loading all data into memory at once. Each iteration returns the next page/batch of results until all data is received.

// Basic usage
import { EnumCrmEntityTypeId } from '@bitrix24/b24jssdk'

const generator = $b24.actions.v2.fetchList.make({
  method: 'crm.item.list',
  params: {
    entityTypeId: EnumCrmEntityTypeId.deal,
    filter: { '>opportunity': 10000 }
  },
  idKey: 'id',
  customKeyForResult: 'items',
  requestId: 'unique-request-id'
})

for await (const chunk of generator) {
  // Process chunk (e.g., save to database, analyze, etc.)
  console.log(`Processing ${chunk.length} items`)
}

When to Use FetchListV2.make()

1. Very large data volumes: When the number of records is in the thousands or tens of thousands.
2. Stream processing: When data needs to be processed as it arrives.
3. Long operations: When processing each record requires significant time.

Method Signature

make<T = unknown>(
  options: ActionFetchListV2
): AsyncGenerator<T[]>

Parameters

The options object contains the following properties:

Return Value

AsyncGenerator<T[]> — an asynchronous generator that returns data chunks as arrays of type T.

Each iteration of the generator returns:

• An array of elements (chunk) with up to 50 records.
• The generator completes when all data is received.

Key Concepts

Automatic Warning

When calling methods that are available in REST API version 3, the system automatically logs a warning:

"The method {method_name} is available in restApi:v3. It's worth migrating to the new API."

This indicates that you should consider migrating to the newer REST API version.

AsyncGenerator vs Promise

Unlike CallListV2.make(), which returns a Promise with all data at once, FetchListV2.make() returns an asynchronous generator:

Performance Optimization

The method implements the Bitrix24 recommended algorithm for efficient work with large data volumes:

1. start: -1: Disables counting the total number of records, significantly speeding up query execution.
2. Filtering by increasing id: Each subsequent query uses a >cursorIdKey filter with the id of the last retrieved element (read via idKey).
3. Stream processing: Data is processed as it is received, saving memory.
4. Automatic data end detection: Requests stop when an empty array is received or the number of elements is less than the page size (50). A short page ends the walk only once the cursor read from it has been checked — a page that is short and repeats the cursor is a server ignoring the page condition, not the end of the data, and is reported rather than returned.

REST API v2 Response Structure

In REST API version 2, various methods may return data in different structures:

• Direct array: { result: [...] }
• Grouped array: { result: { items: [...] } }

The customKeyForResult parameter allows you to specify the key where the data is located in the response.

Limitations

• Page size: Fixed limitation of Bitrix24 REST API version 2 — 50 records per request.
• Sorting is fixed: The method always sorts by cursorIdKey (which defaults to idKey) ascending, because cursor pagination relies on a >cursorIdKey filter to walk the dataset. A user-supplied order value would break that invariant, so the declared order property is Omitted from the type — though the [key: string]: unknown index signature it inherits means the compiler still accepts one — and any value passed at runtime is stripped with a warning log entry. To narrow the result set, use filter instead.
• Conditions go in lowercase filter, and FILTER has to be removed: the method pages by writing its own lowercase filter, order and start, and the portal keeps only the later of two top-level keys that differ by case. Older list methods are documented with uppercase FILTER / SORT / ORDER, so following that documentation here is the mistake. Measured on user.get with four users: FILTER: { ID: 4 } returns all four, filter: { ID: 4 } returns one. Which of the pair is later is not fixed, so the two ways of getting this wrong fail in opposite directions — passing only FILTER drops your conditions, while passing FILTER and filter drops the walker's own cursor and stalls the walk (see the next bullet). Move the conditions across; do not just add a lowercase key beside the uppercase one. SORT fails louder still — on user.get it makes the injected order fail the method's own validation, so the request throws ERROR_ARGUMENT / "Order must be a string". All of these are reported with a warning.
• A cursor that stops advancing is fatal: if the last idKey value on a page equals the one already filtered on, the >cursorIdKey condition was dropped and the same page will keep arriving. A full page cannot end the walk on its own — the length < 50 stop never fires — and a short one is checked before it is taken for the end of the data, so it throws SdkError with code JSSDK_ACTION_CURSOR_STALLED instead of collecting duplicates for ever. The usual cause is idKey naming the response field while cursorIdKey is left to default to it: tasks.task.list returns a lowercase id but filters on an uppercase ID, so it needs both (idKey: 'id', cursorIdKey: 'ID'). A cursor that moves the wrong way is caught too, and separately: a server alternating between two pages never repeats the value just sent, so the equality test above cannot see it — the walk is stopped with JSSDK_ACTION_CURSOR_WENT_BACKWARDS on the first backwards step. And neither check waits for a full page any more: a short page whose cursor did not advance used to end the walk as end-of-data, returning a truncated and overlapping result with no error at all.
• Only for list methods: Intended only for methods that return data arrays.

Error Handling

Since the method returns an asynchronous generator, errors are handled differently than in CallListV2.make():

import { SdkError } from '@bitrix24/b24jssdk'

try {
  const generator = $b24.actions.v2.fetchList.make({
    method: 'some.method',
    params: { /* some_params */ },
    idKey: 'id',
    customKeyForResult: 'items',
    requestId: 'unique-request-id'
  })

  for await (const chunk of generator) {
    // Process chunk (e.g., save to database, analyze, etc.)
    console.log(`Processing ${chunk.length} items`)
  }
} catch (error) {
  // Handling error
  if (
    error instanceof SdkError
    && error.code === 'JSSDK_CORE_B24_FETCH_LIST_METHOD_API_V2'
  ) {
    console.error(`${error.message}`, { code: error.code })
  } else {
    console.error('Some error', error)
  }
}

Examples

Step-by-step processing of a large number of companies

import { B24Hook, EnumCrmEntityTypeId, LoggerFactory, Text, SdkError, AjaxError } from '@bitrix24/b24jssdk'

type Company = {
  id: number
  title: string
}

const devMode = !!(typeof import.meta !== 'undefined' && (import.meta.dev || import.meta.env?.DEV))
const $logger = LoggerFactory.createForBrowser('Example:ProcessCrmItems', devMode)
const $b24 = B24Hook.fromWebhookUrl('https://your_domain.bitrix24.com/rest/1/webhook_code/')

async function processCrmItemList(): Promise<void> {
  let batchNumber = 0
  let totalItems = 0

  const sixMonthAgo = new Date()
  sixMonthAgo.setMonth((new Date()).getMonth() - 6)
  sixMonthAgo.setHours(0, 0, 0)
  
  const requestId = 'some-crm-item-list'
  
  try {
    const generator = $b24.actions.v2.fetchList.make<Company>({
      method: 'crm.item.list',
      params: {
        entityTypeId: EnumCrmEntityTypeId.company,
        filter: {
          // use some filter by title
          '=%title': 'Prime%',
          '>=createdTime': Text.toB24Format(sixMonthAgo) // created at least 6 months ago
        },
        select: ['id', 'title']
      },
      idKey: 'id',
      customKeyForResult: 'items',
      requestId
    })
    
    for await (const chunk of generator) {
      batchNumber++
      totalItems += chunk.length

      $logger.info(`Processing batch #${batchNumber}`, {
        batchSize: chunk.length,
        totalSoFar: totalItems
      })
      
      // Example: saving to database
      await saveToDatabase(chunk)
      
      // Example: sending to message queue
      await sendToMessageQueue(chunk)
    }
    
    $logger.notice(`Processed ${totalItems} elements in ${batchNumber} batches`)
  } catch (error) {
    if (error instanceof SdkError) {
      $logger.error(`Processing error: ${error.message}`, {
        code: error.code,
        batchNumber,
        totalItems
      })
    } else {
      $logger.error('Unknown error', { error, batchNumber, totalItems })
    }
    throw error
  }
}

// Helper functions
// Database save implementation
async function saveToDatabase(items: Company[]): Promise<void> {
  await new Promise(resolve => setTimeout(resolve, 100)) // Simulation
}

// Message queue send implementation
async function sendToMessageQueue(items: Company[]): Promise<void> {
  await new Promise(resolve => setTimeout(resolve, 50)) // Simulation
}

// Usage
try {
  await processCrmItemList()
} catch (error) {
  $logger.critical('A problem occurred', { error })
}

Paginating tasks.task.list (request field ≠ response field)

tasks.task.list sorts and filters by ID (uppercase) but returns each task with a lowercase id. Point idKey at the response field and cursorIdKey at the request field — the cursor then reads the last id from the page and asks for the next one with >ID:

type TaskListItem = { id: string, title: string }

const generator = $b24.actions.v2.fetchList.make<TaskListItem>({
  method: 'tasks.task.list',
  params: {
    filter: { RESPONSIBLE_ID: 1 }, // tasks filter fields are uppercase
    select: ['ID', 'TITLE'] // selected uppercase, but the response carries lowercase id / title
  },
  idKey: 'id', // read task.id from the response
  cursorIdKey: 'ID', // order + ">ID" page filter in the request
  customKeyForResult: 'tasks'
})

let total = 0
for await (const chunk of generator) {
  total += chunk.length
}
console.log(`Processed ${total} tasks`)

Without cursorIdKey, the default idKey: 'ID' can't be read from the lowercase response, so pagination stops after the first 50 tasks — and the SDK logs a warning that points you here.

Alternatives and Recommendations

• Chunk size: Bitrix24 REST API version 2 always returns up to 50 records per request.
• Optimal concurrency: When processing data from the generator, limited concurrency (3-5 simultaneous operations) is recommended.
• Error handling: Always handle errors inside the for await...of loop to prevent the entire process from stopping.
• Progress monitoring: Implement progress logging for long-running operations.
• For sequential requests: Use Call for single calls.
• For batch operations: Use Batch to execute up to 50 commands in a single request.
• On the client-side (browser): Use the built-in B24Frame object.
• See also: Filtering — building the filter (and why order is stripped).

Bounding the walk

The walk pages until the data runs out. Three options bound it.

maxPages stops after that many pages and throws JSSDK_ACTION_MAX_PAGES_EXCEEDED, naming the method, after yielding every page it read. It defaults to 10 000 — a backstop rather than a policy: on restApi:v2 that is 500 000 rows, and at the default drain rate of 2 requests/second roughly 83 minutes, so a walk that never ends is bounded without capping a read anyone performs. Raise it when a read genuinely needs more.

It errors rather than truncating. A short list that looks complete is the failure mode this whole class of bug keeps producing, so nothing is returned — though the pages already yielded are yours to keep.

signal takes an AbortSignal and throws JSSDK_ACTION_ABORTED. It is checked at the top of each iteration, so an already-aborted signal costs no request at all.

The streaming walkers take no progress: they hand you each page as it arrives, so counting them is your own for await body.