FetchListV2.make

Overview

When you need to process large volumes of data from list methods of REST API version 2 in parts (chunks), use FetchListV2.make().

This method implements a fast algorithm for iterating over large data sets without loading all data into memory at once. Each iteration returns the next page/batch of results until all data is received.

// Basic usage
import { EnumCrmEntityTypeId } from '@bitrix24/b24jssdk'

const generator = $b24.actions.v2.fetchList.make({
  method: 'crm.item.list',
  params: {
    entityTypeId: EnumCrmEntityTypeId.deal,
    filter: { '>opportunity': 10000 }
  },
  idKey: 'id',
  customKeyForResult: 'items',
  requestId: 'unique-request-id'
})

for await (const chunk of generator) {
  // Process chunk (e.g., save to database, analyze, etc.)
  console.log(`Processing ${chunk.length} items`)
}

When to Use FetchListV2.make()

1. Very large data volumes: When the number of records is in the thousands or tens of thousands.
2. Stream processing: When data needs to be processed as it arrives.
3. Long operations: When processing each record requires significant time.

Method Signature

make<T = unknown>(
  options: ActionFetchListV2
): AsyncGenerator<T[]>

Parameters

The options object contains the following properties:

Return Value

AsyncGenerator<T[]> — an asynchronous generator that returns data chunks as arrays of type T.

Each iteration of the generator returns:

• An array of elements (chunk) with up to 50 records.
• The generator completes when all data is received.

Key Concepts

Automatic Warning

When calling methods that are available in REST API version 3, the system automatically logs a warning:

"The method {method_name} is available in restApi:v3. It's worth migrating to the new API."

This indicates that you should consider migrating to the newer REST API version.

AsyncGenerator vs Promise

Unlike CallListV2.make(), which returns a Promise with all data at once, FetchListV2.make() returns an asynchronous generator:

Performance Optimization

The method implements the Bitrix24 recommended algorithm for efficient work with large data volumes:

1. start: -1: Disables counting the total number of records, significantly speeding up query execution.
2. Filtering by increasing ID: Each subsequent query uses a >ID filter with the ID of the last retrieved element.
3. Stream processing: Data is processed as it is received, saving memory.
4. Automatic data end detection: Requests stop when an empty array is received or the number of elements is less than the page size (50).

REST API v2 Response Structure

In REST API version 2, various methods may return data in different structures:

• Direct array: { result: [...] }
• Grouped array: { result: { items: [...] } }

The customKeyForResult parameter allows you to specify the key where the data is located in the response.

Limitations

• Page size: Fixed limitation of Bitrix24 REST API version 2 — 50 records per request.
• Sorting: The method always adds sorting by idKey in ascending order for correct pagination.
• Only for list methods: Intended only for methods that return data arrays.

Error Handling

Since the method returns an asynchronous generator, errors are handled differently than in CallListV2.make():

try {
  const generator = $b24.actions.v2.fetchList.make({
    method: 'some.method',
    params: { /* some_params */ },
    idKey: 'id',
    customKeyForResult: 'items',
    requestId: 'unique-request-id'
  })

  for await (const chunk of generator) {
    // Process chunk (e.g., save to database, analyze, etc.)
    console.log(`Processing ${chunk.length} items`)
  }
} catch (error) {
  // Handling error
  if (
    error instanceof SdkError
    && error.code === 'JSSDK_CORE_B24_FETCH_LIST_METHOD_API_V2'
  ) {
    console.error(`${error.message}`, { code: error.code })
  } else {
    console.error('Some error', error)
  }
}

Examples

Step-by-step processing of a large number of companies

import { B24Hook, EnumCrmEntityTypeId, LoggerFactory, Text, SdkError, AjaxError } from '@bitrix24/b24jssdk'

type Company = {
  id: number
  title: string
}

const devMode = typeof import.meta !== 'undefined' && (import.meta.dev || import.meta.env?.DEV)
const $logger = LoggerFactory.createForBrowser('Example:ProcessCrmItems', devMode)
const $b24 = B24Hook.fromWebhookUrl('https://your_domain.bitrix24.com/rest/1/webhook_code/')

async function processCrmItemList(): Promise<void> {
  let batchNumber = 0
  let totalItems = 0

  const sixMonthAgo = new Date()
  sixMonthAgo.setMonth((new Date()).getMonth() - 6)
  sixMonthAgo.setHours(0, 0, 0)
  
  const requestId = 'some-crm-item-list'
  
  try {
    const generator = $b24.actions.v2.fetchList.make<Company>({
      method: 'crm.item.list',
      params: {
        entityTypeId: EnumCrmEntityTypeId.company,
        filter: {
          // use some filter by title
          '=%title': 'Prime%',
          '>=createdTime': Text.toB24Format(sixMonthAgo) // created at least 6 months ago
        },
        select: ['id', 'title']
      },
      idKey: 'id',
      customKeyForResult: 'items',
      requestId
    })
    
    for await (const chunk of generator) {
      batchNumber++
      totalItems += chunk.length

      $logger.info(`Processing batch #${batchNumber}`, {
        batchSize: chunk.length,
        totalSoFar: totalItems
      })
      
      // Example: saving to database
      await saveToDatabase(chunk)
      
      // Example: sending to message queue
      await sendToMessageQueue(chunk)
    }
    
    $logger.notice(`Processed ${totalItems} elements in ${batchNumber} batches`)
  } catch (error) {
    if (error instanceof SdkError) {
      $logger.error(`Processing error: ${error.message}`, {
        code: error.code,
        batchNumber,
        totalItems
      })
    } else {
      $logger.error('Unknown error', { error, batchNumber, totalItems })
    }
    throw error
  }
}

// Helper functions
// Database save implementation
async function saveToDatabase(items: Company[]): Promise<void> {
  await new Promise(resolve => setTimeout(resolve, 100)) // Simulation
}

// Message queue send implementation
async function sendToMessageQueue(items: Company[]): Promise<void> {
  await new Promise(resolve => setTimeout(resolve, 50)) // Simulation
}

// Usage
try {
  await processCrmItemList()
} catch (error) {
  $logger.critical('A problem occurred', { error })
}

Alternatives and Recommendations

• Chunk size: Bitrix24 REST API version 2 always returns up to 50 records per request.
• Optimal concurrency: When processing data from the generator, limited concurrency (3-5 simultaneous operations) is recommended.
• Error handling: Always handle errors inside the for await...of loop to prevent the entire process from stopping.
• Progress monitoring: Implement progress logging for long-running operations.
• For sequential requests: Use Call for single calls.
• For batch operations: Use Batch to execute up to 50 commands in a single request.
• On the client-side (browser): Use the built-in B24Frame object.