Developers/Library/Fetch records in bunches

From Aimeos documentation

Developers
Other languages:
English 100%


2016.x version

Fetch records in bunches efficiently

Retrieving records in the database is always done in bunches (default: 100 records/searchItems, can be changed using \Aimeos\MW\Criteria\Iface::setslice()) and if you need to process more or all items, you have to repeatedly call searchItems() of the manager. The most efficient code for this is:

  1.  $start = 0;
  2.  $search->setSortations( array( $search->sort( '+', '<domain>.id' ) ) );
  3.  
  4.  do
  5.  {
  6.  	$result = $manager->searchItems( $search );
  7.  
  8.  	foreach ( $result as $item ) {
  9.  		// process items
  10.  	}
  11.  
  12.  	$count = count( $result );
  13.  	$start += $count;
  14.  	$search->setSlice( $start );
  15.  }
  16.  while( $count == $search->getSliceSize() );

Setting the sortation is important as MySQL isn't able to return all entries of an unsorted result only once. If you forget to add the sortation, chances are high that you will sometimes get records twice and others not at all. Please replace "<domain>" in the second parameter with the name of the domain, e.g. "product" or "product.list".