Aimeos 2019.04 stable release

 

The stable 2019.04 release contains some major improvements:

  • Improved core APIs and rewritten frontend controllers
  • XML importer for attributes, categories, customers, products and suppliers
  • Increased performance for complex queries
  • More SEO friendly URLs

Improved APIs

The Aimeos core already offered an easy to learn API for working with managers, items and providers. All methods are now supporting fluent interfaces if possible so method calls can be chained like:

$item = $manager->createItem()->fromArray( $map )
    ->setType( 'default' )->addPropertyItem( $propItem );

Additionally, the frontend controller API for working with the shop objects efficiently in the HTML clients and the JSON REST API has been completely overhauled. All objects contain simple, fluent methods that can be concatenated like in the core library. Implementing your own product list in the frontend is now only a matter of a few easy to understand calls:

$products = \Aimeos\Controller\Frontend::create( $context, 'product' )
   ->uses( ['text', 'price', 'media' )->category( 123 )->text( 'sneaker' )
   ->sort( 'name' )->slice( 0, 48 )->search();

This statement would retrieve the first 48 products with texts, prices and images that are in category “123” and contain the text “sneaker” sorted by their names. In Laravel, facades simplify working with the frontend controller even more:

$products = Product::uses( ['text', 'price', 'media' )->category( 123 )
   ->text( 'sneaker' )->sort( 'name' )->slice( 0, 48 )->search();

XML importer

It’s now possible to import attributes, categories, customers, products and suppliers using Aimeos XML files. They are as flexible as the Aimeos data structures so you can import all items in a document oriented way. A simple example for a product XML import file would be:

<products>
    <product ref="test-article">
       <product.type>event</product.type>
       <product.code>test-article</product.code>
       <product.label>Test event</product.label>
       <product.datestart>2000-01-01T10:00:00</product.datestart>
       <lists>
          <text><textitem>...</textitem></text>
          <price><priceitem>...</priceitem></price>
          <media><mediaitem>...</mediaitem></media>
       </lists>
    </product>
</products>

The product XML export job produces the same XML format so you can share data between different Aimeos instances too.

Performance

Aimeos has gotten faster once more. Complex queries which ask for entries that reference items from other domains or have properties of a specific value are now lightning fast in all DBMS when using the *:has and *:prop search functions. To fetch all products which reference a specific attribute use:

$search = $manager->createSearch();
$func = $search->createFunction( 'product:has', ['attribute', 'default', 123 );
$search->setConditions( $search->combine( '!=', $func, null ) );
$products = $manager->searchItems();

To get notfied about queries which take longer than one second, a monitoring similar to the MySQL slowlog is now active. Furthermore, using utf8mb4 as charset in MySQL doesn’t have a performance drawback any more so it’s now used by default. Existing setups are migrated towards this charset if possible (requires MySQL >= 5.7.8).

SEO friendly URLs

In the past, Aimeos required the product ID to be part of the URL. Due to the simplifed index for product texts which contains the product URL name and due to optimized routes, the URLs of all detail pages can now look like

https://shop.com/My_cool_product

The product name in the URL can be localized for each language of corse.

#gigacommerce – Aimeos shop performance with 1 billion items

 

Aimeos has recently proven to be capable of handling one billion articles and more in an online shop using the #gigacommerce extension. This is far more than the number of articles at the Amazon market place which is currently estimated by around 562 million items. But the sheer numbers doesn’t say anything about the performance so how fast can Aimeos delivery content when users browsing the shop pages?

The setup

For a realistic test Aimeos has been set up in a standard cloud hosting environent sponsored by Profihost, an e-commerce hoster from Hannover, Germany. They provided two of their “Flexserver Expert” packages, which are virtual machines running on a cluster with Intel Xeon Skylake 2,6 GHz CPUs.

Hardware

  • LAMP VM: 8 Core, 16GB RAM, 17GB on SSD

  • ElasticSearch VM: 8 Core, 32GB RAM, 465GB on SSD

Software

  • Debian Linux, Apache 2.4, MySQL 5.7, PHP 7.2, ElasticSearch 5.4

  • Laravel 5.7, Symfony 4.1, TYPO3 8.7

  • Aimeos 2018.10, ai-elastic extension 2018.10

Test procedure

All tests have been done with Loader.io, a SaaS provider for load tests. The latency between the clients and the server is about 32ms, which is included in the response times. A typical page request of a detail page therefore looks as follows:

ACTION TIME
Stalled 5.26 ms
Request sent 0.12 ms
Waiting (TTFB) 50.74 ms
Content Download 18.33 ms
Sum 75.28 ms

The response time therefore consists of the network latency (32ms), the processing on the server (50.7ms – 32ms = 18.7ms) and the transmission time of the HTML page (18.3ms). The network latency is the largest influencing factor here and can be reduced only with difficulty in real operation. In general it is lower the closer the server is geographically to the customer and the better the connection of the hoster is.

The Aimeos list and detail page were tested, both with cache and without. The cache does NOT contain the complete page (“full page cache”) as in other shop systems, but only page fragments such as the product list or the product details. The page frame is provided by Laravel, Symfony and TYPO3, in which the fragments are inserted before delivering the page to the client.

Test results

The tests showed that Laravel 5.7 and Symfony 4.1 are about the same speed. In the evaluations of Loader.io, there were no significant differences between the two and therefore the results were summarized. The only measurable difference is about 5ms, which Symfony 4.1 can generate on an unloaded system faster than Laravel 5.7. Under load there is no more difference.

With TYPO3 as frontend, the server can process fewer requests per second, but TYPO3 is not directly comparable to Laravel/Symfony. As a complete content management system, TYPO3 allows the maintenance of pages by editors, which is not easily possible with Laravel/Symfony.

Conclusion

With over 600 PI/second and a processing time on the server less than 20ms, the Aimeos E-Commerce System is certainly one of the fastest shop systems. The differences between list and detail view are mainly due to the different page size (189KB to 66KB) and the resulting longer time for data transfer.

Competitors can only achieve similar results by storing complete pages in a Varnish cache and loading shopping carts and other dynamic content through AJAX queries. However, Aimeos is so fast that this would make response times even worse, because the separate query for e.g. the shopping cart is not necessary at all with Aimeos.

Even without cache the number of 130-150 PI/second and the response times of 162ms incl. network latency (130ms without latency) are excellent values compared to other shop systems, especially since the system can scale up to 1 billion and more items. This makes Aimeos particularly suitable for high-load scenarios such as TV advertising.

More about #gigacommerce: https://aimeos.com/gigacommerce

Aimeos 2019.01 major release

 

The new beta version contains a lot of improvements and focuses on:

  • Even more performance
  • Developer experience
  • Code cleanup

Performance improvements

Aimeos is already known as ultra fast e-commerce solution for Laravel, Symfony, SlimPHP, TYPO3 and Flow/NeosCMS even for complex requirements. This release pushes the limits teared down by Aimeos #gigacommerce further with render times of 20ms and below if content caching is enabled. Without caching, response times are exceptional too compared to other shop systems and in the 2019.01 release, we reduced that time further.

First, we denormalized tables that use types, so the type code is now stored directly instead of the type ID. This saves several database queries per request. Additionally, it’s possible to retrieve related data for e.g. products filtered by type:

$manager->searchItems( $filter, ['text', 'price', 'product' => ['suggestion'] );

This fetches related products of type “suggestion” only instead of all related products. Especially for shops with many relations, this can be a huge performance improvement if components can’t be cached.

PHP 7.3 promises another huge performance improvement and we are testing that version every time. Unfortunately, there’s a bug in PHP 7.3 with prevents production usage yet. It seems that the PHP developers over-optimized opcode caching so PHP 7.3 produces wrong results: https://bugs.php.net/bug.php?id=77310

Simplifications for developers

One of the major goals for this Aimeos release was to simplify interfaces so the first steps for new developers are easier. As a result, creating managers, controllers and clients requires only:

$manager = \Aimeos\MShop::create( $context, 'product' );
$cntl = \Aimeos\Controller\Frontend::create( $context, 'catalog' );
$client = \Aimeos\Client\Html::create( $context, 'catalog/lists' );

Items with relations are fetched recursively now and a decorator limits the recursion depth to two levels. The allowed depth is configurable like everything else in Aimeos. This allows you to get all releated data you need at one without fetching and merging data from several calls to searchItems():

$products = $manager->searchItems( $filter, ['product', 'price', 'text] );

For bundle and selection products, the items in the result list contains the bundled and variant articles including prices, texts and related products.

Also, all newly created items have the status “enabled” by default and must be explicitly disabled if required. This saves many calls to “setStatus()” especially when implementing a new importer. The frontend controllers allow multiple category IDs for filtering so it’s possible to search for
products in multiple categories with one method call.

Code cleanups

Finally, a lot of old code has been removed due to the switch to document oriented managment of items. Especially in the index managers and the filter criteria implementation was some code left after the changes required to use ElasticSearch natively.

The 2019.x releases are the last ones that will support PHP 5 as 5.6 isn’t maintained any more since this year. You should use PHP 7 nevertheless to get the best performance.

For a full list of backward incompatible changes, please have a look at the Wiki article. BC breaking changes in 2019.01 are listed up to 2018-12-31:

https://aimeos.org/docs/Developers/Changelog#Version_2019.x

 

#gigacommerce – Scalable online shops with 1 billion items and more

 

Does an online shop with 1 billion items work on a standard cloud server? And that even with the demand for very low response times? We investigated this question during the #gigacommerce project.

The #gigacommerce setup contains 1 billion articles, which are combined into 10 million products. Each item contains attributes, images, prices and text like in any online store. The basis is a Symfony, Laravel and TYPO3 installation with the Aimeos E-Commerce extension, which access an ElasticSearch server.

All three frontends have been extensively tested to find out where the limits are. The Aimeos components showed that they scale almost infinitely, with response times of around 100 milliseconds. Interestingly, the only absolute limit in the tests was not the available hardware, but the network bandwidth, which was used up at over 600 PI/second. This makes it particularly interesting for peak load situations and TV advertising.

How it this possible?

The special feature of the Aimeos e-commerce system is its high flexibility. Data can be stored in different backends, which use the appropriate technology depending on the requirements – like here ElasticSearch for the product data. This makes the system ultra fast and scalable. Nevertheless, Aimeos is not a special solution, but comparable with other major players in the industry.

The open source software Aimeos can be adapted to the most diverse needs, in particular to complex B2B requirements and delivers everything that is important for an online shop: product catalogs, shopping baskets, payment integration, order management, e-mails and all optimized for mobile devices. Aimeos is available for TYPO3, Laravel, Symfony, SlimPHP and Flow/Neos but the software can be integrated into any PHP application.

More about Aimeos #gigacommerce