To create your own #gigacommerce setup, you should use at least hardware fulfilling that specs:
Furthermore, you need these software:
For the #gigacommerce setup, please install:
This section contains the setup of the #gigacommerce demo setup.
You need a MySQL 5.7+ and ElasticSearch 5+ server installed. Afterwards, you have to create database (e.g. "aimeos") and configure how they can be accessed:
'resource' => [ 'db' => [ 'adapter' => 'mysql', 'host' => '127.0.0.1', 'port' => '3306', 'database' => 'aimeos', 'username' => '<user>', 'password' => '<pass>', 'stmt' => [ "SET SESSION sort_buffer_size=2097144; SET NAMES 'utf8mb4'; SET SESSION sql_mode='ANSI'" ], 'limit' => 3, 'defaultTableOptions' => [ 'charset' => 'utf8mb4', 'collate' => 'utf8mb4_bin', ], ], 'es' => [ 'hosts' => ['127.0.0.1:9200'], 'index' => 'aimeos', 'norefresh' => true, // only for indexing 'selectorClass' => '\Elasticsearch\ConnectionPool\Selectors\StickyRoundRobinSelector', 'settings' => [ 'max_result_window' => 10000, 'number_of_shards' => 8, 'number_of_replicas' => 0, 'refresh_interval' => '-1', ] ], 'es-stock' => [ 'hosts' => ['127.0.0.1:9200'], 'index' => 'aimeos_stock', 'norefresh' => true, // only for indexing 'selectorClass' => '\Elasticsearch\ConnectionPool\Selectors\StickyRoundRobinSelector', 'settings' => [ 'max_result_window' => 10000, 'number_of_shards' => 8, 'number_of_replicas' => 0, 'refresh_interval' => -1, ], ], ],
The database setup is pretty straight forward, in the two ElasticSearch configuration blocks there are a few settings to explain:
By default, Aimeos will use the database configured as db. To use ElasticSearch for product and stock data, you need to add these settings:
'mshop' => [ 'index' => [ 'manager' => [ 'name' => 'Elastic', ] ], 'product' => [ 'manager' => [ 'name' => 'Elastic', ] ], 'stock' => [ 'manager' => [ 'name' => 'Elastic', ] ] ],
In a standard setup, Aimeos caches list and detail views in the database. Due to the amount of data and varations possible in the #gigacommerce setup, this isn't desired. Use these settings to disable caching list and detail views:
'client' => [ 'html' => [ 'catalog' => [ 'lists' => [ 'cache' => false, 'standard' => [ 'subparts' => ['items'], ], ], 'detail' => [ 'cache' => false, ], 'stage' => [ 'standard' => [ 'subparts' => , ], ], ], ], ],
It will also disable two features (promotional product list in list view and last/next product link in detail view) that require at least one more ElasticSearch query and are turned of for raw speed.
To generate the same set of demo data as in the #gigacommerce demo, use these settings:
'setup' => [ 'unitperf' => [ 'max-batch' => 10000, 'num-catlevels' => 3, 'num-categories' => 1000, 'num-catproducts' => 10000, 'num-prodvariants' => 100, ], ],
The meaning of the configuration values is:
You can use different settings for the categories and products but you should use multiples of 10 for each setting. You can also use another distribution of categories and products like 100 categories, 10000 products/category, 1000 articles/product or 10000 categories, 10000 products/category, 10 articles/product. Only keep in mind that if the product or article value is above 10000, you have to adapt the max_result_window settings of your ElasticSearch configuration and MySQL could start choking due to the number of records in the mshop_catalog_list table (category -> product references).
'apc_enabled' => true, 'pcntl_max' => 24,
For maximum performance APCu must be enabled and this requires the PHP apcu extension to be installed. This will cache the used configuration settings and translations in the shared memory of the web server. How to enable APCu in Aimeos depends on the integration. Please check the sample configuration files for Laravel and Symfony resp. the Aimeos settings in the Extension Manager for TYPO3.
The optimal number of parallel processes for task like generating the demo data depends on your server hardware. A good starting point for tests is twice the number of processors used by ElasticSearch, so the processors are always fully occupied.
Nevertheless, indexing the products is an IO intensive task and most likely, the storage will be your bottleneck, not the CPUs.
Now you can start generating the #gigacommerce demo data. Depending on your host application, you have to use:
Now you have to wait. Depending on your hardware it will take a long time until the data is indexed in Elasticsearch. Using a single ElasticSearch node with 16 CPU cores, 48 GB RAM and SSDs it takes approx. 35h until the process finishes. If you have a cluster of several nodes, you only need a fraction of that time because the IO is distributed across all nodes.
To get the best performance in the frontend, you should merge the segments in the index to only one segment by using:
After merging the segments in all shards, you will have ca. 465 GB occupied on your SSDs.