Help for Aimeos

Hi,

We have a developer building us a site on Aimeos. We have over 55 million simple products to import, no images and no configurations.

We have tried importing via the CSV method however it will take 10+ days to do this. We have a dedicated server, 8 core cpu and 64

We attempted a direct SQL import which was fast, but only a few thousand products displayed on the front end. He wasnt sure but he believes it due to the way the tables are set out in Aimeos DB.

Has any attempted importing so much data into Aimeos and the fastest way to do this (with the products showing ok on the front end) ?

Kind rgds

The fastest standard way is the XML import as it's much faster than the CSV importer.
But for 55m articles you should write a custom importer that uses the product manager directly. This is almost as fast as an importer using custom SQL statements but much easier to deal with additional data.

Note, that the frontend only shows 10k products in a single list by default as no one will go through longer lists without using full text search or the facet filter. As long as your articles are in the mshop_product table in indexed in the mshop_index_* tables, they will be found.

55m articles usually consists of a lot of variants of a product. You should add selection products afterwards, that group your articles so the amount of products referenced by the categories as much smaller (55m catalog<->product references would slow down the database).

Hi, thanks for responding. I will pass this to our developer

For fast searching and filtering, you need at least one of this extensions from the Aimeos company:
- ElasticSearch: Products are stored and indexed in ElasticSearch (https://aimeos.com/extensions#c435)
- Solr: Products are stored in DB and indexed in Solr (https://aimeos.com/extensions#c606)

For your amount of products, I would recommend the ElasticSearch (#gigacommere) extension.