Import products with images from CSV

Help for integrating the Laravel package
Forum rules
Always add your Laravel, Aimeos and PHP version as well as your environment (Linux/Mac/Win)
Spam and unrelated posts will be removed immediately!
columbo
Advanced
Posts: 124
Joined: 09 Oct 2019, 09:42

Re: Import products with images from CSV

Post by columbo » 16 Feb 2020, 18:55

Did you install Aimeos in a /shop subdirectory on your server?
yes, and it worked fine so far.
You are right, I used an older version. I updated now to 2019.10.10,
set APP_URL=http://x.x.x.x:80 (wihtout /shop) and added 'absoluteUri' => true in shop.php

unfortunately csv upload is still not working - same as before:

Code: Select all

<div class="media-item">
<img class="" src="http://x.x.x.x:80/shop/" data-src="http://x.x.x.x:80/shop/" data-srcset="http://x.x.x.x:80/shop/ 1w" alt="images/prodimage.jpg" srcset="http://x.x.x.x:80/shop/ 1w">
<meta itemprop="contentUrl" content="http://x.x.x.x:80/shop/">
here the mapping:

Code: Select all

'mapping' => [
	'item' => array(
		1 => 'product.code', // e.g. unique EAN code					
		3 => 'product.label', // UTF-8 encoded text, also used as product name
		2 => 'product.type', // type of the product, e.g. "default" or "selection"
		9 => 'product.status', // enabled (1) or disabled (0)
	),
	'text' => array(
		4 => 'text.type', // e.g. "short" for short description
		5 => 'text.content', // UTF-8 encoded text								
	),
	'media' => array(
		39 => 'media.url', // relative URL of the product image on the server
	),
	'price' => array(
		14 => 'product.lists.type',
		7 => 'price.taxrate', // tax rate with decimals separated by a dot
		16 => 'price.currencyid', // three letter ISO currency code
		17 => 'price.value', // price with decimals separated by a dot
		18 => 'price.quantity', // amount the quantity the price (for block pricing)
		20 => 'product.lists.datestart',
		21 => 'product.lists.dateend',
	),
	'attribute' => array(
		12 => 'product.lists.type', // e.g. "suggestion" for suggested product
		10 => 'attribute.code', // code of an attribute, will be created if not exists
		11 => 'attribute.type', // e.g. "size", "length", "width", "color", etc.
	),
	'product' => array(
		15 => 'product.code', // e.g. EAN code of another product
		22 => 'product.lists.type', // e.g. "suggestion" for suggested product
	),
	'catalog' => array(
		8 => 'catalog.code', // e.g. Unique category code
		38 => 'catalog.code', // e.g. Unique category code
	),
	'stock' => array(
		13 => 'stock.stocklevel',
	),
],

bouuman
Posts: 25
Joined: 17 Feb 2020, 18:37

Re: Import products with images from CSV

Post by bouuman » 17 Feb 2020, 18:44


the csv setup configuration is stored here right?
/httpdocs/typo3conf/ext/aimeos/Resources/Private/Extensions/ai-controller-jobs/controller/common/src/Controller/Common/Product/Import/Csv/base.php
Last edited by bouuman on 18 Feb 2020, 09:59, edited 1 time in total.

User avatar
aimeos
Administrator
Posts: 7866
Joined: 01 Jan 1970, 00:00

Re: Import products with images from CSV

Post by aimeos » 18 Feb 2020, 09:49

columbo wrote: 16 Feb 2020, 18:55 unfortunately csv upload is still not working - same as before:
here the mapping:
The mapping seems to be OK. What is the content of the mshop_media table (link and preview columns)?
Can you also post a few CSV lines as examples you want to import?
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

User avatar
aimeos
Administrator
Posts: 7866
Joined: 01 Jan 1970, 00:00

Re: Import products with images from CSV

Post by aimeos » 18 Feb 2020, 09:52

bouuman wrote: 17 Feb 2020, 18:44 the csv setup configuration is stored here right?
/httpdocs/typo3conf/ext/aimeos/Resources/Private/Extensions/ai-controller-jobs/controller/common/src/Controller/Common/Product/Import/Csv/base.php
For TYPO3, the configuration location is a bit different then for Laravel and we don't want to mix them up in the Laravel forum. Please create a new thread in the TYPO3 forum: typo3-and-flow-neoscms-f16/
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

columbo
Advanced
Posts: 124
Joined: 09 Oct 2019, 09:42

Re: Import products with images from CSV

Post by columbo » 18 Feb 2020, 11:43

aimeos wrote: 18 Feb 2020, 09:49 The mapping seems to be OK. What is the content of the mshop_media table (link and preview columns)?
Can you also post a few CSV lines as examples you want to import?
mshop_media tabel:
  • column link: /images/prodimage.jpg
  • column preview: {"1":""}
csv sample:

Code: Select all

id,str_item_number,producttype,str_name,texttype,desc,str_item_group,f_vat,productcategory,productstatus,str_unit,attributetype,attributeproductlist,stocklevel,productliststype,productcode,currencyid,f_price,f_from_amount,pricetypeid,dt_valid_from,dt_valid_to,priceproductliststype,currencyid_1,price_1,amount_1,valid_from_1,valid_to_1,currencyid_2,price_2,amount_2,valid_from_2,valid_to_2,currencyid_3,price_3,amount_3,valid_from_3,valid_to_3,productcategory2,mediaurl,mediaurl,f_vat,medialabel,mediaproductliststype
11750,0123-kt,default,"XYZ kt",name,"XYZ",0123,10,,1,kt,Verkaufseinheit,variant,,default,,EUR,10.1,1,,"2020-02-14 23:00:00",,,EUR,,,,,EUR,,,,,EUR,,,,,,,,10,,
11750,0123,select,"XYZ",name,"XYZ",0123,10,obst_fruechte,1,,,,,,"0123-kt
                0123-kt",EUR,,,,,,default,,,,,,,,,,,,,,,,frischware,/images/prodimage.jpg,image/jpeg,,tmpmedia,default

User avatar
aimeos
Administrator
Posts: 7866
Joined: 01 Jan 1970, 00:00

Re: Import products with images from CSV

Post by aimeos » 18 Feb 2020, 17:27

Your images are stored in your ./public/ folder before the import e.g. as ./public/images/prodimage.jpg, right?
It looks like your images are either not available or not found for some reason.

If the images are there, are the preview images generated if you execute the media/scale job?

Code: Select all

./artisan aimeos:jobs media/scale
Before execute the job, you should set the "force" flag:
https://aimeos.org/docs/Configuration/C ... dard/force
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

columbo
Advanced
Posts: 124
Joined: 09 Oct 2019, 09:42

Re: Import products with images from CSV

Post by columbo » 18 Feb 2020, 22:08

changed nothing else, just executed:

Code: Select all

artisan aimeos:jobs media/scale
images are displayed now - THX!
its working with and without setting:

Code: Select all

controller/jobs/media/scale/standard/force = 1
But I'm still wondering why? Do I have to execute this manually or is it a configuration issue?
This command should be executed after the csv import job automatically - shouldn't it?

User avatar
aimeos
Administrator
Posts: 7866
Joined: 01 Jan 1970, 00:00

Re: Import products with images from CSV

Post by aimeos » 19 Feb 2020, 08:02

Can you post the output of "composer show"?
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

columbo
Advanced
Posts: 124
Joined: 09 Oct 2019, 09:42

Re: Import products with images from CSV

Post by columbo » 19 Feb 2020, 21:11

composer show:

Code: Select all

aimeos/ai-admin-jqadm                 2019.10.9  Aimeos ai-admin-jqadm extension
aimeos/ai-admin-jsonadm               2019.10.4  Aimeos ai-admin-jsonadm extension
aimeos/ai-client-html                 2019.10.12 Aimeos ai-client-html extension
aimeos/ai-client-jsonapi              2019.10.4  Aimeos JSON API extension
aimeos/ai-controller-frontend         2019.10.8  Aimeos ai-controller-frontend extension
aimeos/ai-controller-jobs             2019.10.10 Aimeos ai-controller-jobs extension
aimeos/ai-gettext                     2019.10.1  Aimeos Gettext extension
aimeos/ai-laravel                     2019.10.5  Laravel adapter for Aimeos web shops and e-commer...
aimeos/ai-swiftmailer                 2019.10.1  SwiftMailer adapter for Aimeos web shops and e-co...
aimeos/aimeos-core                    2019.10.17 Full-featured e-commerce components for high perf...
aimeos/aimeos-laravel                 2019.10.2  Professional, full-featured and high performance ...
composer/installers                   v1.8.0     A multi-framework Composer library installer
dnoegel/php-xdg-base-dir              v0.1.1     implementation of xdg base directory specificatio...
doctrine/cache                        1.10.0     PHP Doctrine Cache library is a popular cache imp...
doctrine/dbal                         v2.10.1    Powerful PHP database abstraction layer (DBAL) wi...
doctrine/event-manager                1.1.0      The Doctrine Event Manager is a simple PHP event ...
doctrine/inflector                    1.3.1      Common String Manipulations with regard to casing...
doctrine/instantiator                 1.3.0      A small, lightweight utility to instantiate objec...
doctrine/lexer                        1.2.0      PHP Doctrine Lexer parser library that can be use...
dragonmantank/cron-expression         v2.3.0     CRON for PHP: Calculate the next or previous run ...
egulias/email-validator               2.1.17     A library for validating emails against several RFCs
facade/flare-client-php               1.3.1      Send PHP errors to Flare
facade/ignition                       1.16.0     A beautiful error page for Laravel applications.
facade/ignition-contracts             1.0.0      Solution contracts for Ignition
fideloper/proxy                       4.2.2      Set trusted proxies for Laravel
filp/whoops                           2.7.1      php error handling for cool kids
fzaninotto/faker                      v1.9.1     Faker is a PHP library that generates fake data f...
hamcrest/hamcrest-php                 v2.0.0     This is the PHP port of Hamcrest Matchers
jakub-onderka/php-console-color       v0.2
jakub-onderka/php-console-highlighter v0.4       Highlight PHP code in terminal
laravel/framework                     v6.15.1    The Laravel Framework.
laravel/tinker                        v1.0.10    Powerful REPL for the Laravel framework.
laravel/ui                            v1.2.0     Laravel UI utilities and presets.
league/commonmark                     1.3.0      Highly-extensible PHP Markdown parser which fully...
league/flysystem                      1.0.64     Filesystem abstraction: Many filesystems, one API.
mockery/mockery                       1.3.1      Mockery is a simple yet flexible PHP mock object ...
monolog/monolog                       2.0.2      Sends your logs to files, sockets, inboxes, datab...
myclabs/deep-copy                     1.9.5      Create deep copies (clones) of your objects
nesbot/carbon                         2.30.0     An API extension for DateTime that supports 281 d...
nikic/php-parser                      v4.3.0     A PHP parser written in PHP
nunomaduro/collision                  v3.0.1     Cli error handling for console/command-line PHP a...
opis/closure                          3.5.1      A library that can be used to serialize closures ...
paragonie/random_compat               v9.99.99   PHP 5.x polyfill for random_bytes() and random_in...
phar-io/manifest                      1.0.3      Component for reading phar.io manifest informatio...
phar-io/version                       2.0.1      Library for handling version information and cons...
phpdocumentor/reflection-common       2.0.0      Common reflection classes used by phpdocumentor t...
phpdocumentor/reflection-docblock     5.0.0      With this component, a library can provide suppor...
phpdocumentor/type-resolver           1.0.1      A PSR-5 based resolver of Class names, Types and ...
phpoption/phpoption                   1.7.2      Option Type for PHP
phpspec/prophecy                      v1.10.2    Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage             7.0.10     Library that provides collection, processing, and...
phpunit/php-file-iterator             2.0.2      FilterIterator implementation that filters files ...
phpunit/php-text-template             1.2.1      Simple template engine.
phpunit/php-timer                     2.1.2      Utility class for timing
phpunit/php-token-stream              3.1.1      Wrapper around PHP's tokenizer extension.
phpunit/phpunit                       8.5.2      The PHP Unit Testing framework.
psr/container                         1.0.0      Common Container Interface (PHP FIG PSR-11)
psr/http-message                      1.0.1      Common interface for HTTP messages
psr/log                               1.1.2      Common interface for logging libraries
psr/simple-cache                      1.0.1      Common interfaces for simple caching
psy/psysh                             v0.9.12    An interactive shell for modern PHP.
ramsey/uuid                           3.9.2      Formerly rhumsaa/uuid. A PHP 5.4+ library for gen...
scrivo/highlight.php                  v9.18.1.0  Server side syntax highlighter that supports 185 ...
sebastian/code-unit-reverse-lookup    1.0.1      Looks up which function or method a line of code ...
sebastian/comparator                  3.0.2      Provides the functionality to compare PHP values ...
sebastian/diff                        3.0.2      Diff implementation
sebastian/environment                 4.2.3      Provides functionality to handle HHVM/PHP environ...
sebastian/exporter                    3.1.2      Provides the functionality to export PHP variable...
sebastian/global-state                3.0.0      Snapshotting of global state
sebastian/object-enumerator           3.0.3      Traverses array structures and object graphs to e...
sebastian/object-reflector            1.1.1      Allows reflection of object attributes, including...
sebastian/recursion-context           3.0.0      Provides functionality to recursively process PHP...
sebastian/resource-operations         2.0.1      Provides a list of PHP built-in functions that op...
sebastian/type                        1.1.3      Collection of value objects that represent the ty...
sebastian/version                     2.0.1      Library that helps with managing the version numb...
swiftmailer/swiftmailer               v6.2.3     Swiftmailer, free feature-rich PHP mailer
symfony/console                       v4.4.4     Symfony Console Component
symfony/css-selector                  v5.0.4     Symfony CssSelector Component
symfony/debug                         v4.4.4     Symfony Debug Component
symfony/error-handler                 v4.4.4     Symfony ErrorHandler Component
symfony/event-dispatcher              v4.4.4     Symfony EventDispatcher Component
symfony/event-dispatcher-contracts    v1.1.7     Generic abstractions related to dispatching event
symfony/finder                        v4.4.4     Symfony Finder Component
symfony/http-foundation               v4.4.4     Symfony HttpFoundation Component
symfony/http-kernel                   v4.4.4     Symfony HttpKernel Component
symfony/mime                          v5.0.4     A library to manipulate MIME messages
symfony/polyfill-ctype                v1.14.0    Symfony polyfill for ctype functions
symfony/polyfill-iconv                v1.14.0    Symfony polyfill for the Iconv extension
symfony/polyfill-intl-idn             v1.14.0    Symfony polyfill for intl's idn_to_ascii and idn_...
symfony/polyfill-mbstring             v1.14.0    Symfony polyfill for the Mbstring extension
symfony/polyfill-php72                v1.14.0    Symfony polyfill backporting some PHP 7.2+ featur...
symfony/polyfill-php73                v1.14.0    Symfony polyfill backporting some PHP 7.3+ featur...
symfony/process                       v4.4.4     Symfony Process Component
symfony/psr-http-message-bridge       v1.3.0     PSR HTTP message bridge
symfony/routing                       v4.4.4     Symfony Routing Component
symfony/service-contracts             v2.0.1     Generic abstractions related to writing services
symfony/translation                   v4.4.4     Symfony Translation Component
symfony/translation-contracts         v2.0.1     Generic abstractions related to translation
symfony/var-dumper                    v4.4.4     Symfony mechanism for exploring and dumping PHP v...
theseer/tokenizer                     1.1.3      A small library for converting tokenized PHP sour...
tijsverkoyen/css-to-inline-styles     2.2.2      CssToInlineStyles is a class that enables you to ...
vlucas/phpdotenv                      v3.6.0     Loads environment variables from `.env` to `geten...
webmozart/assert                      1.7.0      Assertions to validate method input/output with n...
zendframework/zend-diactoros          1.8.7      PSR HTTP Message implementations

User avatar
aimeos
Administrator
Posts: 7866
Joined: 01 Jan 1970, 00:00

Re: Import products with images from CSV

Post by aimeos » 20 Feb 2020, 11:23

The media/scale job isn't executed automatically at the end but the preview images are generated if
- there are no preview URLs in the CSV files
- and the media item is modified (most likely if you haven't deleted the entries before importing them again)

https://github.com/aimeos/ai-controller ... #L174-L180
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

Post Reply