Import Export

Questions around the TYPO3 integration and plugins as well as Flow and NeosCMS
Forum rules
Always add your TYPO3/Flow, Aimeos and PHP version as well as your environment (Linux/Mac/Win)
alpin3rocker
Posts: 28
Joined: 04 Mar 2015, 09:16

Import Export

Post by alpin3rocker » 04 Oct 2015, 11:08

Hello,

to import and update or export, in this case Products, i red about in the Forum and in the documentation. At this time cant figure out primary how to start and where i have to place the needed changes.

Is there out of the box a functional product import/export or should this be configured?
Should i start with an extension or is it in the most cases enough to place the changes in the Scheduler tasks in the TypoScript configuration?
like this:

Code: Select all

controller.jobs.product.export.location = http://www.mydomain.com/aiexport


I clicked the "Export" function in the Product area of the shop admin Interface and than i started a before created Scheduler Task "Product export" manually.
In the Shop Admin i can see that the Product export succeeded. Where is the exported file stored?
Is there a sample csv available with demo-products to import this manually?

Thank you!

User avatar
aimeos
Administrator
Posts: 3097
Joined: 01 Jan 1970, 00:00

Re: Import Export

Post by aimeos » 05 Oct 2015, 08:25

alpin3rocker wrote:Is there out of the box a functional product import/export or should this be configured? Is there a sample csv available with demo-products to import this manually?
For the product importer yes, there's a default format described here: https://aimeos.org/docs/Developers/Cont ... lt_mapping
Currently, there's no product export that you can import again, only the sitemap files can be exported.
alpin3rocker wrote: Should i start with an extension or is it in the most cases enough to place the changes in the Scheduler tasks in the TypoScript configuration?
like this:

Code: Select all

controller.jobs.product.export.location = http://www.mydomain.com/aiexport
You don't need an extension for this, just configure it via the TS in your scheduler task.
alpin3rocker wrote: I clicked the "Export" function in the Product area of the shop admin Interface and than i started a before created Scheduler Task "Product export" manually.
In the Shop Admin i can see that the Product export succeeded. Where is the exported file stored?
The import/export in the admin interface is not for products but only for product texts. You can find the generated files in the Overview tab (first one).

AndyH
Posts: 8
Joined: 25 Oct 2015, 19:50

Re: Import Export

Post by AndyH » 25 Oct 2015, 20:15

Hello aimeos team,

thanks alot for that great extension.

I'm relativly new to it and I'm trying to use it for a customer project. The shop is set up and I want to fill it with the existing products.

Like alpin3rocker I'm trying to figure out how to start the import of product texts.

I put this into TS config:

Code: Select all

plugin.tx_aimeos {
 settings {       
        controller.jobs.product.export.location = /home/.../web/typo3temp/aimeos/

        controller.jobs.product.import.csv {
            location = /home/.../web/typo3temp/aimeos/
            container.content = CSV
        }
}
I also created a simple CSV test file with one product, imported it in the "Product" tab and after a while in the "Overview" tab the message occured "Product text import: test-import.csv" with status green/check. When switching back to tab "Product" unfortunatelly no new entry can be found (and also not in table mshop_product).

So the following questions occured:
Thanks alot for your help!

Andreas

User avatar
aimeos
Administrator
Posts: 3097
Joined: 01 Jan 1970, 00:00

Re: Import Export

Post by aimeos » 27 Oct 2015, 10:21

AndyH wrote:

Code: Select all

plugin.tx_aimeos {
 settings {       
        controller.jobs.product.import.csv {
            location = /home/.../web/typo3temp/aimeos/
        }
}
For importing products from CSV you only need the "location" line and mapping that fits to your file.
AndyH wrote: I also created a simple CSV test file with one product, imported it in the "Product" tab and after a while in the "Overview" tab the message occured "Product text import: test-import.csv" with status green/check. When switching back to tab "Product" unfortunatelly no new entry can be found (and also not in table mshop_product).
This is a common mistake: The import in the admin interface is only for texts, i.e. product texts including those for different languages. The CSV import must be done via the appropriate scheduler task.
AndyH wrote: [*] Are there some minimum requirements for the CSV like the miniumum amount of columns (required columns) beside the ones shown here https://aimeos.org/docs/Developers/Cont ... and_format
The only requirement is that the each line in the CSV file contains a product code. All other data is optional resp. only required in combination with other columns.
AndyH wrote: [*] Is there an example CSV file which can be downloaded?
I've added a link to an example file at the end of the default mapping section: https://aimeos.org/docs/Developers/Cont ... lt_mapping
AndyH wrote: [*] Where do I have to place the configuration array with the mapping codes shown here https://aimeos.org/docs/Developers/Cont ... lt_mapping
You have to put it into the TSConfig field of the scheduler task.
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

AndyH
Posts: 8
Joined: 25 Oct 2015, 19:50

Re: Import Export

Post by AndyH » 15 Nov 2015, 18:42

Hello Aimeos admin,

thanks alot for your fast reply. Thank you also for the example csv. This helps me now. :-)

After I added your csv file and changed the TS config, I tried to run the scheduler with the current logged into be user manually. This failed and created some pseudo products with some php code. So I tried to start the import with the "_cli_scheduler" be user from the bash but I was running into several problems here (which is btw. not a fault of Aimeos but of my system configuration). If someone is having the same problems here are some tips:
  • start your scheduler with the command "php typo3/cli_dispatch.phpsh scheduler" from the bash in your document root of your website (don't jump into the typo3 folder because of the symlinks. The script will assume a wrong path for typo3conf folder). If you're using application contexts in your AdditionalConfiguration.php than you have to use "TYPO3_CONTEXT=Development php typo3/cli_dispatch.phpsh scheduler".
  • Check if you have mysqli also running for the php5-cli. To find out see the phpinfo by typing "php -i"
    • (php5-cli uses a different php.ini as your webserver. On ubuntu/linux you can find the config file with "php -i |grep php\.ini")
    • In /etc/php5/cli/conf.d/20-mysqli.ini make sure that extension=mysqli.so is not commented out (;)
After I got the scheduler working (Last Execution of scheduled task shows now "... (Cron)" in the backend) I got some errors with aimeos which I finally figured out.

At the beginning I got an "Execution failed: 0, Invalid product lines in ".": 316/468" Later I had a "Integrity constraint violation: 1048 Column 'currencyid' cannot be null" and I had to add the currencyid to the mapping. It took me a while that I've to put the following complete code into the TypoScript configuration of scheduler task.

Code: Select all

plugin.tx_aimeos {
    settings {
        controller.jobs.product.import.csv {
            location = /home/.../website_3/web/typo3temp/aimeos/
            mapping {
				item {
					0 = product.code
					1 = product.label
					2 = product.type
					3 = product.status
				}
				text {
					4 = text.type
					5 = text.content
					6 = text.type
					7 = text.content
				}
				media {
					8 = media.url
				}
				price {
					9 = price.quantity
					10 = price.value
					11 = price.taxrate
					12 = price.currencyid
				}
				attribute {
					13 = attribute.type
					14 = attribute.code
				}
				product {
					15 = product.code
					16 = product.list.type
				}
				property {
					17 = product.property.value
					18 = product.property.type
				}
				catalog {
					19 = catalog.code
					20 = catalog.list.type
				}
            }
        }
	}
}
Is there a way to include the TypoScript code in the scheduler task from a file like

Code: Select all

<INCLUDE_TYPOSCRIPT: source="FILE:EXT:myaimeosext/Configuration/TypoScript/ext/aimeos/scheduler.ts">
so that it can be under version control? I did not find a way to get it working.

Best,
Andreas

User avatar
aimeos
Administrator
Posts: 3097
Joined: 01 Jan 1970, 00:00

Re: Import Export

Post by aimeos » 15 Nov 2015, 22:15

AndyH wrote: After I added your csv file and changed the TS config, I tried to run the scheduler with the current logged into be user manually. This failed and created some pseudo products with some php code.
This is the case if the directory is wrong and points to the ./typo3/ directory.
AndyH wrote: Is there a way to include the TypoScript code in the scheduler task from a file like

Code: Select all

<INCLUDE_TYPOSCRIPT: source="FILE:EXT:myaimeosext/Configuration/TypoScript/ext/aimeos/scheduler.ts">
so that it can be under version control? I did not find a way to get it working.
I don't think that will work but you can add the settings to a configuration file in your own Aimeos extension.
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

AndyH
Posts: 8
Joined: 25 Oct 2015, 19:50

Re: Import Export

Post by AndyH » 30 Nov 2015, 08:17

Hello,

and thank's again for your reply and especially for your patience.

I now have propably my last question on the product import which is related to the media section in the import configuration.

Each of my products have an image which will be used language independand and one image which is allways language specific. It works for the language specific part but the images which should be shown in all language versions get also the languageuid fo the following media element.

Code: Select all

media {
	# media (all)
	41 = media.url
	# media language specific (de)
	42 = media.languageid
	43 = media.url
	# media language specific (en)
	44 = media.languageid
	45 = media.url
}
Is there a way to set the languageid to "All" at the import?

Many thanks,
Andreas

User avatar
aimeos
Administrator
Posts: 3097
Joined: 01 Jan 1970, 00:00

Re: Import Export

Post by aimeos » 30 Nov 2015, 09:55

The language ID wasn't set explicitly up to now and the existing items are reused so the language ID of that item will remain if not available in your CSV file. Unfortunately, adding an empty column won't work too because it must be a NULL value.

The behavior is now fixed in dev-master and will be part of the next release: https://github.com/aimeos/aimeos-core/c ... b8c528a445
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

AndyH
Posts: 8
Joined: 25 Oct 2015, 19:50

Re: Import Export

Post by AndyH » 30 Nov 2015, 15:35

Awesome!

It work's great.

AndyH
Posts: 8
Joined: 25 Oct 2015, 19:50

Re: Import Export

Post by AndyH » 05 Dec 2015, 13:54

Hello aimeos team,

thanks to your help all imported products are now available in the backend and I'm close to have the products shown on the website automatically.

Currently one (hopefully) last problem occures. I figured out that a product is shown in the frontend when I open the product in the shop backend and click "save". Then an entry in the table mshop_catalog_index_catalog was created. If I create such an entry manually and flush the T3 caches the product is also shown on the website.

The question now is how can this be created automatically during the import?

The following columns are part of the import. Did I missed a column?

Code: Select all

catalog {
    79 = catalog.code
    80 = catalog.list.type
}
Best,
Andreas

Post Reply