Import CSV with HTTP:// images
Forum rules
Always add your TYPO3, Aimeos and PHP version as well as your environment (Linux/Mac/Win)
Spam and unrelated posts will be removed immediately!
Always add your TYPO3, Aimeos and PHP version as well as your environment (Linux/Mac/Win)
Spam and unrelated posts will be removed immediately!
-
- Posts: 4
- Joined: 01 Jun 2021, 12:03
Import CSV with HTTP:// images
Hello,
i am currently evaluating aimeos for a complex project.
My Setup:
* TYPO3 v10.4.16
* aimeos/aimeos-typo3 21.4.2
* Dockerized Dev Env
* PHP 7.4.19
* MySQL 5.7.34
I am trying to import products from a CSV file, which works just fine.
The images, however, are only provided via HTTP Url, like "https://cdn.example.com/images/products ... 31dd97ce51" (this is a random URL, just to show an example). The URL returns an image and the response header contains the correct mime type.
I know external URLs are supported, because the demo shop's images are HTTP Urls and i can set them in the database.
My Import config:
First of all: The import fails when i add "17 = media.url". I debugged the code and found an exception thrown in vendor/aimeos/aimeos-core/lib/mwlib/src/MW/Filesystem/Standard.php@148
The message is pretty clear. aimeos prefixes the external url with the local base "/project/app/private/uploads/tx_aimeos/" because it thinks it is a relative file path.
First of all, i think the scheduler task should output some errors when an exception was thrown. This would help to discover and investigate import errors.
My actual question:
How can i configure the importer to accept the HTTP URL as is and to treat it as such? (Just save the URL, do not download the file to the local file system)
Additionally: how can i configure the importer to fetch the mime type from the response headers?
i am currently evaluating aimeos for a complex project.
My Setup:
* TYPO3 v10.4.16
* aimeos/aimeos-typo3 21.4.2
* Dockerized Dev Env
* PHP 7.4.19
* MySQL 5.7.34
I am trying to import products from a CSV file, which works just fine.
The images, however, are only provided via HTTP Url, like "https://cdn.example.com/images/products ... 31dd97ce51" (this is a random URL, just to show an example). The URL returns an image and the response header contains the correct mime type.
I know external URLs are supported, because the demo shop's images are HTTP Urls and i can set them in the database.
My Import config:
Code: Select all
controller {
jobs {
product {
import {
csv {
skip-lines = 1
location = fileadmin/shop-import/
mapping {
item {
0 = product.label
3 = product.code
}
text {
4 = text.content
}
media {
17 = media.url
}
price {
18 = price.currencyid
10 = price.quantity
6 = price.value
7 = price.taxrate
}
supplier {
2 = supplier.code
}
}
}
}
}
}
}
Code: Select all
Couldn't get file mtime for "/project/app/private/uploads/tx_aimeos/https://cdn.example.com/images/products?id=fe9cb6ee-b8bd-4440-82a5-df31dd97ce51"
First of all, i think the scheduler task should output some errors when an exception was thrown. This would help to discover and investigate import errors.
My actual question:
How can i configure the importer to accept the HTTP URL as is and to treat it as such? (Just save the URL, do not download the file to the local file system)
Additionally: how can i configure the importer to fetch the mime type from the response headers?
Re: Import CSV with HTTP:// images
Can you investigate, why the has() method returns TRUE for the URL. Here are the relevant lines:VerteXVaaR wrote: ↑01 Jun 2021, 12:56 How can i configure the importer to accept the HTTP URL as is and to treat it as such? (Just save the URL, do not download the file to the local file system)
- https://github.com/aimeos/ai-controller ... .php#L228
- https://github.com/aimeos/aimeos-core/b ... #L182-L185
- https://github.com/aimeos/aimeos-core/b ... #L370-L379
That's not necessary because the mimetype of the generated preview images is used to set that field automatically:VerteXVaaR wrote: ↑01 Jun 2021, 12:56 Additionally: how can i configure the importer to fetch the mime type from the response headers?
https://github.com/aimeos/aimeos-core/b ... #L162-L206
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, give us a star
If you like Aimeos, give us a star
-
- Posts: 4
- Joined: 01 Jun 2021, 12:03
Re: Import CSV with HTTP:// images
Can you investigate, why the has() method returns TRUE for the URL. Here are the relevant lines:
Code: Select all
$fs->has( $url )
Code: Select all
$refItem->getPreviews() === []
I changed the code to make it work, but since my first contact with aimeos was yesterday, i'm not quite sure about the code style and contribution workflow.
Code: Select all
} elseif( $fs->has( $url )
&& (
$refItem->getPreviews() === []
|| $refItem->getUrl() !== $url
|| !( $fs instanceof \Aimeos\MW\Filesystem\MetaIface )
|| date( 'Y-m-d H:i:s', $fs->time( $url ) ) > $refItem->getTimeModified()
)
) {
$refItem = \Aimeos\Controller\Common\Media\Factory::create( $context )->scale( $refItem->setUrl( $url ) );
} elseif( 0 === strpos( $url, 'http://' ) || 0 === strpos( $url, 'https://' ) ) {
$refItem->setUrl( $url );
}
Re: Import CSV with HTTP:// images
Just create a pull request at the htttps://github.com/aimeos/ai-controller-jobs repository. It's easier to care about coding style there.
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, give us a star
If you like Aimeos, give us a star
-
- Posts: 4
- Joined: 01 Jun 2021, 12:03
Re: Import CSV with HTTP:// images
Here you go: https://github.com/aimeos/ai-controller-jobs/pull/30
This topic can be closed, as my question is the result of a bug and a PR is submitted.
This topic can be closed, as my question is the result of a bug and a PR is submitted.
Re: Import CSV with HTTP:// images
In Aimeos, there are objects available for dealing with CSV files:
https://aimeos.org/docs/latest/infrastr ... ite-files/
For saving/fetching data, use the appropriate manager:
https://aimeos.org/docs/latest/models/managing-items/
https://aimeos.org/docs/latest/infrastr ... ite-files/
For saving/fetching data, use the appropriate manager:
https://aimeos.org/docs/latest/models/managing-items/
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, give us a star
If you like Aimeos, give us a star