What is an e-commerce framework?

The term e-commerce framework is related to software frameworks for e-commerce applications. They offer an environment for building e-commerce applications quickly.

E-Commerce frameworks are flexible enough to adapt them to your specific requirements. As result, they are suitable for building virtually all kinds of online shops and e-commerce related (web) applications like the Aimeos e-commerce framework does.

An e-commerce framework must

  • allow replacing all parts of the framework code
  • forbid changes in the framework code itself
  • contain bootstrap code to start the application
  • be extensible by user-written code

E-Commerce frameworks should

  • define the general program flow
  • consist of reusable components
  • be organized in functional domains

Examples of e-commerce frameworks are

They provide an overall structure for e-commerce related applications. Furthermore, they implement the general program flow e.g. how the checkout process works. Contrary to monolithic shop systems, existing program flow can not only be extended but completely changed according to your needs.

Evolution of e-commerce systems

Since the beginning of (internet) e-commerce around 1995, a lot has changed on the technology side. The first generation of e-commerce systems evolved from existing ERP and related systems. This was followed by the 2. generation of standalone shop systems between 2004 and 2008. E-commerce frameworks are the latest generation of e-commerce systems and started around 2012.

Hybris, the shop system owned by SAP is one of the representatives of the 1. generation. It’s strongly connected to the SAP ERP system and Hybris is mainly a shop front-end for SAP. Customer relationship (CRM) and content management (CMS) tools are available in the ERP system but very limited.

The Magento shop system represents the 2. generation of standalone e-commerce systems. They usually contain CRM and CMS and some other functionality but also only at a very basic level. They might be enough for the smallest shops but are unusable for shop owners who run for real profits.

Aimeos is one of the few real e-commerce frameworks that are currently available. These 3rd generation systems excel in their own domain: Present and sell products. For all other e-commerce related tasks, they connect to specialized systems and exchange data in both ways. Thus, shop owners can choose the best systems for their needs.

E-Commerce framework architecture

E-commerce frameworks must be based on a strong architectural model. Usually, they make heavy use of interfaces and design patterns like

  • Dependency Injection (make components independent of used object implementation)
  • Factories (create objects at a central place that instantiates the actual implementation)
  • Decorators (dynamically add functionality to existing objects)
  • Publish/Subscribe model (notify listening objects about changes instead of polling for updates)

A “design pattern” is re-usable solution that solves similar software design problems in an elegant way. They require programming language templates which enforce public class methods and their signatures called “interfaces”.

The basic requirement are independent, side-effect free components that form the building blocks. One or more components care about the functionality of a business “domain”. Such a domain can be the HTML front-end, a JSON REST API and the administration interface.

The vertical separation of code into business domains enables scaling out applications by deploying the application as a bunch of micro services.

Micro services are loosely coupled applications that offer their service over lightweight protocols like HTTP and JSON. They are easier to extend, maintain and especially deploy than monolithic applications.

More about Aimeos

Was ist ein E-Commerce Framework?

Der Term E-Commerce Framework bezieht sich auf Software-Frameworks für E-Commerce Applikationen. Sie ermöglichen es in relativ kurzer Zeit komplette E-Commerce Applikationen zu bauen.

E-Commerce Frameworks sind flexibel genug um sie an kundenspezifische Anforderungen anzupassen. Dadurch können sie als Basis für praktisch alle Arten von Onlineshops und E-Commerce bezogene (Web-)Applikationen dienen, wie beispielsweise das Aimeos E-Commerce Framework.

Ein E-Commerce Framework muss

  • es erlauben, jeden seiner Bestandteile zu ersetzen
  • Änderungen am Framework-Code selbst verbieten
  • Bootstrap-Code beinhalten um die Applikation zu starten
  • erweiterbar für kundenspezifischen Code sein

E-Commerce Frameworks sollen

  • den allgemeinen Programmablauf definieren
  • aus wiederverwendbaren Komponenten bestehen
  • in funktionalen Domänen unterteilt sein

Beispiele für E-Commerce Frameworks sind

Sie bieten eine komplette Infrastruktur für E-Commerce bezogene Applikationen. Weiterhin implementieren sie den allgemeinen Programmfluss, z.B. wie der Checkout-Prozess funktioniert. Im Gegensatz zu monolithischen Shopsystemen kann der bestehende Programmfluss nicht nur erweitert werden, sondern auch komplett für die eigenen Bedürfnisse geändert werden.

Evolution von E-Commerce Systemen

Seit dem Beginn des E-Commerce im Internet um 1995 hat sich vieles auf Seiten der Technologie verändert. Die erste Generation von E-Commerce Systemen entwickelte sich aus existierenden Warenwirtschaftsprogrammen und ähnlichen Systemen. Daraufhin folgte die zweite Generation von eigenständigen Shopsystemen zwischen 2004 und 2008. E-Commerce Frameworks sind die neuste Generation von E-Commerce Systemen und begannen ab 2012 auf den Markt zu kommen.

Hybris, das Shopsystem von SAP ist einer der Vertreter der ersten Generation. Es ist eng mit dem SAP ERP System verzahnt und Hybris ist hauptsächlich eine Shopoberfläche für SAP. Customer Relationship (CRM) und Content Management (CMS) Werkzeuge sind im Umfeld von SAP verfügbar aber mit eingeschränkter Funktionalität im Gegensatz zu spezialisierten Systemen.

Das Magento Shopsystem repräsentiert die zweite Generation von eigenständigen E-Commerce Systemen. Sie beinhalten CRM und CMS Komponenten sowie ein paar weitere Funktionalitäten, aber ebenfalls nur mit einem beschränkten Funktionsumfang. Sie können für Kleinstunternehmen ausreichend sein, für Shopsbetreiber mit größeren Umsätzen aber meistens unbrauchbar.

Aimeos ist eines der wenigen echten E-Commerce Frameworks die im Moment verfügbar sind. Diese Systeme der dritten Generation glänzen in ihrer Kerndomäne: Der Präsentation und dem Verkauf der Produkte. Für alle weiteren E-Commerce bezogenen Aufgaben verbinden sie sich mit spezialisierten Systemen und tauschen Daten in beide Richtungen aus. Dadurch können Shopbetreiber das jeweils beste System für ihre Bedürfnisse wählen.

E-Commerce Framework Architektur

Gute E-Commerce Frameworks müssen einer stringenten Architektur folgen. Dafür verwenden sie ausgiebig Interfaces und Design Patterns wie

  • Dependency Injection (Komponenten sind unabhängig von der konkreten Objektimplementierung)
  • Factories (erzeugen zentral Objekte und instanzieren die tatsächliche Implementierung)
  • Decorators (fügen dynamisch zusätzliche Funktionalität zu existierenden Objekten hinzu)
  • Publish/Subscribe (benachrichtigen registrierte Objekte über Änderungen anstatt wiederholt auf Änderungen zu testen)

Ein “Design Pattern” ist eine wiederverwendbare Lösung, die ähnliche Softwaredesign-Probleme elegant löst. Sie benötigen Schablonen names “Interfaces” in den Programmiersprachen, die öffentliche Klassenmethoden und ihre Signaturen erzwingen können.

Die wichtigste Anforderung sind unabhängige Bausteine ohne Seiteneffekte, die zu Komponenten zusammen gefügt werden können. Ein oder mehrere Bausteine sind für die Funktionalität einer Fachdomäne zuständig. Eine Domäne kann eine HTML Oberfläche, eine JSON REST API oder eine Administrationsoberfläche sein.

Die vertikale Trennung von Code in Fachdomänen ermöglicht es Applikationen beliebig zu skalieren, indem sie als Microservices auf mehreren Serverinstanzen laufen können.

Microservices sind lose gekoppelte Applikationen, die ihre Dienste über leichtgewichtige Protokolle wie HTTP oder JSON anbieten. Sie sind leichter zu erweitern, zu warten und auszurollen als monolithische Applikationen.

Mehr über Aimeos

Aimeos 2019.04 stable release

 

The stable 2019.04 release contains some major improvements:

  • Improved core APIs and rewritten frontend controllers
  • XML importer for attributes, categories, customers, products and suppliers
  • Increased performance for complex queries
  • More SEO friendly URLs

Improved APIs

The Aimeos core already offered an easy to learn API for working with managers, items and providers. All methods are now supporting fluent interfaces if possible so method calls can be chained like:

$item = $manager->createItem()->fromArray( $map )
    ->setType( 'default' )->addPropertyItem( $propItem );

Additionally, the frontend controller API for working with the shop objects efficiently in the HTML clients and the JSON REST API has been completely overhauled. All objects contain simple, fluent methods that can be concatenated like in the core library. Implementing your own product list in the frontend is now only a matter of a few easy to understand calls:

$products = \Aimeos\Controller\Frontend::create( $context, 'product' )
   ->uses( ['text', 'price', 'media' )->category( 123 )->text( 'sneaker' )
   ->sort( 'name' )->slice( 0, 48 )->search();

This statement would retrieve the first 48 products with texts, prices and images that are in category “123” and contain the text “sneaker” sorted by their names. In Laravel, facades simplify working with the frontend controller even more:

$products = Product::uses( ['text', 'price', 'media' )->category( 123 )
   ->text( 'sneaker' )->sort( 'name' )->slice( 0, 48 )->search();

XML importer

It’s now possible to import attributes, categories, customers, products and suppliers using Aimeos XML files. They are as flexible as the Aimeos data structures so you can import all items in a document oriented way. A simple example for a product XML import file would be:

<products>
    <product ref="test-article">
       <product.type>event</product.type>
       <product.code>test-article</product.code>
       <product.label>Test event</product.label>
       <product.datestart>2000-01-01T10:00:00</product.datestart>
       <lists>
          <text><textitem>...</textitem></text>
          <price><priceitem>...</priceitem></price>
          <media><mediaitem>...</mediaitem></media>
       </lists>
    </product>
</products>

The product XML export job produces the same XML format so you can share data between different Aimeos instances too.

Performance

Aimeos has gotten faster once more. Complex queries which ask for entries that reference items from other domains or have properties of a specific value are now lightning fast in all DBMS when using the *:has and *:prop search functions. To fetch all products which reference a specific attribute use:

$search = $manager->createSearch();
$func = $search->createFunction( 'product:has', ['attribute', 'default', 123 );
$search->setConditions( $search->combine( '!=', $func, null ) );
$products = $manager->searchItems();

To get notfied about queries which take longer than one second, a monitoring similar to the MySQL slowlog is now active. Furthermore, using utf8mb4 as charset in MySQL doesn’t have a performance drawback any more so it’s now used by default. Existing setups are migrated towards this charset if possible (requires MySQL >= 5.7.8).

SEO friendly URLs

In the past, Aimeos required the product ID to be part of the URL. Due to the simplifed index for product texts which contains the product URL name and due to optimized routes, the URLs of all detail pages can now look like

https://shop.com/My_cool_product

The product name in the URL can be localized for each language of corse.

#gigacommerce – Aimeos shop performance with 1 billion items

 

Aimeos has recently proven to be capable of handling one billion articles and more in an online shop using the #gigacommerce extension. This is far more than the number of articles at the Amazon market place which is currently estimated by around 562 million items. But the sheer numbers doesn’t say anything about the performance so how fast can Aimeos delivery content when users browsing the shop pages?

The setup

For a realistic test Aimeos has been set up in a standard cloud hosting environent sponsored by Profihost, an e-commerce hoster from Hannover, Germany. They provided two of their “Flexserver Expert” packages, which are virtual machines running on a cluster with Intel Xeon Skylake 2,6 GHz CPUs.

Hardware

  • LAMP VM: 8 Core, 16GB RAM, 17GB on SSD

  • ElasticSearch VM: 8 Core, 32GB RAM, 465GB on SSD

Software

  • Debian Linux, Apache 2.4, MySQL 5.7, PHP 7.2, ElasticSearch 5.4

  • Laravel 5.7, Symfony 4.1, TYPO3 8.7

  • Aimeos 2018.10, ai-elastic extension 2018.10

Test procedure

All tests have been done with Loader.io, a SaaS provider for load tests. The latency between the clients and the server is about 32ms, which is included in the response times. A typical page request of a detail page therefore looks as follows:

ACTION TIME
Stalled 5.26 ms
Request sent 0.12 ms
Waiting (TTFB) 50.74 ms
Content Download 18.33 ms
Sum 75.28 ms

The response time therefore consists of the network latency (32ms), the processing on the server (50.7ms – 32ms = 18.7ms) and the transmission time of the HTML page (18.3ms). The network latency is the largest influencing factor here and can be reduced only with difficulty in real operation. In general it is lower the closer the server is geographically to the customer and the better the connection of the hoster is.

The Aimeos list and detail page were tested, both with cache and without. The cache does NOT contain the complete page (“full page cache”) as in other shop systems, but only page fragments such as the product list or the product details. The page frame is provided by Laravel, Symfony and TYPO3, in which the fragments are inserted before delivering the page to the client.

Test results

The tests showed that Laravel 5.7 and Symfony 4.1 are about the same speed. In the evaluations of Loader.io, there were no significant differences between the two and therefore the results were summarized. The only measurable difference is about 5ms, which Symfony 4.1 can generate on an unloaded system faster than Laravel 5.7. Under load there is no more difference.

With TYPO3 as frontend, the server can process fewer requests per second, but TYPO3 is not directly comparable to Laravel/Symfony. As a complete content management system, TYPO3 allows the maintenance of pages by editors, which is not easily possible with Laravel/Symfony.

Conclusion

With over 600 PI/second and a processing time on the server less than 20ms, the Aimeos E-Commerce System is certainly one of the fastest shop systems. The differences between list and detail view are mainly due to the different page size (189KB to 66KB) and the resulting longer time for data transfer.

Competitors can only achieve similar results by storing complete pages in a Varnish cache and loading shopping carts and other dynamic content through AJAX queries. However, Aimeos is so fast that this would make response times even worse, because the separate query for e.g. the shopping cart is not necessary at all with Aimeos.

Even without cache the number of 130-150 PI/second and the response times of 162ms incl. network latency (130ms without latency) are excellent values compared to other shop systems, especially since the system can scale up to 1 billion and more items. This makes Aimeos particularly suitable for high-load scenarios such as TV advertising.

More about #gigacommerce: https://aimeos.com/gigacommerce

Aimeos 2019.01 major release

 

The new beta version contains a lot of improvements and focuses on:

  • Even more performance
  • Developer experience
  • Code cleanup

Performance improvements

Aimeos is already known as ultra fast e-commerce solution for Laravel, Symfony, SlimPHP, TYPO3 and Flow/NeosCMS even for complex requirements. This release pushes the limits teared down by Aimeos #gigacommerce further with render times of 20ms and below if content caching is enabled. Without caching, response times are exceptional too compared to other shop systems and in the 2019.01 release, we reduced that time further.

First, we denormalized tables that use types, so the type code is now stored directly instead of the type ID. This saves several database queries per request. Additionally, it’s possible to retrieve related data for e.g. products filtered by type:

$manager->searchItems( $filter, ['text', 'price', 'product' => ['suggestion'] );

This fetches related products of type “suggestion” only instead of all related products. Especially for shops with many relations, this can be a huge performance improvement if components can’t be cached.

PHP 7.3 promises another huge performance improvement and we are testing that version every time. Unfortunately, there’s a bug in PHP 7.3 with prevents production usage yet. It seems that the PHP developers over-optimized opcode caching so PHP 7.3 produces wrong results: https://bugs.php.net/bug.php?id=77310

Simplifications for developers

One of the major goals for this Aimeos release was to simplify interfaces so the first steps for new developers are easier. As a result, creating managers, controllers and clients requires only:

$manager = \Aimeos\MShop::create( $context, 'product' );
$cntl = \Aimeos\Controller\Frontend::create( $context, 'catalog' );
$client = \Aimeos\Client\Html::create( $context, 'catalog/lists' );

Items with relations are fetched recursively now and a decorator limits the recursion depth to two levels. The allowed depth is configurable like everything else in Aimeos. This allows you to get all releated data you need at one without fetching and merging data from several calls to searchItems():

$products = $manager->searchItems( $filter, ['product', 'price', 'text] );

For bundle and selection products, the items in the result list contains the bundled and variant articles including prices, texts and related products.

Also, all newly created items have the status “enabled” by default and must be explicitly disabled if required. This saves many calls to “setStatus()” especially when implementing a new importer. The frontend controllers allow multiple category IDs for filtering so it’s possible to search for
products in multiple categories with one method call.

Code cleanups

Finally, a lot of old code has been removed due to the switch to document oriented managment of items. Especially in the index managers and the filter criteria implementation was some code left after the changes required to use ElasticSearch natively.

The 2019.x releases are the last ones that will support PHP 5 as 5.6 isn’t maintained any more since this year. You should use PHP 7 nevertheless to get the best performance.

For a full list of backward incompatible changes, please have a look at the Wiki article. BC breaking changes in 2019.01 are listed up to 2018-12-31:

https://aimeos.org/docs/Developers/Changelog#Version_2019.x

 

#gigacommerce – Scalable online shops with 1 billion items and more

 

Does an online shop with 1 billion items work on a standard cloud server? And that even with the demand for very low response times? We investigated this question during the #gigacommerce project.

The #gigacommerce setup contains 1 billion articles, which are combined into 10 million products. Each item contains attributes, images, prices and text like in any online store. The basis is a Symfony, Laravel and TYPO3 installation with the Aimeos E-Commerce extension, which access an ElasticSearch server.

All three frontends have been extensively tested to find out where the limits are. The Aimeos components showed that they scale almost infinitely, with response times of around 100 milliseconds. Interestingly, the only absolute limit in the tests was not the available hardware, but the network bandwidth, which was used up at over 600 PI/second. This makes it particularly interesting for peak load situations and TV advertising.

How it this possible?

The special feature of the Aimeos e-commerce system is its high flexibility. Data can be stored in different backends, which use the appropriate technology depending on the requirements – like here ElasticSearch for the product data. This makes the system ultra fast and scalable. Nevertheless, Aimeos is not a special solution, but comparable with other major players in the industry.

The open source software Aimeos can be adapted to the most diverse needs, in particular to complex B2B requirements and delivers everything that is important for an online shop: product catalogs, shopping baskets, payment integration, order management, e-mails and all optimized for mobile devices. Aimeos is available for TYPO3, Laravel, Symfony, SlimPHP and Flow/Neos but the software can be integrated into any PHP application.

More about Aimeos #gigacommerce

Aimeos on Content Commerce Summit

Content Commerce Summit

Content commerce, the fusion of editorial content and e-commerce, is a growing area gaining more and more importance for businesses.

We have been on the first European Content Commerce Summit in Leipzig (Germany) for two days spotting the latest trends and and held a session about Aimeos as solution for integrating content and e-commerce to interested participants.

The offered sessions had a wide range of topics, from the very interesting talk from Danny Nauth about the secrets of successful content based on the experience of neural marketing, over content along the customer journey by Kai Wermer and the importance of the right content for SEO by Markus Hoevener to tops and flops in content commerce projects by Hannes Richter from Projecter.

Roman Zenner had some bad luck with the beamer but used that problem as a smart excuse for having no presentation at all 😉

To sum up: A really cool event for everyone interested in all aspects of content and e-commerce!

Modern PHP applications up to 110x slower on network storage

performance-loss-small

Modern PHP applications consists of thousands of PHP files. Even if your own set of files is small, most of the functionality is nowadays provided by 3rd party libraries. Especially composer made it simpler than never before to integrate external libraries and full stack frameworks like Laravel or Symfony contains a lot of them. The Aimeos e-commerce components are not different in this case since this kind of code reuse is the best thing since sliced bread 🙂

Bad old days and brave new world

The downside is that a single request have to touch a lot of PHP files, sometimes up to several hundred. In the “bad old days”, each file must be read from the disk, parsed into byte code and executed by the PHP interpreter if byte code caches like APC wasn’t installed. That took a lot of time and PHP applications could require up to several seconds to finish a request.

Since PHP 5.5, performance improved very much, mostly by the integrated OPcode cache, since PHP7 also by various optimizations inside the PHP interpreter itself. Now, request times of even big PHP applications can be only several 100ms or below. PHP applications can be almost as fast as Java one. Great stuff!

When things turn wrong

There’s only one problem PHP developers and administrators don’t really think about: The applications are only fast as long as the file system the files are stored is fast too! We’re not talking about the sequential read or write performance, which can be fast on network file systems too. It’s the time required for file access that matters if your application consists of several thousand files!

OPcache

Why the file access you might wondering? The OPcache keeps the files in memory so there’s no file access at all? Wrong! Even if you configured the available memory of the OPcache to be big enough for all files, it checks for file modifications regularly. By default, this check is executed every two seconds. A better value would be 60 seconds or more:

opcache.revalidate_freq = 60

Things will get even worse. For every part of the file path, the meta data is retrieved by the file system. The path “/var/www/html/index.php” will result in requests for “/var”, “/var/wwww”, “/var/www/html” and “/var/www/html/index.php”. You see, this can result in a lot of request for hundreds of files stored deep down in a directory tree.

Real path cache

PHP has a feature called “real path cache” that should mitigate the problem. The thought about the this cache is to reduce the amount of file system requests by storing the absolute path of the requested files. So far so good but it’s value is 16KB by default and this is ridiculously low for modern PHP applications. This results in a constant trashing of the cached entries and renders the cache useless with this size. To avoid this, you should start and test with values of 128KB and more:

realpath_cache_size = 128KB

The network stoage disaster

So, where’s the 110x performance loss? Well, retrieving file meta data is fast when it’s already in the cache of the operating system and don’t have to be revalidated. Then, it doesn’t matter much if the OPcache and real path cache settings are misconfigured.

For our tests, we’ve used the Aimeos Core plus the required libraries and extensions (ca. 6500 files and directories) as reference and executed the following command in the Aimeos directory:

# time (find . &gt; /dev/null)

Ext4

Let’s test the meta data performance of an Ext4 file system on a local hard disk.

First test:

real    0m2.722s
user    0m0.017s
sys     0m0.130s

Subsequent tests:

real    0m0.043s
user    0m0.003s
sys     0m0.040s

The first time retrieving the meta data of all files and directories from the hard disk lasted 2.722 seconds. If the data is cached by the operating system, only 0.043 seconds are needed.

NFS

Now we did the same for NFS, the standard Unix network file system:

First test:

real    0m4.197s
user    0m0.032s
sys     0m0.188s

Subsequent tests:

real    0m0.490s
user    0m0.019s
sys     0m0.080s

The first time, it takes almost 1.5x as long than from a local hard disk and afterwards it needs even 10x longer. Thus, performance of PHP applications over NFS will severely suffer!

GlusterFS

Have a look at GlusterFS, another network file system:

First test:

real    0m9.031s
user    0m0.045s
sys     0m0.176s

Subsequent tests:

real    0m4.769s
user    0m0.052s
sys     0m0.156s

Not only that the first time it took more then three times longer than from a local hard disk, the subsequent test have shown a more than 110x worse performance compared to the local hard disk. You can imagine that this will lead to response times of several seconds!

Conclusions

The test results made clear that storing the files of a PHP application on a network storage is really a bad idea. It will make more than a noticeable difference and results in response times which are extremely bad when compared to applications stored on local hard disks.

If you have no other chance than using a network storage, you should at least optimize and test the OPcache and real path cache settings to mitigate the problem. If you use optimal settings, you can decrease the request time penalty to only 2-4 times of a local disk. But it will make a big difference if your customers wait 0.5 or up to 4 seconds!

Posted in PHP

How to use the TYPO3 caching framework with cache groups

The TYPO3 caching framework is a great way to speed up your site if your extension needs to perform tasks that can be cached. In the Aimeos TYPO3 extension, we use it to store content like list or detail views of the web shop to get lightning fast response times. Since version 6.2, a cache can belong to one or more cache groups in the caching framework and it enables editors to control more precisely which caches should be flushed. Unfortunately, this features isn’t well documented yet in the TYPO3 documentation but here’s the remedy!

What are cache groups

In the TYPO3 caching framework, there are currently three types of caches:

  • pages : Front-end related caches
  • system : Low-level caches that shouldn’t normally be flushed
  • all : All other caches that don’t fall into one of the two categories before

Based on the cache type (a.k.a cache group) the caches are flushed on different occasions. This makes a difference for the editors because depending on what they have changed, most often one of the cache groups will be flushed as well. Also, there are at least two options available when editors decide to flush the caches manually:

Aimeos-typo3-cache7The first option will only flush front-end related caches while the second one flushes all caches that are not in the “system” or “pages” cache group. Flushing the “system” caches isn’t that easy any more but those don’t need to be cleared at all by editors. Instead, they are automatically flushed if e.g. a new extension is installed.

Therefore, it makes sense to tell TYPO3 which cache group your cache should be in if you use the caching framework to store data temporarily. This allows at least editors to selectively flush caches instead of the famous “flush all caches to be sure”! More information about which cache in in which cache group can be found in the article about the TYPO3 caching architecture.

Set up your extension cache

To create your own cache in your TYPO3 extension, there are only a few lines in the ext_localconf.php of your extension necessary. You can copy and paste these lines and only have to replace the aimeos string by the name of your own TYPO3 extension:

if( !is_array( $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos'] = array();
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['frontend'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['frontend'] = 'TYPO3\\CMS\\Core\\Cache\\Frontend\\StringFrontend';
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['backend'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\DatabaseBackend';
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['options'] ) ) {
	$TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['options'] = array( 'defaultLifetime' =&gt; 0 );
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['groups'] ) ) {
	$TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['groups'] = array( 'pages' );
}

The whole if/is_array/isset thing is necessary to allow administrators to overwrite your settings in the typo3conf/LocalConfiguration.php file as the configuration of your extension is loaded afterwards. Thus, it would overwrite any previous configuration settings if you don’t check first if they exist.

There are four settings you can use to configure your cache in the TYPO3 caching framework:

  • frontend : What type of data is accepted
  • backend : Where is the data stored
  • options : Additional configuration settings for the back-end
  • groups : The cache groups your extension cache belongs to

There’s the choice between two cache front-ends and several cache back-ends. A complete list of both including their options is available in the TYPO3 cache front-end/back-end article. The groups setting can be either pages, system, all or a combination thereof. Normally, you should use only one cache group and the group all is the default value if you don’t specify one.

If you use the default database back-end, you should be aware that the default lifetime of cached entries is only ONE HOUR if you don’t set it to a different value or to unlimited (“0”) like in the example above. Don’t forget to set up the “Caching framework garbage collection” scheduler task to clean up old entries. Otherwise, your cache tables will grow infinitely!

Use your new cache

Once set up, you can access the cache everywhere in you TYPO3 extension and you only have to write a few lines:

$name = 'TYPO3\\CMS\\Core\\Cache\\CacheManager';
$manager = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance( $name );
$cache = $manager->getCache( 'aimeos' );

They create an instance of the cache manager and retrieve your cache by name. Replace aimeos with the name of your extension resp. the name you’ve used in your ext_localconf.php in the cache setup code. Now you are able to set and get cached entries or remove them from the cache again:

$cache->set( 'auniqueidentifier', $data, array( 'tag1', 'tag2' ) );
$data = $cache->get( 'auniqueidentifier' );
$cache->remove( 'auniqueidentifier' );

So, let’s go and make your TYPO3 extension fast! 🙂

Language mapping in TYPO3 with RealURL and bootstrap package

The cool thing about the TYPO3 bootstrap_package extension is that you can set up a new site within minutes because everything is pre-configured. This also includes the RealURL setup so you already see the page titles in your URLs and the language parameter value is mapped too.

The dark side

You will face the dark side of the bootstrap package when you try to use different languages than those configured as examples. There’s only a mapping between the value of sys_language_id and English, German and Danish available and if you try to add a mapping manually, RealURL immediately switches to the “manual mode” where you have to configure everything yourself once again.

Loosing the auto-configuration implies a lot of work, especially if you are using the Aimeos web shop extension or similar ones, which add a lot of RealURL rules out of the box. Thus, it’s very compelling to try to overwrite only the RealURL configuration which is responsible for the language ID/code mapping.

The bright side

Fortunately, the bootstrap package allows you to overwrite its RealURL auto-configuration rules including the preVars section which is responsible for the language mapping.

First of all, you need an extension for your project where you can add the RealURL rules to. You can either use the Kickstarter extension or the Aimeos extension creator. The automatic configuration requires a PHP class and method that will add new or overwrite existing RealURL rules. You should add it in the Classes/Realurl.php file of your extension. Copy this code to your new file:

  1. <?php
  2.  
  3. namespace <vendor>\<extname>;
  4.  
  5. class Realurl
  6. {
  7.         public function addAutoConfig( array $params, $pObj )
  8.         {
  9.                 $params['config']['preVars'][1] = array(
  10.                         'GETvar' => 'L',
  11.                         'valueMap' => array(
  12.                                 'en' => '0',
  13.                                 'de' => '1',
  14.                                 '...' => '2',
  15.                         ),
  16.                         'noMatch' => 'bypass',
  17.                 );
  18.  
  19.                 return $params['config'];
  20.         }
  21. }

The addAutoConfig() method replaces the second entry of the “preVars” array, which are exactly the language mapping rules in the bootstrap package. You can change or add as many entries in the “valueMap” array as you like. The array keys are the strings that will be shown in the URL while the values are the sys_language_id values of the languages you’ve configured in your TYPO3 page tree and TypoScript configuration.

Remember to replace the <vendor> and <extname> placeholders in the namespace statement in the third line of the file. <vendor> can be an arbitrary string while <extname> must be your extension name which the first letter in upper case as well as all letters that follow an underscore, e.g. “bootstrap_package” becomes “BootstrapPackage”.

Afterwards, you have to add your class to the RealURL hook that will call your method when collecting the rules for auto-configuration. This is done in the ext_localconf.php of your extension by adding these lines:

  1. $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['ext/realurl/class.tx_realurl_autoconfgen.php']['extensionConfiguration']['<extname>'] =
  2. 	'EXT:<extname>/Classes/Realurl.php:<vendor>\\<extname>\\Realurl->addAutoConfig';

As above, you have to replace the <vendor> and <extname> placeholders with the same values you’ve used for the namespace statement.

The order you will install the extensions matters! Make sure the RealURL extension is installed first, then the bootstrap package and afterwards the rest of the extensions which contain RealURL rules including your new one.

Finally, clear your caches (TYPO3 general and frontend caches) and delete the typo3conf/realurl_autoconf.php file. Then, your languages should appear correctly mapped in your URLs.