How to use the TYPO3 caching framework with cache groups

The TYPO3 caching framework is a great way to speed up your site if your extension needs to perform tasks that can be cached. In the Aimeos TYPO3 extension, we use it to store content like list or detail views of the web shop to get lightning fast response times. Since version 6.2, a cache can belong to one or more cache groups in the caching framework and it enables editors to control more precisely which caches should be flushed. Unfortunately, this features isn’t well documented yet in the TYPO3 documentation but here’s the remedy!

What are cache groups

In the TYPO3 caching framework, there are currently three types of caches:

  • pages : Front-end related caches
  • system : Low-level caches that shouldn’t normally be flushed
  • all : All other caches that don’t fall into one of the two categories before

Based on the cache type (a.k.a cache group) the caches are flushed on different occasions. This makes a difference for the editors because depending on what they have changed, most often one of the cache groups will be flushed as well. Also, there are at least two options available when editors decide to flush the caches manually:

Aimeos-typo3-cache7The first option will only flush front-end related caches while the second one flushes all caches that are not in the “system” or “pages” cache group. Flushing the “system” caches isn’t that easy any more but those don’t need to be cleared at all by editors. Instead, they are automatically flushed if e.g. a new extension is installed.

Therefore, it makes sense to tell TYPO3 which cache group your cache should be in if you use the caching framework to store data temporarily. This allows at least editors to selectively flush caches instead of the famous “flush all caches to be sure”! More information about which cache in in which cache group can be found in the article about the TYPO3 caching architecture.

Set up your extension cache

To create your own cache in your TYPO3 extension, there are only a few lines in the ext_localconf.php of your extension necessary. You can copy and paste these lines and only have to replace the aimeos string by the name of your own TYPO3 extension:

if( !is_array( $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos'] = array();
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['frontend'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['frontend'] = 'TYPO3\\CMS\\Core\\Cache\\Frontend\\StringFrontend';
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['backend'] ) ) {
    $TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\DatabaseBackend';
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['options'] ) ) {
	$TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['options'] = array( 'defaultLifetime' => 0 );
}
 
if( !isset($TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['groups'] ) ) {
	$TYPO3_CONF_VARS['SYS']['caching']['cacheConfigurations']['aimeos']['groups'] = array( 'pages' );
}

The whole if/is_array/isset thing is necessary to allow administrators to overwrite your settings in the typo3conf/LocalConfiguration.php file as the configuration of your extension is loaded afterwards. Thus, it would overwrite any previous configuration settings if you don’t check first if they exist.

There are four settings you can use to configure your cache in the TYPO3 caching framework:

  • frontend : What type of data is accepted
  • backend : Where is the data stored
  • options : Additional configuration settings for the back-end
  • groups : The cache groups your extension cache belongs to

There’s the choice between two cache front-ends and several cache back-ends. A complete list of both including their options is available in the TYPO3 cache front-end/back-end article. The groups setting can be either pages, system, all or a combination thereof. Normally, you should use only one cache group and the group all is the default value if you don’t specify one.

If you use the default database back-end, you should be aware that the default lifetime of cached entries is only ONE HOUR if you don’t set it to a different value or to unlimited (“0”) like in the example above. Don’t forget to set up the “Caching framework garbage collection” scheduler task to clean up old entries. Otherwise, your cache tables will grow infinitely!

Use your new cache

Once set up, you can access the cache everywhere in you TYPO3 extension and you only have to write a few lines:

$name = 'TYPO3\\CMS\\Core\\Cache\\CacheManager';
$manager = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance( $name );
$cache = $manager->getCache( 'aimeos' );

They create an instance of the cache manager and retrieve your cache by name. Replace aimeos with the name of your extension resp. the name you’ve used in your ext_localconf.php in the cache setup code. Now you are able to set and get cached entries or remove them from the cache again:

$cache->set( 'auniqueidentifier', $data, array( 'tag1', 'tag2' ) );
$data = $cache->get( 'auniqueidentifier' );
$cache->remove( 'auniqueidentifier' );

So, let’s go and make your TYPO3 extension fast! 🙂

Language mapping in TYPO3 with RealURL and bootstrap package

The cool thing about the TYPO3 bootstrap_package extension is that you can set up a new site within minutes because everything is pre-configured. This also includes the RealURL setup so you already see the page titles in your URLs and the language parameter value is mapped too.

The dark side

You will face the dark side of the bootstrap package when you try to use different languages than those configured as examples. There’s only a mapping between the value of sys_language_id and English, German and Danish available and if you try to add a mapping manually, RealURL immediately switches to the “manual mode” where you have to configure everything yourself once again.

Loosing the auto-configuration implies a lot of work, especially if you are using the Aimeos web shop extension or similar ones, which add a lot of RealURL rules out of the box. Thus, it’s very compelling to try to overwrite only the RealURL configuration which is responsible for the language ID/code mapping.

The bright side

Fortunately, the bootstrap package allows you to overwrite its RealURL auto-configuration rules including the preVars section which is responsible for the language mapping.

First of all, you need an extension for your project where you can add the RealURL rules to. You can either use the Kickstarter extension or the Aimeos extension creator. The automatic configuration requires a PHP class and method that will add new or overwrite existing RealURL rules. You should add it in the Classes/Realurl.php file of your extension. Copy this code to your new file:

  1. <?php
  2.  
  3. namespace <vendor>\<extname>;
  4.  
  5. class Realurl
  6. {
  7.         public function addAutoConfig( array $params, $pObj )
  8.         {
  9.                 $params['config']['preVars'][1] = array(
  10.                         'GETvar' => 'L',
  11.                         'valueMap' => array(
  12.                                 'en' => '0',
  13.                                 'de' => '1',
  14.                                 '...' => '2',
  15.                         ),
  16.                         'noMatch' => 'bypass',
  17.                 );
  18.  
  19.                 return $params['config'];
  20.         }
  21. }

The addAutoConfig() method replaces the second entry of the “preVars” array, which are exactly the language mapping rules in the bootstrap package. You can change or add as many entries in the “valueMap” array as you like. The array keys are the strings that will be shown in the URL while the values are the sys_language_id values of the languages you’ve configured in your TYPO3 page tree and TypoScript configuration.

Remember to replace the <vendor> and <extname> placeholders in the namespace statement in the third line of the file. <vendor> can be an arbitrary string while <extname> must be your extension name which the first letter in upper case as well as all letters that follow an underscore, e.g. “bootstrap_package” becomes “BootstrapPackage”.

Afterwards, you have to add your class to the RealURL hook that will call your method when collecting the rules for auto-configuration. This is done in the ext_localconf.php of your extension by adding these lines:

  1. $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['ext/realurl/class.tx_realurl_autoconfgen.php']['extensionConfiguration']['<extname>'] =
  2. 	'EXT:<extname>/Classes/Realurl.php:<vendor>\\<extname>\\Realurl->addAutoConfig';

As above, you have to replace the <vendor> and <extname> placeholders with the same values you’ve used for the namespace statement.

The order you will install the extensions matters! Make sure the RealURL extension is installed first, then the bootstrap package and afterwards the rest of the extensions which contain RealURL rules including your new one.

Finally, clear your caches (TYPO3 general and frontend caches) and delete the typo3conf/realurl_autoconf.php file. Then, your languages should appear correctly mapped in your URLs.

Steps to configure your TYPO3 extension automatically

Simplicity is key! I think nobody would oppose to this little sentence but in the TYPO3 ecosystem, many things are not very simple – at least not for new users. Even the most popular extensions need a deep knowledge of how to configure them before they can be used. And good documentation is most of time a rare thing!

So why not build an extension that works out of the box and configures itself as far as possible to give users an instant feeling of success? The constants.txt or setup.txt TypoScript files of the extensions can contain almost all default settings. But things get difficult if the required configuration depends on dynamic things like page IDs. Then, all hope of simplicity is lost and users have to add the necessary settings themselves again.

Is this really true?

The signal/slot mechanism described in the article about executing scripts after installing an extension provides the tools to get things done. Although, the article is very generic and I would like to go into more detail about how we’ve implemented things in the Aimeos web shop distribution.

The distribution imports some pages from the data.t3d file and the TYPO3 importer cares about avoiding conflicts with existing pages by reassigning the page IDs. Problems arose after we’ve added some required TypoScript settings for access restricted pages like the “MyAccount” page. In order to redirect to the login form when a user isn’t logged in, you have to configure its page ID. That seems to be impossible at the first glance when the page ID will differ in each TYPO3 installation.

The solution is to look up the page ID from the database dynamically after the extension is installed and generate a constants.txt file that will then contain the actual page ID. The constants in this file can be used in the TypoScript setup.txt file afterwards.

But first, you need to make sure your code will be executed automatically after the extension is installed. Remember the steps from the article about executing scripts? The first one was to register your extension for listening to the signal for extension installation:

  1. if (TYPO3_MODE === 'BE') {
  2.     $class = 'TYPO3\\CMS\\Extbase\\SignalSlot\\Dispatcher';
  3.     $dispatcher = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance($class);
  4.     $dispatcher->connect(
  5.         'TYPO3\\CMS\\Extensionmanager\\Service\\ExtensionManagementService',
  6.         'hasInstalledExtensions',
  7.         '<vendor>\\<extension>\\<classname>',
  8.         '<method>'
  9.     );
  10. }

Please replace <vendor>, <extension>, <classname> and <method> with the strings that fits for your extension. Afterwards, you must implement the method of your class you’ve registered for listening. We use the name of the Aimeos setup class in our example because you can have a look at it’s implementation for reference.

  1. <?php
  2.  
  3. namespace <vendor>\<extension>;
  4.  
  5. use \TYPO3\CMS\Core\Utility\GeneralUtility;
  6. use \TYPO3\CMS\Backend\Utility\BackendUtility;
  7.  
  8. class Setup
  9. {
  10.     public function process( $extname = null )
  11.     {
  12.         if( $extname !== '<extension>' ) {
  13.             return;
  14.         }
  15.  
  16.         $data = '';
  17.         $ds = DIRECTORY_SEPARATOR;
  18.         $filename = dirname( __DIR__ ) . $ds . 'Configuration' . $ds . 'TypoScript' . $ds . 'constants.txt';
  19.  
  20.         $records = BackendUtility::getRecordsByField( 'pages', 'title', 'My account' );
  21.  
  22. 	foreach( $records as $record ) {
  23.             $data .= 'config.typolinkLinkAccessRestrictedPages = ' . intval( $record['uid'] ) . "\n";
  24.         }
  25.  
  26.         GeneralUtility::writeFile( $filename, $data );
  27.     }
  28. }
The example contains placeholders for <vendor> and <extension> and you have to replace them with the names from your extension.

First we have to ensure that the constants.txt file is only created if our own extension is installed. Thus, we check for the extension name to filter out all other extension installations.

The TYPO3 backend utilities contains some nice methods to retrieve records from the database with only little effort. For example, you can use the getRecordsByField() method from the BackendUtility class to fetch all records from a table where a field matches a certain value. Here we want to get all records where the “title” equals “My account” from the “pages” table.

Searching for a page with a title only works as long as the page isn’t renamed! In the best case, your extension has added the page including the page name itself, so this won’t be a problem. It’s not bad if somebody renames the page afterwards because the page ID will stay the same in this case. The only problematic situation could be several pages with the same name.

Make sure you generate valid TypoScript afterwards, including new lines and keep an eye on security by enforcing the types you are expecting! The foreach() loop is a great way to deal with no records and several returned records too because both can an will happen sooner or later. Don’t throw an exception if something unexpected happens because it may lead to not being able to install your extension at all! It’s better to drop those records and add a message to the log file.

You don’t have to add all TypoScript statements to your constants.txt that contains a dynamic value. Instead, you can also define them like “page.myaccount.id = 123” and reference the constants in your setup.txt with “{$page.myaccount.id}”. This is especially handy to keep your constants.txt small and if you need the value several times.

At last, your extension has to write the TypoScript statements to the constants.txt file. The GeneralUtiltiy class from the TYPO3 core contains a little writeFile() helper function to get this done within one line.

Simple, isn’t it?

These few additional lines can make the difference between an easy installable extension and hours of frustrations bugging your users. I think the little effort during the development of your extension is worth the hours saved later on. Your users will thank you for your work by using your extension more often!

A special thanks for the inspiration to this tutorial goes to Stephan Schuler from netlogix.

How to execute scripts after installing TYPO3 extensions

Extensions are a great way to add features to the TYPO3 CMS and there are extension for virtually everything! Simply download them from the TYPO3 extension repository and install them via the Extension Manager in the TYPO3 back-end. So far so good …

Often, the remaining tasks of configuring the extension, adding new pages with the plug-ins from the extension or importing required data into the database becomes much more cumbersome. Why can’t life be not as easy as in other CMS? Just click and everything works?

Yes, we can!

The infrastructure for performing additional tasks after installing extensions is part of TYPO3 since some time. At least since TYPO3 6.2 it works flawlessly, before your way may vary. The trick is to utilize the signal/slot mechanism to connect to the Extension Management service and listen for signals emitted after an extension was installed.

In the Aimeos TYPO3 web shop extension we’ve added the following code to the ext_localconf.php:

  1. if (TYPO3_MODE === 'BE') {
  2.     $class = 'TYPO3\\CMS\\Extbase\\SignalSlot\\Dispatcher';
  3.     $dispatcher = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance($class);
  4.     $dispatcher->connect(
  5.         'TYPO3\\CMS\\Extensionmanager\\Service\\ExtensionManagementService',
  6.         'hasInstalledExtensions',
  7.         'Aimeos\\Aimeos\\Setup',
  8.         'executeOnSignal'
  9.     );
  10. }

The first line ensures that we are connecting only to the Extension Management service if we are in the back-end. If you leave this line out, your script would connect at every front-end request too, which would only slow down response times. The next two lines instantiates the signal/slot dispatcher class and the makeInstance() method of the GeneralUtility class does a good job.

Don’t use the PHP “new” operator for creating the class instance! This would create one instance each time the code is executed instead of sharing a common instance during the request.

The remaining lines connect your method executeOnSignal() of your namespaced class Aimeos\Aimeos\Setup to the Extension Management service whose class name is handed over as first parameter. Here we are listening to the hasInstalledExtensions signal, that is emitted by the service after an extension has been installed. The key of the installed extension will be passed as first parameter to our method.

There’s another signal that is emitted by the Extension Management service named willInstallExtensions. By listening to this signal, your script can be informed about which extensions are going to be installed next. An array of extension keys is given in this case.

Do what you need

At last, you need to create a class that contains the code to be executed after an extension is installed. The Aimeos TYPO3 extension contains a Setup class in the Classes/ directory of the extension with a method named executeOnSignal() to care about this. The basic code of this class is:

  1. <?php
  2.  
  3. namespace Aimeos\Aimeos;
  4.  
  5. class Setup
  6. {
  7.     public function executeOnSignal( $extname = null )
  8.     {
  9.         if( $extname !== 'aimeos' ) {
  10.             return;
  11.         }
  12.  
  13.         // your code
  14.     }
  15. }

The namespace declaration is important but must be adapted to your extension. The first namespace part is the vendor name, e.g. the name of your company. The second part is the extension key with the first letter uppercased. If your extension key contains an underscore, then the underscore is removed and the next character transformed to uppercase as well, e.g. the boostrap_package uses BootstrapPackage as the second namespace part.

You are free to use any valid class and method name in your extension. Only make sure that the third and fourth parameter of the connect() method call are named appropriately. Also, ensure that your class and method are neither abstract nor static so an instance can be created. Like said above, the key name of the extension that has been installed is given as first parameter to your method. For compatibility reasons, you should make that parameter optional by assigning a default value of null.

Use the given extension key name to filter for your one! The Extension Management service will inform your script about every extension that is installed, not only your own one. Otherwise, you would execute your script code for every installed extension and this may slow down the installation process depending on how much your code does.

What’s next?

Pretty simple, isn’t it? Now it’s time to pimp your TYPO3 extension to remove as much burden as possible from the user! In a second article we will go into more detail how this can save your users day when your extension imports pages and depending TypoScript configuration.

A special thanks goes to Benjamin Kott for his bootstrap_package because it was the first that provided a working example that the signal/slot infrastructure can be used for these kind of use cases.