Developers/Controller/Implement job controller

From Aimeos documentation

Developers
Other languages:
English 100% • ‎русский 14%


2016.x+ version

A shop needs to execute several tasks in an asynchronous manner, like sending out e-mails, importing or exporting data as well as rebuilding indexes. The tasks included in Aimeos that can be referred to by these keys:

  • admin/cache (remove old cache entries)
  • admin/log (remove old log entries)
  • catalog/import/csv (import categories from CSV files)
  • customer/email/account (create new customer accounts and send e-mails)
  • customer/email/watch (customer notification e-mails on product updates)
  • index/optimize (optimize the product index for fastest access)
  • index/rebuild (rebuilds the product index)
  • media/scale (rescales the product images to the new sizes)
  • order/cleanup/unfinished (removes unfinished orders)
  • order/cleanup/unpaid (removes unpaid orders)
  • order/email/delivery (order delivery related e-mails)
  • order/email/payment (order payment related e-mails)
  • order/email/voucher (e-mails containing the voucher code)
  • order/export/csv (export orders in admin interface)
  • order/service/async (batch update of payment/delivery status)
  • order/service/delivery (process order delivery services like sending orders to ERP systems)
  • order/service/payment (capture authorized payments)
  • product/bought (automatically generated product suggestions)
  • product/export (export products)
  • product/export/sitemap (generate product sitemaps for search engines)
  • product/import/csv (import products from CSV files)
  • subscription/export/csv (export subscriptions in admin interface)
  • subscription/process/begin (start subscription period and add permissions if applicable)
  • subscription/process/renew (renew subscriptions on next date)
  • subscription/process/end (finish subscription period and revoke permissions if applicable)

These tasks are implemented as job controllers, PHP classes which can be executed from the command line or from a scheduler provided by the application. Each application and framwork offers means to execute them directly:

Location

The job implementations are grouped together, e.g. all administrative tasks are located in the "Admin" subdirectory of the "controller/jobs/src/Controller/Jobs" directory of the core while all order related jobs can be found in the "Order" subdirectory. Depending on the type of task you need to implement or if it depends on another one like the "product/export/sitemap" job controller, you may place your implementation in a subdirectory of one of the existing directories.

The first part of the job controller key (e.g. "product" in "product/export") corresponds to the domain of the managers in the "lib/mshoplib/src" directory. The term "domain" means all classes that care about the same kind of data like the "order" domain for all order related data: Ordered products, customer addresses, used delivery and payment in orders and basic order information.

If you want to implement a job controller that mainly works on data from a domain like "media", feel free to create your controller in the "Controller/Jobs/Media" directory and then it's accessed by the "media/..." key.

Factory

If you create a new job controller that doesn't extend an existing one, you need to implement a factory that cares about creating the job controller object. It will also wrap the decorators around if there are any configured. The factory only contains the static createController() method with those few lines:

  1. namespace Aimeos\Controller\Jobs\Product\Export;
  2.  
  3. class Factory
  4. 	extends \Aimeos\Controller\Jobs\Common\Factory\Base
  5. 	implements \Aimeos\Controller\Jobs\Common\Factory\Iface
  6. {
  7. 	public static function createController( \Aimeos\MShop\Context\Item\Iface $context, \Aimeos\Bootstrap $aimeos, $name = null )
  8. 	{
  9. 		if ( $name === null ) {
  10. 			$name = $context->getConfig()->get( 'controller/jobs/product/export/name', 'Standard' );
  11. 		}
  12.  
  13. 		if ( ctype_alnum($name) === false )
  14. 		{
  15. 			$classname = is_string($name) ? '\\Aimeos\\Controller\\Jobs\\Product\\Export\\' . $name : '<not a string>';
  16. 			throw new \Aimeos\Controller\Jobs\Exception( sprintf( 'Invalid characters in class name "%1$s"', $classname ) );
  17. 		}
  18.  
  19. 		$iface = '\\Aimeos\\Controller\\Jobs\\Iface';
  20. 		$classname = '\\Aimeos\\Controller\\Jobs\\Product\\Export\\' . $name;
  21.  
  22. 		$controller = self::createControllerBase( $context, $aimeos, $classname, $iface );
  23.  
  24. 		return self::addControllerDecorators( $context, $aimeos, $controller, 'product/export' );
  25. 	}
  26. }

The example is from the product export job controller. You need to replace all occurences of "product", "Product", "export" and "Export" by the names that match your directories below "controller/jobs/src/Controller/Jobs/" in your Aimeos extension. If you want more than two levels, e.g. "product/export/gshopping", please extend the namespace, paths and class names by that last part.

Skeleton

Creating a new job controller is rather simple because it needs to contain only three public methods:

  1. namespace Aimeos\Controller\Jobs\Product\Export;
  2.  
  3. class Standard
  4. 	extends \Aimeos\Controller\Jobs\Base
  5. 	implements Aimeos\Controller\Jobs\Iface
  6. {
  7. 	/**
  8. 	 * Returns the localized name of the job.
  9. 	 *
  10. 	 * @return string Name of the job
  11. 	 */
  12. 	public function getName()
  13. 	{
  14. 		return $this->getContext()->getI18n()->dt('controller/jobs', 'Product export');
  15. 	}
  16.  
  17.  
  18. 	/**
  19. 	 * Returns the localized description of the job.
  20. 	 *
  21. 	 * @return string Description of the job
  22. 	 */
  23. 	public function getDescription()
  24. 	{
  25. 		return $this->getContext()->getI18n()->dt('controller/jobs', 'Exports all available products');
  26. 	}
  27.  
  28.  
  29. 	/**
  30. 	 * Executes the job.
  31. 	 *
  32. 	 * @throws Aimeos\Controller\Jobs\Exception If an error occurs
  33. 	 */
  34. 	public function run()
  35. 	{
  36. 		// ...
  37. 	}
  38. }

The name of the job controller corresponds to it's location in the file system where each directory is a part of its name and the slashes (/) between the directory names are replaced by backslashes (\), so

Controller/Jobs/Product/Export/Standard.php

maps to

Aimeos\Controller\Jobs\Product\Export\Standard

For new implementations that are not an alternative to an existing ones, you should always use "Standard" at the end to show that it's the default implementation. This makes it easy to replace your implementation by an alternative one, e.g. "Aimeos\Controller\Jobs\Product\Export\Myexport" only by configuration. This would not be possible if you name your class and file "Controller/Jobs/Product/Export.php".

Furthermore, you need to extend from the base abstract class "Aimeos\Controller\Jobs\Base" to have access to the context and to some helper methods for functionality that is commonly used. Finally, implementing the "Aimeos\Controller\Jobs\Iface" interface makes your class a job controller recognized by the core library.

getName()

To be able to show a name in the language of the shop owner instead of the key (e.g. "product/export") for your job controller, the getName() method should return a string that can be translated. This is done by the dt() method of the internationalization / translation object that is part of the context item:

  1. return $this->getContext()->getI18n()->dt('controller/jobs', 'Product export');

There are several translation domains in the core library but for job controllers you always need to use the "controller/jobs" translation domain as shown above. The second parameter of the dt() method is the name that should be translated, i.e. the name of your job controller in English.

getDescription()

A more descriptive message about the functionality of your job controller should be returned by the getDescription() method. In order to be able to translate it to language of the shop owner, you have to use the dt() method of the internationalization / translation object:

  1. return $this->getContext()->getI18n()->dt('controller/jobs', 'Exports all available products');

The translation domain is also "controller/jobs" like for the name. Descriptions should be short but descriptive enough so people not used to your job controller can understand what it does. Don't make it too long (more than 250 characters are to long for sure) because it depends on the application how the description is shown and there may be not enough space to display long texts.

run()

All the real work is done by the run() method of your job controller. This method performs the tasks that your job controller is implemented for. Normally, it makes use of the managers available in the "lib/mshoplib/src" directory to retrieve, store or delete data in the database or any other storage, e.g.

  1. $manager = \Aimeos\MShop\Factory::createManager($this->getContext(), 'product');
  2. $search = $manager->createSearch();
  3. $search->setConditions($search->compare('==', 'product.type.code', 'selection'));
  4. $result = $manager->searchItems($search, array('product', 'text'));

In installations with two or more shops the site is defined by the one who configures the task so you should not care about the site your code is working on. This also means that you don't have access to data across all shops but only to the data of the current site. Depending on the tasks your code has to do, you can alter the current language and currency in the site item stored in the context or even set it to null to get items of all languages and currencies.

Working code for job controllers of different types can be found in the controller/jobs/src/Controller/Jobs directory of the core.

Templates

Since 2015-04 there's the possibility to use views and templates for generating output in job controllers. They are used in the same way as for the HTML clients, for example:

  1. $view = $this->getContext()->getView();
  2. $view->items = array( 1, 2, 3 );
  3.  
  4. $tplconf = 'controller/jobs/product/export/standard/template-items';
  5. $default = 'product/export/items-body-default.xml';
  6. $result = $view->render( $view->config( $tplconf, $default ) ) );

At first, you can retrieve a new view from the context object by using the getView() method and for repeated calls it always returns a clean object regardless of what has been done with previously returned objects. You can assign data directly like shown above or even assign multiple key/value pairs at once using the assign() method. It's signature and more useful methods can be found in the view class.

To render the output and return the content, you should use the render() method of the view. It expects the path of the template that should be used to generate the content. In combination with "$view->config()" it also checks if there's another template configured that should be used instead of the default one and translates a relative path into an absolute one. The latter is done by looking at the list of directories provided in "custom" => "controller/jobs/templates" of the manifest.php that is part of the core and every extension.

In the templates (usually in the subdirectories of "controller/jobs/templates"), the assigned data can be retrieved by using either

$this->varname

or by

$this->get('varname', 'defaultvalue')

The later one is preferable because it returns a default value if the information is not available and doesn't throw an exception.

Parallel processing

Since 2017.07, you can speed up jobs by splitting them into several independently running tasks. Thus, you can utilize multi-core systems efficiently for e.g. imports or exports. If you run several job controllers at once from a cron job, they are already automatically run in parallel if the system supports it.

It's also possible to e.g. read data from a file in chunks and process each chunk by a different task. For this, you need to create an anonymous function that will do the work after a new child process is spawned:

  1. $fcn = function( \Aimeos\MShop\Context\Item\Iface $context, $data ) {
  2.     echo $data;
  3. };
  4.  
  5. $context = $this->getContext()
  6. $context->getProcess()
  7.     ->start( $fcn, [$context, 'data1'] )
  8.     ->start( $fcn, [$context, 'data2'] )
  9.     ->wait();

This would span two child processes, both executing the anonymous function with a different data set. The result in this case can be "data1data2" or "data2data1", depending on which task completes first.

The anonymous function should only operate on the non-shared arguments passed to the function (objects like the context are automatically cloned before they are passed). You must not facilitate more objects to the function via the "function() use( ... )" construct!

The wait() call cleans up the child processes after they have exited. You can also use it wait for all running tasks to finish before you start the next tasks using a different function (synchronization).

Unit tests

Testing job controllers is an important part of the implementation to ensure that they are working correctly. The implementation of the unit tests cases doesn't differ much from other unit tests and you can use this skeleton for your own tests:

  1. namespace \Aimeos\Controller\Jobs\Product\Export\Sitemap;
  2.  
  3. class StandardTest extends \PHPUnit\Framework\TestCase
  4. {
  5. 	private $object;
  6. 	private $context;
  7. 	private $aimeos;
  8.  
  9.  
  10. 	protected function setUp()
  11. 	{
  12. 		\Aimeos\MShop\Factory::setCache( true );
  13.  
  14. 		$this->context = \TestHelperJobs::getContext();
  15. 		$this->aimeos = \TestHelperJobs::getAimeos();
  16.  
  17. 		$this->object = new \Aimeos\Controller\Jobs\Product\Export\Sitemap\Standard( $this->context, $this->aimeos );
  18. 	}
  19.  
  20.  
  21. 	protected function tearDown()
  22. 	{
  23. 		\Aimeos\MShop\Factory::setCache( false );
  24. 		\Aimeos\MShop\Factory::clear();
  25.  
  26. 		unset( $this->object );
  27. 	}
  28.  
  29.  
  30. 	public function testGetName()
  31. 	{
  32. 		$this->assertEquals( 'Product site map', $this->object->getName() );
  33. 	}
  34.  
  35.  
  36. 	public function testGetDescription()
  37. 	{
  38. 		$text = 'Creates a product site map for search engines';
  39. 		$this->assertEquals( $text, $this->object->getDescription() );
  40. 	}
  41.  
  42.  
  43. 	public function testRun()
  44. 	{
  45. 		$this->object->run();
  46. 		$this->assertFileExists( 'tmp/aimeos-sitemap-1.xml.gz' );
  47. 		unlink( 'tmp/aimeos-sitemap-1.xml.gz' );
  48. 	}
  49. }

If you already know unit tests the implementation is pretty straight forward. The only thing that is special are these lines:

  1. \Aimeos\MShop\Factory::setCache( true );
  2. \Aimeos\MShop\Factory::setCache( false );
  3. \Aimeos\MShop\Factory::clear();

When you use the \Aimeos\MShop\Factory::createManager() method to create manager objects, it caches objects and returns them if its asked for the same kind of object again. In unit tests, this may have undesired side effects and therefore, the lines above enable this caching only for the test cases of this unit test class and clears the object cache afterwards so the next unit test class starts in a defined state.

In this testRun() method, the assertion tests indirectly if the test case succeeded by looking for the generated file. Often you may want to test if methods of objects used in the job controller are called. This is a bit more advanced as you must create mock objects first and let the \Aimeos\MShop\Factory class return them instead of creating a new, real object. An example would be:

  1. $mock = $this->getMockBuilder('\Aimeos\MShop\Product\Manager\Lists\Standard')
  2. 	->setConstructorArgs( array( $this->context ) )
  3. 	->setMethods( array( 'deleteItems', 'saveItem' ) )
  4. 	->getMock();
  5.  
  6. \Aimeos\MShop\Factory::injectManager( $this->context, 'product/list', $mock );
  7.  
  8. $stub->expects( $this->atLeastOnce() )->method( 'deleteItems' );
  9. $stub->expects( $this->atLeastOnce() )->method( 'saveItem' );
  10.  
  11. $this->object->run();

This creates a product list manager as mock object, injects it into the \Aimeos\MShop\Factory class and defines that the deleteItems() and saveItems() method has to be called at least once before the test is marked as successful.

More information about mocking object is available in the test doubles section of the PHPUnit site.