elasticsearch?

Help for integrating the Laravel package
Forum rules
Always add your Laravel, Aimeos and PHP version as well as your environment (Linux/Mac/Win)
Spam and unrelated posts will be removed immediately!
BonoboMagno
Posts: 11
Joined: 30 Jun 2017, 13:22

elasticsearch?

Post by BonoboMagno » 30 Jun 2017, 15:37

Any wrapper for elasticsearch?
Did someone have implemented it before?
I will need it just for full text search on a 10k product db.

User avatar
aimeos
Administrator
Posts: 7836
Joined: 01 Jan 1970, 00:00

Re: elasticsearch?

Post by aimeos » 01 Jul 2017, 10:47

Not yet. ElasticSearch or any other document database can be used by implementing specific managers for the "index" domain: https://github.com/aimeos/aimeos-core/t ... ex/Manager

UPDATE:
There are now extensions available for using ElastisSearch and Solr:
- https://aimeos.com/extensions#c435
- https://aimeos.com/extensions#c606

For 10k products, Aimeos using a MySQL database is almost as fast as ElasticSearch. Up to 100k products, you won't get any problems, even with faceted search.
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

BonoboMagno
Posts: 11
Joined: 30 Jun 2017, 13:22

Re: elasticsearch?

Post by BonoboMagno » 03 Jul 2017, 08:06

realtime fulltext search too?
usually mysql ask you to use something like %word% , but that mean that:
find "word"
1) %woRrd% is ok
2)%Sword% is not

did your index engine make something different?

User avatar
aimeos
Administrator
Posts: 7836
Joined: 01 Jan 1970, 00:00

Re: elasticsearch?

Post by aimeos » 03 Jul 2017, 10:49

MySQL full text search is prefix based. Thus, if you search for "word", you will find:
- word%
- Word%
- words%
But not "sword".

There's a like based search implemented too but that's very slow because it can't use any index.
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

BonoboMagno
Posts: 11
Joined: 30 Jun 2017, 13:22

Re: elasticsearch?

Post by BonoboMagno » 03 Jul 2017, 13:58

aimeos wrote:MySQL full text search is prefix based. Thus, if you search for "word", you will find:
- word%
- Word%
- words%
But not "sword".

There's a like based search implemented too but that's very slow because it can't use any index.
yeah, this is the mysql limit. This is why i wanna try to use elasticsearch.

I found plastic https://github.com/sleimanx2/plastic
that seems to accelerate the work.
Otherwise, doctrine search https://github.com/doctrine/search
and the last is laravel scout with https://github.com/ErickTamayo/laravel-scout-elastic

Any suggest? Where are defined the model? can i extend them (with a trait)?
I can't see any /model directory
How you usually overload/extend them?

edit.
i found something like setup tasks.
You say that aimeos version controll the database too. I like the idea, how to preserve the feature?
any doc?

User avatar
aimeos
Administrator
Posts: 7836
Joined: 01 Jan 1970, 00:00

Re: elasticsearch?

Post by aimeos » 03 Jul 2017, 15:53

BonoboMagno wrote: Any suggest? Where are defined the model? can i extend them (with a trait)?
I can't see any /model directory
How you usually overload/extend them?
Aimeos doesn't use Laravel models or any other ODM for two reasons:
- not portable
- too slow

Instead, there's an index domain in the Aimeos data management layer that cares about indexing. By default, it uses index tables in the database but this can be replaced by a ElasticSearch document store too.
BonoboMagno wrote: i found something like setup tasks.
You say that aimeos version controll the database too. I like the idea, how to preserve the feature?
any doc?
You can find documentation about database schema updates and setup tasks here:
https://aimeos.org/docs/Developers/Library/Setup_tasks
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

BonoboMagno
Posts: 11
Joined: 30 Jun 2017, 13:22

Re: elasticsearch?

Post by BonoboMagno » 04 Jul 2017, 11:47

I read a bitter more after my last reply. You use docrine dbal that i read is a PDO thin layer.
Your reason are right.

I found that you have a wrapper for redis (ai-cache ext), nice thing.

On aimos-laravel/src/default.php
i have this array

Code: Select all

'index' => array(
			'manager' => array(
				'name' => 'MySQL',
				'attribute' => array(
					'name' => 'MySQL',
				),
				'catalog' => array(
					'name' => 'MySQL',
				),
				'price' => array(
					'name' => 'MySQL',
				),
				'text' => array(
					'name' => 'MySQL',
				),
			),
		)
So i can define here elasticsearch (let's say for 'text'),
extending from MShop/Index/manager/?

Code: Select all

	/**
	 * Initializes the search attribute object.
	 *
	 * @param array $params Parameter to be set on initialisation
	 *		[code] string
	 *		[default] string (optional)
	 *		[internalcode] string
	 *		[internaltype] string
	 *		[internaldeps] array (optional)
	 *		[label] string
	 *		[public] boolean (optional)
	 *		[required] booblean (optional)
	 *		[type] string
	 */
but what internalcode/internaldepts do?
on index/manager/text i have

Code: Select all

'index.text.id' => array(
			'code'=>'index.text.id',
			'internalcode'=>'mindte."textid"',
			'internaldeps'=>array( 'LEFT JOIN "mshop_index_text" AS mindte
				USE INDEX ("idx_msindte_value", "idx_msindte_p_s_lt_la_ty_do_va") ON mindte."prodid" = mpro."id"' ),
			'label'=>'Product index text ID',
			'type'=> 'string',
			'internaltype' => \Aimeos\MW\DB\Statement\Base::PARAM_STR,
			'public' => false,
		)
This mean that aimeos run explicit index, but i cant extend the "standard.php" for elasticsearch.
A have to throw away all? Is ok, but zero adapter to use for elasticsearch?

Thanks again, but never used doctrine and is also my first time with elasticsearch :/

User avatar
aimeos
Administrator
Posts: 7836
Joined: 01 Jan 1970, 00:00

Re: elasticsearch?

Post by aimeos » 05 Jul 2017, 07:50

Doctrine DBAL is only used for SQL schema updates. The corresponding DBAL database layer is only necessary to do that.

For ElasticSearch, it doesn't make much sense to extend from the SQL based managers because document oriented databases work very different. "internalcode" is the mapping from the used search keys (e.g. "product.id") to the data structure of the storage (e.g. table.column for RDBMS). If the storage allows the same keys, you should use "product.id" an alike directly. "internaldeps" is for joining the tables in RDBMS. For document oriented databases, this wouldn't be necessary because all information would be stored only in one document.

You also have to create an adapter for the ElasicSearch query language like for SQL:
https://github.com/aimeos/aimeos-core/t ... W/Criteria
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

BonoboMagno
Posts: 11
Joined: 30 Jun 2017, 13:22

Re: elasticsearch?

Post by BonoboMagno » 05 Jul 2017, 14:25

I hoped to didn't have to do it from scratch :/

You use dbal for schema creation/update. But what you use for read the data? Can i switch to ORM ? maybe extending laravel version.
I think that is more futureproof using eloquent standard. You know, laravel is pretty much a monopoly on php framework. If one want speed, he can just pass to elasticsearch/angolia.

I have a question, how to call a product creation?
let see if i understand.
i see on routes that saveAction on JqadmController.php is called.
It call createClient with resource "product" that call the factory (nice pattern, but is better to explain what can builds before calling it)
that call the product factory, that call product/standard classes.

It's really long. And now, if i want to create product with a job, i have only to call the factory?

in this case,

Code: Select all

		$site = 'default;
		$lang = 'en';
		$resource = 'product'

		$aimeos = app( '\Aimeos\Shop\Base\Aimeos' )->get();
		$templatePaths = $aimeos->getCustomPaths( 'admin/jqadm/templates' );

		$context = app( '\Aimeos\Shop\Base\Context' )->get( false, 'backend' );
		$context->setI18n( app('\Aimeos\Shop\Base\I18n')->get( array( $lang, 'en' ) ) );
		$context->setLocale( app('\Aimeos\Shop\Base\Locale')->getBackend( $context, $site ) );
		$context->setView( app( '\Aimeos\Shop\Base\View' )->create( $context, $templatePaths, $lang ) );

		\Aimeos\Admin\JQAdm\Factory::createClient( $context, $templatePaths, $resource )->setAimeos( $aimeos );
	
but where are the post data?! no $request array. maybe on $resource = Route::input( 'resource' ); ?
what it have?

What are the api if i want to use the package only as backend companion?
Why laravel extension have only view resource files?! :/

User avatar
aimeos
Administrator
Posts: 7836
Joined: 01 Jan 1970, 00:00

Re: elasticsearch?

Post by aimeos » 05 Jul 2017, 21:01

BonoboMagno wrote: I think that is more futureproof using eloquent standard. You know, laravel is pretty much a monopoly on php framework. If one want speed, he can just pass to elasticsearch/angolia.
Laravel is just a PHP framework but e.g. no CMS. Aimeos is independent of any framework or application because in its core it's a PHP library that encapsulates a complete e-commerce system. Thus, Laravels Eloquent is not an option.
BonoboMagno wrote: i see on routes that saveAction on JqadmController.php is called.
It call createClient with resource "product" that call the factory (nice pattern, but is better to explain what can builds before calling it) that call the product factory, that call product/standard classes.
You can create a decorator like this one to push the product related data into ElasticSeach:
https://github.com/aimeos/ai-admin-jqad ... /Cache.php
BonoboMagno wrote: but where are the post data?! no $request array. maybe on $resource = Route::input( 'resource' ); ?
what it have?
You can get the GET/POST data using $view->param()
BonoboMagno wrote: What are the api if i want to use the package only as backend companion?
Why laravel extension have only view resource files?! :/
Can you please explain in more detail?
Professional support and custom implementation are available at Aimeos.com
If you like Aimeos, Image give us a star

Post Reply