Page 1 of 2

elasticsearch?

Posted: 30 Jun 2017, 15:37
by BonoboMagno
Any wrapper for elasticsearch?
Did someone have implemented it before?
I will need it just for full text search on a 10k product db.

Re: elasticsearch?

Posted: 01 Jul 2017, 10:47
by aimeos
Not yet. ElasticSearch or any other document database can be used by implementing specific managers for the "index" domain: https://github.com/aimeos/aimeos-core/t ... ex/Manager

UPDATE:
There are now extensions available for using ElastisSearch and Solr:
- https://aimeos.com/extensions#c435
- https://aimeos.com/extensions#c606

For 10k products, Aimeos using a MySQL database is almost as fast as ElasticSearch. Up to 100k products, you won't get any problems, even with faceted search.

Re: elasticsearch?

Posted: 03 Jul 2017, 08:06
by BonoboMagno
realtime fulltext search too?
usually mysql ask you to use something like %word% , but that mean that:
find "word"
1) %woRrd% is ok
2)%Sword% is not

did your index engine make something different?

Re: elasticsearch?

Posted: 03 Jul 2017, 10:49
by aimeos
MySQL full text search is prefix based. Thus, if you search for "word", you will find:
- word%
- Word%
- words%
But not "sword".

There's a like based search implemented too but that's very slow because it can't use any index.

Re: elasticsearch?

Posted: 03 Jul 2017, 13:58
by BonoboMagno
aimeos wrote:MySQL full text search is prefix based. Thus, if you search for "word", you will find:
- word%
- Word%
- words%
But not "sword".

There's a like based search implemented too but that's very slow because it can't use any index.
yeah, this is the mysql limit. This is why i wanna try to use elasticsearch.

I found plastic https://github.com/sleimanx2/plastic
that seems to accelerate the work.
Otherwise, doctrine search https://github.com/doctrine/search
and the last is laravel scout with https://github.com/ErickTamayo/laravel-scout-elastic

Any suggest? Where are defined the model? can i extend them (with a trait)?
I can't see any /model directory
How you usually overload/extend them?

edit.
i found something like setup tasks.
You say that aimeos version controll the database too. I like the idea, how to preserve the feature?
any doc?

Re: elasticsearch?

Posted: 03 Jul 2017, 15:53
by aimeos
BonoboMagno wrote: Any suggest? Where are defined the model? can i extend them (with a trait)?
I can't see any /model directory
How you usually overload/extend them?
Aimeos doesn't use Laravel models or any other ODM for two reasons:
- not portable
- too slow

Instead, there's an index domain in the Aimeos data management layer that cares about indexing. By default, it uses index tables in the database but this can be replaced by a ElasticSearch document store too.
BonoboMagno wrote: i found something like setup tasks.
You say that aimeos version controll the database too. I like the idea, how to preserve the feature?
any doc?
You can find documentation about database schema updates and setup tasks here:
https://aimeos.org/docs/Developers/Library/Setup_tasks

Re: elasticsearch?

Posted: 04 Jul 2017, 11:47
by BonoboMagno
I read a bitter more after my last reply. You use docrine dbal that i read is a PDO thin layer.
Your reason are right.

I found that you have a wrapper for redis (ai-cache ext), nice thing.

On aimos-laravel/src/default.php
i have this array

Code: Select all

'index' => array(
			'manager' => array(
				'name' => 'MySQL',
				'attribute' => array(
					'name' => 'MySQL',
				),
				'catalog' => array(
					'name' => 'MySQL',
				),
				'price' => array(
					'name' => 'MySQL',
				),
				'text' => array(
					'name' => 'MySQL',
				),
			),
		)
So i can define here elasticsearch (let's say for 'text'),
extending from MShop/Index/manager/?

Code: Select all

	/**
	 * Initializes the search attribute object.
	 *
	 * @param array $params Parameter to be set on initialisation
	 *		[code] string
	 *		[default] string (optional)
	 *		[internalcode] string
	 *		[internaltype] string
	 *		[internaldeps] array (optional)
	 *		[label] string
	 *		[public] boolean (optional)
	 *		[required] booblean (optional)
	 *		[type] string
	 */
but what internalcode/internaldepts do?
on index/manager/text i have

Code: Select all

'index.text.id' => array(
			'code'=>'index.text.id',
			'internalcode'=>'mindte."textid"',
			'internaldeps'=>array( 'LEFT JOIN "mshop_index_text" AS mindte
				USE INDEX ("idx_msindte_value", "idx_msindte_p_s_lt_la_ty_do_va") ON mindte."prodid" = mpro."id"' ),
			'label'=>'Product index text ID',
			'type'=> 'string',
			'internaltype' => \Aimeos\MW\DB\Statement\Base::PARAM_STR,
			'public' => false,
		)
This mean that aimeos run explicit index, but i cant extend the "standard.php" for elasticsearch.
A have to throw away all? Is ok, but zero adapter to use for elasticsearch?

Thanks again, but never used doctrine and is also my first time with elasticsearch :/

Re: elasticsearch?

Posted: 05 Jul 2017, 07:50
by aimeos
Doctrine DBAL is only used for SQL schema updates. The corresponding DBAL database layer is only necessary to do that.

For ElasticSearch, it doesn't make much sense to extend from the SQL based managers because document oriented databases work very different. "internalcode" is the mapping from the used search keys (e.g. "product.id") to the data structure of the storage (e.g. table.column for RDBMS). If the storage allows the same keys, you should use "product.id" an alike directly. "internaldeps" is for joining the tables in RDBMS. For document oriented databases, this wouldn't be necessary because all information would be stored only in one document.

You also have to create an adapter for the ElasicSearch query language like for SQL:
https://github.com/aimeos/aimeos-core/t ... W/Criteria

Re: elasticsearch?

Posted: 05 Jul 2017, 14:25
by BonoboMagno
I hoped to didn't have to do it from scratch :/

You use dbal for schema creation/update. But what you use for read the data? Can i switch to ORM ? maybe extending laravel version.
I think that is more futureproof using eloquent standard. You know, laravel is pretty much a monopoly on php framework. If one want speed, he can just pass to elasticsearch/angolia.

I have a question, how to call a product creation?
let see if i understand.
i see on routes that saveAction on JqadmController.php is called.
It call createClient with resource "product" that call the factory (nice pattern, but is better to explain what can builds before calling it)
that call the product factory, that call product/standard classes.

It's really long. And now, if i want to create product with a job, i have only to call the factory?

in this case,

Code: Select all

		$site = 'default;
		$lang = 'en';
		$resource = 'product'

		$aimeos = app( '\Aimeos\Shop\Base\Aimeos' )->get();
		$templatePaths = $aimeos->getCustomPaths( 'admin/jqadm/templates' );

		$context = app( '\Aimeos\Shop\Base\Context' )->get( false, 'backend' );
		$context->setI18n( app('\Aimeos\Shop\Base\I18n')->get( array( $lang, 'en' ) ) );
		$context->setLocale( app('\Aimeos\Shop\Base\Locale')->getBackend( $context, $site ) );
		$context->setView( app( '\Aimeos\Shop\Base\View' )->create( $context, $templatePaths, $lang ) );

		\Aimeos\Admin\JQAdm\Factory::createClient( $context, $templatePaths, $resource )->setAimeos( $aimeos );
	
but where are the post data?! no $request array. maybe on $resource = Route::input( 'resource' ); ?
what it have?

What are the api if i want to use the package only as backend companion?
Why laravel extension have only view resource files?! :/

Re: elasticsearch?

Posted: 05 Jul 2017, 21:01
by aimeos
BonoboMagno wrote: I think that is more futureproof using eloquent standard. You know, laravel is pretty much a monopoly on php framework. If one want speed, he can just pass to elasticsearch/angolia.
Laravel is just a PHP framework but e.g. no CMS. Aimeos is independent of any framework or application because in its core it's a PHP library that encapsulates a complete e-commerce system. Thus, Laravels Eloquent is not an option.
BonoboMagno wrote: i see on routes that saveAction on JqadmController.php is called.
It call createClient with resource "product" that call the factory (nice pattern, but is better to explain what can builds before calling it) that call the product factory, that call product/standard classes.
You can create a decorator like this one to push the product related data into ElasticSeach:
https://github.com/aimeos/ai-admin-jqad ... /Cache.php
BonoboMagno wrote: but where are the post data?! no $request array. maybe on $resource = Route::input( 'resource' ); ?
what it have?
You can get the GET/POST data using $view->param()
BonoboMagno wrote: What are the api if i want to use the package only as backend companion?
Why laravel extension have only view resource files?! :/
Can you please explain in more detail?