What is the best full text search engine for Python?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 75 | |
| Number of Parts | 169 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/21101 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201675 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
Search engine (computing)E-textEmulationSpecial unitary groupSineVarianceCodeProgrammer (hardware)Software testingBasis <Mathematik>2 (number)IdentifiabilityQuery languageProjective planeOrder (biology)Inheritance (object-oriented programming)Social classRevision controlComputing platformWeb 2.0Bit rateSearch engine (computing)InformationArithmetic meanPattern languageBitSelectivity (electronic)BenchmarkProcess (computing)Different (Kate Ryan album)Software developerTask (computing)Direction (geometry)Natural languageBlogSoftware frameworkVolume (thermodynamics)MathematicsData structure1 (number)Single-precision floating-point formatData storage deviceGodMusical ensembleGenderGraph (mathematics)Band matrixInformation securityParticle systemLecture/ConferenceComputer animation
04:25
ExplosionMetric systemRead-only memoryMemory managementBoundary value problemMathematical analysisPrice indexTerm (mathematics)Database transactionGroup actionLengthHydraulic jumpSpecial unitary groupTotal S.A.Database normalizationElasticity (physics)E-textComplex (psychology)Query languageTable (information)Population densityRankingVector spaceCovering spaceGlass floatFrequencyWordElectronic mailing listDisintegrationNormed vector spaceTerm (mathematics)Field (computer science)Revision controlElectronic mailing listRootResultantMatching (graph theory)Query languageTable (information)Limit (category theory)Database normalizationSearch engine (computing)Coefficient of determinationWordFreewareBitSubject indexingContext awarenessFunctional (mathematics)File formatMultiplication signBasis <Mathematik>Projective planeEndliche ModelltheorieRankingVector spacePerfect groupExtension (kinesiology)Different (Kate Ryan album)Elasticity (physics)1 (number)DatabaseNormal (geometry)Bookmark (World Wide Web)InformationInternet service providerPopulation densityMoment (mathematics)Data storage deviceType theoryNeuroinformatikGoodness of fitWeb pageSequenceAdditionForm (programming)Hydraulic jumpMoore's lawSpecial functionsMathematical optimizationSearch treeHash functionScaling (geometry)CASE <Informatik>Vulnerability (computing)GenderDiffuser (automotive)Graph (mathematics)System callOrder (biology)Information securitySparse matrixStreaming mediaQuicksortPerspective (visual)Data conversionContent (media)Film editingGraph coloringInverter (logic gate)Inheritance (object-oriented programming)Arithmetic meanServer (computing)Graph (mathematics)Line (geometry)Set (mathematics)Computer animationLecture/Conference
13:41
Quantum stateInterior (topology)Normed vector spaceUniform resource nameWeb pageObject (grammar)Elasticity (physics)Stack (abstract data type)Buffer overflowNebenläufigkeitskontrolleSynchronizationPrice indexWordWritingSource codeDatabaseTable (information)Server (computing)Attribute grammarField (computer science)Software testingQuery languageVideoconferencingLetterpress printingNP-hardOpen setScalabilityReal-time operating systemAnalytic setInternetworkingRow (database)AlgorithmType theoryQuery languageInstallation artJava appletScaling (geometry)Price indexDefault (computer science)Source codeDifferent (Kate Ryan album)DatabaseSearch engine (computing)Electronic mailing listWordImplementationResultantDirection (geometry)NumberData managementClient (computing)Square numberSequelPoint cloudElasticity (physics)Internet service providerSubject indexingVector spaceInformationMathematicsField (computer science)Attribute grammarCodeCombinational logicParsingGame controllerSet (mathematics)Server (computing)BitTable (information)Limit (category theory)Revision controlConstructor (object-oriented programming)Function (mathematics)Configuration spaceLine (geometry)Stack (abstract data type)Proof theorySoftware testingNatural languageRelational databaseOperator (mathematics)LoginConcurrency (computer science)Slide ruleBuffer overflowBoolean algebraMechanism designRankingFunctional (mathematics)Search algorithmSequenceString (computer science)1 (number)Computer iconBuildingArithmetic meanRight angleStaff (military)GenderTelecommunicationSpring (hydrology)QuicksortData structureStability theorySystem callParallel portCodeTheory of relativityMereologyExecution unitMusical ensembleNumeral (linguistics)OvalShared memoryForcing (mathematics)Rule of inferenceSocial classInstance (computer science)VideoconferencingWeightPoint (geometry)Computer animationLecture/Conference
22:51
WordLine (geometry)Letterpress printingRankingAlgorithmInclusion mapFunctional (mathematics)Search engine (computing)Source codeInformationMarginal distributionReal numberQuery languageTable (information)Pairwise comparisonSound effectWeightProjective planeSet (mathematics)ResultantAlgorithmData compressionMatching (graph theory)WordComputer animation
23:54
SynchronizationElasticity (physics)Client (computing)DialectInformation managementSummierbarkeitFermat's Last TheoremArmCurve fittingTrigonometric functionsMulti-agent systemSinePrice indexSubject indexingWordRankingConfiguration spaceSet (mathematics)Internet service providerDefault (computer science)Source codeSubject indexingDifferent (Kate Ryan album)Functional (mathematics)DatabaseCodeOpen sourceCASE <Informatik>Search treeReal-time operating systemClient (computing)GradientSolid geometryDialectElasticity (physics)Inverse elementDiffuser (automotive)Search engine (computing)Table (information)Endliche ModelltheorieAngleInformationQuery languageFront and back endsResultantArithmetic meanData structureBitPosition operatorSystem callIndependence (probability theory)AlgorithmGenderSpeech synthesisWordData storage deviceSemiconductor memoryTerm (mathematics)FrequencyUniverse (mathematics)Streaming mediaQuicksortMiniDiscObject (grammar)Musical ensembleTask (computing)Java appletPlastikkarteoutputParameter (computer programming)Open setWeightProjective planeInverter (logic gate)RankingPatch (Unix)Modul <Datentyp>MathematicsBuildingLecture/Conference
29:37
Gamma functionScale (map)Elasticity (physics)Metropolitan area networkEmulationDatabaseComa BerenicesAddressing modeMiniDiscSubject indexingCalculationMereologyDefault (computer science)DatabaseRevision controlWordPoint (geometry)Table (information)Electronic mailing listView (database)CuboidRankingGraph (mathematics)Mathematical singularityOffice suiteSlide ruleMatching (graph theory)WeightCombinational logicSoftware testingoutputParameter (computer programming)Subject indexingWater vaporExtension (kinesiology)Distribution (mathematics)Internet forumScaling (geometry)Physical systemInheritance (object-oriented programming)Elasticity (physics)ResultantSearch engine (computing)Set (mathematics)Control flowDampingMagnetic stripe cardWeißes RauschenAlgorithmDirected graphAddress spaceAssociative propertyTask (computing)Functional (mathematics)Query languageBeta functionWell-formed formulaVariancePlastikkarteReading (process)CASE <Informatik>Musical ensemble2 (number)String (computer science)Product (business)DivisorDemosceneInclusion mapScalabilityLink (knot theory)Computer animation
35:20
Electronic meeting systemSubject indexingScale (map)Elasticity (physics)Content (media)Letterpress printingField (computer science)Price indexTable (information)Attribute grammarWritingSource codeDatabaseReal numberConcurrency (computer science)Uniform resource nameCAN busQuantum stateWeb pagePersonal area networkMathematical singularityAddressing modeValue-added networkOperator (mathematics)Dependent and independent variablesMultiplication signRankingLibrary (computing)SurgeryBasis <Mathematik>Revision controlTheoryDifferent (Kate Ryan album)Term (mathematics)Client (computing)Element (mathematics)CASE <Informatik>Task (computing)ResultantExecution unitConfiguration spaceSearch engine (computing)SequelWeightData modelGraph (mathematics)Subject indexingElasticity (physics)Set (mathematics)Source codePhysical systemIdeal (ethics)MathematicsGroup actionSoftware testingCartesian coordinate systemMeeting/InterviewComputer animationLecture/Conference
Transcript: English(auto-generated)
00:33
tools for different Python frameworks Regarding to search engines. It's really difficult to find useful information
00:42
especially benchmarks comparing accuracy and and quality of the search And that's why it's really difficult to select search engine for your project if you start project or you continue project and The agenda is we are talking about a little bit about me. What is full-text search?
01:10
Different full-text search engines like Postgres, Elastic, Whoosh, Sphinx and Search accuracy and search speed and what's next
01:23
I'm both expertise in Python and Golang and I created with my friends Australian startup which helps create Check out very quickly. I speak here and I have blog I can't believe but 18 years ago. There was no Google and
01:47
other other web search engines were around back then and Asta, LaVista, Hotbot, Inkatomy and all the web search and And But what's more unbelievable is that 26 years ago
02:05
there was no web search at all and Now world is rapidly changed and the volume of information available and bandwidth gives us the opportunity to get this information but unfortunately the
02:21
processing rate which human being can Consume information does not change much and this inevitably means Transformed searching from something that only geeks ever care about to something that every single of us
02:41
to deal with on daily basis Let's start it from simple text search text search. It's not new problem and Every day every developer do something like I like to do we have C Python codebase or your project code base and we try to search all occurrences of order deep class and
03:05
The common tool for every Unix platform is grab and you see that you can find it less than three seconds on my laptop and If we try to do this Task using
03:20
ACK it's like improved optimized grab for programmers. It's a little bit less than two seconds and My favorite one but not super fast it's Python version of search through code It's like PSS. It's less than one second and the most my favorite one
03:42
It's Golang based platinum search Maybe everyone use it in V or yeah marks because it's super fast. It's like 50 14 less than second its characteristic where made tests
04:02
But okay, it's it's direct search It's when you have pattern and search it's it's simple problem but what if we talk about full text search and Full text search provide capability to identify natural language documents and satisfy
04:20
a query and sorted by relevance to the query and If you plan to read any books in the end you can find search index and it's by the way search index of one of my favorite book about elastic search and
04:41
The purpose of certain index is to optimize performance and without index the search engine win will scan every document and corpus which would require considerable time and computing power for example while an index of
05:04
10,000 Documents can be acquired with milliseconds Sequential scan every word in 10,000 documents will will take hours
05:20
Advantages disadvantages of index is additional computer computer storage required to store this index and Time to Let's imagine we have simple example with two sentences and we try to build
05:48
Inverted index its common term in full text search for these two sentences and First we split the content field on each document by on word basis and
06:03
Then we create sorted list Like you see in the first column its term like quick they're brown Etc. And then we mark each a currency in each document and place where it's a cure for my example, I exclude the places but only the fact that
06:25
Term exists in document and If we try to search query using quick brown you see that our table or our inverted index can show for us like
06:42
Accuracies brown a quick and we see the brown existing to document documents and quick is only in one But you can you can you can find that I have a little bit redundancy in my index That's why we should apply normalization It means that we should
07:02
lowercase quick Pluralized dogs Using root forms of verbs etc and maybe use synonyms Like sorry like jump and leap Okay, let's talk
07:22
About what search engines we have now at the current moment. So lots of different types of search engines But today we will talk about only four of these it's Postgres full-text search elastic search Python search whoosh and Sphinx
07:43
Let's start from Postgres full-text search Everybody use Postgres. It's Was created by Michael Stonebrake for eight years in 1986 and the interesting fact that full-text search
08:03
supported from version 8.3 it means that you can use it in every project because I'm not sure that you have less versions and eight and Last table version is nine dot five and lots of free advantages for these database
08:23
Let's see example. We have simple query Simple text and we try to full text search through this query In context of Postgres, we should use two functions it's text search vector its format when we transform your data and
08:44
special function text search query and the results will be Looks like this you see that This results return true it means that we find the results of our search
09:03
Okay next we All full text search it's about indexes I mean if you want to understand how it's work You should understand how indexes works and Postgres provide two kind of indexes first
09:20
It's generalized inverted index that I shown before in example and second its generalized search tree based and The last one is a little bit lossy because the index might produce false matches because it has very limited hash function for
09:46
For search for text we should try to search it means that it can represent the same phrase With the same ID and you can find false matches, it's why it's a little bit not recommended but
10:01
the difference is Between this index is very simple when you have data which static it means that it's change Not so often that's why you can use first one if you have dynamic data Which change every day every minute every second and you try to search you should use generalized
10:24
search tree next Important it's ranking search results. It's how To measure how relevant documents are to particular query So when there are many matches the most relevant ones can be showing first sometime it's very useful and
10:48
Postgres provide two C based functions. It's rank and rank close density something like it and You can cover density and you can
11:03
You can use it and I have some small example for you How to use it it's next after this slide Last but not least. It's highlighting results every user wants to see what he search and see what
11:25
occurrences of what he tried to search and for this Postgres provide headline function, which were easy you just Use this function and it will mark Your results with some HTML tags or etc
11:47
Also very important it's stop words stop words. It's like English words, for example, which Unuseful or uninformative etc. Etc. It's like it is
12:02
and working with stop words also included in Settings of Postgres and when you apply using text search vector function you can see that text search vector applied to your column which have some information and
12:21
As a result you see like special format for Postgres where you see only useful information exclude stop words like list stop word instead of Because in there of unnecessary no need to search using this word in some cases
12:44
Next a little bit about Python Postgres full text search Provide for Python if you use Django It's good news because in Django 1.10 already added Postgres search
13:02
Functionality which Rely only Postgres full text search engine and it means it will be super fast if you use it in your project old version of Django model is Django a RAM extension is written a couple years ago and it's working
13:22
Perfect with old version of Django The third one is using a scale alchemy Some example how to apply to your project if you already have some Model which called page you can just create search index. It's
13:41
special field and Override your search manager where you should add configuration and search field and After update search field means on each safe update or delete Index will automatically update it by Postgres and
14:01
as a result you can do some very common Oram queries using keyword search you just use search and you can search documentation and about only limitations that Postgres provide and as a result Django also, it's very
14:21
simple Query Construction mechanism it means you can use only two boolean operators It's and or or and you see that in second example I provide example with about or document or Django etc
14:42
According to Django 1.10 itself you can As you can understand they edit by default using underscore underscore search for each field, that's why you can also use it without any installation like I show in previous slide and
15:06
Or you can annotate it with search vector and Filter by cheese and see results. It's awesome because it will convert it for direct Text search query text search vector SQL query and Postgres will execute it very fast
15:24
and examples Yes, this commit was made by I'm not sure but couple months ago. It's super fresh information There is no any documentation about this only in source code in this commit. You can find it
15:41
Maybe they will update it, but I'm not sure that it's already done Okay, let's talk about Finish with Postgres full text search we have pros like quick implementation. You saw it no dependency Maybe
16:00
Disadvantages is need manual manage indexes because it's not done automatically dependent Postgres if you use my sequel It will not work if you use another database it will not works No analytics data what I mean about this is this it means I can't get
16:25
analytics on Search from Postgres I can only search and that's all if I want to get some important natural language text data, I can't do it and Very
16:42
Simple query builder Okay, let's continue with elastic search Elastic search is distributed scalable real-time search and analytics engine It's very important because it's enable us to search analyze and explore your data
17:03
it based on Apache Lucene search index, which Now is The most advanced and high-performance in the Internet Who use elastic search it's github use elastic search to query
17:23
130 billions of line of code you every day do it stack overflow use Combine full text search with geolocation. Sometimes it's very useful Guardian parse logs like lots of companies and
17:41
Wikipedia try to provide full text search with highlighted data and data doc cloud and other The idea was elastic search very simple. It's It's not quite Equals, but it's like in parallel. You can understand how it works
18:02
You have relation database you have elastic search you have database you have indices you have rows you have types columns equals documents tables equals fields The most Important it's maybe locks elastic search use optimistic concurrency control. It means when you try to change
18:23
document in elastic search they just updated and update Version of this document and it means when you search proof For some document it will use the last version of the document
18:41
According to elastic search there are lots of Python clients. It's default Python client new version made by Honza krall with I think I owe and Also DSL when you can build your queries if you work with elastic, you know how it's difficult sometimes
19:00
Work with these big JSONs and manipulate with it. It's annoying and on the create DSL and it's Looks pretty awesome Some examples you can get data You can create index with number of shards number of replicas. You can scale it
19:22
You can add JSON to index. It's just like how you create data for your index You can manage stop words For instance, you can add list of stop words you can highlight results
19:41
My favorite feature you can select thank you because sometimes it's also useful to select your predefined tag not use default one and Relevance relevance is you can explain query and you see what's wait and I Remove lots of details, but it's big big explanation
20:03
Why this query returns these results and which each value of weights? And I like it. It's difficult to do in Postgres, but it's really Easy to understand and calculate why these results
20:21
First if you for example, override your relevance function or rank function, etc Okay Next very quickly it's things I only put on slide differences things written in C++
20:41
C++ I found back and It's used for example my scale as data source and In comparing with elastic it's written not in Java and Swings assumes that you already have my scale database and all other stuff based on my scale
21:03
But it's not like mandatory. You can use Postgres. You can use any provider and About swing search server a little bit differences. It's DB table. It's things index DB rows It's swings documents and DB columns. It's things fields and attributes
21:24
it's not simple similar to Postgres and elastic search maybe seem similar to elastic search and Query language, it's not Sequel it's swing square language, but it's very similar to
21:42
Default SQL and you can find from your test one It's index name where much you have a Python and it will looks something like it It's I mean them I put only differences all other stuff very similar to elastic
22:01
And last but not least. It's pure Python whoosh Which? Created by Matt chaput his idea was like, okay my clients have no ability to install Java and That's why he create full text search engine in pure Python and it's not super fast
22:24
but in comparing with another pure Python search engines, it's super fast and It has pluggable Scoring algorithms you can add lots of and configure lots of stuff and By the way more information you can find on his talk
22:43
I'm not ready to repeat it and some small examples whoosh Depends a little bit on Postgres because it used Postgres for example stop words and it's create frozen set dynamically for Postgres stop course, but you can select any set of stop words
23:03
I mean, it's just example from source code. You also can highlight search results assumes that we hit we have hits in title and The most interesting it's it use best match 25
23:21
algorithms, which It's by the way, it's ranking function Which used to search engines to rank matching documents according to relevance to given by search queries it's the common use algorithms and It was created developed in 1970s I hope and
23:42
Now I created some compression table for you because when I started work on my first project with full text Search it was difficult to understand lots of information and how to structure it Is it why I created some table where I can
24:01
Find and you see that Python 3 support most search engines swings open PR You have lots of clients you you can use this table like reference Interesting that Postgres and elastic
24:20
Have both are thin clients swings and whoosh no and I added Django just for example, if you use Django sometimes you need some Rams etc. That's why you can find a stack very useful and
24:43
but talking about his deck, it's like provides modular search for Django and It's create one API layer Under couple of different different search engines and provide you
25:00
Django Like Django or RAM functionality for search, but I can't believe that it's really useful When you have project you create search full text search you apply Hey stack, and then you decide okay tomorrow I will be use elastic search today solar to and the day after tomorrow whoosh and etc
25:22
It's strange because it's like only very very simple Set of features all other features different in hand stack that's why I called a stake like Swiss knife and it's useful but Not for specific tasks and
25:40
I created small Prons and cons for you about a stack. Yeah, it's easy to sit up looks like Django or RAM Search engine independent support now for engines If we go deeper search query set API very poor, I mean, it's it's very poor you can't
26:04
create very smart queries Difficult to manage stop words because you need go to search engine back end and do it by by hand by yourself Hey step doesn't care about it Lose performance because you need like convert results to search query sets and work with it
26:26
maybe memory and model based it means that most the last full text search engines try to promote No school concept and when you have like object or when you have
26:41
document not model not one table, that's why it's a little bit difficult and The most I Think the most ugly with haystack It's lots of hard coded sent in settings in search engine if you open
27:00
Source code of haystack you can find hard coded elastic search settings Hardcoded settings for solar etc. And it's annoying if you want to change something You need to change haystack or patch it or something like it Let's continue with my table Next very difficult and interesting things which index each search engine use and I put
27:28
Like elastic user patch Lucina, you can find more information about it. It's default Inverted index As I said before postgres use generalize inverted index and generalized search trees
27:46
Sphinx Has Sphinx has three opportunities its disk indexes real-time indexes in distributed by the way distributed index It's just like container for lots of disks and real-time
28:02
indexes it's how you can scale your Sphinx and who's use very simple index folder as I Said before guy who create who she said just you have only Python and folder without any database Java etc. That's why he
28:21
Use simple approach and last column. It's interesting Sometimes when you have database you need to search In the memory without creating index and it's possible only for postgres I like this feature because you can use it in all Databases no need to create if you want you just could need to create index, but you can search all other
28:47
Search engines you need to put get data from data source put it to index build index and then you only then you can search but Postgres can do it in real time for you
29:03
Next interesting it's ranking relevance and Etc It's how which? probability algorithms each engine use for search elastic use very common turn frequency inverse document frequency
29:23
It means how often your term or your query acute in the whole document database and According to postgres. We already talked about CD rank you can It's interesting and you can put some weights for CD rank like input parameter
29:42
but it's you you can influence to CD rank formula how to calculate rank just only some parameters Sphinx, it's cool because lots of lots of Variance by default it use two factors first. It's
30:05
Major part it's approximately between the document text in the query it's called like longest longest common subsequences or something like it and Very common known best match 25 and who's use from my point of view the most
30:24
smart Relevance because it's improved best bunch 25 but interesting that you can replace any relevance Function to push and Sphinx has big table of a lots of formulas how to I mean
30:44
You also can configure it not like for Postgres or elastic search. You can't do it According to configure stop words. You can do it in all engines You can highlight search results all engines. It's like common features that you need
31:05
Sometimes it's useful to use synonyms and you can find that all This engine supports synonyms only huge but you can do it manually to replace words Or create dictionary which associate like one word with set of words like synonyms
31:26
About scaling I would like to say that The most scalable it's elastic search because it works from scratch and you can use it For Postgres, you should think about partitioning table inheritance, etc
31:42
About swings. I already said that it's used distributed searching and you can include lots of indexes in distribute index and It's how you can do it manually who does not support any scales and In the end, I would like to present for you
32:01
Some low tests that I made in real production I have 1 million music artists and I put it to each search engine and I try to search because most of low tests that I found for search engines use like White noise they generates
32:21
Like combination of letters and try to search. It does not make any sense and performance result, it's interesting because If data I put it In one table, for example for Postgres and Postgres after when I create index Postgres return for fun
32:44
Four milliseconds it last version the last last The latest which I found like nine Dot-six beta or something like it elastic return in nine milliseconds. It's also pretty awesome
33:00
Sphinx return in six milliseconds, but I'm not sure that I configure it correctly that's why maybe some results not super useful and who's also Has less performance The question only if you have more data, which not putting in
33:21
Postgres my next test Task for me. I plan to do small smart queries and I have database with 300 million records, which I'm not I'm not sure that I can put in one table in Postgres and etc and Yeah, maybe results will be different
33:42
In the end I would like to propose you to read some books which I found very useful for me about Elasticsearch the scene if you interesting in strings if you're interesting in swings and Very cool book about
34:01
Don't break which called red book about database systems I created some list of references for you because it's really difficult to share with you details of each index and you can find it in some very useful links and read about it because when you're
34:23
Stacled and your customer decide. Okay relevant should work I mean indexing should work that or these you can read about each index and find for you in which case your index will be more Efficient
34:41
Also about ranking ranking is Really difficult part. That's why I also put it to links you you can read of about each scoring how it's calculated and etc, etc because performance Will depend on two big
35:00
factors first, it's ranking algorithm because you should calculate ranks and Second it's indexing how you build your index and Thank you slides you can find on this link and Thank you for your attention and we hurrying and question, please
35:24
Well any questions I
35:41
Got a questions about operators in the jungle full-text search You mentioned that there are only and and or operators Can we combine it? I mean John and though or Yes, the full and bar. Yes By the way, it's feature. Not about Django its feature. I will show its feature of maybe this like it's
36:06
its feature of Postgres Okay All other questions Yeah, please
36:20
And what's a good way to compare the performance of different search engines not in terms of Speed of response but in terms of the quality of ranking. Yeah, it's I understand. Thank you for question It's what I am doing on everyday basis. I work with
36:41
our application not just full text search through data we try to Much users and etc by his interests and it means that The ranking it's very important for me and I have lots of tests for that how I I built very big queries
37:02
with and or we see no names without see no names, etc, and I Prepare expected result manually and I run my test and see results and etc. Yeah, it's Like only manual unfortunately work. It depends on your real task
37:24
All other questions apart from haystack. Do you have any recommendation from Tango and elasticsearch?
37:42
Django and elasticsearch for combining with those two from my experience, it's like Using Just Python client. I mean you can create manage task which Will refresh your index if you plan to get data from I mean
38:03
maybe you plan to store your data in Postgres or my sequel and You need like on some action you plan to refresh index and search from elasticsearch I found great solutions that you can just use simple Python
38:22
Client elasticsearch DSL or elasticsearch.py which on the crown maintain mostly and just but only add Manage by common to refresh index create like Asynchronous tasks for refresh index etc. If you plan to use haystack
38:43
You can I Don't remember the name but I found interesting library which override some settings from haystack and you can like add your synonyms change configurations, etc, and I recommend to use it if you plan to use haystack, but
39:05
problem of haystack that it's not support last version of elasticsearch and you You you will tackle on I don't know one dot seven dot five or something like it. Yeah
39:20
Yeah, please Is there a reason why you haven't talked a lot about solar? Could you please repeat? Is there a reason why you haven't talked a lot about the solar? search engine Solr. Ah, so I have no experience with soul, but I hope that it will soar also use
39:46
Lucien and Only difference is that I know solar is not easy to scale you can but it's not so easy and If you already use sword Maybe you should continue but for new projects, I think I found elastic more like
40:05
Useful for me something like it Okay Thank you very much