I tried a few searches: $\int_{-\infty}^\infty e^{-x^2}dx$ was good, $\mathbb{R}P^\infty$ was not bad, $M\otimes N$ gave lots of hits for $MN$ or $m\times n$ which is not so useful, $\Omega S^3$ gave results which made no sense to me at all. Still, the basic framework looks very promising, and I am sure that the algorithm can be tweaked. — Neil Strickland 1 hour ago
1:35 AM
BTW search for $xy^3$ gives rather similar results to the previous search. Which should be a partial explanation for the "results which made no sense".
5 hours later…
6:50 AM
I have also tried a few other searches with adding some keywords (not formulas): search for $\Omega S^3$, Omega, search for $\Omega S^3$, homotopy, search for $\Omega S^3$, homotopy, sphere and
Only the last one found this question: Is there a rational homotopy equivalence between $\Omega S^3$ and infinite complex projective space? (But the query already contained two words from the title.)
7:18 AM
7 hours later…
2:07 PM
Good question, it is also interesting to explain why you only get the wanted post by searhcing so many query keywords though.
2:28 PM
First, to answer Neil Strickland's question, recall that Approah0 is trying to evaluate similarity between query and hits by two perspectives: One is symbolic similarity, the other is structural similarity. The possible docs that match $\int_{-\infty}^\infty e^{-x^2}dx$ is relatively less than that matches $M\otimes N$, or $\Omega S^3$. Because "ab", "cd", "x \otimes y" they all match $M\otimes N$ structurally. So the only measure to pick who is a better match is to evaluate symbol similarity.
And also, currently Approach0 does not differentiate \times and \otimes, they are both TIMES token (github.com/approach0/search-engine/blob/master/tex-parser/…)
3:04 PM
As for $\Omega S^3$, well, after I login onto my server and investigate for a while, I think you guys help me find a problem. I would not say it is a bug, it is because my ranking formula is not perfect.
currently approach0 uses a ranking formula like this:
document_score = proximity_score + (math_score * bm25_score)
document_score = proximity_score + (math_score * bm25_score)
where proximity_score is how proximate your query keywords occur in this document, the more far away, this score goes lower. The purpose of this score is, for example, if the user enters "hello world" as query, then document with a "hello world" should rank higher than document with a "hello Alice, you are my entire world."
bm25_score is, for simplicity here, you can think of it as the similarity degree of English words between query and document.
The wanted post has been assigned docID 421893 in my index. To illustrate the problem with current ranking formula, I randomly choose a document which is ranked top 10th on the first page of current result when you search $\Omega S^3$,, it has been assigned docID 58914.
As you can see, although 421893 has a higher math score (which is expected), its prox_score is negative 1.203973.
3:59 PM
I changed one number in my code and this issue gets fixed: github.com/approach0/search-engine/commit/…
4:16 PM
« first day (100 days earlier) ← previous day next day → last day (2660 days later) »
Transcript for
Jan7
Jan '1710
Jan14
In the search of a question
When you are looking for a specific question (using Approach0 ...