Home
>
Firestore Fuzzy Full Text Search

Firestore Fuzzy Full Text Search

8 min read
Jonathan Gamble

jdgamble555 on Sunday, October 2, 2022 (last modified on Sunday, December 17, 2023)

We need to be able to search Firestore without a third-party database, so I created a way. It pains me that Google recommends an external search engine other than a google product for this.

Current Options

Google Recommends 3 of these, but not even Big Query.

The problem is that all the options cost money and time. You have to self-host and manage another database, or you have to pay money and manage another database. I have heard Algolia is way more expensive. They are also all noSQL databases, so this won't help with the join issues. If you had to pay for a third-party database, I would recommend RedisGraph. Redis can get very expensive, but the speed and relational queries alone would make it worth it.

So, I wrote a frontend package that helps with this: j-firebase. I will be eventually adding more useful functions to this package.

Installation

Install the package in any frontend framework of your choice.

	npm i j-firebase

Usage

Creating the Index

Add the index for the data using searchIndex after you have added / updated your document. It will automatically delete the old indexes.

	import { searchIndex } from 'j-firebase';
...
await searchIndex({
  ref: doc(this.afs, 'posts', id),
  data,
  indexFields: ['content', 'title', 'tags']
});

The only variables that you need are ref, data, and indexFields. The data gets saved to the collection _search/{your-collection}/_all. You can change these options.

	 * @param param: {
 *  ref - document ref
 *  data - document data
 *  del - boolean - delete past index
 *  useSoundex - index with soundex
 *  docObj - the document object in case of ssr,
 *  soundexFunc - change out soundex function for other languages,
 *  copyFields - field values to copy from original document
 *  searchCol - the collection to store search index
 *  allCol - the search sub collection to store index docs
 *  termField - the document field to store index
 *  numWords - the number of words to index in a phrase
 * }

I retired my old backend package, adv-firestore-functions, which had a lot of Firebase Functions tools, due to the problems keeping up with it after Firebase 9 changes. However, I prefer to do these things on the frontend these days.

Using the Index

Here is a sample function for searching using searchDocs:

	import { searchDocs } from 'j-firebase';
...
const data = await searchDocs<Post>(
  collection(this.afs, 'posts'),
  term
);

Basically, you just pass in the collection reference and the term you want to search. Simple!

Here are the parameters:

	/**
 * @param collectionRef - the collection reference
 * @param term - the phrase you're searching
 * @param param: {
 *   searchCol - the search collection indexed
 *   allCol - the sub search collection indexed
 *   idField - the name of the id field to return
 *   termField - the term field that is indexed
 *   soundexFunc - the soundex function to use
 *   filters = other query constraints to add (where, startAt, etc)
 * }
 * @returns search document references
 */

Note: Both searchDocs and searchIndex throw errors for error checking, just like other Firestore functions.

Security

You need to make sure you secure your Firestore Rules so that only the resource owner can create an index, and the index matches the document. This assumes your document has a authorId on it, so change that value to fit your field.

	function searchIndex() {
  let docPath = 
/databases/$(database)/documents/$(request.path[4])/$(request.path[6]);
  return get(docPath).data.authorId == request.auth.uid;
} 
match /_search/{document=**} {
  allow read;
  allow write: if searchIndex();
}

Possible Todo List

I need to get this website going, but I have a few possible additions I want to make to these functions:

  1. Work server-side and client-side (possible without over importing?)
  2. Add trigram search options
  3. Option to add index on original document
  4. Better documentation
  5. Built-in soundex for other languages
  6. Stop words maybe?
  7. Add observable version
  8. Other types of search (ideas from ts_vector?)

I may never get to this, so pull requests are welcome.

Until then,

J


Related Posts

© 2024 Code.Build