Home
> Firestore Secure Batch Increment
Get updates on future FREE course and blog posts!
Subscribe

Firestore Secure Batch Increment

25 min read

Jonathan Gamble

jdgamble555 on Monday, February 5, 2024 (last modified on Wednesday, February 21, 2024)

The classic way to keep track of the number of documents in Firestore is to use increment. This is also the necessary way if you want to subscribe to count changes or if you need to sort by the count. You will need to store the counter on an actual document. The total collection count is usually stored in a meta document like _counters/collection_name on a field called count. You will need to use atomic writing techniques with batch to ensure the two documents are changed in one transaction.

TL;DR#

Method ✅ Best For ⚠️ Avoid When
Batch Increment Large collection counts. Sorting by counters. Need for realtime counter. You need to display a count once on small collections. You prefer firebase functions over Firestore rules.

Count a Collection of Todos#

Let’s say you want to keep track of a todos collection count. This assumes the count is stored in the collection _counters/todos on a field called count.

Add a Todo#

You will want to increment the count field at the same time you add a new todos/{id} document.

	export const addTodo = async (text: string, uid: string) => {

  // todo and counter doc references
  const countRef = doc(db, '_counters/todos');
  const todoRef = doc(collection(db, 'todos'));

  const batch = writeBatch(db);

    // increment todo count
  batch.set(countRef, {
      count: increment(1),
      resourceRef: todoRef
  }, { merge: true });

  // add todo
  batch.set(todoRef, {
      uid,
      text,
      complete: false,
      createdAt: serverTimestamp()
  });

  // commit transaction
  batch.commit();
}

Delete a Todo#

You will want to decrement the count field at the same time you delete a todos/{id} document.

	export const deleteTodo = (id: string, uid: string) => {

  // todo and counter doc references
  const countRef = doc(db, '_counters/todos');
  const todoRef = doc(db, 'todos', id);

  const batch = writeBatch(db);

  // decrement counter
  batch.set(countRef, {
      count: increment(-1),
      resourceRef: todoRef
  }, { merge: true });

    // delete todo doc
  batch.delete(todoRef);

  // commit transaction
  batch.commit();
}

Note: For the increment to work correctly, you need to make sure merge is set to true on the counter document(s).

Referring to the Resource#

Notice there is an extra field called resourceRef. This ensures Firestore Rules knows which collection the counter is counting. It is required overhead for full security as you will see.

Firestore Rules#

Also in order to prevent an add post or delete post transaction from succeeding if the counter is not equally updated, you need to have good Firestore Rules. You don’t want a user adding a document without a count or vice versa. You have to ensure a todo document cannot be added or deleted unless the count field is incremented or decremented and that a counter cannot be incremented or decremented unless a todo document is added or deleted. This works both ways.

	match /todos/{document} {
  allow read;
  allow update;
  allow create, delete: if todoCounter();
}
match /_counters/{document} {
  allow read;
  allow write: if isCounter();
}

So, you need a security function for both use cases. Unfortunately, this is not an easy task. So, I created some helper reusable functions for any collection.

Helper Functions#

These are just general Firestore Rules helper functions that I created and find useful. I try to keep things as simple and easy to understand as possible.

	function docDeleted(path) {
  // 1 đź“– doc at path is deleted
  return exists(path) && !existsAfter(path);
}
function docCreated(path) {
  // 1 đź“– doc at path is created
  return existsAfter(path) && !exists(path);
}
function requestData() {
  // get request data
  return request.resource.data;
}
function createResource() {
  // create resource
  return request.method == 'create';
}
function deleteResource() {
  // delete resource
  return request.method == 'delete';
}
function refCollection(docRef) {
  // get doc reference collection
  return docRef[3];
}
function requestCollection() {
  // 3rd path is collection
  return request.path[3];
}
function requestDoc() {
  // 4th path is doc id
  return request.path[4];
}
function isIncrement(before, after, field) {
  // increment
  return after[field] == before[field] + 1;
}
function isDecrement(before, after, field) {
  // decrement
  return after[field] == before[field] - 1;
}
function isCreateField(before, after, field) {
  // field is created
  return !(field in before.data) && (field in after.data);
}
function isDeleteField(before, after, field) {
  // field is deleted
  return (field in before.data) && !(field in after.data);
}
function hasOnly(data, fields) {
  // data has only fields
  return data.keys().hasOnly(fields);
}
function isOne(data, field) {
  // field = 1
  return data[field] == 1;
}

Counter Base Functions#

These are the counter base functions. They determine if a document is a counter. If it is an increment or a decrement document, it must be a counter.

	function isCountDecrement(before, after, field, isDeleteDoc) {
  // field is deleted or doc is deleted from count = 1
  let isDelete = isDeleteDoc || isDeleteField(before, after, field);
  let isLastDecrement = isDelete && isOne(before, field);

  // field is valid decrement at 1 multiple
  return isDecrement(before, after, field) || isLastDecrement;
}

function isCountIncrement(before, after, field, isCreateDoc) {

  // field is created or doc is created to count = 1
  let isCreate = isCreateDoc || isCreateField(before, after, field);
  let isFirstIncrement = isCreate && isOne(after, field);

  // field is valid increment at 1 multiple
  return isIncrement(before, after, field) || isFirstIncrement;
}

function isCount(before, after, field, cCreate, cDelete, rCreate, rDelete) {

  // is increment or count create
  let isValidIncrement = isCountIncrement(before, after, field, cCreate);

  // is decrement or count delete
  let isValidDecrement = isCountDecrement(before, after, field, cDelete);

  // is valid counter with resource create or delete
  return (isValidIncrement && rCreate) || (isValidDecrement && rDelete);
}

Counter Resource Functions#

When you use any counter, you are coming from two directions. First you must check from the resource document being added perspective, then you must check from the counter document being incremented perspective. These are reusable counter functions so that you never have to think about this complexity again.

	function counterDoc(field, refField) {

  let before = resource.data;
  let after = request.resource.data;

  // doc created or deleted, only count as 1 đź“– on resource doc
  let rCreate = docCreated(after[refField]);
  let rDelete = docDeleted(after[refField]);

  // counter doc created or deleted
  let cCreate = createResource();
  let cDelete = deleteResource();

  return isCount(before, after, field, cCreate, cDelete, rCreate, rDelete);
}

function requestCounterDoc(field, path) {

  // counter doc created or deleted, only count as 1 đź“– on count doc
  let before = get(path).data;
  let after = getAfter(path).data;
  let cCreate = docCreated(path);
  let cDelete = docDeleted(path);

  // doc created or deleted
  let rCreate = createResource();
  let rDelete = deleteResource();

  return isCount(before, after, field, cCreate, cDelete, rCreate, rDelete);
}

Custom Counter Functions#

This is how you use the counter resource functions. You can declare you count field whatever you like, and the full path to your counter document. You can also declare your resource reference field.

	function todoCounter() {

  // config
  let countField = 'count';
  let countPath = /databases/$(database)/documents/_counters/$(requestCollection());

  // check for counter
  return requestCounterDoc(countField, countPath);
}

function isCounter() {

  // config
  let countField = 'count';
  let countRefField = 'resourceRef';

  // validate collection ref is same as count document id
  let validRef = requestDoc() == refCollection(requestData()[countRefField]);

  // can only have a count field and a count ref field
  let validFields = hasOnly(requestData(), [countField, countRefField]);

  // check for counter
  return counterDoc(countField, countRefField) && validFields && validRef;
}

This is Insane, what is all this!?#

In order to understand why you need all this, let’s take a look at the security you actually need.

Increment#

  1. The count on the count document is incremented at the same time a new todos document is created.
  2. Firestore Rules (FR) needs to simultaneously do the following:
    • On the todos document change, FR must see if a todo document is created. If the doc is created, FR must get the count document and make sure the count field on the counter document is being incremented by 1.
    • On the counter document, FR must see if the count field is being incremented. It must then fetch the resource reference document, the new todo document, and make sure it is being created (it did not exist before the transaction, but it does exist after the transaction). It must then make sure the reference document is referencing the correct collection (todos), and that there are no other fields on the counter document except count and resourceRef (or whatever you name it).

This is just for increment. The opposite must hold true for decrement; I’m leaving that out for brevity of this already long post. However, it gets even more complicated.

What is an Increment?#

An increment is defined as one of the following 3 situations:

  • A count document with a count field whose value is increased by 1.
  • A count document that is created with a count field whose value is equal to 1.
  • A count document on which a count field is added whose value is equal to 1.

While 99% of situations are going to be the first case, you still have to account for the other two use cases. Same goes for decrement. The document could be removed, the field could be removed, or the count could simply be decremented.

Other Security Rules#

You also need to protect the count document from having extra fields (only count and resourceRef). Lastly, you need to make sure the documentRef field points to the correct collection. Without documentRef, there is no way to verify the document id of the newly created field when verifying the count document. You could slightly modify the code to accept a document id instead if you like.

Price#

These Firestore Rules functions will cost you one document read for the added or removed document write, and one document read for the incremented or decremented count document write. Firestore Rules must confirm the before and after value. However, you will always only be charged one document read when you get the count, instead one of per 1000 results.

  1. Firestore creates the batch for count and todos
    • 1 write on count
    • 1 write on todos
  2. Firestore Rules match intercepts the write on the count document
    • 1 read on the todos document to confirm create or delete
  3. Firestore Rules simultaneous intercepts the write on the todos document
    • 1 read on the count document to confirm it is a valid counter

Note: Even though you call get, getAfter, exists, and existsAfter, on the same document, you only get charged one document read, as the document (and future version) is only actually being read once. Do not use these functions to get a resource or request.resource object, as you’re already on that path and have the object ready to go.

Filtered Counts#

You will also commonly store a subtotal, or individual queries. For example, you may store the total count for a user’s todos, separate from the total count for all todos. If you need to store both, your batching techniques will get complicated real quick. Usually a user’s count would be stored on the user’s document, while a total count would be stored on a counter document or a meta document. This batching gets a little more complex.

User’s Todo Count#

	export const addTodo = async (text: string, uid: string) => {

  const countRef = doc(db, '_counters/todos');
  const todoRef = doc(collection(db, 'todos'));

  const batch = writeBatch(db);

  // counter increment
  batch.set(countRef, {
      count: increment(1),
      resourceRef: todoRef
  }, { merge: true });

  // todo addition
  batch.set(todoRef, {
      uid,
      text,
      complete: false,
      createdAt: serverTimestamp()
  });

  // optional user count
  const userRef = doc(db, 'users/' + uid);
  batch.set(userRef, {
      todoCount: increment(1),
      resourceRef: todoRef
  }, { merge: true });

  batch.commit();
}

You simply add the userCount to the batch as well for both increment and decrement.

	export const deleteTodo = (id: string, uid: string) => {

  const countRef = doc(db, '_counters/todos');
  const todoRef = doc(db, 'todos', id);

  const batch = writeBatch(db);

  batch.set(countRef, {
      count: increment(-1),
      resourceRef: todoRef
  }, { merge: true });

  // optional user count
  const userRef = doc(db, 'users/' + uid);
  batch.set(userRef, {
      todoCount: increment(-1),
      resourceRef: todoRef
  }, { merge: true });

  batch.delete(todoRef);

  batch.commit();
}

Reusable Firestore Rules#

	match /users/{document} {
  allow read;
  allow write: if isUserCounter();
}

match /todos/{document} {
  allow read;
  allow update;
  allow create, delete: if todoCounter() && userCounter();
}

match /_counters/{document} { 
  allow read;
  allow write: if isCounter();
}

The match statement will check the todos and the users documents.

	function todoCounter() {

  // config
  let countField = 'count';
  let countPath = /databases/$(database)/documents/_counters/$(requestCollection());

  // check for counter
  return requestCounterDoc(countField, countPath);
}

function isCounter() {

  // config
  let countField = 'count';
  let countRefField = 'resourceRef';

  // validate collection ref is same as count document id
  let validRef = requestDoc() == refCollection(requestData()[countRefField]);

  // can only have a count field and a count ref field
  let validFields = hasOnly(requestData(), [countField, countRefField]);

  // check for counter
  return counterDoc(countField, countRefField) && validFields && validRef;
}

function isUserCounter() {

  // config
  let countField = 'todoCount';
  let countRefField = 'resourceRef';

  // validate count ref is in todos collection
  let validRef = refCollection(requestData()[countRefField]) == 'todos';

  // check for counter
  return counterDoc(countField, countRefField) && validRef;
}

function userCounter() {

  // config
  let userPath = /databases/$(database)/documents/users/$(request.auth.uid);
  let userField = 'todoCount';

  // check for todo counter
  return requestCounterDoc(userField, userPath);
}

As you can see, thanks to the reusable requestCountDoc and counterDoc functions, it is very easy to add a counter. The long crazy functions have already been declared. I spent tedious time creating these functions to make them easy. I firmly believe this is one of the best ways to create counters in Firestore.

Price#

  1. Firestore creates the batch for count, users, and todos
    • 1 write on count
    • 1 write on todos
    • 1 write on users
  2. Firestore Rules match intercepts the write on the count document
    • 1 read on the todos document to confirm create or delete
  3. Firestore Rules simultaneously intercepts the write on the todos document
    • 1 read on the count document to confirm it is a valid counter
    • 1 read on the users document to confirm it is a valid counter
  4. Firestore Rules simultaneously intercepts the write on the users document
    • 1 read on the todos document to confirm create or delete

Note: I also create my https://github.com/jdgamble555/j-firebase package to help with the frontend batching part, but unfortunately there are no Firestore Rules packages.

Node.js SDK#

If you want to disable writes on your frontend, you could get rid of the Firestore Rules complication all together and use the firebase-admin.

Example App#

Firestore Rules Counter Example

Repo: GitHub
Repo Firestore Rules: firestore.rules

I built a working Todo app. It is not deployed, but you can easily run it on your computer using Firebase Emulators. I also built the exact same app in my Firestore Server Side Counter post, and my Firestore Secure Batch Increment post.


Related Posts

© 2024 Code.Build