Indexes in the Frontend

Whiteboard Arguments - Part 1

My colleague and I have a habit of talking things out in front of a whiteboard until one of us can't argue anymore. Not every decision makes it through intact. Whiteboard Arguments is where we write up the ones worth sharing. The debates that changed the design, the ideas that got cut, and the problems that didn't have an obvious answer until we put them on the wall.

The Debate

This one started less like a whiteboard argument and more like a slow burn.

I had been quietly building something for a few weeks. Mentioning it in passing. Dropping hints about index maps and typed arrays and working sets. My colleague was heads-down on a different part of the project, a mobile companion to the same feature, and mostly nodded along without really engaging.

Then I showed him a working version. It wasn't done, but it was already a complicated system.

"That's a lot. Do you need this? I mean, seems cool, but?"

That's the whiteboard argument. Not a theoretical debate between two experts who already knew the answer. A real challenge in front of running code I was still learning to justify myself. I believed the design was right. I wasn't yet sure I could prove it.

We pulled up to the whiteboard. Drew out the data model. Boxes for each dimension. Arrows to the fact table. Mapped out the access patterns, the filter paths, the views that needed to materialize from the same underlying data. That whiteboard is still on the wall. The dims and facts are still exactly where we drew them.

Some things survived the argument. Some things got cut. The result was better than what either of us would have built alone, which is the whole point.

The Problem That Started It

The UI had a data shape problem that most frontend developers don't run into often, but that anyone who has worked with reporting systems will recognize immediately.

There were multiple dimension-style collections, reference data that describes who and what. And one fact-style collection, transactional records that reference those dimensions. The UI needed to filter across all of them simultaneously, project those filters into multiple views, aggregate values, resolve overrides by date, and do all of this reactively as the user interacted.

If you've ever worked with a data warehouse, you know this shape. Dimensions and facts. The fact table is wide and dense. The dimension tables are smaller but referenced constantly. Every meaningful query touches both.

The problem is that most frontend developers, myself included until this project, reach for the same tools regardless of the shape of the data. An array for each collection. A find() or filter() when you need something. Maybe a Redux slice if it feels important enough.

That works fine until it doesn't. And on this UI, it wouldn't. Getting my colleague to agree on that was the first argument.

The Naive Approach and Where It Breaks

To understand why the engine exists, it helps to see what the alternative looks like.

const products: Product[] = [...];
const warehouses: Warehouse[] = [...];
const suppliers: Supplier[] = [...];
const movements: StockMovement[] = [...];

function getProduct(movement: StockMovement): Product | undefined {
  return products.find(p => p.id === movement.productId); // O(n) every time
}

function applyFilters(movements: StockMovement[], filters: FilterState): StockMovement[] {
  return movements.filter(m => {
    const product = products.find(p => p.id === m.productId);      // O(n) inside O(n)
    const warehouse = warehouses.find(w => w.id === m.warehouseId); // O(n) inside O(n)
    const supplier = suppliers.find(s => s.id === m.supplierId);    // O(n) inside O(n)
    return matchesFilter(product, warehouse, supplier, filters);
  });
}

For a small dataset this is fine. For a dataset with thousands of movements referencing multiple dimensions, filtered and re-filtered as the user interacts, this compounds fast. You're scanning the full collection on every lookup, inside a loop, on every filter change.

The bigger problem isn't even performance. It's that this structure makes it easy to keep adding find() calls without ever noticing the cost accumulating. You don't hit a wall, you just slowly sink.

This was the easiest part to argue. Once you draw it on a whiteboard and count the passes, the problem is obvious. What wasn't obvious was what to do about it.

Index Everything You'll Look Up

The answer I came to, partly through research, partly through trial and error, was to treat the frontend data layer like a database. If you're going to look something up repeatedly by ID, build an index.

// Build once when data loads
const productByKey = new Map<number, Product>();
const warehouseByKey = new Map<number, Warehouse>();
const supplierByKey = new Map<number, Supplier>();
const movementsByProductId = new Map<number, StockMovement[]>();
const movementsByWarehouseId = new Map<number, StockMovement[]>();
const movementsBySupplierId = new Map<number, StockMovement[]>();

for (const p of products) productByKey.set(p.id, p);
for (const w of warehouses) warehouseByKey.set(w.id, w);
for (const s of suppliers) supplierByKey.set(s.id, s);

for (const m of movements) {
  const byProduct = movementsByProductId.get(m.productId) ?? [];
  byProduct.push(m);
  movementsByProductId.set(m.productId, byProduct);

  const byWarehouse = movementsByWarehouseId.get(m.warehouseId) ?? [];
  byWarehouse.push(m);
  movementsByWarehouseId.set(m.warehouseId, byWarehouse);

  const bySupplier = movementsBySupplierId.get(m.supplierId) ?? [];
  bySupplier.push(m);
  movementsBySupplierId.set(m.supplierId, bySupplier);
}

// Every lookup is now O(1)
function getProduct(id: number): Product | undefined {
  return productByKey.get(id);
}

function getMovementsForWarehouse(warehouseId: number): StockMovement[] {
  return movementsByWarehouseId.get(warehouseId) ?? [];
}

You pay a cost upfront when data loads. Every lookup after that is O(1). This was the clearest part of the whiteboard session. Once the indexes were drawn next to the naive approach, the argument basically made itself. Trade memory for speed. Pay once, spend nothing on every lookup after.

My colleague's question shifted from "do you need indexes" to "do you need everything else."

Typed Arrays for Hot-Path Lookups

One pattern I landed on early, partly from research and partly from an AI collaboration that saved me several hours of dead ends, was using typed arrays to store dimension keys parallel to the movement array.

Instead of storing the full dimension object on each movement, or looking it up through a Map on every iteration, you keep a Uint32Array where index i holds the dimension key for movement i. Typed arrays are cache-friendly, memory-compact, and fast to iterate in tight loops.

const productKeyByMovementIdx = new Uint32Array(movements.length);
const warehouseKeyByMovementIdx = new Uint32Array(movements.length);
const supplierKeyByMovementIdx = new Uint32Array(movements.length);

for (let i = 0; i < movements.length; i++) {
  productKeyByMovementIdx[i] = movements[i].productId;
  warehouseKeyByMovementIdx[i] = movements[i].warehouseId;
  supplierKeyByMovementIdx[i] = movements[i].supplierId;
}

// Hot loop: typed array access + O(1) Map lookup, no object traversal
for (let i = 0; i < movements.length; i++) {
  const productKey = productKeyByMovementIdx[i];
  const product = productByKey.get(productKey);
  // ...
}

I'll be honest, this was the part of the design I was least able to defend in the moment. I believed it was right. I understood the concept. But I was still wrapping my head around the implications. Keeping the typed arrays in sync with the movements array as mutations happen is not trivial. The argument here wasn't resolved at the whiteboard, it was resolved over a few days of implementation. The working version did the talking.

The Filter: One Pass, Everything at Once

This is the part I'm most proud of, and also the part that would have been hardest to justify before it existed.

The UI has multiple filter dropdowns, products, warehouses, suppliers, companies, and more. They cross-filter each other: selecting a warehouse should narrow the supplier list to only suppliers who have movements in that warehouse. And when filters are active, the engine needs to produce both the eligible set of movements for the working set and the narrowed options for every dropdown, simultaneously.

The naive approach runs a separate pass for each concern. One pass to determine eligible movements, one pass per dropdown to narrow options. The passes multiply as filters are added.

Before getting to the solution, there's a design decision worth naming explicitly: you have to choose a unit of filtering. In our domain, we had multiple dimensions, but one of them was the natural join point. The entity that had meaningful relationships to all the others and appeared across every projection. Choosing to iterate over that dimension, rather than over the fact records themselves, was a domain design choice. It's not automatic. The wrong choice here breaks the whole pattern.

With the right unit chosen, the solution was a bitmask approach. Here's what that means.

A bitmask uses a single integer to represent multiple boolean states at once. Each bit position in the integer corresponds to one category. You set a bit by OR-ing the integer with that bit's value. You check whether all bits are set by comparing the integer to a constant that has all bits on.

// Each filter category gets its own bit position (powers of 2)
const B_PRODUCTS   = 1;   // binary: 00000001
const B_WAREHOUSES = 2;   // binary: 00000010
const B_SUPPLIERS  = 4;   // binary: 00000100
const B_COMPANIES  = 8;   // binary: 00001000

// ALL has every bit turned on
const ALL = B_PRODUCTS | B_WAREHOUSES | B_SUPPLIERS | B_COMPANIES;
//        = 00000001 | 00000010 | 00000100 | 00001000
//        = 00001111

For each record in the dimension iteration, you build a mask by OR-ing in each bit the record satisfies:

let mask = 0; // 00000000, nothing passes yet

if (!hasProducts  || activeProducts.has(record.productId))    mask |= B_PRODUCTS;
//  mask = 00000001 if record passes the product filter (or no product filter is active)

if (!hasWarehouses || activeWarehouses.has(record.warehouseId)) mask |= B_WAREHOUSES;
//  mask = 00000011 if record also passes the warehouse filter

if (!hasSuppliers || activeSuppliers.has(record.supplierId))  mask |= B_SUPPLIERS;
if (!hasCompanies  || activeCompanies.has(record.companyId))   mask |= B_COMPANIES;

If the record passes every filter, mask === ALL. That's your eligible set check:

if (mask === ALL) eligibleKeys.add(record.id); // 00001111 === 00001111

Now here's where the bitmask earns its place. Cross-filtering, narrowing a dropdown to only show options that are still valid given the other active filters, requires checking whether a record passes all filters except its own category. You get that by AND-ing the mask with the bitwise complement of that category's bit:

// A product option is eligible if the record passes everything EXCEPT the product filter.
// ~B_PRODUCTS flips all bits: 00000001 becomes 11111110
// (mask & ~B_PRODUCTS) === (ALL & ~B_PRODUCTS) checks: does mask have all bits except B_PRODUCTS set?

if ((mask & ~B_PRODUCTS)   === (ALL & ~B_PRODUCTS))   optProductIds.add(record.productId);
if ((mask & ~B_WAREHOUSES) === (ALL & ~B_WAREHOUSES)) optWarehouseIds.add(record.warehouseId);
if ((mask & ~B_SUPPLIERS)  === (ALL & ~B_SUPPLIERS))  optSupplierIds.add(record.supplierId);
if ((mask & ~B_COMPANIES)  === (ALL & ~B_COMPANIES))  optCompanyIds.add(record.companyId);

One loop. The eligible set for the working set and the narrowed options for every dropdown, all derived from the same single pass over the same records.

The setup that makes this efficient is a pre-built flat array of those dimension records, constructed once when data loads. Names are pre-lowercased, associated keys are pre-resolved and deduped, anything that would require a Map lookup or string allocation in the hot loop is done upfront. The loop itself is as lean as possible.

This was hard to sketch on a whiteboard because it didn't exist yet. I knew the multi-pass approach was going to be expensive. I had a rough idea of the bitmask pattern from other contexts. The actual design emerged during implementation, and the whiteboard caught up to the code rather than the other way around.

What Got Cut

Here's the part that doesn't make it into most technical posts.

When you mutate a movement, change a quantity, delete an entry, the indexes need to reflect that change. The clean solution is delta updates: surgically update only the affected entries in the relevant indexes and aggregates.

That's not what this engine does for certain operations. For some mutation paths, it does a full index rebuild instead. The comment in the code says it plainly: for simplicity, rebuild all indexes after this operation. In a more optimized implementation we could update only the changed records. That optimization was not worth the complexity given the actual mutation frequency.

This was a direct outcome of the argument. After a few days of debate I stopped and asked honestly: how often do users actually mutate data in this UI? The answer was not that often. A handful of changes at a time. A full rebuild is fast enough that no user would ever notice.

KISS. Don't optimize what doesn't need optimizing. The pushback didn't just validate the engine, it also put a ceiling on it. Without "do you need this" I might have kept going.

When This Is Wrong

The mobile version of this exact feature uses the same underlying data model, shares some of the same backend logic, and does none of this.

The mobile app is scoped to a single dimension for its entire flow. One unit of work, one day at a time. Its operations are simple. Display records, make edits, sync. Redux with memoized selectors handles it cleanly. No cross-cutting filters. No multi-view projection. No bitmask pass needed.

Same data. Same domain. Completely different right answer.

That's the point. This engine isn't a pattern you reach for because your data looks like facts and dimensions. It's a pattern you reach for when the operations your UI needs to perform on that data don't have a simpler solution.

Before you build anything like this, answer these questions honestly:

• How many times will your UI look up the same record by ID, and how fast does that need to be?
• How many records are you filtering simultaneously, and how often does the filter change?
• Do you need cross-cutting filters that narrow multiple dropdowns simultaneously?
• Do you need undo/redo, dirty tracking, or local mutations against server-synced state?

If most of those answers are "not many" or "not often", use Redux. Use useState. Use a memoized selector. The overhead of an engine like this isn't just build time, it's maintenance time. Someone else has to understand it too.

If the answers point the other way, the complexity earns its place. The UI stays fast, the access patterns are explicit, and you stop hiding a growing pile of find() calls behind code that looks fine until it isn't.

The whiteboard argument isn't always between two people who know the answer. Sometimes it's one person trying to prove something they believe but can't fully articulate yet, with someone else asking the right questions to force the proof.

That's still an argument worth having. Maybe more so.