Skip to content

feat(query): database query engine, scripts, and unified collection views#7

Merged
kooksee merged 8 commits into
mainfrom
feat/database-query-and-scripts
Jun 24, 2026
Merged

feat(query): database query engine, scripts, and unified collection views#7
kooksee merged 8 commits into
mainfrom
feat/database-query-and-scripts

Conversation

@kooksee

@kooksee kooksee commented Jun 23, 2026

Copy link
Copy Markdown

Summary

  • Add unified database.query engine (structured IR + SQL builders) for records, meta types, asset libraries, and system DBs (filter/sort/aggregate + DSL).
  • Add Button field + sandboxed scripts (database.button.run, database.script.run) and action writes via collection.applyActions.
  • Add Settings tools: Script Explorer and Query Explorer (Dataview-style DSL).
  • Unify database collection views on SQL path via useDatabaseSqlCollectionQuery.
  • Add Formula fields (computed at display time).
  • Add Page Query Block (/query) — live DSL queries embedded in pages.
  • Add Page Button Block (/button) — script actions with table result display for query returns.

Test plan

  • packages/client tests: record-query, record-query-dsl, collection-actions, database-script, formula-expression
  • apps/web tests (146 passing, including backup-service)
  • packages/client, packages/core, apps/desktop lint
  • Desktop package build (LOCAL_ONLY=true npm run package)
  • Manual: Query Explorer, Script Explorer, Button field, Page Query/Button blocks
  • @colanode/ui lint (pre-existing debt on main as well)

Follow-ups (post-merge)

  • Markdown import for colanode-query / colanode-button blocks
  • Page Button result persistence (in-memory lastRun today)
  • DSL OR support; configurable result columns on blocks

kooksee and others added 2 commits June 23, 2026 14:47
- Added support for a new 'button' field type across various components, including field creation, display, and handling.
- Updated the UI to include button-specific settings in the field creation popover and table view headers.
- Enhanced query and mutation handlers to accommodate the new button functionality.
- Introduced new routes for script and query explorers in the sidebar settings for better navigation.

This update improves the flexibility of field types and enhances user interaction with button fields in the application.
Route records and meta type collection views through the same SQL-backed database.query path with shared pagination and hydration, so filters and sorts stay consistent across UI, scripts, and DSL usage.

Co-authored-by: Cursor <cursoragent@cursor.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a unified local query engine and Query DSL parser for Colanode databases, alongside a sandboxed scripting environment for button fields and ad-hoc scripts. Key additions include the database.query handler, a sandbox runner, and the Query and Script Explorer UI components. While the implementation is comprehensive, several critical security and stability issues must be addressed. These include a SQL injection vulnerability via unsanitized aggregate aliases, a sandbox escape risk through unblocked global APIs, potential cache corruption from shallow-frozen row snapshots, and potential TypeError crashes in the collection action handler due to missing null guards on concurrent node updates.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +35 to +58
const buildAggregateSqlExpression = (
aggregate: RecordQueryAggregate,
fields: Record<string, FieldAttributes>
): string | null => {
const alias = aggregate.alias ?? `${aggregate.fn}_${aggregate.fieldId}`;

if (aggregate.fn === 'count' && aggregate.fieldId === '*') {
return `COUNT(*) as "${alias}"`;
}

const field = fields[aggregate.fieldId];
if (!field) {
return null;
}

const selector = buildFieldSqlSelector(field);
const fn = aggregate.fn.toUpperCase();

if (aggregate.fn === 'count') {
return `COUNT(${selector}) as "${alias}"`;
}

return `${fn}(CAST(${selector} AS REAL)) as "${alias}"`;
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

The alias property in RecordQueryAggregate is inserted directly into the SQL query string without sanitization or escaping. Since this query can be constructed from user-controlled inputs (such as the Query DSL or client-side API calls), this presents a SQL injection vulnerability.

To prevent this, ensure that the alias is sanitized or escaped (for example, by doubling any double quotes inside the identifier to conform to SQLite's identifier escaping rules, or restricting it to alphanumeric characters and underscores).

const buildAggregateSqlExpression = (
  aggregate: RecordQueryAggregate,
  fields: Record<string, FieldAttributes>
): string | null => {
  const rawAlias = aggregate.alias ?? `${aggregate.fn}_${aggregate.fieldId}`;
  const alias = rawAlias.replace(/"/g, '""');

  if (aggregate.fn === 'count' && aggregate.fieldId === '*') {
    return `COUNT(*) as "${alias}"`;
  }

  const field = fields[aggregate.fieldId];
  if (!field) {
    return null;
  }

  const selector = buildFieldSqlSelector(field);
  const fn = aggregate.fn.toUpperCase();

  if (aggregate.fn === 'count') {
    return `COUNT(${selector}) as "${alias}"`;
  }

  return `${fn}(CAST(${selector} AS REAL)) as "${alias}"`;
};

Comment on lines +35 to +44
const wrappedScript = `"use strict";\n${script}`;
let runner: (...args: unknown[]) => Promise<unknown>;

try {
runner = new AsyncFunction('ctx', wrappedScript);
} catch (error) {
throw new ScriptSandboxError(
error instanceof Error ? error.message : 'Invalid script syntax.'
);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

The sandbox implementation uses AsyncFunction directly in the client worker but does not restrict access to powerful global APIs like fetch, WebSocket, XMLHttpRequest, or indexedDB. A malicious script could easily exfiltrate sensitive database records to an external server.

To mitigate this, shadow these sensitive globals by declaring them as arguments to the AsyncFunction and passing undefined when invoking the runner.

Suggested change
const wrappedScript = `"use strict";\n${script}`;
let runner: (...args: unknown[]) => Promise<unknown>;
try {
runner = new AsyncFunction('ctx', wrappedScript);
} catch (error) {
throw new ScriptSandboxError(
error instanceof Error ? error.message : 'Invalid script syntax.'
);
}
const blockedGlobals = ['fetch', 'WebSocket', 'XMLHttpRequest', 'indexedDB', 'caches', 'self'];
const wrappedScript = `"use strict";\n${script}`;
let runner: (...args: unknown[]) => Promise<unknown>;
try {
runner = new AsyncFunction('ctx', ...blockedGlobals, wrappedScript);
} catch (error) {
throw new ScriptSandboxError(
error instanceof Error ? error.message : 'Invalid script syntax.'
);
}

Comment on lines +108 to +123
const buildRowSnapshot = (
id: string,
attributes: RecordAttributes | PageAttributes
): ScriptRowSnapshot => {
const fields: Record<string, FieldValue> =
attributes.type === 'page'
? (attributes.fields ?? {})
: attributes.fields;

return {
id,
name: attributes.name,
fields,
nodeType: attributes.type,
};
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The buildRowSnapshot function returns a direct reference to attributes.fields (which belongs to the live Yjs/TanStack DB node attributes). Since Object.freeze is shallow, a sandboxed script can directly mutate the nested fields of ctx.record or ctx.row (e.g., ctx.record.fields.status.value = 'mutated'), bypassing the mutation system and corrupting the client-side cache.

To ensure the sandbox is truly read-only, deep clone the fields and attributes before passing them to the context.

Suggested change
const buildRowSnapshot = (
id: string,
attributes: RecordAttributes | PageAttributes
): ScriptRowSnapshot => {
const fields: Record<string, FieldValue> =
attributes.type === 'page'
? (attributes.fields ?? {})
: attributes.fields;
return {
id,
name: attributes.name,
fields,
nodeType: attributes.type,
};
};
const buildRowSnapshot = (
id: string,
attributes: RecordAttributes | PageAttributes
): ScriptRowSnapshot => {
const fields: Record<string, FieldValue> =
attributes.type === 'page'
? (attributes.fields ?? {})
: attributes.fields;
return {
id,
name: attributes.name,
fields: JSON.parse(JSON.stringify(fields)),
nodeType: attributes.type,
};
};

Comment on lines +47 to +55
const result = await workspace.nodes.updateNode<RecordAttributes>(
input.nodeId,
(current) =>
applyCollectionFieldActions(
current,
input.actions,
database.fields ?? {}
)
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The updater function passed to updateNode does not check if current is null or undefined. If the node is concurrently deleted or not found, this will throw a TypeError when attempting to read or spread current.

Add a defensive guard to return early if current is not defined.

      const result = await workspace.nodes.updateNode<RecordAttributes>(
        input.nodeId,
        (current) => {
          if (!current) {
            return current;
          }

          return applyCollectionFieldActions(
            current,
            input.actions,
            database.fields ?? {}
          );
        }
      );

Comment on lines +72 to +86
const result = await workspace.nodes.updateNode<PageAttributes>(
input.nodeId,
(current) => {
const withFields = {
...current,
fields: current.fields ?? {},
};

return applyCollectionFieldActions(
withFields,
input.actions,
database.fields ?? {}
);
}
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The updater function passed to updateNode does not check if current is null or undefined. If the node is concurrently deleted or not found, attempting to spread current or access current.fields will throw a TypeError.

Add a defensive guard to return early if current is not defined.

      const result = await workspace.nodes.updateNode<PageAttributes>(
        input.nodeId,
        (current) => {
          if (!current) {
            return current;
          }

          const withFields = {
            ...current,
            fields: current.fields ?? {},
          };

          return applyCollectionFieldActions(
            withFields,
            input.actions,
            database.fields ?? {}
          );
        }
      );

kooksee and others added 6 commits June 23, 2026 18:52
Introduce computed formula columns, embed live database.query results in
pages via a Query block, and improve atom block selection, deletion, and
popover focus handling in the document editor.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ld creation

- Introduced `getNextFractionalIndex` and `isValidFractionalIndex` functions to streamline index handling for scheduled tasks and related fields.
- Updated `buildRemindersRelationField` to utilize the new index management functions, ensuring proper index assignment for reminders.
- Refactored field creation logic across various components to replace direct index calculations with the new utility functions, improving consistency and maintainability.
- Enhanced UI components to support dynamic index assignment, ensuring better user experience when adding fields.

This update improves the reliability of index management in scheduled tasks and related UI components.
Embed sandboxed database.script.run buttons in pages via /button, with
meta type page row context and markdown export support.

Co-authored-by: Cursor <cursoragent@cursor.com>
…management

- Added a settings toggle for the ButtonBlockPanel to manage script execution settings.
- Introduced state management for the last run output of the script.
- Updated the component to handle selected state and improve user interaction.
- Refactored the action button to enhance UI responsiveness during script execution.
- Integrated new utility functions for better handling of database queries and results.

This update improves the usability and functionality of the ButtonBlockPanel, allowing for a more dynamic user experience when configuring and executing scripts.
Align backup-service tests with content-addressed blob storage and mock
snapshotDatabases for non-SQLite fixtures. Fix core/client/desktop lint
issues and document query/button block result rendering in record-query.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add prepare script to @colanode/mcp-server and declare typed package
exports so @colanode/cli can compile when dist is absent on fresh installs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kooksee kooksee merged commit 2484e5f into main Jun 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant