Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/backtick-column-split.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@hyperdx/common-utils': patch
'@hyperdx/app': patch
---

fix: keep backtick-quoted column names that contain a comma intact when toggling search columns or editing group by
11 changes: 0 additions & 11 deletions packages/app/src/__tests__/utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ import {
parseTimestampToMs,
resolveConditionalColor,
sortingStateToOrderByString,
splitClauseList,
stripTrailingSlash,
useQueryHistory,
withMapKeyAlias,
Expand Down Expand Up @@ -1537,16 +1536,6 @@ describe('isColumnInSelect', () => {
});
});

describe('splitClauseList', () => {
it('keeps a comma inside a backtick-quoted name', () => {
expect(splitClauseList('Timestamp, `a,b`, Body')).toEqual([
'Timestamp',
'`a,b`',
'Body',
]);
});
});

describe('orderByAfterRemovingSelectItem', () => {
const item = `ResourceAttributes['service.name'] AS "service.name"`;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { TableConnection } from '@hyperdx/common-utils/dist/core/metadata';
import { splitAndTrimWithBracket } from '@hyperdx/common-utils/dist/core/utils';
import {
Autocomplete,
Box,
Expand All @@ -25,7 +26,7 @@ import {

import { fieldListSummary } from '@/components/SplitValueButton/SplitValueButton';
import SQLInlineEditor from '@/components/SQLEditor/SQLInlineEditor';
import { isColumnInSelect, splitClauseList, withMapKeyAlias } from '@/utils';
import { isColumnInSelect, withMapKeyAlias } from '@/utils';

import {
clauseApplyDisabled,
Expand Down Expand Up @@ -68,10 +69,10 @@ export function ColumnsPopover({
// An empty stored SELECT means "use the source default". An empty draft does
// not: the user may have removed every column before adding a replacement.
const storedColumns = (select: string) =>
splitClauseList(select || defaultValue);
splitAndTrimWithBracket(select || defaultValue);

const shown = storedColumns(value);
const items = useMemo(() => splitClauseList(draft), [draft]);
const items = useMemo(() => splitAndTrimWithBracket(draft), [draft]);

const options = useMemo(
() => fieldOptions.filter(option => !isColumnInSelect(items, option)),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { splitAndTrimWithBracket } from '@hyperdx/common-utils/dist/core/utils';
import { findBalancedParens } from '@hyperdx/common-utils/dist/variables';

import { splitClauseList } from '@/utils';
import { parseAsSortingStateString } from '@/utils/queryParsers';

export type SortKey = { id: string; desc: boolean };
Expand Down Expand Up @@ -35,15 +35,15 @@ export function parseSortKeys(orderBy: string): SortKey[] | null {
expr.startsWith('(') &&
findBalancedParens(expr, 0) === expr.length - 1
) {
const ids = splitClauseList(expr.slice(1, -1));
const ids = splitAndTrimWithBracket(expr.slice(1, -1));
if (ids.length > 1 && ids.every(id => id.length > 0)) {
const desc = direction[2].toUpperCase() === 'DESC';
return ids.map(id => ({ id, desc }));
}
}

const keys: SortKey[] = [];
for (const part of splitClauseList(trimmed)) {
for (const part of splitAndTrimWithBracket(trimmed)) {
const parsed = parseAsSortingStateString.parse(part);
if (parsed == null || parsed.id.length === 0) return null;
keys.push({ id: parsed.id, desc: parsed.desc });
Expand Down
21 changes: 2 additions & 19 deletions packages/app/src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@ import {
TableConnection,
unquoteIdentifier,
} from '@hyperdx/common-utils/dist/core/metadata';
import {
replaceBacktickedIdentifiers,
restoreReplacements,
splitAndTrimWithBracket,
} from '@hyperdx/common-utils/dist/core/utils';
import { splitAndTrimWithBracket } from '@hyperdx/common-utils/dist/core/utils';
import {
CATEGORICAL_PALETTE_TOKENS,
ChartPaletteToken,
Expand Down Expand Up @@ -1326,19 +1322,6 @@ export const withMapKeyAlias = (
return `${column} AS "${key}"`;
};

/**
* Split a SELECT or ORDER BY list. A comma inside a backtick-quoted name is
* not a separator. Those names are tokenized first so a later join cannot
* insert a space inside one. The shared splitter stays unaware of backticks.
*/
export function splitClauseList(input: string): string[] {
const { sqlWithReplacements, replacements } =
replaceBacktickedIdentifiers(input);
return splitAndTrimWithBracket(sqlWithReplacements).map(part =>
restoreReplacements(part, replacements.quotedText),
);
}

/**
* The sort to keep once a SELECT item is removed. Sorting on an aliased column
* from the table header orders by the alias itself (`"service.name" DESC`),
Expand All @@ -1355,7 +1338,7 @@ export const orderByAfterRemovingSelectItem = (
return orderBy;
}
const quoted = `"${alias}"`;
const keys = splitClauseList(orderBy);
const keys = splitAndTrimWithBracket(orderBy);
const kept = keys.filter(
key => key.replace(/\s+(ASC|DESC)$/i, '') !== quoted,
);
Expand Down
21 changes: 21 additions & 0 deletions packages/common-utils/src/__tests__/utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,27 @@ describe('utils', () => {
expect(splitAndTrimWithBracket(input)).toEqual(expected);
});

it('should keep commas and brackets inside backtick-quoted identifiers', () => {
const input = "Timestamp, `a,b`, `x(y`, `it's` AS label, Body";
const expected = [
'Timestamp',
'`a,b`',
'`x(y`',
"`it's` AS label",
'Body',
];
expect(splitAndTrimWithBracket(input)).toEqual(expected);
});

it('should keep the text after an unclosed quote', () => {
expect(splitAndTrimWithBracket('Timestamp, `a')).toEqual([
'Timestamp',
'`a',
]);
expect(splitAndTrimWithBracket('a`b, c')).toEqual(['a`b, c']);
expect(splitAndTrimWithBracket("x, 'y, z")).toEqual(['x', "'y, z"]);
});

it('should handle mixed quotes with commas', () => {
const input = `col1, "double, quoted", col2, 'single, quoted', col3`;
const expected = [
Expand Down
23 changes: 23 additions & 0 deletions packages/common-utils/src/core/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,29 @@ export function splitAndTrimWithBracket(input: string): string[] {
let squareCount: number = 0;
let inSingleQuote: boolean = false;
let inDoubleQuote: boolean = false;
let inBacktick: boolean = false;

const res: string[] = [];
let cur: string = '';
for (let i = 0; i <= input.length; i++) {
const c = i === input.length ? ',' : input[i];

if (
c === '`' &&
!inSingleQuote &&
!inDoubleQuote &&
!isQuoteEscapedByBackslash(input, i)
) {
inBacktick = !inBacktick;
cur += c;
continue;
}

if (inBacktick) {
cur += c;
continue;
}

if (c === '"' && !inSingleQuote && !isQuoteEscapedByBackslash(input, i)) {
inDoubleQuote = !inDoubleQuote;
cur += c;
Expand Down Expand Up @@ -178,6 +195,12 @@ export function splitAndTrimWithBracket(input: string): string[] {
cur += c;
}
}
// An unclosed quote swallows the closing ','; keep its text so a half-typed
// name survives. Unclosed brackets still drop it, as parseToStartOfFunction expects.
if (inBacktick || inSingleQuote || inDoubleQuote) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Keeping the text after an unclosed quote lets the columns editor absorb columns added after it

ColumnsPopover rebuilds the draft with items.join(', ') and then splits it again (packages/app/src/components/SearchResultsToolbar/ColumnsPopover.tsx:86,93,110). Take the draft Timestamp, aand addBody. The draft becomes Timestamp, a, Body, which splits back into ['Timestamp', 'a, Body']. Bodynever shows up as its own row and stays in the options list, so each further click appends, Bodyagain. Dragging ``a `` above Timestamp has the same effect and merges the two into one item. Before this PR the dangling item was dropped instead. Two possible fixes: disable add and reorder while the last item has an open quote, or have the splitter tell callers which item is unterminated so the editor can block structural edits on it. Add a ColumnsPopover test that adds a column after an unclosed backtick.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Unclosed-quote fallback also fires when brackets are unbalanced, which contradicts the comment and the PR description

The fallback only checks the quote flags. Input with an unclosed bracket that also has an open quote, such as Timestamp, count(aorx, f('y, now returns its last item. The same input without the quote, count(a, still drops it. The comment says unclosed brackets still drop the item, as parseToStartOfFunctionexpects, so this case breaks that contract. AddparenCount === 0 && squareCount === 0` to the condition, and add a test for an unclosed quote inside an open paren.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

const rest = cur.slice(0, -1).trim();
if (rest) res.push(rest);
}
return res;
}

Expand Down
Loading