Skip to content

[SPARK-56426][SQL] Fix LATERAL VIEW column alias with dot in name#55282

Open
jiwen624 wants to merge 1 commit intoapache:masterfrom
jiwen624:fix-lateral-view-dot-alias
Open

[SPARK-56426][SQL] Fix LATERAL VIEW column alias with dot in name#55282
jiwen624 wants to merge 1 commit intoapache:masterfrom
jiwen624:fix-lateral-view-dot-alias

Conversation

@jiwen624
Copy link
Copy Markdown
Contributor

@jiwen624 jiwen624 commented Apr 9, 2026

What changes were proposed in this pull request?

In ResolveGenerate, generator output column names were extracted via UnresolvedAttribute.name, which wraps any name containing a dot in backtick characters (e.g. "skill.inst""`skill.inst`"). This string was then passed directly to makeGeneratorOutput as the resolved attribute name, embedding literal backtick characters into the column name.

The fix uses nameParts.head instead, which returns the raw identifier text without display-formatting.

Why are the changes needed?

Using a backtick-quoted identifier with a dot as a LATERAL VIEW column alias causes an UNRESOLVED_COLUMN error at analysis time:

SELECT id, `skill.inst`
FROM VALUES (1, array('a', 'b')) AS (id, skills)
LATERAL VIEW explode(skills) skills_table AS `skill.inst`
-- ERROR: [UNRESOLVED_COLUMN] `skill.inst` cannot be resolved.
-- Did you mean: `skills_table`.```skill.inst```

The triple-backtick suggestion reveals that the resolved attribute name contains literal backtick characters, caused by UnresolvedAttribute.name being misused as a raw identifier extractor rather than for display only.

This worked in Spark 3.1.x.

Does this PR introduce any user-facing change?

Yes. Queries using a backtick-quoted LATERAL VIEW column alias containing a dot (e.g. `a.b`) now execute correctly instead of failing with UNRESOLVED_COLUMN.

How was this patch tested?

Added a test case in SQLQuerySuite covering LATERAL VIEW with a dot-containing column alias.

Was this patch authored or co-authored using generative AI tooling?

No.

@jiwen624 jiwen624 marked this pull request as ready for review April 9, 2026 23:35
@jiwen624
Copy link
Copy Markdown
Contributor Author

@cloud-fan @dongjoon-hyun could you take a look at this bug fix when you get a chance? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant