Skip to content

[PM-37255] feat: Integrate fill-assist targeting rules into autofill parser#7066

Open
aj-rosado wants to merge 24 commits into
mainfrom
PM-37256/apply-fill-assist-rules
Open

[PM-37255] feat: Integrate fill-assist targeting rules into autofill parser#7066
aj-rosado wants to merge 24 commits into
mainfrom
PM-37256/apply-fill-assist-rules

Conversation

@aj-rosado

Copy link
Copy Markdown
Contributor

🎟️ Tracking

https://bitwarden.atlassian.net/browse/PM-37255

📔 Objective

Wires the fill-assist targeting-rules pipeline end-to-end into the Android autofill framework.

When the FillAssistTargetingRules feature flag is enabled and rules are available for the current host, the autofill parser replaces heuristic field detection with site-specific CSS-selector-based matching. Unmatched nodes are excluded entirely — there is no heuristic fallback when rules are active.

What's included:

  • Network layerFillAssistApi, FillAssistService, FillAssistManifestJson, FillAssistFormsJson: fetches a versioned manifest and forms JSON from the fill-assist CDN endpoint.
  • Data layerFillAssistManagerImpl, FillAssistDiskSource: parses CSS selectors into FillAssistRules (tag, id, name, type, role constraints per field), caches rules on disk, syncs on server-config change with a 6-hour re-fetch throttle.
  • CSS selector parser — handles >>> Shadow DOM notation, space-separated CSS descendant selectors (split on whitespace outside […] attribute brackets to preserve attribute values that contain spaces), #id shorthand, and [attr='value'] / [attr="value"] attribute selectors.
  • IntegrationAutofillParserImpl looks up host rules for the focused view's URI and calls AssistStructure.buildFillAssistViews to replace the heuristic view list when rules match.
  • View-node matchingFillAssistViewNodeExtensions.traverseForFillAssist traverses the AssistStructure tree; HtmlInfoExtensions.matchesSelectorClause does the per-node attribute comparison (kept in HtmlInfoExtensions alongside hints() since HtmlInfo.attributes uses android.util.Pair and is untestable in unit tests).
  • TestsFillAssistManagerTest, FillAssistServiceTest, FillAssistViewNodeExtensionsTest (all previously commented-out tests now passing via mockkStatic(HtmlInfo::matchesSelectorClause)), and AutofillParserTests updated for the new constructor parameters.

@aj-rosado aj-rosado added the ai-review-vnext Request a Claude code review using the vNext workflow label Jun 16, 2026
@github-actions github-actions Bot added the app:password-manager Bitwarden Password Manager app context label Jun 16, 2026
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🤖 Bitwarden Claude Code Review

Overall Assessment: APPROVE

Reviewed the end-to-end wiring of the fill-assist targeting-rules pipeline into AutofillParserImpl: the new resolveEffectiveViews gating (feature flag + host-rule lookup + partition-category coverage), the AssistStructure.buildFillAssistViews traversal and per-node selector matching in HtmlInfoExtensions.matchesSelectorClause, the descendant-selector parsing change in FillAssistManagerImpl, and the accompanying test suites. Production logic is sound, host-prefix normalization matches the existing AccessibilityNodeInfoManagerImpl convention, and the "no heuristic fallback when rules match" behavior is consistently implemented. The new parser, view-node, and selector paths are well covered by tests.

Code Review Details
  • ♻️ : AutofillParserTests.kt:93 stubs getFeatureFlagFlow(FlagKey.ManageDevices) — an unrelated flag the parser never calls; appears to be a copy-paste artifact and is misleading.
  • ♻️ : AutofillParserTests.kt:97 mockIsFillAssistEnabled is written in setup and each fill-assist test but never read; the flag is actually driven by mutableFillAssistFlagFlow.value, so the field is dead and misleading.

Both are test-only cleanups; no production-behavior or correctness impact.

@aj-rosado aj-rosado changed the title Add fill assist logic to Autofill [PM-37255] feat: Integrate fill-assist targeting rules into autofill parser Jun 17, 2026
@github-actions github-actions Bot added the t:feature Change Type - Feature Development label Jun 25, 2026
@aj-rosado aj-rosado marked this pull request as ready for review June 30, 2026 10:04
@aj-rosado aj-rosado requested review from a team and david-livefront as code owners June 30, 2026 10:04
Base automatically changed from PM-37255/fill-assist-integration to main June 30, 2026 10:17
?: return AutofillRequest.Unfillable

val packageName =
traversalDataList.buildPackageNameOrNull(assistStructure = assistStructure)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clean this up a bit

val packageName = traversalDataList.buildPackageNameOrNull(
    assistStructure = assistStructure,
)

?: viewsLists
.flatten()
.filter { it !is AutofillView.Unused })
.map { it.updateWebsiteIfNecessary(website = urlBarWebsite) }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that this was moved out into it's own method.

What do you think of this minor update:

private fun List<ViewNodeTraversalData>.toAutofillViews(
    urlBarWebsite: String?,
): List<AutofillView> {
    val viewsLists = map { it.autofillViews }
    val autofillViewLists = viewsLists
        .filter { views -> views.any { it.data.isFocused } }
        .flatten()
        .filter { it !is AutofillView.Unused }
        .takeUnless { it.isEmpty() }
        ?: viewsLists
            .flatten()
            .filter { it !is AutofillView.Unused }
    return autofillViewLists.map { it.updateWebsiteIfNecessary(website = urlBarWebsite) }
}

I have never found that original block of code particularly readable 😄

.map { it.updateWebsiteIfNecessary(website = urlBarWebsite) }

// Find the focused view, or fallback to the first fillable item on the screen (so
// we at least have something to hook into)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we leave in the comments in place? Autofill is confusing enough as it is, I want to make sure we leave goo notes for the next person.

        // Find the focused view, or fallback to the first fillable item on the screen (so
        // we at least have something to hook into)
        val focusedView = autofillViews
            .firstOrNull { it.data.isFocused }
            ?: autofillViews.firstOrNull()
            // The view is unfillable if there are no focused views.
            ?: return AutofillRequest.Unfillable


val blockListedURIs = settingsRepository.blockedAutofillUris + BLOCK_LISTED_URIS
if (blockListedURIs.contains(uri)) {
// The view is unfillable if the URI is block listed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, let's leave the comments in for posterity.


// Get inline information if available
val isInlineAutofillEnabled = settingsRepository.isInlineAutofillEnabled
Timber.d("Autofill request isInlineEnabled=$isInlineAutofillEnabled -- ${fillRequest?.id}")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing this?

* when the feature flag is enabled and the host rules cover the current partition type;
* otherwise returns the heuristic [autofillViews].
*/
private fun resolveEffectiveViews(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about making this an extension function?

    private fun List<AutofillView>.toEffectiveViews(
        assistStructure: AssistStructure,
        uri: String?,
        focusedView: AutofillView,
        urlBarWebsite: String?,
    ): List<AutofillView> {

?.takeUnless { it.startsWith("androidapp://") }
?.toUri()
?.host
?.takeIf { featureFlagManager.getFeatureFlag(FlagKey.FillAssistTargetingRules) }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just return early instead of putting this check in the chain?

        if (!featureFlagManager.getFeatureFlag(FlagKey.FillAssistTargetingRules)) {
            return autofillViews
        }

val urlBarWebsite = traversalDataList
.flatMap { it.urlBarWebsites }
.firstOrNull()
val autofillViews = traversalDataList.toAutofillViews(urlBarWebsite = urlBarWebsite)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new buildFillAssistViews is awfully similar to the existing logic for parsing the assist structure. It's hard for me to tell if it is possible but can we optimize this to parse the structure only once?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review-vnext Request a Claude code review using the vNext workflow app:password-manager Bitwarden Password Manager app context t:feature Change Type - Feature Development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants