Skip to content

Commit dd69ab2

Browse files
committed
Kotlin: speed up the Dokka source-set collapsing
Node#matches? recompiles the selector and evaluates it against the whole document on every call, and it was called once per child of every .platform-hinted node — 31% of the scrape's wall time on its own, since the cost grows with the document size. A plain class check does the job. Also drop two of the three Nokogiri queries per variant: the version nodes are extracted once up front, and the dedup key is a substitution on inner_html rather than a dup plus two css calls. Throughput over a fixed 120s window goes from 340 to 4081 pages; a full kotlin@2 scrape takes 4m24s. Output is unchanged (4875 pages, 4873 entries, 74 types, 77.8 MB db.json).
1 parent 4cdb7bd commit dd69ab2

1 file changed

Lines changed: 18 additions & 12 deletions

File tree

lib/docs/filters/kotlin/clean_html_v2.rb

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -47,13 +47,17 @@ def collapse_platform_tabs
4747

4848
# Reverse document order so nested widgets are collapsed before their ancestors.
4949
css('.platform-hinted').reverse_each do |node|
50-
variants = node.element_children.select { |child| child.matches?('.sourceset-dependent-content') }
50+
variants = node.element_children.select { |child| sourceset_content?(child) }
5151
next if variants.empty?
5252

5353
groups = {}
5454
variants.each do |variant|
55+
# Pull the version out first so it is part of neither the key nor the kept copy.
56+
since = since_nodes(variant)
57+
source = [platform(variant), since_version(since)]
58+
since.each(&:remove)
59+
5560
key = variant_key(variant)
56-
source = [platform(variant), since_version(variant)]
5761
if (group = groups[key])
5862
group[:sources] << source
5963
variant.remove
@@ -63,7 +67,6 @@ def collapse_platform_tabs
6367
end
6468

6569
groups.each_value do |group|
66-
since_nodes(group[:node]).each(&:remove)
6770
label = platform_label(group[:sources], all)
6871
group[:node].add_previous_sibling(label) if label
6972
group[:node].replace(group[:node].children)
@@ -73,27 +76,30 @@ def collapse_platform_tabs
7376
end
7477
end
7578

79+
# Node#matches? re-evaluates the selector against the whole document, which
80+
# dominates the scrape on the larger package indexes.
81+
def sourceset_content?(node)
82+
node['class'].to_s.split.include?('sourceset-dependent-content')
83+
end
84+
7685
def platform(node)
7786
node['data-togglable'].to_s.split('/').last
7887
end
7988

89+
MODIFIER = %r{<span class="token keyword">(?:expect|actual)\s*</span>}
90+
8091
# Two variants are the same declaration when they only differ in their
81-
# expect/actual modifier and in the Kotlin version they were introduced in.
92+
# expect/actual modifier; the version has already been removed by the caller.
8293
def variant_key(variant)
83-
copy = variant.dup
84-
since_nodes(copy).each(&:remove)
85-
copy.css('.token.keyword').each do |node|
86-
node.remove if %w(expect actual).include?(node.content.strip)
87-
end
88-
copy.inner_html
94+
variant.inner_html.gsub(MODIFIER, '')
8995
end
9096

9197
def since_nodes(node)
9298
node.css('.kdoc-tag, .inline-comment').select { |child| child.content.match?(SINCE) }
9399
end
94100

95-
def since_version(variant)
96-
since_nodes(variant).first&.content&.split(SINCE)&.last&.squish.presence
101+
def since_version(since)
102+
since.first&.content&.split(SINCE)&.last&.squish.presence
97103
end
98104

99105
def platform_label(sources, all)

0 commit comments

Comments
 (0)