Commit dd69ab2
committed
Kotlin: speed up the Dokka source-set collapsing
Node#matches? recompiles the selector and evaluates it against the whole
document on every call, and it was called once per child of every
.platform-hinted node — 31% of the scrape's wall time on its own, since
the cost grows with the document size. A plain class check does the job.
Also drop two of the three Nokogiri queries per variant: the version
nodes are extracted once up front, and the dedup key is a substitution
on inner_html rather than a dup plus two css calls.
Throughput over a fixed 120s window goes from 340 to 4081 pages; a full
kotlin@2 scrape takes 4m24s. Output is unchanged (4875 pages, 4873
entries, 74 types, 77.8 MB db.json).1 parent 4cdb7bd commit dd69ab2
1 file changed
Lines changed: 18 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
| 50 | + | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
55 | 60 | | |
56 | | - | |
57 | 61 | | |
58 | 62 | | |
59 | 63 | | |
| |||
63 | 67 | | |
64 | 68 | | |
65 | 69 | | |
66 | | - | |
67 | 70 | | |
68 | 71 | | |
69 | 72 | | |
| |||
73 | 76 | | |
74 | 77 | | |
75 | 78 | | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
76 | 85 | | |
77 | 86 | | |
78 | 87 | | |
79 | 88 | | |
| 89 | + | |
| 90 | + | |
80 | 91 | | |
81 | | - | |
| 92 | + | |
82 | 93 | | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
| 94 | + | |
89 | 95 | | |
90 | 96 | | |
91 | 97 | | |
92 | 98 | | |
93 | 99 | | |
94 | 100 | | |
95 | | - | |
96 | | - | |
| 101 | + | |
| 102 | + | |
97 | 103 | | |
98 | 104 | | |
99 | 105 | | |
| |||
0 commit comments