Very slow compilation on Elixir main

[Schultzer]

Elixir and Erlang/OTP versions

Erlang/OTP 28 [erts-16.1] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Elixir 1.20.0-rc.3 (528f8e7) (compiled with Erlang/OTP 28)

Operating system

macOS 26.3.1 (25D2128)

Current behavior

I'm seeing slow compilation on the lastet RC and main in my sql library https://github.com/elixir-dbvisor/sql.

This was not an issue in 1.20.0-rc.1-otp-28, looks like it was intruduced in 1.20.0-rc.2-otp-28

➜  sql git:(main) ✗ mix compile --force --profile time
Compiling 16 files (.ex)
[profile]     90ms compiling +      0ms waiting while compiling lib/mix/tasks/compile/sql.ex
[profile]    100ms compiling +      0ms waiting while compiling lib/application.ex
[profile]    100ms compiling +      0ms waiting while compiling lib/formatter.ex
[profile]     47ms compiling +      0ms waiting while compiling lib/token.ex
[profile]    177ms compiling +      0ms waiting while compiling lib/pool.ex
[profile]    223ms compiling +      0ms waiting while compiling lib/mix/tasks/sql.gen.test.ex
[profile]    313ms compiling +      0ms waiting while compiling lib/bnf.ex
[profile]    288ms compiling +      0ms waiting while compiling lib/sql.ex
[profile]   1033ms compiling +      0ms waiting while compiling lib/parser.ex
[profile]   8075ms compiling +      0ms waiting while compiling lib/lexer.ex
Compiling lib/format.ex (it's taking more than 10s)
Compiling lib/adapters/tds.ex (it's taking more than 10s)
Compiling lib/adapters/postgres.ex (it's taking more than 10s)
Compiling lib/adapters/mysql.ex (it's taking more than 10s)
Compiling lib/adapters/ansi.ex (it's taking more than 10s)
[profile]  20569ms compiling +    121ms waiting for module SQL.BNF while compiling lib/adapters/ansi.ex
[profile]                    |    147ms waiting for module SQL.Token while compiling lib/adapters/ansi.ex
[profile]  20784ms compiling +    118ms waiting for module SQL.BNF while compiling lib/adapters/tds.ex
[profile]                    |    139ms waiting for module SQL.Token while compiling lib/adapters/tds.ex
[profile]  21772ms compiling +    126ms waiting for module SQL.BNF while compiling lib/adapters/mysql.ex
[profile]                    |    144ms waiting for module SQL.Token while compiling lib/adapters/mysql.ex
[profile]  22850ms compiling +    116ms waiting for module SQL.BNF while compiling lib/adapters/postgres.ex
[profile]                    |    105ms waiting for module SQL.Token while compiling lib/adapters/postgres.ex
[profile] 197545ms compiling +    190ms waiting for module SQL.BNF while compiling lib/format.ex
[profile]    118ms compiling + 176890ms waiting for module SQL.Format while compiling lib/mix/tasks/sql.get.ex
[profile]                    |  12742ms waiting for module SQL.Adapters.ANSI while compiling lib/mix/tasks/sql.get.ex
[profile]                    |   7734ms waiting for module SQL.Lexer while compiling lib/mix/tasks/sql.get.ex
[profile]                    |    295ms waiting for module SQL while compiling lib/mix/tasks/sql.get.ex
[profile] Finished cycle resolution in 0ms
[profile] Finished compilation cycle of 20 modules in 197794ms
[profile] Finished writing modules to disk in 22ms
[profile] Finished after compile callback in 483ms
Verifying SQL.Format (it's taking more than 10s)
Verifying SQL.Adapters.Postgres (it's taking more than 10s)
Verifying SQL.Adapters.MySQL (it's taking more than 10s)
Verifying SQL.Adapters.TDS (it's taking more than 10s)
Verifying SQL.Adapters.ANSI (it's taking more than 10s)
[profile] Finished group pass check of 20 modules in 291951ms
Generated sql app

Expected behavior

To compile as fast as 1.20.0-rc.1-otp-28 .

➜  sql git:(main) ✗ mix compile --force --profile time
Compiling 16 files (.ex)
[profile]     80ms compiling +      0ms waiting while compiling lib/mix/tasks/compile/sql.ex
[profile]    103ms compiling +      0ms waiting while compiling lib/application.ex
[profile]    105ms compiling +      0ms waiting while compiling lib/formatter.ex
[profile]     54ms compiling +      0ms waiting while compiling lib/token.ex
[profile]    210ms compiling +      0ms waiting while compiling lib/pool.ex
[profile]    244ms compiling +      0ms waiting while compiling lib/mix/tasks/sql.gen.test.ex
[profile]    308ms compiling +      0ms waiting while compiling lib/bnf.ex
[profile]    344ms compiling +      0ms waiting while compiling lib/sql.ex
[profile]   1094ms compiling +      0ms waiting while compiling lib/parser.ex
[profile]   1782ms compiling +    103ms waiting for module SQL.BNF while compiling lib/adapters/ansi.ex
[profile]                    |    136ms waiting for module SQL.Token while compiling lib/adapters/ansi.ex
[profile]   1941ms compiling +    105ms waiting for module SQL.BNF while compiling lib/adapters/tds.ex
[profile]                    |    134ms waiting for module SQL.Token while compiling lib/adapters/tds.ex
[profile]   4002ms compiling +    104ms waiting for module SQL.BNF while compiling lib/adapters/mysql.ex
[profile]                    |    135ms waiting for module SQL.Token while compiling lib/adapters/mysql.ex
[profile]   4669ms compiling +    104ms waiting for module SQL.BNF while compiling lib/adapters/postgres.ex
[profile]                    |    104ms waiting for module SQL.Token while compiling lib/adapters/postgres.ex
[profile]   5335ms compiling +    211ms waiting for module SQL.BNF while compiling lib/format.ex
[profile]   8217ms compiling +      0ms waiting while compiling lib/lexer.ex
[profile]    104ms compiling +   7806ms waiting for module SQL.Lexer while compiling lib/mix/tasks/sql.get.ex
[profile]                    |    344ms waiting for module SQL while compiling lib/mix/tasks/sql.get.ex
[profile] Finished cycle resolution in 0ms
[profile] Finished compilation cycle of 20 modules in 8262ms
[profile] Finished writing modules to disk in 44ms
[profile] Finished after compile callback in 274ms
[profile] Finished group pass check of 20 modules in 172ms
Generated sql app

[Schultzer]

➜  sql git:(main) ✗ MIX_DEBUG=1 mix compile --force --profile time --no-verification
-> Running mix CLI
-> Running mix loadconfig (inside SQL.MixProject)
<- Ran mix loadconfig in 1ms
-> Running mix compile --force --profile time --no-verification (inside SQL.MixProject)
-> Running mix loadpaths --force --profile time --no-verification (inside SQL.MixProject)
-> Running mix deps.loadpaths --force --profile time --no-verification (inside SQL.MixProject)
-> Running mix archive.check --force --profile time --no-verification (inside SQL.MixProject)
<- Ran mix archive.check in 0ms
<- Ran mix deps.loadpaths in 117ms
<- Ran mix loadpaths in 118ms
-> Running mix compile.all --force --profile time --no-verification (inside SQL.MixProject)
-> Running mix compile.yecc --force --profile time --no-verification (inside SQL.MixProject)
<- Ran mix compile.yecc in 2ms
-> Running mix compile.leex --force --profile time --no-verification (inside SQL.MixProject)
<- Ran mix compile.leex in 1ms
-> Running mix compile.erlang --force --profile time --no-verification (inside SQL.MixProject)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --force --profile time --no-verification (inside SQL.MixProject)
Compiling 16 files (.ex)
[profile]     84ms compiling +      0ms waiting while compiling lib/mix/tasks/compile/sql.ex
[profile]    102ms compiling +      0ms waiting while compiling lib/application.ex
[profile]    108ms compiling +      0ms waiting while compiling lib/formatter.ex
[profile]     57ms compiling +      0ms waiting while compiling lib/token.ex
[profile]    152ms compiling +      0ms waiting while compiling lib/pool.ex
[profile]    210ms compiling +      0ms waiting while compiling lib/mix/tasks/sql.gen.test.ex
[profile]    271ms compiling +      0ms waiting while compiling lib/bnf.ex
[profile]    281ms compiling +      0ms waiting while compiling lib/sql.ex
[profile]   1071ms compiling +      0ms waiting while compiling lib/parser.ex
[profile]   8630ms compiling +      0ms waiting while compiling lib/lexer.ex
Compiling lib/format.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/tds.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/postgres.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/mysql.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/ansi.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

[profile]  21501ms compiling +     78ms waiting for module SQL.BNF while compiling lib/adapters/ansi.ex
[profile]                    |    154ms waiting for module SQL.Token while compiling lib/adapters/ansi.ex
[profile]  21572ms compiling +     78ms waiting for module SQL.BNF while compiling lib/adapters/tds.ex
[profile]                    |    143ms waiting for module SQL.Token while compiling lib/adapters/tds.ex
[profile]  21999ms compiling +     76ms waiting for module SQL.BNF while compiling lib/adapters/mysql.ex
[profile]                    |    143ms waiting for module SQL.Token while compiling lib/adapters/mysql.ex
[profile]  22668ms compiling +     74ms waiting for module SQL.BNF while compiling lib/adapters/postgres.ex
[profile]                    |     93ms waiting for module SQL.Token while compiling lib/adapters/postgres.ex
[profile] 199497ms compiling +    151ms waiting for module SQL.BNF while compiling lib/format.ex
[profile]    129ms compiling + 177895ms waiting for module SQL.Format while compiling lib/mix/tasks/sql.get.ex
[profile]                    |  13068ms waiting for module SQL.Adapters.ANSI while compiling lib/mix/tasks/sql.get.ex
[profile]                    |   8279ms waiting for module SQL.Lexer while compiling lib/mix/tasks/sql.get.ex
[profile]                    |    299ms waiting for module SQL while compiling lib/mix/tasks/sql.get.ex
[profile] Finished cycle resolution in 0ms
[profile] Finished compilation cycle of 20 modules in 199679ms
[profile] Finished writing modules to disk in 4ms
[profile] Finished after compile callback in 193ms
<- Ran mix compile.elixir in 199903ms
-> Running mix compile.app --force --profile time --no-verification (inside SQL.MixProject)
Generated sql app
<- Ran mix compile.app in 5ms
<- Ran mix compile.all in 199948ms
<- Ran mix compile in 200083ms
<- Ran mix CLI in 200150ms

[Schultzer]

➜  sql git:(main) ✗ MIX_DEBUG=1 mix compile --force --profile time
-> Running mix CLI
-> Running mix loadconfig (inside SQL.MixProject)
<- Ran mix loadconfig in 1ms
-> Running mix compile --force --profile time (inside SQL.MixProject)
-> Running mix loadpaths --force --profile time (inside SQL.MixProject)
-> Running mix deps.loadpaths --force --profile time (inside SQL.MixProject)
-> Running mix archive.check --force --profile time (inside SQL.MixProject)
<- Ran mix archive.check in 0ms
<- Ran mix deps.loadpaths in 97ms
<- Ran mix loadpaths in 98ms
-> Running mix compile.all --force --profile time (inside SQL.MixProject)
-> Running mix compile.yecc --force --profile time (inside SQL.MixProject)
<- Ran mix compile.yecc in 1ms
-> Running mix compile.leex --force --profile time (inside SQL.MixProject)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --force --profile time (inside SQL.MixProject)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --force --profile time (inside SQL.MixProject)
Compiling 16 files (.ex)
[profile]    131ms compiling +      0ms waiting while compiling lib/mix/tasks/compile/sql.ex
[profile]    147ms compiling +      0ms waiting while compiling lib/application.ex
[profile]    146ms compiling +      0ms waiting while compiling lib/formatter.ex
[profile]     42ms compiling +      0ms waiting while compiling lib/token.ex
[profile]    163ms compiling +      0ms waiting while compiling lib/pool.ex
[profile]    229ms compiling +      0ms waiting while compiling lib/mix/tasks/sql.gen.test.ex
[profile]    337ms compiling +      0ms waiting while compiling lib/bnf.ex
[profile]    299ms compiling +      0ms waiting while compiling lib/sql.ex
[profile]   1124ms compiling +      0ms waiting while compiling lib/parser.ex
Compiling lib/lexer.ex (it's taking more than 10s)
[profile]  10532ms compiling +      0ms waiting while compiling lib/lexer.ex
Compiling lib/format.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/tds.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/postgres.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/mysql.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Compiling lib/adapters/ansi.ex (it's taking more than 10s)
    (elixir 1.20.0-rc.3) src/elixir_erl_compiler.erl:20: :elixir_erl_compiler.spawn/1
    (elixir 1.20.0-rc.3) src/elixir_module.erl:168: :elixir_module.compile/7
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:41: :elixir_compiler.optimize_defmodule/2
    (elixir 1.20.0-rc.3) src/elixir_lexical.erl:18: :elixir_lexical.run/3
    (elixir 1.20.0-rc.3) src/elixir_compiler.erl:21: :elixir_compiler.quoted/3
    (elixir 1.20.0-rc.3) lib/kernel/parallel_compiler.ex:549: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

[profile]  21733ms compiling +    100ms waiting for module SQL.BNF while compiling lib/adapters/tds.ex
[profile]                    |    174ms waiting for module SQL.Token while compiling lib/adapters/tds.ex
[profile]  21945ms compiling +    106ms waiting for module SQL.BNF while compiling lib/adapters/ansi.ex
[profile]                    |    178ms waiting for module SQL.Token while compiling lib/adapters/ansi.ex
[profile]  23180ms compiling +    101ms waiting for module SQL.BNF while compiling lib/adapters/mysql.ex
[profile]                    |    175ms waiting for module SQL.Token while compiling lib/adapters/mysql.ex
[profile]  24365ms compiling +     97ms waiting for module SQL.BNF while compiling lib/adapters/postgres.ex
[profile]                    |    140ms waiting for module SQL.Token while compiling lib/adapters/postgres.ex
[profile] 199660ms compiling +    193ms waiting for module SQL.BNF while compiling lib/format.ex
[profile]    127ms compiling + 177594ms waiting for module SQL.Format while compiling lib/mix/tasks/sql.get.ex
[profile]                    |  11679ms waiting for module SQL.Adapters.ANSI while compiling lib/mix/tasks/sql.get.ex
[profile]                    |  10171ms waiting for module SQL.Lexer while compiling lib/mix/tasks/sql.get.ex
[profile]                    |    300ms waiting for module SQL while compiling lib/mix/tasks/sql.get.ex
[profile] Finished cycle resolution in 0ms
[profile] Finished compilation cycle of 20 modules in 199885ms
[profile] Finished writing modules to disk in 5ms
[profile] Finished after compile callback in 204ms
Verifying SQL.Format (it's taking more than 10s)
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5770: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6210: Module.Types.Descr.iterator_merge/3
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:30: Module.Types.Pattern.previous_subtype?/2
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:218: Module.Types.Pattern.of_head/8
    (elixir 1.20.0-rc.3) lib/module/types.ex:332: anonymous fn/7 in Module.Types.local_handler/8
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types.ex:325: Module.Types.local_handler/8

Verifying SQL.Adapters.Postgres (it's taking more than 10s)
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5768: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5770: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6210: Module.Types.Descr.iterator_merge/3
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:30: Module.Types.Pattern.previous_subtype?/2
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:218: Module.Types.Pattern.of_head/8
    (elixir 1.20.0-rc.3) lib/module/types.ex:332: anonymous fn/7 in Module.Types.local_handler/8
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3

Verifying SQL.Adapters.MySQL (it's taking more than 10s)
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6144: Module.Types.Descr.bdd_compare/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5768: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5770: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6210: Module.Types.Descr.iterator_merge/3
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:30: Module.Types.Pattern.previous_subtype?/2
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:218: Module.Types.Pattern.of_head/8
    (elixir 1.20.0-rc.3) lib/module/types.ex:332: anonymous fn/7 in Module.Types.local_handler/8

Verifying SQL.Adapters.ANSI (it's taking more than 10s)
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6144: Module.Types.Descr.bdd_compare/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5768: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5770: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6210: Module.Types.Descr.iterator_merge/3
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:30: Module.Types.Pattern.previous_subtype?/2
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:218: Module.Types.Pattern.of_head/8
    (elixir 1.20.0-rc.3) lib/module/types.ex:332: anonymous fn/7 in Module.Types.local_handler/8

Verifying SQL.Adapters.TDS (it's taking more than 10s)
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5753: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:5770: Module.Types.Descr.bdd_union/2
    (elixir 1.20.0-rc.3) lib/module/types/descr.ex:6210: Module.Types.Descr.iterator_merge/3
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:30: Module.Types.Pattern.previous_subtype?/2
    (elixir 1.20.0-rc.3) lib/module/types/pattern.ex:218: Module.Types.Pattern.of_head/8
    (elixir 1.20.0-rc.3) lib/module/types.ex:332: anonymous fn/7 in Module.Types.local_handler/8
    (elixir 1.20.0-rc.3) lib/enum.ex:2617: Enum."-reduce/3-lists^foldl/2-0-"/3

[profile] Finished group pass check of 20 modules in 290645ms
<- Ran mix compile.elixir in 490773ms
-> Running mix compile.app --force --profile time (inside SQL.MixProject)
Generated sql app
<- Ran mix compile.app in 5ms
<- Ran mix compile.all in 490812ms
<- Ran mix compile in 490926ms
<- Ran mix CLI in 490995ms

[josevalim]

Thank you. This will help find more optimizations but in some cases you are defining more than 3000 clauses and those will eventually trigger some performance slow down, either in the compiler or the type system. I will send a PR to reduce some of those and then try to improve the rest from here.

[josevalim]

This PR brings it to 10 seconds: elixir-dbvisor/sql#28

[Schultzer]

Amazing, I'll take a look at the rest of the codebase, I suspect we might be running into similiar issues in ex_cldr althught not sure the fix is as simple as that one.

So those two libraries might be good benchmarking cases for compiler optimizations.

cc @kipcole9

[josevalim]

Commit c8c7abc make types checking on sql 40% faster (from 5s to 3s on my machine). I am not sure if I can push it much beyond that. Checking for redundant clauses is quadratic in terms of complexity (as one clause has to compare with all previous clauses), so reducing the amount of clauses (like we did with color) will be essential. I will play with it a bit more tomorrow.

You will also gain by optimizing the lexer and format. They are the ones holding down compilation and are not directly related to the type system.

[Schultzer]

Is there an easy way to see how many clauses are generated, in most of my micro benchmarks I found that generating lots of clauses are more performant, not by a lot, but I still like to push it as far as my abilities takes me, I’m aware that there might be better ways to improve performance then brute force, but I haven’t learned that yet.

[josevalim]

In your case, since you have for loops generating clauses, I simply computed the Enum.count of the generator and multiplied by the number of clauses defined inside the block.

In any case, sql compiles immediately on Elixir main with my sql patch (and type checking takes 1.2s prior to my sql patch).