[AlexaCampusano]

Related read Storage Strategies for Collectionsin Dynamically Typed Languages

What are we trying to do?

This PR is a proposal to add fixnums stride information into the array object, so heap arrays have more information about the data it's storing and we can optimize it.

Prior Art: #1081

In this PR I'm targetting all fixnum arrays, not just frozen ones. This PR affects these common paths:

• At first mutation, arrays collect the stride for the fixnum (whether it's 8bit, 16bit, 32bit or VALUE), it also collects the signedness of the value being appended, both get added to the Array flags. We then proceed to narrow the VALUE to its raw representation for that value. This happens the moment the Array gets a heap buffer (on the embedded->heap spill).
• Fixnum Array literals, when the bytecode compiler sees an all literal fixnum array, it builds a hidden frozen template array and embeds it in the iseq of duparray. Now it narrows that template if it's all fixnums, and at runtime duparray copies the narrow buffer into a new narrow array instead of handing out a shared view of the template. The literal owns its own (small) buffer, and the template is collected with its iseq. There are concerns I have with this approach, that I'd like to discuss.
• Mutations keep the array narrow, if a value doesn't fit the current width we promote W8 → W16 → W32 in place. The first non-fixnum (or a fixnum past 32 bits, or a sign change for now) widens the array back to VALUEs, once, and it stays wide.
• Copies stay narrow. dup, clone, a[i, n], first(n), last(n), pop(n), replace, sort_by! copy the narrow buffer rather than sharing it, so neither the source nor the copy gets widened by being read from.
• Reading an element decodes the raw integer back into a VALUE on the fly, so looking at a narrow array never changes its representation. C extensions are the exception: they read arrays through the public RARRAY_PTR family, which has to return a buffer of actual VALUEs. When an extension asks for that pointer on a narrow array, the array is widened first, once, and stays wide. Extensions keep working unchanged; they just lose the memory saving for the arrays they touch.
• freeze no longer does the narrowing. By the time an array is frozen it's already narrow from how it was built, so freeze just trims the unused capacity. The one case it still narrows is an array that was filled by C code through a raw pointer, since that path never went through our write logic.
• Both JITs already called into C for most array work, and C is stride-aware, so they mostly didn't need changes. The exceptions are the handful of places where a JIT reads or writes the buffer directly in generated code. Those now check the stride first and fall back to the C path for narrow arrays, so compiled code never touches a narrow buffer as if it held VALUEs.

Some numbers!!!!

More narrowed paths exercised vs the previous #1081 work.

 Memory: master -> patch   (base 8fdf4342b5; patch = e8ed7bed71 + rb_ary_widen export)
     Array/total = ObjectSpace.memsize_of_all after full GC, including hidden literal templates
     (dump_all over every ARRAY); RSS = mean per-iteration after warmup

     == Narrow path exercised ==============================================

     blurhash
       1 array x 124,848 elems (pixel buffer, make_shareable)
       Array       1168.0 KiB  ->      311.9 KiB   -73.3%
       total          3.3 MiB  ->        2.4 MiB   -25.7%
       RSS           20.4 MiB  ->       19.5 MiB    -4.2%

     mail
       Ragel parser tables, unfrozen literals (hidden templates 3038 KiB -> 75 KiB)
       Array       3289.7 KiB  ->     1026.4 KiB   -68.8%
       total          8.8 MiB  ->        6.5 MiB   -25.6%
       RSS           74.6 MiB  ->       73.2 MiB    -1.8%

     erubi-rails
       same mail-gem tables via ActionMailer (hidden templates 3172 KiB -> 86 KiB)
       Array       3747.6 KiB  ->     1383.1 KiB   -63.1%
       total         12.3 MiB  ->        9.9 MiB   -18.9%
       RSS          117.6 MiB  ->      118.2 MiB    +0.5%

     railsbench
       same mail-gem tables (hidden templates 3201 KiB -> 104 KiB)
       Array       4149.8 KiB  ->     1784.4 KiB   -57.0%
       total         15.8 MiB  ->       13.5 MiB   -14.7%
       RSS          124.6 MiB  ->      125.1 MiB    +0.5%

     lobsters
       same mail-gem tables (hidden templates 3919 KiB -> 133 KiB)
       Array       5848.7 KiB  ->     2958.3 KiB   -49.4%
       total         34.2 MiB  ->       31.4 MiB    -8.2%
       RSS          374.8 MiB  ->      341.4 MiB    -8.9%

     chunky-png
       Array        599.6 KiB  ->      427.0 KiB   -28.8%
       total          4.4 MiB  ->        4.3 MiB    -4.2%
       RSS           99.5 MiB  ->      100.9 MiB    +1.4%

     rubocop
       Array       2743.0 KiB  ->     2167.3 KiB   -21.0%
       total         13.5 MiB  ->       12.9 MiB    -4.3%
       RSS          109.6 MiB  ->      115.2 MiB    +5.2%

     lee
       Array        310.1 KiB  ->      267.8 KiB   -13.6%
       total          4.1 MiB  ->        4.1 MiB    -1.2%
       RSS           44.3 MiB  ->       41.5 MiB    -6.3%

     optcarrot
       emulator tables, Array.new(n, 0) / [0] * n, mutable
       Array      30834.5 KiB  ->    28995.0 KiB    -6.0%
       total         31.2 MiB  ->       29.4 MiB    -5.8%
       RSS           62.2 MiB  ->       53.6 MiB   -13.8%

     sudoku
       Array        280.3 KiB  ->      260.4 KiB    -7.1%
       total          2.3 MiB  ->        2.3 MiB    -1.4%
       RSS           21.5 MiB  ->       21.2 MiB    -1.5%

     liquid-compile   Array -9.2%   total -3.5%   RSS -2.7%
     knucleotide      Array -9.1%   total +0.1%   RSS  0.0%
     graphql          Array -5.2%   total -0.8%   RSS -0.4%
     nbody            Array -4.9%   total -0.9%   RSS -0.6%
     sequel           Array -4.9%   total -0.1%   RSS +0.2%
     protoboeuf       Array -4.4%   total -0.4%   RSS +0.1%
     liquid-c         Array -4.1%   total -0.9%   RSS -3.2%

     == Narrow path not exercised (Array within -3%, total within -1%) ========

     ruby-lsp, nqueens, ruby-json, liquid-il, binarytrees, liquid-render,
     activerecord, psych-load, fannkuchredux, matmul, rubyboy, hexapdf, shipit

     No benchmark grew.  RSS deltas under ~3% on processes over 100 MiB are noise.

Read paths

 Reads: master -> patch   (base 8fdf4342b5; patch 7d5b74b70b + single-test RARRAY_AREF/RARRAY_LEN)
     Interpreter, ns per call, best of 3 runs, arrays built by push.
     W8: i%200   W16: i%60_000   W32: i*40_000   VALUE: i*2**40 (past 32 bits, stays wide on both)

     == W8 ======================================================================
                                 1,000 elements                      100,000 elements
       include? hit        269 ns  ->    48 ns   -82%        25 ns  ->    30 ns   +20%
       include? miss       2.2 us  ->    23 ns   -99%     204.9 us  ->    25 ns  -100%
       index miss          2.0 us  ->    24 ns   -99%     198.0 us  ->    25 ns  -100%
       count               2.1 us  ->   135 ns   -94%     205.0 us  ->  11.7 us   -94%
       sum                 538 ns  ->    62 ns   -89%      49.8 us  ->   4.4 us   -91%
       max                 467 ns  ->    33 ns   -93%      45.8 us  ->   815 ns   -98%
       min                 464 ns  ->    33 ns   -93%      45.2 us  ->   775 ns   -98%
       ==                  331 ns  ->    90 ns   -73%      24.7 us  ->   1.9 us   -92%
       hash                3.3 us  ->   3.1 us    -6%     329.1 us  -> 299.9 us    -9%
       uniq                6.4 us  ->   6.7 us    +4%     554.2 us  -> 561.8 us    +1%
       each               15.1 us  ->  16.1 us    +7%      1.55 ms  ->  1.60 ms    +3%
       a[i] loop          17.5 us  ->  17.2 us    -2%      1.75 ms  ->  1.81 ms    +4%

     == W16 =====================================================================
                                 1,000 elements                      100,000 elements
       include? hit        1.1 us  ->   157 ns   -86%      98.0 us  ->  11.3 us   -89%
       include? miss       2.2 us  ->    23 ns   -99%     197.2 us  ->    25 ns  -100%
       index miss          2.0 us  ->    26 ns   -99%     198.2 us  ->    25 ns  -100%
       count               2.0 us  ->   137 ns   -93%     198.1 us  ->  11.4 us   -94%
       sum                 663 ns  ->    60 ns   -91%      70.7 us  ->   4.4 us   -94%
       max                 470 ns  ->    42 ns   -91%      46.0 us  ->   1.9 us   -96%
       min                 466 ns  ->    39 ns   -92%      45.7 us  ->   1.9 us   -96%
       ==                  331 ns  ->   116 ns   -65%      24.4 us  ->   3.5 us   -85%
       hash                3.2 us  ->   3.1 us    -3%     321.1 us  -> 303.7 us    -5%
       uniq                9.2 us  ->   8.6 us    -7%     838.9 us  -> 865.8 us    +3%
       each               15.2 us  ->  16.4 us    +8%      1.55 ms  ->  1.61 ms    +4%
       a[i] loop          17.4 us  ->  17.6 us    +1%      1.74 ms  ->  1.73 ms    -1%

     == W32 =====================================================================
                                 1,000 elements                      100,000 elements
       include? hit        1.2 us  ->   157 ns   -87%     120.1 us  ->  11.9 us   -90%
       include? miss       2.4 us  ->    23 ns   -99%     219.1 us  ->    25 ns  -100%
       index miss          2.3 us  ->    26 ns   -99%     197.7 us  ->    25 ns  -100%
       count               2.0 us  ->   126 ns   -94%     216.0 us  ->  10.4 us   -95%
       sum                 534 ns  ->    55 ns   -90%      64.8 us  ->   3.8 us   -94%
       max                 478 ns  ->    60 ns   -87%      46.9 us  ->   3.7 us   -92%
       min                 479 ns  ->    62 ns   -87%      45.7 us  ->   3.8 us   -92%
       ==                  319 ns  ->   151 ns   -53%      24.8 us  ->   6.9 us   -72%
       hash                3.4 us  ->   3.0 us   -10%     314.3 us  -> 303.3 us    -3%
       uniq                8.2 us  ->   8.4 us    +1%     990.6 us  ->  1.01 ms    +2%
       each               15.0 us  ->  16.2 us    +8%      1.56 ms  ->  1.62 ms    +4%
       a[i] loop          17.3 us  ->  16.3 us    -6%      1.77 ms  ->  1.61 ms    -9%

     == VALUE (wide on both; control) ==========================================
                                 1,000 elements                      100,000 elements
       include? hit        1.2 us  ->   1.2 us    +0%     108.3 us  ->  97.5 us   -10%
       include? miss       2.0 us  ->   2.4 us   +24%*    218.4 us  -> 196.8 us   -10%
       index miss          2.0 us  ->   2.2 us   +12%*    220.8 us  -> 220.9 us    +0%
       count               2.0 us  ->   2.2 us   +11%*    220.5 us  -> 198.0 us   -10%
       sum                 536 ns  ->   521 ns    -3%      92.7 us  ->  74.5 us   -20%
       max                 476 ns  ->   483 ns    +1%      45.8 us  ->  47.0 us    +3%
       min                 467 ns  ->   484 ns    +4%      46.3 us  ->  45.9 us    -1%
       ==                  332 ns  ->   321 ns    -3%      24.7 us  ->  24.5 us    -1%
       hash                3.1 us  ->   3.4 us    +9%     332.7 us  -> 315.2 us    -5%
       uniq                6.7 us  ->   6.7 us    -0%     653.2 us  -> 661.4 us    +1%
       each               15.2 us  ->  15.7 us    +3%      1.56 ms  ->  1.61 ms    +4%
       a[i] loop          17.3 us  ->  17.6 us    +2%      1.76 ms  ->  1.75 ms    -1%