OCaml 5.3 compatibility

[fantazio]

Important

To merge after release 1.1.1

Other than updating the OCaml version to 5.3 and fixing the compilation mistakes, this PR includes 2 important changes :

1. .cmti files are used instead of .cmi files. The cmi_infos are now read from either the .cmti if it exists or the .cmt otherwise. This simplifies a lot the initialization of State.File_infos.t. This type has evolved to only expose the information used from the various cm*_infos read. Using .cmti provides information used in the next point.

2. the cmt_infos.cmt_value_dependencies is recreated (as a list of pairs of locations, to only expose what is actually used). Indeed, it is replaced by cmt_declaration_dependencies which exposes a lot more dependencies than only values, and some uids in that list are missing in the cmt_infos.cmt_uid_to_decl. To recreate the value dependencies, the latter field of both the .cmt and .cmti files are used, and missing uids' locations are retrieved by opening their corresponding .cmti/.cmt (if provided).

Introducing a cache for cmt files

Point 2 above may require reading multiple cmt/cmti files to retrieve their cmt_uid_to_decl tables in order to construct a single equivalent to the cmt_value_dependencies. This strongly and negatively impacts the performance, hindering its use on real life projects. E.g. on Frama-C, the execution would take >5min on my machine vs ~10s when using the 5.2-compatible version of the analyzer.

Important

Having to read multiple cmt/cmti files to reconstruct the equivalent of the cmt_value_dependencies is expected as explained in the documentation of Shapes

Note

Merlin relies on caching as well. In particular, it uses a cache in its shape reduction functions.
We cannot rely on shape reductions because shapes contain Uids and we rely on the locations dependencies at the end of the analysis of each compilation unit.
A possible future improvement would be to rely less on locations during the analysis and only resolve uids to their corresponding locations after the analysis (and before reporting of course). In this situation, the uid -> loc associations and uid -> uid dependencies would be stored to be used all at once during the final resolution.

Caching mechanism

The implemented caching mechanism is very naive :

• A maximum size is defined (can be defined using --cmt_cache_size; default is 64);
• Each time a cmt/cmti file is read, it is cached;
• If the number of cached files reached the maximum size, the cache is cleared before caching a new file.

It uses a Hashtbl for speed, and hit/miss statistics are gathered during the execution. These statistics can be displayed by calling Cmt.print_cache_stats ().

Speed improvement

In order to scale the tool the large projects, a cache is introduced to keep cmt/cmti files in memory. Although the cache size is limited (to avoid moving the performance issue from speed to memory), it drastically reduces the IO operations and the result is comparable to the 5.2-compatible version.
The table below details time measurements (using time) of the OCaml 5.2-compatbile version and this PR's version of the dead_code_analyzer on Frama-C, and their relative difference (5.3 / 5.2 - 1) :

cli		--nothing	-a -E all	-a -M all	-a -T all	-a -Oa all	-a -On all	-a -S +all	--all
5.2	11.36s	9.08s	10.01s	10.43s	9.17s	10.25s	10.31s	9s	12.12s
5.3	11.7s	9.18s	10.7s	9.83s	9.28s	9.65s	9.69s	9.12s	12.18s
rel	+2.99%	+1.10%	+6.89%	-5.75%	+1.20%	-5.85%	-6.01%	+1.33%	+0.50%

As we can see, the speed of the new version is of the same order of magnitude as the OCaml 5.2-compatible version. It sometimes appears faster but the methodology used is not precise enough to assert that. The differences are mostly within the error margin.

As a point of comparison, the new version without cache takes 308.82s with the --nothing option, and 333.08s with the --all option, making it ~30 times slower.

Impact on memory

Caching the cmt/cmti files has a cost. The table below details memory measurements (the maximum resident size returned by time) of the OCaml 5.2-compatbile version and this PR's version of the dead_code_analyzer on Frama-C, and their relative difference (5.3 / 5.2 - 1) :

cli		--nothing	-a -E all	-a -M all	-a -T all	-a -Oa all	-a -On all	-a -S +all	--all
5.2	0.59Gb	0.58Gb	0.57Gb	0.60Gb	0.57Gb	0.67Gb	0.67Gb	0.57Gb	0.71Gb
5.3	0.69Gb	0.89Gb	0.71Gb	0.95Gb	0.76Gb	1.01Gb	1.01Gb	0.74Gb	1.04Gb
rel	+16.95%	+53.06%	+23.92%	+59.31%	+33.78%	+50.73%	+50.66%	+28.96%	+45.38%

As we can see, the memory consumption almost reaches a 60% increase in some cases. As before, the methodology is not precise enough to assert anything but the change in order of magnitude is clear.

Selecting the cache size

The default cache size is 64. It has been selected because it is the smallest power of 2 with similar time performance to the OCaml 5.2-compatible version.
A new command line option --cmt_cache_size gives the user control over the maximum amount of cmt/cmti files cached.

The 2 tables below details the time and memory performance of different cache sizes compared to the previous cache size and compared to a cache size = 64, measured on Frama-C.

Cache size relative time performance

cache_size	1	2	4	8	16	32	64	128	256
rel pred	NA	-41.02%	-45.30%	-37.39%	-23.27%	-14.83%	-12.03%	-10.78%	-4.51%
rel 64	+761.06%	+407.88%	+177.80%	+73.93%	+33.47%	+13.68%	+0.00%	-10.78%	-14.80%

Cache size relative memory performance

cache_size	1	2	4	8	16	32	64	128	256
rel pred	NA	-65.49%	-57.14%	-2.21%	-9.68%	+23.44%	+1.40%	-0.26%	+20.45%
rel 64	+511.62%	+111.07%	-9.54%	-11.54%	-20.11%	-1.38%	+0.00%	-0.26%	+20.14%

As we can see, the time performance improves as the cache size increases. The improvement cache size doubling reduces, displaying a logarithmic behavior.

More surprisingly, the maximum resident size decreases as the cache size increases at first, then increases at cache size = 32, stabilizes until cache size = 128, and finally increases again. Unlike for time performance, the impact on the memory is less predictable. I have not investigated this behavior in details (I suspect the measurement methodology and the GC to be mostly responsible).

Results change

The test-suite results are identical (module a few blank lines rightfully removed). The results on Frama-C change a little as detailed in the table below. This table describes the number of reports produced for a given section (or the combination of them all for --all) and a version of the analyzer. It also provides the actual number of deletions and additions from the OCaml 5.2-compatible version to the OCaml 5.3-compatible. Finally, it shows the relative differences, compared to the OCaml 5.2-compatible version for deletions, the 5.3-compatible for additions, and the 5.2-compatible for overall (deletions + additions)

cli	-a -E all	-a -M all	-a -T all	-a -Oa all	-a -On all	-a -S +all	--all
5.2	2868	521	202	1043	1023	311	5968
5.3	2859	521	200	1031	950	311	5872
-	27	0	2	20	99	0	148
+	18	0	0	8	26	0	52
rel -	0.94%	0%	0.99%	1.92%	9.68%	0%	2.48%
rel +	0.63%	0%	0%	0.78%	2.74%	0%	0.89%
rel	1.57%	0%	0.99%	2.68%	12.22%	0%	3.35%

As we can see, the update to OCaml 5.3 does not modify the results too much overall (<4%). However the changes in the optional arguments never used are important (>12%).
After triaging the changes, I was able to obtain the results described in the table below. Deletions that were false positives and additions that were false negatives are discarded from the difference.

cli	-a -E all	-a -M all	-a -T all	-a -Oa all	-a -On all	-a -S +all	--all
-	0	0	0	3	2	0	5
+	18	0	0	6	24	0	48
rel -	0%	0%	0%	0.29%	0.20%	0%	0.08%
rel +	0.63%	0%	0%	0.58%	2.53%	0%	0.82%
rel	0.63%	0%	0%	0.86%	2.54%	0%	0.89%

The changes are very minimal (<1% overall, <3% at worst). The introduced FP seem to be caused by the same issues that caused them in the 5.2-compatible version. Similarly for the FN. Most of the deletions were false positives. After triaging the results, the update actually introduces more additions than deletions (while it was the opposite before) and the amount of corrected errors (147) is clearly above that of introduced errors (53).