Cleanup legacy file duplication checks and feature flags

[lferran]

Description

[sc-14410]

How was this PR tested?

Describe how you tested this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.19%. Comparing base (8ce30da) to head (93855ff).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3618   +/-   ##
=======================================
  Coverage   85.18%   85.19%           
=======================================
  Files         554      554           
  Lines       47517    47485   -32     
  Branches    13640    13640           
=======================================
- Hits        40479    40453   -26     
+ Misses       6449     6444    -5     
+ Partials      589      588    -1     

Flag	Coverage Δ
nidx	79.72% <ø> (+<0.01%)	⬆️
nucliadb	73.33% <87.50%> (+0.03%)	⬆️
nucliadb-ingest	43.67% <62.50%> (+0.01%)	⬆️
nucliadb-reader	43.67% <12.50%> (+<0.01%)	⬆️
nucliadb-search	54.29% <25.00%> (+0.03%)	⬆️
nucliadb-standalone	46.09% <50.00%> (+0.01%)	⬆️
nucliadb-train	44.75% <25.00%> (+0.02%)	⬆️
nucliadb-writer	47.01% <87.50%> (-0.05%)	⬇️
nucliadb_dataset	73.77% <ø> (ø)
nucliadb_models	71.64% <ø> (ø)
nucliadb_sdk	83.49% <ø> (ø)
nucliadb_telemetry	71.59% <ø> (ø)
nucliadb_utils	80.90% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.