fix: apply schema-force-optional recursively in schema evolution path [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]

[waterWang]

Description

When iceberg.tables.schema-force-optional=true is set and schema evolution is enabled, the RecordConverter only makes a table field optional when the incoming Connect record field's schema is already optional. It does not check config.schemaForceOptional(), so required nested struct fields in the Connect schema remain required in the Iceberg table.

This is a gap in the schema evolution path: toIcebergType already applies schemaForceOptional recursively for initial schema creation (auto-create), but the makeOptional logic in RecordConverter does not consult the flag.

Changes

1. RecordConverter.java — convertToStruct(Struct): Changed the makeOptional condition from tableField.isRequired() && recordField.schema().isOptional() to tableField.isRequired() && (config.schemaForceOptional() || recordField.schema().isOptional()).

2. RecordConverter.java — evolveSchemaFromConnectSchema: Same change for the nested-field case.

3. TestSchemaUtils.java — Added testToIcebergTypeNestedStruct parameterized test that verifies toIcebergType recursively applies schemaForceOptional to nested struct fields at every level.

Impact

• schemaForceOptional=true now ensures that required fields in the Connect schema are marked optional in the Iceberg schema during evolution, matching the behavior already present in the auto-create path.
• Fixes the scenario where a CDC pipeline's _cdc.key struct (from DebeziumTransform) keeps required nested fields, causing writer failures when the source PK changes and old key fields disappear from incoming records.

Closes #17555