Null values cause Oura tasks to get killed

[mpgxvii]

• When the Oura connector receives a null value when pulling from the REST API, the task gets killed and needs a manual restart

Error logs:

{"id":4,"state":"FAILED","worker_id":"radar-oura-connector:8083","trace":"org.apache.kafka.connect.errors.DataException: Found null value for non-optional schema\n\tat io.confluent.connect.avro.AvroData.validateSchemaValue(AvroData.java:1369)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1423)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1418)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1679)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1418)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1414)\n\tat io.confluent.connect.avro.AvroData.toConnectData(AvroData.java:1394)\n\tat org.radarbase.connect.rest.oura.OuraSourceTask.lambda$handleRequest$3(OuraSourceTask.java:108)\n\tat java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)\n\tat java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)\n\tat java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)\n\tat java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)\n\tat or

[callumstew]

I think this is a similar to a problem in the recent HRfH testing

A user tried to connect their oura account and it appeared successful from their end. However, the registrations/{id}/authorize endpoint returned an error because the JSON response contains null values.

I believe the OuraAuthUserId object needs a nullable email field.

current class:

data class OuraAuthUserId(
    val age: Int? = null,
    val weight: Float? = null,
    val height: Float? = null,
    val biological_sex: String? = null,
    val email: String,
    @SerialName("id") val userId: String,
)

The error:

io.ktor.serialization.JsonConvertException: Illegal input: Unexpected JSON token at offset 114: Expected string literal but 'null' literal was found at path: $.email
Use 'coerceInputValues = true' in 'Json {}` builder to coerce nulls to default values.
JSON input: {"id":"a72bee9","age":null,"weight":null,"height":null,"biological_sex":null,"email":null}
        at io.ktor.serialization.kotlinx.KotlinxSerializationConverter.deserialize(KotlinxSerializationConverter.kt:90) ~[ktor-serialization-kotlinx-jvm-2.3.11.jar:2.3.11]
        at io.ktor.serialization.ContentConverterKt$deserialize$$inlined$map$1$2.emit(Emitters.kt:224) ~[ktor-serialization-jvm-2.3.11.jar:2.3.11]
        at kotlinx.coroutines.flow.FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$3.collect(SafeCollector.common.kt:114) ~[kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at io.ktor.serialization.ContentConverterKt$deserialize$$inlined$map$1.collect(SafeCollector.common.kt:113) ~[ktor-serialization-jvm-2.3.11.jar:2.3.11]
        at kotlinx.coroutines.flow.FlowKt__ReduceKt.firstOrNull(Reduce.kt:239) ~[kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.flow.FlowKt.firstOrNull(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at io.ktor.serialization.ContentConverterKt.deserialize(ContentConverter.kt:123) ~[ktor-serialization-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.contentnegotiation.ContentNegotiation.convertResponse$ktor_client_content_negotiation(ContentNegotiation.kt:232) ~[ktor-client-content-negotiation-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$2.invokeSuspend(ContentNegotiation.kt:264) ~[ktor-client-content-negotiation-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$2.invoke(ContentNegotiation.kt) ~[ktor-client-content-negotiation-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$2.invoke(ContentNegotiation.kt) ~[ktor-client-content-negotiation-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:131) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:89) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.HttpClient$4.invokeSuspend(HttpClient.kt:177) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.HttpClient$4.invoke(HttpClient.kt) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.HttpClient$4.invoke(HttpClient.kt) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:131) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:89) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:99) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.HttpCallValidator$Companion$install$2.invokeSuspend(HttpCallValidator.kt:142) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.HttpCallValidator$Companion$install$2.invoke(HttpCallValidator.kt) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.plugins.HttpCallValidator$Companion$install$2.invoke(HttpCallValidator.kt) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:131) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:89) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:109) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) ~[ktor-utils-jvm-2.3.11.jar:2.3.11]
        at io.ktor.client.call.HttpClientCall.bodyNullable(HttpClientCall.kt:89) ~[ktor-client-core-jvm-2.3.11.jar:2.3.11]
        at org.radarbase.authorizer.service.OuraAuthorizationService$getExternalId$2.invokeSuspend(OuraAuthorizationService.kt:112) ~[authorizer-app-backend-4.4.9.jar:?]
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) [kotlin-stdlib-1.9.23.jar:1.9.23-release-779]
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104) [kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111) ~[kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99) ~[kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585) [kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802) [kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706) [kotlinx-coroutines-core-jvm-1.8.0.jar:?]
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693) [kotlinx-coroutines-core-jvm-1.8.0.jar:?]
Caused by: kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 114: Expected string literal but 'null' literal was found at path: $.email
Use 'coerceInputValues = true' in 'Json {}` builder to coerce nulls to default values.
JSON input: {"id":"a72bee9","age":null,"weight":null,"height":null,"biological_sex":null,"email":null}
        at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:24) ~[kotlinx-serialization-json-jvm-1.5.1.jar:?]
        at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:32) ~[kotlinx-serialization-json-jvm-1.5.1.jar:?]
        at kotlinx.serialization.json.internal.AbstractJsonLexer.fail(AbstractJsonLexer.kt:584) ~[kotlinx-serialization-json-jvm-1.5.1.jar:?]```