Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/modules/trino/pages/usage-guide/catalogs/delta-lake.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Delta Lake

[Delta Lake](https://delta.io/) is an open-source storage framework that enables building a Lakehouse architecture with support for multiple compute engines.
https://delta.io/[Delta Lake] is an open-source storage framework that enables building a Lakehouse architecture with support for multiple compute engines.
It depends on a Hive metastore being present and makes use of the same metastore ConfigMap used by the Hive connector.

== Example Delta Lake catalog configuration
Expand All @@ -27,8 +27,16 @@ spec:
accessStyle: Path
credentials:
secretClass: minio-credentials
# TLS is required for S3 connections, see the note below
tls:
verification:
server:
caCert:
secretClass: minio-tls-certificates
----

IMPORTANT: The `tls` section is required, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3].

== Connect to S3 store or HDFS

The Delta Lake connector connects to S3 or HDFS in the same way the xref:usage-guide/catalogs/hive.adoc[] connector does.
Expand Down
17 changes: 15 additions & 2 deletions docs/modules/trino/pages/usage-guide/catalogs/hive.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,19 @@ spec:
accessStyle: Path
credentials:
secretClass: minio-credentials
configOverrides: # <4>
tls: # <4>
verification:
server:
caCert:
secretClass: minio-tls-certificates
configOverrides: # <5>
hive.metastore.username: trino
----
<1> The name of the catalog as it will appear in Trino
<2> TrinoCluster can use these labels to select which catalogs to include
<3> The name of your Hive Stacklet
<4> Use `configOverrides` to add arbitrary properties to the Trino catalog configuration
<4> TLS is required for S3 connections, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3]
<5> Use `configOverrides` to add arbitrary properties to the Trino catalog configuration

== Connect to S3 store

Expand All @@ -50,11 +56,18 @@ spec:
accessStyle: Path
credentials:
secretClass: minio-credentials
tls:
verification:
server:
caCert:
secretClass: minio-tls-certificates
# OR
s3:
reference: my-minio
----

IMPORTANT: The `tls` section is required, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3].

See xref:concepts:s3.adoc[] for details about S3 connections.

IMPORTANT: Make sure that the underlying Hive metastore also has access to the S3 store, because it will e.g. check if the directory exists when creating tables.
Expand Down
6 changes: 6 additions & 0 deletions docs/modules/trino/pages/usage-guide/catalogs/iceberg.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,15 @@ spec:
accessStyle: Path
credentials:
secretClass: minio-credentials
tls: # <3>
verification:
server:
caCert:
secretClass: minio-tls-certificates
----
<1> The name of the catalog as it will appear in Trino
<2> This is the name of your Hive Stacklet
<3> TLS is required for S3 connections, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3]

== Connect to S3 store or HDFS
The Iceberg connector can connect to S3 or HDFS in the same way the xref:usage-guide/catalogs/hive.adoc[] connector does.
Expand Down
30 changes: 20 additions & 10 deletions docs/modules/trino/pages/usage-guide/s3.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
:description: Configure S3 connections in Trino either inline within the TrinoCatalog or via an external S3Connection resource for centralized management.

You can specify S3 connection details directly inside the TrinoCatalog specification or by referring to an external S3Connection custom resource.
This mechanism used used across the whole Stackable Data Platform, read the xref:concepts:s3.adoc[S3 concepts page] to learn more.
This mechanism is used across the whole Stackable Data Platform, read the xref:concepts:s3.adoc[S3 concepts page] to learn more.

[#tls-required]
IMPORTANT: Every S3 connection must contain a `tls` section, because Trino uses the native S3 client, which requires TLS.

== Inline

Expand All @@ -14,22 +17,24 @@ s3: # <1>
inline:
host: test-minio # <2>
port: 9000 # <3>
pathStyleAccess: true # <4>
secretClass: minio-credentials # <5>
tls:
accessStyle: Path # <4>
credentials:
secretClass: minio-credentials # <5>
tls: # <6>
verification:
server:
caCert:
secretClass: minio-tls-certificates #<6>
secretClass: minio-tls-certificates
----
<1> Entry point for the connection configuration
<2> Connection host
<3> Optional connection port
<4> Optional flag if path-style URLs should be used; This defaults to `false`
which means virtual hosted-style URLs are used.
<5> Name of the Secret object expected to contain the following keys:
`accessKey` and `secretKey`
<6> Optional TLS settings for encrypted traffic. The `secretClass` can be provided by the Secret Operator or yourself.
<4> Optional access style, either `Path` or `VirtualHosted`; This defaults to
`VirtualHosted`, which means virtual hosted-style URLs are used.
<5> Name of the SecretClass providing the credentials. The Secret behind it is expected
to contain the following keys: `accessKey` and `secretKey`
<6> TLS settings for encrypted traffic. The `secretClass` can be provided by the Secret Operator or yourself.
This section is required, see <<tls-required,the note above>>.

A self provided S3 TLS secret can be specified like this:

Expand Down Expand Up @@ -84,6 +89,11 @@ spec:
accessStyle: Path
credentials:
secretClass: minio-credentials
tls:
verification:
server:
caCert:
secretClass: minio-tls-certificates
----

This has the advantage that the connection configuration is configured in a single place and can be shared across applications, reducing the cost of updating these details.
Loading