You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a table with text to chunk and insert data:
217
236
218
-
-- Create a table with text to chunk and insert data
237
+
```sql
219
238
CREATETABLEtextchunk
220
239
(
221
240
text_id INT IDENTITY (1, 1) PRIMARY KEY,
@@ -224,30 +243,158 @@ CREATE TABLE textchunk
224
243
GO
225
244
226
245
INSERT INTO textchunk (text_to_chunk)
227
-
VALUES
228
-
('All day long we seemed to dawdle through a country which was full of beauty of every kind. Sometimes we saw little towns or castles on the top of steep hills such as we see in old missals; sometimes we ran by rivers and streams which seemed from the wide stony margin on each side of them to be subject to great floods.'),
229
-
('My Friend, Welcome to the Carpathians. I am anxiously expecting you. Sleep well to-night. At three to-morrow the diligence will start for Bukovina; a place on it is kept for you. At the Borgo Pass my carriage will await you and will bring you to me. I trust that your journey from London has been a happy one, and that you will enjoy your stay in my beautiful land. Your friend, DRACULA')
246
+
VALUES ('All day long we seemed to dawdle through a land which was full of beauty of every kind. Sometimes we saw little towns or castles on the top of steep hills such as we see in old missals; sometimes we ran by rivers and streams which seemed from the wide stony margin on each side of them to be subject to great floods.'),
247
+
('My Friend, Welcome to the Carpathians. I am anxiously expecting you. Sleep well to-night. At three to-morrow the diligence will start for Bukovina; a place on it is kept for you. At the Borgo Pass my carriage will await you and will bring you to me. I trust that your journey from London has been a happy one, and that you will enjoy your stay in my beautiful land. Your friend, DRACULA')
230
248
GO
249
+
```
250
+
251
+
Create a new table to hold the chunked text and vector embeddings:
231
252
232
-
-- Create a new table to hold the chunked text and vector embeddings
253
+
```sql
233
254
CREATETABLEtext_embeddings
234
255
(
235
256
embeddings_id INT IDENTITY (1, 1) PRIMARY KEY,
236
257
chunked_text NVARCHAR (MAX),
237
258
vector_embeddings VECTOR(1536)
238
259
);
260
+
```
239
261
240
-
-- Insert the chunked text and vector embeddings into the text_embeddings table using AI_GENERATE_CHUNKS and AI_GENERATE_EMBEDDINGS
262
+
Insert the chunked text and vector embeddings into the text_embeddings table using `AI_GENERATE_CHUNKS` and `AI_GENERATE_EMBEDDINGS`:
263
+
264
+
```sql
241
265
INSERT INTO text_embeddings (chunked_text, vector_embeddings)
242
-
SELECTc.chunk, AI_GENERATE_EMBEDDINGS(c.chunk USE MODEL MyAzureOpenAIModel)
AI_GENERATE_EMBEDDINGS(c.chunk USE MODEL MyAzureOpenAIModel)
268
+
FROM textchunk AS t
269
+
CROSS APPLY AI_GENERATE_CHUNKS (
270
+
SOURCE =t.text_to_chunk,
271
+
CHUNK_TYPE = FIXED,
272
+
CHUNK_SIZE =100
273
+
) AS c;
274
+
```
275
+
276
+
View the results
277
+
278
+
```sql
279
+
SELECT*
280
+
FROM text_embeddings;
281
+
```
282
+
283
+
### [Managed Identity](#tab/managed-identity)
284
+
285
+
The following example demonstrates an end-to-end process for making your data AI-ready using Managed Identity:
246
286
247
-
-- View the results
248
-
SELECT*FROM text_embeddings;
287
+
1. Enable [Managed Identity](../statements/create-external-model-transact-sql.md#managed-identity). This option applies to SQL Server Azure Arc / Azure Virtual Machine (VM) based deployments.
288
+
289
+
1. Use [CREATE EXTERNAL MODEL with Azure OpenAI using Managed Identity](../statements/create-external-model-transact-sql.md#create-an-external-model-with-azure-openai-using-managed-identity) to register and make your embedding model accessible using Managed Identity.
290
+
291
+
1. Split the dataset into smaller chunks with [AI_GENERATE_CHUNKS](ai-generate-chunks-transact-sql.md), so the data fits within the model's context window and improves retrieval accuracy.
292
+
293
+
1. Generate embeddings using `AI_GENERATE_EMBEDDINGS`.
294
+
295
+
1. Insert the results into a table with a [vector data type](../data-types/vector-data-type.md).
296
+
297
+
> [!NOTE]
298
+
> Replace `<password>` with a [valid password](../statements/create-master-key-transact-sql.md#password-password).
299
+
300
+
Enable managed identity for authentication:
301
+
302
+
```sql
303
+
EXECUTE sp_configure 'allow server scoped db credentials', 1;
304
+
RECONFIGURE WITH OVERRIDE;
249
305
```
250
306
307
+
Enable the external REST endpoint invocation on the database server:
Create a table with text to chunk and insert data:
349
+
350
+
```sql
351
+
CREATETABLEtextchunk
352
+
(
353
+
text_id INT IDENTITY (1, 1) PRIMARY KEY,
354
+
text_to_chunk NVARCHAR (MAX)
355
+
);
356
+
GO
357
+
358
+
INSERT INTO textchunk (text_to_chunk)
359
+
VALUES ('All day long we seemed to dawdle through a land which was full of beauty of every kind. Sometimes we saw little towns or castles on the top of steep hills such as we see in old missals; sometimes we ran by rivers and streams which seemed from the wide stony margin on each side of them to be subject to great floods.'),
360
+
('My Friend, Welcome to the Carpathians. I am anxiously expecting you. Sleep well to-night. At three to-morrow the diligence will start for Bukovina; a place on it is kept for you. At the Borgo Pass my carriage will await you and will bring you to me. I trust that your journey from London has been a happy one, and that you will enjoy your stay in my beautiful land. Your friend, DRACULA')
361
+
GO
362
+
```
363
+
364
+
Create a new table to hold the chunked text and vector embeddings:
365
+
366
+
```sql
367
+
CREATETABLEtext_embeddings
368
+
(
369
+
embeddings_id INT IDENTITY (1, 1) PRIMARY KEY,
370
+
chunked_text NVARCHAR (MAX),
371
+
vector_embeddings VECTOR(1536)
372
+
);
373
+
```
374
+
375
+
Insert the chunked text and vector embeddings into the text_embeddings table using `AI_GENERATE_CHUNKS` and `AI_GENERATE_EMBEDDINGS`:
376
+
377
+
```sql
378
+
INSERT INTO text_embeddings (chunked_text, vector_embeddings)
379
+
SELECTc.chunk,
380
+
AI_GENERATE_EMBEDDINGS(c.chunk USE MODEL MyAzureOpenAIModel)
381
+
FROM textchunk AS t
382
+
CROSS APPLY AI_GENERATE_CHUNKS (
383
+
SOURCE =t.text_to_chunk,
384
+
CHUNK_TYPE = FIXED,
385
+
CHUNK_SIZE =100
386
+
) AS c;
387
+
```
388
+
389
+
View the results:
390
+
391
+
```sql
392
+
SELECT*
393
+
FROM text_embeddings;
394
+
```
395
+
396
+
---
397
+
251
398
## Related content
252
399
253
400
-[CREATE EXTERNAL MODEL (Transact-SQL)](../statements/create-external-model-transact-sql.md)
@@ -256,3 +403,4 @@ SELECT * FROM text_embeddings;
0 commit comments