Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions integrations/llms/vertex-ai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1193,6 +1193,32 @@ Grounding is invoked by passing the `google_search` tool (for newer models like
If you mix regular tools with grounding tools, vertex might throw an error saying only one tool can be used at a time.
</Warning>

## Grounding with Google Maps

Vertex AI also supports grounding with Google Maps for location-based queries. This allows your LLM to retrieve and use location data from Google Maps.

To use Google Maps grounding, pass the `google_maps` (or `googleMaps`) tool in the `tools` array with optional retrieval configuration:

```json
"tools": [
{
"type": "function",
"function": {
"name": "google_maps",
"parameters": {
"retrievalConfig": {
// Optional: Configure retrieval settings
}
}
}
}
]
```

<Note>
Google Maps grounding is useful for queries involving locations, directions, places, and geographic information.
</Note>

## gemini-2.0-flash-thinking-exp and other thinking/reasoning models

`gemini-2.0-flash-thinking-exp` models return a Chain of Thought response along with the actual inference text,
Expand Down
13 changes: 13 additions & 0 deletions product/ai-gateway/multimodal-capabilities/image-generation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,19 @@ curl "https://api.portkey.ai/v1/images/generations" \

[Create Image](/provider-endpoints/images/create-image)

### OpenAI gpt-image-1 Parameters

For OpenAI's `gpt-image-1` model, additional parameters are supported:

| Parameter | Type | Description |
|-----------|------|-------------|
| `moderation` | string | Content moderation level for generated images |
| `output_format` | string | Output format for the generated image |
| `output_compression` | number | Compression level (0-100) for the output image |
| `background` | string | Background style for the generated image |
| `partial_images` | number | Number of partial images to return (0-3) |
| `stream` | boolean | Whether to stream partial image results |

On completion, the request will get logged in the logs UI where the image can be viewed.

(_Note that providers may remove the hosted image after a period of time, so some logs might only contain the url_)
Expand Down
4 changes: 3 additions & 1 deletion product/guardrails.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,16 @@ Define a basic orchestration logic for your Guardrail here.
<img src="/images/product/ai-gateway/ai-33.png"/>
</Frame>

### There are 6 Types of Guardrail Actions
### There are 7 Types of Guardrail Actions

| Action | State | Description | Impact |
|:-------------- |:-------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Async** | **TRUE** This is the **default** state | Run the Guardrail checks **asynchronously** along with the LLM request. | Will add no latency to your requestUseful when you only want to log guardrail checks without affecting the request |
| **Async** | **FALSE** | **On Request** Run the Guardrail check **BEFORE** sending the request to the **LLM** **On Response** Run the Guardrail check **BEFORE** sending the response to the **user** | Will add latency to the requestUseful when your Guardrail critical and you want more orchestration over your request based on the Guardrail result |
| **Deny** | **TRUE** | **On Request & Response** If any of the Guardrail checks **FAIL**, the request will be killed with a **446** status code. If all of the Guardrail checks **SUCCEED**, the request/response will be sent further with a **200** status code. | This is useful when your Guardrails are critical and upon them failing, you can not run the requestWe would advice running this action on a subset of your requests to first see the impact |
| **Deny** | **FALSE** This is the **default** state | **On Request & Response** If any of the Guardrail checks **FAIL**, the request will STILL be sent, but with a **246** status code. If all of the Guardrail checks **SUCCEED**, the request/response will be sent further with a **200** status code. | This is useful when you want to log the Guardrail result but do not want it to affect your result |
| **Sequential** | **TRUE** | Run the Guardrail checks **sequentially** one after another. The next check only runs after the previous one completes. | Useful when checks have dependencies or when you want predictable ordering. Adds latency as checks run one at a time. |
| **Sequential** | **FALSE** This is the **default** state | Run the Guardrail checks **in parallel**. All checks execute simultaneously. | Faster execution when checks are independent of each other. |
| **On Success** | **Send Feedback** | If **all of the** Guardrail checks **PASS**, append your custom defined feedback to the request | We recommend setting up this actionThis will help you build an "Evals dataset" of Guardrail results on your requests over time |
| **On Failure** | **Send Feedback** | If **any of the** Guardrail checks **FAIL**, append your custom feedback to the request | We recommend setting up this actionThis will help you build an "Evals dataset" of Guardrail results on your requests over time |

Expand Down
2 changes: 2 additions & 0 deletions product/guardrails/creating-raw-guardrails-in-json.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ In this example:
}],
"deny": false,
"async": false,
"sequential": false,
"on_success": {
"feedback": {"value": 1,"weight": 1}
},
Expand All @@ -58,5 +59,6 @@ In this example,

* `deny`: Is set to `TRUE` or `FALSE`
* `async`: Is set to `TRUE` or `FALSE`
* `sequential`: Is set to `TRUE` or `FALSE` - when `TRUE`, guardrail checks run one after another instead of in parallel
* `on_success`: Used to pass custom `feedback`
* `on_failure`: Used to pass custom `feedback`