Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions examples/deepseek-ocr2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
6 changes: 6 additions & 0 deletions examples/deepseek-ocr2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@

First, make the project name in `pyproject.toml` unique.

```bash
sed -i '' "s/^name = \"deepseek-ocr2\"/name = \"deepseek-ocr2-$(date +%s)\"/" pyproject.toml
```

From within this directory, run `together beta jig deploy`.

## Inference Requests
Expand All @@ -15,6 +19,7 @@ Create your own request, or use the example request provided below after replaci
### Example Request

**Request**

```
curl --header "Authorization: Bearer $TOGETHER_API_KEY" \
-H "Content-Type: application/json" \
Expand Down Expand Up @@ -42,6 +47,7 @@ curl --header "Authorization: Bearer $TOGETHER_API_KEY" \
```

**Response**

```
{
"id": "chatcmpl-af755c4e61de3a2a",
Expand Down
1 change: 1 addition & 0 deletions examples/flux2-dev/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
Dockerfile
uv.lock
2 changes: 2 additions & 0 deletions examples/hello-world/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
2 changes: 2 additions & 0 deletions examples/kitchen-sink/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
2 changes: 2 additions & 0 deletions examples/midnight/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
2 changes: 2 additions & 0 deletions examples/qwen_vllm/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
12 changes: 9 additions & 3 deletions examples/qwen_vllm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,25 @@ instead of `jig submit`.

## How to Run

1. Deploy:
1. Generate a unique deployment name and update `pyproject.toml`:

```bash
sed -i '' "s/^name = \"speech-to-speech\"/name = \"speech-to-speech-$(date +%s)\"/" pyproject.toml
```

2. Deploy:

```bash
together beta jig deploy
```

2. Get the endpoint URL:
3. Get the endpoint URL:

```bash
together beta jig endpoint
```

3. Send a request:
4. Send a request:

```bash
curl $(together beta jig endpoint)/v1/chat/completions \
Expand Down
1 change: 1 addition & 0 deletions examples/speech-to-speech/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
Dockerfile
uv.lock
10 changes: 5 additions & 5 deletions examples/speech-to-speech/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,22 @@ A real-time speech-to-speech application that processes audio input and responds

## How to Run

1. Install client dependencies:
1. Generate a unique deployment name and update `pyproject.toml`:

```bash
uv sync --no-default-groups --group client
sed -i '' "s/^name = \"speech-to-speech\"/name = \"speech-to-speech-$(date +%s)\"/" pyproject.toml
```

2. Set the admin API key:
2. Install client dependencies:

```bash
export ADMIN_API_KEY=<your_admin_api_key>
uv sync --no-default-groups --group client
```

3. Deploy the application:

```bash
jig deploy
together beta jig deploy
```

4. Run the client with your deployment ID:
Expand Down
2 changes: 2 additions & 0 deletions examples/sprocket_wan2.1/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Dockerfile
uv.lock
21 changes: 4 additions & 17 deletions jig.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,11 @@
#!/usr/bin/env python3
"""jig - Simple deployment tool for Together AI"""

#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.11"
# dependencies = ["together @ git+https://github.com/togethercomputer/together-py@next"]
# dependencies = ["together @ git+https://github.com/togethercomputer/together-py@next" ]
# ///
import click
from together import Together

from together.lib.cli.api.beta.jig import add_commands


@click.group()
@click.pass_context
def jig(ctx: click.Context) -> None:
"""Jig - deployment tool for Together AI"""
ctx.obj = Together()

"""jig - Simple deployment tool for Together AI"""

add_commands(jig)
from together.lib.cli.api.beta.jig import jig

if __name__ == "__main__":
jig()
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ Issues = "https://github.com/togethercomputer/sprocket/issues"
[dependency-groups]
dev = [
"ruff>=0.12.5",
"together @ git+https://github.com/togethercomputer/together-py@next",
"pytest",
]

[tool.setuptools]
Expand Down
1 change: 1 addition & 0 deletions sprocket/sprocket.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ class Runner:
def __init__(self, sprocket: "Sprocket | AsyncSprocket", model_name: str) -> None:
self.queue_client = QueueClient(model_name)
self.download_client = httpx.AsyncClient(timeout=None, follow_redirects=True)
Path("inputs").mkdir(exist_ok=True)
self.sprocket = sprocket
# create InputOutputProcessor, potentially overriden by sprocket
self.io_processor = sprocket.processor()
Expand Down
480 changes: 371 additions & 109 deletions uv.lock

Large diffs are not rendered by default.