From 4f199364c35e7cc50b6a94921bd68db16ff77fa0 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Thu, 4 Jun 2026 12:20:00 -0400 Subject: [PATCH 1/3] style: apply /style-guide pass to models/track --- models/track/config.mdx | 58 +++---- models/track/create-an-experiment.mdx | 69 ++++---- models/track/environment-variables.mdx | 77 ++++----- models/track/jupyter.mdx | 39 +++-- models/track/limits.mdx | 110 +++++++------ models/track/log.mdx | 43 ++--- models/track/log/customize-logging-axes.mdx | 17 +- models/track/log/distributed-training.mdx | 117 +++++++------- models/track/log/log-models.mdx | 58 ++++--- models/track/log/log-summary.mdx | 25 ++- models/track/log/log-tables.mdx | 83 +++++----- models/track/log/media.mdx | 143 ++++++++-------- models/track/log/plots.mdx | 53 +++--- models/track/log/working-with-csv.mdx | 51 +++--- models/track/project-page.mdx | 171 +++++++++++--------- models/track/public-api-guide.mdx | 107 ++++++------ models/track/reproduce_experiments.mdx | 27 ++-- models/track/workspaces.mdx | 61 +++---- 18 files changed, 698 insertions(+), 611 deletions(-) diff --git a/models/track/config.mdx b/models/track/config.mdx index 9716b085f2..4e86402233 100644 --- a/models/track/config.mdx +++ b/models/track/config.mdx @@ -1,37 +1,37 @@ --- description: Use a dictionary-like object to save your experiment configuration title: Configure experiments +keywords: ["wandb.config", "argparse", "absl.FLAGS", "config-defaults.yaml", "hyperparameters"] --- import { ColabLink } from '/snippets/_includes/colab-link.mdx'; -Use the `config` property of a run to save your training configuration: -- hyperparameter -- input settings such as the dataset name or model type -- any other independent variables for your experiments. +This page shows you how to save the configuration of a W&B run so that you can analyze, compare, and reproduce your experiments later. Use the `config` property of a run to save your training configuration: -The `wandb.Run.config` property makes it easy to analyze your experiments and reproduce your work in the future. You can group by configuration values in the W&B App, compare the configurations of different W&B runs, and evaluate how each training configuration affects the output. The `config` property is a dictionary-like object that can be composed from multiple dictionary-like objects. +- Hyperparameters. +- Input settings such as the dataset name or model type. +- Any other independent variables for your experiments. + +The `wandb.Run.config` property helps you analyze your experiments and reproduce your work. You can group by configuration values in the W&B App, compare the configurations of different W&B runs, and evaluate how each training configuration affects the output. The `config` property is a dictionary-like object that you can compose from multiple dictionary-like objects. To save output metrics or dependent variables like loss and accuracy, use `wandb.Run.log()` instead of `wandb.Run.config`. - - ## Set up an experiment configuration -Configurations are typically defined in the beginning of a training script. Machine learning workflows may vary, however, so you are not required to define a configuration at the beginning of your training script. +You typically define configurations at the beginning of a training script. Machine learning workflows vary, so you don't have to. -Use dashes (`-`) or underscores (`_`) instead of periods (`.`) in your config variable names. - -Use the dictionary access syntax `["key"]["value"]` instead of the attribute access syntax `config.key.value` if your script accesses `wandb.Run.config` keys below the root. +You must use dashes (`-`) or underscores (`_`) instead of periods (`.`) in your config variable names. -The following sections outline different common scenarios of how to define your experiments configuration. +If your script accesses `wandb.Run.config` keys below the root, you must use the dictionary access syntax `["key"]["value"]` instead of the attribute access syntax `config.key.value`. + +The following sections outline common scenarios for how to define your experiment's configuration. ### Set the configuration at initialization -Pass a dictionary at the beginning of your script when you call the `wandb.init()` API to generate a background process to sync and log data as a W&B Run. +Pass a dictionary at the beginning of your script when you call the `wandb.init()` API to start a background process that syncs and logs your data as a run. -The following code snippet demonstrates how to define a Python dictionary with configuration values and how to pass that dictionary as an argument when you initialize a W&B Run. +The following code snippet demonstrates how to define a Python dictionary with configuration values and how to pass that dictionary as an argument when you initialize a run. ```python import wandb @@ -68,15 +68,15 @@ activation = run.config.get("activation") ``` -Throughout the Developer Guide and examples we copy the configuration values into separate variables. This step is optional. It is done for readability. +Throughout the Developer Guide and examples, we copy the configuration values into separate variables. This step is optional and improves readability. ### Set the configuration with argparse -You can set your configuration with an argparse object. [argparse](https://docs.python.org/3/library/argparse.html), short for argument parser, is a standard library module in Python 3.2 and above that makes it easy to write scripts that take advantage of all the flexibility and power of command line arguments. +You can set your configuration with an argparse object. [argparse](https://docs.python.org/3/library/argparse.html) is a standard library module in Python 3.2 and above. Use it to write scripts that take advantage of command-line arguments. -This is useful for tracking results from scripts that are launched from the command line. +This approach is useful for tracking results from scripts that you launch from the command line. -The following Python script demonstrates how to define a parser object to define and set your experiment config. The functions `train_one_epoch` and `evaluate_one_epoch` are provided to simulate a training loop for the purpose of this demonstration: +The following Python script demonstrates how to define a parser object to define and set your experiment config. The functions `train_one_epoch` and `evaluate_one_epoch` simulate a training loop for this demonstration: ```python # config_experiment.py @@ -141,6 +141,7 @@ if __name__ == "__main__": args = parser.parse_args() main(args) ``` + ### Set the configuration throughout your script You can add more parameters to your config object throughout your script. The following code snippet demonstrates how to add new key-value pairs to your config object: @@ -171,10 +172,10 @@ You can update multiple values at a time: run.config.update({"lr": 0.1, "channels": 16}) ``` -### Set the configuration after your Run has finished -Use the [W&B Public API](/models/ref/python/public-api/) to update a completed run's config. +### Set the configuration after your run has finished +If a run has already completed and you need to add or correct config values, use the [W&B Public API](/models/ref/python/public-api/) to update the stored configuration. -You must provide the API with your entity, project name and the run's ID. You can find these details in the Run object or in the [W&B App](/models/track/workspaces/): +You must provide the API with your entity, project name, and the run's ID. You can find these details in the run object or in the [W&B App](/models/track/workspaces/): ```python with wandb.init() as run: @@ -195,7 +196,7 @@ api_run.update() ## Highlight config values -Pin config keys to the **References** section at the top of a run's overview page. +Pin important config keys to the **References** section at the top of a run's overview page so that you can find them quickly when reviewing the run. Use [`wandb.Run.pin_config_keys`](/models/ref/python/experiments/run#method-run-pin_config_keys) to pin one or more config keys with the Python SDK. @@ -217,10 +218,9 @@ with wandb.init(config=config) as run: run.pin_config_keys(["grafana_url"]) ``` +## Use absl flags -## `absl.FLAGS` - -You can also pass in [`absl` flags](https://abseil.io/docs/python/guides/flags). +If your training script uses [`absl` flags](https://abseil.io/docs/python/guides/flags) for configuration, you can pass them directly into your run's config. ```python flags.DEFINE_string("model", None, "model to run") # name, default, help @@ -228,8 +228,8 @@ flags.DEFINE_string("model", None, "model to run") # name, default, help run.config.update(flags.FLAGS) # adds absl flags to config ``` -## File-Based Configs -If you place a file named `config-defaults.yaml` in the same directory as your run script, the run automatically picks up the key-value pairs defined in the file and passes them to `wandb.Run.config`. +## File-based configs +If you prefer to keep configuration outside your Python code, you can load it from a YAML file. If you place a file named `config-defaults.yaml` in the same directory as your run script, the run automatically picks up the key-value pairs defined in the file and passes them to `wandb.Run.config`. The following code snippet shows a sample `config-defaults.yaml` YAML file: @@ -249,7 +249,7 @@ with wandb.init(config={"epochs": 200, "batch_size": 64}) as run: ... ``` -To load a configuration file other than `config-defaults.yaml`, use the `--configs command-line` argument and specify the path to the file: +To load a configuration file other than `config-defaults.yaml`, use the `--configs` command-line argument and specify the path to the file: ```bash python train.py --configs other-config.yaml @@ -276,7 +276,7 @@ with wandb.init(config=config_dictionary) as run: ## TensorFlow v1 flags -You can pass TensorFlow flags into the `wandb.Run.config` object directly. +If you use TensorFlow v1, you can pass TensorFlow flags into the `wandb.Run.config` object directly. ```python with wandb.init() as run: diff --git a/models/track/create-an-experiment.mdx b/models/track/create-an-experiment.mdx index 73613fc3ce..66c8e286d6 100644 --- a/models/track/create-an-experiment.mdx +++ b/models/track/create-an-experiment.mdx @@ -1,25 +1,29 @@ --- description: "Create a W&B Experiment using the Python SDK to track run initialization, hyperparameters, and metric logging." title: Create an experiment +keywords: ["wandb.init", "run.log", "log_artifact", "hyperparameters", "training loop"] --- Use the W&B Python SDK to track machine learning experiments. You can then review the results in an interactive dashboard or export your data to Python for programmatic access with the [W&B Public API](/models/ref/python/public-api/). -This guide describes how to use W&B building blocks to create a W&B Experiment. +This guide describes how to use W&B building blocks to create a W&B Experiment so you can capture hyperparameters, log metrics, and save model artifacts for later analysis and comparison. It's intended for users who train machine learning models in Python and want to track their work with W&B. -## How to create a W&B Experiment +## Create a W&B Experiment Create a W&B Experiment in four steps: -1. [Initialize a W&B Run](#initialize-a-wb-run) -2. [Capture a dictionary of hyperparameters](#capture-a-dictionary-of-hyperparameters) -3. [Log metrics inside your training loop](#log-metrics-inside-your-training-loop) -4. [Log an artifact to W&B](#log-an-artifact-to-wb) +1. [Initialize a W&B Run](#initialize-a-wb-run). +2. [Capture a dictionary of hyperparameters](#capture-a-dictionary-of-hyperparameters). +3. [Log metrics inside your training loop](#log-metrics-inside-your-training-loop). +4. [Log an artifact to W&B](#log-an-artifact-to-wb). + +The following sections describe each step in detail. ### Initialize a W&B run -Use [`wandb.init()`](/models/ref/python/functions/init) to create a W&B Run. -The following snippet creates a run in a W&B project named `“cat-classification”` with the description `“My first experiment”` to help identify this run. Tags `“baseline”` and `“paper1”` are included to remind us that this run is a baseline experiment intended for a future paper publication. +A run is the basic unit of computation tracked by W&B, and every experiment begins by creating one. Use [`wandb.init()`](/models/ref/python/functions/init) to create a W&B Run. + +The following snippet creates a run in a W&B project named `"cat-classification"` with the description `"My first experiment"` to help identify this run. The tags `"baseline"` and `"paper1"` indicate that this run is a baseline experiment intended for a future paper publication. ```python import wandb @@ -35,10 +39,11 @@ with wandb.init( `wandb.init()` returns a [Run](/models/ref/python/experiments/run) object. -Note: Runs are added to pre-existing projects if that project already exists when you call `wandb.init()`. For example, if you already have a project called `“cat-classification”`, that project will continue to exist and not be deleted. Instead, a new run is added to that project. +W&B adds runs to pre-existing projects if that project already exists when you call `wandb.init()`. For example, if you already have a project called `"cat-classification"`, that project continues to exist and isn't deleted. Instead, W&B adds a new run to that project. ### Capture a dictionary of hyperparameters + Save a dictionary of hyperparameters such as learning rate or model type. The model settings you capture in config are useful later to organize and query your results. ```python @@ -49,10 +54,11 @@ with wandb.init( ... ``` -For more information on how to configure an experiment, see [Configure Experiments](./config). +For more information about how to configure an experiment, see [Configure Experiments](./config). ### Log metrics inside your training loop -Call [`run.log()`](/models/ref/python/experiments/run/#method-runlog) to log metrics about each training step such as accuracy and loss. + +Log metrics during training to monitor progress and compare runs in the W&B dashboard. Call [`run.log()`](/models/ref/python/experiments/run/#method-runlog) to log metrics about each training step such as accuracy and loss. ```python model, dataloader = get_model(), get_data() @@ -63,21 +69,25 @@ for epoch in range(run.config.epochs): run.log({"accuracy": accuracy, "loss": loss}) ``` -For more information on different data types you can log with W&B, see [Log Data During Experiments](/models/track/log/). +For more information about different data types you can log with W&B, see [Log Data During Experiments](/models/track/log/). + +### Log an artifact to W&B + +Optionally log a W&B Artifact. Artifacts let you version datasets and models. -### Log an artifact to W&B -Optionally log a W&B Artifact. Artifacts make it easy to version datasets and models. ```python -# You can save any file or even a directory. In this example, we pretend +# You can save any file or even a directory. This example assumes # the model has a save() method that outputs an ONNX file. model.save("path_to_model.onnx") run.log_artifact("path_to_model.onnx", name="trained-model", type="model") ``` + Learn more about [Artifacts](/models/artifacts/) or about versioning models in [Registry](/models/registry/). +### Complete example + +The following script combines the preceding code snippets into a complete experiment that initializes a run, captures configuration, logs metrics, and saves a model artifact: -### Putting it all together -The full script with the preceding code snippets is found below: ```python import wandb @@ -102,21 +112,22 @@ with wandb.init( run.log_artifact("path_to_model.onnx", name="trained-model", type="model") ``` -## Next steps: Visualize your experiment -Use the W&B Dashboard as a central place to organize and visualize results from your machine learning models. With just a few clicks, construct rich, interactive charts like [parallel coordinates plots](/models/app/features/panels/parallel-coordinates/), [parameter importance analyses](/models/app/features/panels/parameter-importance/), and [additional chart types](/models/app/features/panels/). +## Next steps: visualize your experiment + +After your run completes, the metrics, configuration, and artifacts you logged are available in the W&B App. Use the W&B Dashboard as a central place to organize and visualize results from your machine learning models. With a few clicks, construct interactive charts such as [parallel coordinates plots](/models/app/features/panels/parallel-coordinates/), [parameter importance analyses](/models/app/features/panels/parameter-importance/), and [additional chart types](/models/app/features/panels/). Quickstart Sweeps Dashboard example -For more information on how to view experiments and specific runs, see [Visualize results from experiments](/models/track/workspaces/). - +For more information about how to view experiments and specific runs, see [Visualize results from experiments](/models/track/workspaces/). ## Best practices + The following are some suggested guidelines to consider when you create experiments: -1. **Finish your runs**: Use `wandb.init()` in a `with` statement to automatically mark the run as finished when the code completes or raises an exception. - * In Jupyter notebooks, it may be more convenient to manage the Run object yourself. In this case, you can explicitly call `finish()` on the Run object to mark it complete: +- **Finish your runs**: Use `wandb.init()` in a `with` statement to automatically mark the run as finished when the code completes or raises an exception. + * In Jupyter notebooks, it might be more convenient to manage the Run object yourself. In this case, you can explicitly call `finish()` on the Run object to mark it complete: ```python # In a notebook cell: @@ -125,13 +136,13 @@ The following are some suggested guidelines to consider when you create experime # In a different cell: run.finish() ``` -2. **Config**: Track hyperparameters, architecture, dataset, and anything else you'd like to use to reproduce your model. These will show up in columns— use config columns to group, sort, and filter runs dynamically in the app. -3. **Project**: A project is a set of experiments you can compare together. Each project gets a dedicated dashboard page, and you can easily turn on and off different groups of runs to compare different model versions. -4. **Notes**: Set a quick commit message directly from your script. Edit and access notes in the Overview section of a run in the W&B App. -5. **Tags**: Identify baseline runs and favorite runs. You can filter runs using tags. You can edit tags at a later time on the Overview section of your project's dashboard on the W&B App. -6. **Create multiple run sets to compare experiments**: When comparing experiments, create multiple run sets to make metrics easy to compare. You can toggle run sets on or off on the same chart or group of charts. +- **Config**: Track hyperparameters, architecture, dataset, and anything else you'd like to use to reproduce your model. These show up in columns. Use config columns to group, sort, and filter runs dynamically in the app. +- **Project**: A project is a set of experiments you can compare together. Each project gets a dedicated dashboard page, and you can turn on and off different groups of runs to compare different model versions. +- **Notes**: Set a quick commit message directly from your script. Edit and access notes in the Overview section of a run in the W&B App. +- **Tags**: Identify baseline runs and favorite runs. You can filter runs using tags. You can edit tags later on the Overview section of your project's dashboard on the W&B App. +- **Create multiple run sets to compare experiments**: When you compare experiments, create multiple run sets to make metrics easy to compare. You can toggle run sets on or off on the same chart or group of charts. -The following code snippet demonstrates how to define a W&B Experiment using the best practices listed above: +The following code snippet demonstrates how to define a W&B Experiment using the preceding best practices: ```python import wandb diff --git a/models/track/environment-variables.mdx b/models/track/environment-variables.mdx index 4899075806..1079588fab 100644 --- a/models/track/environment-variables.mdx +++ b/models/track/environment-variables.mdx @@ -1,9 +1,12 @@ --- description: "Configure W&B SDK behavior using environment variables for authentication, project settings, logging modes, and more." title: Environment variables +keywords: ["WANDB_API_KEY", "WANDB_MODE", "Singularity", "AWS batch", "offline mode"] --- -When you're running a script in an automated environment, you can control W&B with environment variables set before the script runs or within the script. +When you're running a script in an automated environment, you can control W&B with environment variables set before the script runs or within the script. Use environment variables to configure authentication, project and run metadata, logging behavior, and storage locations without modifying your training code. This is useful for CI systems, batch jobs, and shared infrastructure. + +The following examples show common patterns for setting W&B environment variables in shell scripts and Python code. ```bash # This is secret and shouldn't be checked into version control @@ -29,48 +32,48 @@ os.environ["WANDB_SWEEP_ID"] = "b05fq58z" ## Optional environment variables -Use these optional environment variables to do things like set up authentication on remote machines. +Use these optional environment variables to do things such as set up authentication on remote machines. The following table lists each variable and how W&B uses it. | Variable name | Usage | | --------------------------- | ---------- | -| `WANDB_API_KEY` | Sets the authentication key associated with your account. Create an API key at [User Settings](https://wandb.ai/settings). This must be set if `wandb login` hasn't been run on the remote machine. | -| `WANDB_BASE_URL` | If you're using [wandb/local](/platform/hosting/) you should set this environment variable to `http://YOUR_IP:YOUR_PORT` | -| `WANDB_CACHE_DIR` | This defaults to \~/.cache/wandb, you can override this location with this environment variable | -| `WANDB_CONFIG_DIR` | This defaults to \~/.config/wandb, you can override this location with this environment variable | -| `WANDB_CONFIG_PATHS` | Comma separated list of yaml files to load into wandb.config. See [config](./config#file-based-configs). | -| `WANDB_CONSOLE` | Set this to "off" to disable stdout / stderr logging. This defaults to "on" in environments that support it. | +| `WANDB_API_KEY` | Sets the authentication key associated with your account. Create an API key at [User Settings](https://wandb.ai/settings). You must set this if you haven't run `wandb login` on the remote machine. | +| `WANDB_BASE_URL` | If you're using [wandb/local](/platform/hosting/), set this environment variable to `http://[YOUR_IP]:[YOUR_PORT]`. | +| `WANDB_CACHE_DIR` | Defaults to `~/.cache/wandb`. Use this environment variable to override the location. | +| `WANDB_CONFIG_DIR` | Defaults to `~/.config/wandb`. Use this environment variable to override the location. | +| `WANDB_CONFIG_PATHS` | Comma-separated list of YAML files to load into `wandb.config`. See [config](./config#file-based-configs). | +| `WANDB_CONSOLE` | Set this to `off` to disable stdout/stderr logging. Defaults to `on` in environments that support it. | | `WANDB_DATA_DIR` | Where to upload staging artifacts. The default location depends on your platform, because it uses the value of `user_data_dir` from the `platformdirs` Python package. Make sure this directory exists and the running user has permission to write to it. | -| `WANDB_DIR` | Where to store all generated files. If unset, defaults to the `wandb` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This does not control the location of downloaded artifacts, which you can set using the `WANDB_ARTIFACT_DIR` environment variable. | -| `WANDB_ARTIFACT_DIR` | Where to store all downloaded artifacts. If unset, defaults to the `artifacts` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This does not control the location of generated metadata files, which you can set using the `WANDB_DIR` environment variable. | -| `WANDB_DISABLE_GIT` | Prevent wandb from probing for a git repository and capturing the latest commit / diff. | -| `WANDB_DISABLE_CODE` | Set this to true to prevent wandb from saving notebooks or git diffs. We'll still save the current commit if we're in a git repo. | -| `WANDB_DOCKER` | Set this to a docker image digest to enable restoring of runs. This is set automatically with the wandb docker command. You can obtain an image digest by running `wandb docker my/image/name:tag --digest` | -| `WANDB_ENTITY` | The entity associated with your run. If you have run `wandb init` in the directory of your training script, it will create a directory named _wandb_ and will save a default entity which can be checked into source control. If you don't want to create that file or want to override the file you can use the environmental variable. | -| `WANDB_ERROR_REPORTING` | Set this to false to prevent wandb from logging fatal errors to its error tracking system. | -| `WANDB_HOST` | Set this to the hostname you want to see in the wandb interface if you don't want to use the system provided hostname | -| `WANDB_IGNORE_GLOBS` | Set this to a comma separated list of file globs to ignore. These files will not be synced to the cloud. | -| `WANDB_JOB_NAME` | Specify a name for any jobs created by `wandb`. | -| `WANDB_JOB_TYPE` | Specify the job type, like "training" or "evaluation" to indicate different types of runs. See [grouping](/models/runs/grouping/) for more info. | -| `WANDB_MODE` | If you set this to "offline" wandb will save your run metadata locally and not sync to the server. If you set this to `disabled` wandb will turn off completely. | -| `WANDB_NAME` | The human-readable name of your run. If not set it will be randomly generated for you | -| `WANDB_NOTEBOOK_NAME` | If you're running in jupyter you can set the name of the notebook with this variable. We attempt to auto detect this. | -| `WANDB_NOTES` | Longer notes about your run. Markdown is allowed and you can edit this later in the UI. | -| `WANDB_PROJECT` | The project associated with your run. This can also be set with `wandb init`, but the environmental variable will override the value. | -| `WANDB_RESUME` | By default this is set to _never_. If set to _auto_ wandb will automatically resume failed runs. If set to _must_ forces the run to exist on startup. If you want to always generate your own unique ids, set this to _allow_ and always set `WANDB_RUN_ID`. | -| `WANDB_RUN_GROUP` | Specify the experiment name to automatically group runs together. See [grouping](/models/runs/grouping/) for more info. | -| `WANDB_RUN_ID` | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. All non-word characters will be converted to dashes. This can be used to resume an existing run in cases of failure. | -| `WANDB_QUIET` | Set this to `true` to limit statements logged to standard output to critical statements only. If this is set all logs will be written to `$WANDB_DIR/debug.log`. | -| `WANDB_SILENT` | Set this to `true` to silence wandb log statements. This is useful for scripted commands. If this is set all logs will be written to `$WANDB_DIR/debug.log`. | -| `WANDB_SHOW_RUN` | Set this to `true` to automatically open a browser with the run url if your operating system supports it. | -| `WANDB_SWEEP_ID` | Add sweep ID tracking to `Run` objects and related classes, and display in the UI. | -| `WANDB_TAGS` | A comma separated list of tags to be applied to the run. | -| `WANDB_USERNAME` | The username of a member of your team associated with the run. This can be used along with a service account API key to enable attribution of automated runs to members of your team. | -| `WANDB_USER_EMAIL` | The email of a member of your team associated with the run. This can be used along with a service account API key to enable attribution of automated runs to members of your team. | +| `WANDB_DIR` | Where to store all generated files. If unset, defaults to the `wandb` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This doesn't control the location of downloaded artifacts, which you can set using the `WANDB_ARTIFACT_DIR` environment variable. | +| `WANDB_ARTIFACT_DIR` | Where to store all downloaded artifacts. If unset, defaults to the `artifacts` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This doesn't control the location of generated metadata files, which you can set using the `WANDB_DIR` environment variable. | +| `WANDB_DISABLE_GIT` | Prevents `wandb` from probing for a Git repository and capturing the latest commit and diff. | +| `WANDB_DISABLE_CODE` | Set this to `true` to prevent `wandb` from saving notebooks or Git diffs. `wandb` still saves the current commit if you're in a Git repo. | +| `WANDB_DOCKER` | Set this to a Docker image digest to enable restoring of runs. This is set automatically with the `wandb docker` command. You can obtain an image digest by running `wandb docker [IMAGE_NAME]:[TAG] --digest`. | +| `WANDB_ENTITY` | The entity associated with your run. If you have run `wandb init` in the directory of your training script, `wandb` creates a directory named `wandb` and saves a default entity, which you can check into source control. If you don't want to create that file or want to override the file, use the environment variable. | +| `WANDB_ERROR_REPORTING` | Set this to `false` to prevent `wandb` from logging fatal errors to its error tracking system. | +| `WANDB_HOST` | Set this to the hostname you want to see in the `wandb` interface if you don't want to use the system-provided hostname. | +| `WANDB_IGNORE_GLOBS` | Set this to a comma-separated list of file globs to ignore. `wandb` doesn't sync these files to the cloud. | +| `WANDB_JOB_NAME` | Specifies a name for any jobs created by `wandb`. | +| `WANDB_JOB_TYPE` | Specifies the job type, such as `training` or `evaluation`, to indicate different types of runs. See [grouping](/models/runs/grouping/) for more info. | +| `WANDB_MODE` | If you set this to `offline`, `wandb` saves your run metadata locally and doesn't sync to the server. If you set this to `disabled`, `wandb` turns off completely. | +| `WANDB_NAME` | The human-readable name of your run. If not set, `wandb` randomly generates it for you. | +| `WANDB_NOTEBOOK_NAME` | If you're running in Jupyter, you can set the name of the notebook with this variable. `wandb` attempts to autodetect this. | +| `WANDB_NOTES` | Longer notes about your run. Markdown is allowed, and you can edit this later in the UI. | +| `WANDB_PROJECT` | The project associated with your run. You can also set this with `wandb init`, but the environment variable overrides the value. | +| `WANDB_RESUME` | By default, this is set to `never`. If set to `auto`, `wandb` automatically resumes failed runs. If set to `must`, forces the run to exist on startup. If you want to always generate your own unique IDs, set this to `allow` and always set `WANDB_RUN_ID`. | +| `WANDB_RUN_GROUP` | Specifies the experiment name to automatically group runs together. See [grouping](/models/runs/grouping/) for more info. | +| `WANDB_RUN_ID` | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. `wandb` converts all non-word characters to dashes. You can use this to resume an existing run in cases of failure. | +| `WANDB_QUIET` | Set this to `true` to limit statements logged to standard output to critical statements only. If this is set, `wandb` writes all logs to `$WANDB_DIR/debug.log`. | +| `WANDB_SILENT` | Set this to `true` to silence `wandb` log statements. This is useful for scripted commands. If this is set, `wandb` writes all logs to `$WANDB_DIR/debug.log`. | +| `WANDB_SHOW_RUN` | Set this to `true` to automatically open a browser with the run URL if your operating system supports it. | +| `WANDB_SWEEP_ID` | Adds sweep ID tracking to `Run` objects and related classes, and displays it in the UI. | +| `WANDB_TAGS` | A comma-separated list of tags to apply to the run. | +| `WANDB_USERNAME` | The username of a member of your team associated with the run. Use this along with a service account API key to enable attribution of automated runs to members of your team. | +| `WANDB_USER_EMAIL` | The email of a member of your team associated with the run. Use this along with a service account API key to enable attribution of automated runs to members of your team. | ## Singularity environments -If you're running containers in [Singularity](https://singularity.lbl.gov/index.html) you can pass environment variables by pre-pending the above variables with `SINGULARITYENV_`. More details about Singularity environment variables can be found [here](https://singularity.lbl.gov/docs-environment-metadata#environment). +If you're running containers in [Singularity](https://singularity.lbl.gov/index.html), you can pass environment variables by prepending any of the preceding variables with `SINGULARITYENV_`. For more details about Singularity environment variables, see the [Singularity documentation](https://singularity.lbl.gov/docs-environment-metadata#environment). -## Running on AWS +## Run on AWS -If you're running batch jobs in AWS, it's easy to authenticate your machines with your W&B credentials. Create an API key at [User Settings](https://wandb.ai/settings), and set the `WANDB_API_KEY` environment variable in the [AWS batch job spec](https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html#parameters). +If you're running batch jobs in AWS, you can authenticate your machines with your W&B credentials. Create an API key at [User Settings](https://wandb.ai/settings), and set the `WANDB_API_KEY` environment variable in the [AWS batch job spec](https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html#parameters). diff --git a/models/track/jupyter.mdx b/models/track/jupyter.mdx index 92046f9b81..5adc029b25 100644 --- a/models/track/jupyter.mdx +++ b/models/track/jupyter.mdx @@ -2,17 +2,20 @@ description: Use W&B with Jupyter to get interactive visualizations without leaving your notebook. title: Track Jupyter notebooks +keywords: ["Jupyter", "Colab", "notebook", "wandb magic", "interactive visualization"] --- Use W&B with Jupyter to get interactive visualizations without leaving your notebook. Combine custom analysis, experiments, and prototypes, all fully logged. +This page shows you how to install W&B in a notebook, initialize a run, render live W&B interfaces inline, and take advantage of notebook-specific features such as Colab authentication and cell-order tolerance. + ## Use cases for W&B with Jupyter notebooks -1. **Iterative experimentation**: Run and re-run experiments, tweaking parameters, and have all the runs you do saved automatically to W&B without having to take manual notes along the way. -2. **Code saving**: When reproducing a model, it's hard to know which cells in a notebook ran, and in which order. Turn on code saving on your [settings page](/platform/app/settings-page/) to save a record of cell execution for each experiment. -3. **Custom analysis**: Once runs are logged to W&B, it's easy to get a dataframe from the API and do custom analysis, then log those results to W&B to save and share in reports. +- **Iterative experimentation**: Run and re-run experiments, tweak parameters, and have all your runs saved automatically to W&B without taking manual notes. +- **Code saving**: When reproducing a model, it's hard to know which cells in a notebook ran, and in which order. Turn on code saving on your [settings page](/platform/app/settings-page/) to save a record of cell execution for each experiment. +- **Custom analysis**: Once runs are logged to W&B, you can get a dataframe from the API and do custom analysis, then log those results to W&B to save and share in reports. -## Getting started in a notebook +## Get started in a notebook Start your notebook with the following code to install W&B and link your account: @@ -35,7 +38,7 @@ wandb.init( ) ``` -After running `wandb.init()` , start a new cell with `%%wandb` to see live graphs in the notebook. If you run this cell multiple times, data will be appended to the run. +To see live graphs in the notebook as your experiment runs, after `wandb.init()` start a new cell with `%%wandb`. If you run this cell multiple times, W&B appends data to the run. ```notebook %%wandb @@ -49,24 +52,24 @@ Try it for yourself in this [example notebook](https://wandb.me/jupyter-interact Jupyter W&B widget -### Rendering live W&B interfaces directly in your notebooks +### Render live W&B interfaces directly in your notebooks You can also display any existing dashboards, sweeps, or reports directly in your notebook using the `%wandb` magic: ```notebook # Display a project workspace -%wandb USERNAME/PROJECT +%wandb [USERNAME]/[PROJECT] # Display a single run -%wandb USERNAME/PROJECT/runs/RUN_ID +%wandb [USERNAME]/[PROJECT]/runs/[RUN-ID] # Display a sweep -%wandb USERNAME/PROJECT/sweeps/SWEEP_ID +%wandb [USERNAME]/[PROJECT]/sweeps/[SWEEP-ID] # Display a report -%wandb USERNAME/PROJECT/reports/REPORT_ID +%wandb [USERNAME]/[PROJECT]/reports/[REPORT-ID] # Specify the height of embedded iframe -%wandb USERNAME/PROJECT -h 2048 +%wandb [USERNAME]/[PROJECT] -h 2048 ``` -As an alternative to the `%%wandb` or `%wandb` magics, after running `wandb.init()` you can end any cell with `wandb.Run.finish()` to show in-line graphs, or call `ipython.display(...)` on any report, sweep, or run object returned from our apis. +As an alternative to the `%%wandb` or `%wandb` magics, after running `wandb.init()` you can end any cell with `wandb.Run.finish()` to show inline graphs, or call `ipython.display(...)` on any report, sweep, or run object returned from the W&B APIs. ```python import wandb @@ -79,15 +82,17 @@ run.finish() ``` -Want to know more about what you can do with W&B? Check out our [guide to logging data and media](/models/track/log/), learn [how to integrate us with your favorite ML toolkits](/models/integrations), or just dive straight into the [reference docs](/models/ref/python/) or our [repo of examples](https://github.com/wandb/examples). +To learn more about what you can do with W&B, see the [guide to logging data and media](/models/track/log/), learn [how to integrate W&B with ML toolkits](/models/integrations), or go to the [reference docs](/models/ref/python/) or the [examples repo](https://github.com/wandb/examples). ## Additional Jupyter features in W&B -1. **Easy authentication in Colab**: When you call `wandb.init()` for the first time in a Colab, we automatically authenticate your runtime if you're currently logged in to W&B in your browser. On the overview tab of your run page, you'll see a link to the Colab. -2. **Jupyter Magic:** Display dashboards, sweeps and reports directly in your notebooks. The `%wandb` magic accepts a path to your project, sweeps or reports and will render the W&B interface directly in the notebook. -3. **Launch dockerized Jupyter**: Call `wandb docker --jupyter` to launch a docker container, mount your code in it, ensure Jupyter is installed, and launch on port 8888. -4. **Run cells in arbitrary order without fear**: By default, we wait until the next time `wandb.init()` is called to mark a run as `finished`. That allows you to run multiple cells (say, one to set up data, one to train, one to test) in whatever order you like and have them all log to the same run. If you turn on code saving in [User Settings](https://wandb.ai/settings), you'll also log the cells that were executed, in order and in the state in which they were run, enabling you to reproduce even the most non-linear of pipelines. To mark a run as complete manually in a Jupyter notebook, call `wandb.Run.finish()`. +W&B includes several conveniences designed specifically for notebook workflows. + +- **Streamlined authentication in Colab**: When you call `wandb.init()` for the first time in a Colab, W&B automatically authenticates your runtime if you're logged in to W&B in your browser. On the **Overview** tab of your run page, you'll see a link to the Colab. +- **Jupyter magic**: Display dashboards, sweeps, and reports directly in your notebooks. The `%wandb` magic accepts a path to your project, sweeps, or reports and renders the W&B interface directly in the notebook. +- **Launch dockerized Jupyter**: Call `wandb docker --jupyter` to launch a Docker container, mount your code in it, ensure Jupyter is installed, and launch on port `8888`. +- **Run cells in arbitrary order**: By default, W&B waits until the next time `wandb.init()` is called to mark a run as `finished`. That lets you run multiple cells (say, one to set up data, one to train, one to test) in any order and have them all log to the same run. If you turn on code saving in [**User Settings**](https://wandb.ai/settings), you also log the cells that were executed, in order and in the state in which they were run, which lets you reproduce non-linear pipelines. To mark a run as complete manually in a Jupyter notebook, call `wandb.Run.finish()`. ```python import wandb diff --git a/models/track/limits.mdx b/models/track/limits.mdx index 2df69b952f..9de55e254a 100644 --- a/models/track/limits.mdx +++ b/models/track/limits.mdx @@ -1,18 +1,21 @@ --- description: This page describes how logging patterns impact performance in W&B and provides guidance for scaling experiment tracking in large projects. title: Logging at scale and performance +keywords: ["rate limits", "metric cardinality", "throughput", "workspace performance", "wandb.Run.log"] --- -Performance is usually influenced by a combination of: +This page helps you understand how your logging patterns affect W&B performance and how to scale experiment tracking in large projects. Use it to plan logging strategies, diagnose slow workspaces, and stay within recommended operating ranges. -* the number of runs in a project -* the number of steps in each run -* the number of distinct metrics you log -* how often you call `wandb.Run.log()` -* how much data you send in each log call -* how your workspace is configured +Several factors usually influence performance: -In most cases, performance issues are caused more often by logging too many distinct metrics than by logging too many steps. +* The number of runs in a project. +* The number of steps in each run. +* The number of distinct metrics you log. +* How often you call `wandb.Run.log()`. +* How much data you send in each log call. +* How your workspace is configured. + +Usually, logging too many distinct metrics causes more performance issues than logging too many steps. ## Key terms @@ -20,7 +23,7 @@ The following terms are used throughout this page. ### Steps -A **step** is a single logical row of metrics in a run. A step is finalized when `wandb.Run.log()` is called with `commit=True`, or implicitly when neither `commit` nor `step` is specified. +A **step** is a single logical row of metrics in a run. A step finalizes when you call `wandb.Run.log()` with `commit=True`, or implicitly when you specify neither `commit` nor `step`. ```python import wandb @@ -103,6 +106,8 @@ throughput = logged points × log frequency ## Recommendations at scale +This section lists the operating ranges that W&B recommends for large-scale logging. Use these values as targets when planning a project or diagnosing performance issues. + The recommendations described in this section only apply to W&B Multi-tenant Cloud. If you use a different W&B deployment type, check with your administrator for deployment-specific guidance or limits. @@ -119,12 +124,12 @@ The following table summarizes recommended operating ranges for large-scale logg | Video throughput | 40 MB per minute | -These values are guidelines for maintaining good performance at scale. W&B may continue to accept data beyond these recommendations, but pages can become slower to load and use. +These values are guidelines for maintaining good performance at scale. W&B might continue to accept data beyond these recommendations, but pages can become slower to load and use. ## Throughput examples -Different logging patterns can produce the same throughput. +Different logging patterns can produce the same throughput. The following sections describe how to reach the same throughput targets through different combinations of metrics per call and log frequency. ### Scalar logging examples @@ -159,7 +164,7 @@ The values listed in the table only apply to W&B Multi-tenant Cloud. If you use ## Logging considerations -Use `wandb.Run.log()` to track experiment metrics. +Use `wandb.Run.log()` to track experiment metrics. The following considerations help you log efficiently and avoid common performance pitfalls. ### Metric cardinality @@ -190,13 +195,13 @@ with wandb.init() as run: {/* Metric names must follow GraphQL naming constraints. See [Metric naming constraints](/models/track/log/#metric-naming-constraints). */} -If your workspace suddenly slows down, check whether recent runs introduced a large number of new metric keys. This often appears as many plots with only one or two runs visible. If this happened unintentionally, consider deleting and recreating those runs with a smaller, more stable set of metric names. +If your workspace suddenly slows down, check whether recent runs introduced many new metric keys. This often appears as many plots with only one or two runs visible. If this happened unintentionally, consider deleting and recreating those runs with a smaller, more stable set of metric names. ### Value size Keep the size of a single logged value under 1 MB and the total size of a single `wandb.Run.log()` call under 25 MB. -These recommendations do not apply to `wandb.Media` types such as `wandb.Image` and `wandb.Audio`, which are handled differently. +These recommendations don't apply to `wandb.Media` types such as `wandb.Image` and `wandb.Audio`, which W&B handles differently. ```python import json @@ -215,18 +220,18 @@ with wandb.init(project="wide-values") as run: Large values can slow plot loading for the entire run, not just for the metric that contains the large value. -W&B still stores logged data that exceeds these recommendations, but pages may load more slowly. +W&B still stores logged data that exceeds these recommendations, but pages might load more slowly. ### Log frequency and throughput -Choose a logging frequency that matches the value of the data you are collecting. Logging too often can increase SDK overhead and make the app slower, especially when combined with high metric cardinality or large payloads. +Choose a logging frequency that matches the value of the data you're collecting. Logging too often can increase SDK overhead and make the app slower, especially when combined with high metric cardinality or large payloads. As a starting point, keep logging within these guidelines: -* Log frequency: less than 1,000 `wandb.Run.log()` calls per minute -* Throughput: less than 100,000 logged values per minute -* Video throughput: less than 40 MB per minute +* Log frequency: less than 1,000 `wandb.Run.log()` calls per minute. +* Throughput: less than 100,000 logged values per minute. +* Video throughput: less than 40 MB per minute. Batch related metrics into the same step when possible. For example, the following code snippet logs three metrics in the same step, which is more efficient than logging them separately. @@ -284,7 +289,7 @@ with open("large_config.json", "r") as f: ## Workspace performance -Workspace performance depends on both the underlying project data and workspace configuration. +Workspace performance depends on both the underlying project data and workspace configuration. This section describes settings and project shapes that affect how quickly a workspace loads and responds. ### Runs per project @@ -296,7 +301,7 @@ If your team regularly works with only a subset of runs, consider moving older o By default, a workspace in automatic mode creates standard panels for each logged key. In large projects, this can produce too many panels and slow the workspace. -To improve performance: +To improve performance, switch to manual mode so the workspace only renders the panels you select: 1. Reset the workspace to manual mode. 2. Use [Quick add](/models/app/features/panels/#quick-add) to add only the panels you need. @@ -321,7 +326,7 @@ Create sections based on high-level metric groupings rather than one section per When logging thousands of metrics per run, use a manual workspace so that you can choose which metrics to visualize. -A focused set of panels loads faster. Metrics that are not plotted are still collected and stored. +A focused set of panels loads faster. W&B still collects and stores metrics that aren't plotted. To reset a workspace to manual mode, click the workspace's **action ()** menu, then click **Reset workspace**. Resetting a workspace has no impact on stored metrics for runs. See [workspace panel management](/models/app/features/panels/). @@ -329,7 +334,7 @@ To reset a workspace to manual mode, click the workspace's **action ( -W&B does not enforce hard product limits for these recommendations beyond API rate limiting. If you exceed the guidance on this page, W&B may continue to accept your data, but the app or SDK may become slower. +W&B doesn't enforce hard product limits for these recommendations beyond API rate limiting. If you exceed the guidance on this page, W&B might continue to accept your data, but the app or SDK might become slower. ## Rate limits -W&B Multi-tenant Cloud APIs use rate limits to maintain service reliability and availability. +W&B Multi-tenant Cloud APIs use rate limits to maintain service reliability and availability. This section explains the headers that signal rate-limit status and the limits that apply to metric logging and the GraphQL API. Rate limits are subject to change. @@ -365,11 +372,11 @@ If you hit a rate limit, the server returns HTTP `429 Rate limit exceeded` and i ### Rate-limit HTTP headers -| Header name | Description | -| --------------------- | ----------------------------------------------------------------- | -| `RateLimit-Limit` | Quota available in the current time window, scaled from 0 to 1000 | -| `RateLimit-Remaining` | Remaining quota in the current window, scaled from 0 to 1000 | -| `RateLimit-Reset` | Number of seconds until the current quota resets | +| Header name | Description | +| --------------------- | ------------------------------------------------------------------ | +| `RateLimit-Limit` | Quota available in the current time window, scaled from 0 to 1,000 | +| `RateLimit-Remaining` | Remaining quota in the current window, scaled from 0 to 1,000 | +| `RateLimit-Reset` | Number of seconds until the current quota resets | ### Metric logging API rate limits @@ -377,7 +384,7 @@ If you hit a rate limit, the server returns HTTP `429 Rate limit exceeded` and i Rate limits for metric logging apply at the project level and include both request rate and total request size over a rolling time window. Paid plans have higher limits than free plans. -If you exceed a rate limit, the W&B SDK automatically retries requests with backoff. In some cases, this can delay `run.finish()` until the rate-limit window resets. +If you exceed a rate limit, the W&B SDK automatically retries requests with backoff. Sometimes, this can delay `run.finish()` until the rate-limit window resets. To reduce the chance of rate limiting: @@ -399,7 +406,7 @@ with wandb.init(project="basic-intro") as run: run.log({"acc": accuracy, "loss": loss}) ``` -For manual syncing, use `wandb sync `. See [`wandb sync`](/models/ref/cli/wandb-sync). +For manual syncing, use `wandb sync [RUN-FILE-PATH]`, where `[RUN-FILE-PATH]` is the path to the offline run directory. See [`wandb sync`](/models/ref/cli/wandb-sync). ### GraphQL API rate limits @@ -407,34 +414,33 @@ The W&B app and the [public API](/models/ref/python/public-api/api) use GraphQL For Multi-tenant Cloud: -* unauthorized requests are rate-limited per IP address -* authorized requests are rate-limited per user -* some SDK requests that specify a project path can also be limited per project based on database query time +* Unauthorized requests are rate-limited per IP address. +* Authorized requests are rate-limited per user. +* Some SDK requests that specify a project path can also be limited per project based on database query time. Teams and Enterprise plans have higher limits than Free plans. -If you are making a large number of public API requests, wait at least one second between requests when possible. If you receive HTTP `429 Rate limit exceeded` or see `RateLimit-Remaining=0`, wait for the number of seconds in `RateLimit-Reset` before retrying. - -## Troubleshooting slow projects +If you're making many public API requests, wait at least 1 second between requests when possible. If you receive HTTP `429 Rate limit exceeded` or see `RateLimit-Remaining=0`, wait for the number of seconds in `RateLimit-Reset` before retrying. -If a project or workspace feels slow, check the following first: +## Troubleshoot slow projects -1. Did recent runs introduce a large number of new metric names? -2. Are you logging too frequently? -3. Are individual `run.log()` calls very large? -4. Is the workspace in automatic mode with too many panels or sections? -5. Does the project contain more runs than your team actively uses? +If a project or workspace feels slow, the following checklist helps you isolate the most common causes before contacting support: -In many cases, performance improves after reducing metric cardinality, batching log calls, and switching large workspaces to manual mode. +* Did recent runs introduce many new metric names? +* Are you logging too frequently? +* Are individual `run.log()` calls unusually large? +* Is the workspace in automatic mode with too many panels or sections? +* Does the project contain more runs than your team actively uses? +Often, performance improves after reducing metric cardinality, batching log calls, and switching large workspaces to manual mode. ## Browser considerations -The W&B app can be memory-intensive and performs best in Chrome. Depending on your computer's memory, having W&B active in 3+ tabs at once can cause performance to degrade. If you encounter unexpectedly slow performance, consider closing other tabs or applications. +The W&B app can be memory-intensive and performs best in Chrome. Depending on your computer's memory, having W&B active in three or more tabs at once can cause performance to degrade. If you encounter unexpectedly slow performance, consider closing other tabs or applications. -## Reporting performance issues to W&B +## Report performance issues to W&B -W&B takes performance seriously and investigates every report of lag. To expedite investigation, when reporting slow loading times consider invoking W&B's built-in performance logger that captures key metrics and performance events. Append the URL parameter `&PERF_LOGGING` to a page that is loading slowly, then share the output of your console with your account team or Support. +W&B takes performance seriously and investigates every report of lag. To speed up investigation, when reporting slow loading times consider invoking W&B's built-in performance logger that captures key metrics and performance events. Append the URL parameter `&PERF_LOGGING` to a page that's loading slowly, then share the output of your console with your account team or Support. Adding PERF_LOGGING diff --git a/models/track/log.mdx b/models/track/log.mdx index ecb302c8fd..049af7d71c 100644 --- a/models/track/log.mdx +++ b/models/track/log.mdx @@ -1,15 +1,18 @@ --- description: Keep track of metrics, videos, custom plots, and more title: Overview +keywords: ["wandb.Run.log", "experiment tracking", "metrics", "step", "x-axis"] --- import MetricNamingRules from "/snippets/_includes/metric-naming-rules.mdx"; import MetricNamingExamples from "/snippets/_includes/metric-naming-examples.mdx"; -Log a dictionary of metrics, media, or custom objects to a step with the W&B Python SDK. W&B collects the key-value pairs during each step and stores them in one unified dictionary each time you log data with `wandb.Run.log()`. Data logged from your script is saved locally to your machine in a directory called `wandb`, then synced to the W&B cloud or your [private server](/platform/hosting/). +Use the W&B Python SDK to log metrics, media, and custom objects from your training scripts so you can visualize, compare, and analyze them in the W&B app. This page introduces how logging works, what W&B captures automatically, and the common patterns you'll use to track experiments. + +Log a dictionary of metrics, media, or custom objects to a step with the W&B Python SDK. W&B collects the key-value pairs during each step and stores them in a single dictionary each time you log data with `wandb.Run.log()`. W&B saves the data your script logs locally to your machine in a directory called `wandb`, then syncs it to the W&B cloud or your [private server](/platform/hosting/). -Key-value pairs are stored in one unified dictionary only if you pass the same value for each step. W&B writes all of the collected keys and values to memory if you log a different value for `step`. +W&B stores key-value pairs in a single dictionary only if you pass the same value for each step. W&B writes all of the collected keys and values to memory if you log a different value for `step`. Each call to `wandb.Run.log()` is a new `step` by default. W&B uses steps as the default x-axis when it creates charts and panels. You can optionally create and use a custom x-axis or capture a custom summary metric. For more information, see [Customize log axes](/models/track/log/customize-logging-axes/). @@ -24,7 +27,7 @@ run.log({'loss': 0.2}, step=step) {/* [INSERT EXAMPLE] */} -Use `wandb.Run.log()` to log consecutive values for each `step`: 0, 1, 2, and so on. It is not possible to write to a specific history step. W&B only writes to the "current" and "next" step. +Use `wandb.Run.log()` to log consecutive values for each `step`: 0, 1, 2, and so on. You can't write to a specific history step. W&B only writes to the "current" and "next" step. {/* You can set `commit=False` in `wandb.Run.log()` to accumulate metrics, just be sure to eventually call `run.log()` with `commit=True` (the default) to persist the metrics. @@ -41,22 +44,22 @@ run.log({'accuracy': 0.8}) W&B automatically logs the following information during a W&B Experiment: -* **System metrics**: CPU and GPU utilization, network, etc. For the GPU, these are fetched with [`nvidia-smi`](https://developer.nvidia.com/nvidia-system-management-interface). -* **Command line**: The stdout and stderr are picked up and show in the logs tab on the [run page.](/models/runs/) +* **System metrics**: CPU and GPU utilization, network, and similar metrics. For the GPU, these are fetched with [`nvidia-smi`](https://developer.nvidia.com/nvidia-system-management-interface). +* **Command line**: W&B picks up `stdout` and `stderr` and shows them in the logs tab on the [run page](/models/runs/). Turn on [Code Saving](https://wandb.me/code-save-colab) in your account's [Settings page](https://wandb.ai/settings) to log: -* **Git commit**: Pick up the latest git commit and see it on the overview tab of the run page, as well as a `diff.patch` file if there are any uncommitted changes. -* **Dependencies**: The `requirements.txt` file will be uploaded and shown on the files tab of the run page, along with any files you save to the `wandb` directory for the run. +* **Git commit**: W&B picks up the latest `git` commit and shows it on the overview tab of the run page, along with a `diff.patch` file if you have any uncommitted changes. +* **Dependencies**: W&B uploads the `requirements.txt` file and shows it on the files tab of the run page, along with any files you save to the `wandb` directory for the run. -## What data is logged with specific W&B API calls? +## Data logged with specific W&B API calls -With W&B, you can decide exactly what you want to log. The following lists some commonly logged objects: +Beyond the data W&B captures automatically, you decide exactly what to log from your code. The following list describes some commonly logged objects and the API calls used to capture them: -* **Datasets**: You have to specifically log images or other dataset samples for them to stream to W&B. -* **Plots**: Use `wandb.plot()` with `wandb.Run.log()` to track charts. See [Log Plots](/models/track/log/plots/) for more information. -* **Tables**: Use `wandb.Table` to log data to visualize and query with W&B. See [Log Tables](/models/track/log/log-tables/) for more information. +* **Datasets**: You must explicitly log images or other dataset samples for them to stream to W&B. +* **Plots**: Use `wandb.plot()` with `wandb.Run.log()` to track charts. For more information, see [Create and track plots from experiments](/models/track/log/plots/). +* **Tables**: Use `wandb.Table` to log data to visualize and query with W&B. For more information, see [Log tables](/models/track/log/log-tables/). * **PyTorch gradients**: Add `wandb.Run.watch(model)` to see gradients of the weights as histograms in the UI. * **Configuration information**: Log hyperparameters, a link to your dataset, or the name of the architecture you're using as config parameters, passed in like this: `wandb.init(config=your_config_dictionary)`. * **Metrics**: Use `wandb.Run.log()` to see metrics from your model. If you log metrics like accuracy and loss from inside your training loop, you'll get live updating graphs in the UI. @@ -65,7 +68,7 @@ With W&B, you can decide exactly what you want to log. The following lists some ## Metric naming constraints -Due to GraphQL limitations, metric names in W&B must follow specific naming rules: +Due to GraphQL limitations, metric names in W&B must follow specific naming rules. When you choose names for the metrics you log, follow the rules in this section so that your metrics sort, query, and display correctly in the W&B UI: @@ -77,23 +80,23 @@ Avoid naming metrics with invalid characters (such as commas, spaces, or special ## Common workflows -1. **Compare the best accuracy**: To compare the best value of a metric across runs, set the summary value for that metric. By default, summary is set to the last value you logged for each key. This is useful in the table in the UI, where you can sort and filter runs based on their summary metrics, to help compare runs in a table or bar chart based on their _best_ accuracy, instead of final accuracy. For example: `wandb.run.summary["best_accuracy"] = best_accuracy` +The following workflows show how to combine `wandb.Run.log()` with related APIs to address frequent experiment tracking tasks: -2. **View multiple metrics on one chart**: Log multiple metrics in the same call. For example: +- **Compare the best accuracy**: To compare the best value of a metric across runs, set the summary value for that metric. By default, W&B sets the summary to the last value you logged for each key. This is useful in the table in the UI, where you can sort and filter runs based on their summary metrics to compare runs in a table or bar chart based on their _best_ accuracy instead of final accuracy. For example: `wandb.run.summary["best_accuracy"] = best_accuracy`. +- **View multiple metrics on one chart**: Log multiple metrics in the same call. For example: ```python with wandb.init() as run: run.log({"acc": 0.9, "loss": 0.1}) ``` You can then plot both metrics in the UI. -3. **Customize the x-axis**: Add a custom x-axis to the same log call to visualize your metrics against a different axis in the W&B dashboard. For example: +- **Customize the x-axis**: Add a custom x-axis to the same log call to visualize your metrics against a different axis in the W&B dashboard. For example: ```python with wandb.init() as run: run.log({'acc': 0.9, 'epoch': 3, 'batch': 117}) ``` - To set the default x-axis for a given metric use [Run.define_metric()](/models/ref/python/experiments/run#define_metric). - -4. **Log rich media and charts**: `wandb.Run.log()` supports the logging of a wide variety of data types, from [media like images and videos](/models/track/log/media/) to [tables](/models/track/log/log-tables/) and [charts](/models/app/features/custom-charts/). + To set the default x-axis for a given metric, use [Run.define_metric()](/models/ref/python/experiments/run#define_metric). +- **Log rich media and charts**: `wandb.Run.log()` supports many data types, from [media like images and videos](/models/track/log/media/) to [tables](/models/track/log/log-tables/) and [charts](/models/app/features/custom-charts/). -## Best practices and tips +## Best practices and tips For best practices and tips for Experiments and logging, see [Best Practices: Experiments and Logging](https://wandb.ai/wandb/pytorch-lightning-e2e/reports/W-B-Best-Practices-Guide--VmlldzozNTU1ODY1#w&b-experiments-and-logging). diff --git a/models/track/log/customize-logging-axes.mdx b/models/track/log/customize-logging-axes.mdx index f10cc783e3..e518bb4f7d 100644 --- a/models/track/log/customize-logging-axes.mdx +++ b/models/track/log/customize-logging-axes.mdx @@ -1,6 +1,7 @@ --- title: Customize log axes description: "Use define_metric() to set a custom x-axis for logged metrics instead of the default W&B step counter." +keywords: ["define_metric", "custom x-axis", "step_metric", "metric globs"] --- Set a custom x-axis when you log metrics to W&B. By default, W&B logs metrics as *steps*. Each step corresponds to a `wandb.Run.log()` API call. @@ -19,23 +20,23 @@ with wandb.init() as run: run.log(log_dict) ``` -In the project's workspace, the `validation_loss` metric is plotted against the `step` x-axis, which increments by 1 each time `wandb.Run.log()` is called. From the previous code, the x-axis shows the step numbers 0, 1, 2, ..., 9. +In the project's workspace, the `validation_loss` metric is plotted against the `step` x-axis, which increments by 1 each time `wandb.Run.log()` is called. From the previous code, the x-axis shows the step numbers from 0 to 9. Line plot panel that uses `step` as the x-axis. -In certain situations, it makes more sense to log metrics against a different x-axis such as a logarithmic x-axis. Use the [`define_metric()`](/models/ref/python/experiments/run/#define_metric) method to use any metric you log as a custom x-axis. +Sometimes you want to log metrics against a different x-axis, such as a logarithmic scale. Use the [`define_metric()`](/models/ref/python/experiments/run/#define_metric) method to use any metric you log as a custom x-axis. Specify the metric that you want to appear as the y-axis with the `name` parameter. The `step_metric` parameter specifies the metric you want to use as the x-axis. When you log a custom metric, specify a value for both the x-axis and the y-axis as key-value pairs in a dictionary. -Copy and paste the following code snippet to set a custom x-axis metric. Replace the values within `<>` with your own values: +To set a custom x-axis metric, use the following code snippet. Replace the values within `[]` with your own values: ```python import wandb -custom_step = "" # Name of custom x-axis -metric_name = "" # Name of y-axis metric +custom_step = "[CUSTOM-STEP]" # Name of custom x-axis +metric_name = "[METRIC-NAME]" # Name of y-axis metric with wandb.init() as run: # Specify the step metric (x-axis) and the metric to log against it (y-axis) @@ -49,7 +50,7 @@ with wandb.init() as run: run.log(log_dict) ``` -As an example, the following code snippet creates a custom x-axis called `x_axis_squared`. The value of the custom x-axis is the square of the for loop index `i` (`i**2`). The y-axis consists of mock values for validation loss (`"validation_loss"`) using Python's built-in `random` module: +For example, the following code snippet creates a custom x-axis called `x_axis_squared`. The value of the custom x-axis is the square of the for loop index `i` (`i**2`). The y-axis consists of mock values for validation loss (`"validation_loss"`) using Python's built-in `random` module: ```python import wandb @@ -66,13 +67,13 @@ with wandb.init() as run: run.log(log_dict) ``` -The following image shows the resulting plot in the W&B App UI. The `validation_loss` metric is plotted against the custom x-axis `x_axis_squared`, which is the square of the for loop index `i`. Note that the x-axis values are `0, 1, 4, 9, 16, 25, 36, 49, 64, 81`, which correspond to the squares of `0, 1, 2, ..., 9` respectively. +The following image shows the resulting plot in the W&B App UI. The `validation_loss` metric is plotted against the custom x-axis `x_axis_squared`, which is the square of the for loop index `i`. The x-axis values `0, 1, 4, 9, 16, 25, 36, 49, 64, 81` are the squares of `0` through `9`. Line plot panel that uses a custom x axis. Values are logged to W&B as the square of the loop number. -You can set a custom x-axis for multiple metrics using `globs` with string prefixes. As an example, the following code snippet plots logged metrics with the prefix `train/*` to the x-axis `train/step`: +You can set a custom x-axis for multiple metrics using `globs` with string prefixes. For example, the following code snippet plots logged metrics with the prefix `train/*` to the x-axis `train/step`: ```python import wandb diff --git a/models/track/log/distributed-training.mdx b/models/track/log/distributed-training.mdx index 985ffcb006..32a15a4c53 100644 --- a/models/track/log/distributed-training.mdx +++ b/models/track/log/distributed-training.mdx @@ -1,11 +1,14 @@ --- description: Use W&B to log distributed training experiments with multiple GPUs. title: Log distributed training experiments +keywords: ["DDP", "PyTorch distributed", "multi-node training", "shared mode", "rank 0"] --- -During a distributed training experiment, you train a model using multiple machines or clients in parallel. W&B can help you track distributed training experiments. Based on your use case, track distributed training experiments using one of the following approaches: +During a distributed training experiment, you train a model using multiple machines or clients in parallel. W&B can help you track distributed training experiments so that you can monitor metrics, system utilization, and model behavior across all participating processes. This guide is for ML practitioners running distributed training jobs who want to log experiment data with W&B. -* **Track a single process**: Track a rank 0 process (also known as a "leader" or "coordinator") with W&B. This is a common solution for logging distributed training experiments with the [PyTorch Distributed Data Parallel](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html#torch.nn.parallel.DistributedDataParallel) (DDP) Class. +Based on your use case, track distributed training experiments using one of the following approaches: + +* **Track a single process**: Track a rank 0 process (also known as a "leader" or "coordinator") with W&B. This is a common solution for logging distributed training experiments with the [PyTorch Distributed Data Parallel](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html#torch.nn.parallel.DistributedDataParallel) (DDP) class. * **Track multiple processes**: For multiple processes, you can either: * Track each process separately using one run per process. You can optionally group them together in the W&B App UI. * Track all processes to a single run. @@ -15,9 +18,9 @@ During a distributed training experiment, you train a model using multiple machi Each concurrent connection takes compute, memory, and network resources. Even empty client connections that don't log metrics regularly push system metric updates, leading to slower performance when loading charts. -W&B recommends that you limit the maximum number of concurrent client connections as appropriate for your workload and that you monitor resource usage over time. W&B has tested with a hard limit of 300 concurrent client connections in **Dedicated Cloud**. +W&B recommends that you limit the maximum number of concurrent client connections as appropriate for your workload and that you monitor resource usage over time. W&B has tested with a hard limit of 300 concurrent client connections in Dedicated Cloud. -In **Multi-tenant Cloud** organizations, client connections for distributed training are subject to the same [rate limits](/models/track/limits#rate-limits) as regular training runs. Users on [Teams and Enterprise plans](https://wandb.ai/site/pricing) receive higher rate limits than those on the Free plan. +In Multi-tenant Cloud organizations, client connections for distributed training are subject to the same [rate limits](/models/track/limits#rate-limits) as regular training runs. Users on [Teams and Enterprise plans](https://wandb.ai/site/pricing) receive higher rate limits than those on the Free plan. {/* The following examples demonstrate how to track metrics with W&B using PyTorch DDP on two GPUs on a single machine. [PyTorch DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) (`DistributedDataParallel` in`torch.nn`) is a popular library for distributed training. The basic principles apply to any distributed training setup, but the details of implementation may differ. @@ -28,16 +31,17 @@ Explore the code behind these examples in the W&B GitHub examples repository [he ## Track a single process -This section describes how to track values and metrics available to your rank 0 process. Use this approach to track only metrics that are available from a single process. Typical metrics include GPU/CPU utilization, behavior on a shared validation set, gradients and parameters, and loss values on representative data examples. +The following sections describe how to track values and metrics available to your rank 0 process. Use this approach to track only metrics that are available from a single process. Typical metrics include GPU/CPU utilization, behavior on a shared validation set, gradients and parameters, and loss values on representative data examples. Within the rank 0 process, initialize a W&B run with [`wandb.init()`](/models/ref/python/functions/init) and log experiments ([`wandb.Run.log()`](/models/ref/python/experiments/run/#method-runlog)) to that run. -The following [sample Python script (`log-ddp.py`)](https://github.com/wandb/examples/blob/master/examples/pytorch/pytorch-ddp/log-ddp.py) demonstrates one way to track metrics on two GPUs on a single machine using PyTorch DDP. [PyTorch DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) (`DistributedDataParallel` in`torch.nn`) is a popular library for distributed training. The basic principles apply to any distributed training setup, but the implementation may differ. +The following [sample Python script (`log-ddp.py`)](https://github.com/wandb/examples/blob/master/examples/pytorch/pytorch-ddp/log-ddp.py) demonstrates one way to track metrics on two GPUs on a single machine using PyTorch DDP. [PyTorch DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) (`DistributedDataParallel` in `torch.nn`) is a library for distributed training. The basic principles apply to any distributed training setup, but the implementation may differ. + +The Python script does the following: -The Python script: 1. Starts multiple processes with `torch.distributed.launch`. -1. Checks the rank with the `--local_rank` command line argument. -1. If the rank is set to 0, sets up `wandb` logging conditionally in the [`train()`](https://github.com/wandb/examples/blob/master/examples/pytorch/pytorch-ddp/log-ddp.py#L24) function. +2. Checks the rank with the `--local_rank` command-line argument. +3. If the rank is set to 0, sets up `wandb` logging conditionally in the [`train()`](https://github.com/wandb/examples/blob/master/examples/pytorch/pytorch-ddp/log-ddp.py#L24) function. ```python if __name__ == "__main__": @@ -72,18 +76,19 @@ However, the loss values as a function epoch and batch size were only logged fro ## Track multiple processes -Track multiple processes with W&B with one of the following approaches: -* [Tracking each process separately](/models/track/log/distributed-training/#track-each-process-separately) by creating a run for each process. -* [Tracking all processes to a single run](/models/track/log/distributed-training/#track-all-processes-to-a-single-run). +Track multiple processes with W&B when you need metrics, logs, or artifacts from more than only the rank 0 process. The following sections describe two approaches: + +* [Track each process separately](/models/track/log/distributed-training/#track-each-process-separately) by creating a run for each process. +* [Track all processes to a single run](/models/track/log/distributed-training/#track-all-processes-to-a-single-run). ### Track each process separately -This section describes how to track each process separately by creating a run for each process. Within each run you log metrics, artifacts, and forth to their respective run. Call `wandb.Run.finish()` at the end of training, to mark that the run has completed so that all processes exit properly. +The following sections describe how to track each process separately by creating a run for each process. Within each run you log metrics, artifacts, and so forth to their respective run. Call `wandb.Run.finish()` at the end of training, to mark that the run has completed so that all processes exit properly. -You might find it difficult to keep track of runs across multiple experiments. To mitigate this, provide a value to the `group` parameter when you initialize W&B (`wandb.init(group='group-name')`) to keep track of which run belongs to a given experiment. For more information about how to keep track of training and evaluation W&B Runs in experiments, see [Group Runs](/models/runs/grouping/). +You might find it difficult to keep track of runs across multiple experiments. To mitigate this, provide a value to the `group` parameter when you initialize W&B (`wandb.init(group='group-name')`) to keep track of which run belongs to a given experiment. For more information about how to keep track of training and evaluation runs in experiments, see [Group Runs](/models/runs/grouping/). -**Use this approach if you want to track metrics from individual processes**. Typical examples include the data and predictions on each node (for debugging data distribution) and metrics on individual batches outside of the main node. This approach is not necessary to get system metrics from all nodes nor to get summary statistics available on the main node. +**Use this approach if you want to track metrics from individual processes**. Typical examples include the data and predictions on each node (for debugging data distribution) and metrics on individual batches outside of the main node. This approach isn't necessary to get system metrics from all nodes nor to get summary statistics available on the main node. The following Python code snippet demonstrates how to set the group parameter when you initialize W&B: @@ -104,13 +109,13 @@ if __name__ == "__main__": run.finish() # mark the run as finished ``` -Explore the W&B App UI to view an [example dashboard](https://wandb.ai/ayush-thakur/DDP?workspace=user-noahluna) of metrics tracked from multiple processes. Note that there are two W&B Runs grouped together in the left sidebar. Click on a group to view the dedicated group page for the experiment. The dedicated group page displays metrics from each process separately. +Explore the W&B App UI to view an [example dashboard](https://wandb.ai/ayush-thakur/DDP?workspace=user-noahluna) of metrics tracked from multiple processes. The two runs are grouped together in the left sidebar. Click a group to view the dedicated group page for the experiment. The dedicated group page displays metrics from each process separately. Grouped distributed runs -The preceding image demonstrates the W&B App UI dashboard. On the sidebar we see two experiments. One labeled 'null' and a second (bound by a yellow box) called 'DPP'. If you expand the group (select the Group dropdown) you will see the W&B Runs that are associated to that experiment. +The preceding image demonstrates the W&B App UI dashboard. The sidebar shows two experiments: one labeled "null" and a second (bound by a yellow box) called "DPP". Expand the group (select the **Group** dropdown) to see the runs associated with that experiment. ### Organize distributed runs @@ -118,29 +123,29 @@ Set the `job_type` parameter when you initialize W&B (`wandb.init(job_type='type ```python # Main coordinating node - with wandb.init(project="", job_type="main", group="experiment_1") as run: + with wandb.init(project="[PROJECT-NAME]", job_type="main", group="experiment_1") as run: # Training code # Reporting worker nodes - with wandb.init(project="", job_type="worker", group="experiment_1") as run: + with wandb.init(project="[PROJECT-NAME]", job_type="worker", group="experiment_1") as run: # Training code ``` -Once you have set the `job_type` for your nodes, you can create [saved views](/models/track/workspaces/#create-a-new-saved-workspace-view) in your workspace to organize your runs. Click the **action ()** menu at the top right and click **Save as new view**. +Once you have set the `job_type` for your nodes, you can create [saved views](/models/track/workspaces/#create-a-new-saved-workspace-view) in your workspace to organize your runs. Click the **action ()** menu and click **Save as new view**. For example, you could create the following saved views: - - **Default view**: Filter out worker nodes to reduce noise + - **Default view**: Filter out worker nodes to reduce noise. - Click **Filter**, then set **Job Type** to `worker`. - - Shows only your reporting nodes + - Shows only your reporting nodes. - - **Debug view**: Focus on worker nodes for troubleshooting - - Click **Filter**, then set **Job Type** `==` `worker` and set **State** to `IN` `crashed`. - - Shows only worker nodes that have crashed or are in error states + - **Debug view**: Focus on worker nodes for troubleshooting. + - Click **Filter**, then set **Job Type** `==` `worker` and set **State** to `IN` `crashed`. + - Shows only worker nodes that have crashed or are in error states. - - **All nodes view**: See everything together - - No filter - - Useful for comprehensive monitoring + - **All nodes view**: See everything together. + - No filter. + - Useful for comprehensive monitoring. To open a saved view, click **Workspaces** in the project sidebar, then click the menu. Workspaces appear at the top of the list and saved views appear at the bottom. @@ -155,7 +160,6 @@ Parameters prefixed by `x_` (such as `x_label`) are in public preview. Create a To track multiple processes to a single run, you must have: - W&B Python SDK version `v0.19.9` or newer. - - W&B Server v0.68 or newer. @@ -166,12 +170,12 @@ Within the primary node, initialize a W&B run with [`wandb.init()`](/models/ref/ 1. The `mode` parameter set to `"shared"` to enable shared mode. 2. A unique label for [`x_label`](https://github.com/wandb/wandb/blob/main/wandb/sdk/wandb_settings.py#L638). You use the value you specify for `x_label` to identify which node the data is coming from in logs and system metrics in the W&B App UI. If left unspecified, W&B creates a label for you using the hostname and a random hash. 3. Set the [`x_primary`](https://github.com/wandb/wandb/blob/main/wandb/sdk/wandb_settings.py#L660) parameter to `True` to indicate that this is the primary node. -4. Optionally provide a list of GPU indexes ([0,1,2]) to `x_stats_gpu_device_ids` to specify which GPUs W&B tracks metrics for. If you do not provide a list, W&B tracks metrics for all GPUs on the machine. +4. Optionally provide a list of GPU indexes (`[0, 1, 2]`) to `x_stats_gpu_device_ids` to specify which GPUs W&B tracks metrics for. If you don't provide a list, W&B tracks metrics for all GPUs on the machine. -Make note of the run ID of the primary node. Each worker node needs the run ID of the primary node. +Make note of the run ID of the primary node. Each worker node requires the run ID of the primary node. -`x_primary=True` distinguishes a primary node from worker nodes. Primary nodes are the only nodes that upload files shared across nodes such as configuration files, telemetry and more. Worker nodes do not upload these files. +`x_primary=True` distinguishes a primary node from worker nodes. Primary nodes are the only nodes that upload files shared across nodes such as configuration files, telemetry, and more. Worker nodes don't upload these files. For each worker node, initialize a W&B run with [`wandb.init()`](/models/ref/python/functions/init) and provide the following: @@ -187,13 +191,13 @@ For each worker node, initialize a W&B run with [`wandb.init()`](/models/ref/pyt * Consider defining an environment variable on each worker node to set the run ID of the primary node. -The following sample code demonstrates the high level requirements for tracking multiple processes to a single run: +The following sample code demonstrates the high-level requirements for tracking multiple processes to a single run: ```python import wandb -entity = "" -project = "" +entity = "[TEAM-ENTITY]" +project = "[PROJECT-NAME]" # Initialize a run in the primary node run = wandb.init( @@ -228,20 +232,20 @@ run = wandb.init( ) ``` -In a real world example, each worker node might be on a separate machine. +In a real-world example, each worker node might be on a separate machine. See the [Distributed Training with Shared Mode](https://wandb.ai/dimaduev/simple-cnn-ddp/reports/Distributed-Training-with-Shared-Mode--VmlldzoxMTI0NTE1NA) report for an end-to-end example on how to train a model on a multi-node and multi-GPU Kubernetes cluster in GKE. -View console logs from multi node processes in the project that the run logs to: +To view console logs from multi-node processes in the project that the run logs to: 1. Navigate to the project that contains the run. -2. Click on the **Runs** tab in the project sidebar. -3. Click on the run you want to view. -4. Click on the **Logs** tab in the project sidebar. +2. Click the **Runs** tab in the project sidebar. +3. Click the run you want to view. +4. Click the **Logs** tab in the project sidebar. -You can filter console logs based on the labels you provide for `x_label` in the UI search bar located at the top of the console log page. For example, the following image shows which options are available to filter the console log by if values `rank0`, `rank1`, `rank2`, `rank3`, `rank4`, `rank5`, and `rank6` are provided to `x_label`.` +You can filter console logs based on the labels you provide for `x_label` in the UI search bar on the console log page. For example, the following image shows the available filter options when you provide values `rank0`, `rank1`, `rank2`, `rank3`, `rank4`, `rank5`, and `rank6` to `x_label`. Multi-node console logs @@ -255,15 +259,15 @@ W&B aggregates system metrics from all nodes and displays them in the W&B App UI Multi-node system metrics -See [Line plots](/models/app/features/panels/line-plot/) for information on how to customize line plot panels. +For more information about how to customize line plot panels, see [Line plots](/models/app/features/panels/line-plot/). ## Example use cases -The following code snippets demonstrate common scenarios for advanced distributed use cases. +The following code snippets demonstrate common scenarios for advanced distributed use cases. Use these patterns when your training setup involves spawned processes or when you need to share a single run across processes. ### Spawn process -Use the `wandb.setup()`method in your main function if you initiate a run in a spawned process: +Use the `wandb.setup()` method in your main function if you initiate a run in a spawned process: ```python import multiprocessing as mp @@ -303,28 +307,29 @@ if __name__ == "__main__": main() ``` -W&B can not guarantee the logging order. Synchronization should be done by the author of the script. +W&B can't guarantee the logging order. The author of the script must handle synchronization. ## Troubleshooting -There are two common issues you might encounter when using W&B and distributed training: +The following sections describe two common issues you might encounter when using W&B with distributed training, along with guidance about enabling W&B Service to improve reliability: + +- **Becoming unresponsive at the beginning of training**: A `wandb` process can stop responding if the `wandb` multiprocessing interferes with the multiprocessing from distributed training. +- **Becoming unresponsive at the end of training**: A training job might stop responding if the `wandb` process can't determine when to exit. Call the `wandb.Run.finish()` API at the end of your Python script to signal W&B that the run finished. The `wandb.Run.finish()` API finishes uploading data and causes W&B to exit. -1. **Hanging at the beginning of training** - A `wandb` process can hang if the `wandb` multiprocessing interferes with the multiprocessing from distributed training. -2. **Hanging at the end of training** - A training job might hang if the `wandb` process does not know when it needs to exit. Call the `wandb.Run.finish()` API at the end of your Python script to tell W&B that the run finished. The `wandb.Run.finish()` API will finish uploading data and will cause W&B to exit. -W&B recommends using `wandb service` command to improve the reliability of your distributed jobs. Both of the preceding training issues are commonly found in versions of the W&B SDK where wandb service is unavailable. +W&B recommends using `wandb service` command to improve the reliability of your distributed jobs. Both of the preceding training issues commonly occur in versions of the W&B SDK where `wandb service` is unavailable. ### Enable W&B Service Depending on your version of the W&B SDK, you might already have W&B Service enabled by default. -#### W&B SDK 0.13.0 and above +#### W&B SDK 0.13.0 and later -W&B Service is enabled by default for versions of the W&B SDK `0.13.0` and above. +W&B Service is enabled by default for versions of the W&B SDK `0.13.0` and later. -#### W&B SDK 0.12.5 and above +#### W&B SDK 0.12.5 and later -Modify your Python script to enable W&B Service for W&B SDK version 0.12.5 and above. Use the `wandb.require()` method and pass the string `"service"` within your main function: +Modify your Python script to enable W&B Service for W&B SDK version `0.12.5` and later. Use the `wandb.require()` method and pass the string `"service"` within your main function: ```python if __name__ == "__main__": @@ -336,8 +341,8 @@ def main(): # rest-of-your-script-goes-here ``` -For optimal experience we do recommend you upgrade to the latest version. +W&B recommends that you upgrade to the latest version. -**W&B SDK 0.12.4 and below** +#### W&B SDK 0.12.4 and earlier -Set the `WANDB_START_METHOD` environment variable to `"thread"` to use multithreading instead if you use a W&B SDK version 0.12.4 and below. +Set the `WANDB_START_METHOD` environment variable to `"thread"` to use multithreading instead if you use a W&B SDK version `0.12.4` or earlier. diff --git a/models/track/log/log-models.mdx b/models/track/log/log-models.mdx index 326ce381ba..3bd038b60b 100644 --- a/models/track/log/log-models.mdx +++ b/models/track/log/log-models.mdx @@ -1,6 +1,7 @@ --- title: Log models description: "Log model artifacts to a W&B run and retrieve them later using the log_model and use_model SDK methods." +keywords: ["log_model", "use_model", "model artifact", "model lineage", "experiment tracking"] --- import { ColabLink } from '/snippets/_includes/colab-link.mdx'; @@ -8,45 +9,48 @@ import { ColabLink } from '/snippets/_includes/colab-link.mdx'; # Log models -The following guide describes how to log models to a W&B run and interact with them. +This guide shows you how to log model files to a W&B run as a model artifact, and how to retrieve and reuse those files in later runs. Logging models to a run lets you track which experiment produced a given model, capture its lineage, and make it available for downstream use. This page is for users who want lightweight model tracking as part of an experiment tracking workflow. The following APIs are useful for tracking models as a part of your experiment tracking workflow. Use the APIs listed on this page to log models to a run, and to access metrics, tables, media, and other objects. W&B suggests that you use [W&B Artifacts](/models/artifacts/) if you want to: -- Create and keep track of different versions of serialized data besides models, such as datasets, prompts, and more. +- Create and keep track of different versions of serialized data besides models, such as datasets and prompts. - Explore [lineage graphs](/models/artifacts/explore-and-traverse-an-artifact-graph/) of a model or any other objects tracked in W&B. -- Interact with the model artifacts these methods created, such as [updating properties](/models/artifacts/update-an-artifact/) (metadata, aliases, and descriptions) +- Interact with the model artifacts these methods created, such as [updating properties](/models/artifacts/update-an-artifact/) (metadata, aliases, and descriptions). -For more information on W&B Artifacts and advanced versioning use cases, see the [Artifacts](/models/artifacts/) documentation. +For more information about W&B Artifacts and advanced versioning use cases, see the [Artifacts](/models/artifacts/) documentation. ## Log a model to a run -Use the [`log_model`](/models/ref/python/experiments/run#log_model) to log a model artifact that contains content within a directory you specify. The [`log_model`](/models/ref/python/experiments/run#log_model) method also marks the resulting model artifact as an output of the W&B run. -You can track a model's dependencies and the model's associations if you mark the model as the input or output of a W&B run. View the lineage of the model within the W&B App UI. See the [Explore and traverse artifact graphs](/models/artifacts/explore-and-traverse-an-artifact-graph/) page within the [Artifacts](/models/artifacts/) chapter for more information. +This section explains how to use `log_model` to log a model artifact from a W&B run. -Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or [reference URI](/models/artifacts/track-external-files/#amazon-s3--gcs--azure-blob-storage-references) to an external bucket such as `s3://bucket/path`. +Use the [`log_model`](/models/ref/python/experiments/run#log_model) method to log a model artifact that contains content within a directory you specify. The [`log_model`](/models/ref/python/experiments/run#log_model) method also marks the resulting model artifact as an output of the W&B run. -Ensure to replace values enclosed in `<>` with your own. +You can track a model's dependencies and the model's associations if you mark the model as the input or output of a W&B run. View the lineage of the model within the W&B App UI. For more information, see the [Explore and traverse artifact graphs](/models/artifacts/explore-and-traverse-an-artifact-graph/) page within the [Artifacts](/models/artifacts/) chapter. + +Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or [reference URI](/models/artifacts/track-external-files/#amazon-s3--gcs--azure-blob-storage-references) to an external bucket such as `s3://bucket/path`. + +Replace `[YOUR-PROJECT]` with your project name, `[YOUR-ENTITY]` with your entity name, `[PATH-TO-MODEL]` with the path to your model files, and `[NAME]` with a name for the model artifact. ```python import wandb # Initialize a W&B run -with wandb.init(project="", entity="") as run: +with wandb.init(project="[YOUR-PROJECT]", entity="[YOUR-ENTITY]") as run: # Log the model - run.log_model(path="", name="") + run.log_model(path="[PATH-TO-MODEL]", name="[NAME]") ``` -Optionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B will use the basename of the input path prepended with the run ID as the name. +Optionally provide a name for the model artifact for the `name` parameter. If `name` isn't specified, W&B uses the basename of the input path prepended with the run ID as the name. -Keep track of the `name` that you, or W&B assigns, to the model. You will need the name of the model to retrieve the model path with the [`wandb.Run.use_model()`](/models/ref/python/experiments/run#use_model) method. +Keep track of the `name` that you, or W&B assigns, to the model. You need the name of the model to retrieve the model path with the [`wandb.Run.use_model()`](/models/ref/python/experiments/run#use_model) method. -See [`log_model`](/models/ref/python/experiments/run#log_model) in the API Reference for parameters. +For parameters, see [`log_model`](/models/ref/python/experiments/run#log_model) in the API Reference.
@@ -97,9 +101,9 @@ with wandb.init(entity="charlie", project="mnist-experiments", config=config) as run.log_model(path=full_path, name="MNIST") ``` -When you call `wandb.Run.log_model()`, a model artifact named `MNIST` is created and the file `model.h5` is added to the model artifact. Your terminal or notebook will print information of where to find information about the run the model was logged to. +When you call `wandb.Run.log_model()`, W&B creates a model artifact named `MNIST` and adds the file `model.h5` to the model artifact. Your terminal or notebook prints a link to the run the model was logged to. -```python +```text View run different-surf-5 at: https://wandb.ai/charlie/mnist-experiments/runs/wlby6fuw Synced 5 W&B file(s), 0 media file(s), 1 artifact file(s) and 0 other file(s) Find logs at: ./wandb/run-20231206_103511-wlby6fuw/logs @@ -108,32 +112,38 @@ Find logs at: ./wandb/run-20231206_103511-wlby6fuw/logs
+After the run completes, the model files are stored as a model artifact in W&B and are linked to the run as an output. + ## Download and use a logged model -Use the [`use_model`](/models/ref/python/experiments/run#use_model) function to access and download models files previously logged to a W&B run. + +After you log a model, you can retrieve its files in a later run to evaluate it, fine-tune it, or use it for inference. + +Use the [`use_model`](/models/ref/python/experiments/run#use_model) function to access and download model files previously logged to a W&B run. Provide the name of the model artifact where the model files you want to retrieve are stored. The name you provide must match the name of an existing logged model artifact. -If you did not define `name` when you originally logged the files with `log_model`, the default name assigned is the basename of the input path, prepended with the run ID. +If you didn't define `name` when you originally logged the files with `log_model`, the default name assigned is the basename of the input path, prepended with the run ID. + +Replace `[YOUR-PROJECT]` with your project name, `[YOUR-ENTITY]` with your entity name, and `[YOUR-MODEL-NAME]` with the name of the model artifact to retrieve. + -Ensure to replace the values enclosed in `<>` with your own: - ```python import wandb # Initialize a run -with wandb.init(project="", entity="") as run: +with wandb.init(project="[YOUR-PROJECT]", entity="[YOUR-ENTITY]") as run: # Access and download model. Returns path to downloaded artifact - downloaded_model_path = run.use_model(name="") + downloaded_model_path = run.use_model(name="[YOUR-MODEL-NAME]") ``` -The [use_model](/models/ref/python/experiments/run#use_model) function returns the path of downloaded model files. Keep track of this path if you want to link this model later. In the preceding code snippet, the returned path is stored in a variable called `downloaded_model_path`. +The [`use_model`](/models/ref/python/experiments/run#use_model) function returns the path of downloaded model files. Keep track of this path if you want to link this model later. In the preceding code snippet, the returned path is stored in a variable called `downloaded_model_path`.
Example: Download and use a logged model -For example, in the following code snippet a user called the `use_model` API. They specified the name of the model artifact they want to fetch and they also provided a version/alias. They then stored the path that is returned from the API to the `downloaded_model_path` variable. +In the following code snippet, you call the `use_model` API. You specify the name of the model artifact to fetch along with a version or alias, then store the returned path in the `downloaded_model_path` variable. ```python import wandb @@ -150,4 +160,4 @@ with wandb.init(project=project, entity=entity) as run: ```
-See [`use_model`](/models/ref/python/experiments/run#use_model) in the API Reference for parameters and return type. \ No newline at end of file +For parameters and return type, see [`use_model`](/models/ref/python/experiments/run#use_model) in the API Reference. \ No newline at end of file diff --git a/models/track/log/log-summary.mdx b/models/track/log/log-summary.mdx index 0617541c33..a9bfbbcd74 100644 --- a/models/track/log/log-summary.mdx +++ b/models/track/log/log-summary.mdx @@ -1,11 +1,12 @@ --- title: Log summary metrics description: "Track and customize single summary metrics like best accuracy or minimum loss on a W&B run using run.summary." +keywords: ["run.summary", "define_metric", "best accuracy", "summary dictionary", "Public API"] --- -In addition to values that change over time during training, it is often important to track a single value that summarizes a model or a preprocessing step. Log this information in a W&B Run's `summary` dictionary. A Run's summary dictionary can handle numpy arrays, PyTorch tensors or TensorFlow tensors. When a value is one of these types we persist the entire tensor in a binary file and store high level metrics in the summary object, such as min, mean, variance, percentiles, and more. +In addition to values that change over time during training, it's often important to track a single value that summarizes a model or a preprocessing step. Log this information in a W&B run's `summary` dictionary. A run's summary dictionary can handle numpy arrays, PyTorch tensors, or TensorFlow tensors. When a value is one of these types, W&B persists the entire tensor in a binary file and stores high-level metrics in the summary object, such as min, mean, variance, and percentiles. -The last value logged with `wandb.Run.log()` is automatically set as the summary dictionary in a W&B Run. If a summary metric dictionary is modified, the previous value is lost. +The last value logged with `wandb.Run.log()` is automatically set as the summary dictionary in a W&B run. If you modify a summary metric dictionary, the previous value is lost. The following code snippet demonstrates how to provide a custom summary metric to W&B: @@ -22,7 +23,7 @@ with wandb.init(config=args) as run: best_accuracy = test_accuracy ``` -You can update the summary attribute of an existing W&B Run after training has completed. Use the [W&B Public API](/models/ref/python/public-api/) to update the summary attribute: +You can update the summary attribute of an existing W&B run after training completes. Use the [W&B Public API](/models/ref/python/public-api/) to update the summary attribute: ```python api = wandb.Api() @@ -44,7 +45,7 @@ By default, the summary uses the final value from history. To customize summary * `"last"` * `"none"` -You can use `"best"` only when you also set the optional `objective` argument to `"minimize"` or `"maximize"`. +You can use `"best"` only when you also set the optional `objective` argument to `"minimize"` or `"maximize"`. The following example adds the min and max values of loss and accuracy to the summary: @@ -73,7 +74,7 @@ with wandb.init() as run: ## View summary metrics -View summary values in a run's **Overview** page or the project's runs table. +After you log summary values, you can view them on a run's **Overview** page or in the project's runs table. The following tabs describe each option. @@ -89,10 +90,10 @@ View summary values in a run's **Overview** page or the project's runs table. 1. Navigate to the W&B App. 2. Select the **Runs** tab. -3. Within the runs table, you can view the summary values within the columns based on the name of the summary value. +3. In the runs table, view the summary values in the columns based on the name of the summary value. -You can use the W&B Public API to fetch the summary values of a run. +You can use the W&B Public API to fetch the summary values of a run. The following code example demonstrates one way to retrieve the summary values logged to a specific run using the W&B Public API and pandas: @@ -100,9 +101,9 @@ The following code example demonstrates one way to retrieve the summary values l import wandb import pandas -entity = "" -project = "" -run_name = "" # Name of run with summary values +entity = "[YOUR-ENTITY]" +project = "[YOUR-PROJECT]" +run_name = "[YOUR-RUN-NAME]" # Name of run with summary values all_runs = [] @@ -139,7 +140,3 @@ df[df['name']==run_name].summary.reset_index(drop=True).to_dict() - - - - diff --git a/models/track/log/log-tables.mdx b/models/track/log/log-tables.mdx index c74927b0e6..6224d195db 100644 --- a/models/track/log/log-tables.mdx +++ b/models/track/log/log-tables.mdx @@ -1,33 +1,36 @@ --- description: "Create, populate, and log W&B Tables to visualize and query structured data from your experiment runs." title: Log tables +keywords: [wandb.Table, JoinedTable, add_data, log table, artifact tables] --- -Use `wandb.Table` to log data to visualize and query with W&B. In this guide, learn how to: +Use `wandb.Table` to log structured data that you can visualize and query in W&B. Tables let you inspect model predictions, dataset samples, and other per-row results alongside your metrics, so you can debug model behavior and share findings with collaborators. -1. [Create Tables](./log-tables#create-tables) -2. [Add Data](./log-tables#add-data) -3. [Retrieve Data](./log-tables#retrieve-data) -4. [Save Tables](./log-tables#save-tables) +This guide walks through the typical table workflow: + +1. [Create tables](./log-tables#create-tables) +2. [Add data](./log-tables#add-data) +3. [Retrieve data](./log-tables#retrieve-data) +4. [Save tables](./log-tables#save-tables) ## Create tables -To define a Table, specify the columns you want to see for each row of data. Each row might be a single item in your training dataset, a particular step or epoch during training, a prediction made by your model on a test item, an object generated by your model, etc. Each column has a fixed type: numeric, text, boolean, image, video, audio, etc. You do not need to specify the type in advance. Give each column a name, and make sure to only pass data of that type into that column index. For a more detailed example, see the [W&B Tables guide](https://wandb.ai/stacey/mnist-viz/reports/Guide-to-W-B-Tables--Vmlldzo2NTAzOTk#1.-how-to-log-a-wandb.table). +To define a table, specify the columns you want to see for each row of data. Each row might be a single item in your training dataset, a step or epoch during training, a prediction your model made on a test item, or an object your model generated. Each column has a fixed type: numeric, text, boolean, image, video, audio, and so on. You don't need to specify the type in advance. Give each column a name, and only pass data of that type into that column index. For a more detailed example, see the [W&B Tables guide](https://wandb.ai/stacey/mnist-viz/reports/Guide-to-W-B-Tables--Vmlldzo2NTAzOTk#1.-how-to-log-a-wandb.table). Use the `wandb.Table` constructor in one of two ways: -1. **List of Rows:** +1. **List of rows:** - Log named columns and rows of data. For example the following code snippet generates a table with two rows and three columns: + Log named columns and rows of data. For example, the following code snippet generates a table with two rows and three columns: ```python wandb.Table(columns=["a", "b", "c"], data=[["1a", "1b", "1c"], ["2a", "2b", "2c"]]) ``` -2. **Pandas DataFrame:** Log a DataFrame using `wandb.Table(dataframe=my_df)`. Column names will be extracted from the DataFrame. +2. **Pandas DataFrame:** Log a DataFrame using `wandb.Table(dataframe=my_df)`. W&B extracts column names from the DataFrame. -#### From an existing array or dataframe +### From an existing array or dataframe ```python # assume a model has returned predictions on four images @@ -50,14 +53,14 @@ test_table = wandb.Table(data=my_data, columns=columns) ## Add data -Tables are mutable. As your script executes you can add more data to your table, up to 200,000 rows. There are two ways to add data to a table: +After you create a table, populate it with the rows or columns you want to track. Tables are mutable. As your script executes, you can add more data to your table, up to 200,000 rows. You can add data to a table in two ways: -1. **Add a Row**: `table.add_data("3a", "3b", "3c")`. Note that the new row is not represented as a list. If your row is in list format, use the star notation, `*` ,to expand the list to positional arguments: `table.add_data(*my_row_list)`. The row must contain the same number of entries as there are columns in the table. -2. **Add a Column**: `table.add_column(name="col_name", data=col_data)`. Note that the length of `col_data` must be equal to the table's current number of rows. Here, `col_data` can be a list data, or a NumPy NDArray. +1. **Add a row**: `table.add_data("3a", "3b", "3c")`. The new row isn't represented as a list. If your row is in list format, use the star notation, `*`, to expand the list to positional arguments: `table.add_data(*my_row_list)`. The row must contain the same number of entries as there are columns in the table. +2. **Add a column**: `table.add_column(name="col_name", data=col_data)`. The length of `col_data` must equal the table's current number of rows. Here, `col_data` can be a list data, or a NumPy NDArray. -### Adding data incrementally +### Add data incrementally -This code sample shows how to create and populate a W&B table incrementally. You define the table with predefined columns, including confidence scores for all possible labels, and add data row by row during inference. You can also [add data to tables incrementally when resuming runs](#adding-data-to-resumed-runs). +This code sample shows how to create and populate a W&B table incrementally. You define the table with predefined columns, including confidence scores for all possible labels, and add data row by row during inference. You can also [add data to tables incrementally when resuming runs](#add-data-to-resumed-runs). ```python # Define the columns for the table, including confidence scores for each label @@ -78,14 +81,14 @@ for img_id, img in enumerate(mnist_test_data): ) # Add row data to the table ``` -#### Adding data to resumed runs +### Add data to resumed runs You can incrementally update a W&B table in resumed runs by loading an existing table from an artifact, retrieving the last row of data, and adding the updated metrics. Then, reinitialize the table for compatibility and log the updated version back to W&B. ```python import wandb -# Initialize a run +# Initialize a run with wandb.init(project="my_project") as run: # Load the existing table from the artifact @@ -113,18 +116,18 @@ with wandb.init(project="my_project") as run: ## Retrieve data -Once data is in a Table, access it by column or by row: +After data is in a table, you can read it back to compute statistics, feed it into downstream code, or inspect specific rows. Access the data by column or by row: -1. **Row Iterator**: Use the row iterator of Table such as `for ndx, row in table.iterrows(): ...` to efficiently iterate over the data's rows. -2. **Get a Column**: Retrieve a column of data using `table.get_column("col_name")` . As a convenience, you can pass `convert_to="numpy"` to convert the column to a NumPy NDArray of primitives. This is useful if your column contains media types such as `wandb.Image` so that you can access the underlying data directly. +1. **Row iterator**: Use the row iterator of the table such as `for ndx, row in table.iterrows(): ...` to efficiently iterate over the data's rows. +2. **Get a column**: Retrieve a column of data using `table.get_column("col_name")`. As a convenience, you can pass `convert_to="numpy"` to convert the column to a NumPy NDArray of primitives. This is useful if your column contains media types such as `wandb.Image`, so that you can access the underlying data directly. ## Save tables -After you generate a table of data in your script, for example a table of model predictions, save it to W&B to visualize the results live. +After you generate a table of data in your script, for example, a table of model predictions, save it to W&B to visualize the results live. Saving a table persists it to the W&B backend, so that you and your collaborators can view, query, and compare it in the UI. ### Log a table to a run -Use `wandb.Run.log()` to save your table to the run, like so: +To attach a table to a specific run so that it appears in that run's workspace, use `wandb.Run.log()`: ```python with wandb.init() as run: @@ -132,35 +135,35 @@ with wandb.init() as run: run.log({"table_key": my_table}) ``` -Each time a table is logged to the same key, a new version of the table is created and stored in the backend. This means you can log the same table across multiple training steps to see how model predictions improve over time, or compare tables across different runs, as long as they're logged to the same key. You can log up to 200,000 rows. +Each time you log a table to the same key, W&B creates a new version of the table and stores it in the backend. You can log the same table across multiple training steps to see how model predictions improve over time, or compare tables across different runs, as long as you log them to the same key. You can log up to 200,000 rows. To log more than 200,000 rows, you can override the limit with: `wandb.Table.MAX_ARTIFACT_ROWS = X` -However, this would likely cause performance issues, such as slower queries, in the UI. +However, this can cause performance issues, such as slower queries, in the UI. ### Access tables programmatically -In the backend, Tables are persisted as Artifacts. If you are interested in accessing a specific version, you can do so with the artifact API: +In the backend, W&B persists tables as artifacts. To access a specific version of a logged table from code, for example, to reload predictions for further analysis, use the artifact API. Replace `[RUN-ID]` with the run ID, `[TABLE-NAME]` with the table name, and `[TAG]` with the artifact alias or version tag: ```python with wandb.init() as run: - my_table = run.use_artifact("run--:").get("") + my_table = run.use_artifact("run-[RUN-ID]-[TABLE-NAME]:[TAG]").get("[TABLE-NAME]") ``` -For more information on Artifacts, see the [Artifacts Chapter](/models/artifacts/) in the Developer Guide. +For more information about artifacts, see the [Artifacts chapter](/models/artifacts/) in the Developer Guide. ### Visualize tables -Any table logged this way will show up in your Workspace on both the Run Page and the Project Page. For more information, see [Visualize and Analyze Tables](/models/tables/visualize-tables/). +Any table logged this way appears in your workspace on both the run page and the project page. For more information, see [Visualize and Analyze Tables](/models/tables/visualize-tables/). ## Artifact tables -Use `artifact.add()` to log tables to the Artifacts section of your run instead of the workspace. This could be useful if you have a dataset that you want to log once and then reference for future runs. +Use `artifact.add()` to log tables to the Artifacts section of your run instead of the workspace. This approach is useful when you have a dataset that you want to log once and then reference from future runs, so you don't re-upload the same data each time. ```python with wandb.init(project="my_project") as run: @@ -173,22 +176,22 @@ with wandb.init(project="my_project") as run: run.log_artifact(test_predictions) ``` -Refer to this Colab for a [detailed example of artifact.add() with image data](https://wandb.me/dsviz-nature-colab) and this Report for an example of how to use Artifacts and Tables to [version control and deduplicate tabular data](https://wandb.me/TBV-Dedup). +Refer to this Colab for a [detailed example of artifact.add() with image data](https://wandb.me/dsviz-nature-colab) and this report for an example of how to use artifacts and tables to [version control and deduplicate tabular data](https://wandb.me/TBV-Dedup). -### Join Artifact tables +### Join artifact tables -You can join tables you have locally constructed or tables you have retrieved from other artifacts using `wandb.JoinedTable(table_1, table_2, join_key)`. +To compare or correlate data across two tables, for example, to align ground truth with predictions or pair original and generated samples, join them with `wandb.JoinedTable(table_1, table_2, join_key)`. You can join tables you constructed locally or tables you retrieved from other artifacts. -| Args | Description | -| --------- | ------------------------------------------------------------------------------------------------------------------ | -| table_1 | (str, `wandb.Table`, ArtifactEntry) the path to a `wandb.Table` in an artifact, the table object, or ArtifactEntry | -| table_2 | (str, `wandb.Table`, ArtifactEntry) the path to a `wandb.Table` in an artifact, the table object, or ArtifactEntry | -| join_key | (str, [str, str]) key or keys on which to perform the join | +| Args | Description | +| ----------- | ------------------------------------------------------------------------------------------------------------------------ | +| `table_1` | (`str`, `wandb.Table`, `ArtifactEntry`) the path to a `wandb.Table` in an artifact, the table object, or `ArtifactEntry` | +| `table_2` | (`str`, `wandb.Table`, `ArtifactEntry`) the path to a `wandb.Table` in an artifact, the table object, or `ArtifactEntry` | +| `join_key` | (`str`, [`str`, `str`]) key or keys on which to perform the join | -To join two Tables you have logged previously in an artifact context, fetch them from the artifact and join the result into a new Table. +To join two tables you logged previously in an artifact context, fetch them from the artifact and join the result into a new table. -For example, the following code example demonstrates how to read one Table of original songs called `'original_songs'` and another Table of synthesized versions of the same songs called `'synth_songs'`. The code joins the two tables on `"song_id"`, and uploads the resulting table as a new W&B Table: +For example, the following code reads one table of original songs called `'original_songs'` and another table of synthesized versions of the same songs called `'synth_songs'`. It joins the two tables on `"song_id"` and uploads the result as a new W&B Table: ```python import wandb @@ -212,4 +215,4 @@ with wandb.init(project="my_project") as run: run.log_artifact(join_at) ``` -[Read this tutorial](https://wandb.ai/stacey/cshanty/reports/Whale2Song-W-B-Tables-for-Audio--Vmlldzo4NDI3NzM) for an example on how to combine two previously stored tables stored in different Artifact objects. +[Read this tutorial](https://wandb.ai/stacey/cshanty/reports/Whale2Song-W-B-Tables-for-Audio--Vmlldzo4NDI3NzM) for an example of combining two tables stored in different artifact objects. diff --git a/models/track/log/media.mdx b/models/track/log/media.mdx index 6d8a0d8348..e7d7281a3d 100644 --- a/models/track/log/media.mdx +++ b/models/track/log/media.mdx @@ -1,24 +1,25 @@ --- description: Log rich media, from 3D point clouds and molecules to HTML and histograms title: Log media and objects +keywords: ["wandb.Image", "wandb.Video", "wandb.Object3D", "wandb.Molecule", "segmentation masks"] --- import { ColabLink } from '/snippets/_includes/colab-link.mdx'; -We support images, video, audio, and more. Log rich media to explore your results and visually compare your runs, models, and datasets. Read on for examples and how-to guides. +This page shows you how to log rich media, including images, video, audio, 3D point clouds, molecules, histograms, HTML, and text, so you can explore your results and visually compare your runs, models, and datasets. Use these examples as a reference when you want to attach media objects to a W&B run. For details, see the [Data types reference](/models/ref/python/data-types/). -For more details, check out a [demo report about visualize model predictions](https://wandb.ai/lavanyashukla/visualize-predictions/reports/Visualize-Model-Predictions--Vmlldzo1NjM4OA) or watch a [video walkthrough](https://www.youtube.com/watch?v=96MxRvx15Ts). +For more details, see a [demo report on visualizing model predictions](https://wandb.ai/lavanyashukla/visualize-predictions/reports/Visualize-Model-Predictions--Vmlldzo1NjM4OA) or watch a [video walkthrough](https://www.youtube.com/watch?v=96MxRvx15Ts). -## Pre-requisites -In order to log media objects with the W&B SDK, you may need to install additional dependencies. -You can install these dependencies by running the following command: +## Prerequisites + +To log media objects with the W&B SDK, you may need to install additional dependencies that support the media types you plan to use. Install them by running the following command. ```bash pip install wandb[media] @@ -26,27 +27,27 @@ pip install wandb[media] ## Images -Log images to track inputs, outputs, filter weights, activations, and more. +Log images to track inputs, outputs, filter weights, and activations. Autoencoder inputs and outputs -Images can be logged directly from NumPy arrays, as PIL images, or from the filesystem. +You can log images directly from NumPy arrays, as PIL images, or from the filesystem. -Each time you log images from a step, they are available in the UI. Expand the image panel, then use the step slider to look at images from different steps. This makes it easy to compare how a model's output changes during training. Click a media panel to view an image in full-screen mode; there you can zoom and pan, including with [keyboard shortcuts](/models/app/keyboard-shortcuts#media-panels). +Each time you log images from a step, they are available in the UI. Expand the image panel, then use the step slider to look at images from different steps. This helps you compare how a model's output changes during training. Click a media panel to view an image in full-screen mode. In full-screen mode you can zoom and pan, including with [keyboard shortcuts](/models/app/keyboard-shortcuts#media-panels). To compare images or videos from different runs, steps, or indices in one view, use [Compare mode](/models/app/features/panels/media#compare-mode) in a media panel. -It's recommended to log fewer than 50 images per step to prevent logging from becoming a bottleneck during training and image loading from becoming a bottleneck when viewing results. +Log fewer than 50 images per step to prevent logging from becoming a bottleneck during training, and to prevent image loading from becoming a bottleneck when viewing results. - + Provide arrays directly when constructing images manually, such as by using [`make_grid` from `torchvision`](https://pytorch.org/vision/stable/utils.html#torchvision.utils.make_grid). -Arrays are converted to png using [Pillow](https://pillow.readthedocs.io/en/stable/index.html). +The SDK converts arrays to PNG using [Pillow](https://pillow.readthedocs.io/en/stable/index.html). ```python import wandb @@ -58,9 +59,9 @@ with wandb.init(project="image-log-example") as run: run.log({"examples": images}) ``` -We assume the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, we convert them to integers between `0` and `255`. If you want to normalize your images differently, you can specify the [`mode`](https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes) manually or just supply a [`PIL.Image`](https://pillow.readthedocs.io/en/stable/reference/Image.html), as described in the "Logging PIL Images" tab of this panel. +The SDK assumes the image is gray scale if the last dimension is 1, RGB if it's 3, and RGBA if it's 4. If the array contains floats, the SDK converts them to integers between `0` and `255`. To normalize your images differently, specify the [`mode`](https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes) manually or supply a [`PIL.Image`](https://pillow.readthedocs.io/en/stable/reference/Image.html), as described in the "Log PIL images" tab of this panel. - + For full control over the conversion of arrays to images, construct the [`PIL.Image`](https://pillow.readthedocs.io/en/stable/reference/Image.html) yourself and provide it directly. ```python @@ -78,7 +79,7 @@ with wandb.init(project="") as run: run.log({"example": wandb.Image(image, caption="My Image")}) ``` - + For even more control, create images however you like, save them to disk, and provide a filepath. ```python @@ -99,10 +100,11 @@ with wandb.init(project="") as run: ## Image overlays +You can attach overlays such as segmentation masks and bounding boxes to logged images so you can inspect predictions and ground-truth annotations directly in the W&B UI. The following tabs show how to log each overlay type. - -Log semantic segmentation masks and interact with them (altering opacity, viewing changes over time, and more) via the W&B UI. + +Log semantic segmentation masks and interact with them in the W&B UI, such as by altering opacity and viewing changes over time. Interactive mask viewing @@ -115,7 +117,7 @@ To log an overlay, provide a dictionary with the following keys and values to th * `"path"`: (string) a path to a saved image mask file * `"class_labels"`: (optional) a dictionary mapping the integer class labels in the image mask to their readable class names -To log multiple masks, log a mask dictionary with multiple keys, as in the code snippet below. +To log multiple masks, log a mask dictionary with multiple keys, as in the following code snippet. [See a live example](https://app.wandb.ai/stacey/deep-drive/reports/Image-Masks-for-Semantic-Segmentation--Vmlldzo4MTUwMw) @@ -138,11 +140,11 @@ mask_img = wandb.Image( ) ``` -Segmentation masks for a key are defined at each step (each call to `run.log()`). +Segmentation masks for a key are defined at each step (each call to `run.log()`). - If steps provide different values for the same mask key, only the most recent value for the key is applied to the image. -- If steps provide different mask keys, all values for each key are shown, but only those defined in the step being viewed are applied to the image. Toggling the visibility of masks not defined in the step do not change the image. +- If steps provide different mask keys, all values for each key are shown, but only those defined in the step being viewed are applied to the image. Toggling the visibility of masks not defined in the step doesn't change the image. - + Log bounding boxes with images, and use filters and toggles to dynamically visualize different sets of boxes in the UI. @@ -151,19 +153,19 @@ Log bounding boxes with images, and use filters and toggles to dynamically visua [See a live example](https://app.wandb.ai/stacey/yolo-drive/reports/Bounding-Boxes-for-Object-Detection--Vmlldzo4Nzg4MQ) -To log a bounding box, you'll need to provide a dictionary with the following keys and values to the boxes keyword argument of `wandb.Image`: +To log a bounding box, provide a dictionary with the following keys and values to the `boxes` keyword argument of `wandb.Image`: -* `box_data`: a list of dictionaries, one for each box. The box dictionary format is described below. - * `position`: a dictionary representing the position and size of the box in one of two formats, as described below. Boxes need not all use the same format. +* `box_data`: a list of dictionaries, one for each box. The box dictionary format is described in the following list. + * `position`: a dictionary representing the position and size of the box in one of two formats, as described in the following list. Boxes need not all use the same format. * _Option 1:_ `{"minX", "maxX", "minY", "maxY"}`. Provide a set of coordinates defining the upper and lower bounds of each box dimension. * _Option 2:_ `{"middle", "width", "height"}`. Provide a set of coordinates specifying the `middle` coordinates as `[x,y]`, and `width` and `height` as scalars. - * `class_id`: an integer representing the class identity of the box. See `class_labels` key below. - * `scores`: a dictionary of string labels and numeric values for scores. Can be used for filtering boxes in the UI. - * `domain`: specify the units/format of the box coordinates. **Set this to "pixel"** if the box coordinates are expressed in pixel space, such as integers within the bounds of the image dimensions. By default, the domain is assumed to be a fraction/percentage of the image, expressed as a floating point number between 0 and 1. - * `box_caption`: (optional) a string to be displayed as the label text on this box -* `class_labels`: (optional) A dictionary mapping `class_id`s to strings. By default we will generate class labels `class_0`, `class_1`, etc. + * `class_id`: an integer representing the class identity of the box. See the `class_labels` key in this list. + * `scores`: a dictionary of string labels and numeric values for scores. You can use it to filter boxes in the UI. + * `domain`: specify the units or format of the box coordinates. **Set this to `"pixel"`** if the box coordinates are expressed in pixel space, such as integers within the bounds of the image dimensions. By default, the domain is a fraction of the image, expressed as a floating point number between 0 and 1. + * `box_caption`: (optional) a string to display as the label text on this box. +* `class_labels`: (optional) a dictionary mapping `class_id`s to strings. By default, the SDK generates class labels such as `class_0` and `class_1`. -Check out this example: +See this example: ```python import wandb @@ -215,17 +217,19 @@ with wandb.init(project="my_project") as run: -## Image overlays in Tables +## Image overlays in tables + +To include image overlays inside a [`wandb.Table`](/models/ref/python/data-types/table), construct a `wandb.Image` for each row and pass it as a table cell. The following tabs show how to do this for segmentation masks and bounding boxes. - + Interactive Segmentation Masks in Tables -To log Segmentation Masks in tables, you will need to provide a `wandb.Image` object for each row in the table. +To log segmentation masks in tables, provide a `wandb.Image` object for each row in the table. -An example is provided in the Code snippet below: +The following code snippet shows an example. ```python table = wandb.Table(columns=["ID", "Image"]) @@ -245,14 +249,14 @@ with wandb.init(project="my_project") as run: run.log({"Table": table}) ``` - + Interactive Bounding Boxes in Tables -To log Images with Bounding Boxes in tables, you will need to provide a `wandb.Image` object for each row in the table. +To log images with bounding boxes in tables, provide a `wandb.Image` object for each row in the table. -An example is provided in the code snippet below: +The following code snippet shows an example. ```python table = wandb.Table(columns=["ID", "Image"]) @@ -288,11 +292,13 @@ for id, img, boxes in zip(ids, images, boxes_set): ## Histograms +Log histograms to track the distribution of values, such as gradients or activations, across training steps. The following tabs show basic and flexible approaches. + - -If a sequence of numbers, such as a list, array, or tensor, is provided as the first argument, we will construct the histogram automatically by calling `np.histogram()`. All arrays/tensors are flattened. You can use the optional `num_bins` keyword argument to override the default of `64` bins. The maximum number of bins supported is `512`. + +If you provide a sequence of numbers, such as a list, array, or tensor, as the first argument, the SDK constructs the histogram automatically by calling `np.histogram()`. The SDK flattens all arrays/tensors. Use the optional `num_bins` keyword argument to override the default of `64` bins. The maximum number of bins supported is `512`. -In the UI, histograms are plotted with the training step on the x-axis, the metric value on the y-axis, and the count represented by color, to ease comparison of histograms logged throughout training. See the "Histograms in Summary" tab of this panel for details on logging one-off histograms. +In the UI, histograms are plotted with the training step on the x-axis, the metric value on the y-axis, and the count represented by color, which eases comparison of histograms logged throughout training. See the "Histograms in Summary" tab of this panel for details on logging one-off histograms. ```python run.log({"gradients": wandb.Histogram(grads)}) @@ -302,8 +308,8 @@ run.log({"gradients": wandb.Histogram(grads)}) GAN discriminator gradients - -If you want more control, call `np.histogram()` and pass the returned tuple to the `np_histogram` keyword argument. + +For more control, call `np.histogram()` and pass the returned tuple to the `np_histogram` keyword argument. ```python np_hist_grads = np.histogram(grads, density=True, range=(0.0, 1.0)) @@ -314,11 +320,11 @@ run.log({"gradients": wandb.Histogram(np_hist_grads)}) -If histograms are in your summary they will appear on the Overview tab of the [Run Page](/models/runs/). If they are in your history, we plot a heatmap of bins over time on the Charts tab. +If histograms are in your summary, they appear on the Overview tab of the [run page](/models/runs/). If they are in your history, the UI plots a heatmap of bins over time on the Charts tab. ## 3D visualizations -Log 3D point clouds and Lidar scenes with bounding boxes. Pass in a NumPy array containing coordinates and colors for the points to render. +Log 3D point clouds and Lidar scenes with bounding boxes. Pass in a NumPy array containing coordinates and colors for the points to render. ```python point_cloud = np.array([[0, 0, 0, COLOR]]) @@ -330,25 +336,25 @@ run.log({"point_cloud": wandb.Object3D(point_cloud)}) The W&B UI truncates the data at 300,000 points. -#### NumPy array formats +### NumPy array formats -Three different formats of NumPy arrays are supported for flexible color schemes. +The SDK supports three different formats of NumPy arrays for flexible color schemes. * `[[x, y, z], ...]` `nx3` * `[[x, y, z, c], ...]` `nx4` `| c is a category` in the range `[1, 14]` (Useful for segmentation) * `[[x, y, z, r, g, b], ...]` `nx6 | r,g,b` are values in the range `[0,255]`for red, green, and blue color channels. -#### Python object +### Python object -Using this schema, you can define a Python object and pass it in to [the `from_point_cloud` method](/models/ref/python/#from_point_cloud). +With this schema, you can define a Python object and pass it to [the `from_point_cloud` method](/models/ref/python/#from_point_cloud). -* `points`is a NumPy array containing coordinates and colors for the points to render using [the same formats as the simple point cloud renderer shown above](#python-object). -* `boxes` is a NumPy array of python dictionaries with three attributes: - * `corners`- a list of eight corners - * `label`- a string representing the label to be rendered on the box (Optional) - * `color`- rgb values representing the color of the box - * `score` - a numeric value that will be displayed on the bounding box that can be used to filter the bounding boxes shown (for example, to only show bounding boxes where `score` > `0.75`). (Optional) -* `type` is a string representing the scene type to render. Currently the only supported value is `lidar/beta` +* `points` is a NumPy array containing coordinates and colors for the points to render using [the same formats as the simple point cloud renderer](#python-object). +* `boxes` is a NumPy array of Python dictionaries with three attributes: + * `corners` - a list of eight corners + * `label` - a string representing the label to render on the box (optional) + * `color` - RGB values representing the color of the box + * `score` - a numeric value that displays on the bounding box and that you can use to filter the bounding boxes shown (for example, to only show bounding boxes where `score` > `0.75`). (optional) +* `type` is a string representing the scene type to render. The only supported value is `lidar/beta`. ```python point_list = [ @@ -395,9 +401,9 @@ run.log({"my_first_point_cloud": wandb.Object3D.from_point_cloud( When viewing a point cloud, you can hold control and use the mouse to move around inside the space. -#### Point cloud files +### Point cloud files -You can use [the `from_file` method](/models/ref/python/#from_file) to load in a JSON file full of point cloud data. +Use [the `from_file` method](/models/ref/python/#from_file) to load a JSON file full of point cloud data. ```python run.log({"my_cloud_from_file": wandb.Object3D.from_file( @@ -405,7 +411,7 @@ run.log({"my_cloud_from_file": wandb.Object3D.from_file( )}) ``` -An example of how to format the point cloud data is shown below. +The following shows an example of how to format the point cloud data. ```json { @@ -491,9 +497,9 @@ An example of how to format the point cloud data is shown below. "type": "lidar/beta" } ``` -#### NumPy arrays +### NumPy arrays -Using [the same array formats defined above](#numpy-array-formats), you can use `numpy` arrays directly with [the `from_numpy` method](/models/ref/python/#from_numpy) to define a point cloud. +With [the same array formats](#numpy-array-formats), you can use `numpy` arrays directly with [the `from_numpy` method](/models/ref/python/#from_numpy) to define a point cloud. ```python run.log({"my_cloud_from_numpy_xyz": wandb.Object3D.from_numpy( @@ -538,12 +544,13 @@ run.log({"my_cloud_from_numpy_rgb": wandb.Object3D.from_numpy( )}) ``` +You can also log molecular data, such as proteins, by passing the path to a molecular data file to `wandb.Molecule`: ```python run.log({"protein": wandb.Molecule("6lu7.pdb")}) ``` -Log molecular data in any of 10 file types:`pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, or `mmtf.` +Log molecular data in any of 10 file types: `pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, or `mmtf`. W&B also supports logging molecular data from SMILES strings, [`rdkit`](https://www.rdkit.org/docs/index.html) `mol` files, and `rdkit.Chem.rdchem.Mol` objects. @@ -559,7 +566,7 @@ run.log( ) ``` -When your run finishes, you'll be able to interact with 3D visualizations of your molecules in the UI. +When your run finishes, you can interact with 3D visualizations of your molecules in the UI. [See a live example using AlphaFold](https://wandb.me/alphafold-workspace) @@ -579,13 +586,13 @@ run.log({"example": [wandb.Image(...) for img in images]}) ### Video -Videos are logged using the [`wandb.Video`](/models/ref/python/) data type: +Log videos with the [`wandb.Video`](/models/ref/python/) data type: ```python run.log({"example": wandb.Video("myvideo.mp4")}) ``` -Now you can view videos in the media browser. Go to your project workspace, run workspace, or report and click **Add visualization** to add a rich media panel. +You can view videos in the media browser. Go to your project workspace, run workspace, or report and click **Add visualization** to add a rich media panel. ## 2D view of a molecule @@ -603,7 +610,7 @@ run.log({"acetic_acid": wandb.Image(pil_image)}) ## Other media -W&B also supports logging of a variety of other media types. +W&B also supports logging other media types, including audio, video, text tables, and HTML. The following sections show how to log each type. ### Audio @@ -619,15 +626,15 @@ A maximum of 100 audio clips can be logged per step. For more usage information, run.log({"video": wandb.Video(numpy_array_or_path_to_video, fps=4, format="gif")}) ``` -If a numpy array is supplied we assume the dimensions are, in order: time, channels, width, height. By default we create a 4 fps gif image ([`ffmpeg`](https://www.ffmpeg.org) and the [`moviepy`](https://pypi.org/project/moviepy/) python library are required when passing numpy objects). Supported formats are `"gif"`, `"mp4"`, `"webm"`, and `"ogg"`. If you pass a string to `wandb.Video` we assert the file exists and is a supported format before uploading to wandb. Passing a `BytesIO` object will create a temporary file with the specified format as the extension. +If you supply a NumPy array, the SDK assumes the dimensions are, in order: time, channels, width, and height. By default, the SDK creates a 4 fps GIF image ([`ffmpeg`](https://www.ffmpeg.org) and the [`moviepy`](https://pypi.org/project/moviepy/) Python library are required when you pass NumPy objects). Supported formats are `"gif"`, `"mp4"`, `"webm"`, and `"ogg"`. If you pass a string to `wandb.Video`, the SDK asserts the file exists and is a supported format before uploading to W&B. If you pass a `BytesIO` object, the SDK creates a temporary file with the specified format as the extension. -On the W&B [Run](/models/runs/) and [Project](/models/track/project-page/) Pages, you will see your videos in the Media section. +Your videos appear in the Media section of the W&B [run](/models/runs/) and [project](/models/track/project-page/) pages. For more usage information, see [`video-file`](/models/ref/query-panel/video-file). ### Text -Use `wandb.Table` to log text in tables to show up in the UI. By default, the column headers are `["Input", "Output", "Expected"]`. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with `wandb.Table.MAX_ROWS = {DESIRED_MAX}`. +Use `wandb.Table` to log text in tables that appear in the UI. By default, the column headers are `["Input", "Output", "Expected"]`. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with `wandb.Table.MAX_ROWS = {DESIRED_MAX}`. ```python with wandb.init(project="my_project") as run: @@ -659,7 +666,7 @@ run.log({"custom_file": wandb.Html(open("some.html"))}) run.log({"custom_string": wandb.Html('Link')}) ``` -Log custom HTML at any key to expose an HTML panel on the run page. By default, we inject default styles; you can turn off default styles by passing `inject=False`. +Log custom HTML at any key to expose an HTML panel on the run page. By default, the SDK injects default styles. To turn off default styles, pass `inject=False`. ```python run.log({"custom_file": wandb.Html(open("some.html"), inject=False)}) diff --git a/models/track/log/plots.mdx b/models/track/log/plots.mdx index 2f0b8aac24..51ed4b50d2 100644 --- a/models/track/log/plots.mdx +++ b/models/track/log/plots.mdx @@ -1,18 +1,23 @@ --- description: Create and track plots from machine learning experiments. title: Create and track plots from experiments +keywords: ["wandb.plot", "custom charts", "matplotlib", "Plotly", "Bokeh"] --- +This page shows you how to create and log plots from your machine learning experiments so that you can visualize metrics, evaluate models, and share interactive charts in W&B. It's for users who already log runs with `wandb` and want to add custom or built-in visualizations to their experiments. + In W&B Models, methods in `wandb.plot` let you track charts with `wandb.Run.log()`, including charts that change over time during training. To learn more about the custom charting framework, see the [custom charts walkthrough](/models/app/features/custom-charts/walkthrough/). -### Basic charts +## Basic charts + +This section covers the general workflow for creating a W&B chart and provides examples of common chart types (line, scatter, bar, histogram, and multi-line plots) that you can use to visualize metrics and results. To create a W&B chart: 1. Create a `wandb.Table` object and add the data you want to visualize. -1. Generate a plot using one of the W&B's built-in [helper functions](/models/ref/python/custom-charts) -1. Log the plot with `wandb.Run.log()`. +2. Generate a plot using one of W&B's built-in [helper functions](/models/ref/python/custom-charts). +3. Log the plot with `wandb.Run.log()`. -The following basic charts can be used to construct basic visualizations of metrics and results. +Use the following basic charts to construct basic visualizations of metrics and results. @@ -33,7 +38,7 @@ with wandb.init() as run: ) ``` -You can use this to log curves on any two dimensions. If you're plotting two lists of values against each other, the number of values in the lists must match exactly. For example, each point must have an x and a y. +Use this to log curves on any two dimensions. If you plot two lists of values against each other, the number of values in the lists must match exactly. For example, each point must have an x and a y. Custom line plot @@ -55,7 +60,7 @@ with wandb.init() as run: run.log({"my_custom_id": wandb.plot.scatter(table, "class_x", "class_y")}) ``` -You can use this to log scatter points on any two dimensions. If you're plotting two lists of values against each other, the number of values in the lists must match exactly. For example, each point must have an x and a y. +Use this to log scatter points on any two dimensions. If you plot two lists of values against each other, the number of values in the lists must match exactly. For example, each point must have an x and a y. Custom scatter plot @@ -66,7 +71,7 @@ For more information, see the [Creating Custom Scatter Plots With W&B](https://w [Run the code](https://tiny.cc/custom-charts) -Log a custom bar chart (a list of labeled values as bars) natively in a few lines: +Log a custom bar chart (a list of labeled values as bars) in a few lines: ```python import wandb @@ -83,7 +88,7 @@ with wandb.init() as run: ) ``` -You can use this to log arbitrary bar charts. The number of labels and values in the lists must match exactly. Each data point must have both. +Use this to log arbitrary bar charts. The number of labels and values in the lists must match exactly. Each data point must have both. Custom bar chart @@ -94,7 +99,7 @@ For more information, see the [Custom Bar Charts](https://wandb.ai/wandb/plots/r [Run the code](https://tiny.cc/custom-charts) -Log a custom histogram (sort a list of values into bins by count or frequency of occurrence) natively in a few lines. If you have a list of prediction confidence scores (`scores`), you can visualize the distribution like this: +Log a custom histogram (sort a list of values into bins by count or frequency of occurrence) in a few lines. If you have a list of prediction confidence scores (`scores`), you can visualize the distribution like this: ```python import wandb @@ -105,7 +110,7 @@ with wandb.init() as run: run.log({"my_histogram": wandb.plot.histogram(table, "scores", title="Histogram")}) ``` -You can use this to log arbitrary histograms. Note that `data` is a list of lists, intended to support a 2D array of rows and columns. +Use this to log arbitrary histograms. `data` is a list of lists, which supports a 2D array of rows and columns. Custom histogram @@ -135,7 +140,7 @@ with wandb.init() as run: ) ``` -Note that the number of x and y points must match exactly. You can supply one list of x values to match multiple lists of y values, or a separate list of x values for each list of y values. +The number of x and y points must match exactly. You can supply one list of x values to match multiple lists of y values, or a separate list of x values for each list of y values. Multi-line plot @@ -147,9 +152,11 @@ For more information, see the [Custom Multi-Line Plots](https://wandb.ai/wandb/p -### Model evaluation charts +## Model evaluation charts + +This section describes preset charts you can use to evaluate model performance, including precision-recall curves, ROC curves, and confusion matrices. -These preset charts have built-in `wandb.plot()` methods that make it quick and easy to log charts directly from your script and see the exact information you're looking for in the UI. +These preset charts have built-in `wandb.plot()` methods that let you log charts directly from your script and see the information you want in the UI. @@ -240,7 +247,9 @@ For more information, see the [Confusion Matrix: Usage and Examples](https://wan -### Interactive custom charts +## Interactive custom charts + +This section describes how to log data to a custom chart preset so that you can fully control how the chart is rendered in the W&B UI. For full customization, tweak a built-in [Custom Chart preset](/models/app/features/custom-charts/walkthrough/) or create a new preset, then save the chart. Use the chart ID to log data to that custom preset directly from your script. @@ -269,7 +278,9 @@ with wandb.init() as run: [Run the code](https://tiny.cc/custom-charts) -### Matplotlib and Plotly plots +## Matplotlib and Plotly plots + +This section describes how to log charts that you have already generated with matplotlib or Plotly, instead of building charts with the W&B custom charts framework. Instead of using W&B [Custom Charts](/models/app/features/custom-charts/walkthrough/) with `wandb.plot()`, you can log charts generated with [matplotlib](https://matplotlib.org/) and [Plotly](https://plotly.com/). @@ -287,17 +298,19 @@ with wandb.init() as run: run.log({"chart": plt}) ``` -Just pass a `matplotlib` plot or figure object to `wandb.Run.log()`. By default we'll convert the plot into a [Plotly](https://plot.ly/) plot. If you'd rather log the plot as an image, you can pass the plot into `wandb.Image`. We also accept Plotly charts directly. +Pass a `matplotlib` plot or figure object to `wandb.Run.log()`. By default, W&B converts the plot into a [Plotly](https://plot.ly/) plot. To log the plot as an image instead, pass the plot into `wandb.Image`. W&B also accepts Plotly charts directly. If you get an error like "You attempted to log an empty plot", store the figure separately from the plot with `fig = plt.figure()` and then log `fig` in your call to `wandb.Run.log()`. -### Log custom HTML to W&B Tables +## Log custom HTML to W&B Tables + +This section describes how to embed interactive charts inside a W&B Table by logging them as HTML. Use this approach when you want to keep chart interactivity alongside other tabular data. -W&B supports logging interactive charts from Plotly and Bokeh as HTML and adding them to Tables. +W&B supports logging interactive charts from Plotly and Bokeh as HTML and adding them to Tables. The following sections describe how to log Plotly and Bokeh figures. -#### Log Plotly figures to Tables as HTML +### Log Plotly figures to Tables as HTML You can log interactive Plotly charts to W&B Tables by converting them to HTML. @@ -329,7 +342,7 @@ with wandb.init(project="log-plotly-fig-tables", name="plotly_html") as run: run.log({"test_table": table}) ``` -#### Log Bokeh figures to Tables as HTML +### Log Bokeh figures to Tables as HTML You can log interactive Bokeh charts to W&B Tables by converting them to HTML. diff --git a/models/track/log/working-with-csv.mdx b/models/track/log/working-with-csv.mdx index 44bae53e97..da2660736d 100644 --- a/models/track/log/working-with-csv.mdx +++ b/models/track/log/working-with-csv.mdx @@ -1,17 +1,18 @@ --- description: "Import CSV files into W&B as Tables and Artifacts for visualization, comparison, and analysis in dashboards." title: Track CSV files with experiments +keywords: ["CSV import", "pandas DataFrame", "wandb.Table", "dataset artifact", "experiment import"] --- -Use the W&B Python Library to log a CSV file and visualize it in a [W&B Dashboard](/models/track/workspaces/). W&B Dashboard are the central place to organize and visualize results from your machine learning models. This is particularly useful if you have a [CSV file that contains information of previous machine learning experiments](#import-and-log-your-csv-of-experiments) that are not logged in W&B or if you have [CSV file that contains a dataset](#import-and-log-your-dataset-csv-file). +Use the W&B Python Library to log a CSV file and visualize it in a [W&B Dashboard](/models/track/workspaces/). W&B Dashboards are the central place to organize and visualize results from your machine learning models. By following this page, you can bring existing CSV data into W&B so you can compare, query, and analyze it alongside other experiments. This is useful if you have a [CSV file that contains information about previous machine learning experiments](#import-and-log-your-csv-of-experiments) that you didn't log in W&B, or if you have a [CSV file that contains a dataset](#import-and-log-your-dataset-csv-file). ## Import and log your dataset CSV file {/* {% embed url="https://drive.google.com/file/d/1jBG3M4VnaMgeclRzowYZEYvFxvwb9SXF/view?usp=sharing" %} */} -We suggest you use W&B Artifacts to make the contents of the CSV file easier to re-use. +This section shows how to import a dataset stored in a CSV file, convert it into a W&B Table, and store it in an artifact for reuse across runs. Use W&B artifacts to make the contents of the CSV file easier to reuse. -1. To get started, first import your CSV file. In the following code snippet, replace the `iris.csv` filename with the name of your CSV filename: +1. Import your CSV file. In the following code snippet, replace the `iris.csv` filename with the name of your CSV file: ```python import wandb @@ -21,14 +22,14 @@ import pandas as pd new_iris_dataframe = pd.read_csv("iris.csv") ``` -2. Convert the CSV file to a W&B Table to utilize [W&B Dashboards](/models/track/workspaces/). +2. Convert the CSV file to a W&B Table to use with [W&B Dashboards](/models/track/workspaces/): ```python # Convert the DataFrame into a W&B Table iris_table = wandb.Table(dataframe=new_iris_dataframe) ``` -3. Next, create a W&B Artifact and add the table to the Artifact: +3. Create a W&B artifact and add the table to the artifact: ```python # Add the table to an Artifact to increase the row @@ -39,9 +40,9 @@ iris_table_artifact.add(iris_table, "iris_table") # Log the raw csv file within an artifact to preserve our data iris_table_artifact.add_file("iris.csv") ``` -For more information about W&B Artifacts, see the [Artifacts chapter](/models/artifacts/). +For more information about W&B artifacts, see [Artifacts](/models/artifacts/). -4. Lastly, start a new W&B Run to track and log to W&B with `wandb.init()`: +4. Start a new W&B run to track and log to W&B with `wandb.init()`: ```python # Start a W&B run to log data @@ -54,14 +55,14 @@ with wandb.init(project="tables-walkthrough") as run: run.log_artifact(iris_table_artifact) ``` -The `wandb.init()` API spawns a new background process to log data to a Run, and it synchronizes data to wandb.ai (by default). View live visualizations on your W&B Workspace Dashboard. The following image demonstrates the output of the code snippet demonstration. +The `wandb.init()` API spawns a new background process to log data to a run, and it synchronizes data to `wandb.ai` by default. View live visualizations on your W&B Workspace Dashboard. The following image shows the output of the preceding code snippet. - CSV file imported into W&B Dashboard + CSV file imported into W&B Dashboard -The full script with the preceding code snippets is found below: +The full script with the preceding code snippets follows: ```python import wandb @@ -78,7 +79,7 @@ iris_table = wandb.Table(dataframe=new_iris_dataframe) iris_table_artifact = wandb.Artifact("iris_artifact", type="dataset") iris_table_artifact.add(iris_table, "iris_table") -# log the raw csv file within an artifact to preserve our data +# Log the raw csv file within an artifact to preserve our data iris_table_artifact.add_file("iris.csv") # Start a W&B run to log data @@ -91,18 +92,18 @@ with wandb.init(project="tables-walkthrough") as run: run.log_artifact(iris_table_artifact) ``` -## Import and log your CSV of Experiments +## Import and log your CSV of experiments {/* {% embed url="https://drive.google.com/file/d/1PL4RSdopHEptDR5Gi0DEzECXuoW_5B0f/view?usp=sharing" %} The below table becomes this W&B Dashboard after conversion {% endembed %} */} -In some cases, you might have your experiment details in a CSV file. Common details found in such CSV files include: +This section shows how to take a CSV file that records past experiments and convert each row into a W&B run so you can visualize and compare them in a Dashboard. Sometimes, you might have your experiment details in a CSV file. Common details in such CSV files include: -* A name for the experiment run -* Initial [notes](/models/runs/#add-a-note-to-a-run) -* [Tags](/models/runs/tags/) to differentiate the experiments -* Configurations needed for your experiment (with the added benefit of being able to utilize our [Sweeps Hyperparameter Tuning](/models/sweeps/)). +* A name for the experiment run. +* Initial [notes](/models/runs/#add-a-note-to-a-run). +* [Tags](/models/runs/tags/) to differentiate the experiments. +* Configurations needed for your experiment (with the added benefit of being able to use our [Sweeps Hyperparameter Tuning](/models/sweeps/)). | Experiment | Model Name | Notes | Tags | Num Layers | Final Train Acc | Final Val Acc | Training Losses | | ------------ | ---------------- | ------------------------------------------------ | ------------- | ---------- | --------------- | ------------- | ------------------------------------- | @@ -112,9 +113,9 @@ In some cases, you might have your experiment details in a CSV file. Common deta | ... | ... | ... | ... | ... | ... | ... | | | Experiment N | mnist-X-layers | NOTES | ... | ... | ... | ... | \[..., ...] | -W&B can take CSV files of experiments and convert it into a W&B Experiment Run. The following code snippets and code script demonstrates how to import and log your CSV file of experiments: +W&B can take CSV files of experiments and convert each row into a W&B experiment run. The following code snippets and code script show how to import and log your CSV file of experiments: -1. To get started, first read in your CSV file and convert it into a Pandas DataFrame. Replace `"experiments.csv"` with the name of your CSV file: +1. Read in your CSV file and convert it into a Pandas DataFrame. Replace `"experiments.csv"` with the name of your CSV file: ```python import wandb @@ -152,7 +153,7 @@ for i, row in loaded_experiment_df.iterrows(): ``` -2. Next, start a new W&B Run to track and log to W&B with [`wandb.init()`](/models/ref/python/functions/init): +2. Start a new W&B run to track and log to W&B with [`wandb.init()`](/models/ref/python/functions/init): ```python with wandb.init( @@ -160,21 +161,21 @@ for i, row in loaded_experiment_df.iterrows(): ) as run: ``` -As an experiment runs, you might want to log every instance of your metrics so they are available to view, query, and analyze with W&B. Use the [`run.log()`](/models/ref/python/experiments/run/#method-runlog) command to accomplish this: +As an experiment runs, you might want to log every instance of your metrics so they're available to view, query, and analyze with W&B. Use the [`run.log()`](/models/ref/python/experiments/run/#method-runlog) command: ```python run.log({key: val}) ``` -You can optionally log a final summary metric to define the outcome of the run using the [`define_metric`](/models/ref/python/experiments/run#define_metric) API. This example adds the summary metrics to our run with `run.summary.update()`: +You can optionally log a final summary metric to define the outcome of the run using the [`define_metric`](/models/ref/python/experiments/run#define_metric) API. This example adds the summary metrics to the run with `run.summary.update()`: ```python run.summary.update(summaries) ``` -For more information about summary metrics, see [Log Summary Metrics](./log-summary). +For more information about summary metrics, see [Log summary metrics](./log-summary). -Below is the full example script that converts the above sample table into a [W&B Dashboard](/models/track/workspaces/): +The following full example script converts the preceding sample table into a [W&B Dashboard](/models/track/workspaces/): ```python FILENAME = "experiments.csv" @@ -206,7 +207,7 @@ for i, row in loaded_experiment_df.iterrows(): for summary_col in SUMMARY_COLS: summaries[summary_col] = row[summary_col] - with wandb.init( + with wandb.init( project=PROJECT_NAME, name=run_name, tags=tags, notes=notes, config=config ) as run: diff --git a/models/track/project-page.mdx b/models/track/project-page.mdx index 9063c688f1..d9004723e1 100644 --- a/models/track/project-page.mdx +++ b/models/track/project-page.mdx @@ -2,12 +2,13 @@ description: Compare versions of your model, explore results in a scratch workspace, and export findings to a report to save notes and visualizations title: Projects +keywords: ["workspace", "runs table", "artifacts tab", "starred projects", "project visibility"] --- import ProjectVisibilitySettings from "/snippets/_includes/project-visibility-settings.mdx"; -A *project* is a central location where you visualize results, compare experiments, view and download artifacts, create an automation, and more. +A *project* is a central location where you visualize results, compare experiments, view and download artifacts, create an automation, and more. Use projects to organize related runs so that you and your team can analyze, share, and manage your work in one place. Each project has a visibility setting that determines who can access it. For more information about who can access a project, see [Project visibility](/platform/hosting/iam/access-management/restricted-projects). @@ -15,18 +16,22 @@ Each project has a visibility setting that determines who can access it. For mor Each project contains the following tabs: -* [Overview](/models/track/project-page/#overview-tab): snapshot of your project -* [Workspace](/models/track/project-page/#workspace-tab): personal visualization sandbox -* [Runs](#runs-tab): A table that lists all the runs in your project -* [Automations](#automations-tab): Automations configured in your project -* [Sweeps](/models/sweeps): automated exploration and optimization -* [Reports](/models/track/project-page/#reports-tab): saved snapshots of notes, runs, and graphs -* [Artifacts](#artifacts-tab): Contains all runs and the artifacts associated with that run +* [Overview](/models/track/project-page/#overview-tab): Snapshot of your project. +* [Workspace](/models/track/project-page/#workspace-tab): Personal visualization sandbox. +* [Runs](#runs-tab): Table that lists all the runs in your project. +* [Automations](#automations-tab): Automations configured in your project. +* [Sweeps](/models/sweeps): Automated exploration and optimization. +* [Reports](/models/track/project-page/#reports-tab): Saved snapshots of notes, runs, and graphs. +* [Artifacts](#artifacts-tab): All runs and the artifacts associated with each run. ## Overview tab + +The **Overview** tab provides a snapshot of the project and gives you access to project-level settings. + W&B creates a project for you when you initialize a run with the name you provide for the project field. -From the **Overview** tab, you can find the project name and manage the project. +From the **Overview** tab, you can find the project name and manage the project: + - To change the project's name, description, or team, click **Edit** in the upper right corner. - To undelete recently deleted runs, click the **action ()** menu in the upper right corner, then click **Undelete recently deleted runs**. - To delete the project, click the **action ()** menu in the upper right corner, then click **Delete project**. Read the confirmation dialog and follow the instructions. Deleting a project is not reversible. @@ -34,15 +39,16 @@ From the **Overview** tab, you can find the project name and manage the project. The rest of the **Overview** page is divided into **Details** and **Project roles** tabs. ### Details + Details about the project include: -* **Project visibility**: The visibility of the project. The visibility setting that determines who can access it. See [Project visibility](/platform/hosting/iam/access-management/restricted-projects) for more information. -* **Last active**: Timestamp of the last time data is logged to this project -* **Contributors**: The number of users that contribute to this project -* **Total runs**: The total number of runs in this project -* **Total compute**: we add up all the run times in your project to get this total +* **Project visibility**: The visibility setting that determines who can access the project. For more information, see [Project visibility](/platform/hosting/iam/access-management/restricted-projects). +* **Last active**: Timestamp of the last time data was logged to this project. +* **Contributors**: The number of users who contribute to this project. +* **Total runs**: The total number of runs in this project. +* **Total compute**: The sum of all the run times in your project. -The **Details** tab also includes instructions for accessing and exporting the project's data and metrics, downloading the best model from a sweep, and more. +The **Details** tab also includes instructions for accessing and exporting the project's data and metrics, downloading the best model from a sweep, and more. [View a live example project](https://wandb.ai/example-team/sweep-demo/overview) @@ -55,29 +61,30 @@ To change a project's privacy from the **Overview** tab: ### Project roles + The **Project roles** tab is visible only to the project owner and those with the **Admin** role. List and search for users with access to the project or change a member's role. ## Workspace tab -A project's *workspace* gives you a personal sandbox to compare experiments. Use projects to organize models that can be compared, working on the same problem with different architectures, hyperparameters, datasets, preprocessing etc. +A project's *workspace* gives you a personal sandbox to compare experiments. Use the workspace to organize models that can be compared, working on the same problem with different architectures, hyperparameters, datasets, preprocessing, and so on. -**Runs sidebar**: list of all the runs in your project. +**Runs sidebar**: List of all the runs in your project. -* **action ()** menu: Hover over a row in the sidebar to see the menu appear on the left side. Use this menu to rename a run, delete a run, or stop and active run. -* **Visibility icon**: click the eye to turn on and off runs on graphs -* **Color**: change the run color to another one of our presets or a custom color -* **Search**: search runs by name. This also filters visible runs in the plots. -* **Filter**: use the sidebar filter to narrow down the set of runs visible -* **Group**: select a config column to dynamically group your runs, for example by architecture. Grouping makes plots show up with a line along the mean value, and a shaded region for the variance of points on the graph. -* **Sort**: pick a value to sort your runs by, for example runs with the lowest loss or highest accuracy. Sorting will affect which runs show up on the graphs. -* **Expand button**: expand the sidebar into the full table -* **Minimize**: press **Cmd+.** (macOS) or **Ctrl+.** (Windows/Linux) to collapse or restore the Runs selector. See [Keyboard shortcuts](/models/app/keyboard-shortcuts) for details. -* **Run count**: the number in parentheses at the top is the total number of runs in the project. The number (N visualized) is the number of runs that have the eye turned on and are available to be visualized in each plot. In the example below, the graphs are only showing the first 10 of 183 runs. Edit a graph to increase the max number of runs visible. +* **Action ()** menu: Hover over a row in the sidebar to see the menu appear on the left side. Use this menu to rename a run, delete a run, or stop an active run. +* **Visibility icon**: Click the eye to turn on and off runs on graphs. +* **Color**: Change the run color to another preset or a custom color. +* **Search**: Search runs by name. This also filters visible runs in the plots. +* **Filter**: Use the sidebar filter to narrow down the set of runs visible. +* **Group**: Select a config column to dynamically group your runs, for example by architecture. Grouping makes plots show up with a line along the mean value, and a shaded region for the variance of points on the graph. +* **Sort**: Pick a value to sort your runs by, for example runs with the lowest loss or highest accuracy. Sorting affects which runs show up on the graphs. +* **Expand button**: Expand the sidebar into the full table. +* **Minimize**: Press **Cmd+.** (macOS) or **Ctrl+.** (Windows/Linux) to collapse or restore the Runs selector. See [Keyboard shortcuts](/models/app/keyboard-shortcuts) for details. +* **Run count**: The number in parentheses at the top is the total number of runs in the project. The number (N visualized) is the number of runs that have the eye turned on and are available to be visualized in each plot. In the following example, the graphs are only showing the first 10 of 183 runs. Edit a graph to increase the max number of runs visible. If you pin, hide, or change the order of columns in the [Runs tab](#runs-tab), the Runs sidebar reflects these customizations. -**Panels layout**: use this scratch space to explore results, add and remove charts, and compare versions of your models based on different metrics +**Panels layout**: Use this scratch space to explore results, add and remove charts, and compare versions of your models based on different metrics. [View a live example](https://wandb.ai/example-team/sweep-demo) @@ -95,7 +102,7 @@ Each section has options in the upper right corner: * **Add section**: Add a section above or below from the dropdown menu, or click the button at the bottom of the page to add a new section. * **Rename section**: Change the title for your section. * **Export section to report**: Save this section of panels to a new report. -* **Delete section**: Remove the whole section and all the charts. This can be undone with the undo button at the bottom of the page in the workspace bar. +* **Delete section**: Remove the whole section and all the charts. To undo this, click the undo button at the bottom of the page in the workspace bar. * **Add panel**: Click the plus button to add a panel to the section. ### Move panels between sections @@ -108,8 +115,8 @@ Drag and drop panels to reorder and organize into sections. You can also click t ### Resize panels -All panels maintain the same size, and there are pages of panels. -Resize the section by clicking and dragging the lower right corner of the section, which will display a corner icon when hovering. +All panels maintain the same size, and there are pages of panels. +Resize the section by clicking and dragging the lower right corner of the section, which displays a corner icon when you hover. Resizing panels @@ -125,7 +132,7 @@ Use the search box in the workspace to filter down the panels. This search match ## Runs tab {/* Keep this in sync with /guide/models/track/runs/_index.md */} -Use the Runs tab to filter, group, and sort your runs. +Use the Runs tab to filter, group, and sort your runs so you can find and compare the runs you care about. Runs table @@ -135,16 +142,18 @@ The following tabs demonstrate some common actions you can take in the Runs tab. -The Runs tab shows details about runs in the project. It shows a large number of columns by default. +The Runs tab shows details about runs in the project. It shows many columns by default. When you customize the Runs tab, the customization is also reflected in the **Runs** selector of the [Workspace tab](#workspace-tab). +You can customize the columns as follows: + - To view all visible columns, scroll the page horizontally. - To change the order of the columns, drag a column to the left or right. - To pin a column, hover over the column name, click the **action ()** menu that appears, then click **Pin column**. Pinned columns appear near the left of the page, after the **Name** column. To unpin a pinned column, choose **Unpin column**. -- To hide a column, hover over the column name, click the **action ()** menu that appears, then click **Hide column**. To view all columns that are currently hidden, click **Columns**. +- To hide a column, hover over the column name, click the **action ()** menu that appears, then click **Hide column**. To view all columns that are hidden, click **Columns**. - To show, hide, pin, and unpin multiple columns at once, click **Columns**. - Click the name of a hidden column to unhide it. - Click the name of a visible column to hide it. @@ -153,17 +162,17 @@ When you customize the Runs tab, the customization is also reflected in the **Ru Sort all rows in a Table by the value in a given column using one of these options: -1. Click the **action ()** menu when hovering over any column header to quickly sort the table by that column's values. -1. Click the **Sort** button and select the desired columns from the drop-down. You can use this panel to sort by multiple columns in priority order. +- Click the **action ()** menu when hovering over any column header to sort the table by that column's values. +- Click the **Sort** button and select the desired columns from the drop-down. You can use this panel to sort by multiple columns in priority order. Configuring a run table sort -The preceding image demonstrates how to view sorting options for a Table column called `val_acc`. +The sort image demonstrates how to view sorting options for a Table column called `val_acc`. - + Apply filter expressions to narrow down the runs you want to visualize. @@ -183,6 +192,7 @@ Click the **Group** button above the table to group all rows by the value in a p {/* ## Automations tab */} ## Automations tab + Automate downstream actions for versioning artifacts. To create an automation, define trigger events and resulting actions. Actions include executing a webhook or launching a W&B job. For more information, see [Automations](/models/automations). ## Sweeps tab @@ -195,7 +205,7 @@ Start a new [sweep](/models/sweeps) from your project. ## Reports tab -See all the snapshots of results in one place, and share findings with your team. +The **Reports** tab lets you see all the snapshots of results in one place, and share findings with your team. Reports tab @@ -205,7 +215,7 @@ To view a report, select the **Reports** tab from the project sidebar, then sele ## Artifacts tab -View all [artifacts](/models/artifacts) associated with a project, from training datasets and [fine-tuned models](/models/registry) to [tables of metrics and media](/models/tables/tables-walkthrough). +View all [artifacts](/models/artifacts) associated with a project, from training datasets and [fine-tuned models](/models/registry) to [tables of metrics and media](/models/tables/tables-walkthrough). The following sections describe each panel and tab available for an artifact. ### Metadata panel @@ -214,7 +224,7 @@ View all [artifacts](/models/artifacts) associated with a project, from training Artifact metadata panel -The metadata panel provides access to the artifact's metadata. This metadata might include configuration arguments required to reconstruct the artifact, URLs where more information can be found, or metrics produced during the run which logged the artifact. +The metadata panel provides access to the artifact's metadata. This metadata might include configuration arguments required to reconstruct the artifact, URLs where more information can be found, or metrics produced during the run that logged the artifact. View the configuration for the run that produced the artifact as well as history metrics. To view an artifact's metadata: @@ -229,7 +239,7 @@ To view an artifact's metadata: Artifact usage panel -The usage panel provides a code snippet for downloading the artifact for use outside of the web app, for example on a local machine. This section also indicates and links to the run which output the artifact and any runs which use the artifact as an input. +The usage panel provides a code snippet for downloading the artifact for use outside of the web app, for example on a local machine. This section also indicates and links to the run that output the artifact and any runs that use the artifact as an input. To view the artifact usage: @@ -243,7 +253,7 @@ To view the artifact usage: Artifact files panel -The files panel lists the files and folders associated with the artifact. W&B uploads certain files for a run automatically. For example, `requirements.txt` shows the versions of each library the run used, and `wandb-metadata.json`, and `wandb-summary.json` include information about the run. Other files may be uploaded, such as artifacts or media, depending on the run's configuration. +The files panel lists the files and folders associated with the artifact. W&B uploads certain files for a run automatically. For example, `requirements.txt` shows the versions of each library the run used, and `wandb-metadata.json` and `wandb-summary.json` include information about the run. Other files may be uploaded, such as artifacts or media, depending on the run's configuration. To view files logged to an artifact: @@ -269,42 +279,47 @@ Click the Explode toggle to view all of the individual artifact versions and the Artifact versions tab -The versions tab shows all versions of the artifact. Select an artifact to view the details for that specific version. +The versions tab shows all versions of the artifact. Select an artifact to view the details for that specific version. + +To view all versions of an artifact: 1. Select the **Artifacts** tab from the project sidebar. 2. Select an artifact from the list to view the details page for the latest version of that artifact. 3. Select the **Versions** tab to view all versions of that artifact. -4. From the dropdown (next to the artifact name), select **All Versions**. +4. From the dropdown (next to the artifact name), select **All Versions**. -For example, the previous image shows different model artifact versions for a model artifact named `"zoo-wyhak4p0"`. +For example, the **Versions tab** image shows different model artifact versions for a model artifact named `"zoo-wyhak4p0"`. ## Create a project -You can create a project in the W&B App or programmatically by specifying a project in a call to `wandb.init()`. + +Create a project to start organizing your runs, artifacts, and reports. You can create a project in the W&B App or programmatically by specifying a project in a call to `wandb.init()`. In the W&B App, you can create a project from the **Projects** page or from a team's landing page. From the **Projects** page: + 1. Click the global navigation icon in the upper left. The project sidebar opens. 1. In the **Projects** section of the navigation, click **View all** to open the project overview page. 1. Click **Create new project**. -1. Set **Team** to the name of the team that will own the project. -1. Specify a name for your project using the **Name** field. +1. Set **Team** to the name of the team that owns the project. +1. Specify a name for your project using the **Name** field. 1. Set **Project visibility**, which defaults to **Team**. -1. Optionally, provide a **Description**. +1. Optional: Provide a **Description**. 1. Click **Create project**. From a team's landing page: + 1. Click the global navigation icon in the upper left. The project sidebar opens. 1. In the **Teams** section of the navigation, click the name of a team to open its landing page. 1. In the landing page, click **Create new project**. -1. **Team** is automatically set to the team that owns the landing page you were viewing. If necessary, change the team. -1. Specify a name for your project using the **Name** field. +1. **Team** is automatically set to the team that owns the landing page you're viewing. If necessary, change the team. +1. Specify a name for your project using the **Name** field. 1. Set **Project visibility**, which defaults to **Team**. -1. Optionally, provide a **Description**. +1. Optional: Provide a **Description**. 1. Click **Create project**. @@ -324,19 +339,19 @@ In the W&B App, the organization home page summarizes recent work. The **Recent Activity** viewer shows recent runs. Choose **Your projects** to see your own runs, or **Organization** to see recent runs by your colleagues. -* **Search** to filter the activity list. -* **Sortable columns** let you reorder rows by the column you care about. -* **Notes** and **tags** columns display automatically if a run includes these details. +* Use **Search** to filter the activity list. +* Use **Sortable columns** to reorder rows by the column you care about. +* The **Notes** and **tags** columns display automatically if a run includes these details. -Below the **Recent Activity** viewer, quickly access your recently created reports. +Below the **Recent Activity** viewer, access your recently created reports. ## Star a project -Add a star to a project to mark that project as important. +Add a star to a project to mark that project as important so you and your team can find it. -Starred projects are grouped at the top of the **Projects** page. Quickly access up to 10 recent projects in the **Projects** list in the top-left **Global navigation menu**, with starred projects listed first. +Starred projects are grouped at the top of the **Projects** page. Access up to 10 recent projects in the **Projects** list in the top-left **Global navigation menu**, with starred projects listed first. -There are two ways to mark a project as important: within a project's overview tab or within your team's profile page. +You can mark a project as important in two ways: from a project's overview tab or from your team's profile page. @@ -351,7 +366,7 @@ There are two ways to mark a project as important: within a project's overview t 1. Navigate to your team's profile page at `https://wandb.ai//projects`. 2. Select the **Projects** tab. -3. Hover your mouse next to the project you want to star. Click on star icon that appears. +3. Hover your mouse next to the project you want to star. Click the star icon that appears. For example, the following image shows the star icon next to the "Compare_Zoo_Models" project. @@ -360,19 +375,15 @@ For example, the following image shows the star icon next to the "Compare_Zoo_Mo - - - - -Confirm that your project is available.. In the **Project** section of the left navigation, click **View all** to open the **Project** page. Search or filter to find your project by its name, team, or other metadata. Starred projects appear at the top of the**Projects** page, and at the top of the **Projects** section of the left navigation. +Confirm that your project is available. In the **Project** section of the left navigation, click **View all** to open the **Project** page. Search or filter to find your project by its name, team, or other metadata. Starred projects appear at the top of the **Projects** page, and at the top of the **Projects** section of the left navigation. ## Delete a project You can delete your project using the **action ()** menu on the right of the overview tab. -1. Navigate to your W&B project +1. Navigate to your W&B project. 2. Select the **Overview** tab from the project sidebar. -3. Click the **action ()** menu in the upper right hand corner. +3. Click the **action ()** menu in the upper right corner. 4. Select **Delete project** from the dropdown menu. @@ -386,17 +397,17 @@ W&B does not terminate active [sweeps](/models/sweeps) or agents when you delete ## Add notes to a project -Add notes to your project either as a description overview or as a markdown panel within your workspace. +Add notes to your project to document context, decisions, or observations. You can add notes either as a description overview or as a markdown panel within your workspace. ### Add description overview to a project Descriptions you add to your page appear in the **Overview** tab of your profile. -1. Navigate to your W&B project -2. Select the **Overview** tab from the project sidebar -3. Click **Edit** in the upper right hand corner -4. Add your notes in the **Description** field -5. Select the **Save** button +1. Navigate to your W&B project. +2. Select the **Overview** tab from the project sidebar. +3. Click **Edit** in the upper right corner. +4. Add your notes in the **Description** field. +5. Select the **Save** button. **Create reports to create descriptive notes comparing runs** @@ -407,9 +418,9 @@ You can also create a W&B Report to add plots and markdown side by side. Use dif ### Add notes to run workspace -1. Navigate to your W&B project -2. Select the **Workspace** tab from the project sidebar -3. Click the **Add panels** button from the top right corner -4. Select the **TEXT AND CODE** dropdown from the modal that appears -5. Select **Markdown** -6. Add your notes in the markdown panel that appears in your workspace +1. Navigate to your W&B project. +2. Select the **Workspace** tab from the project sidebar. +3. Click the **Add panels** button from the top right corner. +4. Select the **TEXT AND CODE** dropdown from the modal that appears. +5. Select **Markdown**. +6. Add your notes in the markdown panel that appears in your workspace. diff --git a/models/track/public-api-guide.mdx b/models/track/public-api-guide.mdx index 2022c24c5b..0f0407e6c2 100644 --- a/models/track/public-api-guide.mdx +++ b/models/track/public-api-guide.mdx @@ -2,29 +2,30 @@ description: Import data from MLFlow, export or update data that you have saved to W&B title: Import and export data +keywords: [public API, MLFlow import, run export, run history, sweeps] --- import ApiKeyCreate from "/snippets/_includes/api-key-create.mdx"; import ApiKeySecurity from "/snippets/_includes/api-key-security.mdx"; -Export data or import data with W&B Public APIs. +Use the W&B Public APIs to move experiment data into and out of W&B. This guide shows you how to import runs from MLFlow, export run data for custom analysis, and update data on runs that you have already logged to W&B. -This feature requires python>=3.8 +This feature requires `python>=3.8`. ## Import data from MLFlow -W&B supports importing data from MLFlow, including experiments, runs, artifacts, metrics, and other metadata. +Migrate your existing MLFlow tracking data to W&B so that you can continue analyzing past experiments alongside new ones. W&B supports importing data from MLFlow, including experiments, runs, artifacts, metrics, and other metadata. -Install dependencies: +Install the importer dependencies, which provide the MLFlow integration: ```shell # note: this requires py38+ pip install wandb[importers] ``` -Log in to W&B. Follow the prompts if you have not logged in before. +Log in to W&B so that the importer can authenticate to your account. Follow the prompts if you haven't logged in before. ```shell wandb login @@ -41,7 +42,7 @@ runs = importer.collect_runs() importer.import_runs(runs) ``` -By default, `importer.collect_runs()` collects all runs from the MLFlow server. If you prefer to upload a special subset, you can construct your own runs iterable and pass it to the importer. +By default, `importer.collect_runs()` collects all runs from the MLFlow server. To upload a specific subset instead, construct your own runs iterable and pass it to the importer: ```py import mlflow @@ -57,18 +58,18 @@ importer.import_runs(runs) ``` -You might need to [configure the Databricks CLI first](https://docs.databricks.com/dev-tools/cli/index.html) if you import from Databricks MLFlow. +If you import from Databricks MLFlow, you might need to [configure the Databricks CLI first](https://docs.databricks.com/dev-tools/cli/index.html). Set `mlflow-tracking-uri="databricks"` in the previous step. -To skip importing artifacts, you can pass `artifacts=False`: +To skip importing artifacts, pass `artifacts=False`: ```py importer.import_runs(runs, artifacts=False) ``` -To import to a specific W&B entity and project, you can pass a `Namespace`: +To import to a specific W&B entity and project, pass a `Namespace`: ```py from wandb.apis.importers import Namespace @@ -150,18 +151,18 @@ importer.import_all( ## Export data -Use the Public API to export or update data that you have saved to W&B. Before using this API, log data from your script. Check the [Quickstart](/models/quickstart/) for more details. +Use the Public API to export or update data that you have saved to W&B. Before using this API, log data from your script. For more details, see the [Quickstart](/models/quickstart/). -**Use Cases for the Public API** +Common use cases for the Public API include: -- **Export Data**: Pull down a dataframe for custom analysis in a Jupyter Notebook. Once you have explored the data, you can sync your findings by creating a new analysis run and logging results, for example: `wandb.init(job_type="analysis")` -- **Update Existing Runs**: You can update the data logged in association with a W&B run. For example, you might want to update the config of a set of runs to include additional information, like the architecture or a hyperparameter that wasn't originally logged. +- **Export data**: Download a dataframe for custom analysis in a Jupyter Notebook. After you have explored the data, you can sync your findings by creating a new analysis run and logging results, for example: `wandb.init(job_type="analysis")`. +- **Update existing runs**: Update the data logged in association with a W&B run. For example, you might want to update the config of a set of runs to include additional information, like the architecture or a hyperparameter that wasn't originally logged. See the [Generated Reference Docs](/models/ref/python/public-api/) for details on available functions. ### Create an API key -An API key authenticates your machine to W&B. +The Public API requires an API key to authenticate requests from your machine to W&B. @@ -170,12 +171,12 @@ An API key authenticates your machine to W&B. ### Find the run path -To use the Public API, you'll often need the run path which is `//`. In the app UI, open a run page and click the [Overview tab ](/models/track/public-api-guide/#overview-tab)to get the run path. +Most Public API calls identify a run by its run path, which has the form `//`. To find the run path, open a run page in the app UI and click the [Overview tab](/models/track/public-api-guide/#overview-tab). ### Export run data -Download data from a finished or active run. Common usage includes downloading a dataframe for custom analysis in a Jupyter notebook, or using custom logic in an automated environment. +Download data from a finished or active run so that you can analyze it outside of W&B. Common usage includes downloading a dataframe for custom analysis in a Jupyter notebook, or using custom logic in an automated environment. ```python import wandb @@ -184,7 +185,7 @@ api = wandb.Api() run = api.run("//") ``` -The most commonly used attributes of a run object are: +The most-used attributes of a run object are: | Attribute | Meaning | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -192,9 +193,9 @@ The most commonly used attributes of a run object are: | `run.history()` | A list of dictionaries meant to store values that change while the model is training such as loss. The command `run.log()` appends to this object. | | `run.summary` | A dictionary of information that summarizes the run's results. This can be scalars like accuracy and loss, or large files. By default, `run.log()` sets the summary to the final value of a logged time series. The contents of the summary can also be set directly. Think of the summary as the run's outputs. | -You can also modify or update the data of past runs. By default a single instance of an api object will cache all network requests. If your use case requires real time information in a running script, call `api.flush()` to get updated values. +You can also modify or update the data of past runs. By default, a single instance of an api object caches all network requests. If your use case requires real-time information in a running script, call `api.flush()` to get updated values. -### Understanding different run attributes +### Run attributes The following code snippet shows how to create a run, log some data, and then access the run's attributes: @@ -212,9 +213,9 @@ with wandb.init(project="public-api-example") as run: ) ``` -The following sections describe the different outputs for the above run object attributes +The following sections show the outputs for the run object attributes accessed in the previous example. -##### `run.config` +#### `run.config` ```python {"n_epochs": 5} @@ -232,15 +233,17 @@ The following sections describe the different outputs for the above run object a } ``` -### Sampling +### Sample size + +The default history method samples the metrics to a fixed number of samples (the default is 500, which you can change with the `samples` argument). To export all of the data on a large run, use the `run.scan_history()` method. For more details, see the [API Reference](/models/ref/python/public-api). -The default history method samples the metrics to a fixed number of samples (the default is 500, you can change this with the `samples` __ argument). If you want to export all of the data on a large run, you can use the `run.scan_history()` method. For more details see the [API Reference](/models/ref/python/public-api). +### Query multiple runs -### Querying multiple runs +Use the following examples to fetch and filter multiple runs at once, which is useful when you want to compare runs in a project or aggregate results across an experiment. -This example script finds a project and outputs a CSV of runs with name, configs and summary stats. Replace `` and `` with your W&B entity and the name of your project, respectively. +This example script finds a project and outputs a CSV of runs with name, configs, and summary stats. Replace `` and `` with your W&B entity and the name of your project, respectively. ```python import pandas as pd @@ -274,7 +277,7 @@ run.finish() ``` -The W&B API also provides a way for you to query across runs in a project with api.runs(). The most common use case is exporting runs data for custom analysis. The query interface is the same as the one [MongoDB uses](https://www.mongodb.com/docs/manual/reference/mql/query-predicates/). +The W&B API also lets you query across runs in a project with `api.runs()`. A common use case is exporting runs data for custom analysis. The query interface is the same as the one [MongoDB uses](https://www.mongodb.com/docs/manual/reference/mql/query-predicates/). ```python runs = api.runs( @@ -287,37 +290,39 @@ print(f"Found {len(runs)} runs") -Calling `api.runs` returns a `Runs` object that is iterable and acts like a list. By default the object loads 50 runs at a time in sequence as required, but you can change the number loaded per page with the `per_page` keyword argument. +Calling `api.runs` returns a `Runs` object that is iterable and acts like a list. By default, the object loads 50 runs at a time in sequence as required. To change the number loaded per page, use the `per_page` keyword argument. -`api.runs` also accepts an `order` keyword argument. The default order is `-created_at`. To order results ascending, specify `+created_at`. You can also sort by config or summary values. For example, `summary.val_acc` or `config.experiment_name`. +`api.runs` also accepts an `order` keyword argument. The default order is `-created_at`. To order results in ascending order, specify `+created_at`. You can also sort by config or summary values. For example, `summary.val_acc` or `config.experiment_name`. ### Error handling -If errors occur while talking to W&B servers a `wandb.CommError` will be raised. The original exception can be introspected via the `exc` attribute. +If errors occur while communicating with W&B servers, a `wandb.CommError` is raised. To introspect the original exception, use the `exc` attribute. ### Get the latest git commit through the API -In the UI, click on a run and then click the Overview tab on the run page to see the latest git commit. It's also in the file `wandb-metadata.json` . Using the public API, you can get the git hash with `run.commit`. +To see the latest git commit in the UI, click a run and then click the **Overview** tab on the run page. The commit is also in the file `wandb-metadata.json`. Using the public API, get the git hash with `run.commit`. ### Get a run's name and ID during a run -After calling `wandb.init()` you can access the random run ID or the human readable run name from your script like this: +After calling `wandb.init()`, you can access the random run ID or the human-readable run name from your script as follows: -- Unique run ID (8 character hash): `run.id` +- Unique run ID (8-character hash): `run.id` - Random run name (human readable): `run.name` -If you're thinking about ways to set useful identifiers for your runs, here's what we recommend: +To set useful identifiers for your runs, follow these recommendations: -- **Run ID**: leave it as the generated hash. This needs to be unique across runs in your project. -- **Run name**: This should be something short, readable, and preferably unique so that you can tell the difference between different lines on your charts. -- **Run notes**: This is a great place to put a quick description of what you're doing in your run. You can set this with `wandb.init(notes="your notes here")` -- **Run tags**: Track things dynamically in run tags, and use filters in the UI to filter your table down to just the runs you care about. You can set tags from your script and then edit them in the UI, both in the runs table and the overview tab of the run page. See the detailed instructions [here](/models/runs/tags/). +- **Run ID**: Leave it as the generated hash. The run ID must be unique across runs in your project. +- **Run name**: Use something short, readable, and preferably unique so that you can tell the difference between lines on your charts. +- **Run notes**: A good place for a short description of what you're doing in your run. Set this with `wandb.init(notes="your notes here")`. +- **Run tags**: Track things dynamically in run tags, and use filters in the UI to filter your table down to only the runs you care about. Set tags from your script and then edit them in the UI, both in the runs table and the **Overview** tab of the run page. For details, see [Tag runs](/models/runs/tags/). -## Public API Examples +## Public API examples + +The following sections show common patterns for working with the Public API, including reading and filtering runs, updating run data, and downloading files. ### Export data to visualize in matplotlib or seaborn -Check out our [API examples](/models/ref/python/public-api/) for some common export patterns. You can also click the download button on a custom plot or on the expanded runs table to download a CSV from your browser. +For common export patterns, see the [API examples](/models/ref/python/public-api/). To download a CSV from your browser, click the download button on a custom plot or on the expanded runs table. ### Read metrics from a run @@ -336,7 +341,7 @@ if run.state == "finished": ### Filter runs -You can filter by using the MongoDB Query Language. +To filter runs, use the MongoDB Query Language. #### Date @@ -349,7 +354,7 @@ runs = api.runs( ### Read specific metrics from a run -To pull specific metrics from a run, use the `keys` argument. The default number of samples when using `run.history()` is 500. Logged steps that do not include a specific metric will appear in the output dataframe as `NaN`. The `keys` argument will cause the API to sample steps that include the listed metric keys more frequently. +To retrieve specific metrics from a run, use the `keys` argument. The default number of samples when using `run.history()` is 500. Logged steps that don't include a specific metric appear in the output dataframe as `NaN`. The `keys` argument causes the API to sample steps that include the listed metric keys more frequently. ```python import wandb @@ -364,7 +369,7 @@ if run.state == "finished": ### Compare two runs -This will output the config parameters that are different between `run1` and `run2`. +This example outputs the config parameters that differ between `run1` and `run2`. ```python import pandas as pd @@ -385,7 +390,7 @@ print(df[df[run1.name] != df[run2.name]]) Outputs: -``` +```text c_10_sgd_0.025_0.01_long_switch base_adam_4_conv_2fc batch_size 32 16 n_conv_layers 5 4 @@ -423,7 +428,7 @@ run.summary.update() ``` -Renaming a column only applies to tables. Charts will still refer to metrics by their original names. +Renaming a column only applies to tables. Charts still refer to metrics by their original names. @@ -444,7 +449,7 @@ run.update() ### Export system resource consumptions to a CSV file -The snippet below would find the system resource consumptions and then, save them to a CSV. +The following snippet finds the system resource consumptions and saves them to a CSV. ```python import wandb @@ -457,7 +462,7 @@ with wandb.Api().run("//") as run: ### Get unsampled metric data -When you pull data from history, by default it's sampled to 500 points. Get all the logged data points using `run.scan_history()`. Here's an example downloading all the `loss` data points logged in history. +When you retrieve data from history, by default it's sampled to 500 points. To get all the logged data points, use `run.scan_history()`. The following example downloads all the `loss` data points logged in history. ```python import wandb @@ -471,7 +476,7 @@ losses = [row["loss"] for row in history] ### Get paginated data from history -If metrics are being fetched slowly on our backend or API requests are timing out, you can try lowering the page size in `scan_history` so that individual requests don't time out. The default page size is 500, so you can experiment with different sizes to see what works best: +If the backend fetches metrics slowly or API requests time out, lower the page size in `scan_history` so that individual requests don't time out. The default page size is 500. Experiment with different sizes to see what works best: ```python import wandb @@ -484,7 +489,7 @@ run.scan_history(keys=sorted(cols), page_size=100) ### Export metrics from all runs in a project to a CSV file -This script pulls down the runs in a project and produces a dataframe and a CSV of runs including their names, configs, and summary stats. Replace `` and `` with your W&B entity and the name of your project, respectively. +This script downloads the runs in a project and produces a dataframe and a CSV of runs including their names, configs, and summary stats. Replace `` and `` with your W&B entity and the name of your project, respectively. ```python import pandas as pd @@ -530,7 +535,7 @@ start_time = run.created_at ### Upload files to a finished run -The code snippet below uploads a selected file to a finished run. +The following code snippet uploads a selected file to a finished run. ```python import wandb @@ -648,7 +653,7 @@ system_metrics.to_csv("sys_metrics.csv") ### Update summary metrics -You can pass a dictionary to update summary metrics. +To update summary metrics, pass a dictionary. ```python summary.update({"key": val}) @@ -656,7 +661,7 @@ summary.update({"key": val}) ### Get the command that ran the run -Each run captures the command that launched it on the run overview page. To pull this command down from the API, you can run: +Each run captures the command that launched it on the run overview page. To retrieve this command from the API, run the following: ```python import wandb diff --git a/models/track/reproduce_experiments.mdx b/models/track/reproduce_experiments.mdx index aef3e3caa2..2e3f0738fe 100644 --- a/models/track/reproduce_experiments.mdx +++ b/models/track/reproduce_experiments.mdx @@ -1,50 +1,53 @@ --- title: Reproduce experiments description: "Reproduce a teammate's W&B experiment by downloading the associated code, dependencies, and configuration from a run." +keywords: [reproduce, requirements.txt, git state, run code, dependencies] --- -Reproduce an experiment that a team member creates to verify and validate their results. +Reproduce an experiment that a team member creates to verify and validate their results. This page explains how to retrieve the code, dependencies, and configuration from a logged run so you can re-execute it in your own environment. -Before you reproduce an experiment, you need to make note of the: +Before you reproduce an experiment, note the: * Name of the project the run was logged to * Name of the run you want to reproduce To reproduce an experiment: -1. Navigate to the project where the run is logged to. +1. Navigate to the project that contains the run. 2. Select the **Workspace** tab from the project sidebar. 3. Click the run that you want to reproduce. The run page opens with the **Overview** tab shown by default. -To continue, download the experiment's code at a given hash or clone the experiment's entire repository. +With the run page open, you next need the exact code that produced the experiment. Download the experiment's code at a given hash, or clone the experiment's entire repository. Download the experiment's Python script or notebook: -1. On the **Overview** tab (shown by default), in the **Command** field, make a note of the name of the script that created the experiment. +1. On the **Overview** tab (shown by default), in the **Command** field, note the name of the script that created the experiment. 2. Select the **Code** tab on the run page. 3. Click **Download** next to the file that corresponds to the script or notebook. -Clone the GitHub repository your teammate used when creating the experiment. To do this: +Clone the GitHub repository your teammate used when they created the experiment. To do this: 1. If necessary, gain access to the GitHub repository that your teammate used to create the experiment. 2. Copy the **Git repository** field, which contains the GitHub repository URL. 3. Clone the repository: ```bash - git clone https://github.com/your-repo.git && cd your-repo + git clone https://github.com/[YOUR-REPO].git && cd [YOUR-REPO] ``` 4. Copy and paste the **Git state** field into your terminal. The Git state is a set of Git commands that checks out the exact commit that your teammate used to create the experiment. Replace values specified in the following code snippet with your own: ```bash - git checkout -b "" 0123456789012345678901234567890123456789 + git checkout -b "[RUN-NAME]" [COMMIT-SHA] ``` +After you have the code, retrieve the dependencies your teammate used so your environment matches theirs: + 5. On the run page, select the **Files** tab. 6. Download the `requirements.txt` file and store it in your working directory. This directory should contain either the cloned GitHub repository or the downloaded Python script or notebook. -7. (Recommended) Create a Python virtual environment. +7. Optional: Create a Python virtual environment. 8. Install the requirements specified in the `requirements.txt` file. ```bash pip install -r requirements.txt @@ -54,16 +57,16 @@ Clone the GitHub repository your teammate used when creating the experiment. To -If you downloaded a Python notebook, navigate to the directory where you downloaded the notebook and run the following command in your terminal: +If you downloaded a Python notebook, navigate to the directory where you downloaded the notebook and start Jupyter: ```bash jupyter notebook ``` -If you downloaded a Python script, navigate to the directory where you downloaded the script and run the following command in your terminal; Replace values enclosed in `<>` with your own: +If you downloaded a Python script, navigate to the directory where you downloaded the script and run it with `python`. Replace values enclosed in `[]` with your own: ```bash -python .py +python [YOUR-SCRIPT-NAME].py ``` \ No newline at end of file diff --git a/models/track/workspaces.mdx b/models/track/workspaces.mdx index cf291b840f..16f7da234b 100644 --- a/models/track/workspaces.mdx +++ b/models/track/workspaces.mdx @@ -1,9 +1,10 @@ --- description: A playground for exploring run data with interactive visualizations title: View experiments results +keywords: ["panels", "saved views", "templates", "wandb-workspaces"] --- -W&B workspace is your personal sandbox to customize charts and explore model results. A W&B workspace consists of *Tables* and *Panel sections*: +A W&B workspace is where you customize charts and explore model results. Use a workspace to visualize and compare runs across your project, organize panels for analysis, and share findings with your team. A W&B workspace consists of **Tables** and **Panel sections**: * **Tables**: All runs logged to your project are listed in the project's table. Turn on and off runs, change colors, and expand the table to see notes, config, and summary metrics for each run. * **Panel sections**: A section that contains one or more [panels](/models/app/features/panels/). Create new panels, organize them, and export to reports to save snapshots of your workspace. @@ -13,18 +14,18 @@ W&B workspace is your personal sandbox to customize charts and explore model res ## Workspace types -There are two main workspace categories: **Personal workspaces** and **Saved views**. +There are two main workspace categories: **Personal workspaces** and **Saved views**. -* **Personal workspaces:** A customizable workspace for in-depth analysis of models and data visualizations. Only the owner of the workspace can edit and save changes. Teammates can view a personal workspace but teammates can not make changes to someone else's personal workspace. +* **Personal workspaces:** A customizable workspace for in-depth analysis of models and data visualizations. Only the owner of the workspace can edit and save changes. Teammates can view a personal workspace but teammates can't make changes to someone else's personal workspace. * **Saved views:** Saved views are collaborative snapshots of a workspace. Anyone on your team can view, edit, and save changes to saved workspace views. Use saved workspace views for reviewing and discussing experiments, runs, and more. -The following image shows multiple personal workspaces created by Cécile-parker's teammates. In this project, there are no saved views: +The following image shows multiple personal workspaces created by Cécile-parker's teammates. This project has no saved views: No saved views ## Saved workspace views -Improve team collaboration with tailored workspace views. Create Saved Views to organize your preferred setup of charts and data. +Create saved views to organize your preferred setup of charts and data, and to collaborate with your team. ### Create a new saved workspace view @@ -38,12 +39,12 @@ New saved views appear in the workspace navigation menu. Saved views menu -### Update a saved workspace view -Saved changes overwrite the previous state of the saved view. Unsaved changes are not retained. To update a saved workspace view in W&B: +### Update a saved workspace view +Saved changes overwrite the previous state of the saved view. W&B doesn't retain unsaved changes. To update a saved workspace view in W&B: 1. Navigate to a saved view. 2. Make the desired changes to your charts and data within the workspace. -3. Click the **Save** button to confirm your changes. +3. Click the **Save** button to confirm your changes. A confirmation dialog appears when you save your updates to a workspace view. If you prefer not to see this prompt in the future, select the option **Do not show this modal next time** before confirming your save. @@ -58,14 +59,14 @@ Remove saved views that are no longer needed. 4. Confirm the deletion to remove the view from your workspace menu. ### Share a workspace view -Share your customized workspace with your team by sharing the workspace URL directly. All users with access to the workspace project can see the saved Views of that workspace. +Share your customized workspace with your team by sharing the workspace URL directly. All users with access to the workspace project can see the saved views of that workspace. ## Workspace templates This feature requires an [Enterprise](https://wandb.ai/site/pricing/) license. -Use _workspace templates_ to create workspaces using the same settings as an existing workspace instead of the [default settings for new workspaces](#default-workspace-settings). +Use workspace templates to create workspaces using the same settings as an existing workspace instead of the [default settings for new workspaces](#default-workspace-settings). Templates let you standardize how new workspaces look and behave so you don't have to reconfigure each one. ### Default workspace settings @@ -74,11 +75,11 @@ W&B uses the following default settings for workspaces: #### Hide empty sections during search -By default, W&B does not show empty sections in the workspace. You can change this setting to show empty sections in the workspace. +By default, W&B doesn't show empty sections in the workspace. You can change this setting to show empty sections in the workspace. #### Sort panels alphabetically -By default, W&B does not sort panels by plot title alphabetically. You can change this setting to sort panels alphabetically by their plot title. +By default, W&B doesn't sort panels by plot title alphabetically. You can change this setting to sort panels alphabetically by their plot title. #### Section organization By default, W&B organizes panels into sections based on the first prefix of the metric name. For example, if your workspace includes the metrics `a/b/c/d` and `a/e/f`, W&B organizes these metrics into a section called `a`. If your workspace includes the metrics `a/b/c/d` and `e/f/g`, W&B organizes these metrics into sections called `a` and `e`. @@ -123,28 +124,30 @@ By default, new workspaces use these default settings for line plots: | Show X range in tooltip | Off | | Tooltip runs | Default | | Sync zoom across charts | Off | -| Show highlighted run only | Off | +| Show highlighted run only | Off | ### Configure your workspace template 1. Open any workspace or create a new one. -1. Click on the **Settings** button at the top right corner of the workspace. +1. Click the **Settings** button at the top right corner of the workspace. 1. Select **Workspace layout** from the panel. 1. Configure the workspace's settings according to your preferences. ### Save your workspace template +After you configure a workspace's settings, save them as your personal template so future workspaces inherit the same configuration. + 1. At the top of the workspace, click the **action ()** menu near the **Undo** and **Redo** arrow icons. 1. Click **Save personal workspace template**. 1. Review the settings for the template, then click **Save**. -New workspaces will use these settings instead of the defaults. +New workspaces use these settings instead of the defaults. ### View your workspace template To view your workspace template's current configuration: 1. From any page, select your user icon on the top right corner. From the dropdown, choose **User Settings**. -1. Navigate to the **Personal workspace template** section. If you are using a workspace template, its configuration displays. Otherwise, the section includes no details. +1. Navigate to the **Personal workspace template** section. If you're using a workspace template, its configuration displays. Otherwise, the section includes no details. ### Update your workspace template To update your workspace template: @@ -152,29 +155,29 @@ To update your workspace template: 1. Open any workspace. 1. Modify the workspace's settings. For example, set the number of runs to include to `11`. 1. To save the changes to the template, click the **action ()** menu near the **Undo** and **Redo** arrow icons, then click **Update personal workspace template**. -1. Verify the settings, then click **Update**. The template is updated, and reapplied to all workspaces that use it. +1. Verify the settings, then click **Update**. W&B updates the template and reapplies it to all workspaces that use it. ### Delete your workspace template To delete your workspace template and go back to the default settings: 1. From any page, select your user icon on the top right corner. From the dropdown, choose **User Settings**. 1. Navigate to the **Personal workspace template** section. Your workspace template's configuration displays. -1. Click the trash icon next to **Settings**. +1. Click the trash icon next to **Settings**. -For Dedicated Cloud and Self-Managed, deleting your workspace template is supported on v0.70 and above. On older Server versions, update your workspace template to use the [default settings](#default-workspace-settings) instead. +For Dedicated Cloud and Self-Managed, deleting your workspace template is supported on v0.70 and later. On earlier Server versions, update your workspace template to use the [default settings](#default-workspace-settings) instead. ## Programmatically create workspaces +Create and customize workspaces from Python code when you want to automate workspace setup, version-control workspace definitions, or replicate the same layout across multiple projects. + For an end-to-end example, see [Programmatic Workspaces](https://colab.research.google.com/github/wandb/wandb-workspaces/blob/Update-wandb-workspaces-tuturial/Workspace_tutorial.ipynb) notebook. [`wandb-workspaces`](https://github.com/wandb/wandb-workspaces/tree/main) is a Python library for programmatically working with [W&B](https://wandb.ai/) workspaces and reports. -Define a workspace programmatically with [`wandb-workspaces`](https://github.com/wandb/wandb-workspaces/tree/main). [`wandb-workspaces`](https://github.com/wandb/wandb-workspaces/tree/main) is a Python library for programmatically working with [W&B](https://wandb.ai/) workspaces and reports. - You can define the workspace's properties, such as: * Set panel layouts, colors, and section orders. @@ -182,7 +185,7 @@ You can define the workspace's properties, such as: * Add and customize panels within sections to organize workspace views. * Load and modify existing workspaces using a URL. * Save changes to existing workspaces or save as new views. -* Filter, group, and sort runs programmatically using simple expressions. +* Filter, group, and sort runs programmatically using expressions. * Customize run appearance with settings like colors and visibility. * Copy views from one workspace to another for integration and reuse. @@ -201,7 +204,7 @@ You can define the workspace's properties, such as: - **Cross-workspace integration:** - Copy views from one workspace to another for seamless integration and reuse. */} -### Install Workspace API +### Install the Workspace API In addition to `wandb`, ensure that you install `wandb-workspaces`: @@ -217,25 +220,25 @@ pip install wandb wandb-workspaces ```python import wandb_workspaces.reports.v2 as ws -workspace = ws.Workspace(entity="your-entity", project="your-project", views=[...]) +workspace = ws.Workspace(entity="[ENTITY-NAME]", project="[PROJECT-NAME]", views=[...]) workspace.save() ``` ### Edit an existing view ```python -existing_workspace = ws.Workspace.from_url("workspace-url") -existing_workspace.views[0] = ws.View(name="my-new-view", sections=[...]) +existing_workspace = ws.Workspace.from_url("[WORKSPACE-URL]") +existing_workspace.views[0] = ws.View(name="[VIEW-NAME]", sections=[...]) existing_workspace.save() ``` ### Copy a workspace `saved view` to another workspace ```python -old_workspace = ws.Workspace.from_url("old-workspace-url") +old_workspace = ws.Workspace.from_url("[OLD-WORKSPACE-URL]") old_workspace_view = old_workspace.views[0] -new_workspace = ws.Workspace(entity="new-entity", project="new-project", views=[old_workspace_view]) +new_workspace = ws.Workspace(entity="[NEW-ENTITY-NAME]", project="[NEW-PROJECT-NAME]", views=[old_workspace_view]) new_workspace.save() ``` -See [`wandb-workspace examples`](https://github.com/wandb/wandb-workspaces/tree/main/examples/workspaces) for comprehensive workspace API examples. +See [`wandb-workspace examples`](https://github.com/wandb/wandb-workspaces/tree/main/examples/workspaces) for workspace API examples. From 2a44a7774525db838dd458be055b530dbf96ff17 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Fri, 5 Jun 2026 15:07:20 -0400 Subject: [PATCH 2/3] style: apply review feedback to models/track Apply recurring style rules from ngrayluna's review of the integrations PR (#2673) to the models/track set: - "preceding" -> "previous" - passive -> active voice; W&B (not `wandb`) as the actor performing actions - third person -> second person (address the reader as "you") - gloss `wandb` as the W&B Python SDK on first mention where context was missing Co-Authored-By: Claude Opus 4.8 (1M context) --- models/track/create-an-experiment.mdx | 6 ++--- models/track/environment-variables.mdx | 30 +++++++++++------------ models/track/limits.mdx | 2 +- models/track/log/distributed-training.mdx | 8 +++--- models/track/log/log-models.mdx | 4 +-- models/track/log/log-summary.mdx | 2 +- models/track/log/plots.mdx | 2 +- models/track/log/working-with-csv.mdx | 6 ++--- models/track/project-page.mdx | 2 +- models/track/public-api-guide.mdx | 2 +- 10 files changed, 32 insertions(+), 32 deletions(-) diff --git a/models/track/create-an-experiment.mdx b/models/track/create-an-experiment.mdx index 66c8e286d6..403183079a 100644 --- a/models/track/create-an-experiment.mdx +++ b/models/track/create-an-experiment.mdx @@ -6,7 +6,7 @@ keywords: ["wandb.init", "run.log", "log_artifact", "hyperparameters", "training Use the W&B Python SDK to track machine learning experiments. You can then review the results in an interactive dashboard or export your data to Python for programmatic access with the [W&B Public API](/models/ref/python/public-api/). -This guide describes how to use W&B building blocks to create a W&B Experiment so you can capture hyperparameters, log metrics, and save model artifacts for later analysis and comparison. It's intended for users who train machine learning models in Python and want to track their work with W&B. +This guide describes how to use W&B building blocks to create a W&B Experiment so you can capture hyperparameters, log metrics, and save model artifacts for later analysis and comparison. This page is for you if you train machine learning models in Python and want to track your work with W&B. ## Create a W&B Experiment @@ -86,7 +86,7 @@ Learn more about [Artifacts](/models/artifacts/) or about versioning models in [ ### Complete example -The following script combines the preceding code snippets into a complete experiment that initializes a run, captures configuration, logs metrics, and saves a model artifact: +The following script combines the previous code snippets into a complete experiment that initializes a run, captures configuration, logs metrics, and saves a model artifact: ```python import wandb @@ -142,7 +142,7 @@ The following are some suggested guidelines to consider when you create experime - **Tags**: Identify baseline runs and favorite runs. You can filter runs using tags. You can edit tags later on the Overview section of your project's dashboard on the W&B App. - **Create multiple run sets to compare experiments**: When you compare experiments, create multiple run sets to make metrics easy to compare. You can toggle run sets on or off on the same chart or group of charts. -The following code snippet demonstrates how to define a W&B Experiment using the preceding best practices: +The following code snippet demonstrates how to define a W&B Experiment using the previous best practices: ```python import wandb diff --git a/models/track/environment-variables.mdx b/models/track/environment-variables.mdx index 1079588fab..2d82a5c6cc 100644 --- a/models/track/environment-variables.mdx +++ b/models/track/environment-variables.mdx @@ -45,25 +45,25 @@ Use these optional environment variables to do things such as set up authenticat | `WANDB_DATA_DIR` | Where to upload staging artifacts. The default location depends on your platform, because it uses the value of `user_data_dir` from the `platformdirs` Python package. Make sure this directory exists and the running user has permission to write to it. | | `WANDB_DIR` | Where to store all generated files. If unset, defaults to the `wandb` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This doesn't control the location of downloaded artifacts, which you can set using the `WANDB_ARTIFACT_DIR` environment variable. | | `WANDB_ARTIFACT_DIR` | Where to store all downloaded artifacts. If unset, defaults to the `artifacts` directory relative to your training script. Make sure this directory exists and the running user has permission to write to it. This doesn't control the location of generated metadata files, which you can set using the `WANDB_DIR` environment variable. | -| `WANDB_DISABLE_GIT` | Prevents `wandb` from probing for a Git repository and capturing the latest commit and diff. | -| `WANDB_DISABLE_CODE` | Set this to `true` to prevent `wandb` from saving notebooks or Git diffs. `wandb` still saves the current commit if you're in a Git repo. | +| `WANDB_DISABLE_GIT` | Prevents W&B from probing for a Git repository and capturing the latest commit and diff. | +| `WANDB_DISABLE_CODE` | Set this to `true` to prevent W&B from saving notebooks or Git diffs. W&B still saves the current commit if you're in a Git repo. | | `WANDB_DOCKER` | Set this to a Docker image digest to enable restoring of runs. This is set automatically with the `wandb docker` command. You can obtain an image digest by running `wandb docker [IMAGE_NAME]:[TAG] --digest`. | -| `WANDB_ENTITY` | The entity associated with your run. If you have run `wandb init` in the directory of your training script, `wandb` creates a directory named `wandb` and saves a default entity, which you can check into source control. If you don't want to create that file or want to override the file, use the environment variable. | -| `WANDB_ERROR_REPORTING` | Set this to `false` to prevent `wandb` from logging fatal errors to its error tracking system. | -| `WANDB_HOST` | Set this to the hostname you want to see in the `wandb` interface if you don't want to use the system-provided hostname. | -| `WANDB_IGNORE_GLOBS` | Set this to a comma-separated list of file globs to ignore. `wandb` doesn't sync these files to the cloud. | -| `WANDB_JOB_NAME` | Specifies a name for any jobs created by `wandb`. | +| `WANDB_ENTITY` | The entity associated with your run. If you have run `wandb init` in the directory of your training script, W&B creates a directory named `wandb` and saves a default entity, which you can check into source control. If you don't want to create that file or want to override the file, use the environment variable. | +| `WANDB_ERROR_REPORTING` | Set this to `false` to prevent W&B from logging fatal errors to its error tracking system. | +| `WANDB_HOST` | Set this to the hostname you want to see in the W&B interface if you don't want to use the system-provided hostname. | +| `WANDB_IGNORE_GLOBS` | Set this to a comma-separated list of file globs to ignore. W&B doesn't sync these files to the cloud. | +| `WANDB_JOB_NAME` | Specifies a name for any jobs created by W&B. | | `WANDB_JOB_TYPE` | Specifies the job type, such as `training` or `evaluation`, to indicate different types of runs. See [grouping](/models/runs/grouping/) for more info. | -| `WANDB_MODE` | If you set this to `offline`, `wandb` saves your run metadata locally and doesn't sync to the server. If you set this to `disabled`, `wandb` turns off completely. | -| `WANDB_NAME` | The human-readable name of your run. If not set, `wandb` randomly generates it for you. | -| `WANDB_NOTEBOOK_NAME` | If you're running in Jupyter, you can set the name of the notebook with this variable. `wandb` attempts to autodetect this. | +| `WANDB_MODE` | If you set this to `offline`, W&B saves your run metadata locally and doesn't sync to the server. If you set this to `disabled`, W&B turns off completely. | +| `WANDB_NAME` | The human-readable name of your run. If not set, W&B randomly generates it for you. | +| `WANDB_NOTEBOOK_NAME` | If you're running in Jupyter, you can set the name of the notebook with this variable. W&B attempts to autodetect this. | | `WANDB_NOTES` | Longer notes about your run. Markdown is allowed, and you can edit this later in the UI. | | `WANDB_PROJECT` | The project associated with your run. You can also set this with `wandb init`, but the environment variable overrides the value. | -| `WANDB_RESUME` | By default, this is set to `never`. If set to `auto`, `wandb` automatically resumes failed runs. If set to `must`, forces the run to exist on startup. If you want to always generate your own unique IDs, set this to `allow` and always set `WANDB_RUN_ID`. | +| `WANDB_RESUME` | By default, this is set to `never`. If set to `auto`, W&B automatically resumes failed runs. If set to `must`, forces the run to exist on startup. If you want to always generate your own unique IDs, set this to `allow` and always set `WANDB_RUN_ID`. | | `WANDB_RUN_GROUP` | Specifies the experiment name to automatically group runs together. See [grouping](/models/runs/grouping/) for more info. | -| `WANDB_RUN_ID` | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. `wandb` converts all non-word characters to dashes. You can use this to resume an existing run in cases of failure. | -| `WANDB_QUIET` | Set this to `true` to limit statements logged to standard output to critical statements only. If this is set, `wandb` writes all logs to `$WANDB_DIR/debug.log`. | -| `WANDB_SILENT` | Set this to `true` to silence `wandb` log statements. This is useful for scripted commands. If this is set, `wandb` writes all logs to `$WANDB_DIR/debug.log`. | +| `WANDB_RUN_ID` | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. W&B converts all non-word characters to dashes. You can use this to resume an existing run in cases of failure. | +| `WANDB_QUIET` | Set this to `true` to limit statements logged to standard output to critical statements only. If this is set, W&B writes all logs to `$WANDB_DIR/debug.log`. | +| `WANDB_SILENT` | Set this to `true` to silence `wandb` log statements. This is useful for scripted commands. If this is set, W&B writes all logs to `$WANDB_DIR/debug.log`. | | `WANDB_SHOW_RUN` | Set this to `true` to automatically open a browser with the run URL if your operating system supports it. | | `WANDB_SWEEP_ID` | Adds sweep ID tracking to `Run` objects and related classes, and displays it in the UI. | | `WANDB_TAGS` | A comma-separated list of tags to apply to the run. | @@ -72,7 +72,7 @@ Use these optional environment variables to do things such as set up authenticat ## Singularity environments -If you're running containers in [Singularity](https://singularity.lbl.gov/index.html), you can pass environment variables by prepending any of the preceding variables with `SINGULARITYENV_`. For more details about Singularity environment variables, see the [Singularity documentation](https://singularity.lbl.gov/docs-environment-metadata#environment). +If you're running containers in [Singularity](https://singularity.lbl.gov/index.html), you can pass environment variables by prepending any of the previous variables with `SINGULARITYENV_`. For more details about Singularity environment variables, see the [Singularity documentation](https://singularity.lbl.gov/docs-environment-metadata#environment). ## Run on AWS diff --git a/models/track/limits.mdx b/models/track/limits.mdx index 9de55e254a..c7e1b4175a 100644 --- a/models/track/limits.mdx +++ b/models/track/limits.mdx @@ -418,7 +418,7 @@ For Multi-tenant Cloud: * Authorized requests are rate-limited per user. * Some SDK requests that specify a project path can also be limited per project based on database query time. -Teams and Enterprise plans have higher limits than Free plans. +If you're on a Teams or Enterprise plan, you have higher limits than on the Free plan. If you're making many public API requests, wait at least 1 second between requests when possible. If you receive HTTP `429 Rate limit exceeded` or see `RateLimit-Remaining=0`, wait for the number of seconds in `RateLimit-Reset` before retrying. diff --git a/models/track/log/distributed-training.mdx b/models/track/log/distributed-training.mdx index 32a15a4c53..57f69b34e3 100644 --- a/models/track/log/distributed-training.mdx +++ b/models/track/log/distributed-training.mdx @@ -4,7 +4,7 @@ title: Log distributed training experiments keywords: ["DDP", "PyTorch distributed", "multi-node training", "shared mode", "rank 0"] --- -During a distributed training experiment, you train a model using multiple machines or clients in parallel. W&B can help you track distributed training experiments so that you can monitor metrics, system utilization, and model behavior across all participating processes. This guide is for ML practitioners running distributed training jobs who want to log experiment data with W&B. +During a distributed training experiment, you train a model using multiple machines or clients in parallel. W&B can help you track distributed training experiments so that you can monitor metrics, system utilization, and model behavior across all participating processes. Use this guide if you run distributed training jobs and want to log experiment data with W&B. Based on your use case, track distributed training experiments using one of the following approaches: @@ -20,7 +20,7 @@ Each concurrent connection takes compute, memory, and network resources. Even em W&B recommends that you limit the maximum number of concurrent client connections as appropriate for your workload and that you monitor resource usage over time. W&B has tested with a hard limit of 300 concurrent client connections in Dedicated Cloud. -In Multi-tenant Cloud organizations, client connections for distributed training are subject to the same [rate limits](/models/track/limits#rate-limits) as regular training runs. Users on [Teams and Enterprise plans](https://wandb.ai/site/pricing) receive higher rate limits than those on the Free plan. +In Multi-tenant Cloud organizations, client connections for distributed training are subject to the same [rate limits](/models/track/limits#rate-limits) as regular training runs. On [Teams and Enterprise plans](https://wandb.ai/site/pricing), you receive higher rate limits than on the Free plan. {/* The following examples demonstrate how to track metrics with W&B using PyTorch DDP on two GPUs on a single machine. [PyTorch DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) (`DistributedDataParallel` in`torch.nn`) is a popular library for distributed training. The basic principles apply to any distributed training setup, but the details of implementation may differ. @@ -115,7 +115,7 @@ Explore the W&B App UI to view an [example dashboard](https://wandb.ai/ayush-tha Grouped distributed runs -The preceding image demonstrates the W&B App UI dashboard. The sidebar shows two experiments: one labeled "null" and a second (bound by a yellow box) called "DPP". Expand the group (select the **Group** dropdown) to see the runs associated with that experiment. +The previous image demonstrates the W&B App UI dashboard. The sidebar shows two experiments: one labeled "null" and a second (bound by a yellow box) called "DPP". Expand the group (select the **Group** dropdown) to see the runs associated with that experiment. ### Organize distributed runs @@ -317,7 +317,7 @@ The following sections describe two common issues you might encounter when using - **Becoming unresponsive at the beginning of training**: A `wandb` process can stop responding if the `wandb` multiprocessing interferes with the multiprocessing from distributed training. - **Becoming unresponsive at the end of training**: A training job might stop responding if the `wandb` process can't determine when to exit. Call the `wandb.Run.finish()` API at the end of your Python script to signal W&B that the run finished. The `wandb.Run.finish()` API finishes uploading data and causes W&B to exit. -W&B recommends using `wandb service` command to improve the reliability of your distributed jobs. Both of the preceding training issues commonly occur in versions of the W&B SDK where `wandb service` is unavailable. +W&B recommends using `wandb service` command to improve the reliability of your distributed jobs. Both of the previous training issues commonly occur in versions of the W&B SDK where `wandb service` is unavailable. ### Enable W&B Service diff --git a/models/track/log/log-models.mdx b/models/track/log/log-models.mdx index 3bd038b60b..5bc43f0cda 100644 --- a/models/track/log/log-models.mdx +++ b/models/track/log/log-models.mdx @@ -9,7 +9,7 @@ import { ColabLink } from '/snippets/_includes/colab-link.mdx'; # Log models -This guide shows you how to log model files to a W&B run as a model artifact, and how to retrieve and reuse those files in later runs. Logging models to a run lets you track which experiment produced a given model, capture its lineage, and make it available for downstream use. This page is for users who want lightweight model tracking as part of an experiment tracking workflow. +This guide shows you how to log model files to a W&B run as a model artifact, and how to retrieve and reuse those files in later runs. Logging models to a run lets you track which experiment produced a given model, capture its lineage, and make it available for downstream use. Use this page if you want lightweight model tracking as part of an experiment tracking workflow. The following APIs are useful for tracking models as a part of your experiment tracking workflow. Use the APIs listed on this page to log models to a run, and to access metrics, tables, media, and other objects. @@ -137,7 +137,7 @@ with wandb.init(project="[YOUR-PROJECT]", entity="[YOUR-ENTITY]") as run: downloaded_model_path = run.use_model(name="[YOUR-MODEL-NAME]") ``` -The [`use_model`](/models/ref/python/experiments/run#use_model) function returns the path of downloaded model files. Keep track of this path if you want to link this model later. In the preceding code snippet, the returned path is stored in a variable called `downloaded_model_path`. +The [`use_model`](/models/ref/python/experiments/run#use_model) function returns the path of downloaded model files. Keep track of this path if you want to link this model later. In the previous code snippet, the returned path is stored in a variable called `downloaded_model_path`.
diff --git a/models/track/log/log-summary.mdx b/models/track/log/log-summary.mdx index a9bfbbcd74..7515df4f8c 100644 --- a/models/track/log/log-summary.mdx +++ b/models/track/log/log-summary.mdx @@ -6,7 +6,7 @@ keywords: ["run.summary", "define_metric", "best accuracy", "summary dictionary" In addition to values that change over time during training, it's often important to track a single value that summarizes a model or a preprocessing step. Log this information in a W&B run's `summary` dictionary. A run's summary dictionary can handle numpy arrays, PyTorch tensors, or TensorFlow tensors. When a value is one of these types, W&B persists the entire tensor in a binary file and stores high-level metrics in the summary object, such as min, mean, variance, and percentiles. -The last value logged with `wandb.Run.log()` is automatically set as the summary dictionary in a W&B run. If you modify a summary metric dictionary, the previous value is lost. +The last value logged with `wandb.Run.log()` is automatically set as the summary dictionary in a W&B run. If you modify a summary metric dictionary, you lose the previous value. The following code snippet demonstrates how to provide a custom summary metric to W&B: diff --git a/models/track/log/plots.mdx b/models/track/log/plots.mdx index 51ed4b50d2..e6da0120ad 100644 --- a/models/track/log/plots.mdx +++ b/models/track/log/plots.mdx @@ -4,7 +4,7 @@ title: Create and track plots from experiments keywords: ["wandb.plot", "custom charts", "matplotlib", "Plotly", "Bokeh"] --- -This page shows you how to create and log plots from your machine learning experiments so that you can visualize metrics, evaluate models, and share interactive charts in W&B. It's for users who already log runs with `wandb` and want to add custom or built-in visualizations to their experiments. +This page shows you how to create and log plots from your machine learning experiments so that you can visualize metrics, evaluate models, and share interactive charts in W&B. This page is for you if you already log runs with the W&B Python SDK (`wandb`) and want to add custom or built-in visualizations to your experiments. In W&B Models, methods in `wandb.plot` let you track charts with `wandb.Run.log()`, including charts that change over time during training. To learn more about the custom charting framework, see the [custom charts walkthrough](/models/app/features/custom-charts/walkthrough/). diff --git a/models/track/log/working-with-csv.mdx b/models/track/log/working-with-csv.mdx index da2660736d..d8631c68f2 100644 --- a/models/track/log/working-with-csv.mdx +++ b/models/track/log/working-with-csv.mdx @@ -55,14 +55,14 @@ with wandb.init(project="tables-walkthrough") as run: run.log_artifact(iris_table_artifact) ``` -The `wandb.init()` API spawns a new background process to log data to a run, and it synchronizes data to `wandb.ai` by default. View live visualizations on your W&B Workspace Dashboard. The following image shows the output of the preceding code snippet. +The `wandb.init()` API spawns a new background process to log data to a run, and it synchronizes data to `wandb.ai` by default. View live visualizations on your W&B Workspace Dashboard. The following image shows the output of the previous code snippet. CSV file imported into W&B Dashboard -The full script with the preceding code snippets follows: +The full script with the previous code snippets follows: ```python import wandb @@ -175,7 +175,7 @@ run.summary.update(summaries) For more information about summary metrics, see [Log summary metrics](./log-summary). -The following full example script converts the preceding sample table into a [W&B Dashboard](/models/track/workspaces/): +The following full example script converts the previous sample table into a [W&B Dashboard](/models/track/workspaces/): ```python FILENAME = "experiments.csv" diff --git a/models/track/project-page.mdx b/models/track/project-page.mdx index d9004723e1..d5cbf7938e 100644 --- a/models/track/project-page.mdx +++ b/models/track/project-page.mdx @@ -323,7 +323,7 @@ From a team's landing page: 1. Click **Create project**. -To create a project programmatically, specify a `project` when calling `wandb.init()`. If the project does not yet exist, it is created automatically, and is owned by the specified entity. For example: +To create a project programmatically, specify a `project` when calling `wandb.init()`. If the project does not yet exist, W&B creates it automatically and assigns ownership to the specified entity. For example: ```python import wandb with wandb.init(entity="", project="") as run: run.log({"accuracy": .95}) diff --git a/models/track/public-api-guide.mdx b/models/track/public-api-guide.mdx index 0f0407e6c2..9769382a12 100644 --- a/models/track/public-api-guide.mdx +++ b/models/track/public-api-guide.mdx @@ -296,7 +296,7 @@ Calling `api.runs` returns a `Runs` object that is iterable and acts like a list ### Error handling -If errors occur while communicating with W&B servers, a `wandb.CommError` is raised. To introspect the original exception, use the `exc` attribute. +If errors occur while communicating with W&B servers, W&B raises a `wandb.CommError`. To introspect the original exception, use the `exc` attribute. ### Get the latest git commit through the API From b1e6a42ea3691eb5b0d2734015adc779da6c7b2b Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Fri, 5 Jun 2026 15:36:48 -0400 Subject: [PATCH 3/3] fix: address Copilot review comments on models/track - log/log-summary.mdx: fix Public API example so it runs (add `api = wandb.Api()`, `import pandas as pd`, `{project_name}` -> `{project}`) - jupyter.mdx: correct `ipython.display(...)` -> `IPython.display(...)` - public-api-guide.mdx: remove broken `#overview-tab` anchor link; refer to the Overview tab without a URL - project-page.mdx: format the collapsed Python SDK snippet across multiple lines - log/media.mdx: replace `{DESIRED_MAX}` with a concrete integer - workspaces.mdx: fix title grammar ("experiments" -> "experiment") Co-Authored-By: Claude Opus 4.8 (1M context) --- models/track/jupyter.mdx | 2 +- models/track/log/log-summary.mdx | 6 ++++-- models/track/log/media.mdx | 2 +- models/track/project-page.mdx | 5 ++++- models/track/public-api-guide.mdx | 2 +- models/track/workspaces.mdx | 2 +- 6 files changed, 12 insertions(+), 7 deletions(-) diff --git a/models/track/jupyter.mdx b/models/track/jupyter.mdx index 5adc029b25..f1c8580e54 100644 --- a/models/track/jupyter.mdx +++ b/models/track/jupyter.mdx @@ -69,7 +69,7 @@ You can also display any existing dashboards, sweeps, or reports directly in you %wandb [USERNAME]/[PROJECT] -h 2048 ``` -As an alternative to the `%%wandb` or `%wandb` magics, after running `wandb.init()` you can end any cell with `wandb.Run.finish()` to show inline graphs, or call `ipython.display(...)` on any report, sweep, or run object returned from the W&B APIs. +As an alternative to the `%%wandb` or `%wandb` magics, after running `wandb.init()` you can end any cell with `wandb.Run.finish()` to show inline graphs, or call `IPython.display(...)` on any report, sweep, or run object returned from the W&B APIs. ```python import wandb diff --git a/models/track/log/log-summary.mdx b/models/track/log/log-summary.mdx index 7515df4f8c..330e6843dd 100644 --- a/models/track/log/log-summary.mdx +++ b/models/track/log/log-summary.mdx @@ -99,15 +99,17 @@ The following code example demonstrates one way to retrieve the summary values l ```python import wandb -import pandas +import pandas as pd entity = "[YOUR-ENTITY]" project = "[YOUR-PROJECT]" run_name = "[YOUR-RUN-NAME]" # Name of run with summary values +api = wandb.Api() + all_runs = [] -for run in api.runs(f"{entity}/{project_name}"): +for run in api.runs(f"{entity}/{project}"): print("Fetching details for run: ", run.id, run.name) run_data = { "id": run.id, diff --git a/models/track/log/media.mdx b/models/track/log/media.mdx index e7d7281a3d..a88180d01e 100644 --- a/models/track/log/media.mdx +++ b/models/track/log/media.mdx @@ -634,7 +634,7 @@ For more usage information, see [`video-file`](/models/ref/query-panel/video-fil ### Text -Use `wandb.Table` to log text in tables that appear in the UI. By default, the column headers are `["Input", "Output", "Expected"]`. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with `wandb.Table.MAX_ROWS = {DESIRED_MAX}`. +Use `wandb.Table` to log text in tables that appear in the UI. By default, the column headers are `["Input", "Output", "Expected"]`. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with `wandb.Table.MAX_ROWS = 200000`. ```python with wandb.init(project="my_project") as run: diff --git a/models/track/project-page.mdx b/models/track/project-page.mdx index d5cbf7938e..b458333096 100644 --- a/models/track/project-page.mdx +++ b/models/track/project-page.mdx @@ -326,7 +326,10 @@ From a team's landing page: To create a project programmatically, specify a `project` when calling `wandb.init()`. If the project does not yet exist, W&B creates it automatically and assigns ownership to the specified entity. For example: ```python -import wandb with wandb.init(entity="", project="") as run: run.log({"accuracy": .95}) +import wandb + +with wandb.init(entity="", project="") as run: + run.log({"accuracy": 0.95}) ``` Refer to the [`wandb.init()` API reference](/models/ref/python/functions/init/#examples). diff --git a/models/track/public-api-guide.mdx b/models/track/public-api-guide.mdx index 9769382a12..b3ee801eb3 100644 --- a/models/track/public-api-guide.mdx +++ b/models/track/public-api-guide.mdx @@ -171,7 +171,7 @@ The Public API requires an API key to authenticate requests from your machine to ### Find the run path -Most Public API calls identify a run by its run path, which has the form `//`. To find the run path, open a run page in the app UI and click the [Overview tab](/models/track/public-api-guide/#overview-tab). +Most Public API calls identify a run by its run path, which has the form `//`. To find the run path, open a run page in the app UI and click the **Overview** tab. ### Export run data diff --git a/models/track/workspaces.mdx b/models/track/workspaces.mdx index 16f7da234b..7aa35a672e 100644 --- a/models/track/workspaces.mdx +++ b/models/track/workspaces.mdx @@ -1,6 +1,6 @@ --- description: A playground for exploring run data with interactive visualizations -title: View experiments results +title: View experiment results keywords: ["panels", "saved views", "templates", "wandb-workspaces"] ---