Skip to content

Commit 668ebcb

Browse files
vdusekclaude
andcommitted
docs: Document missing SDK features across concept pages
- Add documentation for 13 previously undocumented SDK features across 8 concept pages - Create 8 new Python code example files with runnable examples - Features documented: `Actor.use_state()`, `Actor.abort()`, `Actor.get_env()`, `ApifyRequestList`, storage `alias` parameter, tiered proxy URLs, `EXIT` event, advanced `ChargingManager` API, `ChargeResult` from `push_data`, `Actor.is_at_home()`, secret input fields, storage client architecture, and Actor instantiation parameters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 916a62d commit 668ebcb

17 files changed

+336
-4
lines changed

docs/02_concepts/01_actor_lifecycle.mdx

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,15 @@ When the Actor exits, either normally or due to an exception, the SDK performs a
4343
</TabItem>
4444
</Tabs>
4545

46-
You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization, such as disabling automatic `sys.exit()` calls or customizing timeouts. The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.
46+
You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization. The key parameters are:
47+
48+
- `configuration` — a custom [`Configuration`](https://docs.apify.com/sdk/python/reference/class/Configuration) instance to control storage paths, API URLs, and other settings.
49+
- `configure_logging` — whether to set up default logging configuration (default `True`). Set to `False` if you configure logging yourself.
50+
- `exit_process` — whether the Actor calls `sys.exit()` when the context manager exits. Defaults to `True`, except in IPython, Pytest, and Scrapy environments.
51+
- `event_listeners_timeout` — maximum time to wait for Actor event listeners to complete before exiting.
52+
- `cleanup_timeout` — maximum time to wait for cleanup tasks to finish (default 30 seconds).
53+
54+
The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.
4755

4856
<Tabs groupId="request_queue">
4957
<TabItem value="actor_instance_with_context_manager" label="Actor instance with context manager" default>

docs/02_concepts/02_actor_input.mdx

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ description: Read and validate input data passed to your Actor at runtime.
77
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
88

99
import InputExample from '!!raw-loader!roa-loader!./code/02_input.py';
10+
import RequestListExample from '!!raw-loader!roa-loader!./code/02_request_list.py';
11+
import ApiLink from '@site/src/components/ApiLink';
1012

1113
The Actor gets its [input](https://docs.apify.com/platform/actors/running/input) from the input record in its default [key-value store](https://docs.apify.com/platform/storage/key-value-store).
1214

@@ -17,3 +19,19 @@ For example, if an Actor received a JSON input with two fields, `{ "firstNumber"
1719
<RunnableCodeBlock className="language-python" language="python">
1820
{InputExample}
1921
</RunnableCodeBlock>
22+
23+
## Loading URLs from Actor input
24+
25+
Actors commonly receive a list of URLs to process via their input. The <ApiLink to="class/ApifyRequestList">`ApifyRequestList`</ApiLink> class (from `apify.request_loaders`) can parse the standard Apify input format for URL sources. It supports both direct URL objects (`{"url": "https://example.com"}`) and remote URL lists (`{"requestsFromUrl": "https://example.com/urls.txt"}`), where the remote file contains one URL per line.
26+
27+
<RunnableCodeBlock className="language-python" language="python">
28+
{RequestListExample}
29+
</RunnableCodeBlock>
30+
31+
## Secret input fields
32+
33+
The Apify platform supports [secret input fields](https://docs.apify.com/platform/actors/development/secret-input) that are encrypted before being stored. When you mark an input field as `"isSecret": true` in your Actor's [input schema](https://docs.apify.com/platform/actors/development/input-schema), the platform encrypts the value with the Actor's public key.
34+
35+
No special handling is needed in your code — when you call [`Actor.get_input`](../../reference/class/Actor#get_input), encrypted fields are automatically decrypted using the Actor's private key, which is provided by the platform via environment variables. You receive the plaintext values directly.
36+
37+
For more details on Actor input and how to define input schemas, see the [Actor input](https://docs.apify.com/platform/actors/running/input) and [input schema](https://docs.apify.com/platform/actors/development/input-schema) documentation on the Apify platform.

docs/02_concepts/03_storages.mdx

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ description: Use datasets, key-value stores, and request queues to persist Actor
77
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
88

99
import OpeningStoragesExample from '!!raw-loader!roa-loader!./code/03_opening_storages.py';
10+
import OpeningStoragesAliasExample from '!!raw-loader!roa-loader!./code/03_opening_storages_alias.py';
11+
import ApiLink from '@site/src/components/ApiLink';
1012
import DeletingStoragesExample from '!!raw-loader!roa-loader!./code/03_deleting_storages.py';
1113
import DatasetReadWriteExample from '!!raw-loader!roa-loader!./code/03_dataset_read_write.py';
1214
import DatasetExportsExample from '!!raw-loader!roa-loader!./code/03_dataset_exports.py';
@@ -60,7 +62,7 @@ There are several methods for directly working with the default key-value store
6062
- [`Actor.get_value('my-record')`](../../reference/class/Actor#get_value) reads a record from the default key-value store of the Actor.
6163
- [`Actor.set_value('my-record', 'my-value')`](../../reference/class/Actor#set_value) saves a new value to the record in the default key-value store.
6264
- [`Actor.get_input`](../../reference/class/Actor#get_input) reads the Actor input from the default key-value store of the Actor.
63-
- [`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor.
65+
- [`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor. When using the [pay-per-event pricing model](./pay-per-event), `push_data` returns a `ChargeResult` object that indicates whether the charge limit has been reached. You can also pass a `charged_event_name` parameter to charge for a custom event for each pushed item.
6466

6567
## Opening named and unnamed storages
6668

@@ -70,6 +72,12 @@ The [`Actor.open_dataset`](../../reference/class/Actor#open_dataset), [`Actor.op
7072
{OpeningStoragesExample}
7173
</RunnableCodeBlock>
7274

75+
Besides `id` and `name`, the `open_*` methods also accept an `alias` parameter. An alias creates an unnamed storage scoped to the current Actor run — it does not persist across runs, but lets you reference the same storage within a single run using a human-readable label. The `alias` parameter is mutually exclusive with `id` and `name`.
76+
77+
<RunnableCodeBlock className="language-python" language="python">
78+
{OpeningStoragesAliasExample}
79+
</RunnableCodeBlock>
80+
7381
## Deleting storages
7482

7583
To delete a storage, you can use the [`Dataset.drop`](../../reference/class/Dataset#drop),
@@ -172,3 +180,16 @@ To check if all the requests in the queue are handled, you can use the [`Request
172180
<RunnableCodeBlock className="language-python" language="python">
173181
{RqExample}
174182
</RunnableCodeBlock>
183+
184+
## Storage clients
185+
186+
Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The SDK automatically selects the appropriate client based on the runtime environment:
187+
188+
- **`SmartApifyStorageClient`** (default on the Apify platform) — a hybrid client that writes to both the Apify API and the local filesystem for resilience.
189+
- **`ApifyStorageClient`** — communicates directly with the Apify platform API for cloud storage.
190+
- **`FileSystemStorageClient`** — stores data on the local filesystem (in the `storage/` directory). Used when running locally.
191+
- **`MemoryStorageClient`** (from Crawlee) — stores data in memory. Useful for testing.
192+
193+
For most use cases, the default storage client selection is sufficient. All storage clients are available from the `apify.storage_clients` module. For details, see the <ApiLink to="class/ApifyStorageClient">`ApifyStorageClient`</ApiLink> API reference.
194+
195+
For comprehensive information about storage on the Apify platform, see the [storage documentation](https://docs.apify.com/platform/storage), including the pages on [datasets](https://docs.apify.com/platform/storage/dataset), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), and [request queues](https://docs.apify.com/platform/storage/request-queue).

docs/02_concepts/04_actor_events.mdx

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ description: Handle platform events like state persistence and graceful shutdown
77
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
88

99
import ActorEventsExample from '!!raw-loader!roa-loader!./code/04_actor_events.py';
10+
import UseStateExample from '!!raw-loader!roa-loader!./code/04_use_state.py';
11+
import ApiLink from '@site/src/components/ApiLink';
1012

1113
During its runtime, the Actor receives Actor events sent by the Apify platform or generated by the Apify SDK itself.
1214

@@ -69,6 +71,14 @@ During its runtime, the Actor receives Actor events sent by the Apify platform o
6971
you can achieve the same effect by persisting the state regularly in an interval and listening for the migrating event.
7072
</td>
7173
</tr>
74+
<tr>
75+
<td><code>EXIT</code></td>
76+
<td><code>None</code></td>
77+
<td>
78+
Emitted by the SDK (not the platform) when the Actor is about to exit. You can use this event to perform final cleanup tasks,
79+
such as closing external connections or sending notifications, before the Actor shuts down.
80+
</td>
81+
</tr>
7282
</tbody>
7383
</table>
7484

@@ -80,3 +90,17 @@ and to remove them, you use the [`Actor.off`](../../reference/class/Actor#off) m
8090
<RunnableCodeBlock className="language-python" language="python">
8191
{ActorEventsExample}
8292
</RunnableCodeBlock>
93+
94+
## Automatic state persistence with use_state
95+
96+
The example above shows how to manually persist state using the `PERSIST_STATE` event. For most use cases, you can use the <ApiLink to="class/Actor#use_state">`Actor.use_state`</ApiLink> method instead, which handles state persistence automatically.
97+
98+
`Actor.use_state` returns a dictionary that is automatically saved to the default key-value store at regular intervals and whenever a migration or shutdown occurs. You can modify the dictionary in place, and changes are persisted without any manual `set_value` calls.
99+
100+
You can optionally specify a `key` (the key-value store key under which the state is stored) and a `kvs_name` (the name of the key-value store to use). By default, the state is stored in the default key-value store under a default key.
101+
102+
<RunnableCodeBlock className="language-python" language="python">
103+
{UseStateExample}
104+
</RunnableCodeBlock>
105+
106+
For more details on platform events and state persistence, see the [system events](https://docs.apify.com/platform/actors/development/programming-interface/system-events) and [state persistence](https://docs.apify.com/platform/actors/development/state-persistence) documentation on the Apify platform.

docs/02_concepts/05_proxy_management.mdx

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import ApifyProxyConfig from '!!raw-loader!roa-loader!./code/05_apify_proxy_conf
1313
import CustomProxyFunctionExample from '!!raw-loader!roa-loader!./code/05_custom_proxy_function.py';
1414
import ProxyActorInputExample from '!!raw-loader!roa-loader!./code/05_proxy_actor_input.py';
1515
import ProxyHttpxExample from '!!raw-loader!roa-loader!./code/05_proxy_httpx.py';
16+
import TieredProxyExample from '!!raw-loader!roa-loader!./code/05_tiered_proxy.py';
1617
import ApiLink from '@site/src/components/ApiLink';
1718

1819
The Apify SDK provides built-in proxy management through the <ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> class, supporting both [Apify Proxy](https://apify.com/proxy) and custom proxy servers. Proxies are essential for web scraping to avoid [IP address blocking](https://en.wikipedia.org/wiki/IP_address_blocking) and distribute requests across multiple addresses.
@@ -81,6 +82,18 @@ Or you can pass it a method (accepting one optional argument, the session ID), t
8182
{CustomProxyFunctionExample}
8283
</RunnableCodeBlock>
8384

85+
### Tiered proxy rotation
86+
87+
<ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> supports tiered proxy URLs via the `tiered_proxy_urls` parameter. This accepts a list of lists of proxy URLs, where each inner list represents a tier. The proxy rotator starts with the first (cheapest) tier and automatically escalates to higher tiers when lower-tier proxies get blocked. This is useful for optimizing proxy costs — you use cheap datacenter proxies for most requests and only switch to expensive residential proxies when necessary.
88+
89+
:::info
90+
The `tiered_proxy_urls` parameter is only available when constructing `ProxyConfiguration` directly. It is not supported by `Actor.create_proxy_configuration()`.
91+
:::
92+
93+
<RunnableCodeBlock className="language-python" language="python">
94+
{TieredProxyExample}
95+
</RunnableCodeBlock>
96+
8497
### Configuring proxy based on Actor input
8598

8699
To make selecting the proxies that the Actor uses easier, you can use an input field with the editor [`proxy` in your input schema](https://docs.apify.com/platform/actors/development/input-schema#object). This input will then be filled with a dictionary containing the proxy settings you or the users of your Actor selected for the Actor run.

docs/02_concepts/06_interacting_with_other_actors.mdx

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import InteractingStartExample from '!!raw-loader!roa-loader!./code/06_interacti
1010
import InteractingCallExample from '!!raw-loader!roa-loader!./code/06_interacting_call.py';
1111
import InteractingCallTaskExample from '!!raw-loader!roa-loader!./code/06_interacting_call_task.py';
1212
import InteractingMetamorphExample from '!!raw-loader!roa-loader!./code/06_interacting_metamorph.py';
13+
import InteractingAbortExample from '!!raw-loader!roa-loader!./code/06_interacting_abort.py';
1314
import ApiLink from '@site/src/components/ApiLink';
1415

1516
The Apify SDK lets you start, call, and transform (metamorph) other Actors directly from your Actor code. This is useful for composing complex workflows from smaller, reusable Actors.
@@ -52,4 +53,14 @@ For example, imagine you have an Actor that accepts a hotel URL on input, and th
5253
{InteractingMetamorphExample}
5354
</RunnableCodeBlock>
5455

55-
For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference.
56+
## Aborting an Actor run
57+
58+
The [`Actor.abort`](../../reference/class/Actor#abort) method aborts a running Actor on the Apify platform. You can use it to cancel a long-running Actor that is no longer needed.
59+
60+
When you set `gracefully=True`, the platform sends `ABORTING` and `PERSIST_STATE` events to the target Actor, giving it time to save its state, and then force-stops it after 30 seconds. Without the `gracefully` flag, the Actor is stopped immediately.
61+
62+
<RunnableCodeBlock className="language-python" language="python">
63+
{InteractingAbortExample}
64+
</RunnableCodeBlock>
65+
66+
For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference. For more details on running Actors and Actor tasks on the platform, see the [Actors](https://docs.apify.com/platform/actors) and [Actor tasks](https://docs.apify.com/platform/actors/tasks) documentation.

docs/02_concepts/10_configuration.mdx

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ description: Customize Actor behavior through the Configuration class or environ
77
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
88

99
import ConfigExample from '!!raw-loader!roa-loader!./code/10_config.py';
10+
import GetEnvExample from '!!raw-loader!roa-loader!./code/10_get_env.py';
11+
import PlatformDetectionExample from '!!raw-loader!roa-loader!./code/10_platform_detection.py';
1012
import ApiLink from '@site/src/components/ApiLink';
1113

1214
The <ApiLink to="class/Actor">`Actor`</ApiLink> class is configured through the <ApiLink to="class/Configuration">`Configuration`</ApiLink> class, which reads its settings from environment variables. When running on the Apify platform or through the Apify CLI, configuration is automatic — manual setup is only needed for custom requirements.
@@ -33,4 +35,20 @@ This Actor run will not persist its local storages to the filesystem:
3335
APIFY_PERSIST_STORAGE=0 apify run
3436
```
3537

36-
For the full list of configuration options, see the <ApiLink to="class/Configuration">`Configuration`</ApiLink> API reference.
38+
## Reading the runtime environment
39+
40+
The <ApiLink to="class/Actor#get_env">`Actor.get_env`</ApiLink> method returns a dictionary with all `APIFY_*` environment variables parsed into their typed values. This is useful for inspecting the Actor's runtime context, such as the Actor ID, run ID, or default storage IDs. Variables that are not set or are invalid will have a value of `None`.
41+
42+
<RunnableCodeBlock className="language-python" language="python">
43+
{GetEnvExample}
44+
</RunnableCodeBlock>
45+
46+
## Platform detection
47+
48+
The <ApiLink to="class/Actor#is_at_home">`Actor.is_at_home`</ApiLink> method returns `True` when the Actor is running on the Apify platform, and `False` when running locally. This is useful for branching behavior based on the environment, such as using different storage backends or skipping proxy configuration during local development.
49+
50+
<RunnableCodeBlock className="language-python" language="python">
51+
{PlatformDetectionExample}
52+
</RunnableCodeBlock>
53+
54+
For the full list of configuration options, see the <ApiLink to="class/Configuration">`Configuration`</ApiLink> API reference. For a complete list of environment variables available on the platform, see the [environment variables](https://docs.apify.com/platform/actors/development/programming-interface/environment-variables) documentation.

0 commit comments

Comments
 (0)