Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .github/musl_build.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/bin/bash

set -e

if [ -z "$1" ]; then
echo "Usage: $0 <target> [maturin_args]"
exit 1
Expand All @@ -9,8 +11,15 @@ TARGET=$1
ARGS=$2

IMAGE="ghcr.io/0x676e67/rust-musl-cross"
VOLUME_MAPPING="-v $(pwd):/home/rust/src"
VOLUME_MAPPING=("-v" "$(pwd):/home/rust/src")
MATURIN_CMD="maturin build --release --out dist $ARGS"
EXTRA_ENV=()

for var in BORING_BSSL_RUST_CPPLIB MATURIN_VERSION CFLAGS CXXFLAGS LDFLAGS RUSTFLAGS; do
if [ -n "${!var}" ]; then
EXTRA_ENV+=("-e" "$var=${!var}")
fi
done

case $TARGET in
x86_64-unknown-linux-musl | \
Expand All @@ -26,6 +35,6 @@ esac

echo "Building for $TARGET..."
docker pull $IMAGE:$TARGET
docker run --rm $VOLUME_MAPPING $IMAGE:$TARGET /bin/bash -c "$MATURIN_CMD"
docker run --rm "${VOLUME_MAPPING[@]}" "${EXTRA_ENV[@]}" $IMAGE:$TARGET /bin/bash -c "$MATURIN_CMD"

echo "Build completed for target: $TARGET"
21 changes: 14 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ jobs:
custom_env:
CC: i686-linux-gnu-gcc
CXX: i686-linux-gnu-g++
CFLAGS: -msse2
CXXFLAGS: -msse2
CFLAGS_i686_unknown_linux_gnu: -msse2
CXXFLAGS_i686_unknown_linux_gnu: -msse2
CARGO_TARGET_I686_UNKNOWN_LINUX_GNU_LINKER: i686-linux-gnu-g++
- runner: ubuntu-latest
target: aarch64
Expand All @@ -127,10 +131,6 @@ jobs:
- uses: actions/setup-python@v6
with:
python-version: 3.x
- name: Install LLVM and Clang
uses: KyleMayes/install-llvm-action@v2
with:
version: "13.0"
- name: Install target-specific APT dependencies
if: "matrix.platform.apt_packages != ''"
run: sudo apt-get update && sudo apt-get install -y ${{ matrix.platform.apt_packages }}
Expand Down Expand Up @@ -177,9 +177,16 @@ jobs:
steps:
- uses: actions/checkout@v6
- name: Build wheels
run: bash .github/musl_build.sh ${{ matrix.platform.target }} "${{ matrix.build_type.maturin_args }}" --features "${{ matrix.platform.allocator }}"
env:
MATURIN_VERSION: ${{ env.MATURIN_VERSION }}
BORING_BSSL_RUST_CPPLIB: static=stdc++
run: |
export RUSTFLAGS="${RUSTFLAGS:+$RUSTFLAGS }-L native=/usr/local/musl/${{ matrix.platform.target }}/lib -L native=/usr/local/musl/lib/gcc/${{ matrix.platform.target }}/11.4.0 -C link-arg=-static-libgcc -l static=gcc"
if [ "${{ matrix.platform.target }}" = "i686-unknown-linux-musl" ]; then
export CFLAGS="${CFLAGS} -msse2"
export CXXFLAGS="${CXXFLAGS} -msse2"
fi
bash .github/musl_build.sh ${{ matrix.platform.target }} "${{ matrix.build_type.maturin_args }}" --features "${{ matrix.platform.allocator }}"
- name: Upload wheels
uses: actions/upload-artifact@v7
with:
Expand Down Expand Up @@ -242,10 +249,10 @@ jobs:
- name: free-threaded
maturin_args: "--interpreter 3.13t 3.14t"
platform:
- runner: macos-14
- runner: macos-latest
target: x86_64
allocator: jemalloc
- runner: macos-14
- runner: macos-latest
target: aarch64
allocator: jemalloc
steps:
Expand Down
6 changes: 4 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ wreq = { version = "6.0.0-rc.27", features = [
"deflate",
"zstd",
"ws",
"parking_lot",
] }
wreq-util = { version = "3.0.0-rc.10", features = ["emulation-rand", "emulation-compression"] }
wreq-util = { version = "3.0.0-rc.10", features = ["emulation-compression"] }
hickory-resolver = "0.25.2"
cookie = "0.18"
mimalloc = { version = "0.1.43", default-features = false, features = [
Expand All @@ -86,4 +87,5 @@ panic = "abort"
strip = true

[patch.crates-io]
wreq = { git = "https://github.com/0x676e67/wreq", rev = "240d84eab5eb4df8548933ee2c13337d86e1afe1" }
wreq = { git = "https://github.com/0x676e67/wreq" }
wreq-util = { git = "https://github.com/0x676e67/wreq-util" }
4 changes: 3 additions & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ markdown_extensions:
- pymdownx.details
- attr_list
- md_in_html
- pymdownx.emoji
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
- pymdownx.tasklist
- pymdownx.caret
- pymdownx.mark
Expand Down
97 changes: 77 additions & 20 deletions docs/source/getting-started/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,26 +55,83 @@ Due to the complexity of **TLS** encryption and the widespread adoption of **HTT

**TLS** and **HTTP/2** fingerprints are often identical across various browser models because these underlying protocols evolve slower than browser release cycles. **100+ browser device emulation profiles** are maintained in **[wreq-python]**.

??? note "Available OS emulations"

| **OS** | **Description** |
| ----------- | ------------------------------ |
| **Windows** | Windows (any version) |
| **MacOS** | macOS (any version) |
| **Linux** | Linux (any distribution) |
| **Android** | Android (mobile) |
| **iOS** | iOS (iPhone/iPad) |

??? note "Available browser emulations"

| **Browser** | **Versions** |
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Chrome** | `Chrome100`, `Chrome101`, `Chrome104`, `Chrome105`, `Chrome106`, `Chrome107`, `Chrome108`, `Chrome109`, `Chrome110`, `Chrome114`, `Chrome116`, `Chrome117`, `Chrome118`, `Chrome119`, `Chrome120`, `Chrome123`, `Chrome124`, `Chrome126`, `Chrome127`, `Chrome128`, `Chrome129`, `Chrome130`, `Chrome131`, `Chrome132`, `Chrome133`, `Chrome134`, `Chrome135`, `Chrome136`, `Chrome137`, `Chrome138`, `Chrome139`, `Chrome140`, `Chrome141`, `Chrome142`, `Chrome143`, `Chrome144`, `Chrome145` |
| **Safari** | `SafariIos17_2`, `SafariIos17_4_1`, `SafariIos16_5`, `Safari15_3`, `Safari15_5`, `Safari15_6_1`, `Safari16`, `Safari16_5`, `Safari17_0`, `Safari17_2_1`, `Safari17_4_1`, `Safari17_5`, `Safari18`, `SafariIPad18`, `Safari18_2`, `SafariIos18_1_1`, `Safari18_3`, `Safari18_3_1`, `Safari18_5`, `Safari26`, `Safari26_1`, `Safari26_2`, `SafariIos26`, `SafariIos26_2`, `SafariIPad26`, `SafariIPad26_2` |
| **Firefox** | `Firefox109`, `Firefox117`, `Firefox128`, `Firefox133`, `Firefox135`, `FirefoxPrivate135`, `FirefoxAndroid135`, `Firefox136`, `FirefoxPrivate136`, `Firefox139`, `Firefox142`, `Firefox143`, `Firefox144`, `Firefox145`, `Firefox146`, `Firefox147` |
| **OkHttp** | `OkHttp3_9`, `OkHttp3_11`, `OkHttp3_13`, `OkHttp3_14`, `OkHttp4_9`, `OkHttp4_10`, `OkHttp4_12`, `OkHttp5` |
| **Edge** | `Edge101`, `Edge122`, `Edge127`, `Edge131`, `Edge134`, `Edge135`, `Edge136`, `Edge137`, `Edge138`, `Edge139`, `Edge140`, `Edge141`, `Edge142`, `Edge143`, `Edge144`, `Edge145` |
| **Opera** | `Opera116`, `Opera117`, `Opera118`, `Opera119`
???+ note "Available OS emulations"

<div class="grid cards" markdown>

- :fontawesome-brands-windows: **Windows**

---

Windows (any version)

- :fontawesome-brands-apple: **macOS**

---

macOS (any version)

- :fontawesome-brands-linux: **Linux**

---

Linux (any distribution)

- :fontawesome-brands-android: **Android**

---

Android (mobile)

- :fontawesome-brands-apple: **iOS**

---

iOS (iPhone / iPad)

</div>

???+ note "Available browser emulations"

<div class="grid cards" markdown>

- :fontawesome-brands-chrome: **Chrome**

---

`Chrome100` `Chrome101` `Chrome104` `Chrome105` `Chrome106` `Chrome107` `Chrome108` `Chrome109` `Chrome110` `Chrome114` `Chrome116` `Chrome117` `Chrome118` `Chrome119` `Chrome120` `Chrome123` `Chrome124` `Chrome126` `Chrome127` `Chrome128` `Chrome129` `Chrome130` `Chrome131` `Chrome132` `Chrome133` `Chrome134` `Chrome135` `Chrome136` `Chrome137` `Chrome138` `Chrome139` `Chrome140` `Chrome141` `Chrome142` `Chrome143` `Chrome144` `Chrome145` `Chrome146` `Chrome147`

- :fontawesome-brands-edge: **Edge**

---

`Edge101` `Edge122` `Edge127` `Edge131` `Edge134` `Edge135` `Edge136` `Edge137` `Edge138` `Edge139` `Edge140` `Edge141` `Edge142` `Edge143` `Edge144` `Edge145`

- :fontawesome-brands-firefox: **Firefox**

---

`Firefox109` `Firefox117` `Firefox128` `Firefox133` `Firefox135` `FirefoxPrivate135` `FirefoxAndroid135` `Firefox136` `FirefoxPrivate136` `Firefox139` `Firefox142` `Firefox143` `Firefox144` `Firefox145` `Firefox146` `Firefox147` `Firefox148` `Firefox149`

- :fontawesome-brands-safari: **Safari**

---

`SafariIos17_2` `SafariIos17_4_1` `SafariIos16_5` `Safari15_3` `Safari15_5` `Safari15_6_1` `Safari16` `Safari16_5` `Safari17_0` `Safari17_2_1` `Safari17_4_1` `Safari17_5` `Safari18` `SafariIPad18` `Safari18_2` `Safari18_3` `Safari18_3_1` `SafariIos18_1_1` `Safari18_5` `Safari26` `Safari26_1` `Safari26_2` `SafariIos26` `SafariIos26_2` `SafariIPad26` `SafariIpad26_2`

- :fontawesome-brands-opera: **Opera**

---

`Opera116` `Opera117` `Opera118` `Opera119` `Opera120` `Opera121` `Opera122` `Opera123` `Opera124` `Opera125` `Opera126` `Opera127` `Opera128` `Opera129` `Opera130`

- :material-web: **OkHttp**

---

`OkHttp3_9` `OkHttp3_11` `OkHttp3_13` `OkHttp3_14` `OkHttp4_9` `OkHttp4_10` `OkHttp4_12` `OkHttp5`

</div>


## Performance
Expand Down
8 changes: 4 additions & 4 deletions docs/source/guide/emulation.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,16 @@ if __name__ == "__main__":
```python
import asyncio
from wreq import Client
from wreq.emulation import Emulation, EmulationOS, EmulationOption
from wreq.emulation import Emulation, Profile, Platform


async def main():
client = Client()
resp = await client.get(
"https://tls.peet.ws/api/all",
emulation=EmulationOption(
emulation=Emulation.Chrome134,
emulation_os=EmulationOS.Android,
emulation=Emulation(
profile=Profile.Chrome134,
platform=Platform.Android,
),
# Disable client default headers
default_headers=False,
Expand Down
6 changes: 3 additions & 3 deletions docs/source/guide/websocket.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import asyncio
import datetime
import wreq
from wreq import Message, WebSocket
from wreq import Message, WebSocket, Version
from wreq import exceptions


Expand Down Expand Up @@ -95,11 +95,11 @@ async def recv_message(ws):


async def main():
# Connect to HTTP/2 WebSocket server with force_http2=True
# Connect to HTTP/2 WebSocket server
client = wreq.Client(verify=False)
ws: WebSocket = await client.websocket(
"wss://127.0.0.1:3000/ws",
force_http2=True
version=Version.HTTP_2
)
async with ws:
print("Status Code: ", ws.status)
Expand Down
8 changes: 4 additions & 4 deletions examples/emulation.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import asyncio
from wreq import Client, Response
from wreq.emulation import Emulation, EmulationOS, EmulationOption
from wreq.emulation import Emulation, Profile, Platform
from wreq.tls import TlsOptions, TlsVersion, AlpnProtocol
from wreq.http2 import Http2Options, PseudoId, PseudoOrder
from wreq.header import HeaderMap, OrigHeaderMap
Expand Down Expand Up @@ -49,9 +49,9 @@ async def request_chrome_android(client: Client):
print("\n[Testing Chrome on Android Emulation]")
resp = await client.get(
"https://tls.peet.ws/api/all",
emulation=EmulationOption(
emulation=Emulation.Chrome134,
emulation_os=EmulationOS.Android,
emulation=Emulation(
profile=Profile.Chrome134,
platform=Platform.Android,
),
# Disable client default headers
default_headers=False,
Expand Down
6 changes: 4 additions & 2 deletions examples/http2_websocket.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import datetime
import signal
import wreq
from wreq import Message, WebSocket
from wreq import Message, WebSocket, Version
from wreq import exceptions


Expand Down Expand Up @@ -45,7 +45,9 @@ async def recv_message(ws):

async def main():
client = wreq.Client(tls_verify=False)
ws: WebSocket = await client.websocket("wss://127.0.0.1:3000/ws", force_http2=True)
ws: WebSocket = await client.websocket(
"wss://127.0.0.1:3000/ws", version=Version.HTTP_2
)
async with ws:
print("Status Code: ", ws.status)
print("Version: ", ws.version)
Expand Down
19 changes: 3 additions & 16 deletions python/wreq/blocking.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,22 +114,9 @@ def close(self) -> None:
r"""
Close the response.

**Current behavior:**

- When connection pooling is **disabled**: This method closes the network connection.
- When connection pooling is **enabled**: This method closes the response, prevents further body reads,
and returns the connection to the pool for reuse.

**Future changes:**

In future versions, this method will be changed to always close the network connection regardless of
whether connection pooling is enabled or not.

**Recommendation:**

It is **not recommended** to manually call this method at present. Instead, use context managers
(with statement) to properly manage response lifecycle. Wait for the improved implementation
in future versions.
This method closes the network connection regardless of whether connection pooling is
enabled or not. It is recommended to use context managers (`with` statement) to properly
manage response lifecycle instead of calling this method manually.
"""

def __enter__(self) -> Any: ...
Expand Down
Loading
Loading