Skip to content

Latest commit

ย 

History

History
47 lines (37 loc) ยท 2.8 KB

async-tps.md

File metadata and controls

47 lines (37 loc) ยท 2.8 KB

๋น„๋™๊ธฐ API์™€ ๊ณตํ†ต TPS

KmaClient, DataGoKrClient, ApiHubClient, ApiHubGeneratedClient๋Š” ๋น„๋™๊ธฐ ์ „์šฉ์ด๋‹ค. ๋„คํŠธ์›Œํฌ ํ˜ธ์ถœยท๋””๋ฒ„๊ทธ์—๋Š” await, ํŽ˜์ด์ง€ ์ˆœํšŒ์—๋Š” async for, ์ข…๋ฃŒ์—๋Š” async with ๋˜๋Š” await client.aclose()๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค. Async ์ ‘๋‘์‚ฌ ํด๋ž˜์Šค์™€ aio ํŒฉํ„ฐ๋ฆฌ, a ์ ‘๋‘์‚ฌ ๋ฉ”์„œ๋“œ๋Š” ์ œ๊ฑฐํ–ˆ๋‹ค. ๊ธฐ์กด ์ผ๋ฐ˜ ๋ฉ”์„œ๋“œ ์ด๋ฆ„์„ ๊ทธ๋Œ€๋กœ awaitํ•œ๋‹ค. ForecastService์˜ ํ˜ธ์ถœ๊ณผ now/short/vilage/version๋„ async๋‹ค.

import asyncio
from kma import AsyncTokenBucket, KmaClient, DataGoKrClient, latest_mid_fcst_time


async def main() -> None:
    budget = AsyncTokenBucket(2, capacity=1)
    async with (
        KmaClient.from_env(rate_limiter=budget) as weather,
        DataGoKrClient.from_env(rate_limiter=budget) as client,
    ):
        print(await weather.now(nx=60, ny=127))
        async for page in (client.iter_pages(
            "MidFcstInfoService", "getMidFcst", {"stnId": "108", "tmFc": latest_mid_fcst_time()}, num_of_rows=10,
        )):
            print(page)


asyncio.run(main())

๊ธฐ๋ณธ max_rps=5.0, ๊ธฐ๋ณธ capacity=max(1, max_rps)๋Š” ์ดˆ๊ธฐ burst๋ฅผ ํ—ˆ์šฉํ•œ๋‹ค. ์ผ์ •ํ•œ ์†ก์‹  ๊ฐ„๊ฒฉ์€ capacity=1์„ ์ง€์ •ํ•œ๋‹ค. rate_limiter๋ฅผ ์ฃผ์ž…ํ•˜๋ฉด max_rps๋ณด๋‹ค ์šฐ์„ ํ•˜๋ฉฐ, ์—ฌ๋Ÿฌ ํด๋ผ์ด์–ธํŠธ๊ฐ€ ๊ฐ™์€ ๋ฒ„ํ‚ท์„ ๊ณต์œ ํ•˜๋ฉด ํ•ฉ์‚ฐ TPS๊ฐ€ ์ œํ•œ๋œ๋‹ค. ํ•œ ์ด๋ฒคํŠธ ๋ฃจํ”„์—์„œ ์‚ฌ์šฉํ•˜๋ฉฐ, ๋Œ€๊ธฐ ์ทจ์†Œ๋Š” ํ† ํฐ์„ ์†Œ๋น„ํ•˜์ง€ ์•Š๋Š”๋‹ค. ๊ณต๊ธ‰์ž์˜ ์ผ์ผ ์‚ฌ์šฉ๋Ÿ‰ ์ œํ•œ์€ ๋ณ„๋„๋‹ค.

์ผ๋ฐ˜ยทํƒ€์ž…ํ™” ์กฐํšŒ, ๋””๋ฒ„๊ทธ, ํŽ˜์ด์ง€, APIHub 470๊ฐœ ์ƒ์„ฑ ๋ฉ”์„œ๋“œ, ํƒ์ƒ‰ ํ˜ธ์ถœ์€ ๊ฐ™์€ ๊ฒฝ๋กœ๋ฅผ ์“ด๋‹ค. ์ฒซ ์†ก์‹ ยท์žฌ์‹œ๋„ยท๊ฐ ๋ฆฌ๋‹ค์ด๋ ‰ํŠธ๋งˆ๋‹ค ํ•œ ํ† ํฐ์„ ์†Œ๋น„ํ•œ๋‹ค. ๊ธฐ์กด 429/500/502/503/504 ๋ฐ ์ผ์‹œ์ ์ธ ์—ฐ๊ฒฐยทtimeout ์žฌ์‹œ๋„์™€ jitter/Retry-After๋ฅผ ์œ ์ง€ํ•˜๋ฉฐ, 401/403๊ณผ resultCode=22๋Š” ์ฆ‰์‹œ ์‹คํŒจํ•œ๋‹ค. HTTPX์˜ BasicAuth/๊ธฐ๋ณธ ์ธ์ฆ ํ๋ฆ„์„ ์ง€์›ํ•˜๊ณ  ์ถ”๊ฐ€ ์†ก์‹ ์„ ์ˆจ๊ธฐ๋Š” Digest/custom Auth๋Š” ๊ฑฐ๋ถ€ํ•œ๋‹ค. ์‚ฌ์šฉ์ž ์ •์˜ transport ๋‚ด๋ถ€์˜ ์ถ”๊ฐ€ ์†ก์‹ ์€ ํ•ด๋‹น transport์˜ ์ฑ…์ž„์ด๋‹ค.

๋‚ด๋ถ€ ์„ธ์…˜์€ ์ฒซ ์š”์ฒญ์— ๋งŒ๋“ค๊ณ  ์žฌ์‚ฌ์šฉํ•˜๋ฉฐ aclose๊ฐ€ ๋‹ซ๋Š”๋‹ค. ์ฃผ์ž…ํ•œ ๋น„๋™๊ธฐ ์„ธ์…˜์€ ํ˜ธ์ถœ์ž๊ฐ€ ๋‹ซ๋Š”๋‹ค. ํ† ํฐ ๋Œ€๊ธฐ ์ค‘ ์ข…๋ฃŒ๋˜๊ฑฐ๋‚˜ ์ธ์ฆ ํ๋ฆ„์ด ๋ฐ”๋€Œ๋ฉด ์†ก์‹  ์ „์— ๋‹ค์‹œ ๊ฒ€์‚ฌํ•œ๋‹ค. ๊ณต๊ฐœ ์˜ค๋ฅ˜์™€ DebugRun์˜ ๋ชจ๋“  ํ•„๋“œ์—์„œ ์‹ค์ œ ํ‚ค์™€ ์ธ์ฝ”๋”ฉ๋œ ํ‚ค๋ฅผ ๋งˆ์Šคํ‚นํ•œ๋‹ค.

์ผ๋ฐ˜ ํ•จ์ˆ˜๋กœ ๋‚จ๋Š” ๊ฒƒ์€ ์ขŒํ‘œยท์‹œ๊ฐ„ยท์ฝ”๋“œํ‘œยท๋ชจ๋ธยท์นดํƒˆ๋กœ๊ทธยท์‘๋‹ต ํŒŒ์‹ฑยทfixture ์ €์žฅ ๋“ฑ ๋กœ์ปฌ ์ž‘์—…์ด๋‹ค. JSON/XML NO_DATA03 ๋นˆ ๊ฒฐ๊ณผ, quota22 ๋น„์žฌ์‹œ๋„, tmFc ์š”์ฒญ๊ฐ’ ํด๋ฐฑ, PCP/SNO ๋ฒ”์ฃผ ๋ฌธ์ž์—ด, ํŽ˜์ด์ง€ ์ƒํ•œ ๊ฒฝ๊ณ ์™€ APIHub bare query ์ˆœ์„œ๋ฅผ ์œ ์ง€ํ•œ๋‹ค. CLI์™€ Streamlit์€ ์ง„์ž… ๊ฒฝ๊ณ„์—์„œ๋งŒ asyncio.run์„ ์‚ฌ์šฉํ•˜๊ณ  ์ข…๋ฃŒ ์‹œ ์„ธ์…˜์„ ๋‹ซ๋Š”๋‹ค. ์ƒ์„ฑ๊ธฐ ์—ญ์‹œ ๋น„๋™๊ธฐ HTTP์™€ ๊ณตํ†ต TPS๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค.