Skip to content

README.md

Latest commit

 

History

History
387 lines (286 loc) · 9.55 KB

README.md

File metadata and controls

387 lines (286 loc) · 9.55 KB

Bayut Scraper API - Python

A powerful Python scraper for extracting UAE real estate data from Bayut. Search properties in Dubai, Abu Dhabi, Sharjah, and all UAE emirates. Get property details, agent profiles, agency data, and TruBroker verified agents using the Bayut Scraper API by RealtyAPI.

Features

  • Property Search - Search for-sale and for-rent listings with 20 filters (price, rooms, area, furnishing, category, and more)
  • Autocomplete - Get location IDs for any UAE location (Dubai Marina, Palm Jumeirah, etc.)
  • Property Details - Full property data by external ID
  • Agency Details - Agency profiles with agent listings
  • Agent Profiles - Individual agent details by slug
  • Agents by Agency - List all agents in an agency
  • Agent Deals - Claimed deals count and total price
  • Agent Stories - Agent posts and content
  • TruBroker Search - Find TruBroker verified agents by location
  • Property Types - Apartments, Villas, Penthouses, Townhouses, Commercial, Offices, and more
  • Off-Plan Search - Filter under-construction and completed properties
  • Bulk Mode - Process hundreds of properties from input.json with progress tracking
  • Export to JSON or CSV - Save results in your preferred format

Quick Start

1. Get Your API Key

Sign up at realtyapi.io to get your API key.

2. Install

git clone https://github.com/realtyapi/bayut-scraper-python.git
cd bayut-scraper-python
pip install -r requirements.txt

3. Run the Interactive Scraper

python scraper.py

4. Or Use the API Directly in Your Code

import requests

url = "https://bayut.realtyapi.io/search/property"

params = {
    "location_external_id": "6901",
    "purpose": "for-sale",
    "category": "residential",
    "rooms": "2"
}

headers = {
    "x-realtyapi-key": "YOUR_API_KEY"
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

Usage Examples

Autocomplete (Get Location IDs)

from bayut_api import BayutAPI

api = BayutAPI("YOUR_API_KEY")

# Get location_external_id for use in property search
data = api.autocomplete("dubai marina")
print(data)

Search Properties for Sale

data = api.search_property(
    location_external_id="6901",  # Dubai Marina
    purpose="for-sale",
    category="residential",
    rooms="2",
    min_price="500000",
    max_price="3000000",
)
print(data)

Search Rentals

data = api.search_property(
    location_external_id="6901",
    purpose="for-rent",
    rent_frequency="yearly",
    rooms="1",
    furnishing_status="furnished",
    max_price="100000",
)
print(data)

Search Villas

data = api.search_property(
    location_external_id="6901",
    purpose="for-sale",
    category_external_id="3",  # Villas
    min_price="1000000",
)
print(data)

Search Commercial Properties

data = api.search_property(
    location_external_id="6901",
    purpose="for-rent",
    category="commercial",
    category_external_id="5",  # Office
)
print(data)

Search Off-Plan / Under Construction

data = api.search_property(
    location_external_id="6901",
    purpose="for-sale",
    completion_status="under-construction",
)
print(data)

Property Details

# Get externalID from search results
data = api.get_property_details("12372178")
print(data)

Agency Details

data = api.get_agency_details("10499")
print(data)

Agent Details

data = api.get_agent_details("salma-mohamed-mahmoud-hashem-2521536")
print(data)

Agents by Agency

data = api.get_agents_by_agency("10499")
print(data)

Agent Claimed Deals

# Deal count
count = api.get_agent_claimed_deals_count("1919589")
print(count)

# Total deal value
total = api.get_agent_claimed_deals_total_price("1919589")
print(total)

Agent Stories

data = api.get_agent_stories("1919589")
print(data)

TruBroker Agent Search

data = api.search_trubroker_agents(
    location_external_id="5152",
    category="residential",
    purpose="for-sale",
)
print(data)

Scraper Modes

Test Mode (Single Input)

Enter one input manually. Shows a preview (first 1000 characters), then asks to save as JSON or CSV.

Bulk Mode (from input.json)

Process hundreds of inputs from input.json. Shows a progress bar, saves all results at the end.

  [========================------] 80% (8/10) {"location_external_id": "6901",

Bulk Mode - input.json Format

  1. Copy input_sample.json to input.json
  2. Fill in only the sections you need
  3. Run python scraper.py, pick your operation, select "Bulk run"

input.json Structure

{
  "autocomplete_queries": ["dubai marina", "palm jumeirah"],

  "property_searches": [
    {
      "location_external_id": "6901",
      "purpose": "for-sale",
      "rooms": "2",
      "min_price": "500000",
      "max_price": "3000000"
    },
    "6901"
  ],

  "property_ids": ["12372178"],

  "agency_ids": ["10499"],

  "agent_slugs": ["salma-mohamed-mahmoud-hashem-2521536"],

  "agent_ids": ["1919589"],

  "trubroker_searches": [
    {"location_external_id": "5152", "category": "residential", "purpose": "for-sale"}
  ]
}

Output Formats

Option Format Best for
1 JSON (.json) Full nested data, programmatic access
2 CSV (.csv) Spreadsheets, Excel, data analysis
3 Both Keep JSON for detail + CSV for quick review

All output is saved to the output/ directory.

API Configuration

Option 1: Environment Variable

export REALTYAPI_KEY=rt_your_key_here
python scraper.py

Option 2: Config File

{
  "api_key": "rt_your_key_here"
}

Option 3: Direct in Code

from bayut_api import BayutAPI
api = BayutAPI("rt_your_key_here")

Search Filter Reference

Purpose

Value Description
for-sale Properties for sale
for-rent Properties for rent

Category

Value Description
residential Residential properties
commercial Commercial properties

Rent Frequency (for-rent only)

monthly, yearly, weekly, daily

Furnishing Status

furnished, unfurnished

Completion Status (for-sale only)

completed, under-construction

Product (Featured Listings)

superhot, hot

Property Type (category_external_id)

ID Type ID Type
4 Apartment 5 Office
3 Villas 6 Shop
16 Townhouses 7 Warehouse
18 Penthouses 9 Labour Camp
21 Hotel Apartments 25 Commercial Villa
19 Villa Compound 20 Bulk Units
14 Residential Plot 15 Commercial Plot
12 Residential Floor 13 Commercial Floor
17 Residential Building 10 Commercial Building
8 Factory 22 Industrial Land
23 Mixed Use Land 24 Showroom
11 Other Commercial

All Available Endpoints

Category Endpoint Description
Search /search/property Property search (20 filters)
Search /autocomplete Location autocomplete
Details /details/property Full property details
Agent /details/agency Agency profile and agents
Agent /details/agent Agent profile by slug
Agent /agent/agency All agents in an agency
Agent /agent/claimedDeals/count Agent deal count
Agent /agent/claimedDeals/totalPrice Agent total deal value
Agent /stories/agent Agent stories/posts
TruBroker /truBroker/agent Verified agent search

File Structure

bayut-scraper-python/
├── scraper.py          # Interactive CLI (test one or bulk run)
├── bayut_api.py        # Bayut API client library (10 endpoints)
├── config.py           # API key management
├── examples.py         # Code examples for every endpoint
├── input_sample.json   # Template for bulk input (copy to input.json)
├── input.json          # Your bulk input data (gitignored)
├── requirements.txt    # Python dependencies
├── config.json         # Your API key (auto-created, gitignored)
├── output/             # Saved JSON/CSV outputs (auto-created)
└── README.md           # This file

Requirements

  • Python 3.7+
  • requests library

Get Your API Key

Sign up at realtyapi.io to get your Bayut Scraper API key.

Related APIs

RealtyAPI also offers scraper APIs for:

License

MIT

Keywords: bayut scraper, bayut api, bayut data scraper, bayut property data, bayut scraper python, bayut listing scraper, bayut real estate api, scrape bayut, bayut scraping tool, bayut api python, bayut property scraper, bayut dubai data, bayut rental data, bayut uae scraper, dubai real estate scraper, dubai property api, uae real estate api, dubai apartment scraper, dubai rental scraper, dubai property data api, abu dhabi property data, sharjah real estate, bayut search api, bayut agent api, bayut trubroker api, bayut villa data, bayut off plan, bayut commercial property, bayut automation, bayut data mining, bayut web scraper, bayut crawler, bayut property lookup, dubai housing data, uae property scraper, palm jumeirah data, realtyapi, realty api