Skip to content

Add wrangler airtable sync via dashbored - #64

Open
autowattage wants to merge 1 commit into
hackclub:mainfrom
autowattage:dashbored/add-wrangler
Open

Add wrangler airtable sync via dashbored#64
autowattage wants to merge 1 commit into
hackclub:mainfrom
autowattage:dashbored/add-wrangler

Conversation

@autowattage

Copy link
Copy Markdown

add wrangler to the warehouse (airtable).

tables: ysws_project_submission, shop_queue, currency, ysws_config
sync: full-refresh via DLT

field IDs for the new base are included in generated_ids.py, read from the base schema at wizard time. sync uses the existing AIRTABLE_PERSONAL_ACCESS_TOKEN, which needs access to this base before the first materialization.

hackatime claims CTE (draft, needs review)
wrangler_ht_claims AS (
    SELECT 'wrangler'::text AS program_name,
        CASE WHEN POSITION('@' IN LOWER(BTRIM(hp."email"))) > 0
             THEN SPLIT_PART(SPLIT_PART(LOWER(BTRIM(hp."email")), '@', 1), '+', 1)
                  || '@' || SPLIT_PART(LOWER(BTRIM(hp."email")), '@', 2)
             ELSE SPLIT_PART(LOWER(BTRIM(hp."email")), '+', 1)
        END AS user_email,
        LOWER(BTRIM(hp."linked_lapse_lookout_hackatime_links")) AS hackatime_alias,
        NULL::text AS project_name,
        NULL::text AS code_url,
        TIMESTAMP WITH TIME ZONE '2026-08-10 00:00:00+00' AS claim_start_ts
    FROM {{ source('airtable_wrangler', 'ysws_project_submission') }} hp
    WHERE hp."linked_lapse_lookout_hackatime_links" IS NOT NULL AND hp."linked_lapse_lookout_hackatime_links" <> ''
),

via dashbored

NULL::text AS code_url,
TIMESTAMP WITH TIME ZONE '2026-08-10 00:00:00+00' AS claim_start_ts
FROM {{ source('airtable_wrangler', 'ysws_project_submission') }} hp
WHERE hp."linked_lapse_lookout_hackatime_links" IS NOT NULL AND hp."linked_lapse_lookout_hackatime_links" <> ''

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linked_lapse_lookout_hackatime_links field contains full URLs (e.g. https://hackatime.hackclub.com/@user/project/Name+Here), not bare project aliases — and some rows have multiple, newline-separated. The current CTE passes the whole URL string as the alias, which will never match project_name in the warehouse → 0 DAU.

Suggested change
WHERE hp."linked_lapse_lookout_hackatime_links" IS NOT NULL AND hp."linked_lapse_lookout_hackatime_links" <> ''
LOWER(REPLACE(BTRIM(SPLIT_PART(url, '/project/', 2)), '+', ' ')) AS hackatime_alias,
NULL::text AS project_name,
NULL::text AS code_url,
TIMESTAMP WITH TIME ZONE '2026-08-10 00:00:00+00' AS claim_start_ts
FROM {{ source('airtable_wrangler', 'ysws_project_submission') }} hp
CROSS JOIN LATERAL unnest(string_to_array(hp."linked_lapse_lookout_hackatime_links", E'\n')) AS url
WHERE url LIKE '%/project/%'

Splits on newlines, extracts the segment after /project/, decodes + → space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants