How this thing actually works

Look, most “how it works” pages are marketing fluff wrapped in gradients. This one is not. This is me explaining what happens from the moment you paste your domain to the moment a post goes live on your site. Every API, every model, every cron job. No hand waving, no “proprietary AI” nonsense. Just the actual stack.

If you are a developer evaluating whether this thing is legit or just another wrapper, this page is for you. If you are not technical, it still might be interesting. Or it might be boring as shit. Either way everything here is real.

Step 1. We scrape your website

When you paste your URL we need to understand what your business actually does. So we scrape the homepage. The primary path uses Bright Data Web Unlocker, which is a proxy service that handles JavaScript rendering, CAPTCHAs, and all the crap that makes scraping hard. We POST to their API with your URL and get back raw HTML.

If Bright Data is not configured or the response is too thin (less than 300 characters of extracted text, which means the page is probably behind a paywall or returned garbage) we fall back to a plain fetch with a custom user agent. No headless browser, no Puppeteer, just a fetch call with a 12 second timeout.

The HTML gets stripped down with regex. Scripts, styles, SVGs, and comments get nuked. Block tags become newlines. HTML entities get decoded. The whole thing is truncated to 12,000 characters because anything longer is just footer links and legal text that add nothing.

That cleaned up text plus the page title and meta description get sent to Claude Haiku 4.5 running on AWS Bedrock. Haiku reads the content and extracts a structured brand profile: your company name, what you do, who your audience is, your tone of voice, your core topics, and your products. This is a single cheap model call. The output is a typed Zod schema so we get structured JSON back, not free text we have to parse.

Step 2. Keyword research

This is where it gets interesting. We do not just ask an AI to guess keywords. That approach sucks because AI models have no idea what people actually search for in 2025. They hallucinate search volumes and invent keywords nobody has ever typed into Google.

Instead we use SE Ranking, which is a real SEO data provider with actual search volume data from clickstream panels and Google Ads integration. The pipeline works in two paths.

The good path (when SE Ranking discovery is configured): First, Haiku extracts 3 to 5 seed topics from your brand profile. Short noun phrases like “email marketing” or “crm software”. Then for each seed we hit two SE Ranking endpoints in parallel: /v1/keywords/related and /v1/keywords/questions. Both return real keywords that real people search for, with real monthly search volumes. We filter server side for volume of at least 50, difficulty under 40, and at least 3 words (because head terms are useless for a small site trying to rank).

SE Ranking charges 10 credits per returned keyword. We hard cap at 280 credits per discovery run, which means a maximum of 28 keywords across all endpoints. The budget gets divided evenly across requests so there is no way to accidentally blow past it.

All discovered keywords go through a two level deduplication: exact match (case insensitive, stripped punctuation) and a similarity signature that collapses near duplicates by stemming significant words and comparing the sorted set. So “best email marketing tools” and “best tools for email marketing” collapse to one entry, and we keep the version with higher volume.

Then we score every keyword for winnability. The scoring function rewards search demand (log scaled, so a 5000 volume term does not drown out a solid 200 volume one) and penalizes difficulty. A low difficulty keyword with decent volume beats a high difficulty vanity term every time. This is how we pick the 12 best keywords from the pool.

The fallback path (when SE Ranking is not configured): Haiku generates a pool of 40 keyword candidates from your brand profile. The system prompt is extremely detailed about what makes a good long tail keyword and forces variety across categories like how to, comparison, alternative, question, best of, and so on. Then if the SE Ranking enrichment API is available we batch all 40 keywords into a single POST to /v1/keywords/export which costs a flat 100 credits regardless of how many keywords you send (up to 5000). That overwrites the AI guesses with real volume, CPC, competition, and difficulty data. Same winnability scoring, same top 12 selection.

The result is that when SE Ranking is configured you get keywords backed by real data. When it is not, you get AI estimates that are at least structured and scored sensibly. The system degrades gracefully instead of breaking.

Step 3. Writing the actual blog post

This is the expensive part. We use Claude Sonnet 4.6 on AWS Bedrock for the writing. Not Haiku, not GPT, not some fine tuned open source model. Sonnet is the best general purpose writing model we have access to and the quality difference is noticeable.

The generation pipeline is chunked. A single 2700 word article cannot be reliably generated in one model call within a 60 second serverless function timeout (Vercel free tier limit). So we split it into steps.

First, Haiku plans the outline. Title, meta title (60 char limit), meta description (155 char limit), a TL;DR summary, image search terms, and an ordered list of 3 sections: one intro, one body, one closing. This is cheap and fast.

Then each section gets its own Sonnet call, each targeting around 900 words. Each call receives the full outline plus one sentence summaries of all previously written sections so it stays coherent and does not repeat itself. The intro section must start with a TL;DR block. The closing section must contain a FAQ (4 to 6 real questions, each answered in the first sentence) and a key takeaways bullet list. Body sections should include at least one comparison table in GFM Markdown.

The system prompt is long and specific. Every section must open with a self contained answer before the supporting detail. This matters because AI answer engines like ChatGPT and Perplexity extract and quote individual passages. If your first sentence in a section completely answers the implied question, you are more likely to get cited.

Each Sonnet call has a 6000 output token cap. Three sections times 6000 is 18,000 potential output tokens, but in practice they use around 1200 to 1500 each (900 words of actual content plus the JSON schema overhead). The structured output uses Zod schemas through the Vercel AI SDK so we get typed objects back, not raw text.

After all sections are written they get stitched together. The slug is computed, a cover image is resolved from Pexels, and the final WrittenPost object is stored in the database. The shape is identical whether we used the chunked path or the single call path so everything downstream does not care which one ran.

Step 4. Cover images

We use Pexels. Free API, good quality stock photos. The outline produces 3 to 6 concrete visual search terms (things like “coffee beans” or “laptop desk”, never abstract crap like “innovation”). We query Pexels for landscape photos, take the top 3 most relevant results, and pick one deterministically based on a hash of the post slug so the same post always gets the same image.

If Pexels returns nothing relevant we return null. No image is better than a wrong image. We never ship a random stock photo of a handshake on an article about email marketing. The only image in a post is the cover. No inline images are injected into the body.

Step 5. Scheduling and publishing

All the scheduling runs on Inngest, which is a serverless job queue. Five functions total.

The publish scheduler runs every 15 minutes. It queries the database for posts whose scheduled time has arrived and fans out one event per post. It does zero heavy work itself.

Each event triggers a publish post function. Concurrency capped at 3, retries set to 3. If the post already has a body it just flips the status to published. If it needs generation it runs the full chunked pipeline described above, each section in its own Inngest step so each gets a fresh 60 second timeout budget.

There is also a pre generation scheduler that runs hourly and finds posts due within the next 24 hours that still have empty bodies. It generates the content ahead of time so when the publish scheduler fires later the post is ready instantly. No last minute generation under time pressure.

A cleanup function runs hourly to reset posts stuck in a “generating” state (which happens when a function crashes mid generation) and enforce grace periods for cancelled subscriptions.

Step 6. How your blog actually gets served

This is probably the part people are most confused about. Your blog lives at yourdomain.com/blogs but we are the ones serving it. How does that work.

It is a reverse proxy. You add a rewrite rule to your Vercel config (or Cloudflare Worker, or nginx, whatever) that says “any request to /blogs/* should be proxied to rankmy.blog/proxy/yourdomain.com/blogs/*”. The visitor never leaves your domain. From Google and from the browser it looks like your site is serving those pages. Because technically it is. The HTML comes from us but it is served under your domain with your SSL certificate.

There are two routing paths in the codebase. The host based path reads the forwarded host header and looks up the site by domain. The path based path extracts the domain from the URL. Both end up at the same rendering code.

Pages are server rendered with Next.js and cached at the edge for 5 minutes with a stale while revalidate of 24 hours. Static assets are served from our origin with CORS headers so they load correctly even when the HTML is on your domain. The proxy paths are marked noindex so Google only sees and indexes the canonical URLs on your domain.

Step 7. Database

Supabase Postgres. Four main tables: users (plan, subscription status, payment IDs), sites (domain, slug, hosting mode, theme, customization), posts (title, body markdown, target keyword, status, scheduled time, cover image), and post events (analytics: views and clicks with referrer and country).

Public blog reads go through the Supabase anon client with RLS policies active. Publishing, webhook processing, and admin operations use the service role client that bypasses RLS.

Step 8. Payments

Dodo Payments. Not Stripe, not LemonSqueezy. Dodo. The webhook receives events, verifies the signature, maps the event type to our subscription status (active, trialing, past due, canceled), resolves the plan from the Dodo product ID by matching against env vars, and patches the users table. Lifetime deal purchases get permanently set to active with no trial and no grace period. Cancellations get a 90 day grace window where published posts stay live before the system unpublishes them.

Step 9. SEO outputs per blog

Every customer blog gets its own robots.txt, sitemap.xml, and llms.txt. The robots.txt explicitly allows 17 named AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google Extended, Applebot, and others) because the whole point is to get cited by AI. The llms.txt follows the llmstxt.org spec and gives AI crawlers a clean structured list of every published post with a one line summary.

Every blog post page ships with three JSON LD blocks: BlogPosting (with publisher, author, word count, dates, canonical URL), BreadcrumbList (site root, blog index, article), and FAQPage (extracted from the FAQ section in the markdown). Plus proper Open Graph tags, Twitter cards, and a self referencing canonical.

Step 10. Analytics

A lightweight beacon component on every blog page fires a POST to our /api/track endpoint with the site ID, post ID, event type (view or click), path, and referrer. The endpoint filters bots by user agent regex, rate limits at 60 requests per IP per minute, and inserts into the post events table with the country extracted from Vercel or Cloudflare geo headers.

That is it. No third party analytics script, no cookie banner needed for tracking, no GDPR consent required for this specific data. The cookie banner you see on our marketing site is for Google Analytics and iubenda on our own pages, not for the analytics on your blog.

What this costs us per blog post

Since I am being transparent about everything else, here is the cost breakdown. One blog post through the chunked pipeline: 1 Haiku call for the outline (fractions of a cent), 3 Sonnet calls for the sections (about 2.5 cents each), Pexels is free. Total is roughly 8 to 10 cents per post. Worst case if every Sonnet call maxes out its 6000 token cap, maybe 16 cents. The keyword research is a one time cost per discovery run, not per post.

On the $49 a month Pro plan at 30 posts that is about $3 in AI costs. On the $199 lifetime deal at 10 posts a month that is about $1 per month forever. The margins are good. I am telling you this because hiding it would be stupid and you would find out anyway.

What is not here

There is no RAG pipeline. No vector database. No fine tuned models. No multi agent orchestration. No LangChain. None of that shit. It is structured prompts with Zod schemas sent to Bedrock models through the Vercel AI SDK. The complexity is in the pipeline design (chunked generation, winnability scoring, deduplication, graceful degradation) not in the model layer. Simple tools composed well beat complicated architectures every time.

If you have questions about any of this, email hello@rankmy.blog. I will answer them honestly because as you can tell by now that is sort of the whole vibe.