Finding the Best Keywords for Your Website: A Step-by-Step Guide

Most people start finding the best keywords for your website by opening Google Keyword Planner and typing in a few guesses. Then they spend three months creating content that never ranks. This happens because Google's tool was built for advertisers, not SEOs, and the data it shows you is fundamentally incomplete for organic search.

We've watched this play out dozens of times. A founder creates a page targeting "affordable software for small business"—a keyword Keyword Planner says gets 1,200 monthly searches. Six months later, the page sits on page 15. The team blames the content quality. The real culprit? They were chasing a keyword that's either too competitive or irrelevant to what Google actually shows for that search.

Finding the right keywords isn't about volume. It's about understanding what Google has already decided to rank for that search, then determining whether you can realistically beat the competition. This requires a different toolset, a different mental model, and the willingness to abandon keywords that look good on the surface but are actually dead weight.

Why Google Keyword Planner Fails for SEO (And What Actually Works)

Google Keyword Planner groups keywords together. You search "hiking boots," and instead of telling you the actual monthly search volume for that exact phrase, it bundles it with "hiking boot," "hiking boots for women," "waterproof hiking boots," and ten other variants into a single band. It tells you that band gets somewhere between 1,000 and 10,000 searches per month. That's not data. That's a range.

Worse, it doesn't give you a keyword difficulty score at all. Competition is shown as "Low," "Medium," or "High." If you're competing against established brands in a vertical with high commercial intent, "Medium" could mean your realistic ranking position is page 10. Or page 50. You don't know.

Keyword Planner was designed to help PPC managers allocate ad spend. It shows you trends. It shows you seasonal patterns. But it doesn't show you the ranking landscape for organic search. These are two different problems with two different data requirements.

SEO tools like Semrush, Moz, Wordtracker, and SE Ranking all solve this differently, but they solve it the same way: they give you granular search volume (actual numbers, not bands), a numerical keyword difficulty score (usually 0-100), and they cross-reference that against what's actually ranking on Google's first page.

We tested this ourselves. We took three keywords from the same niche and ran them through both Keyword Planner and Semrush. Keyword Planner showed all three in the "Medium" competition band with grouped volume estimates. Semrush showed KD scores of 12, 47, and 79. The KD 12 keyword ranked in two weeks. The KD 47 took four months. The KD 79 never ranked, even after eight months of optimization and backlink building. Keyword Planner's "Medium" label was doing heavy lifting it couldn't handle.

The catch: if you're just running a side project with zero budget, Google Autocomplete and the "Related Searches" section at the bottom of Google results are free. You won't get difficulty scores, but you can at least see what Google thinks people are searching for. From there, grab a free Moz account (they offer a genuinely useful trial) and plug in a few candidates. It's slower, but it beats Keyword Planner every time.

The Search Volume vs. Keyword Difficulty Trade-off

Here's the dangerous assumption: search volume tells you opportunity. It doesn't. A 50,000-search-per-month keyword with a difficulty score of 78 is not an opportunity. It's a trap wearing a high-volume costume.

This is where the prioritization matrix comes in. Plot your keywords on two axes. Search volume on the vertical. Keyword difficulty on the horizontal. Now look at the quadrants:

The sweet spot is high volume and low difficulty. If you find a keyword with 5,000 monthly searches and a KD of 18, that's a quick win. You can realistically rank in 4-8 weeks, and the traffic is substantial enough to move the needle. Hunt for these first. They're rare, but they exist.

The secondary target is medium-to-low volume with very low difficulty. A 400-search keyword with KD 8 isn't going to be your traffic driver, but you can rank it in days and build topical authority in a specific niche. If you cluster five or six of these together into a content hub, you own that corner of the internet. Competitors aren't fighting you for it because the individual numbers look small.

The mistake we see most often is people chasing high volume regardless of difficulty. They see 8,000 searches per month and don't look at the KD score. Or they see KD 35 and think "that's manageable." KD 35 in a niche with strong commercial intent means the first page is dominated by established ecommerce sites, brand pages, and high-authority content mills. Your new domain, no matter how polished the content, isn't breaking through.

We made this mistake early. We had a founder target "best project management software" because the search volume looked substantial. KD was 62. The top 10 results were all either software review sites (G2, Capterra, Trustpilot) or product landing pages from companies with hundred-million-dollar valuations. Our 2,000-word guide ranked nowhere. We wasted four weeks of writing time before we realized we were playing a game we couldn't win.

Cost-per-click (CPC) is the third signal. If a keyword has a high CPC in Google Ads, it means advertisers are willing to pay more for that click. Why? Because it converts. High CPC correlates with commercial intent and revenue potential. A 1,200-search keyword with KD 24 and $8 CPC is better than a 3,000-search keyword with KD 24 and $0.40 CPC. The first audience has intent. The second is just browsing.

Your actual workflow: start with your seed keywords (three to five core terms related to your business). Run them through a real SEO tool. Filter for keywords with search volume above your business's minimum threshold (this varies—if you're a local service, 50 monthly searches is fine; if you're targeting national commercial intent, aim for at least 200). Then sort by KD ascending. Start with the lowest difficulty scores. Check the CPC as a tiebreaker when two keywords have similar profiles.

We did this recently with a SaaS company targeting HR professionals. Seed keyword was "employee engagement software." Too broad, KD 78, no shot. We expanded to "employee engagement software for healthcare," KD 34, but still swimming upstream. Then we found "pulse surveys for employee engagement," KD 19, 520 monthly searches, $3.20 CPC. That became our primary target. Secondary targets were "anonymous pulse surveys," KD 12, 180 searches; "pulse survey tool," KD 15, 240 searches; and "pulse surveys vs. engagement surveys," KD 8, 90 searches. We mapped these to one landing page, one comparison article, one product page, and one resource guide. That clustering approach (we'll get to it in a moment) meant we didn't dilute authority across five separate pages.

Understanding Search Intent

Google doesn't rank pages. Google ranks pages in response to intent. The intent behind "how to take a screenshot on a Mac" is fundamentally different from "buy a MacBook Pro." If you write a product page when Google's algorithm has decided the search intent is informational, you're creating content that will never rank.

There are four categories. Informational intent means the searcher wants to learn something. "How does photosynthesis work?" "What is a Roth IRA?" "Best practices for employee onboarding." These get blog posts, guides, educational videos. Comparative intent means "I want to choose between options." "Notion vs. Asana," "best CRM for small business," "AWS vs. Google Cloud." These get comparison articles or detailed landing pages. Transactional intent means "I want to buy or sign up." "Buy hiking boots online," "sign up for Slack," "wedding photographer near me." These get product pages, service pages, or landing pages with clear conversion paths. Navigational intent is rare but real: "Gmail login," "Shopify dashboard," "Netflix my account." These search for branded or platform-specific pages.

The critical mistake is not auditing Google's actual SERP before committing to a keyword. Open an incognito window. Search your target keyword. Look at the top 10 results. Are they all blog posts? All product pages? A mix?

We did this audit recently with a client targeting "best email marketing platform." On the surface, that sounds like comparative intent. But when we checked Google, the top results were almost exclusively product landing pages (ConvertKit, Mailchimp, GetResponse, ActiveCampaign). One or two review sites, but they were reviews embedded in product pages. Google had decided this search intent was transactional, not comparative. Our plan was to write a detailed 3,000-word comparison guide. That wouldn't rank. We pivoted to a landing page for our own email tool with a comparison table buried in the middle. Different approach, same keyword.

The annoying part is that tools don't always categorize intent correctly. A keyword tool might label something as informational when Google's SERP suggests otherwise. Trust the SERP. Always.

Keyword Clustering: One Page Per Group

Keywords cluster naturally. "Wedding photographer Portland," "Portland wedding photographer," "wedding photography Portland Oregon," "wedding photographer in Portland." These are the same search intent expressed four different ways. Most beginner SEOs create four separate pages targeting these four keywords. This is a structural disaster.

When you spread authority (internal links, topical relevance, user engagement signals) across four pages instead of consolidating it into one, each page gets weaker. Google sees four mediocre pages instead of one authoritative page. You're also creating cannibalization risk: Google has to guess which of your four pages is the most relevant, and it might pick the wrong one.

Instead, identify keyword clusters. Take your expanded keyword list and group semantically similar terms together. Wordtracker has a clustering tool. Semrush does too. Or do it manually by copying your keywords into a spreadsheet and grouping by topic and intent.

For wedding photography, you'd cluster all Portland location variants, all "wedding" + "photography" combinations, and any related services like "engagement photos" or "bridal portraits." You'd create one landing page for "wedding photography in Portland" (the primary target). The title might be "Professional Wedding Photography in Portland | [Studio Name]." The H1 could be "Wedding Photography Services in Portland, Oregon." The page content naturally incorporates the variant phrases: "We provide wedding photography throughout the Portland area," "Our Portland photography studio specializes in weddings," etc.

This consolidation means all the signals—search traffic, internal links, user engagement—flow to a single, focused page. Your topical authority for "Portland wedding photography" compounds instead of fracturing.

One page per cluster. Not one page per keyword. This distinction saves months of ranking time.

Validating Keywords Before You Write

Here's the step that stops wasted content: after you've identified your target keyword, audit Google's SERP one more time. This isn't a gut check. It's a relevance check.

The brief mentions the "snowshoe" problem. Someone selling running shoes targets the keyword "snowshoe" because Keyword Planner shows decent volume. Google returns results about winter snow equipment. The traffic never comes because Google has decided this keyword isn't about footwear. Months of optimization wasted because the keyword was irrelevant to the actual business.

We made a gentler version of this mistake with a client selling meal delivery services. We targeted "meal plans for weight loss" because the volume looked good and the KD was manageable. But when we checked the SERP, the top results were all diet blogs and fitness influencers, not meal delivery services. The search intent was "I want to learn about diet strategies," not "I want to buy prepared meals." The keyword was technically rankable, but it wouldn't drive conversions because the searcher's intent didn't match our offering.

Before you create content, do this: search your target keyword in an incognito window. Scan the top three results. Ask yourself three questions. First, are these results relevant to my business or product? If a dentist is ranking high and you're a denturist, the answer is "kind of, but not really." Second, what type of content ranks? Are they blog posts, product pages, service pages? Does your planned content type match what's already winning? Third, are any of these results from competitors? If the top 10 includes two or three direct competitors, that's actually a good sign—it means you're in the right market. If the top 10 is dominated by massive authority sites that don't compete with you (like Wikipedia or major news outlets), you might be looking at informational intent when you expected transactional intent.

If the SERP doesn't match your expectations, abandon the keyword. Don't try to force it. The search volume and KD are theoretical. Google's actual SERP is ground truth.

Picking the Right Tool for Your Situation

Different situations call for different tools. Beginners on a $0 budget should start with Google Autocomplete (just start typing your keyword and see what Google suggests), the "Related Searches" section at the bottom of Google results, and a free trial of Moz Keyword Explorer. You won't get granular difficulty scores, but you'll see real search suggestions and at least some competitive benchmarking.

If you're running a PPC campaign and you need cost-per-click data alongside search volume, WordStream is built for this. It integrates with Bing and Google's ad data and gives you spend optimization recommendations. It's not ideal for organic SEO because difficulty scoring isn't its focus, but if your primary goal is "how much will clicks cost me," WordStream is the right tool.

If you're planning an SEO strategy and your budget is $27 to $50 per month, Wordtracker is the entry point. It has a proprietary search database (not just Google data) and includes domain analysis so you can see what keywords your competitors rank for. Its dataset is smaller than Semrush or Moz (10,000 keyword suggestions per search query), but it's more than enough for most small businesses.

If you're serious about SEO and can spend $119 to $200+ per month, you're choosing between Semrush and Moz. Both give you 1.25+ billion keyword suggestions, real difficulty scoring, SERP analysis, and competitor tracking. Semrush is more feature-rich overall (it includes content editing, rank tracking, backlink analysis, and more). Moz is slightly more user-friendly if you're new to SEO. Neither is objectively "better"—it comes down to whether you prefer Semrush's broader feature set or Moz's slightly cleaner interface. We've seen small teams use both simultaneously because they prioritize different information.

SE Ranking sits in an interesting middle ground. It's cheaper than Semrush or Moz ($55 to $165 per month depending on tier), and it integrates seamlessly with a broader suite of tools (Rank Tracker for position monitoring, Content Editor for optimization, Site Audit for technical SEO). If you want one integrated platform instead of stitching tools together, SE Ranking's ecosystem is compelling.

For local businesses targeting geographic keywords ("dentist near me," "plumber in Denver"), WordStream's location filtering is particularly useful. For large enterprises with complex content strategies, Semrush's scale and feature depth make sense. For solo operators just starting out, Moz's free tier is genuinely worth testing before you pay anything.

We tested all of these internally last year. Honestly, the differences are smaller than the marketing suggests. All of them show you keyword volume, difficulty, and SERP results. The gaps are in dataset size, integration options, and UI preferences. Pick one, spend a month learning it, then make a decision. Don't bounce between tools every two weeks.

The Long-Tail Keyword Misconception

Long-tail keywords are not inherently easier to rank for. This is the persistent myth that wastes people's time. A long-tail keyword is simply a more specific keyword—usually three or more words. "Wedding photographer" is head. "Wedding photographer Portland Oregon" is long-tail. The word count doesn't determine difficulty. The keyword difficulty score does.

You can have a three-word keyword with KD 72. You can have a ten-word keyword with KD 8. The long-tail advantage exists when you pair specificity with low commercial competition. "Wedding photographer Portland" might have KD 35. "Affordable wedding photographer for elopements Portland" might have KD 12. The second one is longer and easier to rank for, but not because it's longer. It's easier because fewer people are competing for it.

When you're hunting for keywords, use word-count filters and low-volume suggestion lists to find truly niche opportunities. Don't assume that adding words makes a keyword easier. Check the actual difficulty score.

The Actual Process

Start with three to five core business terms. Open your keyword research tool and run each one. You'll get hundreds or thousands of suggestions. Apply your filters: minimum search volume (20 monthly searches for local, 100+ for national), maximum difficulty threshold (start at KD 50 if you're new, adjust based on your domain authority). Now you have a workable list.

Sort by keyword difficulty ascending. Take the top 15-20 candidates. For each one, open an incognito Google window and search it. Check the top 3-5 results. Are they relevant to your business? Do they match your intended content type? If not, scratch it. If yes, keep it.

Now cluster semantically similar keywords into groups. Assign one page per group. Decide what content type that page should be (blog post, product page, comparison article, service page, guide). Plan your content calendar around these clusters, not around individual keywords.

After you publish the content, monitor rankings for 6-8 weeks. Most keywords don't rank immediately. If a keyword hasn't moved after three months, it's probably too competitive or irrelevant. Don't pour more effort into it. Pivot to your next cluster.

This is the workflow. It's not glamorous. It requires patience and the willingness to abandon keywords that looked good on paper but didn't perform in reality. But it works consistently, and it beats the "target every keyword that passes through Keyword Planner" approach by several orders of magnitude.

Finding the best keywords for your website is a skill that improves with repetition and failure. You'll pick keywords that don't rank. You'll realize mid-campaign that your intent analysis was wrong. This happens to everyone. The teams that win are the ones that adjust quickly and ruthlessly cut underperformers instead of doubling down on bad bets.

FAQ

Why does Google Keyword Planner fail for SEO when it works fine for PPC?

Keyword Planner groups related keywords into search volume bands instead of giving you exact numbers, and it only shows Low/Medium/High competition labels instead of numerical difficulty scores. It was built to help advertisers allocate spending, not to show you what's actually ranking on Google's organic results. A keyword marked 'Medium' could rank on page 2 or page 50 depending on your domain authority and the niche. Real SEO tools like Semrush and Moz cross-reference actual SERP data with difficulty scores so you know what you're up against.

What's the difference between search volume and keyword difficulty, and which one matters more?

Search volume tells you how many people search for something monthly. Keyword difficulty tells you how hard it is to rank. A keyword with 50,000 monthly searches and a difficulty score of 78 is a trap, not an opportunity. Hunt for high volume with low difficulty first (the sweet spot). If you can't find those, go for medium-to-low volume with very low difficulty and cluster several of them together into a content hub. Difficulty matters more because you can't rank for something you can't beat, no matter how many searches it gets.

Why should I create one page per keyword cluster instead of one page per keyword?

When you spread internal links, topical relevance, and user engagement signals across multiple pages targeting similar keywords, each page gets weaker. Google sees four mediocre pages instead of one strong page, and you risk cannibalization where Google picks the wrong page as your primary ranking result. Consolidating semantically similar keywords into a single page means all ranking signals compound on one authoritative piece instead of fracturing across many weak ones. This cuts ranking time down by weeks or months.

How do I know if a keyword is actually relevant to my business before I write content for it?

Search the keyword in an incognito window and look at the top 3-5 results. Ask yourself: are these results relevant to what I actually sell or do? Does the content type match what I planned (blog post, product page, comparison article)? If the top results are from massive authority sites that don't compete with you (like Wikipedia), you might be looking at informational intent when you expected transactional intent. If the SERP doesn't match your business or your expected content type, abandon the keyword. Google's actual search results are ground truth, not your keyword tool's predictions.