1
2
3
4
online marketing
SEO Foundations
How does search organize information? How does search know what users intentions are? Without knowing the foundations of how search works it is difficult to understand how you can use search as a marketing strategy for your business. This masterclass is all about giving you the right foundation.
Course at a glance
50 minutes
3 quizzes
6 Videos
1 gallery
5 pdf

Introduction

In this chapter we try to understand how do we use search. What are our intentions and how we try to find answers based on questions that pop up in our head.

How we use search?

So what exactly is SEO? Before we get into defining what SEO is, let us understand how we humans use the internet. This fundamental concept is important to understand. Everything around SEO revolves around how we use the internet and how we use search engines. 

To get a deeper understanding of how search works, let us analyse how we use search right from a thought to typing a phrase on a search box to getting results. Let us use a fictional character Alice. 

To capture users questions/ phrases that they are actively looking for.

After Alice types in the phrase / words on the search bar there are few things to consider:
  • She is looking for a particular answer (e.g Age of a particular athlete )
  • She is looking for a comprehensive answer (e.g. Biography of an athlete )
  • She is just looking to get educated on a particular subject (e.g. Why is the sky blue? )
  • She is doing market research (e.g. Which car is the best choice for my budget and use case? )
  • She is trying to make a purchase and looking for the best option (e.g. Who is selling the cheapest shoe? )
  • She is trying to learn something specific and needs instructions (e.g. How to cook pan cakes? )
  • She does not even know what kind of answer she wants. She is just searching for something that keeps her entertained and engaged (e.g. The latest global news, picture of dragons)
  • And the most confusing, She may have not typed the phrase/question accurately that would give her the answer she is looking for (e.g. She types in spelling mistakes, grammar mistakes, incorrect words)

This is where it gets interesting. Without knowing what intention Alice has, search needs to make an accurate prediction. How does 'search' know what to show when there are 10 trillion websites and it can only show 10 of them per search result? How does it organize all that information? How does it know when to display what? A subtle difference in text or phrase can change the whole meaning. Can search understand these subtleties.  To answer this we need to understand how search works.
 
It makes prediction based on user patterns. If there are other users like Alice who have asked similar questions, search makes note of it and tries to show Alice the same results that satisfied the other users.   

How does search work?

In this Chapter we will try to understand how search works. What is it doing under the hood. When we press enter after typing in a key phrase what is happening. Let us uncover the black box.

Organizing information

The goal of search is to organize information. It does this in 4 critical steps. 
 
1. CRAWL
2. INDEXES
3. RANKS
3. DISPLAY

Let us understand each of these four steps in details.

CRAWL

The 1st step for search is to find all the website that are available in the internet. It does this by using a software called web-crawler. This is no different than a library trying to get a copy of all the books in the world. Once it finds all the different web pages it is time to index them.

INDEXES

Once it crawls, search needs to record what it has crawled. This is called indexing. It is like the index section in the back of a book. Where each word that is used and which exact pages it was in is shown. This enables navigating a particular word easily through the book. 

RANKS

Once it has indexed all the pages, it now need to organize the information in a more meaningful manner. Which is to say, it needs to create association between what this page is all about when and where to show this page. For what keywords or key-phrases users type should this page show up. It needs to make judgment based on various parameters. This stage is more like a dictionary. Where you try search for a word and you need to find its meaning. In the search world, the word you are trying to search is what keyword or key phrase you enter in the search box, and the meaning of the world is synonymous to search trying to organize the best web-pages that will give you the answer to your search query.  

DISPLAY

Once it ranks, it now needs to display the result. The result can be displayed in various ways. Like images, text, video, audio etc. On search for the most part only 10 or so results are displayed on the 1st page. This part of the search process where it shows the results, the order, the ranking, maps, etc is all part of its display feature.  

Summary

Below is a brief gallery which summarizes the 4 components of how search organizes information.

<

Crawl

Index

Rank

Display

>

The librarian

In the previous lesson we briefly went through how search works. In this lesson we will go a bit deeper. 

Let us consider a Librarian who is in-charge of a library that stores trillions of books. This is no different that what search needs to do. It needs a method to store all those books based on certain categories, topic, edition, recency, relevancy, language, etc. So how does a librarian organize all the books? Once the books are organized properly retrieving them is easy. It is like a dictionary. It does not matter how thick the dictionary is you can systematically find your word and its meaning. But the organization is the tricky part.

To understand how books are organized let us consider you. How would you select a book you want from the library? If a topic is relevant to your interest, you start to look for books based on that topic. Then you might search for reviews and choose the one that is highly rated. You might read the summary, the back cover, the front cover, judge it based on thickness, the content, the story, cover picture, the smell of the book and it's pages, the font size and type, hardcover, softcover or e-book. In a nutshell you systematically navigate a book you want through a sequence of steps before you select one to read. 

For each person, each of those parameters will have different priorities. For some thickness of the book is a high priority and for some the table of content and the story is more important. It depends person to person. Nonetheless, each one has some form of way to judge if a book is worthy enough for them to read it. 

The process of prioritizing and making judgments is what 'search'  does in its ranking step. It needs to categorize websites and pages in a systematic way so that it display the best possible result to its users. This is what defines a search algorithm.   

Search Algorithms, parameters and priorities

When it comes to displaying an appropriate web-page based on a users request which is called a query, search needs to assort web-pages based on certain parameters. Each of those parameters will then be assigned different priorities. After calculating, it them shows the best page it finds fit. 
The question is, what exactly are those different parameters and how does it judge what parameter has a higher priority over the other? This is the essence of this lesson.

There are 5 core parameters that 'search' needs to evaluate before displaying appropriate results.

- Meaning of your query
- Relevance of webpages
- Quality of content
- Usability of webpages
- Context and settings
Source: Google - How search works

Let us describe each one of them in detail.   

Meaning of Your query - In order for 'search' to return relevant information, it first needs to understand the meaning of your query.  What are you trying to look for. Based on Alices example above, 'search' needs to find or guess what is the intent of her query. Understanding intent is based on the language and dialect of the user. Without understanding the meaning, search will not be able to return relevant answer. Back in the day the results that it returned would be exact matches. If it found a website that had the exact key phrase as what the user typed it would return those website as results. Unfortunately there are many problems to this approach. 

One being, the website might contain a bunch of text and phrases that are exact matches but has no intrinsic value. Meaning, it does not have valuable content around those key phrases. This is what is called spamming. To make search give more robust results, it needs to filter out pages that are spam vs those that actually contain value.

Remember, search engines are a business. Their job is to give you relevant results based on your query. If they do not you will top using them. It is in their best interest to  always return the most relevant result it can find. For that, it needs to understand the meaning of your query. Not just give you exact matches based on the words you typed. 

Relevance of webpages - After it understands the meaning of your query, it now has the responsibility to make a guess as to what webpages is best suited to answer your query. This is by no means an easy task. One way is to find if the webpage has popularity. If many other webpages links to a particular webpage it sends a strong signal that this webpage has value. Theoretically it makes sense. A webpage that has many other webpages referencing it does not necessarily mean it can answers the query of the user. It is just one of the signals search takes into account when organizing information.  

A more robust way for 'search' to analyze relevancy is to understand and make sense of the actual content on the page. This again is no easy task. Let us get back to our example of Alice. 

Say Alice is searching for information on Michael Jackson. If a webpage has the word Michael Jackson, that would be the 1st step. Accumulate all pages with the word Michael Jackson. That in it self would be a lot of pages. They next step is to see if the content on the page on Michael Jackson is comprehensive. If all the webpage has is multiple occurrence of the word Michael Jackson, the page would be quite useless. After all Alice is not looking for a webpage with how many times the work Micheal Jackson is repeated. She is looking to learn more about him. On the other hand if a webpage has all the information related to the singer such as, when was he born, where did he study, when did he 1st start to sing, what was his 1st song, when did he release his 1st album, how many times were his song #1 on the top charts, when did he reach super stardom, information about his personal life, comprehensive information on his professional life, references to facts, video of him performing a concert, audio snippet of his music, this would be a much more appropriate result for her search query.  

This in a nutshell is what relevance really means. How valuable and comprehensive is the content surrounding a particular subject. This is also why Wikipedia ranks pretty much on the top for many search queries and rightfully so. The content is comprehensive, factual, backed by appropriate references, up to date and relevant.  

Quality of Content -  Now that search has found relevant pages it is now time for it to do quality control. Is the content on the page accurate. Does it have valid references. Is the language easy to understand. Is there any grammar or spelling errors on the page. Is the page updated frequently. Is the author an expert on the particular subject. Is the webpage trustworthy. Doing quality check is important to filter out webpages that are spam like to the one that have actual value. It also prevents webpages that want to rank high due to deceptive or manipulative behavior.  

Usability of webpages - Now that 'search'  has found relevant content, it has done its quality control of the content it now needs to check if the page is usable. Which is to say, when the user Alice visits this website, does it load properly on her device and does it look trustworthy. Looks matter. Is the webpage responsive. Which is to say, does it load appropriately on mobile devises or tablet or on desktop, is the font legible, is the color easy on the eyes, does the page load fast, is it easy to navigate, is the layout appropriately designed. It needs to take in to consideration all these factors before return a result. After all if Alice visits a page that has relevant content, the quality of the content is very good but the webpage it self is difficult to read, takes a long time to load and difficult to navigate, Alice will have a bad experience. She rather visit a webpage that has mediocre content but usable than one which has great content but takes a long time to load and not easy to use.

Context and settings -  The last part before it display the results, it now need to make sure the context of the query is relevant to the user. A good example is the term football. If you search for the word football in North America the sport is different from when you search for the word in Europe. Context matters. If someone is searching for butter chicken near me, search needs to understand the context. It should make the judgement that the user is looking for places that service butter chicken around the users area. Makes sense of all these parameters is the job of search. 

Going back to how Alice searches, when she types in a key phrase, search takes into consideration all of the above parameters before returning its top results. This whole process is how search works in a nutshell.  

Display

After search has organized and ranked its content it needs to display the information appropriately. Should it show your result on maps, as a text, as a video as a snippet as a scorecard. How should it display the result of your query.
How results are displayed is important for the user experience. For example when you search for what the weather is going to be like, a visual representation if it is sunny, cloudy, rainy, snowy what will be the temperature is easily explained using simple graphics rather than text. Similarly if you search for a location, displaying a map and the time it will take you to reach the location from where you are makes more sense than a website with links to the location. similarly for scorecards or sporting events. If you type cricket or tennis and if there is a tournament going on, most likely you might want to know what the current score is. A live scoreboard will make the user experience more meaningful compared to a link to website. 

How results are displayed is a critical components of search. Looks matter. If the information that a user is looking for can be conveyed in the easier and most efficient manner, it is searches responsibility to constantly thrive to achieve such an outcome.    

Different ways search displays results

Below is a gallery that show for different types of queries how search tries to display and organize the information. By doing this it gets to cater to different user intentions in one shot. This is critical component to search. How it displays information.

<

Weather

If you want to know the weather in your area, search does a good job displaying the weather forecast in a intuitive way.

Video

A Video carousel displayed on search result. Many times if written is not the best way to communicate something. Video might be a better modality.

Rich Snippet

Below a search result, there are options to add F.A.Q or other snippet that explain more about the page before the user visits it. These are called rich snippets.

Questions

When you type in a query on search and search returns with better or different questions related to your query, this is search helping you have a greater search experience. This is done by displaying a set of question in a tabular form.

Podcast

Search not only displays videos and text links but it has the ability to display audio content too. Podcast in audio formats are displayed elegantly on search results.

Layout

If you type in a query related to general information like 'Michael Jackson' search does a great job of laying out all the different information related to your query in a very comprehensive but elegant way.

Maps

If you have a search query where the result is best described on a map, search will return a result appropriately. For e.g. if you search for 'restaurants near me' search is smart enough to know that the result is related to a location and it is best to return the result on a map.

Snippet

Search's goal is to make it easy for users to get the information they want as quickly as possible. If you search for 'how an engine works', search tries to find the most trusted website that has the answer and displays a snippet of that as a search result. This allows the user to get a brief understanding of the topic in a convenient manner.

Layout

At the time of writing this masterclass, the covid-19 epidemic is on. In search, if you type in a query that needs factual data as the answer, search will layout all the results in an elegant manner to help the user make sense of the results.

Business Card

If you own a business or are looking to learn more about a business, search makes it easy for businesses to enter critical information. This is displayed as a business card on search results for users to scan through a business quickly and effectively.

>

Positioning

Having understood how search works, in this Chapter we describe how you can position your website based on the above concepts. By positioning your website accurately, you have a greater chance of making your website show up on the 1st page of search.

The 4 pillars

Having a high level understanding of search should give us the right ammunition to position our business accordingly. At the end it is all about the end users experience. There is nothing you can do to violate this rule. If what you produce does not help the end user it makes no sense for search to promote your content. Having said that, let us divide our efforts into three main pillars than can help us better position our brand on search. 

Content

The 1st pillar for ranking on search is producing valuable content. This is the most overlooked aspect. Many people start to 1st get into the intricacies of 1st building a website or learning how to use a tool, but in reality producing content, show or long must be the 1st focus. Content takes various shapes and forms. It can be in the from of a video, audio (podcast), written blogs, images. The ability to share information in and easy and elegant manner is critical. At the end all of what search is trying to do is find relevant and engaging content.

Usability and User Experience

To serve your reader better, you need to give them a great experience. Search wants to deliver to the user a usable website. There are many things that account for making your website not usable. One being how fast your website loads. If it takes too much time, users will not want to use your site. How are the fonts, the colors, the buttons, do the links work, the layout the design and smoothness, interface. There are so many factors that affect user experience. In a nutshell does the site seem authoritative and is it trustworthy. Those are the two most critical signals your user interface must communicate. 

Feedback and Analytics

After you have written valuable content, you have a website that is usable and now you want to measure how it performs. You need to monitor how your webpage is performing, for what keywords does it show up, how are users interacting and engaging with your page. All of these are important signals to track. Once you capture the data, making sense of it is the next step. Some signals are easy to analyze and fix and some data needs more insights. Things like a button not working after users repeatedly clicks on a link might be an easy fix, where as if you find users not engaging with your content even though it is comprehensive might be a bit difficult to figure out. May be your content is not relevant to what they are looking for, or there may be a usability issue that is difficult to measure or there may find exactly what they were looking for from the 1st paragraph of what you have written. Understanding your data is important if you want to keep your webpage relevant. 

Continuous Improvement 

What is important to note here is that you need to continuously improve on the above. Take examples from Wikipedia. If a new event occurs in one of their articles, the content is immediately updated. This is what makes it a very trust worthy source. The ability to continuously improve/update. Keeping things on your website up to date is the landmark of what SEO is. One comprehensive article that is constantly refined and updated requires constant monitoring. You might think writing more content on different topics is important. Yes, there is truth to that statement. But having content that is out of date is worse. No one wants to read it and it becomes outdated. Not improving on your content and webpage sends a strong signal to search. Search will be inclined to display relevant, recent content to its users that something that is outdated. The goal is always to continuously improve. 

Bridging the gap

The last critical part is to make the connection between what the user is searching for and what you have to offer. Writing content for the sake of writing content might not necessarily yield the right results. After all if no one is searching for what you produce then the net value of your webpage is zero. So how do you bridge the gap?

There are 4 ways how you can bridge the gap in a systematic way:

Trends

Market value and demand

Knowledgeable Experience 



The Benefits

How can your business benefit from ranking high on search?


Brand Awareness
Recruitment
Lead Generation
Educating
Selling

conclusion

In this chapter we summarize what we have learnt and will give you ideas on what to do next.

Summary

What's Next