Omi Sido https://omisido.com/ is a seasoned international speaker and is known in the industry for his humour and ability to deliver actionable insights that audiences can immediately start using.
From SEO consulting with some of the world’s largest telecommunications and travel companies to managing in-house SEO at Daily Mail and Canon, Omi loves diving into complex data and finding the bright spots.
Hi, guys. Thank you for coming here today. My name is Omi Sido and as most of you know I am very active online and I love talking about Digital Marketing and SEO.
Server logs analysis
Today I’m gonna be talking about Server logs analysis and how understanding what the Googlebot does and what the Googlebot sees on your website can massively boost your digital marketing efforts.
I’m gonna start with a short story. A few months ago – this is a real story by the way – a few months ago a guy came to me and asked me to teach him the basics of SEO so he can collaborate with his SEO team. Of course, I said yes. So I gave him some books and some blogs to read and we agreed to meet again in two weeks. Hi, David. So in two weeks, in two weeks we went to Costa and of course, I asked the guy him about his SEO knowledge. His answer: Content is king and currently he is reading a book about content creation and semantic search.
While he was talking I wrote a short poem, an SEO poem on a piece of paper and after he stopped talking paused I simply asked him: Can go and upload this poem to the Internet, on the Internet. He was like confused obviously: Omi what do you mean? And I was like: Well, you’ve just told me that content is king. So upload this poem on the Internet and make me famous. I wanna be the King of the Internet. Of course again confused And he said something like “But Omi I need a website”. End of story.
A lot of people come to me. A lot of companies and tell me: Omi, our website is beautiful. Our pictures are amazing. We publish content on a regular basis but somehow we are not ranking well. Why?
I said it before and I would say it again: Yes, content is King. But every King needs a castle, a home to live in. Technical SEO is this castle and by analysing your server logs you basically know whether you’ve got a solid structure or not.
If every time when the Googlebot comes to your website it can’t understand the structure of your website if every time when the Googlebot comes to your website and ignores strategic sections of your website your content counts for nothing. Shall I repeat this one? Your content counts for nothing.
Sorry, I can see a lot of fo people tagging me.
The only way for the Googlebot to understand what’s on your website is to literary come and crawl all your pages. Now some of you may say ‘But what is a log file?’ This is a log file. I know it’s a bit confusing but in all honesty, you don’t have to be very technical unless you wanna be an SEO Geek to benefit from the data, from the information, from the wisdom coming from the server logs. From the server logs.
So to make all these very simple I will give you another story.
Six months ago a company came to me and asked me to analyse their website and give them some recommendations for improving their rankings.
Obviously, I started crawling their website and the first thing I saw was half a million pages but only roughly 20 000 of them active pages. You all know this website. I’m not gonna mention the name because of the NDA but you’ve been there at least once for the last one month.
Half a million pages but only twenty thousand of them active pages. By active pages, you know what I mean pages that get organic traffic.
So I’ve started analysing their website, I’ve started crawling it and of course analysing the server logs so this is what I see. OnCrawl.
154 orphan pages. Yes, 154 000 orphan pages. Some websites don’t even have 154 000 pages. Out of 154 000 pages only 3 000 of them active pages.
Now, what are orphan pages?
Chelsea: Omi, what’s an active page? How do you qualify that?
Getting organic visits.
3 000, remember this number.
So what are orphan pages?
Orphan pages are pages that are not linked from anywhere in your website structure.
This is the definition you see online.
What is my definition for orphan pages? Omi Sido’s definition for orphan pages. Stop hurting your SEO. Please.
How do we find Orphan pages?
The only way to find Orphan pages is to crawl, fully crawl a website. Take all the log file data, combine it together and analyse it. In this case out of 154 000 pages only 3 000 active pages. I had no choice but to literary delete all inactive pages. And I know it sounds a little bit harsh.
Then I continued analysing this website and I find that the bot is spending a lot of time, literary stuck in a section full of non-complaint pages instead of crawling the sections with complaint pages. Remember my what I said earlier about the strategic crawling of your website. I had no choice but to delete another big chunk of this website.
And then what I call duplicate URL crawling. By analysing this website I realised that the bot is spending a lot of its resources crawling pages with parameters even although they were properly canonicalized.
We had to literally reshuffle the whole navigation and stuff like that.
And now have a look at this picture. Yeah, this graph.
Pages crawled and not crawled by depth against SEO visits distribution by depth.
As I told you earlier you don’t have to be very technical to understand the importance of analysing your log files.
Have a look at this section of the graph. Only 49% are crawled. Yet, this section gives the most SEO organic visits.
By the way, I don’t like calling them organic. For me, they are just SEO visits.
Very strange for this website page depth five is giving more organic visits than those two. So I had a lot of conversations basically the idea was what’s gonna happen if I literary delete this group. What’s gonna happen if I force the bot to crawl this section more often and index more pages? What’s gonna happen if I actually combine those two without deleting this one or I combine those two? Notice this is only 19%. I hope you can see it from far. Nineteen percent.
You have to really think how you wanna spend your Crawl Budget.
We deleted, in all honesty, we’ve deleted more than 60% of this website went to the bin.
Just for clarity. These 60% any of these is unique content stuff like that. You are not suggesting
No, I am not suggesting that. Normally Orphan pages that are not visited by anybody even the bots are.
So ok. So to explain. Let’s go a little bit to Oprpahn pages. Normally those are development mistakes, expired product pages. Do we agree on this one? Ok, thanks.
So we’ve deleted. It’s ok. No, no of course yeah. The point I am trying to make and thank you very much because many people don’t actually know what Orphan pages are. You are absolutely right.
We’ve deleted more than 60%. More than 60% of this website went to the bin. Yet, six months down the line this client sells more products than ever. I didn’t say visits I said money. They sell more products than ever. Now that the bot is allowed to crawl the good pages more often resulting in more pages present in the SERPs and-and in a better position.
Some of you may say ‘Omi, this is a big website. They had a literary room for deleting pages’ and in fact, I have a lot of clients coming to me telling me ‘Omi, I don’t care about analysing log files because I only have 10-20 thousand pages’. So let me give you a quick example of a relatively small website.
This website was about to be migrated 6-7 months ago and I was asked to analyse it.
22 000 pages in the structure. 8 000 orphan pages. 8 thousand. After finding this one they nearly fired the whole Digital Marketing team. Thank God they didn’t. I’ve got more followers on LInkedIn.
23% are only bringing 3% of organic visits. Literary. Three percent of organic visits. Is it worth keeping those pages?
On the other side in the previous example – sorry I can’t find it now – in the previous example the Orphan pages were actually bringing 37% of organic visits. Why are you not linking to them?
Internally linking to them first so your customers can find them when they come on your website and second you can improve their SEO value so they bring even more visits in the future.
Guys, I hope I gave you, I’ve given you a good idea of how to.
Girl: I have a question.
Omi: Of course.
Girl: Have you done any analysis of incoming backlinks to those pages that you.
Omi: Yes, you have to do that. You have to do that.
Girl: How to do that all feed into the whole process?
Omi: With the example, I gave you there were no literary no backlinks. But you have to do that, you know.
Guys, I hope I’ve given you enough information but by all means, ask questions.
Tips & tools to successfully implement AMP in common scenarios.
Whos is Aleyda Solis?
Aleyda Solis is an International SEO Consultant – a service that she provides with her boutique consultancy, Orainti – a blogger (Search Engine Land, State of Digital and Moz), speaker (with more than 70 conferences in 18 countries in English and Spanish) & author (of “SEO, Las Claves Esenciales”).
After working in different SEO roles at European and American companies, both from the agency as well as in the in-house side, she founded her own consultancy to help from startups in competitive industries to multinational companies with complex Web environments to grow their search visibility & achieve their SEO goals with strategic, technical & in-depth SEO consulting.
So first up we’ve got Omi. Most of you know him so I am not really sure how much of an introduction he really needs. He is an expert in everything technical SEO. He makes great videos so if you are not following him online already you definitely should be. He’s got about six years experience in everything digital. And he’s gonna talk all about going beyond the basics of a website migration.
Hi, guys. Thank for coming here today. Kelvin thank you for inviting me to this wonderful conference.
My name is Omi Sido and as most of you know I love talking about SEO and the way SEO can help you achieve your business goals.
Today I’m gonna to be talking about website migration but as you can see from the title I’m not gonna talk about SEO checklists. You can google them. You can find online and you can follow them.
When SEOs and indeed a lot of SEO speakers talk about website migration normally they talk about disasters. Big fat juicy disasters.
But this is not the way I think about SEO. In my opinion and experience, SEO is one of those rare moments in your SEO career when you can massively and I will repeat massively increase the SEO value of your website in one go. What do I mean by that? Stay with me and I’ll take you on a short journey.
First, website migration is one of those rare moments where you can educate your whole SEO team on all aspects of SEO.
A lot of SEOs are talking only about content and also I see a lot of SEO teams out there built with only content SEOs. To be honest with you yes ‘Content is King’ but if you follow me online you know what I normally say to this statement. Content is King but every King needs a castle home to live with his queen or a lover. You know the story.
Website migration is also one of those rare moments where you can explain what SEO is to every single department of your company especially development and content so they can help you achieve your SEO goals in the future.
Saying that for successful website migration you need a cross-functional migration team. A lot of speakers – SEO speakers say the SEO guy should be in charge. But why would the SEO guy waste his time and effort with the development process or the content process? The SEO guy should concentrate on the SEO tasks.
So you are an SEO – you are in charge. Your SEO plan should have minimum minimum 4 SEO sections. The SEO migration plan the thing that you all love the checklist. The content migration plan that you gonna build with your content team. The development migration plan that you gonna build with your development team and timelines.
Timelines are crucial especially for e-commerce websites as you have to make sure that your website is working properly at peak times when your business is making money. And then paying your salary.
But going back SEO guy and the SEO plan. Many SEOs many speakers talk about the SEO migration as a two-stage process. Your SEO plan should have minimum three stages. Let me just go back to the two-stage process. The two-stage process normally is the SEO checklist press the button disaster. Minimum three stages and in my opinion and my experience the more time you spend on the pre-migration stage the better your chances are of a successful website migration. Also in my experience ninety-nine percent of the time the SEO checks that you do in pre-migration stage, you should be doing anyway on a monthly basis. Welcome to the pre pre-migration stage of the website migration. If you don’t have a name for this stage just call it the Omi Stage. Yeah, I am honest.
So I can’t see the numbers there. Anyway.
So this is a website that was migrated a few months ago. Literally, five six months ago and those are real numbers.
Twenty thousand primary pages. Fourteen thousand pages with duplicate descriptions. Three thousand pages 301. Three thousand pages non-200. Do you really want to migrate those issues to the new platform? In my experience, those issues on the new platform will double triple quadruple. On the other side, your beautifully crafted content counts for nothing if the bot comes and goes into a loop or it’s redirected to the pages that don’t matter for your business. Ups, sorry.
Something I want to say as it just came to mind now. Let’s forget about the website migration for a second. As digital marketers I want you to go back to your jobs on Monday and ask the SEO guy. When was the last time you performed a full crawl of our website? Give me the number of 404s the reason and what we are doing about it. Give me the number 301s the reason and what we are doing about it.
Saying that there is one subject that not many SEOs are talking about – orphan pages. What are orphan pages? Orphan pages are pages on our website that are linked from nowhere. How do we find orphan pages? We perform a full crawl of our website. Then we take all the server logs. We combine the data and we analyze it. How many of you are doing on a monthly basis analysis of server logs? Raise your hands. Oh, yeah. It’s about time. Honestly, I only saw three hands. It’s about time you stop blaming Google that Google doesn’t understand your website structure or it’s not indexing your pages properly if you don’t analyze the server logs of your website.
But going back to the website migration. Eight thousand pages – orphan pages. This is twenty-three percent – those are real numbers and this is the same website from the previous slide – 23% are bringing only three percent of organic visits. Do you really want to migrate those pages to the new platform? Of course, a lot of you don’t know about them because your crawls are not finding those pages because they’re not linked in your system.
On the other side image that this was not three percent. This was – I’ve seen this case as well where the orphan pages are bringing say 30% of organic visits. Why are you not linking to those orphan pages?
Talking about similar to orphan pages 404 pages. My advice. If there are no links pointing at them please discard them. Now before the website migration takes place. Not on staging. Now.
On the other side, if a lot of good links are pointing at them now is the time to work with your content team to either reinstate them, improve the content. You should do something about it. Now. Not on staging and not after the migration has taken place.
So now we know everything about our website. We know from a technical point of view it’s time to benchmark check and benchmark our rankings. This is the only way by benchmarking our rankings this is the only way to know how successful our migration was.
But you have to do it now. Not on staging. And not after the migration has taken place.
Talking about rankings – sorry for going back and forwards – ninety percent of the cases parallel to your typical technical migration there is a content migration going on. Now is the time to work with your content team and either combine those pages/content create new sections.
You are doing this now. Before you go on staging.
Now that we are on staging with or without a website migration it amazes me that in the year 2017 I can still see staging environments crawled and ranked by Google. Because you cannot be asked to do the simplest of technical SEO tasks. You are diluting your rankings and the only reason to do that or shall I say due to your laziness.
We are still on staging 301 mapping we all know this is one of the most important parts of the website migration. But I’m not gonna talk about that.
I’m gonna talk about the fact that during your 301 mapping this is the best chance for you to clean your product offerings. Expired pages, pages that don’t have equivalents on the new platform. Many developers not instructed properly by their SEO, the SEOs that are in charge will redirect pages to irrelevant sections on your website or God forbid the home page. The thing that I am saying now is with or without a website migration. Many SEOs are arguing online what’s happening here and saying this is the best way to preserve your SEO juice. Hard earned SEO juice. You can’t be more wrong. After some time Google will start treating those 301 redirects to relevant sections on your website as soft 404s and after some time it will deindex them. So you gonna lose all the hard-earned SEO juice.
And to give you a good example. Last year royal.gov.uk was migrated to royal.uk.
Before we continue ‘The home of the Royal Family.’ What did I tell you ten minutes ago? Every King needs a castle, a home to live with his King…queen.
So the people. It’s a funny story as well I know I was laughing when I found this website. So the people who were migrating the home/technical SEO of the Royal Family last year couldn’t be bothered to redirect the expired pages, pages with no equivalents to the equivalent parts of the website so they ended up with roughly 80% of pages redirected to the home page. The result.
Within one week those clever developers/SEOs destroyed completely the castle the home/technical SEO of the Royal Family. This website lost around 80% of its rankings in one single week. Go and check it out. Everything is still in place so you can actually learn from mistakes.
I can see Purna finding it very funny. It’s not actually funny. Yeah, it’s tragic.
We are still talking about 301 mapping and this is the time for you to clean your chain redirects. If you’ve been following me online you know I hate chain redirects. And it’s very simple. If page A on the old system was 301 redirecting to page B now page A will 301 redirect to page C – this is the new platform and this page is equivalent to page B and page B will 301 redirect to page C. Simple and effective.
Now is the time to make sure on staging that the tracking code is implemented is present literary on every single section on your website.
Now that you managed to educate your whole SEO team. You’ve managed to educate your whole company of what SEO is. You are doing your full website crawls every month and you are analyzing the data you are ready to press the button.
Guys, I did include some post-migration checks because they are very very important. But for now, thank you very much and hope to see you soon.
I am so pumped up about this week. I am going to Holland and as many of you know I just love this country. But first I am meeting Steven van Vessum – one of the founders of Content King.
Guys, if you are a small business and you want to avoid the hassle of checking your website manually on a daily basis this software is for you.
Omi: So Steven, tell us about your software.
Steven: So Content King is a website monitoring system which does continuous crawling of your website and whenever we find issues we alert you. And whenever we see changes we save them.
In doing so we try to make the life of SEOs and digital marketers a lot easier.
Omi: You heard the man. Your life easier.
Steven: Isn’t that what we all want?
Steven: So remember Content King.
It’s Thursday morning. It’s 3 o’clock to be precise and I am on my way to Holland to teach local businessmen the benefits of having a solid technical SEO.
Not many people are talking about it but 68% of revenue (on average of course) across all markets and industries is still coming from organic visits.
Do you spend your time, money and effort towards the right channel
“To be successful in SEO you need a well-defined SEO strategy.” – Omi Sido
Very often described as a process of altering and strengthening a website’s online relevancy, an SEO Audit typically covers important components of a website such as content-related issues, site architecture, indexing, backlink analysis and social-media engagement.
Guys, understanding these components and how they work, is an important part of identifying the strength, weaknesses, and potentials a website has in natural search.
Friday. The second day of my Dutch journey. Let’s remind ourselves something. SEO is one of the most cost-effective marketing strategies because it targets users who are actively and I will repeat this one ACTIVELY looking for your products or services online.
When it comes to website and cars there are a lot of similarities. Both of them need to be well designed. Both of them need maintenance. and both of them need everything to be well connected inside in order for you to move forward.