- Feb 16, 2024
Posted: Oct 21, 2021
Updated: Feb 16, 2024
After a site migration, it's common to temporarily lose some traffic (15~20%), because it takes some time for Google to crawl your new site. In most cases, this is nothing to worry about. However, problems with redirects, duplicate content, metadata, etc., may prevent you from recovering the organic traffic. The below guide serves as a basic checklist to ensure a smooth site migration without losing SEO.
1. Page Indexing
Export the Search Console data from your legacy site: search analytics queries & pages, crawl errors, blocked resources, URL parameters, structured data, etc.
Then, set up a new property for your new domain. Ensure that it is set up for the proper version, accounting for http, https, and www. However, note that creating a property does not automatically make Googlebot aware of your site.
If the Indexing status remains stuck at "Processing data, please check again in a day or so" for many days, it's possible that Google hasn't found your site. You can try to resolve crawling issues by submitting a sitemap, adding links that point to your site, or submitting an indexing request for your homepage. If your pages are well linked among themselves, Google should be able to find all your pages from your home page.
Don't worry if your new domain doesn't get indexed right away. It can take weeks for new pages to be crawled, even when you submit a crawl request. This lag is even more common for migrations, in part because Google wants to be confident that the move is permanent, not some sort of temporary mistake. If the old domain is still indexed, then Google will see your new domain as duplicate content (i.e. not worth indexing). In most cases, this doesn't really matter because users can still find your site, they just need to go through the redirect. You don't need to do anything, just wait until 1) the old domain is recrawled and indexed as a nothing but a redirect, and 2) the new domain is crawled and eventually indexed in its own right.
On the other hand, you could be experiencing specific indexing issues that do require some action. If so, start by cross-checking the page indexing status on the URL Inspection Tool in GSC. If the status is different from what is on the page indexing report, treat the URL Inspection Tool as the source of truth.
This page outlines all of the possible indexing statuses and how to fix them. I will briefly mention two of the most common ones here below.
Discovered - currently not indexed
First, the "Discovered - currently not indexed" status means that Google knows about your page but hasn't crawled or indexed it yet. If you only have a few pages with this status, go ahead and request indexing via the URL inspection tool. There is a limit on how many URLs you can submit. While this is not defined in the docs, you can typically submit 10–15 URLs daily.
Crawled - currently not indexed
"Crawled - currently not indexed" is another GSC status, which means that Googlebot visited your page but didn't index it. Consequently, the page won't appear in Google Search. This could be due to low-quality or duplicate content, or poor website architecture. If the page(s) with "Crawled - currently not indexed" status are things like category pages or archive pages with thin content, then it's understandable that Google deemed those as not providing any value to the users to be indexed. If that's the case, you may ignore the "Crawled - currently not indexed" status.
Another question that often comes up here is, should you noindex category pages? However, there is no need to do so. In fact, noindexing too many category pages could even have a long-term negative effect. Google previously confirmed that if they see a persistent noindex, they will begin to treat the page a soft 404. As a side note, you may consider noindexing category pages if you see that they are causing direct conflicts with rankings, but that wouldn't be the case if the GSC status is "Crawled - currently not indexed."
Remember that Google can't index every page on the Internet. Its storage space is limited, so it needs to exclude low-quality content. In addition, Google's content evaluation is ongoing, so your SEO practices should be as well. Even if your page is indexed today, don't assume that it will always stay that way. So focus on making sure that your important pages are receiving internal links, and that you don't have orphan pages.
2. XML Sitemap
Generate a new XML sitemap with the new pages, site structure, and hreflang if needed. After the site migration, upload the new XML sitemap to Google Search Console. You can submit both the old and new sitemaps to clarify that a migration has taken place.
3. Canonical Tags
During a migration, you may temporarily have duplicate content, making it difficult for Google to decide which page to show in the SERPs. To make sure that Google shows the page that you intend, use canonical tags to signal which is the main version. Google will usually honor your signal and prioritize the canonical URL.
4. robots.txt
The robots.txt file tells crawlers which pages to access. If it is not properly configured, your new pages could be hidden from Google. After the migration, double check robots.txt to make sure that it includes the proper instructions for showing important pages and hiding old or irrelevant pages.
5. 301 Mapping
Typically, 301 permanent redirects are used to tell Google to transfer ranking signals and index the new pages. On the other hand, you can use 302 temporary redirects for pages that you are planning to sunset after the migration. However, if the original page has no backlinks or traffic, simply remove or replace any internal links pointing to them and then return a 404. Here's a flowchart by ahrefs explaining the process.
6. URL Status Codes
After the migration, crawl the new site to verify that there are no error status codes. 404 and 501 errors are the biggest SEO offenders that you should prioritize resolving. The second biggest offenders are links to 301 pages on the old domain. Update your links to point directly to the new domain, not through a redirect.
It's also good practice to create a custom 404 page to help users navigate your new site, in case they land on a page that no longer exists.
Use this Google Apps Script to bulk check status codes.
// Get http server response status code
function getStatusCode(url){
var options = {
'muteHttpExceptions': true,
'followRedirects': false
};
var statusCode ;
try {
statusCode = UrlFetchApp .fetch(url) .getResponseCode() .toString() ;
}
catch( error ) {
statusCode = error .toString() .match( / returned code (\d\d\d)\./ )[1] ;
}
finally {
return statusCode ;
}
}
// Exceed importxml limit
function importRegex(url, regexInput) {
var output = '';
var fetchedUrl = UrlFetchApp.fetch(url, {muteHttpExceptions: true});
if (fetchedUrl) {
var html = fetchedUrl.getContentText();
if (html.length && regexInput.length) {
output = html.match(new RegExp(regexInput, 'i'))[1];
}
}
Utilities.sleep(1000);
return unescapeHTML(output);
} 7. Keep the Old Domain
Unless the purpose of the migration was to sell the old domain, I recommend keeping it even after Google stops indexing it. When preparing the site migration, you should have redirected old pages to new pages on a per-page basis. But if those redirects are lost, the backlinks earned on the old domain will also be lost.
