Sitemaps and Google Verify
By Walt on Wednesday, January 2 2008, 12:16 - Technical info - Permalink
The wait is over. We took on board the number of requests for these two tools and decided you'd waited long enough. All users can now publish sitemaps, as well as Verify their websites.
For details on how to check for the sitemap and how to add Google Verify, read on...
Verify To add Google Verify to your site, please just follow the simple instructions below:
- Log in to your website
- Select “Admin” on the toolbar
- Visit the “Site Information” tab
- Click on the “Sign up for Google Webmaster Tools” button
- On the bottom of the Webmaster Tools panel click “Get your Google ID”
- Google should open in a new window and ask you to sign in
- You’ll be taken to the Google "Dashboard"
- Add your website address in blank field and click "Add Site"
- Look for the "Next step" box and select, "Verify Site"
- Go to the "Verification Method" drop down and select "Add a meta tag"
- Copy the code string e.g.: <meta name="verify-v1" content="UqbdIVXyU8m8PIvMSJLj+W6bf5Sa4An1CZcxazTgafs=" />
- Return to your website and paste the code string into the blank field in the "Webmaster Tools" panel
- Click "OK"
- Click "OK" on "Admin" panel
- Your site will now have the Google Verification code added to the HTML
N.B. The code string given above (<meta name="verify-v1" content="UqbdIVXyU8m8PIvMSJLj+W6bf5Sa4An1CZcxazTgafs=" />) is an example, please do not use this, it won't work! You must register with Google and request your own.
Sitemap We automatically generate an XML file which search engines will index when ever they visit your site. We automatically publish all pages visible in the menu and withhold those that are hidden. If you wish hidden pages to be published you will need to visit the hidden page, click on "Edit" and then go to "Page Settings". You can then select the check box for "Include this page in my Site Map".
To see your site map, you only need to add "/sitemap.xml" to the end of your website address (or URL) in the web browser address bar e.g. www.mysitename.com/sitemap.xml
If you see the message: "This XML file does not appear to have any style information associated with it. The document tree is shown below.", don't panic! This is being generated by the browser (most likely Firefox) and relates to 'how' the information is being displayed in your browser and does not affect what the search engines 'see'.
UPDATE (18/01/08):
For those of you checking site verification you will probably encounter some "403 (Forbidden)" errors. They will appear in list with entries similar to the two below:
- (http://www.mysite.com/_app/8296/partner/moonfruit/modules/) 403 (Forbidden) _?_
- (http://www.mysite.com/_app/8296/resources/) 403 (Forbidden) _?_
These are folders that the the web crawlers (or robots) should not be accessing as they contain and relate to software that runs your site and they do not contain any of your site content.
Our mistake has been to omit an 'exclude' comment telling robots not to look in these directories, which have restricted access anyway. So the web crawlers attempt to read these directories, are not permitted and subsequently generate a 'Forbidden' error.
This bug will be fixed very soon. We will simply request that the web crawlers do not check those directories and this will mean errors will not be generated.
These errors will have no impact on the indexing of your site or your page rankings!
UPDATE (08/02/08):
We have now modified the 'robots.txt' file to stop the web crawlers reading our application directories and throwing up 'errors'. However, now that we have blocked these directories you will probably see a report from Google that says: "URL restricted by robots.txt " and listing the same '.../_app/...' directories (as mentioned above). Google themselves recognise that some directories can be restricted for a logical reason and then simply continue crawling your site for valid content and links.
Just to reiterate, seeing a 'restricted' message is not a cause for alarm and this will not affect your site being indexed or impact your page rankings.
Comments
where can people see our sitemap?
Hi criptor,
This site map is really for search engines, not for navigation, so it's not indended to be viewed by your visitors. Thanks,
Joe
Thanks for listening guys. This type of moving forward is good for everyone. There is more to this than meets the eye. Excellent Job... now to verify 10+ sites....
Jeff
Do the Meta tags link to the site in general or do you set different tags for each page??
Hi Nick,
There are two places to set meta tags, either on the 'site information' panel in Admin, or on the 'page settings tab'. If you set them on the site information panel, these become the default tags for all of your pages, unless further tags have been set on the page settings panel, in which case these take priority. So the answer is that you can set tags on each page, but also set a default for any page without its specific tags. I hope that helps,
Joe
Hey Joe
Would just like to point one thing out...... With the google verify, it will only allow you to put 1 address in... The point im gettin at is if you have more than 1 domain name, like me i have 4, i want to be able to put all 4 of my domains on google, which it will then generate me a code per domain, so basically im asking could you edit the feature and allow sitemaker to accept more than 1 verify code.
Hope you understand what i mean. Thanks - keep up the good work!
The meta verify code is only checked once when you click verify on the Webmaster Tools page. If you have multiple domains, just repeat the process with each domain.
Hi guys,
I need to resubmit my site to google because my (dipstick) friend added lots of keywords in white on a white background as a favour! Now, Google have removed me. Anyway, Google said they will invite me back once i have verified the site. However, copying/pasting the code string places the tag Google are looking for in a different part of the page, so it says I haven't done it. Any help would be appreciated.
Hi Matt,
Have you managed to correct the Google Verify problem yet? If not can you submit a query to Support so that we can log your site name and we can investigate for you.
Thanks
hello, i want to ask why after verified my website on google webmaster tools, and check the sitemap informations, google tell me that they coldnt find my site map, andthis shows:
(General HTTP error: 404 not found
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit),
please help me as becouse of this problem google still didnt give me a pagerank.
thanks
Great job on making verification with our Webmaster Tools pain-free for the webmaster!
I just wanted to add a small comment regarding verifying a site for multiple owners: you will need to keep the verification code on your site for all users. We periodically re-confirm the status of all sites so if for instance you decide to buy a site from someone else, they will not have full access to the Webmaster Tools for that site when you remove their code.
Hi Daniele,
Verify and sitemap are two entirely different features. One is solely offered by Google, the other allows all search engines to more accurately define your site pages (although Google does make good use of this).
There are two ways to make use of sitemap. You can submit a file to Google or we can auto-generate one for you. We chose the latter as it enables other search engines to index your site, and not just Google.
You do not and should not be supplying Google with any files relating to sitemap. This is probably why you are getting an error. Although just in case I have misunderstood I'll highlight the steps required for enabling sitemaps:
As for page ranking, while sitemap will improve the process, it not the be-all-and-end-all. Before we introduced sitemap at Christmas time last year we had many customers who for years, consistently ranked at number one in their category by using simple SEO procedures.
In a nutshell Daniele, we look after sitemap automatically for you so you don't need to supply Google with any files. Your page ranking can be improved by simple steps that have been outlined in other articles on this blog. Please just have a read of those articles and try some or all of the recommendations and you should see an improvement in the ranking of your site.
I hope that helps.
Walt
On On Thursday, January 17 2008, 22:49 by Eirik said "
The meta verify code is only checked once when you click verify on the Webmaster Tools page. If you have multiple domains, just repeat the process with each domain."
About "The meta verify code is only checked once "
Can one remove the verify line once Google has verifed it and accepted the site?
Or must it stay for as long as one uses the Google analytics?
Thanks
Roland
SORRY! just saw this advice from John:
------------------------------------------------
On Wednesday, February 6 2008, 11:38 by JohnMu
Great job on making verification with our Webmaster Tools pain-free for the webmaster!
I just wanted to add a small comment regarding verifying a site for multiple owners: you will need to keep the verification code on your site for all users. We periodically re-confirm the status of all sites so if for instance you decide to buy a site from someone else, they will not have full access to the Webmaster Tools for that site when you remove their code.
-----------------------------
So I guess this means one should not have a line like this in their code:
<meta name="verify-v1" content="#">
Roland
unfortunately, I do have <meta name etc as my site name now - anyone know how to remove it?!
Cheers
Hi! I am indeed getting message: "This XML file does not appear to have any style information associated with it. The document tree is shown below." for my sitemap.xml. I am not panicking, but since it is displayed on my site, I'd prefer it to have the same style as the rest of my site. It is Wordpress Semiologic Pro custom theme. Is there any way I can achieve it?
Thanks,
Anjelina
Just an update on the sitemaps for Moonfruit staff:
If you login to Google Webmasters you will notice all sites created with Sitemaker are showing a warning for the Sitemaps submitted.
The warning states the following:
Sitemap errors and warnings
Line Status Details
- All the URLs in your Sitemap have the same priority.
All the URLs in your Sitemap are set to the same priority (not the default priority). Priority indicates the importance of a particular URL relative to other URLs on your site and doesn't impact your site's performance in search results. If all URLs have the same priority, Google can't tell which are more important.
Can Moonfruit please advise.
Thanks.
Hi James,
At present we don't have a facility for users to indicate page priority so we've flagged all pages as having the same priority. While this isn't ideal it shouldn't be a massive hindrance to your SEO efforts (if any at all).
Priority is just a guide to google and it only counts internally to the site (you can't for example say that your site is more important than another). If you have 2 pages that return with the same ranking on a particular term then the higher priority one would be considered first.
Ultimately it'll work things out on it's own.
I'm not sure if there are plans to add user set priority, at the moment we have some exciting new features that are taking up a lot of our development time.
cheers,
Dan
web developer
Hi, I'm having difficulty verifying my website. I think the meta tag keeps ending up in the wrong place. I've tried them on the 'site information' panel & the page settings in sitemaker. I just get the message, 'we couldn't find the verification meta tag.' The site map I submitted has been accepted but unless I can verify I'm the official owner of my site Google's robots won't be able to index my site & consequently it won't get properly ranked. I'm working to promote an artist & raise their public profile so rankings are particularly important. Any help at all with this would be greatly appreciated. Thanks
Katti
Hi Katti,
You won't be able to verify your site unless you 'point' your A-record. At present you are using 'masked forwarding' which means your site sits within an HTML frame so Google is actually indexing the frame not your site. This is important as your site content which includes your meta data, verification code, sitemap etc are being ignored.
To correct this you will need to contact your domain name provider and ask them, unless you can do this yourself, to point your 'DNS A-record' to our web servers at 146.101.249.107. More on this is available in our FAQs.
Thanks
Can you help someone with chronic stupidy?
I'm trying to sort out Google analytics for which this forum is proving extremely helpful. The latest problem I've encountered however relates to sitemaps. When I enter my site address/sitemap.xml in the browser window nothing appears but it does when I enter mysite.moonfruit.com/sitemap.xml. Unfortunately Google Webmaster Tools only allows me to suffix www.bailang.co.uk and not bailang.moonfruit.com.
I suppose that I could create new accounts for bailang.moonfruit.com but this would be a pain changing dozens of pages and I run around 9 sites. Plus I dont want the SE's to list me as bailang.moonfruit.com just bailang.co.uk
Any help please guys. Please bear in mind that I'm an idiot!
Hi,
The problem you are highlighting is a common mistake. Your domain names are using masked forwarding, which is like a frame around your site. There is little content in this frame and certainly no site content, no META DATA, and no sitemap.xml. All the content is in the site and the only way to ensure that search engines like Google pick up the essential content including the sitemap is to using DNS A-record pointing.
This means that your domain name provider 'points' your A-record towards our web servers (146.101.249.107) so that anyone using one of your domain names actually visits the site not a frame containing the site.
We do provide FAQ on how to do this as well as details within the Site Addresses tab of Admin. The only thing to be wary of is if your domain name provider asks about 'nameservers'. This is a different process entirely. They only need to know that you require the A-record (or Zone file) to point to our web servers using the IP address given above.
Once set up this will ensure that you, or the search engines, will not have problems seeing the sitemap file.
If you have any other questions about this process it's probably best if you use the Submit a Query form so that we can give you more specific answers and much quicker.
Walt
Are we able to submit our sitemap to multiple search engines?
I've submitted to google without a problem but now trying to submit to MSN too but there doesn't seem to be anyway to do this.
Pasting the code into the box for the google code doesn't work as the meta name stays as verify 1.
Is it impossible?
Hi Wing,
Yes you can submit your sitemap to different search engines but not by using a code block. We automatically generate an XML file called "sitemap.xml" which is picked up by web crawlers.
For sites built using SiteMaker software, you should only be adding your site address to the search engine sitemap section not code of any sort.
You can check your site map by adding "/sitemap.xml" to the end of your site address e.g. mysitename.moonfruit.com/sitemap.xml. This like the meta data you would have added will be read by the web crawlers from each search engine. And please note that only pages that you have marked as public will show up.
As for MSN you could have a read of http://www.seo-expert-services.co.u... which has useful information.
Basically you just need to insert your site address (replace the 'yoursite' bit) into the following link: http://api.moreover.com/ping?u=http...
So click on the link then change the site address bit in the browser window and hit enter and it should check your sitemap.
Give that a try and see how you get on.
Walt
Hi Walt,
Firstly, 'Moonfruit' - what a fantastic setup you guys are running. I've only been a member for a couple of weeks but I can safely say that the software, the service and the end results are far beyond anything I've used in the past.
Secondly, I really hope you can help me. I'm having huge problems getting indexed by google and its absolutely paramount to my business that I get it done soon.
I have fr the pst couple of days followed all of the excellent past advice you and your colleagues have given to my fellow subscribers and I think I have followed it all correctly. However, I think I have made a booboo some where along the line. Just to summarise where I am:
My domain name is from 1&1.co.uk.
Last week I used their control panel to point the A-record to the address you give and the site worked.
A few days ago I went through the procedure as instructed on google. ie. verification via meta tag; site map upload.
I added all the necessary keywords and descriptions on my website.
I don't hve any external links yet but I thought at least typing my website into google would yield a result. None.
Googlewebmaster shows no errors so wonder if Ihave done something wrong at the moonfruit end. Is there anything you can point me at to look at or check?
Also checking http://www.freewebsubmission.com/cg...
gives a couple of red errors.
Hope you can help!
Many thanks,
Jon
I don't have any beef with 1and1's web hosting. However, I curse the day I ever listed any domains with them. Check out Red Flag for some customer comments.
It should be a warning that when their phone is answered by a recording that tells you if you have gotten a notice from NCO, a collection agency, to press a certain number. If they are a reputable business, why do they have to send so many accounts to a collection agency?
The Eastern Pennsylvania /Washington DC Better Business Bureau have given 1and1 an “unsatisfactory” rating. It is my understanding that 1and1 gets a complaint almost every day. An investigative journalist, Kelli Jack has written a series of articles about them and has a pending law suit. She says: “1and1 Should Be Shut Down.”
I have submitted my sitemap ...with all the information as suggested by you guys ... but when i try to verify it in from webmaster tools in google ... i get a
"We've detected that your verification file returns a status of 404 (Not found) in the header"- message saying verification is "NOT VERIFIED" i have had a number of these.... in the past ... please advise
J
Return to your website and paste the code string into the blank field in the "Webmaster Tools" panel
I cant find the above - aplogies for my presumed stupidity but I just dont see where this would go!
Thanks
Dave
Hi Dave,
Go to (Edit > Manage your site >) Admin > Site Information and if you haven't set this up you will see Sign up for Google Webmaster tools. Once signed up you can drop the code block into the field on the Google Webmaster Tools panel and save.
If you get stuck please contact Support and we will send you more details.
Thanks
Hi Folks
I've recived this from Google-any ideas.Thanks and apologies if already covered!
Google was unable to crawl the URL due to a robots.txt restriction. This can happen for a number of reasons. For instance, your robots.txt file might prohibit the Googlebot entirely; it might prohibit access to the directory in which this URL is located; or it might prohibit access to the URL specifically. Often, this is not an error. You may have specifically set up a robots.txt file to prevent us from crawling this URL. If that is the case, there's no need to fix this; we will continue to respect robots.txt for this file.
If a URL redirects to a URL that is blocked by a robots.txt file, the first URL will be reported as being blocked by robots.txt (even if the URL is listed as Allowed in the robots.txt analysis tool).
Hi Dave,
This was covered in the UPDATE 08/02/08 above. But to reiterate this, we do block specific directories for technical reasons and this will have no impact on your site being crawled. All other pages on your site are free to be indexed by the web crawlers.
Hello
I've just set up AdSense for content on my site. I want to do the same for AdSense for search. However, I cant see any feature supporting it. Does this exist?
Thanks
Barry
I am getting a xx4 HTTP crawl error. What does this mean, and what should I do to correct it. I am still not getting indexed with google which is preventing my web site from operating.
any help would be greatly appreciated.
Thanks
Hi Andy,
I checked the URL you supplied in your post versus what you have on your account and they differ slightly. It may just be a typo in the URL on your post above - an extra 's' but it may be worth just checking what you have registered with Google.
If that checks out OK, can you please submit a ticket and give us more detail on the error message. Was it actually a XX4 error or a 4XX error - there is a difference and any additional info will help. For example a full error description, the list of URLs it applies to and any other details about your site or pages on it will help. As we have no access to your Analytics account, nor control over what Google do, we are running blind to some degree and need your help in establishing the facts so we can resolve this quickly.
Thanks
Hi, could somebody from moonfruit please respond, Bing now have web master tools which work quite well with the ecommerce sites I manage, however this does require a Meta Tag just as google webmaster tools is there any way of adding this tag to MF/Sitemaker sites at the moment?
Thanks in advance
John
Hi,
While Bing might be a Microsoft product and a replacement for Live Search, it has only 'recently' been unveiled. We will monitor its popularity and look at support for the webmaster tools if the market penetration reaches a suitable adoption level. This is just to ensure our resources are channelled into the right developments. While we have only had a few requests for Bing support, we will watch it's pick-up closely.
Thanks for the request though.
- Walt