# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-agent: * # Disallow: / # The sitemap is generated straight to S3 by `rake sitemap:refresh` (config/sitemap.rb sets # an AwsSdkAdapter), so it is NOT served from this host -- nothing routes /sitemap.xml and # public/sitemap* is gitignored, which is why the old www URL here 404'd. robots.txt may # name a sitemap on another host, so the S3 URL is pointed at directly. # # This is the sitemap *index*; it lists one child per locale. `compress = false` in # config/sitemap.rb, so the file is .xml -- the ".xml.gz" this line used to advertise has not # been regenerated since compression was turned off. Sitemap: https://sweetescape-production-sg.s3-ap-southeast-1.amazonaws.com/misc/sitemaps/sitemap.xml