robot.txt and noindex update