Robot.txt is a text file not an HTML file.By using robot.txt file we tell search engine whether follow these links or not

follow from our website.

Syntax for follow all links in our websites:
User-agent: *
Disallow:

Syntax for don'tfollow the specified links in the robot.txt file:

User-agent: *
Disallow:/cig/example


search engine optimization in Bangalore
mobile application development company

Thank you for the post, it seems that Google is one of such search engines that pays attention to robot.txt file.
Many SE don't care, as such one should know this because of his/her files.