Recently picked up "Python" and decided to try my hand at a "n00b" script to dump the contents of "Robots.txt" file residing on a webserver.
# Ask for Protocol and store it in protocol
protocol = input('Enter HTTP or HTTPS: ')
# Ask for URL or IP and store it in domain
domain = input('Enter URL or IP: ')
robots = "/robots.txt"
from urllib.request import Request, urlopen
print('Checking "Robots.txt" for:')
print(domain)
print()
from urllib.error import URLError, HTTPError
req = Request(protocol+"://"+domain+robots)
try:
response = urlopen(req)
except HTTPError as e:
print('The server couldn\'t fulfill the request.')
print('Error code: ', e.code)
except URLError as e:
print('We failed to reach the server.')
print('Reason: ', e.reason)
else:
print('Contents of "Robots.txt" is as follows.')
print()
with urlopen((protocol+"://"+domain+robots)) as stream:
print(stream.read().decode("utf-8"))
#Written by commandrine.
#Last updated on 22 Jun 2017.
Subscribe to:
Posts (Atom)
dnsaudit.py
Since I was on a roll with Copilot, I decided to automate DNSSEC auditing with the following Python script. import subprocess import sys im...
-
This annoying message popped up after I ran the update in avast! in Ubuntu yesterday. avast! crashes every time I attempt to launch it after...
-
I decided to install a digital certificate for my Gmail account. This is simple and free to set up. Apply for a free certificate from Comod...