Move the content-type check back to the proper indent level, hopefully this

makes the bot check for an html Content-Type and not download every file.
This commit is contained in:
Reese Moore
2011-12-30 22:37:42 -05:00
parent 92ea5b5310
commit d972e70529

View File

@@ -153,6 +153,7 @@ def gettitle(uri):
try: mtype = info['content-type']
except:
return None
if not (('/html' in mtype) or ('/xhtml' in mtype)):
return None