Move the content-type check back to the proper indent level, hopefully this
makes the bot check for an html Content-Type and not download every file.
This commit is contained in:
@@ -153,6 +153,7 @@ def gettitle(uri):
|
||||
try: mtype = info['content-type']
|
||||
except:
|
||||
return None
|
||||
|
||||
if not (('/html' in mtype) or ('/xhtml' in mtype)):
|
||||
return None
|
||||
|
||||
|
||||
Reference in New Issue
Block a user