Python/Spider - Revision history

Tacoball at 03:46, 6 August 2019

2019-08-06T03:46:28Z

← Older revision		Revision as of 03:46, 6 August 2019
Line 1:		Line 1:
	=== Soup ===		== Soup ==
	<source lang="python">		<source lang="python">
	</source>		</source>

	=== Download ===		== Download ==
	<source lang="python">		<source lang="python">
	def download_file(url):		def download_file(url):

Tacoball at 03:46, 6 August 2019

2019-08-06T03:46:15Z

← Older revision		Revision as of 03:46, 6 August 2019
Line 1:		Line 1:
			=== Soup ===
			<source lang="python">
			</source>

			=== Download ===
	<source lang="python">		<source lang="python">
	def download_file(url):		def download_file(url):

Tacoball: Created page with " def download_file(url): local_filename = url.split('/')[-1] # NOTE the stream=True parameter below with requests.get(url, stream=True) as r:..."

2019-05-10T01:49:00Z

Created page with "<source lang="python"> def download_file(url): local_filename = url.split('/')[-1] # NOTE the stream=True parameter below with requests.get(url, stream=True) as r:..."

New page

<source lang="python">
def download_file(url):
local_filename = url.split('/')[-1]
# NOTE the stream=True parameter below
with requests.get(url, stream=True) as r:
r.raise_for_status()
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
# f.flush()
return local_filename
</source>