Thursday, January 11, 2018

How to Download Files in Python

How to Download Files in Python

Python provides several ways to download files from the internet. This can be done over HTTP using the urllib package or the requests library. This tutorial will discuss how to use these libraries to download files from URLs using Python.

REQUESTS

The requests library is one of the most popular libraries in Python. Requests allow you to send  HTTP/1.1 requests without the need to manually add query strings to your URLs, or form-encode your POST data.

With the requests library, you can perform a lot of functions including:

  • adding form data,
  • adding multipart files,
  • and accessing the response data of Python

MAKING REQUESTS

The first you need to do is to install the library and it's as simple as:

To test if the installation has been successful, you can do a very easy test in your  python interpreter by simply typing:

If the installation has been successful, there will be no errors.

HTTP requests include:

  • GET
  • POST
  • PUT
  • DELETE
  • OPTIONS
  • HEAD

Making a GET request

Making requests is very easy as illustrated below.

The above command will get the google web page and store the information in thereq variable. We can then go on to get other attributes as well.

For instance, to know if fetching the google web page was successful, we will query the status_code.

What if we want to find out the encoding type of the Google web page?

You might also want to know the contents of the response.

This is just a truncated content of the response.

Making a POST Request

In simple terms, a POST request used to create or update data. This is especially used in the submission of forms.

Let's assume you have a registration form that takes an email address and password as input data, when you click on the submit button for registration, the post request will be as shown below.

Making a PUT Request

A PUT request is similar to a POST request. Its used to update data.For instance, the API below shows how to do a PUT request.

Making a DELETE Request

A DELETE request, like the name suggests, is used to delete data. Below is an example of a DELETE request

urllib Package

urllib is a package that collects several modules for working with URLs namely:

urllib.request  offers a very simple interface, in the form of the urlopen function capable of fetching URLs using a variety of different protocols. It also offers a slightly more complex interface for handling basic authentication, cookies, proxies e.t. c.

How to Fetch URLs With urllib

The simplest way to use urllib.request is as follows:

If you wish to retrieve an internet resource and store it, you can do so via the urlretrieve() function.

Downloading Images With Python

In this example, we want to download the image available on this link using both the request llibrary and urllib module. 

Download PDF Files With Python

In this example, we will download a pdf about google trends from this link.

Download Zip Files With Python

In this example, we are going to download the contents of a GitHub repository found in this link and store the file locally.

Download Videos With Python

In this example, we want to download  the video lecture available on this page

Conclusion

This tutorial has covered the most commonly used methods to download files as well as the most common file formats. Even though you will write less code  when using  the urllib module, the requests module is preferred due to its simplicity, popularity and a wide array of features including:

  • Keep-Alive & Connection Pooling
  • International Domains and URLs
  • Sessions with Cookie Persistence
  • Browser-style SSL Verification
  • Automatic Content Decoding
  • Basic/Digest Authentication
  • Elegant Key/Value Cookies
  • Automatic Decompression
  • Unicode Response Bodies
  • HTTP(S) Proxy Support
  • Multipart File Uploads
  • Streaming Downloads
  • Connection Timeouts
  • Chunked Requests
  • .netrc Support


1 comment: