How to Retrieve Data From Internet in Python

As if you are now familiar with Python Programming Language, you are excited to get in the game by going big. From Going big, I meant to develop something that looks cool to everyone rather than just making Calculators and all basic things. if not read this article on how to set up python environment. So, now today I am going to teach you about How to retrieve Data from Internet. So, sit tight and open your Python IDE. For this tutorial, I am going to use the python 2.7.10 and you’re gonna need an Internet Connection for this.

Retrieve Data From Internet in Python

So first let me give you a brief idea what we gonna do here. First, we will make a connection to a particular website. If the connection is made it will it print out the connection code and if not, it’ll do the same. So let’s begin.

retrieve data from internet in python

Open Python IDLE or any other IDE you’re using and type the following code.

import urllib2
def main():

  webUrl=urllib2.urlopen("http://kamalthakur.com")
  print "result code: "+str(webUrl.getcode())

if __name__ == "__main__":
  main()

Now let me Explain what I did here. In the first line, I imported something called, “urllib2”. The urllib2 is a module which provides the utilities for connecting to a web addresses and retrieve data from it. In the second line of code, we defined the main function. If you have some previous programming background like in C/C++ you know what the main function is.

Now below that, we are making a variable called “webUrl” and we are calling “urlopen()” method from “urllib2” library to make a connection to the website, which in this case, is my own blog. Now there are many outcomes of this like maybe connection were made, maybe not or maybe you have some internet connection problem. Now we want to know whether the connection was made or not. So, we print the string “result code ” with the connection code. For example, if the connection was made, it will print code 200 and if not, 404 will be the output. This line will print the connection code using the getcode() method. Note that in order to concatenate it with the string, we need to manually make the code to a string. In next line, we are simply calling main.

Now if we run the code, you’ll see output “result code: 200”. Now you’ll say we didn’t retrieve data from internet in python. Now let’s do that. To retrieve data from internet in Python, add the line of code –

import urllib2
def main():

  webUrl=urllib2.urlopen("http://kamalthakur.com")
  print "result code: "+str(webUrl.getcode())
  data=webUrl.read()
  print data

if __name__ == "__main__":
  main()

Now I added two more line of code. I made a new variable, “data” and used read() method to read all the data inside the “webUrl” variable I created above. It will put all the data inside the new “data” variable. After it, we will print the data. By printing it, we will receive all the source code of the web address. So that is how to retrieve data from internet in python.

If you have any queries or question, please feel free to comment below. We are always happy to help. Happy Coding.

20 Comments

  1. Marek Turnovec January 19, 2016
    • Kamal Thakur January 22, 2016
      • Marek Turnovec January 31, 2016
        • Kamal Thakur February 1, 2016
  2. Davi Wesley January 29, 2016
  3. Brada March 19, 2016
    • Kamal Thakur April 9, 2016
    • Fabrizio May 10, 2016
      • Fabrizio May 10, 2016
  4. Eddie March 20, 2016
    • Kamal Thakur April 9, 2016
  5. Sameer March 20, 2016
    • Kamal Thakur April 9, 2016
  6. zeydi April 5, 2016
    • Kamal Thakur April 7, 2016
      • zeydi April 11, 2016
  7. Yandi May 10, 2016
  8. Edward Mccain May 16, 2016
  9. Gilbert Sholler October 18, 2016