1Sheeld, The Ultimate Arduino Shield Review and Giveaway

A single Arduino shield that can link with an Android device and provide a multitude of sensors and advanced functions? Yep – that’s a thing now. The 1Sheeld, in fact.

From developers Integreight, 1Sheeld blew it’s admittedly humble Kickstarter goal of $10k out of the water 8 times over, but is now available to all from Amazon for $55. The 1Sheeld isn’t designed to replace the Arduino though – it supplements it, giving your Arduino projects access to a full complement of smartphone sensor data, and far more besides. I can’t tell you about alternatives, because there aren’t any.

Design

No larger than any other standard Arduino shield, the 1Sheeld is based around an ATMEGA162, and includes a BlueTooth module underneath. The pin configuration is such that you’ll need an Arduino Uno revision 3 or later (other Arduino models will vary, but anything newer than the Uno rev3 should be fine). Each pin is passed through, so you can place additional shields on top, or use the standard I/O pins as needed with jump leads.

1sheeld - contents

Two toggle switches are attached to the board. The first chooses between 5v and 3.3v; 5v is the standard for use with most Arduinos, so if you’re unsure which to use then stick with 5v.

The second toggle – labelled UART SWITCH – will be in constant use; it switches between BlueTooth link, and Arduino programming mode. This is due to the fact it uses the same set of serial communication pins for both functions, but it’s not difficult to figure out. If you find no sensor data is being received, toggle it over – it won’t break the board if you get it wrong.

Ads by Google

1sheeld - underside

3 micro LEDs indicate send, receive, and BlueTooth status.

All in, it’s a very professionally made shield, with solid solder joints and a well laid out design. The white on black labels are clear and easy to read, with no misprints or irregular placements.

Features

By my count, there are currently 37 different “shields” that the 1Sheeld can emulate. 11 of those are sensor shields –things like: accelerometer, light, gyroscope, pressure. Another 10 are basic I/O operations, such as: buzzer, keyboard, microphone, gamepad, and a variety of buttons. 7 more handle communications and social networking: Twitter, Facebook, phone, email, Skype. Finally, there are 9 special function shields, including: voice recognition, speech synthesis, notifications, camera, LCD and datalogger. You simply tap them on the app to add that functionality, though some will require additional setup such as authorizing Twitter.

1sheeld - linked to android

I think we can all agree it’s a phenomenal list of features – the sheer value for money of having all these features from a single $55 purchase is quite astonishing (not factoring in your phone, of course).

Programming The 1Sheeld

For your very first project, I’d recommend just running through the Getting Started tutorial on the 1Sheeld website. In it, you’ll load some sample code on your Arduino which listens to the smartphone mic level, and then handles the logic to Tweet out a message.

Once you’ve uploaded the sample code, switch the UART-SWITCH to the linked icon (which enables connection with your phone), and hit SCAN in the app. After that, it’s a simple case of enabling the mic and Twitter shields.

At first, my microphone level never reached above 50, so the tweet just wasn’t triggered – you may also need to adjust the sample Arduino code to work with a lower level. That worked nicely, and here’s the tweet it produced (you need to authenticate on the Android side so it knows who to send from):

It’s important to note that the logic for this was running on the Arduino itself – the 1Sheeld didn’t connect the mic and Twitter shields in any way – it merely relayed the mic level to the Arduino, and provided a function for posting tweets. At any point you would be able to drop in your physical components or replacement functions, and the program would still function as intended.

1sheeld - mounted

After that, I was eager to try out the voice recognition features to interact with my newly built giant LED display (a whopping 210 pixels in total – tutorial coming soon!). Integrating the example voice recognition sketch with my Adafruit Matrix library test code took all of 5 minutes, producing the voice control demo you can see in the video at the start of the review.

If I’m honest, I felt like I was cheating. Making incredible things with the Arduino really shouldn’t be this easy!

Should You Buy The 1Sheeld?

The 1Sheeld is an unbelievably clever bit of kit, no doubt about it. The range of features it adds; the sensors it can become; the sheer versatility is outstanding.

1sheeld - mounted side

There are two situations I think the 1Sheeld is ideal for.

First, if you’ve purchased an Arduino starter kit (what to expect in your starter kit), completed all the included projects, and are starting to get bored, the 1Sheeld will keep you amused for at least another year or so – there’s just so much you can do with it. You’ll add some whole new dimensions to your project that just wouldn’t be possible with regular hardware shields (like voice recognition), so your imagination will be the new limit.

That said, it’s not going to be useful for long term embedded projects – it needs to stay tethered to your phone after all. If you’re only using some relativity simple aspect of the 1Sheeld that could be accomplished with a $5 component, it’s completely unfeasible to keep your $55 1Sheeld and Android phone dedicated to the task. Which brings me onto my second use case: when you’re desperate to get started programming, but some critical sensor or dedicated shield haven’t arrived yet. Instead of delaying the project or fudging the function to work with random numbers, you can get stuck right in.

Advertisements

ELECTRONICS 101-PROGRAMMING WITH PYTHON-4(ADVANCED)

Python Sending Email using SMTP

Simple Mail Transfer Protocol (SMTP) is a protocol, which handles sending e-mail and routing e-mail between mail servers.

Python provides smtplib module, which defines an SMTP client session object that can be used to send mail to any Internet machine with an SMTP or ESMTP listener daemon.

Here is a simple syntax to create one SMTP object, which can later be used to send an e-mail:

import smtplib

smtpObj = smtplib.SMTP( [host [, port [, local_hostname]]] )

Here is the detail of the parameters:

  • host: This is the host running your SMTP server. You can specifiy IP address of the host or a domain name like tutorialspoint.com. This is optional argument.
  • port: If you are providing host argument, then you need to specify a port, where SMTP server is listening. Usually this port would be 25.
  • local_hostname: If your SMTP server is running on your local machine, then you can specify just localhost as of this option.

An SMTP object has an instance method called sendmail, which will typically be used to do the work of mailing a message. It takes three parameters:

  • The sender – A string with the address of the sender.
  • The receivers – A list of strings, one for each recipient.
  • The message – A message as a string formatted as specified in the various RFCs.

Example:

Here is a simple way to send one e-mail using Python script. Try it once:

#!/usr/bin/python

import smtplib

sender = 'from@fromdomain.com'
receivers = ['to@todomain.com']

message = """From: From Person <from@fromdomain.com>
To: To Person <to@todomain.com>
Subject: SMTP e-mail test

This is a test e-mail message.
"""

try:
   smtpObj = smtplib.SMTP('localhost')
   smtpObj.sendmail(sender, receivers, message)         
   print "Successfully sent email"
except SMTPException:
   print "Error: unable to send email"

Here, you have placed a basic e-mail in message, using a triple quote, taking care to format the headers correctly. An e-mail requires a From, To, and Subject header, separated from the body of the e-mail with a blank line.

To send the mail you use smtpObj to connect to the SMTP server on the local machine and then use the sendmail method along with the message, the from address, and the destination address as parameters (even though the from and to addresses are within the e-mail itself, these aren’t always used to route mail).

If you’re not running an SMTP server on your local machine, you can use smtplib client to communicate with a remote SMTP server. Unless you’re using a webmail service (such as Hotmail or Yahoo! Mail), your e-mail provider will have provided you with outgoing mail server details that you can supply them, as follows:

smtplib.SMTP('mail.your-domain.com', 25)

Sending an HTML e-mail using Python:

When you send a text message using Python, then all the content will be treated as simple text. Even if you will include HTML tags in a text message, it will be displayed as simple text and HTML tags will not be formatted according to HTML syntax. But Python provides option to send an HTML message as actual HTML message.

While sending an e-mail message, you can specify a Mime version, content type and character set to send an HTML e-mail.

Example:

Following is the example to send HTML content as an e-mail. Try it once:

#!/usr/bin/python

import smtplib

message = """From: From Person <from@fromdomain.com>
To: To Person <to@todomain.com>
MIME-Version: 1.0
Content-type: text/html
Subject: SMTP HTML e-mail test

This is an e-mail message to be sent in HTML format

<b>This is HTML message.</b>
<h1>This is headline.</h1>
"""

try:
   smtpObj = smtplib.SMTP('localhost')
   smtpObj.sendmail(sender, receivers, message)         
   print "Successfully sent email"
except SMTPException:
   print "Error: unable to send email"

Sending Attachments as an e-mail:

To send an e-mail with mixed content requires to set Content-type header to multipart/mixed. Then, text and attachment sections can be specified within boundaries.

A boundary is started with two hyphens followed by a unique number, which can not appear in the message part of the e-mail. A final boundary denoting the e-mail’s final section must also end with two hyphens.

Attached files should be encoded with the pack(“m”) function to have base64 encoding before transmission.

Example:

Following is the example, which will send a file /tmp/test.txt as an attachment. Try it once:

#!/usr/bin/python

import smtplib
import base64

filename = "/tmp/test.txt"

# Read a file and encode it into base64 format
fo = open(filename, "rb")
filecontent = fo.read()
encodedcontent = base64.b64encode(filecontent)  # base64

sender = 'webmaster@tutorialpoint.com'
reciever = 'amrood.admin@gmail.com'

marker = "AUNIQUEMARKER"

body ="""
This is a test email to send an attachement.
"""
# Define the main headers.
part1 = """From: From Person <me@fromdomain.net>
To: To Person <amrood.admin@gmail.com>
Subject: Sending Attachement
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=%s
--%s
""" % (marker, marker)

# Define the message action
part2 = """Content-Type: text/plain
Content-Transfer-Encoding:8bit

%s
--%s
""" % (body,marker)

# Define the attachment section
part3 = """Content-Type: multipart/mixed; name=\"%s\"
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename=%s

%s
--%s--
""" %(filename, filename, encodedcontent, marker)
message = part1 + part2 + part3

try:
   smtpObj = smtplib.SMTP('localhost')
   smtpObj.sendmail(sender, reciever, message)
   print "Successfully sent email"
except Exception:
   print "Error: unable to send email"

Python Multithreaded Programming

Running several threads is similar to running several different programs concurrently, but with the following benefits:

  • Multiple threads within a process share the same data space with the main thread and can therefore share information or communicate with each other more easily than if they were separate processes.
  • Threads sometimes called light-weight processes and they do not require much memory overhead; they care cheaper than processes.

A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer that keeps track of where within its context it is currently running.

  • It can be pre-empted (interrupted)
  • It can temporarily be put on hold (also known as sleeping) while other threads are running – this is called yielding.

Starting a New Thread:

To spawn another thread, you need to call following method available in thread module:

thread.start_new_thread ( function, args[, kwargs] )

This method call enables a fast and efficient way to create new threads in both Linux and Windows.

The method call returns immediately and the child thread starts and calls function with the passed list of agrs. When function returns, the thread terminates.

Here, args is a tuple of arguments; use an empty tuple to call function without passing any arguments. kwargs is an optional dictionary of keyword arguments.

Example:

#!/usr/bin/python

import thread
import time

# Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s" % ( threadName, time.ctime(time.time()) )

# Create two threads as follows
try:
   thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print "Error: unable to start thread"

while 1:
   pass

When the above code is executed, it produces the following result:

Thread-1: Thu Jan 22 15:42:17 2009
Thread-1: Thu Jan 22 15:42:19 2009
Thread-2: Thu Jan 22 15:42:19 2009
Thread-1: Thu Jan 22 15:42:21 2009
Thread-2: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:25 2009
Thread-2: Thu Jan 22 15:42:27 2009
Thread-2: Thu Jan 22 15:42:31 2009
Thread-2: Thu Jan 22 15:42:35 2009

Although it is very effective for low-level threading, but the thread module is very limited compared to the newer threading module.

The Threading Module:

The newer threading module included with Python 2.4 provides much more powerful, high-level support for threads than the thread module discussed in the previous section.

The threading module exposes all the methods of the thread module and provides some additional methods:

  • threading.activeCount(): Returns the number of thread objects that are active.
  • threading.currentThread(): Returns the number of thread objects in the caller’s thread control.
  • threading.enumerate(): Returns a list of all thread objects that are currently active.

In addition to the methods, the threading module has the Thread class that implements threading. The methods provided by the Thread class are as follows:

  • run(): The run() method is the entry point for a thread.
  • start(): The start() method starts a thread by calling the run method.
  • join([time]): The join() waits for threads to terminate.
  • isAlive(): The isAlive() method checks whether a thread is still executing.
  • getName(): The getName() method returns the name of a thread.
  • setName(): The setName() method sets the name of a thread.

Creating Thread using Threading Module:

To implement a new thread using the threading module, you have to do the following:

  • Define a new subclass of the Thread class.
  • Override the __init__(self [,args]) method to add additional arguments.
  • Then, override the run(self [,args]) method to implement what the thread should do when started.

Once you have created the new Thread subclass, you can create an instance of it and then start a new thread by invoking the start(), which will in turn call run() method.

Example:

#!/usr/bin/python

import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print "Starting " + self.name
        print_time(self.name, self.counter, 5)
        print "Exiting " + self.name

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            thread.exit()
        time.sleep(delay)
        print "%s: %s" % (threadName, time.ctime(time.time()))
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

print "Exiting Main Thread"

When the above code is executed, it produces the following result:

Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Thu Mar 21 09:10:03 2013
Thread-1: Thu Mar 21 09:10:04 2013
Thread-2: Thu Mar 21 09:10:04 2013
Thread-1: Thu Mar 21 09:10:05 2013
Thread-1: Thu Mar 21 09:10:06 2013
Thread-2: Thu Mar 21 09:10:06 2013
Thread-1: Thu Mar 21 09:10:07 2013
Exiting Thread-1
Thread-2: Thu Mar 21 09:10:08 2013
Thread-2: Thu Mar 21 09:10:10 2013
Thread-2: Thu Mar 21 09:10:12 2013
Exiting Thread-2

Synchronizing Threads:

The threading module provided with Python includes a simple-to-implement locking mechanism that will allow you to synchronize threads. A new lock is created by calling the Lock() method, which returns the new lock.

The acquire(blocking) method of the new lock object would be used to force threads to run synchronously. The optional blocking parameter enables you to control whether the thread will wait to acquire the lock.

If blocking is set to 0, the thread will return immediately with a 0 value if the lock cannot be acquired and with a 1 if the lock was acquired. If blocking is set to 1, the thread will block and wait for the lock to be released.

The release() method of the the new lock object would be used to release the lock when it is no longer required.

Example:

#!/usr/bin/python

import threading
import time

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print "Starting " + self.name
        # Get lock to synchronize threads
        threadLock.acquire()
        print_time(self.name, self.counter, 3)
        # Free lock to release next thread
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print "%s: %s" % (threadName, time.ctime(time.time()))
        counter -= 1

threadLock = threading.Lock()
threads = []

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

# Add threads to thread list
threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()
print "Exiting Main Thread"

When the above code is executed, it produces the following result:

Starting Thread-1
Starting Thread-2
Thread-1: Thu Mar 21 09:11:28 2013
Thread-1: Thu Mar 21 09:11:29 2013
Thread-1: Thu Mar 21 09:11:30 2013
Thread-2: Thu Mar 21 09:11:32 2013
Thread-2: Thu Mar 21 09:11:34 2013
Thread-2: Thu Mar 21 09:11:36 2013
Exiting Main Thread

Multithreaded Priority Queue:

The Queue module allows you to create a new queue object that can hold a specific number of items. There are following methods to control the Queue:

  • get(): The get() removes and returns an item from the queue.
  • put(): The put adds item to a queue.
  • qsize() : The qsize() returns the number of items that are currently in the queue.
  • empty(): The empty( ) returns True if queue is empty; otherwise, False.
  • full(): the full() returns True if queue is full; otherwise, False.

Example:

#!/usr/bin/python

import Queue
import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q
    def run(self):
        print "Starting " + self.name
        process_data(self.name, self.q)
        print "Exiting " + self.name

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print "%s processing %s" % (threadName, data)
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = Queue.Queue(10)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# Fill the queue
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# Wait for queue to empty
while not workQueue.empty():
    pass

# Notify threads it's time to exit
exitFlag = 1

# Wait for all threads to complete
for t in threads:
    t.join()
print "Exiting Main Thread"

When the above code is executed, it produces the following result:

Starting Thread-1
Starting Thread-2
Starting Thread-3
Thread-1 processing One
Thread-2 processing Two
Thread-3 processing Three
Thread-1 processing Four
Thread-2 processing Five
Exiting Thread-3
Exiting Thread-1
Exiting Thread-2
Exiting Main Thread

Python XML Processing

What is XML?

The Extensible Markup Language (XML) is a markup language much like HTML or SGML. This is recommended by the World Wide Web Consortium and available as an open standard.

XML is a portable, open source language that allows programmers to develop applications that can be read by other applications, regardless of operating system and/or developmental language.

XML is extremely useful for keeping track of small to medium amounts of data without requiring a SQL-based backbone.

XML Parser Architectures and APIs:

The Python standard library provides a minimal but useful set of interfaces to work with XML.

The two most basic and broadly used APIs to XML data are the SAX and DOM interfaces.

  • Simple API for XML (SAX) : Here, you register callbacks for events of interest and then let the parser proceed through the document. This is useful when your documents are large or you have memory limitations, it parses the file as it reads it from disk and the entire file is never stored in memory.
  • Document Object Model (DOM) API : This is a World Wide Web Consortium recommendation wherein the entire file is read into memory and stored in a hierarchical (tree-based) form to represent all the features of an XML document.

SAX obviously can’t process information as fast as DOM can when working with large files. On the other hand, using DOM exclusively can really kill your resources, especially if used on a lot of small files.

SAX is read-only, while DOM allows changes to the XML file. Since these two different APIs literally complement each other, there is no reason why you can’t use them both for large projects.

For all our XML code examples, let’s use a simple XML file movies.xml as an input:

<collection shelf="New Arrivals">
<movie title="Enemy Behind">
   <type>War, Thriller</type>
   <format>DVD</format>
   <year>2003</year>
   <rating>PG</rating>
   <stars>10</stars>
   <description>Talk about a US-Japan war</description>
</movie>
<movie title="Transformers">
   <type>Anime, Science Fiction</type>
   <format>DVD</format>
   <year>1989</year>
   <rating>R</rating>
   <stars>8</stars>
   <description>A schientific fiction</description>
</movie>
   <movie title="Trigun">
   <type>Anime, Action</type>
   <format>DVD</format>
   <episodes>4</episodes>
   <rating>PG</rating>
   <stars>10</stars>
   <description>Vash the Stampede!</description>
</movie>
<movie title="Ishtar">
   <type>Comedy</type>
   <format>VHS</format>
   <rating>PG</rating>
   <stars>2</stars>
   <description>Viewable boredom</description>
</movie>
</collection>

Parsing XML with SAX APIs:

SAX is a standard interface for event-driven XML parsing. Parsing XML with SAX generally requires you to create your own ContentHandler by subclassing xml.sax.ContentHandler.

Your ContentHandler handles the particular tags and attributes of your flavor(s) of XML. A ContentHandler object provides methods to handle various parsing events. Its owning parser calls ContentHandler methods as it parses the XML file.

The methods startDocument and endDocument are called at the start and the end of the XML file. The method characters(text) is passed character data of the XML file via the parameter text.

The ContentHandler is called at the start and end of each element. If the parser is not in namespace mode, the methods startElement(tag, attributes) and endElement(tag) are called; otherwise, the corresponding methods startElementNS and endElementNS are called. Here, tag is the element tag, and attributes is an Attributes object.

Here are other important methods to understand before proceeding:

The make_parser Method:

Following method creates a new parser object and returns it. The parser object created will be of the first parser type the system finds.

xml.sax.make_parser( [parser_list] )

Here is the detail of the parameters:

  • parser_list: The optional argument consisting of a list of parsers to use which must all implement the make_parser method.

The parse Method:

Following method creates a SAX parser and uses it to parse a document.

xml.sax.parse( xmlfile, contenthandler[, errorhandler])

Here is the detail of the parameters:

  • xmlfile: This is the name of the XML file to read from.
  • contenthandler: This must be a ContentHandler object.
  • errorhandler: If specified, errorhandler must be a SAX ErrorHandler object.

The parseString Method:

There is one more method to create a SAX parser and to parse the specified XML string.

xml.sax.parseString(xmlstring, contenthandler[, errorhandler])

Here is the detail of the parameters:

  • xmlstring: This is the name of the XML string to read from.
  • contenthandler: This must be a ContentHandler object.
  • errorhandler: If specified, errorhandler must be a SAX ErrorHandler object.

Example:

#!/usr/bin/python

import xml.sax

class MovieHandler( xml.sax.ContentHandler ):
   def __init__(self):
      self.CurrentData = ""
      self.type = ""
      self.format = ""
      self.year = ""
      self.rating = ""
      self.stars = ""
      self.description = ""

   # Call when an element starts
   def startElement(self, tag, attributes):
      self.CurrentData = tag
      if tag == "movie":
         print "*****Movie*****"
         title = attributes["title"]
         print "Title:", title

   # Call when an elements ends
   def endElement(self, tag):
      if self.CurrentData == "type":
         print "Type:", self.type
      elif self.CurrentData == "format":
         print "Format:", self.format
      elif self.CurrentData == "year":
         print "Year:", self.year
      elif self.CurrentData == "rating":
         print "Rating:", self.rating
      elif self.CurrentData == "stars":
         print "Stars:", self.stars
      elif self.CurrentData == "description":
         print "Description:", self.description
      self.CurrentData = ""

   # Call when a character is read
   def characters(self, content):
      if self.CurrentData == "type":
         self.type = content
      elif self.CurrentData == "format":
         self.format = content
      elif self.CurrentData == "year":
         self.year = content
      elif self.CurrentData == "rating":
         self.rating = content
      elif self.CurrentData == "stars":
         self.stars = content
      elif self.CurrentData == "description":
         self.description = content
  
if ( __name__ == "__main__"):
   
   # create an XMLReader
   parser = xml.sax.make_parser()
   # turn off namepsaces
   parser.setFeature(xml.sax.handler.feature_namespaces, 0)

   # override the default ContextHandler
   Handler = MovieHandler()
   parser.setContentHandler( Handler )
   
   parser.parse("movies.xml")

This would produce following result:

*****Movie*****
Title: Enemy Behind
Type: War, Thriller
Format: DVD
Year: 2003
Rating: PG
Stars: 10
Description: Talk about a US-Japan war
*****Movie*****
Title: Transformers
Type: Anime, Science Fiction
Format: DVD
Year: 1989
Rating: R
Stars: 8
Description: A schientific fiction
*****Movie*****
Title: Trigun
Type: Anime, Action
Format: DVD
Rating: PG
Stars: 10
Description: Vash the Stampede!
*****Movie*****
Title: Ishtar
Type: Comedy
Format: VHS
Rating: PG
Stars: 2
Description: Viewable boredom

For a complete detail on SAX API documentation, please refer to standard Python SAX APIs.

Parsing XML with DOM APIs:

The Document Object Model, or “DOM,” is a cross-language API from the World Wide Web Consortium (W3C) for accessing and modifying XML documents.

The DOM is extremely useful for random-access applications. SAX only allows you a view of one bit of the document at a time. If you are looking at one SAX element, you have no access to another.

Here is the easiest way to quickly load an XML document and to create a minidom object using the xml.dom module. The minidom object provides a simple parser method that will quickly create a DOM tree from the XML file.

The sample phrase calls the parse( file [,parser] ) function of the minidom object to parse the XML file designated by file into a DOM tree object.

#!/usr/bin/python

from xml.dom.minidom import parse
import xml.dom.minidom

# Open XML document using minidom parser
DOMTree = xml.dom.minidom.parse("movies.xml")
collection = DOMTree.documentElement
if collection.hasAttribute("shelf"):
   print "Root element : %s" % collection.getAttribute("shelf")

# Get all the movies in the collection
movies = collection.getElementsByTagName("movie")

# Print detail of each movie.
for movie in movies:
   print "*****Movie*****"
   if movie.hasAttribute("title"):
      print "Title: %s" % movie.getAttribute("title")

   type = movie.getElementsByTagName('type')[0]
   print "Type: %s" % type.childNodes[0].data
   format = movie.getElementsByTagName('format')[0]
   print "Format: %s" % format.childNodes[0].data
   rating = movie.getElementsByTagName('rating')[0]
   print "Rating: %s" % rating.childNodes[0].data
   description = movie.getElementsByTagName('description')[0]
   print "Description: %s" % description.childNodes[0].data

This would produce the following result:

Root element : New Arrivals
*****Movie*****
Title: Enemy Behind
Type: War, Thriller
Format: DVD
Rating: PG
Description: Talk about a US-Japan war
*****Movie*****
Title: Transformers
Type: Anime, Science Fiction
Format: DVD
Rating: R
Description: A schientific fiction
*****Movie*****
Title: Trigun
Type: Anime, Action
Format: DVD
Rating: PG
Description: Vash the Stampede!
*****Movie*****
Title: Ishtar
Type: Comedy
Format: VHS
Rating: PG
Description: Viewable boredom

Python GUI Programming (Tkinter)

Python provides various options for developing graphical user interfaces (GUIs). Most important are listed below:

  • Tkinter: Tkinter is the Python interface to the Tk GUI toolkit shipped with Python. We would look this option in this tutorial.
  • wxPython: This is an open-source Python interface for wxWindows http://wxpython.org.
  • JPython: JPython is a Python port for Java which gives Python scripts seamless access to Java class libraries on the local machine http://www.jython.org.

There are many other interfaces available which I’m not listing here. You can find them over the net.

Tkinter Programming

Tkinter is the standard GUI library for Python. Python when combined with Tkinter provides a fast and easy way to create GUI applications. Tkinter provides a powerful object-oriented interface to the Tk GUI toolkit.

Creating a GUI application using Tkinter is an easy task. All you need to do is perform the following steps:

  • Import the Tkinter module.
  • Create the GUI application main window.
  • Add one or more of the above-mentioned widgets to the GUI application.
  • Enter the main event loop to take action against each event triggered by the user.

Example:

#!/usr/bin/python

import Tkinter
top = Tkinter.Tk()
# Code to add widgets will go here...
top.mainloop()

This would create a following window:

TK Window

Tkinter Widgets

Tkinter provides various controls, such as buttons, labels and text boxes used in a GUI application. These controls are commonly called widgets.

There are currently 15 types of widgets in Tkinter. We present these widgets as well as a brief description in the following table:

Operator Description
Button The Button widget is used to display buttons in your application.
Canvas The Canvas widget is used to draw shapes, such as lines, ovals, polygons and rectangles, in your application.
Checkbutton The Checkbutton widget is used to display a number of options as checkboxes. The user can select multiple options at a time.
Entry The Entry widget is used to display a single-line text field for accepting values from a user.
Frame The Frame widget is used as a container widget to organize other widgets.
Label The Label widget is used to provide a single-line caption for other widgets. It can also contain images.
Listbox The Listbox widget is used to provide a list of options to a user.
Menubutton The Menubutton widget is used to display menus in your application.
Menu The Menu widget is used to provide various commands to a user. These commands are contained inside Menubutton.
Message The Message widget is used to display multiline text fields for accepting values from a user.
Radiobutton The Radiobutton widget is used to display a number of options as radio buttons. The user can select only one option at a time.
Scale The Scale widget is used to provide a slider widget.
Scrollbar The Scrollbar widget is used to add scrolling capability to various widgets, such as list boxes.
Text The Text widget is used to display text in multiple lines.
Toplevel The Toplevel widget is used to provide a separate window container.
Spinbox The Spinbox widget is a variant of the standard Tkinter Entry widget, which can be used to select from a fixed number of values.
PanedWindow A PanedWindow is a container widget that may contain any number of panes, arranged horizontally or vertically.
LabelFrame A labelframe is a simple container widget. Its primary purpose is to act as a spacer or container for complex window layouts.
tkMessageBox This module is used to display message boxes in your applications.

Standard attributes:

Let’s take a look at how some of their common attributes.such as sizes, colors and fonts are specified.

  • Dimensions
  • Colors
  • Fonts
  • Anchors
  • Relief styles
  • Bitmaps
  • Cursors

Geometry Management:

All Tkinter widgets have access to specific geometry management methods, which have the purpose of organizing widgets throughout the parent widget area. Tkinter exposes the following geometry manager classes: pack, grid, and place.

  • The pack() Method – This geometry manager organizes widgets in blocks before placing them in the parent widget.
  • The grid() Method – This geometry manager organizes widgets in a table-like structure in the parent widget.
  • The place() Method -This geometry manager organizes widgets by placing them in a specific position in the parent widget.

Python Extension Programming with C

Any code that you write using any compiled language like C, C++ or Java can be integrated or imported into another Python script. This code is considered as an “extension.”

A Python extension module is nothing more than a normal C library. On Unix machines, these libraries usually end in .so (for shared object). On Windows machines, you typically see .dll (for dynamically linked library).

Pre-Requisite:

To start writing your extension, you are going to need the Python header files.

  • On Unix machines, this usually requires installing a developer-specific package such as python2.5-dev.
  • Windows users get these headers as part of the package when they use the binary Python installer.

Additionally, it is assumed that you have good knowledge of C or C++ to write any Python Extension using C programming.

First look at a Python extension:

For your first look at a Python extension module, you’ll be grouping your code into four parts:

  • The header file Python.h.
  • The C functions you want to expose as the interface from your module.
  • A table mapping the names of your functions as Python developers will see them to C functions inside the extension module.
  • An initialization function.

The header file Python.h

Start including Python.h header file in your C source file, which will give you access to the internal Python API used to hook your module into the interpreter.

Be sure to include Python.h before any other headers you might need. You’ll follow the includes with the functions you want to call from Python.

The C functions:

The signatures of the C implementations of your functions will always take one of the following three forms:

static PyObject *MyFunction( PyObject *self, PyObject *args );

static PyObject *MyFunctionWithKeywords(PyObject *self,
                                 PyObject *args,
                                 PyObject *kw);

static PyObject *MyFunctionWithNoArgs( PyObject *self );

Each one of the preceding declarations returns a Python object. There’s no such thing as a void function in Python as there is in C. If you don’t want your functions to return a value, return the C equivalent of Python’s None value. The Python headers define a macro, Py_RETURN_NONE, that does this for us.

The names of your C functions can be whatever you like as they will never be seen outside of the extension module. So they would be defined as static function.

Your C functions usually are named by combining the Python module and function names together, as shown here:

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Do your stuff here. */
   Py_RETURN_NONE;
}

This would be a Python function called func inside of the module module. You’ll be putting pointers to your C functions into the method table for the module that usually comes next in your source code.

The method mapping table:

This method table is a simple array of PyMethodDef structures. That structure looks something like this:

struct PyMethodDef {
   char *ml_name;
   PyCFunction ml_meth;
   int ml_flags;
   char *ml_doc;
};

Here is the description of the members of this structure:

  • ml_name: This is the name of the function as the Python interpreter will present it when it is used in Python programs.
  • ml_meth: This must be the address to a function that has any one of the signatures described in previous seection.
  • ml_flags: This tells the interpreter which of the three signatures ml_meth is using.
    • This flag will usually have a value of METH_VARARGS.
    • This flag can be bitwise or’ed with METH_KEYWORDS if you want to allow keyword arguments into your function.
    • This can also have a value of METH_NOARGS that indicates you don’t want to accept any arguments.
  • ml_doc: This is the docstring for the function, which could be NULL if you don’t feel like writing one

This table needs to be terminated with a sentinel that consists of NULL and 0 values for the appropriate members.

Example:

For the above-defined function, we would have following method mapping table:

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { NULL, NULL, 0, NULL }
};

The initialization function:

The last part of your extension module is the initialization function. This function is called by the Python interpreter when the module is loaded. It’s required that the function be named initModule, where Module is the name of the module.

The initialization function needs to be exported from the library you’ll be building. The Python headers define PyMODINIT_FUNC to include the appropriate incantations for that to happen for the particular environment in which we’re compiling. All you have to do is use it when defining the function.

Your C initialization function generally has the following overall structure:

PyMODINIT_FUNC initModule() {
   Py_InitModule3(func, module_methods, "docstring...");
}

Here is the description of Py_InitModule3 function:

  • func: This is the function to be exported.
  • module_methods: This is the mapping table name defined above.
  • docstring: This is the comment you want to give in your extension.

Putting this all together looks like the following:

#include <Python.h>

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Do your stuff here. */
   Py_RETURN_NONE;
}

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { NULL, NULL, 0, NULL }
};

PyMODINIT_FUNC initModule() {
   Py_InitModule3(func, module_methods, "docstring...");
}

Example:

A simple example that makes use of all the above concepts:

#include <Python.h>

static PyObject* helloworld(PyObject* self)
{
    return Py_BuildValue("s", "Hello, Python extensions!!");
}

static char helloworld_docs[] =
    "helloworld( ): Any message you want to put here!!\n";

static PyMethodDef helloworld_funcs[] = {
    {"helloworld", (PyCFunction)helloworld, 
     METH_NOARGS, helloworld_docs},
    {NULL}
};

void inithelloworld(void)
{
    Py_InitModule3("helloworld", helloworld_funcs,
                   "Extension module example!");
}

Here the Py_BuildValue function is used to build a Python value. Save above code in hello.c file. We would see how to compile and install this module to be called from Python script.

Building and Installing Extensions:

The distutils package makes it very easy to distribute Python modules, both pure Python and extension modules, in a standard way. Modules are distributed in source form and built and installed via a setup script usually called setup.py as follows.

For the above module, you would have to prepare following setup.py script:

from distutils.core import setup, Extension
setup(name='helloworld', version='1.0',  \
      ext_modules=[Extension('helloworld', ['hello.c'])])

Now, use the following command, which would perform all needed compilation and linking steps, with the right compiler and linker commands and flags, and copies the resulting dynamic library into an appropriate directory:

$ python setup.py install

On Unix-based systems, you’ll most likely need to run this command as root in order to have permissions to write to the site-packages directory. This usually isn’t a problem on Windows

Import Extensions:

Once you installed your extension, you would be able to import and call that extension in your Python script as follows:

#!/usr/bin/python
import helloworld

print helloworld.helloworld()

This would produce the following result:

Hello, Python extensions!!

Passing Function Parameters:

Because you’ll most likely want to define functions that do accept arguments, you can use one of the other signatures for your C functions. For example, following function, that accepts some number of parameters, would be defined like this:

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Parse args and do something interesting here. */
   Py_RETURN_NONE;
}

The method table containing an entry for the new function would look like this:

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { "func", module_func, METH_VARARGS, NULL },
   { NULL, NULL, 0, NULL }
};

You can use API PyArg_ParseTuple function to extract the arguments from the one PyObject pointer passed into your C function.

The first argument to PyArg_ParseTuple is the args argument. This is the object you’ll be parsing. The second argument is a format string describing the arguments as you expect them to appear. Each argument is represented by one or more characters in the format string as follows.

static PyObject *module_func(PyObject *self, PyObject *args) {
   int i;
   double d;
   char *s;

   if (!PyArg_ParseTuple(args, "ids", &i, &d, &s)) {
      return NULL;
   }
   
   /* Do something interesting here. */
   Py_RETURN_NONE;
}

Compiling the new version of your module and importing it will enable you to invoke the new function with any number of arguments of any type:

module.func(1, s="three", d=2.0)
module.func(i=1, d=2.0, s="three")
module.func(s="three", d=2.0, i=1)

You can probably come up with even more variations.

The PyArg_ParseTuple Function:

Here is the standard signature for PyArg_ParseTuple function:

int PyArg_ParseTuple(PyObject* tuple,char* format,...)

This function returns 0 for errors, and a value not equal to 0 for success. tuple is the PyObject* that was the C function’s second argument. Here format is a C string that describes mandatory and optional arguments.

Here is a list of format codes for PyArg_ParseTuple function:

Code C type Meaning
c char A Python string of length 1 becomes a C char.
d double A Python float becomes a C double.
f float A Python float becomes a C float.
i int A Python int becomes a C int.
l long A Python int becomes a C long.
L long long A Python int becomes a C long long
O PyObject* Gets non-NULL borrowed reference to Python argument.
s char* Python string without embedded nulls to C char*.
s# char*+int Any Python string to C address and length.
t# char*+int Read-only single-segment buffer to C address and length.
u Py_UNICODE* Python Unicode without embedded nulls to C.
u# Py_UNICODE*+int Any Python Unicode C address and length.
w# char*+int Read/write single-segment buffer to C address and length.
z char* Like s, also accepts None (sets C char* to NULL).
z# char*+int Like s#, also accepts None (sets C char* to NULL).
(…) as per … A Python sequence is treated as one argument per item.
| The following arguments are optional.
: Format end, followed by function name for error messages.
; Format end, followed by entire error message text.

Returning Values:

Py_BuildValue takes in a format string much like PyArg_ParseTuple does. Instead of passing in the addresses of the values you’re building, you pass in the actual values. Here’s an example showing how to implement an add function:

static PyObject *foo_add(PyObject *self, PyObject *args) {
   int a;
   int b;

   if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
      return NULL;
   }
   return Py_BuildValue("i", a + b);
}

This is what it would look like if implemented in Python:

def add(a, b):
   return (a + b)

You can return two values from your function as follows, this would be cauptured using a list in Python.

static PyObject *foo_add_subtract(PyObject *self, PyObject *args) {
   int a;
   int b;

   if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
      return NULL;
   }
   return Py_BuildValue("ii", a + b, a - b);
}

This is what it would look like if implemented in Python:

def add_subtract(a, b):
   return (a + b, a - b)

The Py_BuildValue Function:

Here is the standard signature for Py_BuildValue function:

PyObject* Py_BuildValue(char* format,...)

Here format is a C string that describes the Python object to build. The following arguments of Py_BuildValue are C values from which the result is built. The PyObject* result is a new reference.

Following table lists the commonly used code strings, of which zero or more are joined into string format.

Code C type Meaning
c char A C char becomes a Python string of length 1.
d double A C double becomes a Python float.
f float A C float becomes a Python float.
i int A C int becomes a Python int.
l long A C long becomes a Python int.
N PyObject* Passes a Python object and steals a reference.
O PyObject* Passes a Python object and INCREFs it as normal.
O& convert+void* Arbitrary conversion
s char* C 0-terminated char* to Python string, or NULL to None.
s# char*+int C char* and length to Python string, or NULL to None.
u Py_UNICODE* C-wide, null-terminated string to Python Unicode, or NULL to None.
u# Py_UNICODE*+int C-wide string and length to Python Unicode, or NULL to None.
w# char*+int Read/write single-segment buffer to C address and length.
z char* Like s, also accepts None (sets C char* to NULL).
z# char*+int Like s#, also accepts None (sets C char* to NULL).
(…) as per … Builds Python tuple from C values.
[…] as per … Builds Python list from C values.
{…} as per … Builds Python dictionary from C values, alternating keys and values.

Code {…} builds dictionaries from an even number of C values, alternately keys and values. For example, Py_BuildValue(“{issi}”,23,”zig”,”zag”,42) returns a dictionary like Python’s {23:’zig’,’zag’:42}.

ELECTRONICS 101-PROGRAMMING WITH PYTHON-3 (ADVANCED)

Python Object Oriented

Python has been an object-oriented language from day one. Because of this, creating and using classes and objects are downright easy. This chapter helps you become an expert in using Python’s object-oriented programming support.

If you don’t have any previous experience with object-oriented (OO) programming, you may want to consult an introductory course on it or at least a tutorial of some sort so that you have a grasp of the basic concepts.

However, here is small introduction of Object-Oriented Programming (OOP) to bring you at speed:

Overview of OOP Terminology

  • Class: A user-defined prototype for an object that defines a set of attributes that characterize any object of the class. The attributes are data members (class variables and instance variables) and methods, accessed via dot notation.
  • Class variable: A variable that is shared by all instances of a class. Class variables are defined within a class but outside any of the class’s methods. Class variables aren’t used as frequently as instance variables are.
  • Data member: A class variable or instance variable that holds data associated with a class and its objects.
  • Function overloading: The assignment of more than one behavior to a particular function. The operation performed varies by the types of objects (arguments) involved.
  • Instance variable: A variable that is defined inside a method and belongs only to the current instance of a class.
  • Inheritance : The transfer of the characteristics of a class to other classes that are derived from it.
  • Instance: An individual object of a certain class. An object obj that belongs to a class Circle, for example, is an instance of the class Circle.
  • Instantiation : The creation of an instance of a class.
  • Method : A special kind of function that is defined in a class definition.
  • Object : A unique instance of a data structure that’s defined by its class. An object comprises both data members (class variables and instance variables) and methods.
  • Operator overloading: The assignment of more than one function to a particular operator.

Creating Classes:

The class statement creates a new class definition. The name of the class immediately follows the keyword class followed by a colon as follows:

class ClassName:
   'Optional class documentation string'
   class_suite
  • The class has a documentation string, which can be accessed via ClassName.__doc__.
  • The class_suite consists of all the component statements defining class members, data attributes and functions.

Example:

Following is the example of a simple Python class:

class Employee:
   'Common base class for all employees'
   empCount = 0

   def __init__(self, name, salary):
      self.name = name
      self.salary = salary
      Employee.empCount += 1
   
   def displayCount(self):
     print "Total Employee %d" % Employee.empCount

   def displayEmployee(self):
      print "Name : ", self.name,  ", Salary: ", self.salary
  • The variable empCount is a class variable whose value would be shared among all instances of a this class. This can be accessed as Employee.empCount from inside the class or outside the class.
  • The first method __init__() is a special method, which is called class constructor or initialization method that Python calls when you create a new instance of this class.
  • You declare other class methods like normal functions with the exception that the first argument to each method is self. Python adds the self argument to the list for you; you don’t need to include it when you call the methods.

Creating instance objects:

To create instances of a class, you call the class using class name and pass in whatever arguments its __init__ method accepts.

"This would create first object of Employee class"
emp1 = Employee("Zara", 2000)
"This would create second object of Employee class"
emp2 = Employee("Manni", 5000)

Accessing attributes:

You access the object’s attributes using the dot operator with object. Class variable would be accessed using class name as follows:

emp1.displayEmployee()
emp2.displayEmployee()
print "Total Employee %d" % Employee.empCount

Now, putting all the concepts together:

#!/usr/bin/python

class Employee:
   'Common base class for all employees'
   empCount = 0

   def __init__(self, name, salary):
      self.name = name
      self.salary = salary
      Employee.empCount += 1
   
   def displayCount(self):
     print "Total Employee %d" % Employee.empCount

   def displayEmployee(self):
      print "Name : ", self.name,  ", Salary: ", self.salary

"This would create first object of Employee class"
emp1 = Employee("Zara", 2000)
"This would create second object of Employee class"
emp2 = Employee("Manni", 5000)
emp1.displayEmployee()
emp2.displayEmployee()
print "Total Employee %d" % Employee.empCount

When the above code is executed, it produces the following result:

Name :  Zara ,Salary:  2000
Name :  Manni ,Salary:  5000
Total Employee 2

You can add, remove or modify attributes of classes and objects at any time:

emp1.age = 7  # Add an 'age' attribute.
emp1.age = 8  # Modify 'age' attribute.
del emp1.age  # Delete 'age' attribute.

Instead of using the normal statements to access attributes, you can use following functions:

  • The getattr(obj, name[, default]) : to access the attribute of object.
  • The hasattr(obj,name) : to check if an attribute exists or not.
  • The setattr(obj,name,value) : to set an attribute. If attribute does not exist, then it would be created.
  • The delattr(obj, name) : to delete an attribute.
hasattr(emp1, 'age')    # Returns true if 'age' attribute exists
getattr(emp1, 'age')    # Returns value of 'age' attribute
setattr(emp1, 'age', 8) # Set attribute 'age' at 8
delattr(empl, 'age')    # Delete attribute 'age'

Built-In Class Attributes:

Every Python class keeps following built-in attributes and they can be accessed using dot operator like any other attribute:

  • __dict__ : Dictionary containing the class’s namespace.
  • __doc__ : Class documentation string or None if undefined.
  • __name__: Class name.
  • __module__: Module name in which the class is defined. This attribute is “__main__” in interactive mode.
  • __bases__ : A possibly empty tuple containing the base classes, in the order of their occurrence in the base class list.

For the above class let’s try to access all these attributes:

#!/usr/bin/python

class Employee:
   'Common base class for all employees'
   empCount = 0

   def __init__(self, name, salary):
      self.name = name
      self.salary = salary
      Employee.empCount += 1
   
   def displayCount(self):
     print "Total Employee %d" % Employee.empCount

   def displayEmployee(self):
      print "Name : ", self.name,  ", Salary: ", self.salary

print "Employee.__doc__:", Employee.__doc__
print "Employee.__name__:", Employee.__name__
print "Employee.__module__:", Employee.__module__
print "Employee.__bases__:", Employee.__bases__
print "Employee.__dict__:", Employee.__dict__

When the above code is executed, it produces the following result:

Employee.__doc__: Common base class for all employees
Employee.__name__: Employee
Employee.__module__: __main__
Employee.__bases__: ()
Employee.__dict__: {'__module__': '__main__', 'displayCount':
<function displayCount at 0xb7c84994>, 'empCount': 2, 
'displayEmployee': <function displayEmployee at 0xb7c8441c>, 
'__doc__': 'Common base class for all employees', 
'__init__': <function __init__ at 0xb7c846bc>}

Destroying Objects (Garbage Collection):

Python deletes unneeded objects (built-in types or class instances) automatically to free memory space. The process by which Python periodically reclaims blocks of memory that no longer are in use is termed garbage collection.

Python’s garbage collector runs during program execution and is triggered when an object’s reference count reaches zero. An object’s reference count changes as the number of aliases that point to it changes.

An object’s reference count increases when it’s assigned a new name or placed in a container (list, tuple or dictionary). The object’s reference count decreases when it’s deleted with del, its reference is reassigned, or its reference goes out of scope. When an object’s reference count reaches zero, Python collects it automatically.

a = 40      # Create object <40>
b = a       # Increase ref. count  of <40> 
c = [b]     # Increase ref. count  of <40> 

del a       # Decrease ref. count  of <40>
b = 100     # Decrease ref. count  of <40> 
c[0] = -1   # Decrease ref. count  of <40> 

You normally won’t notice when the garbage collector destroys an orphaned instance and reclaims its space. But a class can implement the special method __del__(), called a destructor, that is invoked when the instance is about to be destroyed. This method might be used to clean up any nonmemory resources used by an instance.

Example:

This __del__() destructor prints the class name of an instance that is about to be destroyed:

#!/usr/bin/python

class Point:
   def __init( self, x=0, y=0):
      self.x = x
      self.y = y
   def __del__(self):
      class_name = self.__class__.__name__
      print class_name, "destroyed"

pt1 = Point()
pt2 = pt1
pt3 = pt1
print id(pt1), id(pt2), id(pt3) # prints the ids of the obejcts
del pt1
del pt2
del pt3

When the above code is executed, it produces following result:

3083401324 3083401324 3083401324
Point destroyed

Note: Ideally, you should define your classes in separate file, then you should import them in your main program file using import statement. Kindly check Python – Modules chapter for more details on importing modules and classes.

Class Inheritance:

Instead of starting from scratch, you can create a class by deriving it from a preexisting class by listing the parent class in parentheses after the new class name.

The child class inherits the attributes of its parent class, and you can use those attributes as if they were defined in the child class. A child class can also override data members and methods from the parent.

Syntax:

Derived classes are declared much like their parent class; however, a list of base classes to inherit from are given after the class name:

class SubClassName (ParentClass1[, ParentClass2, ...]):
   'Optional class documentation string'
   class_suite

Example:

#!/usr/bin/python

class Parent:        # define parent class
   parentAttr = 100
   def __init__(self):
      print "Calling parent constructor"

   def parentMethod(self):
      print 'Calling parent method'

   def setAttr(self, attr):
      Parent.parentAttr = attr

   def getAttr(self):
      print "Parent attribute :", Parent.parentAttr

class Child(Parent): # define child class
   def __init__(self):
      print "Calling child constructor"

   def childMethod(self):
      print 'Calling child method'

c = Child()          # instance of child
c.childMethod()      # child calls its method
c.parentMethod()     # calls parent's method
c.setAttr(200)       # again call parent's method
c.getAttr()          # again call parent's method

When the above code is executed, it produces the following result:

Calling child constructor
Calling child method
Calling parent method
Parent attribute : 200

Similar way, you can drive a class from multiple parent classes as follows:

class A:        # define your class A
.....

class B:         # define your calss B
.....

class C(A, B):   # subclass of A and B
.....

You can use issubclass() or isinstance() functions to check a relationships of two classes and instances.

  • The issubclass(sub, sup) boolean function returns true if the given subclass sub is indeed a subclass of the superclass sup.
  • The isinstance(obj, Class) boolean function returns true if obj is an instance of class Class or is an instance of a subclass of Class

Overriding Methods:

You can always override your parent class methods. One reason for overriding parent’s methods is because you may want special or different functionality in your subclass.

Example:

#!/usr/bin/python

class Parent:        # define parent class
   def myMethod(self):
      print 'Calling parent method'

class Child(Parent): # define child class
   def myMethod(self):
      print 'Calling child method'

c = Child()          # instance of child
c.myMethod()         # child calls overridden method

When the above code is executed, it produces the following result:

Calling child method

Base Overloading Methods:

Following table lists some generic functionality that you can override in your own classes:

SN Method, Description & Sample Call
1 __init__ ( self [,args…] )
Constructor (with any optional arguments)
Sample Call : obj = className(args)
2 __del__( self )
Destructor, deletes an object
Sample Call : del obj
3 __repr__( self )
Evaluatable string representation
Sample Call : repr(obj)
4 __str__( self )
Printable string representation
Sample Call : str(obj)
5 __cmp__ ( self, x )
Object comparison
Sample Call : cmp(obj, x)

Overloading Operators:

Suppose you’ve created a Vector class to represent two-dimensional vectors, what happens when you use the plus operator to add them? Most likely Python will yell at you.

You could, however, define the __add__ method in your class to perform vector addition and then the plus operator would behave as per expectation:

Example:

#!/usr/bin/python

class Vector:
   def __init__(self, a, b):
      self.a = a
      self.b = b

   def __str__(self):
      return 'Vector (%d, %d)' % (self.a, self.b)
   
   def __add__(self,other):
      return Vector(self.a + other.a, self.b + other.b)

v1 = Vector(2,10)
v2 = Vector(5,-2)
print v1 + v2

When the above code is executed, it produces the following result:

Vector(7,8)

Data Hiding:

An object’s attributes may or may not be visible outside the class definition. For these cases, you can name attributes with a double underscore prefix, and those attributes will not be directly visible to outsiders.

Example:

#!/usr/bin/python

class JustCounter:
   __secretCount = 0
  
   def count(self):
      self.__secretCount += 1
      print self.__secretCount

counter = JustCounter()
counter.count()
counter.count()
print counter.__secretCount

When the above code is executed, it produces the following result:

1
2
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    print counter.__secretCount
AttributeError: JustCounter instance has no attribute '__secretCount'

Python protects those members by internally changing the name to include the class name. You can access such attributes as object._className__attrName. If you would replace your last line as following, then it would work for you:

.........................
print counter._JustCounter__secretCount

When the above code is executed, it produces the following result:

1
2
2

Python Regular Expressions

A regular expression is a special sequence of characters that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world.

The module re provides full support for Perl-like regular expressions in Python. The re module raises the exception re.error if an error occurs while compiling or using a regular expression.

We would cover two important functions, which would be used to handle regular expressions. But a small thing first: There are various characters, which would have special meaning when they are used in regular expression. To avoid any confusion while dealing with regular expressions, we would use Raw Strings as r’expression’.

The match Function

This function attempts to match RE pattern to string with optional flags.

Here is the syntax for this function:

re.match(pattern, string, flags=0)

Here is the description of the parameters:

Parameter Description
pattern This is the regular expression to be matched.
string This is the string, which would be searched to match the pattern at the beginning of string.
flags You can specify different flags using bitwise OR (|). These are modifiers, which are listed in the table below.

The re.match function returns a match object on success, None on failure. We would use group(num) or groups() function of match object to get matched expression.

Match Object Methods Description
group(num=0) This method returns entire match (or specific subgroup num)
groups() This method returns all matching subgroups in a tuple (empty if there weren’t any)

Example:

#!/usr/bin/python
import re

line = "Cats are smarter than dogs"

matchObj = re.match( r'(.*) are (.*?) .*', line, re.M|re.I)

if matchObj:
   print "matchObj.group() : ", matchObj.group()
   print "matchObj.group(1) : ", matchObj.group(1)
   print "matchObj.group(2) : ", matchObj.group(2)
else:
   print "No match!!"

When the above code is executed, it produces following result:

matchObj.group() :  Cats are smarter than dogs
matchObj.group(1) :  Cats
matchObj.group(2) :  smarter

The search Function

This function searches for first occurrence of RE pattern within string with optional flags.

Here is the syntax for this function:

re.search(pattern, string, flags=0)

Here is the description of the parameters:

Parameter Description
pattern This is the regular expression to be matched.
string This is the string, which would be searched to match the pattern anywhere in the string.
flags You can specify different flags using bitwise OR (|). These are modifiers, which are listed in the table below.

The re.search function returns a match object on success, None on failure. We would use group(num) or groups() function of match object to get matched expression.

Match Object Methods Description
group(num=0) This method returns entire match (or specific subgroup num)
groups() This method returns all matching subgroups in a tuple (empty if there weren’t any)

Example:

#!/usr/bin/python
import re

line = "Cats are smarter than dogs";

searchObj = re.search( r'(.*) are (.*?) .*', line, re.M|re.I)

if searchObj:
   print "searchObj.group() : ", searchObj.group()
   print "searchObj.group(1) : ", searchObj.group(1)
   print "searchObj.group(2) : ", searchObj.group(2)
else:
   print "Nothing found!!"

When the above code is executed, it produces following result:

matchObj.group() :  Cats are smarter than dogs
matchObj.group(1) :  Cats
matchObj.group(2) :  smarter

Matching vs Searching:

Python offers two different primitive operations based on regular expressions: match checks for a match only at the beginning of the string, while search checks for a match anywhere in the string (this is what Perl does by default).

Example:

#!/usr/bin/python
import re

line = "Cats are smarter than dogs";

matchObj = re.match( r'dogs', line, re.M|re.I)
if matchObj:
   print "match --> matchObj.group() : ", matchObj.group()
else:
   print "No match!!"

searchObj = re.search( r'dogs', line, re.M|re.I)
if searchObj:
   print "search --> searchObj.group() : ", searchObj.group()
else:
   print "Nothing found!!"

When the above code is executed, it produces the following result:

No match!!
search --> matchObj.group() :  dogs

Search and Replace:

Some of the most important re methods that use regular expressions is sub.

Syntax:

re.sub(pattern, repl, string, max=0)

This method replaces all occurrences of the RE pattern in string with repl, substituting all occurrences unless max provided. This method would return modified string.

Example:

Following is the example:

#!/usr/bin/python
import re

phone = "2004-959-559 # This is Phone Number"

# Delete Python-style comments
num = re.sub(r'#.*$', "", phone)
print "Phone Num : ", num

# Remove anything other than digits
num = re.sub(r'\D', "", phone)    
print "Phone Num : ", num

When the above code is executed, it produces the following result:

Phone Num :  2004-959-559
Phone Num :  2004959559

Regular-expression Modifiers – Option Flags

Regular expression literals may include an optional modifier to control various aspects of matching. The modifiers are specified as an optional flag. You can provide multiple modifiers using exclusive OR (|), as shown previously and may be represented by one of these:

Modifier Description
re.I Performs case-insensitive matching.
re.L Interprets words according to the current locale. This interpretation affects the alphabetic group (\w and \W), as well as word boundary behavior (\b and \B).
re.M Makes $ match the end of a line (not just the end of the string) and makes ^ match the start of any line (not just the start of the string).
re.S Makes a period (dot) match any character, including a newline.
re.U Interprets letters according to the Unicode character set. This flag affects the behavior of \w, \W, \b, \B.
re.X Permits “cuter” regular expression syntax. It ignores whitespace (except inside a set [] or when escaped by a backslash) and treats unescaped # as a comment marker.

Regular-expression patterns:

Except for control characters, (+ ? . * ^ $ ( ) [ ] { } | \), all characters match themselves. You can escape a control character by preceding it with a backslash.

Following table lists the regular expression syntax that is available in Python:

Pattern Description
^ Matches beginning of line.
$ Matches end of line.
. Matches any single character except newline. Using m option allows it to match newline as well.
[…] Matches any single character in brackets.
[^…] Matches any single character not in brackets
re* Matches 0 or more occurrences of preceding expression.
re+ Matches 1 or more occurrence of preceding expression.
re? Matches 0 or 1 occurrence of preceding expression.
re{ n} Matches exactly n number of occurrences of preceding expression.
re{ n,} Matches n or more occurrences of preceding expression.
re{ n, m} Matches at least n and at most m occurrences of preceding expression.
a| b Matches either a or b.
(re) Groups regular expressions and remembers matched text.
(?imx) Temporarily toggles on i, m, or x options within a regular expression. If in parentheses, only that area is affected.
(?-imx) Temporarily toggles off i, m, or x options within a regular expression. If in parentheses, only that area is affected.
(?: re) Groups regular expressions without remembering matched text.
(?imx: re) Temporarily toggles on i, m, or x options within parentheses.
(?-imx: re) Temporarily toggles off i, m, or x options within parentheses.
(?#…) Comment.
(?= re) Specifies position using a pattern. Doesn’t have a range.
(?! re) Specifies position using pattern negation. Doesn’t have a range.
(?> re) Matches independent pattern without backtracking.
\w Matches word characters.
\W Matches nonword characters.
\s Matches whitespace. Equivalent to [\t\n\r\f].
\S Matches nonwhitespace.
\d Matches digits. Equivalent to [0-9].
\D Matches nondigits.
\A Matches beginning of string.
\Z Matches end of string. If a newline exists, it matches just before newline.
\z Matches end of string.
\G Matches point where last match finished.
\b Matches word boundaries when outside brackets. Matches backspace (0x08) when inside brackets.
\B Matches nonword boundaries.
\n, \t, etc. Matches newlines, carriage returns, tabs, etc.
\1…\9 Matches nth grouped subexpression.
\10 Matches nth grouped subexpression if it matched already. Otherwise refers to the octal representation of a character code.

Regular-expression Examples

Literal characters:

Example Description
python Match “python”.

Character classes:

Example Description
[Pp]ython Match “Python” or “python”
rub[ye] Match “ruby” or “rube”
[aeiou] Match any one lowercase vowel
[0-9] Match any digit; same as [0123456789]
[a-z] Match any lowercase ASCII letter
[A-Z] Match any uppercase ASCII letter
[a-zA-Z0-9] Match any of the above
[^aeiou] Match anything other than a lowercase vowel
[^0-9] Match anything other than a digit

Special Character Classes:

Example Description
. Match any character except newline
\d Match a digit: [0-9]
\D Match a nondigit: [^0-9]
\s Match a whitespace character: [ \t\r\n\f]
\S Match nonwhitespace: [^ \t\r\n\f]
\w Match a single word character: [A-Za-z0-9_]
\W Match a nonword character: [^A-Za-z0-9_]

Repetition Cases:

Example Description
ruby? Match “rub” or “ruby”: the y is optional
ruby* Match “rub” plus 0 or more ys
ruby+ Match “rub” plus 1 or more ys
\d{3} Match exactly 3 digits
\d{3,} Match 3 or more digits
\d{3,5} Match 3, 4, or 5 digits

Nongreedy repetition:

This matches the smallest number of repetitions:

Example Description
<.*> Greedy repetition: matches “<python>perl>”
<.*?> Nongreedy: matches “<python>” in “<python>perl>”

Grouping with parentheses:

Example Description
\D\d+ No group: + repeats \d
(\D\d)+ Grouped: + repeats \D\d pair
([Pp]ython(, )?)+ Match “Python”, “Python, python, python”, etc.

Backreferences:

This matches a previously matched group again:

Example Description
([Pp])ython&\1ails Match python&pails or Python&Pails
([‘”])[^\1]*\1 Single or double-quoted string. \1 matches whatever the 1st group matched . \2 matches whatever the 2nd group matched, etc.

Alternatives:

Example Description
python|perl Match “python” or “perl”
rub(y|le)) Match “ruby” or “ruble”
Python(!+|\?) “Python” followed by one or more ! or one ?

Anchors:

This needs to specify match position.

Example Description
^Python Match “Python” at the start of a string or internal line
Python$ Match “Python” at the end of a string or line
\APython Match “Python” at the start of a string
Python\Z Match “Python” at the end of a string
\bPython\b Match “Python” at a word boundary
\brub\B \B is nonword boundary: match “rub” in “rube” and “ruby” but not alone
Python(?=!) Match “Python”, if followed by an exclamation point
Python(?!!) Match “Python”, if not followed by an exclamation point

Special syntax with parentheses:

Example Description
R(?#comment) Matches “R”. All the rest is a comment
R(?i)uby Case-insensitive while matching “uby”
R(?i:uby) Same as above
rub(?:y|le)) Group only without creating \1 backreference

Python CGI Programming

What is CGI?

  • The Common Gateway Interface, or CGI, is a set of standards that define how information is exchanged between the web server and a custom script.
  • The CGI specs are currently maintained by the NCSA and NCSA defines CGI is as follows:
  • The Common Gateway Interface, or CGI, is a standard for external gateway programs to interface with information servers such as HTTP servers.
  • The current version is CGI/1.1 and CGI/1.2 is under progress.

Web Browsing

To understand the concept of CGI, lets see what happens when we click a hyper link to browse a particular web page or URL.

  • Your browser contacts the HTTP web server and demands for the URL i.e., filename.
  • Web Server will parse the URL and will look for the filename in if it finds that file then sends it back to the browser, otherwise sends an error message indicating that you have requested a wrong file.
  • Web browser takes response from web server and displays either the received file or error message.

However, it is possible to set up the HTTP server so that whenever a file in a certain directory is requested that file is not sent back; instead it is executed as a program, and whatever that program outputs is sent back for your browser to display. This function is called the Common Gateway Interface or CGI and the programs are called CGI scripts. These CGI programs can be a Python Script, PERL Script, Shell Script, C or C++ program, etc.

CGI Architecture Diagram

CGI Architecture

Web Server Support & Configuration

Before you proceed with CGI Programming, make sure that your Web Server supports CGI and it is configured to handle CGI Programs. All the CGI Programs to be executed by the HTTP server are kept in a pre-configured directory. This directory is called CGI Directory and by convention it is named as /var/www/cgi-bin. By convention, CGI files will have extension as .cgi, but you can keep your files with python extension .py as well.

By default, the Linux server is configured to run only the scripts in the cgi-bin directory in /var/www. If you want to specify any other directory to run your CGI scripts, comment the following lines in the httpd.conf file:

<Directory "/var/www/cgi-bin">
   AllowOverride None
   Options ExecCGI
   Order allow,deny
   Allow from all
</Directory>

<Directory "/var/www/cgi-bin">
Options All
</Directory>

Here, I assumed that you have Web Server up and running successfully and you are able to run any other CGI program like Perl or Shell, etc.

First CGI Program

Here is a simple link, which is linked to a CGI script called hello.py. This file is being kept in /var/www/cgi-bin directory and it has following content. Before running your CGI program, make sure you have change mode of file using chmod 755 hello.py UNIX command to make file executable.

#!/usr/bin/python

print "Content-type:text/html\r\n\r\n"
print '<html>'
print '<head>'
print '<title>Hello Word - First CGI Program</title>'
print '</head>'
print '<body>'
print '<h2>Hello Word! This is my first CGI program</h2>'
print '</body>'
print '</html>'

If you click hello.py, then this produces the following output:

Hello Word! This is my first CGI program

This hello.py script is a simple Python script, which is writing its output on STDOUT file i.e., screen. There is one important and extra feature available which is first line to be printed Content-type:text/html\r\n\r\n. This line is sent back to the browser and specifiy the content type to be displayed on the browser screen.

Now, you must have understood basic concept of CGI and you can write many complicated CGI programs using Python. This script can interact with any other external system also to exchange information such as RDBMS.

HTTP Header

The line Content-type:text/html\r\n\r\n is part of HTTP header which is sent to the browser to understand the content. All the HTTP header will be in the following form:

HTTP Field Name: Field Content

For Example
Content-type: text/html\r\n\r\n

There are few other important HTTP headers, which you will use frequently in your CGI Programming.

Header Description
Content-type: A MIME string defining the format of the file being returned. Example is Content-type:text/html
Expires: Date The date the information becomes invalid. This should be used by the browser to decide when a page needs to be refreshed. A valid date string should be in the format 01 Jan 1998 12:00:00 GMT.
Location: URL The URL that should be returned instead of the URL requested. You can use this field to redirect a request to any file.
Last-modified: Date The date of last modification of the resource.
Content-length: N The length, in bytes, of the data being returned. The browser uses this value to report the estimated download time for a file.
Set-Cookie: String Set the cookie passed through the string

CGI Environment Variables

All the CGI program will have access to the following environment variables. These variables play an important role while writing any CGI program.

Variable Name Description
CONTENT_TYPE The data type of the content. Used when the client is sending attached content to the server. For example, file upload, etc.
CONTENT_LENGTH The length of the query information. It’s available only for POST requests.
HTTP_COOKIE Returns the set cookies in the form of key & value pair.
HTTP_USER_AGENT The User-Agent request-header field contains information about the user agent originating the request. Its name of the web browser.
PATH_INFO The path for the CGI script.
QUERY_STRING The URL-encoded information that is sent with GET method request.
REMOTE_ADDR The IP address of the remote host making the request. This can be useful for logging or for authentication purpose.
REMOTE_HOST The fully qualified name of the host making the request. If this information is not available then REMOTE_ADDR can be used to get IR address.
REQUEST_METHOD The method used to make the request. The most common methods are GET and POST.
SCRIPT_FILENAME The full path to the CGI script.
SCRIPT_NAME The name of the CGI script.
SERVER_NAME The server’s hostname or IP Address
SERVER_SOFTWARE The name and version of the software the server is running.

Here is small CGI program to list out all the CGI variables. Click this link to see the result Get Environment

#!/usr/bin/python

import os

print "Content-type: text/html\r\n\r\n";
print "<font size=+1>Environment</font><\br>";
for param in os.environ.keys():
  print "<b>%20s</b>: %s<\br>" % (param, os.environ[param])

GET and POST Methods

You must have come across many situations when you need to pass some information from your browser to web server and ultimately to your CGI Program. Most frequently, browser uses two methods two pass this information to web server. These methods are GET Method and POST Method.

Passing Information using GET method:

The GET method sends the encoded user information appended to the page request. The page and the encoded information are separated by the ? character as follows:

http://www.test.com/cgi-bin/hello.py?key1=value1&key2=value2

The GET method is the default method to pass information from browser to web server and it produces a long string that appears in your browser’s Location:box. Never use GET method if you have password or other sensitive information to pass to the server. The GET method has size limtation: only 1024 characters can be sent in a request string. The GET method sends information using QUERY_STRING header and will be accessible in your CGI Program through QUERY_STRING environment variable.

You can pass information by simply concatenating key and value pairs along with any URL or you can use HTML <FORM> tags to pass information using GET method.

Simple URL Example : Get Method

Here is a simple URL, which will pass two values to hello_get.py program using GET method.

/cgi-bin/hello_get.py?first_name=ZARA&last_name=ALI

Below is hello_get.py script to handle input given by web browser. We are going to use cgi module, which makes it very easy to access passed information:

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
first_name = form.getvalue('first_name')
last_name  = form.getvalue('last_name')

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Hello - Second CGI Program</title>"
print "</head>"
print "<body>"
print "<h2>Hello %s %s</h2>" % (first_name, last_name)
print "</body>"
print "</html>"

This would generate the following result:

Hello ZARA ALI

Simple FORM Example: GET Method

Here is a simple example which passes two values using HTML FORM and submit button. We are going to use same CGI script hello_get.py to handle this imput.

<form action="/cgi-bin/hello_get.py" method="get">
First Name: <input type="text" name="first_name">  <br />

Last Name: <input type="text" name="last_name" />
<input type="submit" value="Submit" />
</form>

Here is the actual output of the above form, You enter First and Last Name and then click submit button to see the result.

First Name:
Last Name:

Passing Information using POST method:

A generally more reliable method of passing information to a CGI program is the POST method. This packages the information in exactly the same way as GET methods, but instead of sending it as a text string after a ? in the URL it sends it as a separate message. This message comes into the CGI script in the form of the standard input.

Below is same hello_get.py script which handles GET as well as POST method.

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
first_name = form.getvalue('first_name')
last_name  = form.getvalue('last_name')

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Hello - Second CGI Program</title>"
print "</head>"
print "<body>"
print "<h2>Hello %s %s</h2>" % (first_name, last_name)
print "</body>"
print "</html>"

Let us take again same example as above which passes two values using HTML FORM and submit button. We are going to use same CGI script hello_get.py to handle this imput.

<form action="/cgi-bin/hello_get.py" method="post">
First Name: <input type="text" name="first_name"><br />
Last Name: <input type="text" name="last_name" />

<input type="submit" value="Submit" />
</form>

Here is the actual output of the above form. You enter First and Last Name and then click submit button to see the result.

First Name:
Last Name:

Passing Checkbox Data to CGI Program

Checkboxes are used when more than one option is required to be selected.

Here is example HTML code for a form with two checkboxes:

<form action="/cgi-bin/checkbox.cgi" method="POST" target="_blank">
<input type="checkbox" name="maths" value="on" /> Maths
<input type="checkbox" name="physics" value="on" /> Physics
<input type="submit" value="Select Subject" />
</form>

The result of this code is the following form:

Maths PhysicsBelow is checkbox.cgi script to handle input given by web browser for checkbox button.

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
if form.getvalue('maths'):
   math_flag = "ON"
else:
   math_flag = "OFF"

if form.getvalue('physics'):
   physics_flag = "ON"
else:
   physics_flag = "OFF"

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Checkbox - Third CGI Program</title>"
print "</head>"
print "<body>"
print "<h2> CheckBox Maths is : %s</h2>" % math_flag
print "<h2> CheckBox Physics is : %s</h2>" % physics_flag
print "</body>"
print "</html>"

Passing Radio Button Data to CGI Program

Radio Buttons are used when only one option is required to be selected.

Here is example HTML code for a form with two radio buttons:

<form action="/cgi-bin/radiobutton.py" method="post" target="_blank">
<input type="radio" name="subject" value="maths" /> Maths
<input type="radio" name="subject" value="physics" /> Physics
<input type="submit" value="Select Subject" />
</form>

The result of this code is the following form:

Maths PhysicsBelow is radiobutton.py script to handle input given by web browser for radio button:

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
if form.getvalue('subject'):
   subject = form.getvalue('subject')
else:
   subject = "Not set"

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Radio - Fourth CGI Program</title>"
print "</head>"
print "<body>"
print "<h2> Selected Subject is %s</h2>" % subject
print "</body>"
print "</html>"

Passing Text Area Data to CGI Program

TEXTAREA element is used when multiline text has to be passed to the CGI Program.

Here is example HTML code for a form with a TEXTAREA box:

<form action="/cgi-bin/textarea.py" method="post" target="_blank">
<textarea name="textcontent" cols="40" rows="4">
Type your text here...
</textarea>
<input type="submit" value="Submit" />
</form>

The result of this code is the following form:

Type your text here…Below is textarea.cgi script to handle input given by web browser:

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
if form.getvalue('textcontent'):
   text_content = form.getvalue('textcontent')
else:
   text_content = "Not entered"

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>";
print "<title>Text Area - Fifth CGI Program</title>"
print "</head>"
print "<body>"
print "<h2> Entered Text Content is %s</h2>" % text_content
print "</body>"

Passing Drop Down Box Data to CGI Program

Drop Down Box is used when we have many options available but only one or two will be selected.

Here is example HTML code for a form with one drop down box:

<form action="/cgi-bin/dropdown.py" method="post" target="_blank">
<select name="dropdown">
<option value="Maths" selected>Maths</option>
<option value="Physics">Physics</option>
</select>
<input type="submit" value="Submit"/>
</form>

The result of this code is the following form:

Below is dropdown.py script to handle input given by web browser.

#!/usr/bin/python

# Import modules for CGI handling 
import cgi, cgitb 

# Create instance of FieldStorage 
form = cgi.FieldStorage() 

# Get data from fields
if form.getvalue('dropdown'):
   subject = form.getvalue('dropdown')
else:
   subject = "Not entered"

print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Dropdown Box - Sixth CGI Program</title>"
print "</head>"
print "<body>"
print "<h2> Selected Subject is %s</h2>" % subject
print "</body>"
print "</html>"

Using Cookies in CGI

HTTP protocol is a stateless protocol. But for a commercial website, it is required to maintain session information among different pages. For example, one user registration ends after completing many pages. But how to maintain user’s session information across all the web pages.

In many situations, using cookies is the most efficient method of remembering and tracking preferences, purchases, commissions, and other information required for better visitor experience or site statistics.

How It Works?

Your server sends some data to the visitor’s browser in the form of a cookie. The browser may accept the cookie. If it does, it is stored as a plain text record on the visitor’s hard drive. Now, when the visitor arrives at another page on your site, the cookie is available for retrieval. Once retrieved, your server knows/remembers what was stored.

Cookies are a plain text data record of 5 variable-length fields:

  • Expires : The date the cookie will expire. If this is blank, the cookie will expire when the visitor quits the browser.
  • Domain : The domain name of your site.
  • Path : The path to the directory or web page that sets the cookie. This may be blank if you want to retrieve the cookie from any directory or page.
  • Secure : If this field contains the word “secure”, then the cookie may only be retrieved with a secure server. If this field is blank, no such restriction exists.
  • Name=Value : Cookies are set and retrieved in the form of key and value pairs.

Setting up Cookies

It is very easy to send cookies to browser. These cookies will be sent along with HTTP Header before to Content-type field. Assuming you want to set UserID and Password as cookies. So cookies setting will be done as follows:

#!/usr/bin/python

print "Set-Cookie:UserID=XYZ;\r\n"
print "Set-Cookie:Password=XYZ123;\r\n"
print "Set-Cookie:Expires=Tuesday, 31-Dec-2007 23:12:40 GMT";\r\n"
print "Set-Cookie:Domain=www.tutorialspoint.com;\r\n"
print "Set-Cookie:Path=/perl;\n"
print "Content-type:text/html\r\n\r\n"
...........Rest of the HTML Content....

From this example, you must have understood how to set cookies. We use Set-Cookie HTTP header to set cookies.

Here, it is optional to set cookies attributes like Expires, Domain and Path. It is notable that cookies are set before sending magic line “Content-type:text/html\r\n\r\n.

Retrieving Cookies

It is very easy to retrieve all the set cookies. Cookies are stored in CGI environment variable HTTP_COOKIE and they will have following form:

key1=value1;key2=value2;key3=value3....

Here is an example of how to retrieve cookies.

#!/usr/bin/python

# Import modules for CGI handling 
from os import environ
import cgi, cgitb

if environ.has_key('HTTP_COOKIE'):
   for cookie in map(strip, split(environ['HTTP_COOKIE'], ';')):
      (key, value ) = split(cookie, '=');
      if key == "UserID":
         user_id = value

      if key == "Password":
         password = value

print "User ID  = %s" % user_id
print "Password = %s" % password

This will produce the following result for the cookies set by above script:

User ID = XYZ
Password = XYZ123

File Upload Example:

To upload a file, the HTML form must have the enctype attribute set to multipart/form-data. The input tag with the file type will create a “Browse” button.

<html>
<body>
   <form enctype="multipart/form-data" 
                     action="save_file.py" method="post">
   <p>File: <input type="file" name="filename" /></p>
   <p><input type="submit" value="Upload" /></p>
   </form>
</body>
</html>

The result of this code is the following form:

File:

Above example has been disabled intentionally to save people uploading file on our server, but you can try above code with your server.

Here is the script save_file.py to handle file upload:

#!/usr/bin/python

import cgi, os
import cgitb; cgitb.enable()

form = cgi.FieldStorage()

# Get filename here.
fileitem = form['filename']

# Test if the file was uploaded
if fileitem.filename:
   # strip leading path from file name to avoid 
   # directory traversal attacks
   fn = os.path.basename(fileitem.filename)
   open('/tmp/' + fn, 'wb').write(fileitem.file.read())

   message = 'The file "' + fn + '" was uploaded successfully'
   
else:
   message = 'No file was uploaded'
   
print """\
Content-Type: text/html\n
<html>
<body>
   <p>%s</p>
</body>
</html>
""" % (message,)

If you are running above script on Unix/Linux, then you would have to take care of replacing file separator as follows, otherwise on your windows machine above open() statement should work fine.

fn = os.path.basename(fileitem.filename.replace("\\", "/" ))

How To Raise a “File Download” Dialog Box ?

Sometimes, it is desired that you want to give option where a user will click a link and it will pop up a “File Download” dialogue box to the user instead of displaying actual content. This is very easy and will be achieved through HTTP header. This HTTP header will be different from the header mentioned in previous section.

For example,if you want make a FileName file downloadable from a given link, then its syntax will be as follows:

#!/usr/bin/python

# HTTP Header
print "Content-Type:application/octet-stream; name=\"FileName\"\r\n";
print "Content-Disposition: attachment; filename=\"FileName\"\r\n\n";

# Actual File Content will go hear.
fo = open("foo.txt", "rb")

str = fo.read();
print str

# Close opend file
fo.close()

Python MySQL Database Access

The Python standard for database interfaces is the Python DB-API. Most Python database interfaces adhere to this standard.

You can choose the right database for your application. Python Database API supports a wide range of database servers:

  • GadFly
  • mSQL
  • MySQL
  • PostgreSQL
  • Microsoft SQL Server 2000
  • Informix
  • Interbase
  • Oracle
  • Sybase

Here is the list of available Python database interfaces: Python Database Interfaces and APIs .You must download a separate DB API module for each database you need to access. For example, if you need to access an Oracle database as well as a MySQL database, you must download both the Oracle and the MySQL database modules.

The DB API provides a minimal standard for working with databases using Python structures and syntax wherever possible. This API includes the following:

  • Importing the API module.
  • Acquiring a connection with the database.
  • Issuing SQL statements and stored procedures.
  • Closing the connection

We would learn all the concepts using MySQL, so let’s talk about MySQLdb module only.

What is MySQLdb?

MySQLdb is an interface for connecting to a MySQL database server from Python. It implements the Python Database API v2.0 and is built on top of the MySQL C API.

How do I install the MySQLdb?

Before proceeding, you make sure you have MySQLdb installed on your machine. Just type the following in your Python script and execute it:

#!/usr/bin/python

import MySQLdb

If it produces the following result, then it means MySQLdb module is not installed:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    import MySQLdb
ImportError: No module named MySQLdb

To install MySQLdb module, download it from MySQLdb Download page and proceed as follows:

$ gunzip MySQL-python-1.2.2.tar.gz
$ tar -xvf MySQL-python-1.2.2.tar
$ cd MySQL-python-1.2.2
$ python setup.py build
$ python setup.py install

Note: Make sure you have root privilege to install above module.

Database Connection:

Before connecting to a MySQL database, make sure of the followings:

  • You have created a database TESTDB.
  • You have created a table EMPLOYEE in TESTDB.
  • This table is having fields FIRST_NAME, LAST_NAME, AGE, SEX and INCOME.
  • User ID “testuser” and password “test123” are set to access TESTDB.
  • Python module MySQLdb is installed properly on your machine.
  • You have gone through MySQL tutorial to understand MySQL Basics.

Example:

Following is the example of connecting with MySQL database “TESTDB”

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")

# Fetch a single row using fetchone() method.
data = cursor.fetchone()

print "Database version : %s " % data

# disconnect from server
db.close()

While running this script, it is producing the following result in my Linux machine.

Database version : 5.0.45

If a connection is established with the datasource, then a Connection Object is returned and saved into db for further use, otherwise db is set to None. Next, db object is used to create a cursor object, which in turn is used to execute SQL queries. Finally, before coming out, it ensures that database connection is closed and resources are released.

Creating Database Table:

Once a database connection is established, we are ready to create tables or records into the database tables using execute method of the created cursor.

Example:

First, let’s create Database table EMPLOYEE:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Drop table if it already exist using execute() method.
cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")

# Create table as per requirement
sql = """CREATE TABLE EMPLOYEE (
         FIRST_NAME  CHAR(20) NOT NULL,
         LAST_NAME  CHAR(20),
         AGE INT,  
         SEX CHAR(1),
         INCOME FLOAT )"""

cursor.execute(sql)

# disconnect from server
db.close()

INSERT Operation:

INSERT operation is required when you want to create your records into a database table.

Example:

Following is the example, which executes SQL INSERT statement to create a record into EMPLOYEE table:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = """INSERT INTO EMPLOYEE(FIRST_NAME,
         LAST_NAME, AGE, SEX, INCOME)
         VALUES ('Mac', 'Mohan', 20, 'M', 2000)"""
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Above example can be written as follows to create SQL queries dynamically:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
       LAST_NAME, AGE, SEX, INCOME) \
       VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
       ('Mac', 'Mohan', 20, 'M', 2000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Example:

Following code segment is another form of execution where you can pass parameters directly:

..................................
user_id = "test123"
password = "password"

con.execute('insert into Login values("%s", "%s")' % \
             (user_id, password))
..................................

READ Operation:

READ Operation on any databasse means to fetch some useful information from the database.

Once our database connection is established, we are ready to make a query into this database. We can use either fetchone() method to fetch single record or fetchall() method to fetech multiple values from a database table.

  • fetchone(): This method fetches the next row of a query result set. A result set is an object that is returned when a cursor object is used to query a table.
  • fetchall(): This method fetches all the rows in a result set. If some rows have already been extracted from the result set, the fetchall() method retrieves the remaining rows from the result set.
  • rowcount: This is a read-only attribute and returns the number of rows that were affected by an execute() method.

Example:

Following is the procedure to query all the records from EMPLOYEE table having salary more than 1000:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "SELECT * FROM EMPLOYEE \
       WHERE INCOME > '%d'" % (1000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Fetch all the rows in a list of lists.
   results = cursor.fetchall()
   for row in results:
      fname = row[0]
      lname = row[1]
      age = row[2]
      sex = row[3]
      income = row[4]
      # Now print fetched result
      print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
             (fname, lname, age, sex, income )
except:
   print "Error: unable to fecth data"

# disconnect from server
db.close()

This will produce the following result:

fname=Mac, lname=Mohan, age=20, sex=M, income=2000

Update Operation:

UPDATE Operation on any databasse means to update one or more records, which are already available in the database. Following is the procedure to update all the records having SEX as ‘M’. Here, we will increase AGE of all the males by one year.

Example:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to UPDATE required records
sql = "UPDATE EMPLOYEE SET AGE = AGE + 1
                          WHERE SEX = '%c'" % ('M')
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

DELETE Operation:

DELETE operation is required when you want to delete some records from your database. Following is the procedure to delete all the records from EMPLOYEE where AGE is more than 20:

Example:

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to DELETE required records
sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Performing Transactions:

Transactions are a mechanism that ensures data consistency. Transactions should have the following four properties:

  • Atomicity: Either a transaction completes or nothing happens at all.
  • Consistency: A transaction must start in a consistent state and leave the system in a consistent state.
  • Isolation: Intermediate results of a transaction are not visible outside the current transaction.
  • Durability: Once a transaction was committed, the effects are persistent, even after a system failure.

The Python DB API 2.0 provides two methods to either commit or rollback a transaction.

Example:

You already have seen how we have implemented transations. Here is again similar example:

# Prepare SQL query to DELETE required records
sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

COMMIT Operation:

Commit is the operation, which gives a green signal to database to finalize the changes, and after this operation, no change can be reverted back.

Here is a simple example to call commit method.

 db.commit()

ROLLBACK Operation:

If you are not satisfied with one or more of the changes and you want to revert back those changes completely, then use rollback() method.

Here is a simple example to call rollback() method.

 db.rollback()

Disconnecting Database:

To disconnect Database connection, use close() method.

 db.close()

If the connection to a database is closed by the user with the close() method, any outstanding transactions are rolled back by the DB. However, instead of depending on any of DB lower level implementation details, your application would be better off calling commit or rollback explicitly.

Handling Errors:

There are many sources of errors. A few examples are a syntax error in an executed SQL statement, a connection failure, or calling the fetch method for an already canceled or finished statement handle.

The DB API defines a number of errors that must exist in each database module. The following table lists these exceptions.

Exception Description
Warning Used for non-fatal issues. Must subclass StandardError.
Error Base class for errors. Must subclass StandardError.
InterfaceError Used for errors in the database module, not the database itself. Must subclass Error.
DatabaseError Used for errors in the database. Must subclass Error.
DataError Subclass of DatabaseError that refers to errors in the data.
OperationalError Subclass of DatabaseError that refers to errors such as the loss of a connection to the database. These errors are generally outside of the control of the Python scripter.
IntegrityError Subclass of DatabaseError for situations that would damage the relational integrity, such as uniqueness constraints or foreign keys.
InternalError Subclass of DatabaseError that refers to errors internal to the database module, such as a cursor no longer being active.
ProgrammingError Subclass of DatabaseError that refers to errors such as a bad table name and other things that can safely be blamed on you.
NotSupportedError Subclass of DatabaseError that refers to trying to call unsupported functionality.

Your Python scripts should handle these errors, but before using any of the above exceptions, make sure your MySQLdb has support for that exception. You can get more information about them by reading the DB API 2.0 specification.

Python Network Programming

Python provides two levels of access to network services. At a low level, you can access the basic socket support in the underlying operating system, which allows you to implement clients and servers for both connection-oriented and connectionless protocols.

Python also has libraries that provide higher-level access to specific application-level network protocols, such as FTP, HTTP, and so on.

This tutorial gives you understanding on most famous concept in Networking – Socket Programming

What is Sockets?

Sockets are the endpoints of a bidirectional communications channel. Sockets may communicate within a process, between processes on the same machine, or between processes on different continents.

Sockets may be implemented over a number of different channel types: Unix domain sockets, TCP, UDP, and so on. The socket library provides specific classes for handling the common transports as well as a generic interface for handling the rest.

Sockets have their own vocabulary:

Term Description
domain The family of protocols that will be used as the transport mechanism. These values are constants such as AF_INET, PF_INET, PF_UNIX, PF_X25, and so on.
type The type of communications between the two endpoints, typically SOCK_STREAM for connection-oriented protocols and SOCK_DGRAM for connectionless protocols.
protocol Typically zero, this may be used to identify a variant of a protocol within a domain and type.
hostname The identifier of a network interface:

  • A string, which can be a host name, a dotted-quad address, or an IPV6 address in colon (and possibly dot) notation
  • A string “<broadcast>”, which specifies an INADDR_BROADCAST address.
  • A zero-length string, which specifies INADDR_ANY, or
  • An Integer, interpreted as a binary address in host byte order.
port Each server listens for clients calling on one or more ports. A port may be a Fixnum port number, a string containing a port number, or the name of a service.

The socket Module:

To create a socket, you must use the socket.socket() function available in socket module, which has the general syntax:

s = socket.socket (socket_family, socket_type, protocol=0)

Here is the description of the parameters:

  • socket_family: This is either AF_UNIX or AF_INET, as explained earlier.
  • socket_type: This is either SOCK_STREAM or SOCK_DGRAM.
  • protocol: This is usually left out, defaulting to 0.

Once you have socket object, then you can use required functions to create your client or server program. Following is the list of functions required:

Server Socket Methods:

Method Description
s.bind() This method binds address (hostname, port number pair) to socket.
s.listen() This method sets up and start TCP listener.
s.accept() This passively accept TCP client connection, waiting until connection arrives (blocking).

Client Socket Methods:

Method Description
s.connect() This method actively initiates TCP server connection.

General Socket Methods:

Method Description
s.recv() This method receives TCP message
s.send() This method transmits TCP message
s.recvfrom() This method receives UDP message
s.sendto() This method transmits UDP message
s.close() This method closes socket
socket.gethostname() Returns the hostname.

A Simple Server:

To write Internet servers, we use the socket function available in socket module to create a socket object. A socket object is then used to call other functions to setup a socket server.

Now call bind(hostname, port function to specify a port for your service on the given host.

Next, call the accept method of the returned object. This method waits until a client connects to the port you specified, and then returns a connection object that represents the connection to that client.

#!/usr/bin/python           # This is server.py file

import socket               # Import socket module

s = socket.socket()         # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345                # Reserve a port for your service.
s.bind((host, port))        # Bind to the port

s.listen(5)                 # Now wait for client connection.
while True:
   c, addr = s.accept()     # Establish connection with client.
   print 'Got connection from', addr
   c.send('Thank you for connecting')
   c.close()                # Close the connection

A Simple Client:

Now we will write a very simple client program which will open a connection to a given port 12345 and given host. This is very simple to create a socket client using Python’s socket module function.

The socket.connect(hosname, port ) opens a TCP connection to hostname on the port. Once you have a socket open, you can read from it like any IO object. When done, remember to close it, as you would close a file.

The following code is a very simple client that connects to a given host and port, reads any available data from the socket, and then exits:

#!/usr/bin/python           # This is client.py file

import socket               # Import socket module

s = socket.socket()         # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345                # Reserve a port for your service.

s.connect((host, port))
print s.recv(1024)
s.close                     # Close the socket when done

Now run this server.py in background and then run above client.py to see the result.

# Following would start a server in background.
$ python server.py & 

# Once server is started run client as follows:

$ python client.py

This would produce following result:

Got connection from ('127.0.0.1', 48437)
Thank you for connecting

Python Internet modules

A list of some important modules which could be used in Python Network/Internet programming.

Protocol Common function Port No Python module
HTTP Web pages 80 httplib, urllib, xmlrpclib
NNTP Usenet news 119 nntplib
FTP File transfers 20 ftplib, urllib
SMTP Sending email 25 smtplib
POP3 Fetching email 110 poplib
IMAP4 Fetching email 143 imaplib
Telnet Command lines 23 telnetlib
Gopher Document transfers 70 gopherlib, urllib

Please check all the libraries mentioned above to work with FTP, SMTP, POP, and IMAP protocols.

Python Regular Expression

ELECTRONICS 101-SERVO MOTOR

What is Servo Motor?

This is nothing but a simple electrical motor, controlled with the help of servomechanism. If the motor as controlled device, associated with servomechanism is DC motor, then it is commonly known DC Servo Motor. If the controlled motor is operated by AC, it is called AC Servo Motor.

There are some special types of application of electrical motor where rotation of the motor is required for just a certain angle not continuously for long period of time. For these applications some special types of motor are required with some special arrangement which makes the motor to rotate a certain angle for a given electrical input (signal). For this purpose servo motor comes into picture.

Now day’s servo system has huge industrial applications. Servo motor applications are also commonly seen in remote controlled toy cars for controlling direction of motion and it is also very commonly used as the motor which moves the tray of a CD or DVD player. Beside these there are other hundreds of servo motor applications we see in our daily life. The main reason behind using a servo is that it provides angular precision, i.e. it will only rotate as much we want and then stop and wait for next signal to take further action. This is unlike a normal electrical motor which starts rotating as and when power is applied to it and the rotation continues until we switch off the power. We cannot control the rotational progress of electrical motor; but we can only control the speed of rotation and can turn it ON and OFF.

Servo Motor System

Servo Motor Working Principle

Before understanding the working principle of servo motor we should understand first the basic of servomechanism.

Servomechanism

A servo system mainly consists of three basic components – a controlled device, a output sensor, a feedback system.

This is an automatic closed loop control system. Here instead of controlling a device by applying variable input signal, the device is controlled by a feedback signal generated by comparing output signal and reference input signal.

When reference input signal or command signal is applied to the system, it is compared with output reference signal of the system produced by output sensor, and a third signal produced by feedback system. This third signal acts as input signal of controlled device. This input signal to the device presents as long as there is a logical difference between reference input signal and output signal of the system. After the device achieves its desired output, there will be no longer logical difference between reference input signal and reference output signal of the system. Then, third signal produced by comparing theses above said signals will not remain enough to operate the device further and to produce further output of the system until the next reference input signal or command signal is applied to the system. Hence the primary task of a servomechanism is to maintain the output of a system at the desired value in the presence of disturbances.

Working Principle of Servo Motor

A servo motor is basically a DC motor(in some special cases it is AC motor) along with some other special purpose components that make a DC motor a servo. In a servo unit, you will find a small DC motor, a potentiometer, gear arrangement and an intelligent circuitry. The intelligent circuitry along with the potentiometer makes the servo to rotate according to our wishes.

As we know, a small DC motor will rotate with high speed but the torque generated by its rotation will not be enough to move even a light load. This is where the gear system inside a servomechanism comes into picture. The gear mechanism will take high input speed of the motor (fast) and at the output, we will get a output speed which is slower than original input speed but more practical and widely applicable.

Say at initial position of servo motor shaft, the position of the potentiometer knob is such that there is no electrical signal generated at the output port of the potentiometer . This output port of the potentiometer is connected with one of the input terminals of the error detector amplifier. Now an electrical signal is given to another input terminal of the error detector amplifier. Now difference between these two signals, one comes from potentiometer and another comes from external source, will be amplified in the error detector amplifier and feeds the DC motor. This amplified error signal acts as the input power of the dc motor and the motor starts rotating in desired direction. As the motor shaft progresses the potentiometer knob also rotates as it is coupled with motor shaft with help of gear arrangement. As the position of the potentiometer knob changes there will be an electrical signal produced at the potentiometer port. As the angular position of the potentiometer knob progresses the output or feedback signal increases. After desired angular position of motor shaft the potentiometer knob is reaches at such position the electrical signal generated in the potentiometer becomes same as of external electrical signal given to amplifier. At this condition, there will be no output signal from the amplifier to the motor input as there is no difference between external applied signal and the signal generated at potentiometer . As the input signal to the motor is nil at that position, the motor stops rotating. This is how a simple conceptual servo motor works.

Servo Motor Control

For understanding servo motor control let us consider an example of servomotor that we have given a signal to rotate by an angle of 45° and then stop and wait for further instruction.

The shaft of the DC motor is coupled with another shaft called output shaft, with help of gear assembly. This gear assembly is used to step down the high rpm of the motor’s shaft to low rpm at output shaft of the servo system.

servo-motor-1

The voltage adjusting knob of a potentiometer is so arranged with the output shaft by means of another gear assembly, that during rotation of the shaft, the knob also rotates and creates an varying electrical potential according to the principle of potentiometer . This signal i.e. electrical potential is increased with angular movement of potentiometer knob along with the system shaft from 0° to 45°. This electrical potential or voltage is taken to the error detector feedback amplifier along with the input reference commends i.e. input signal voltage.

servo-motor-2

As the angle of rotation of the shaft increases from 0° to 45° the voltage from potentiometer increases. At 45° this voltage reaches to a value which is equal to the given input command voltage to the system. As at this position of the shaft, there is no difference between the signal voltage coming from the potentiometer and reference input voltage (command signal) to the system, the output voltage of the amplifier becomes zero.

Servo Motor Circuit

As per the picture given above the output electrical voltage signal of the amplifier, acts as input voltage of the DC motor. Hence the motor will stop rotating after the shaft rotates by 45°. The motor will be at this rest position until another command is given to the system for further movement of the shaft in desired direction. From this example we can understand the most basic servo motor theory and how servo motor control is achieved.

NB: Although in practical servo motor control system, instead of using simple potentiometer we use digital or analog position sensor encoder.

From this basic working principle of servo motor it can be concluded. The shaft of the servo is connected to a potentiometer . The circuitry inside the servo, to which the potentiometer is connected, knows the position of the servo. The current position will be compared with the desired position continuously with the help of an Error Detection Amplifier. If a mismatch is found, then an error signal is provided at the output of the error amplifier and the shaft will rotate to go the exact location required. Once the desired location is reached, it stops and waits.

Continuous Rotation Servo Motors

Continuous rotation servo motors are actually a modified version of what the servos are actually meant to do, that is, control the shaft position. The 360° rotation servos are actually made by changing certain mechanical connections inside the servo. However, certain manufacturer like parallax sells these servos as well. With the continuous rotation servo you can only control the direction and speed of the servo, but not the position.
Two of the most popular Servo motor manufacturers are FUTABA and HITEC.

ARDUINO PROJECT-5:ROBOTIC HAND

Material used –

Arduino UNO

5 servo motors

5 sensors bending

5 resistance

the material to build the robotic hand (tubes, paper, glue, wire, etc …)

glove.

In this tutorial we make a robotic hand (low cost) can replicate the movements of our hand.
We insert the sensor flex on the fingers of a glove, we analyze the value of these sensors and servo motors move 180 ° pulling a wire connected to each finger of the robotic hand.
The electronics then consists in creating a glove with sensors and robotic hand with the five scooters.

SENSORE_DI_FLESSIONE

The sensors of flexion (picture above) are small flexible rods that indicate the amount of bending to which they are subjected. Attacking the fingers of the glove know exactly the movement of each finger.
The flex sensors must be connected to Arduino with resistors. They can be purchased online at Robot-Italy.
Sensor: www.robot-italy.com/it/flex-sensor-2-2.html – 8 €
Resistances: www.robot-italy.com/it/resk-kit-500-resistenze.html – 5 € (Kit 500)
Otherwise you can buy at any electronics store of resistance from 2.2 Kohms
This tutorial was made ​​in 2014, prior to purchase online if you check other sites sell cheaper sensors downturn. Currently reported this is the cheapest.

flex_sensor_1_bb

The sensor must be connected bending this way: the sidebar closer must be powered with 5V Arduino. The largest bar must be connected to GND Arduino passing through a resistance. The cable that comes to analog pin Arduino (used to read the value of the sensor) must be paid before the resistance (see diagram above). What is the resistance?
In this case the resistance serves to decrease and make visible the sensor value.
Inserting a resistance of 2.2 Kohms as we read analog value equal to 50 if the sensor is not bent, equal to 90 if the sensor is flexed in one direction and a value of 10 if the sensor is bent in the opposite direction.
Below is a small chart to see the sensor values ​​of flexion.
NB: if you use another type of resistance these values ​​will be different.

hands

SENSOR VALUE = 50

SENSOR VALUE = 10

We start with only one sensor. Made the circuit of the previous page and open the Arduino software to start programming. For the moment we do not use either the glove or the robotic hand.
The assembly will be done at the end, when the code and the circuit of a single finger will be completed.
Computer Science

To test the operation of the sensor open the file inside the Arduino program for reading analog sensor (FILE – EXAMPLES – BASIC – ANALOGREADSERIAL).
This sketch allows you to read any sensor positioned in pin 0 Arduino.

Uplodate code tab and open the Serial Monitor.
Holding the sensor stopped the displayed number should be a 50.
Bending it forward reach the number 10, by folding it in the reverse direction the number 90.
If you used a different resistance value changes.

Write down the values ​​of the sensor retainer and bent forward. You will have to replace them with those that I will use in the tutorial.

COMMAND MAP (Open the sketc _2_1_sensore_1_motore_val)
Now to the servo motors.
We connect a servant Arduino following the diagram below.

flex_sensor_1_motor_1

When the sensor will be bending bent, the motor will move proportionally.
We have already seen that with the command myservo.write (90); we move our servant in any angle between 0 and 180 °. Is there a command in the Arduino program that allows us to pass a value to the motor in proportion to that which occurs in the sensor.
The sensor sends a fact that there is between 90 and 10. The motor needs a value between 0 and 180.
The map command executes the relationship between the two values. Ex: value = map (value, 90, 10, 0, 179);
The first two numbers refer to flex sensor while the other two to the servant. Based on the incoming number have a value suitable for the motor.
Try to load the code below to better understand the operation of the MAP command (or open the file _2_1_sensore_1_motore_val)

#include <Servo.h>
Servo myservo;
int index;
void setup () {
Serial.begin (9600);
myservo.attach (9);
// We attack the servant to pin 9
}
void loop () {
index = analogRead (A0);
Serial.print (“original value”);
Serial.println (index);
index = map (index, 90, 10, 0, 179);
myservo.write (index);
Serial.print (“scaled value:”);
Serial.println (index);
// We print the original value of the sensor and then modified by the map command
delay (150);
}

If we bend the sensor motor moves and pulls the wire which is connected the finger of the robotic hand.
We begin to realize the glove and the robotic hand.
We take a glove and attach the sensor on the index finger. The best way to fix it is to sew it or fix it with duct tape.
If possible we settle the long electric cable to the sensor and link to them with Arduino.
We’re still making a very basic prototype, once you understand the operation and the amount of cables that we’ll use to mount the glove final.

robot_hand_1

With diagram we settle five red wires to the positive pin sensor flex. The ends of the red wires must be welded together on the back of the hand. To 5 wires we add a longer one that connect to pin 5V Arduino.

flex_sensor_5_dita_nero

Take 5 small wires blacks and we settle them at the other end of the sensor flex. For every black cable must weld resistance. At the other end of the resistance we settle another 5 wires blacks who join at the center of the hand. From the latter part of a last longest black wire that we insert into Pin GND Arduino.

flex_sensor_5_dita_giallo

At this point we have to connect 5 wires long yellow. Each yellow wire must be welded in the first end of the resistance and must be connected to analog pin Arduino. Each of the yellow wires in the end, therefore, pin A0, A1, A2, A3 and A4. These wires will send to Arduino data of each sensor flex.

flex_sensor_5_dita_bb

robot_hand_arduino

glove-interactive

Now for the robotic hand.
Realize the hand is the hardest part of the entire tutorial.
Stumble this tutorial (click here) made very well. The purpose is to create a hand with fingers connected to wires. When the motor pulls the wire, the finger bends.
If you do not want to build the robotic hand alone can buy this toy 4M:
Robotic Hand Kit – www.amazon.com/4M-3774-Robotic-Hand-Kit/dp/B005MK0OPO
robot_hand_4m

Once mounted just connect the various threads to 5 scooters.
The realization of the hand I leave it to your imagination. Please send me email ( puccimarco76@yahoo.it ) your creations, I will publish on the site and on the page
Facebook Tutorials Arduino:
https://www.facebook.com/pages/Tutorial-Arduino/1377902315765530?fref=ts .

Below are some pictures of the robotic hand made with a wooden frame and fingers toy presented above.
The structure is the key part of the hand. The five motors must not move (glue them to the wooden dowels or screws to secure them with metal bars).
With the fishing line to connect each finger scooter built and assemble the electronic part of the 5 scooters.

mano_robotica

mano_robotica_arduino

mano_robotica2

mano_robotica_guanto-arduino

(Open the sketc _4_tutti_sensori_tutti_motori_val)

Below is the complete code to manage the sensors 5 and 5 scooters.
The values ​​to be included in the MAP command, for example. map (inch, 20, 40, 0, 179);
may vary depending on your sensor, your strength, the type of scooters, etc …
I recommend you do the tests by changing the numbers until you find the right compromise between movement of the glove and answer the robotic hand.

#include <Servo.h>
Servo myservo_pollice;
Servo myservo_indice;
Servo myservo_medio;
Servo myservo_anulare;
Servo myservo_mignolo;
int inch;
int index;
int average;
int ring;
int little finger;

void setup () {
Serial.begin (9600);
myservo_pollice.attach (9);
myservo_indice.attach (10);
myservo_medio.attach (11);
myservo_anulare.attach (12);
myservo_mignolo.attach (13);
}

void loop () {
analogRead inch = (A0);
index = analogRead (A1);
average = analogRead (A2);
annular analogRead = (A3);
pinky = analogRead (A4);
Serial.println (little finger);

inch = map (inch, 20, 40, 0, 179);
index = map (index, 20, 40, 0, 179);
average = map (average, 20, 40, 0, 179);
ring = map (ring, 20, 40, 0, 179);
= map pinky (little finger, 20, 40, 0, 179);

myservo_pollice.write (thumb);
myservo_indice.write (index);
myservo_medio.write (average);
myservo_anulare.write (road);
myservo_mignolo.write (little finger);

delay (150);
}

ELECTRONICS 101-AVR MICROCONTROLLER

AVR was developed in the year 1996 by Atmel Corporation. The architecture of AVR was developed by Alf-Egil Bogen and Vegard Wollan. AVR derives its name from its developers and stands for Alf-Egil Bogen Vegard Wollan RISC microcontroller, also known as Advanced Virtual RISC. The AT90S8515 was the first microcontroller which was based on AVR architecture however the first microcontroller to hit the commercial market was AT90S1200 in the year 1997.
AVR microcontrollers are available in three categories:
1.      TinyAVR – Less memory, small size, suitable only for simpler applications
2.      MegaAVR – These are the most popular ones having good amount of memory (upto 256 KB), higher number of inbuilt peripherals and suitable for moderate to complex applications.
3.      XmegaAVR – Used commercially for complex applications, which require large program memory and high speed.
The following table compares the above mentioned AVR series of microcontrollers:
Series Name
Pins
Flash Memory
Special Feature
TinyAVR
6-32
0.5-8 KB
Small in size
MegaAVR
28-100
4-256KB
Extended peripherals
XmegaAVR
44-100
16-384KB
DMA , Event System included
What’s special about AVR?
They are fast: AVR microcontroller executes most of the instructions in single execution cycle. AVRs are about 4 times faster than PICs, they consume less power and can be operated in different power saving modes. Let’s do the comparison between the three most commonly used families of microcontrollers.
8051
PIC
AVR
SPEED
Slow
Moderate
Fast
MEMORY
Small
Large
Large
ARCHITECTURE
CISC
RISC
RISC
ADC
Not Present
Inbuilt
Inbuilt
Timers
Inbuilt
Inbuilt
Inbuilt
PWM Channels
Not Present
Inbuilt
Inbuilt
AVR is an 8-bit microcontroller belonging to the family of Reduced Instruction Set Computer (RISC). In RISC architecture the instruction set of the computer are not only fewer in number but also simpler and faster in operation. The other type of categorization is CISC (Complex Instruction Set Computers). Click to find out differences between RISC and CISC. We will explore more on this when we will learn about the architecture of AVR microcontrollers in following section.
Let’s see what all this means. What is 8-bit? This means that the microcontroller is capable of transmitting and receiving 8-bit data. The input/output registers available are of 8-bits. The AVR family controllers have register based architecture which means that both the operands for an operation are stored in a register and the result of the operation is also stored in a register. Following figure shows a simple example performing OR operation between two input registers and storing the value in Output Register.
Working of AVR Microcontroller

The CPU takes values from two input registers INPUT-1 and INPUT-2, performs the logical operation and stores the value into the OUTPUT register. All this happens in 1 execution cycle.

In our journey with the AVR we will be working on Atmega16 microcontroller, which is a 40-pin IC and belongs to the megaAVR category of AVR family. Some of the features of Atmega16 are:

·      16KB of Flash memory
·         1KB of SRAM
·         512 Bytes of EEPROM
·         Available in 40-Pin DIP
·         8-Channel 10-bit ADC
·         Two 8-bit Timers/Counters
·         One 16-bit Timer/Counter
·         4 PWM Channels
·         In System Programmer (ISP)
·         Serial USART
·         SPI Interface
·         Digital to Analog Comparator.
Architecture of AVR
The AVR microcontrollers are based on the advanced RISC architecture and consist of 32 x 8-bit general purpose working registers. Within one single clock cycle, AVR can take inputs from two general purpose registers and put them to ALU for carrying out the requested operation, and transfer back the result to an arbitrary register. The ALU can perform arithmetic as well as logical operations
over the inputs from the register or between the register and a constant. Single register operations like taking a complement can also be executed in ALU. We can see that AVR does not have any register like accumulator as in 8051 family of microcontrollers; the operations can be performed between any of the registers and can be stored in either of them.
AVR follows Harvard Architecture format in which the processor is equipped with separate memories and buses for Program and the Data information. Here while an instruction is being executed, the next instruction is pre-fetched from the program memory.
AVR Family Architecture
Since AVR can perform single cycle execution, it means that AVR can execute 1 million instructions per second if cycle frequency is 1MHz. The higher is the operating frequency of the controller, the higher will be its processing speed. We need to optimize the power consumption with processing speed and hence need to select the operating frequency accordingly.
There are two flavors for Atmega16 microcontroller:
1.      Atmega16:- Operating frequency range is 0 – 16 MHz.
2.      Atmega16L:- Operating frequency range is 0 – 8 MHz.
If we are using a crystal of 8 MHz = 8 x 106 Hertz = 8 Million cycles, then AVR can execute 8 million instructions.
Naming Convention.!
The AT refers to Atmel the manufacturer, Mega means that the microcontroller belong to MegaAVR category, 16 signifies the memory of the controller, which is 16KB.
Naming Convention of ATmega16 - AVR Family
Architecture Diagram: Atmega16
Following points explain the building blocks of Atmega16 architecture:
·           I/O Ports: Atmega16 has four (PORTA, PORTB, PORTC and PORTD) 8-bit input-output ports.
·           Internal Calibrated Oscillator: Atmega16 is equipped with an internal oscillator for driving its clock. By default Atmega16 is set to operate at internal calibrated oscillator of 1 MHz. The maximum frequency of internal oscillator is 8Mhz. Alternatively, ATmega16 can be operated using an external crystal oscillator with a maximum frequency of 16MHz. In this case you need to modify the fuse bits. (Fuse Bits will be explained in a separate tutorial).
Architecture of ATmega16 - AVR Microcontrollers | Building Blocks Architecture of ATmega16 – AVR Microcontrollers
·           ADC Interface: Atmega16 is equipped with an 8 channel ADC (Analog to Digital Converter) with a resolution of 10-bits. ADC reads the analog input for e.g., a sensor input and converts it into digital information which is understandable by the microcontroller.
·           Timers/Counters: Atmega16 consists of two 8-bit and one 16-bit timer/counter. Timers are useful for generating precision actions for e.g., creating time delays between two operations.
·           Watchdog Timer: Watchdog timer is present with internal oscillator. Watchdog timer continuously monitors and resets the controller if the code gets stuck at any execution action for more than a defined time interval.
·           Interrupts: Atmega16 consists of 21 interrupt sources out of which four are external. The remaining are internal interrupts which support the peripherals like USART, ADC, Timers etc.
·           USART: Universal Synchronous and Asynchronous Receiver and Transmitter interface is available for interfacing with external device capable of communicating serially (data transmission bit by bit).
 General Purpose Registers: Atmega16 is equipped with 32 general purpose registers which are coupled directly with the Arithmetic Logical Unit (ALU) of CPU.
·           Memory: Atmega16 consist of three different memory sections:
1.         Flash EEPROM: Flash EEPROM or simple flash memory is used to store the program dumped or burnt by the user on to the microcontroller. It can be easily erased electrically as a single unit. Flash memory is non-volatile i.e., it retains the program even if the power is cut-off. Atmega16 is available with 16KB of in system programmable Flash EEPROM.
2.        Byte Addressable EEPROM: This is also a nonvolatile memory used to store data like values of certain variables. Atmega16 has 512 bytes of EEPROM, this memory can be useful for storing the lock code if we are designing an application like electronic door lock.
3.        SRAM: Static Random Access Memory, this is the volatile memory of microcontroller i.e., data is lost as soon as power is turned off. Atmega16 is equipped with 1KB of internal SRAM. A small portion of SRAM is set aside for general purpose registers used by CPU and some for the peripheral subsystems of the microcontroller.
·           ISP: AVR family of controllers have In System Programmable Flash Memory which can be programmed without removing the IC from the circuit, ISP allows to reprogram the controller while it is in the application circuit.
·          SPI: Serial Peripheral Interface, SPI port is used for serial communication between two devices on a common clock source. The data transmission rate of SPI is more than that of USART.
·           TWI: Two Wire Interface (TWI) can be used to set up a network of devices, many devices can be connected over TWI interface forming a network, the devices can simultaneously transmit and receive and have their own unique address.
·           DAC: Atmega16 is also equipped with a Digital to Analog Converter (DAC) interface which can be used for reverse action performed by ADC. DAC can be used when there is a need of converting a digital signal to analog signal.
Various microcontroller of MegaAVR series:
ATmega8 and Atmega32 are other members of MegaAVR series controllers. They are quite similar to ATmega16 in architecture. Low power version MegaAVR controllers are also available in markets. The following table shows the comparison between different members of MegaAVR family:
Part Name
ROM
RAM
EEPROM
I/0 Pins
Timer
Interrupts
Operation Voltage
Operating frequency
Packaging
ATmega8
8KB
1KB
512B
23
3
19
4.5-5.5 V
0-16 MHz
28
ATmega8L
8KB
1KB
512B
23
3
19
2.7-5.5 V
0-8 MHz
28
ATmega16
16KB
1KB
512B
32
3
21
4.5-5.5 V
0-16 MHz
40
ATmega16L
16KB
1KB
512B
32
3
21
2.7-5.5 V
0-8 MHz
40
ATmega32
32KB
2KB
1KB
32
3
21
4.5-5.5 V
0-16 MHz
40
ATmega32L
32KB
2KB
1KB
32
3
21
2.7-5.5 V
0-8 MHz
40

BEGINNER PROJECT-4:SIMPLE AUTOMATIC PLANT WATERING CIRCUIT

The post explains a simple yet effective automatic plant irrigation system circuit which can be used for automatically sensing soil humidity and triggering a water pump when the ground gets parched below a predetermined level (adjustable).

https://celebratelife24x7.files.wordpress.com/2014/11/3871a-automaticplantwaterirrigationcircuit.png

The circuit is rather straightforward and uses a single IC 555 as the main active component.

Referring to the automatic plant irrigation circuit shown below we can see the IC 555 is wired in a completely unique and in the quickest possible mode.
Here it’s configured as a comparator, and works better than an opamp because the IC 555 has built in opamps which are at par with any single opamp and also the output of a 555 IC is able to sink sufficient current in order to drive a relay without a transistor driver stage.

The above features particularly makes the above design very simple, low cost and yet too effective with its functions.

The pin#2 here becomes the sensing pinout of the IC, and is held at ground level via R2 which must be calculated as per the desired soil humidity triggering threshold.

The points A and B can be seen fixed inside the soil which needs to be monitored for the intended automatic watering from the water pump.

As long as the points A and B senses some level of humidity corresponding to a resistance value which may be lower than R2, the IC 555 output is held low, which in turn keeps the relay deactivated.

However as the soil tends to get dryer, the resistance across the probes starts getting higher and at some moment of time it becomes higher than R2, creating a potential below 1/3rd supply voltage at pin#2 of IC555.

The above situation instantly prompts pin#3 of the IC to become high, triggering the connected relay.

The relay activation switches ON the water pump which now starts pumping water to the particular area of the soil via a distributing water channel.

As this happen, the soil gradually gets wetter and as soon as the predetermined level is reached, the probes immediately sense the lower resistance and revert the IC ouput pin#3 to a low again switching OFF the relay and the water pump consequently.

C1 ensures a slight hysteresis in the operations ensuring that the relay triggering is not sudden or abrupt, rather it switches only after sensing a genuine response from the soil conditions.

8051 PROJECT-1:HOME_MADE 8051 TRAINING BOARD

Every beginner of microcontrollers all ways think of buying a prototyping board for practicing.Why don’t you make your own board for working on your projects.I made a simple 8051 development board for prototyping my projects.You can make one by just spending less then 100 rupees.

For that you need very little equipment.

  • pin heads
  • 40 pin IC base(i didn’t get a 40 pin so i used a 42 pin)
  • a 7805 voltage regulator
  • crystal (11.0592Mhz or 12.0Mhz as per your requirement)
  • 10kohm resistor
  • 10kohm resistor 9 pin array
  • 10uF electrolytic capacitor
  • 33pF disc capacitors
  • some connecting wires
  • a special PCB that consists of all row holes will be connected

PROCEDURE

first take a PCB with all its row holes connected as shown in the figure

Cut the PCB as per the need of your size of board.First of all take a look at the pin diagram of AT89S52 microcontroller. The EA/VPP(pin 31) must be connected to VCC. The PORT0 must be pulled up through external pullups.So use the 9 pin 10K resistor array for pull up such that common pin to VCC and remaining 8 pins to PORT0 pins.

The power on reset circuit for 8051 will be as shown in the figure

Connection of external ceramic crystal to the AT89S51

arrange all the components as shown in the figure below. step by step

cut the PCB like this

place the 40 pin IC base(i used 42 pin as 40 pin is out of stock)

take some pin heads

place them as per the port pins

and place the remaining required components like crystal,reset circuit,7805 voltage regulator for supply

your home made prototyping board for AT89S51.
for programing the  MCU you need a separate ISP programmer.
so many of them are available in the market.
if you use P89V51RD2 microcontroller you just need a max232 IC and a serial cable to program it

ELECTRONICS 101-8051 MICROCONTROLLER

A microcontroller is an economical computer-on-a-chip built for dealing with specific tasks, such as displaying or receiving information through LEDs or remote controlled devices. The most commonly used set of microcontrollers belong to 8051 Family. 8051 Microcontrollers continue to remain a preferred choice for a vast community of hobbyists and professionals. Through 8051, the world became witness to the most revolutionary set of microcontrollers.

8051 Microcontroller, 8051 Family

8051 Family

Intel fabricated the original 8051 which is known as MCS-51. The other two members of the 8051 family are:
i.     8052 – This microcontroller has 256 bytes of RAM and 3 timers. In addition to the standard features of 8051, this microcontroller has an added 128 bytes of RAM and timer. It has 8K bytes of on chip program ROM. The programs written for projects using 8051 microcontroller can be used to run on the projects using 8052 microcontroller as 8051 is a subset of 8052.
ii.    8031 – This microcontroller has all the features of 8051 except for it to be ROM-less.  An external ROM that can be as large as 64 K bytes should be programmed and added to this chip for execution. The disadvantage of adding external ROM is that 2 ports (out of the 4 ports) are used. Hence, only 2 ports are left for I/O operations which can also be added externally if required for execution.
Comparison of 8051 family members:
Features 8051 8052 8031
RAM(bytes)
128
256
128
ROM
4K
8K
0K
Timers
2
3
2
Serial port
1
1
1
I/O pins
32
32
32
Interrupt sources
6
8
6
Table – 1.1 Comparison of 8051 family members
Various 8051 microcontrollers
8051 microcontrollers use two different kinds of memory such as UV- EPROM, Flash and NV-RAM. Hence 8051 will not be seen in the part number even though it is the most popular member of the 8051 family.
        i.            8751 – This microcontroller is the UV-EPROM version of 8051. This chip has only 4K bytes of UV-EPROM. It is required to have access to the PROM burner and the UV-EPROM eraser to erase the contents inside the chip before it is programmed again. The disadvantage of using this memory is the waiting time of around 20 minutes to erase the contents in order to program it again. Due to this limitation, manufacturers fabricated flash and NV-RAM versions of 8051.
      ii.            AT89C51 from Atmel Corporation – Atmel fabricated the flash ROM version of 8051 which is popularly known as AT89C51 (‘C’ in the part number indicates CMOS). The flash memory can erase the contents within seconds which is best for fast growth. Therefore, 8751 is replaced by AT89C51 to eradicate the waiting time required to erase the contents and hence expedite the development time. To build up a microcontroller based system using AT89C51, it is essential to have ROM burner that supports flash memory. Note that in Flash memory, entire contents must be erased to program it again. The contents are erased by the ROM burner. Atmel is working on a newer version of AT89C51 that can be programmed using the serial COM port of IBM PC in order to get rid of the ROM burner.
Part Number ROM RAM I/O pins Timer Interrupt Vcc Packaging
AT89C51
4K
128
32
2
6
5V
40
AT89C52
8K
256
32
3
8
5V
40
AT89C1051
1K
64
15
1
3
3V
20
AT89C2051
2K
128
32
3
8
3V
20
AT89LV51
4K
128
32
2
6
3V
40
AT89LV52
8K
128
32
3
8
3V
40
Table 1.2 – Versions of 8951 from Atmel (All ROM Flash)
There are different versions of packaging and various speed of the products mentioned in the above table.
Part Number Speed Pins Packaging Use
AT89C51-12PC
12MHz
40
DIP Plastic
Commercial
AT89C51-16PC
16MHz
40
DIP Plastic
Commercial
AT89C51-20PC
20MHz
40
DIP Plastic
Commercial
Table 1.3 – Various Speeds of 8051 from Atmel
Note: The part number AT89C51-16PC where AT – Atmel, C – CMOS (consumes less power), 12 – Speed as 12 MHz, P – Plastic DIP packaging, C – Commercial use. AT89C51-12PC is favourably used by students for their projects.
iii.   DS5000 from Dallas Semiconductor – Dallas Semiconductor fabricated the NV-RAM version of the 8051 which is known as DS5000. The PC serial port is utilized to load the program onto the in-built ROM. The advantage of NV-RAM memory is the facility to erase the contents one byte at a time.
      v.
Part Number RAM ROM Timers I/O pins Interrupts Vcc Packaging
DS5000-8
128
8K
2
32
6
5V
40
DS5000-32
128
32K
2
32
6
5V
40
DS5000T-8
128
8K
2
32
6
5V
40
DS5000T-8
128
32K
2
32
6
5V
40
Table 1.4 – Versions of 8051 from Dallas Semiconductor
Note: ‘T’ signifies the real time clock (RTC) which is different from the timer. Even though the power is turned off, the real – time clock generates and displays the date and the time of day.
The variety range of speed and packaging versions of DS5000 which is shown in the Table 1.5
Part Number NV-RAM Speed
DS5000-8-8
8K
8MHz
DS5000-8-12
8K
12MHz
DS5000-32-8
32K
8MHz
DS5000-32-12
32K
12MHz
DS5000T-8-12
8K
12MHz
DS5000T-32-8
32K
8MHz
Table 1.5 – Versions of 8051 Dallas Semiconductor
Note: The part number DS5000-8-8 where 8 – 8K NV-RAM, 8 – Speed of 8MHz. DS5000-8-12 is favourably used by students for their projects.
iv.    One – Time – Programmable (OTP) versions of the 8051 – This version of microcontroller is cheaper and available from various manufacturers. The manufacturers use OTP microcontroller for mass production because the price per unit is very cheap.

Features of 8051

The main features of 8051 microcontroller are:
        i.            RAM – 128 Bytes (Data memory)
      ii.            ROM – 4Kbytes (ROM signify the on – chip program space)
    iii.            Serial Port – Using UART makes it simpler to interface for serial communication.
    iv.            Two 16 bit Timer/ Counter
      v.            Input/output Pins – 4 Ports of 8 bits each on a single chip.
    vi.            6 Interrupt Sources
 vii.            8 – bit ALU (Arithmetic Logic Unit)
viii.            Harvard Memory Architecture – It has 16 bit Address bus (each of RAM and ROM) and 8 bit Data Bus.
    ix.            8051 can execute 1 million one-cycle instructions per second with a clock frequency of 12MHz.
This microcontroller is also called as “System on a chip” because it has all the features on a single chip. The Block Diagram of 8051 Microcontroller is as shown in Figure 1.1
8051 Block Diagram, 8051 Architecture
Figure 1.1 Block Diagram of 8051 Microcontroller
Memory Architecture
The 4 discrete types of memory in 8051 are:
        i.            Internal RAM – This memory is located from address 0 to 0xff.  The memory locations from 0x00 to 0x7F are accessed directly. The bytes from 0x20 to 0x2F are bit-addressable. Loading R0 and R1 the memory location from 0x80 to 0xFF can easily accessed.
      ii.            Special Function Registers (SFR) – Located from address 0x80 to 0xFF of the memory location. The same instructions used for lower half of Internal RAM can be used to access SFR’s. The SFR’s are bit addressable too.
    iii.            Program Memory – This is read only memory which is located at address 0. With the help of 16 bit Special Function Register DPTR, this memory can also save the tables of constants.
    iv.            External Data Memory – Located at address 0. The Instruction MOVX (Move External) should be used to access the external data memory.
Programming environment and programmer
Formerly, programmers used machine language for coding. A machine language is a program that consists of 0s and 1s which was very dreary for the humans to 8031, 8051 Microcontroller Pin Diagram/PinOut program any computer. In due course of time, assembly language was developed in order to speed up the programming and make it error-free.  Assembly language is a low level language which uses an assembler to translate the program into machine code. The high level programming languages such as BASIC, Pascal, Forth, C, C++, and Java are available to code the program for 8051. These high level languages make use of a Compiler to translate into machine code. For example, when a program is written in C, the program needs to be translated into machine language using C compiler. Usually, Assembly and C language is widely used for 8051 programs as compared to the other high level languages.
The 8051 provides a total of four ports for I/O operations. 8051 has 40 pins, of which 32 pins are set aside for the four ports. PO, P1, P2, and P3 each have 8 pins and can be used for either input or output. The remaining pins are designated as Vrt, GND, XTAL1, XTAL2, RST, EA, ALE/PROG and PSEN.
8051 allows you to manipulate one or all of the bits of a port, thus providing programmers with a unique and powerful feature. 8051 provides the programmer with the ability to read, write and modify each port to customize applications as much as possible.

ELECTRONICS 101-MICROCONTROLLER VS MICROPROCESSOR

MICROPROCESSOR:

Microprocessor

A microprocessor is an electronic computer component crafted from miniature sized transistors & some other circuitry elements on a solitary semi-conductor IC (integrated circuit) or micro chip. The abbreviated of microprocessor is µP or uP. The CPU (central processing unit) is the most renowned micro-processor, but numerous other constituents in a computer contain them, like- the GPU (Graphics Processing Unit) on a video card. In the race of personal computers, the names CPU & microprocessor are applied inter-changeably. At the Center of all PCs and at maximum of the workstations a microprocessor is incorporated. Microprocessors control the logic of approximately all digital machines, from radios clock to fuel injection structures for automobile. Microprocessor is a single IC package in which several functions are integrated.

There are mainly five sorts of microprocessors; these minute units provide computers with a “brain”. Inside a usual silicon micro-processor, there will be a number of minute transistors and tremendously tiny parts. All of these components are employed to assist a computer work as planned.

Types of Microprocessors:

Different Types of Microprocessors

  • Complex Instruction Set Microprocessors:This type of microprocessor is also known as CISM. CISM classify a micro-processor in which each & every order can be executed together with several other low-level functions. These Functions are intended to carry out actions such as- uploading data into memory card, re-calling or downloading data from memory card or a complex mathematics computation in a single command.
  • Reduced Instruction Set Microprocessors: Also known as RISC, this was intended to pace up computer microprocessors. These chips are built up under the guideline that permits the microprocessor to do a smaller amount of things within each command and this will permit it to complete more commands more rapidly.
  • Superscalar processors:This type of processor replica the hardware on the micro-processor so that it can perform numerous instructions at the same time. These replica resources can be committed arithmetic logic units or multipliers. Superscalars comprise of several operational units. Superscalar micro-processors carry out more than one command throughout a single clock cycle by concurrently transmitting numerous instructions to superfluous operational units in the processor.
  • The Application Specific Integrated Circuit:Also known as ASIC microprocessor is intended for extremely precise purposes, which possibly will comprise- automotive emissions control or Personal Digital Assistants computers. ASICs at times is produced to specification, but can also be manufactured by making use of off-the-shelf gears.
  • Digital Signal Multiprocessors (DSPs):DSPs are unique micro-processors employed to decode & encode video, or convert digital or video to analog and vice-versa. These operations need a micro-processor particularly excellent at carrying out mathematical calculations. DSP chips are generally employed in SONAR, mobile telephones, RADAR, home theater audio gears and cable set-top boxes.

MICROCONTROLLER:

Microcontroller

Microcontroller is a computer on‐a‐chip optimized to manage electric gadgets. It is intended particularly for precise tasks like controlling a particular system. A microcontroller sometimes uses abbreviated uC, µC, or MCU, is fundamentally a specialized variety of microprocessor that is intended to be self-satisfactory and lucrative. Also, a microcontroller is a fraction of a set in system, which is fundamentally a complete circuit board. A fixed-in system is a computer system intended to carry out one or more functions over and over again with real‐time work out limits. It is embedded as an element of a full machine often counting hardware and motorized elements as well. Some external electronic peripherals are interfaced to microcontroller for a better performance depends on applications.

Illustrations of microcontrollers are the 8051, Intel’s 80196, Microchip’s PIC and Motorola’s 68HCxx series. Microcontrollers which are normally incorporated in toys, automobiles, appliances and office machines are gears which amalgamate a number of constituents of a microprocessor system on a solo microchip:

  • Memory (both ROM and RAM)
  • The CPU core (microprocessor)
  • Some parallel digital I/O

The microcontroller witnesses the incorporation of a number of helpful functions into a solitary IC pack. These functions are:-

  • The capability to carry out an accumulated set of commands to perform user described jobs.
  • The capability to be proficient to use peripheral memory chips to both read & write data from and to the memory.

Types Of Microcontrollers:

The micro-controllers are sorted in terms of in-house bus width, in-built micro-controller, order set, memory structural design, IC chip or VLSI core or Verilog file & family unit. For the similar family, there may be a range of editions with different sources. Here we are giving few types of microcontroller used in different applications.

Different Types of Microcontrollers

  • 8-bit microcontroller:When in-house bus in a MCU is 8-bit bus then the ALU carries out the logic & arithmetic operations on a byte at an order. The MCU is 8-bit micro-controller. The illustrations of 8-bit MCU are- Intel 8031/8051, Motorola MC68HC11 & PIC1x families.
  • 16-bit microcontroller:The 16 bit Microcontroller consists of a 16 bit bus and the ALU performs arithmetic and logic operations on the 16 bit operand. It provides greater precision and performance as compared to 8 bit MCU.
  • 32-bit microcontroller:When in-house bus for the data transmitting function in an MCU is 32-bit bus then the ALU carries out logic & arithmetic functions on operand words of 32 bits at the orders. The MCU is 32-bit micro-controller. These supply better accuracy and performance in comparison to the 16-bit MCUs.
  • Embedded microcontroller:When a fixed or embedded system includes an MCU that has each hardware and software parts in a solo unit, the MCU is known as embedded micro-controller. Extremely few or no extra peripheral unit or system exists for processing during the control or making use of the peripheral devices. For illustration, a telephone receiver circuit employs an in-built or embedded micro-controller.
  • External memory micro-controller:When an in-built or embedded system adds an MCU that has each of the hardware and software parts present not as a solo component and has all or a piece of the memory component externally interfaced by bringing into play an interfacing circuit which is known as the glue circuit, the MCU is known as a peripheral or external memory micro-controller. For instance, 8031 comprise of a program memory which is interfaced outwardly to it. The 8051 has both in-house as well as peripheral program memory.

Difference Between Microprocessors and Microcontrollers

The dissimilarity amid the two is that a microcontroller integrates the features of a microprocessor (ALU, CPU, Registers) together with the existence of extra characteristics such as existence of ROM, RAM, counter, Input/Output ports, etc. Here a microcontroller controls the function of a device by employing fixed programs accumulated in ROM that does no amend with duration.

Difference between Microprocessor and Microcontroller

From one more point of view, the major dissimilarity amid a usual microprocessors and microcontrollers parting there architectural terms is the area of their application. Usual microprocessors such as the Pentium family or Intel Core family processors or alike processors are in computers as a universally functioning programmable machine. In its life span it has to manage numerous different assignments and programs specified to it.

In contrast a microcontroller of PIC family or 8051 family or any other have noticed their applications in tiny embedded systems such as- control system of traffic signals or some sort of robotic system. Also these gadgets manage similar task or similar program all through their entire life cycle. Another difference is that the microcontroller generally has to handle instantaneous tasks whereas on the contrary the micro-processors in a computer system perhaps will not handle an instantaneous task at all times.

Nowadays many engineering students are showing lot of interest towards the microprocessor and microcontroller projects as they are very interesting and helpful to build good career in electronics.