Python and The Very Slow Server

November 28, 2009 – 2:23 pm

I don’t usually do a lot of Python programming, but I always enjoy it when the opportunity arises. Python is in no way a “clean” language, it has all sorts of warts and limitations that mean that it tends to not get used for big projects. Despite this (or maybe because of it), Python remains my go-to language for Getting Small Things Done Quickly. It is impossible to overstate the utility of just being able to start coding a function by bashing away at the python console – nothing else has given me the same sense of instant gratification since I started programming in BASIC back in the 80s.

The other big advantage of Python is the useful utility libraries that come with it as standard. Want to send twenty thousand emails? Just import smtplib. Want to generate code based on data from a spreadsheet? Import csv and away you go. Need a file that is exactly 32Mb is size? No problem. These are real examples from my job where Python has saved me many hours.

The most recent use I have put Python to is a slow server. For various murky and uninteresting reasons I need a rate-limiting HTTP server, one that I can easily control the speed at which it sends data. Enter Python’s very handy BaseHTTPServer module, which allows you to create custom HTTP servers with only a few lines of code by subclassing a request handler. Although the BaseHTTPServer is fairly useless for serving real files, it is perfect for this type of thing since it does all the boring work of parsing headers and returning status codes.

I don’t care about the contents of the data, just its size and how long it takes to serve. Since I will be varying these parameters a lot, I decided to make them part of each request so that each request could take a different amount of time – this means I don’t have to restart the server between each test run. Modifying the code to serve actual file data would be very simple.

I enjoyed writing this server so much that I regret that it didn’t take longer. Now I actually have to use it for its intended purpose, which I can assure you is not going to be as pleasant.

Here is the complete Python source:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
# A very simple HTTP server designed to for testing situations where the data returned
# is not important but the rate at which it comes down is. This server can be started
# using the command: python delayserver.py
#
# Once started, it will listen for requests on port 8000
# Requests should be of the form http://<address>:8000/size=<bytes>,duration=<seconds>
# where: <bytes> is the size of the response data
# and    <seconds> is how long you want it to take (at minimum, it may take longer)
#
# Notes:
# * The timing is pretty inaccurate for small byte sizes, this isn't a problem for
#   what I need it for
# * Press ctrl-c to stop serving
 
 
import time
import BaseHTTPServer
 
class MyHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
 
	def do_GET(self):
		request = self.path.strip("/")
		duration = 1
		size = 1024
 
        	validRequest = False
		params = request.split(",")
		for p in params:
                    temp = p.partition("=")
                    if (temp[0] == "size"):
                        size = int(temp[2])
                        validRequest = True
                    elif (temp[0] == "duration"):
                        duration = int(temp[2])
                        validRequest = True
 
                if (validRequest == False):
                   self.send_error(404)
                   return
 
                self.send_response( 200 )
                self.send_header( "Content-Length", str(size) )
                self.send_header( "Pragma", "no-cache" )
                self.end_headers()
                self.slowWrite( self.wfile, size, duration )
 
 
        def slowWrite(self, output, size, duration):
                bytesWritten = 0
                startTime = time.time()
                while ( bytesWritten < size ):
                        now = time.time()
                        if (duration != 0):
                                desiredBytes = ( (now - startTime) / duration ) * size
                        else:
                                desiredBytes = size
                        desiredBytes = min( size, desiredBytes )
                        if (desiredBytes < bytesWritten ):
                                time.sleep(0.2)
                        else:
                                while (bytesWritten < desiredBytes):
                                        output.write('A')
                                        bytesWritten = bytesWritten + 1
                                output.flush()
                now = time.time()
                self.log_message( "Request took %f seconds",   now - startTime  )	
 
if __name__ == "__main__":
    http = BaseHTTPServer.HTTPServer( ('', 8000), MyHTTPRequestHandler )
    print "Listening on 8000 - press ctrl-c to stop"
    http.serve_forever()

I should point out that I am by no means an expert at Python, so take this code with a pinch of salt.

Post to Twitter Post to Facebook Post to Reddit Post to StumbleUpon

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

  1. One Response to “Python and The Very Slow Server”

  2. Python is great – the language seems pretty clean to me (and is a million times better than perl), and the libraries are great.

    You’re right about that ability to just execute some code, ala the BASIC interpreters we all remember – it’s a compelling feature that keep drawing you to the language.

    Nice server too – great tool for exercising your code with.

    By Edouard on Nov 29, 2009

Post a Comment

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Click to hear an audio file of the anti-spam word