08.28.08
Posted in python, linux at 4:05 pm by karl
I recently wrote some python scripts that pull information from a SQL Server 2005 database. The scripts are hosted on a redhat machine, so I installed freetds and pymssql to help wire things up. I then started python and typed:
>import _mssql
>mssql=_mssql.connect('123.123.123.123:1433', 'user', 'pass')
which gave me this rather cryptic error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
_mssql.error: DB-Lib error message 20017, severity 9:
Unexpected EOF from the server
Net-Lib error during Operation now in progress
Error 115 - Operation now in progressDB-Lib error message 20002, severity 9:
Adaptive Server connection failed
It turns out the problem is related to the TDS protocol version: for SQL Server 2005 it should be 8.0. To set this up, edit /usr/local/etc/freetds.conf and change it. The entry for my SQL Server machine looks like
[mssqlhost]
host = 123.123.123.123
port = 1433
tds version = 8.0
encryption = request
Once I made that change everything worked. Huzzah!
Permalink
08.18.08
Posted in Web authoring, python, linux at 10:29 am by karl
I recently had to install python2.5 and mod_python on some RedHat ES4 x86_64 machines. It turned out to be a little more complicated than I anticipated. First I installed python and everything seemed OK, but later on I discovered (while trying to install some python modules) that it wasn’t compiled against zlib correctly. To add further pain, having figured out the above I tried to compile mod_python and got the following error:
/usr/bin/ld:
/usr/lib/python2.4/config/libpython2.4.a(abstract.o):
relocation R_X86_64_32 against `a local symbol' can not be
used when making a shared object; recompile with -fPIC
/usr/lib/python2.4/config/libpython2.4.a: could not read
symbols: Bad value
collect2: ld returned 1 exit status
apxs:Error: Command failed with rc=65536
After some trawling the interwebs I found some answers on the python mailing list and on agile testing. Basically it comes down to how shared libraries are compiled - in certain circumstances “relocatable code” needs to be generated (presumably something to do with the larger address space on 64 bit architectures) and this isn’t done by default when you build python on a 64 bit system (the python mailing list entry above explains the rationale).
Here are the steps I used to make mod_python/python2.5 work on RedHat ES4 (x86_64):
1. install zlib1.2.3 (edit makefile with CFLAGS = "-fPIC")
> ./configure --shared
> make
> sudo make install This puts this version of zlib in /usr/local/lib (the original is in /usr/lib)
2. install python 2.5:
> ./configure --enable-shared
> CFLAGS="-fPIC" CXXFLAGS="-fPIC" LDFLAGS="/usr/local/lib" make
> su
# make install
# vi /etc/ld.so.conf.d/python2.5.conf
add text '/usr/local/bin' to file and :wq (see other files in that folder for examples)
# /sbin/ldconfig
# exit
now check if you've got 64 bit python:
> python
>>> import sys
>>> sys.maxint
9223372036854775807 # hurray! this is a 64 bit number
>>> quit()
make sure you're linking against 64 bit libs:
> ldd /usr/local/bin/python
The listed libs should all start with /lib64
3. make sure you've got apache2 and related libs
> up2date -i http http-devel
5. install mod_python 3.3.1:
>./configure --with-apxs=/usr/sbin/apxs --with-python=/usr/local/bin/python
> make
>sudo make install
Permalink
08.08.08
Posted in Web authoring, javascript, python at 10:06 am by karl
This caught me out this morning. Cheetah uses $var, ${var}, $(var) or $[var] as a placeholder for the variable var. Prototype uses $("id") as a turbo-charged replacement for document.getElementById("id") . This can cause problems with inline javascript in cheetah templates, as cheetah will try to substitute the $("elementId") in this code:
<script language="javascript" type="text/javascript">
var el = $("elementId")
</script>
The solution I’m using at the moment is to rewrite the above as:
<script language="javascript" type="text/javascript">
var el = $ ("elementId")
</script>
Note the space between the $ and the (”elementId”). Cheetah ignores $s that are followed by whitespace and javascript pretty much ignores all whitespace., so it seems to work.
Permalink
08.04.08
Posted in Web authoring, scripting, python at 3:12 pm by karl
I am beginning to understand why the mighty python is not more widely adopted for web development. I just tried to install mod_python on my MacBook Pro running OS X 10.5. I did the usual thing of downloading the source, untarring it and running
./configure --with-apsx=/usr/sbin/apsx
make
sudo make install
No complaints from any of these 3 commands. “Hurray”, I thought. I then edited /etc/apache2/httpd.conf to include the line
LoadModule mod_python libexec/apache2/mod_python.so
and restarted apache. No joy. Apache dead.
It turns out I had to edit the Makefile in the src/ directory after I ran configure. Isn’t that the whole point of configure? Bah! The solution is to be found on the Wired blog (you need to add -arch x86_64 in 2 places in the Makefile).
I had similar problems getting mod_python to run under RedHat ES 4 on a 64-bit machine - it looks like configure fails silently on 64 bot architectures. I promised myself I would blog the steps required to get everything running on RedHat but I haven’t done it yet. It would be lovely if someone in mod_python would take action over this stuff, as it’s a great shame that an otherwise excellent module falls over at the first hurdle. Hmmm, maybe I should do it instead of just moaning (alas I don’t think I have the l33t skillz required…)
EDIT: Graham Dumpleton pointed out that the problems with RedHat were not in fact to do with mod_python, but rather that a shared library 64-bit version of python needs to be installed. He is of course right. Thanks Graham!
Permalink
07.07.08
Posted in Web authoring, scripting, javascript at 6:15 pm by karl
I was playing with my computer last night and wrote a super-simple version of Conway’s Game of Life in javascript. Here’s how it looks.
Use the step button to advance a single generation, or use start/stop to run it as an animation. There are 2 classes in use here: life and lifeViewer. The life class does all the computing relating to Conway’s game, and lifeViewer displays it on the screen (using prototype for DOM manipulation and events).
Permalink
06.17.08
Posted in Web authoring, scripting, python at 3:16 pm by karl
As part of the exciting task of refactoring the ticket-text web codebase we decided to go the mod_python/cheetah templates route (previously we had a JSP based solution). So far I’m liking it, if for no other reason than it makes the code easy to hold in your head all at once. One of the first things I had to write was a dispatcher, i.e. a module that reads the incoming URI and decides which module to load and which function to call in that module to handle the request. Given a URI like this:
http://www.mydomain.com/foo/bar/baz/bip
the dispatcher will load the module foo and call the function bar with arguments baz and bip. In the ticket-text case, there are a few exceptions designed to make the resulting URLs even shorter and more intuitive for the end user. Anyway, dispatcher.py comes to 65 lines, of which 11 are whitespace and 18 are comments. That’s 36 lines of code. Having worked with the Zend Framework (PHP) and MVC.NET (C#) in the past, I am amazed at how simple and powerful the pythonic approach is. How did I ever manage before?
Permalink
05.27.08
Posted in scripting, python at 3:07 pm by karl
I recently had to produce a list of password-style codes for e-xamit. They had to be unique and they could not clash with codes that were already in use. I previously used a shell script (that called mkpasswd to generate the codes, remove duplicates and then grep though a list of in-use codes to find any that were already in use. Having recently started using python, I decided to reimplement this (mostly because the shell scripts were on my old computer and I was too lazy to boot it up and find them)…
from random import choice
existingCodes = set(['poipoi', 'blinky']) # in real life we read these from a file
# no 'l' or 'q' as they can be confusing when printed
letters = ['a','b','c','d','e','f','g','h','i','j','k',
'm','n','o','p','r','s','t','u','v','w','x','y','z']
codes = {} # we use a dict since then we get unique keys without writing any code. Yay!
for i in range(500): # we're making 500 codes here
tmp = ""
for j in range(6): # each code has 6 letters
tmp = tmp + choice(letters)
codes[tmp] = 0 # we're only interested in the key, not the value
codeList = set(codes.keys())
duplicates = codeList & existingCodes
print codeList
print "CODES ALREADY IN USE: "
print duplicates
OK so it’s not very exciting. But there are 2 beautiful things in this little script:
- if we store the generated codes as keys in a dictionary, we will never have any duplicates. For free! No coding! And more importantly, no unit testing of custom duplicate removal code!
- the set datatype means we can find the common elements in 2 lists of codes using the & operator. That’s it - no nested
for loops. Yay!
Am I glad I didn’t bother spending too much time with PowerShell!
Permalink
Comments off
03.31.08
Posted in Web authoring, scripting at 7:26 pm by karl
Here are some principles to follow that will reduce the load on your web server and make web pages appear on screen faster:
- make as few HTTP requests as possible
- concatenate all your CSS into a single file.
- concatenate all your JavaScript into a single file.
- put all your icons into a CSS sprite.
- the filesystem is your friend
- if you have the choice between storing information in the database or on the filesystem, use the filesystem if possible.
- make your URLs predictable
- this is related to (1) and (2). For example, if you have an image with a thumbnail, make the thumbnail URL derive from the image URL. You could, for generality, give the thumbnail an arbitrary URL and have a call to the server to find out what the thumbnail for a given image is called. But why would you want to do that? Much better to put images in (for example) an
images/ folder and thumbs in a thumbs/ folder (or whatever). That way you can avoid unnecessary calls to the server to find out things you should already know. If this seems like a straw-man argument, consider the difference between finding the thumbnail for a youtube video (URL convention) and finding the thumbnail for a google video (pull out an RSS feed and parse the XML to get the URL. Pain.).
- avoid talking to the database
- database calls are slow. Making lots of database calls will make you unpopular with your site visitors. Making lots of database calls on shared hosting will make you unpopular with your hosting provider. So…
- use RESTful services with HTTP GET and leverage the caching built into the browser
- print content to the file system if you can
- use
memcached to store frequently accessed data in memory
- don’t listen to me, ask the experts!
Permalink
02.19.08
Posted in Web authoring, javascript at 4:19 pm by karl
I recently did some work on the Leitrim Design House web site. I used the Zend Framework for the skeleton, YUI for the javascript bits and bolted WordPress into it to allow the nice people at the LDH to manage the content. I mostly developed it with Aptana and vi (more on that later), so I got lots of mileage out of free tools.
In hindsight, I think I should have used prototype instead of YUI (as I mostly ended up writing all the JS from scratch - I only used YUI to get at the DOM really). I had a really frustrating time editing my javascript with Aptana though. On my iBook G4 it kept freezing and crashing, mostly when it was trying to autocomplete my variable names. After a while I just gave up and finished off the JS stuff in vi.
I was impressed with the Zend Framework, especially after a long while writing fairly amorphous web apps in PHP. Forcing the MVC approach is a great way to put structure on your application. In fact, that would be my major criticism of ASP.NET - in trying to be all things to everyone it means the programmer has to be quite disciplined, which doesn’t always come naturally. Not to me at least…
Permalink
02.01.08
Posted in Web authoring, javascript at 11:28 pm by karl
When writing javascript code to run in a browser, it pays to use a library to pave over some of the cross browser issues that make your life difficult.
I have used
- YUI, which is designed in a way that makes your javascript look like java or C#. That is, readable and fairly maintainable but pretty long-winded. Readability is a big plus for me (bad memory), but long-windedness ultimately means more waiting while scripts download. Your users won’t like that.
- jQuery, which in my hands at least turns javascript into perl. This is the opposite of YUI, in that it produces dense code that is hard to understand. Dense code is good for your users, but bad for your sanity when you’re trying to debug it.
- prototype, which seems to turn javascript into Ruby. Kind of. It behaves a bit like jQuery (other way round, actually - jQuery is based on prototype), but less so. There are nice shortcut functions like
$ and $$, but I don’t feel it forces you to write complex DOM queries.
Right now I like prototype because it’s small and simple and makes lots of routine stuff (finding DOM elements and dynamically creating new ones) easy. Maybe as I get better at grokking DOM selectors I’ll start using jQuery more.
Permalink
« Previous entries