benchmarks
==========
All measurements on my Athlon64@1.1GHz, 1 GB RAM, RAID5, Hardened Gentoo Linux 2.6.23
From svn47 all measurements on my Athlon64 X2@1GHz, 2GB RAM
All measurements were ran several times in a row, unless otherwise noted the computer was mostly idle
svn12/15 were with PostgreSQL, svn23 onwards with MySQL
All import measurements were taken immediately after reseting the tables

svn12, whilst compiling open office in the background
=====
reseting tables (incl filling default data): 0.5-0.6s
importing one hand from a file: 0.5-0.7s

svn15
=====
reseting tables (incl filling default data): 0.35-0.4s
importing one FTP hand from a file: 0.2-0.3s or 12-18k/hr
importing 51 FTP hands: 2.8-2.9s or 63-66k/hr
The large speedup will be partially due to not compiling in the background, and
partially due to less connecting/disconnecting the DB I'd imagine.
I'm VERY happy with this performance, but large gains could probably be made by
making the importer just a bit more sensible. 

svn23
=====
reseting tables (incl filling default data): 0.3-0.35s
importing 75 PS hands: 3.2s or 84k/hr
importing 180 FTP hands: 7.7-7.9s 82k/hr
This is the rewrite in import_FTP_and_PS.py, svn12/15 are with import_file.py
Nice, it's substantially faster inspite of supporting PS and FTP (ish) rather 
than just FTP and handling more special cases. 

svn24
=====
importing 75 PS hands: 3.2-3.3s or 82k/hr
importing 507 FTP hands: 23.5-23.8s or 77k/hr
importing 947 FTP hands: 76.2-72.7s or 44k/hr
As you can see FTP has gotten about 6% slower on 500 hands - but the much expanded handling
of new cases (particularly the ante folding will be ALOT of extra string
comparisons) easily explains that. In fact I expected a much bigger impact.
 (I suspect this is due to the parsing pausing whilst my shell catches up)

svn47
=====
It significantly slows down as the database gets larger, after a few thousand
hands it got to just one hand per second and slower. However, around 90% of the
CPU load was MySQL. Any tips are most welcome..

svn52 (whilst watching movie in a small window)
=====
Importing 2500 hands (mixed sites+types): 6m15s or 31k/hr (ran twice)
Of these 230 were skipped as duplicate or partial so counting 2270 hands

svn53 (whilst watching movie in a small window)
=====
Importing the 2270 hands from svn52: 1m50 or 74k/hr (82 if counting dupes/partial)
Importing my 35661 (plus 427 dupe/part) hands: 51m51s or 41k/hr (ran once)

The dbsize still has a significant impact, but with these sizes certainly it's
just not a problem. If anyone runs a larger database let me know how long it takes.
Thanks bwarycha for the tip with the foreign keys!


License
=======
Trademarks of third parties have been used under Fair Use or similar laws.

Copyright 2008 Steffen Jobbagy-Felso
Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License,
Version 1.2 as published by the Free Software Foundation; with
no Invariant Sections, no Front-Cover Texts, and with no Back-Cover
Texts. A copy of the license can be found in fdl-1.2.txt

The program itself is licensed under AGPLv3, see agpl-3.0.txt