Commit Graph

274 Commits

Author SHA1 Message Date
Chaz Littlejohn
c518c482ac Fixed several errors in the new storeHudCache which were preventing the data from being imported in full 2011-04-04 06:32:57 +00:00
Chaz Littlejohn
f55cd5569d Added methods for storing the filename and import statistics in a new 'Files' table which links with the Hands table via Hands.fileId. Extended support for storing mixed game information for Full Tilt Poker coming from the filename, and updated mixed game dictionaries for PokerStars. Added a field in GameTypes to track this mixed game variable entitled 'mix' which takes on the string value of 'none' if not a mix game hand. 2011-04-02 23:56:52 +00:00
Worros
59d22bed5d Import: Remove unicode translation from addImportFile
A filename is an almost arbitrary binary sequence, assuming it can be encoded
in utf-8 caused crashes on certain file names.
2011-03-29 17:20:13 +08:00
Worros
ab5e28e73d Import: Add to_hud back to init list 2011-03-29 15:36:18 +08:00
Chaz Littlejohn
7c31a19077 Modified HHC so gtk & pygtk are imported to execute a callback only if import is run from the gui 2011-03-23 19:34:15 +00:00
Chaz Littlejohn
8c4d5d42b4 Added printtest back in and fixed a couple of SessionsCache bugs 2011-03-23 19:27:55 +00:00
Chaz Littlejohn
3ac088b748 Fixed a timezone issue in storeSessionsCache 2011-03-23 19:27:55 +00:00
Chaz Littlejohn
66e1cc3704 This commit includes a set of updates which:
* implement the new SessionsCache table
    - The SessionsCache table can be used to track overall or game sepecific sessions
    - The totalProfit field is summed by gameTypeId for cash games allowing for multiple currencies
    - Tournament profit (cashes - buy-ins) are also recorded in totalProfit and its grouped by tourneyId
* change the sequence and methodology surrounding the import of hands
    - fpdb_import.py implements a unique Hand.py method for each table
    - Hands SessionCache and HudCache records themselves are 'cached' to allow for 'bulk insert' at EOF
    - import is reorganized to allow for efficient locking in multiple connection environments
* changes the name of the index created by addTPlayersIndex (so that it is unique) to accommodate a bug in MySQL 5.5

TODO
* A 'rebuild_sessionsCache' method is still required
* Further commits are expected to fix bugs created during the porting of this code
2011-03-23 19:27:55 +00:00
Worros
867c547ca0 Revert "Modified hhc & fpdb_import so gtk & pygtk are imported only if import is run from the"
This reverts commit 64f46181fa.
2011-03-22 15:20:50 +08:00
Chaz Littlejohn
64f46181fa Modified hhc & fpdb_import so gtk & pygtk are imported only if import is run from the
gui
2011-03-22 03:22:38 +00:00
Steffen Schaumburg
43b071515d remove hharchivebase from code and test config 2011-03-19 20:15:53 +01:00
Steffen Schaumburg
8a519699b5 update copyright year 2011-03-10 06:16:31 +01:00
atinm
644b2d7849 replaced /usr/bin/python with /usr/bin/env python 2011-02-27 21:17:07 +08:00
Worros
e4a3de1272 Merge branch 'chaz' of git://github.com/ChazDazzle/fpdb-chaz 2011-02-17 11:28:44 +08:00
chaz@pokeit.co
ec477110cf fixed an import bug which prevented handsplayers and handsactions records from ever getting inserted 2011-01-18 03:53:35 -05:00
Worros
503c00e1b0 Remove unused minPrint option 2011-01-07 16:08:31 +08:00
Worros
f12bc7bfaa Add printdata capability for Gametype info. 2011-01-07 13:58:15 +08:00
chaz@pokeit.co
38e6e15a26 Modified import_file_dict() so that it stores HandsPlayers and HandsActions inserts in a temprorary list. Once the end of the handlist is reached, an executemany() bulk insert is done. 2011-01-01 15:40:06 -05:00
chaz@pokeit.co
990e226254 storeSessionsCache() now adds a sessionId to the Hands table 2011-01-01 03:35:14 -05:00
Worros
18ccea9a67 fpdb_import: Correct mainline print 2010-12-23 14:18:27 +08:00
Chaz
925db93157 * Added 'cacheSessions' flag
* Added a cacheSessions update call to import_file_dict() in fpdb_import.py
* Added updateSessionsCache() method to Hands
* Added storeSessionsCache() to Database --- WORK IN PROGRESS and set to pass
TO DOs
- add sessions queries to SQL
- complete storeSessionsCache() so it works like we've discussed (save for adding sessionId to HP)
- add SessionsCache keys
- add SessionsCache to the default table creation and deletion methods in Database.py
- update storeSessionsCache() and all the relevant db & sql code so it writes a sessionId to HP
2010-12-02 01:58:02 -05:00
Chaz
a7f3b1326f Added some logic so the importer won't try to send hands to HUD when HudCache is generated from the command line 2010-12-02 00:40:31 -05:00
Chaz
cbd6aa242e Added the 'cacheSessions' config option 2010-12-02 00:35:18 -05:00
Worros
9f64602338 Database, Importer: Convert print to log.info 2010-10-01 10:44:27 +08:00
Worros
591543d468 L10n updates: C->F 2010-09-23 00:10:32 +08:00
Worros
fbfaf0176c Import: First step to making FTP archive files parse
Change regex to be explicit about the number of '*'s

Pass the ftpArchive flag to HHC init
2010-09-17 10:58:47 +08:00
Worros
a7d450d384 Importer: Add command line option for FTP archive format.
To import

./GuiBulkImport -F -c "Full Tilt Poker" -f filename

Completely untested at the moment.
2010-09-15 13:23:32 +08:00
gimick
9e3df066f5 import: cli mode, don't display progress bar; tweak progress bar 2010-09-04 23:12:29 +01:00
gimick
c4203c87b1 import: fix error in progress bar 2010-09-04 13:08:58 +01:00
gimick
3fd2b2f838 Import: Show progress bar during bulk import 2010-09-04 12:30:43 +01:00
Worros
7aeb5e4531 Merge branch 'master' of git://git.assembla.com/fpdb-sql 2010-09-01 11:08:04 +08:00
sqlcoder
06b34e8648 clarify difference between site and filter to allow 2 sites from same network, prompt to add site to db if missing 2010-08-31 22:44:41 +01:00
steffen123
3990b40e3d Some string changes, mostly consistency to make translation easier 2010-08-29 20:35:16 +02:00
Worros
75e1dbbfba Importer; Add missing default option 2010-08-19 18:33:43 +08:00
Worros
0fab203a53 Importer: Add utility functions for regression testing
setPrintTestData: Indicate you would like to print test data when importing
setFakeCacheHHC: Indicate you want to access the HHC after an import run
getCachedHHC: Retrieve HHC

Also modified the main import loop to pass self.settings['testData'] to the database insert and records the HHC if requested
2010-08-19 18:25:26 +08:00
steffen123
7a5340f2d8 gettextify fpdb_import.py 2010-08-16 02:57:03 +02:00
Worros
53c796dddc Importer: Add excetion handler to hud call.
Had a report on the 2+2 thread that:

  File "C:\Documents and Settings\b\Desktop\fpdb\pyfpdb\GuiAutoImport.py", line 160, in do_import
    self.importer.runUpdated()
  File "C:\Documents and Settings\b\Desktop\fpdb\pyfpdb\fpdb_import.py", line 371, in runUpdated
    (stored, duplicates, partial, errors, ttime) = self.import_file_dict(self.database, file, self.filelist[file][0], self.filelist[file][1], None)
  File "C:\Documents and Settings\b\Desktop\fpdb\pyfpdb\fpdb_import.py", line 467, in import_file_dict
    print "fpdb_import: sending hand to hud", hand.dbid_hands, "pipe =", self.caller.pipe_to_hud
IOError: [Errno 9] Bad file descriptor

Was happening, which is a crash attempting to print self.caller.pipe_to_hud

This patch doesn't fix the problem, but should give some indication in the log that it happened.
2010-08-03 19:27:34 +08:00
sqlcoder
397e87b800 autoimport passes filenames to fpdb_import as unicode, so make bulk import do this as well so that add_import_file() always gets same type 2010-07-13 23:12:50 +01:00
steffen123
1d571cabee windows: by sqlcoder, fixes importing files with greek characters 2010-07-12 18:04:30 +02:00
steffen123
178b528d3b change calls to python2 back to python 2010-07-08 20:01:03 +02:00
steffen123
3b24fd18b2 some cleanups:
- updated copyright notices
- added copyright notices that were missing
- changed python to python2
- added gpl-2/3 license texts
- removed THANKS.txt file as the wiki version is more up to date
- removed empty fpdb_db.py
- added GPL2 to debian license file
2010-07-04 03:05:16 +02:00
steffen123
e8864bf241 change email address for bug reports from my personal one to mailing list 2010-07-02 23:48:01 +02:00
Worros
6ba7621f2a Fix 'errors' stat in importer
Instead of:
GuiBulkImport done: Stored: 32  Duplicates: 0   Partial: 0  Errors: 32 in 0.530081987381 seconds - 0/sec

We have:
GuiBulkImport done: Stored: 0   Duplicates: 0   Partial: 0  Errors: 32 in 0.530081987381 seconds - 0/sec
2010-04-23 00:28:30 +08:00
sqlcoder
141b88ecfd stop hudcache updating again when importing duplicate hands 2010-02-20 17:49:03 +00:00
sqlcoder
864454f31c change 2 commits to 1, may help sqlite stop locking up 2010-02-03 01:03:36 +00:00
sqlcoder
d4248706e4 remove surplus get_logger calls, tidy up log messages some more 2010-02-01 22:31:00 +00:00
sqlcoder
86330e536a tidy up logging so that log isn't created in /fpdb/log/ unless it is being used 2010-02-01 21:03:51 +00:00
sqlcoder
f7b402bcdb remove execute from permissions on some files in last commit 2010-01-31 11:27:37 +00:00
sqlcoder
5e9486aa3d config changes for logging: logs rotate and go to APPDATA, hud_config now goes in /fpdb/ if they must use it but logging.conf stays in /pyfpdb/. TODO: Some logging still into /fpdb/log/ 2010-01-31 11:25:24 +00:00
Eric Blade
3381527afc raise FpdbHandDuplicate error on duplicate insert attempt, catch it in import_file_dict, increment duplicates count 2010-01-28 05:56:17 -05:00