evilny0/fpdb
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
5,298 Commits 1 Branch 0 Tags 14 MiB
398a2609ef
Commit Graph

274 Commits

Author SHA1 Message Date
Worros
ff2cca361c New insert player function for HHC import 
Added getSqlPlayerIDs to Database.py - returns a hash {playername:sqlid}

Function uses a caching hash in Database.py to reduce the number of round trips to the database needed just to fetch the player ids for later inserts.

Need to do a performance comparison on a larger import.
 2009-08-02 12:19:33 +08:00
Worros
40a0fe428b Variable name change - hhc is better imho 2009-08-02 11:03:35 +08:00
sqlcoder
fbab3c7e24 make sure handsid is always returned and show size of queue when threaded 2009-08-01 00:07:31 +01:00
sqlcoder
f3a4a8b0d0 fix previous merge 2009-07-31 22:30:41 +01:00
sqlcoder
8d77b961d7 merge from eric @6df03cb... 2009-07-31 22:07:27 +01:00
sqlcoder
36f09ebc82 merge from ray @f5ba0b5b7... 2009-07-31 21:58:39 +01:00
sqlcoder
b503626c2b add db writer threading 2009-07-31 21:24:21 +01:00
eblade
6df03cb234 HUD_main: if new hand id int is not available, we can't print it, so that generates a second error, double fault 
fpdb_import: ok, we're keeping two lists now, one with updated st_size and one with m_time.  grrr.
 2009-07-31 01:15:28 -04:00
eblade
f5ba0b5b7f Database: insert missing HandToWrite.payin_amounts var 
EverleafToFpdb: remove DEBUG XXXXXXX line
FulltiltToFpdb: pass on converting tournament hands
Hand: add Hand.fee var that Database is looking for
fpdb_import: clean up file update scan code, queue files that have changed in the last 60 seconds for the next importer pass, instead of locking it up reading them now (not that this does anything since FT no longer updates the mtime until it closes the table, but maybe it works with Stars or others)
 2009-07-30 21:50:08 -04:00
eblade
3ce0f91b27 useless change of a comment just to test my rss output 2009-07-30 19:53:44 -04:00
eblade
2c7e00f02a fpdb_import: use stat_info.st_size instead of stat_info.st_mtime to determine hh file updates 2009-07-30 17:29:57 -04:00
sqlcoder
38ee98e492 pass sql list from fpdb through guibulkimport to fpdb_import 2009-07-29 06:37:06 +01:00
sqlcoder
298de5dc15 move most stuff from fpdb_db.py to Database.py and simplify db connections in fpdb_import 2009-07-28 23:58:10 +01:00
sqlcoder
12e1da0a5e add lock_for_insert routine but comment out for now (hangs) 2009-07-28 21:19:31 +01:00
sqlcoder
859d18734c minor corrections to output messages 2009-07-25 17:39:06 +01:00
sqlcoder
f69281e2fd finish hudcache rebuild code which speeds up bulk imports nicely - turn off permanently via allow_hudcache_rebuild in fpdb_import.py. Also some more moves into Database.py and cosmetic stuff 2009-07-21 22:26:23 +01:00
Worros
d546557ef7 Make new db insert code "usable" 
Next step, make the players insert. Function doesn't exist at the moment and crashes when NEWIMPROT set to true
 2009-07-19 15:45:18 +08:00
Worros
34932c14ee Add index parameter to Plugins 
Change constructor to HHC to accept an index arg, so that file reads can seek
 2009-07-17 17:45:22 +08:00
Worros
61b0b8f4fe Prep work for inserting directly from HHC 2009-07-17 17:16:06 +08:00
Matt Turnbull
79ca83c95a Merge from ray, minor conflicts 2009-07-15 00:20:26 +01:00
Matt Turnbull
dc6a1c45f9 sqlite - connecting and (re)creating some tables 2009-07-14 00:04:10 +01:00
sqlcoder
a2db28ebf2 create Database instance in import to pass into parser - will eventually replace fpdb_db 2009-06-25 23:14:32 +01:00
Worros
22ff98d7c9 Remove passthrough section from import_file_dict 2009-06-20 14:02:03 +08:00
Worros
f1921bebde Move functions from fpdb_save_to_db into Database.py 
Doesn't actually do much cleanup.

Intend on moving all db functions into Database.py and fpdb_db.py and
providing and API, the functionality of fpdb_simple et all to shift into
the HHC
 2009-06-20 04:15:52 +08:00
sqlcoder
fdf4b65136 pass all settings through so that a new one can be used to skip updating hudcache (not used yet) 2009-06-20 03:21:24 +08:00
Worros
ed9f02b113 Revert "pass all settings through so that a new one can be used to skip updating hudcache (not used yet)" 
This reverts commit b935a5ad67.
 2009-06-10 20:48:33 +08:00
sqlcoder
b935a5ad67 pass all settings through so that a new one can be used to skip updating hudcache (not used yet) 2009-06-09 22:55:18 +01:00
sqlcoder
680e5c2f89 use new more flexible function to decide whether to drop indexes 2009-06-09 22:22:10 +01:00
Worros
c3d2c5a2be Merge branch 'master' of git://git.assembla.com/free_poker_tools 
Conflicts:

	pyfpdb/Database.py
	pyfpdb/SQL.py
	pyfpdb/fpdb_db.py
	pyfpdb/fpdb_save_to_db.py

Also fix datetime issue with Everleaf
 2009-06-09 22:48:48 +08:00
sqlcoder
0a7c340656 Added timing info for index drop/create. Some small mods to output during import (obviously just change any you don't like) 2009-06-07 22:21:58 +01:00
sqlcoder
e662279acd remove parameter to analyzeDB() (error in previous release) 2009-06-07 20:49:35 +01:00
sqlcoder
7e8b80948c move index etc functions from fpdb_simple.py into fpdb_db.py 2009-06-07 20:45:09 +01:00
Ray
109727e764 Postgres changes for unicode. 2009-06-06 11:17:49 -04:00
sqlcoder
d44b093144 put debug message back in, this point should not be reached 2009-05-28 22:40:58 +01:00
sqlcoder
d41a8beacd add self. to variable reference and comment out debug message 2009-05-27 22:21:22 +01:00
sqlcoder
828b23510f add rollback to release locks (we should create a set transaction read_only method in fpdb_db) 2009-05-21 11:23:50 +01:00
sqlcoder
326f29b5b6 always release locks when db work finished 2009-04-27 10:47:22 +08:00
Worros
ae2835cd6b GuiBulkImport cli breakage fix 2009-03-28 22:24:40 +09:00
Worros
f6ca8c1bc1 GuiBulkImport cli breakage fix 2009-03-28 19:38:37 +09:00
Worros
710893b186 Fix merge breakage 2009-03-28 00:54:30 +09:00
Worros
0b3b7f66d6 Merge branch 'master' into siteneutral 
Conflicts:

	pyfpdb/fpdb_import.py
	pyfpdb/fpdb_parse_logic.py
	pyfpdb/fpdb_simple.py
 2009-03-28 00:54:13 +09:00
eblade
465f27af4d mostly spacing cleanup 2009-03-26 18:55:16 -04:00
Worros
497354d4f4 Make BulkImport work only with HHCs 
Added new section to config for converters for ease of creating the gui.

Removed the section in the site line for filter and supported games
 2009-03-24 22:58:45 +09:00
Worros
c673897fa3 Merge branch 'master' into siteneutral 
Conflicts:

	pyfpdb/fpdb_import.py
	pyfpdb/fpdb_save_to_db.py
	pyfpdb/fpdb_simple.py
 2009-03-24 21:46:18 +09:00
eblade
65e2087062 fix fix 2009-03-24 00:30:27 -04:00
Matt Turnbull
3f879ddb3d psql drop fk hang: seemed to be caused by having more than one 
connection to the db open. Made GuiBulkImport use the importer's
connection instead of its own. It still has the one given to it by
fpdb.py. I really don't know. Perhaps it needed a commit.
 2009-03-21 21:38:32 +00:00
Worros
495c39ef69 Merge branch 'master' into siteneutral 
Conflicts:

	pyfpdb/fpdb_import.py
	pyfpdb/fpdb_save_to_db.py
 2009-03-22 01:41:50 +09:00
Worros
1e788621a5 Merge branch 'master' of git://git.assembla.com/mctfpdb 
Conflicts:

	pyfpdb/FulltiltToFpdb.py
 2009-03-22 01:35:03 +09:00
Matt Turnbull
239330ae7b filterCrap after checking if hand is in db 
refactor GuiBulkImport : add main() so that it can be called from
interactive shell (for profiling purposes).
 2009-03-21 15:34:23 +00:00
Matt Turnbull
451a9b3ab1 GuiBulkImport: import directories from cmdline. Basically now does what 
CliFpdb should do, but reads most settings from xml config.
 2009-03-21 14:27:49 +00:00
Worros
ae4704b8dc First working(?) version of siteneutral import 2009-03-21 21:23:51 +09:00
Worros
444e780f29 Merge branch 'master' into siteneutral 
Conflicts:

	pyfpdb/fpdb_import.py
	pyfpdb/fpdb_parse_logic.py
	pyfpdb/fpdb_simple.py
 2009-03-20 00:27:08 +09:00
eblade
8b933bbd7b fpdb_import: formatting cleanup 
fpdb_simple: significant performance enhancements likely in checkPositions, convertCardValuesBoard, filterCrap, float2int, isActionLine, isWinLine
 2009-03-18 21:22:04 -04:00
eblade
a93718f953 Merge branch 'master' of git://git.assembla.com/free_poker_tools.git 2009-03-16 21:00:56 -04:00
Ray
6d292c50d9 Use saveActions and fastStoreHudCache in fpdb_import, etc. 2009-03-15 17:40:01 -04:00
eblade
6eb489c3a5 Merge branch 'master' of git://git.assembla.com/fpdboz.git 
Conflicts:
	pyfpdb/Hud.py
 2009-03-14 03:56:43 -04:00
Worros
3683a82e0f Merge branch 'master' into siteneutral 2009-03-14 05:32:16 +09:00
eblade
d35184cc79 Merge branch 'master' of git://git.assembla.com/fpdboz.git 
Conflicts:
	pyfpdb/fpdb_import.py
 2009-03-13 16:10:44 -04:00
Worros
bbdbe7e47d Remove TODO for previous patch 2009-03-14 05:10:13 +09:00
Worros
6ebab06981 Make converters dynamically loadable in autoimport 
Adds 1 restriction to the HHC sub classes, the classname must be the
same as the filename - ToFpdb.py

FulltiltToFpdb.py must contain a hhc class named Fulltilt
EverleafToFpdb.py must contain a hhc class named Everleaf
 2009-03-14 05:00:12 +09:00
eblade
a5b9ea860d fix runUpdated() to not freak out if a file disappears on it 2009-03-13 06:21:48 -04:00
Worros
3aaf5b9716 Merge branch 'killftp' into siteneutral 
Conflicts:

	pyfpdb/fpdb_simple.py
 2009-03-13 11:09:19 +09:00
eblade
104ca5fd9e Merge branch 'master' of git://git.assembla.com/fpdboz.git 
Conflicts:
	pyfpdb/fpdb_import.py
 2009-03-12 13:27:06 -04:00
eblade
9caa8b9361 Merge branch 'master' of git://git.assembla.com/mctfpdb.git 
Conflicts:
	pyfpdb/fpdb_import.py
 2009-03-12 13:23:32 -04:00
Worros
d8a689b9fe Quick fix 2009-03-13 02:18:34 +09:00
Worros
2d849c69e7 Refactor HHC call a bit, make sure out_fh gets closed 2009-03-13 00:31:29 +09:00
Worros
c523fce123 Merge branch 'master' of git://git.assembla.com/mctfpdb 
Conflicts:

	pyfpdb/fpdb_import.py
 2009-03-12 22:57:43 +09:00
Matt Turnbull
76e467a5d4 get rid of thread 2009-03-12 12:24:23 +00:00
eblade
02c5cfb39e remove call to hhc.processFile as someone removed that function 2009-03-12 01:18:31 -04:00
Matt Turnbull
ff675fb3ae import working? 2009-03-12 03:15:34 +00:00
eblade
9d7395a61c unbreak psuedo-recursion 2009-03-11 23:10:31 -04:00
Worros
264b188633 [KILLFTP 5/x] Remove references to the named site "PokerStars" or 
"Full Tilt Poker"
 2009-03-10 23:03:02 +09:00
Worros
161485e44e [KILLFTP 4/x] paper bag fix caused iun patch 1. 
This code needs replacing
 2009-03-10 22:09:32 +09:00
Worros
14b58d6ad5 [KILLFTP 3/x] Fix screwup from patch 1 2009-03-10 22:03:19 +09:00
Worros
72dff55f6f [KILLFTP 1/x] Tear out FTP specific hand import code 2009-03-10 22:03:18 +09:00
Worros
d6b5f5d019 Merge branch 'master' of git://git.assembla.com/fpdb-eric 
Conflicts:

	pyfpdb/Configuration.py
 2009-03-09 19:22:54 +09:00
eblade
635d59b25d use some lambda map/filter calls where they make sense instead of for loops 
localize some variables in the HUD update function
 2009-03-08 16:28:09 -04:00
Matt Turnbull
ca6f6e513b message 2009-03-06 18:10:04 +00:00
Matt Turnbull
75dcf002f9 Configuration: _getframe(0) so that GuiAutoImport can run standalone 
(stack not deep enough to go one deeper)
fpdb_import: Everleaf plumbing back in place.
GuiAutoImport.py: run standalone.
 2009-03-05 01:12:15 +00:00
Matt Turnbull
5c26bb028d Merge branch 'master' into monkeyfutz 
Everleaf converts hands
Moved process hand logic into the hand subclass that knows what it
wants, it calls back to the hhc (which knows about the format) to populate itself

Conflicts:

	pyfpdb/EverleafToFpdb.py
	pyfpdb/Hand.py
	pyfpdb/HandHistoryConverter.py
 2009-03-01 23:22:47 +00:00
Matt Turnbull
da602b2908 temp 2009-02-26 15:36:23 +00:00
Worros
7f1fd2ca38 Merge branch 'master' of git://git.assembla.com/fpdb-eric 2009-02-26 23:32:41 +09:00
eblade
c7541d9f14 return appropriate tuple from import_fpdb_file on file error 2009-02-25 23:17:36 -05:00
eblade
560cb1a543 fpdb_import runUpdated() will now remove files that were found to be newly missing on it's most recent pass from it's list of files to check 2009-02-25 22:44:03 -05:00
eblade
3c9f5537ea import should stop erroring out and crashing if a file it wanted to read has been deleted 2009-02-25 12:40:39 -05:00
Matt Turnbull
d5b7b17b21 bugfix 2009-02-25 15:16:18 +00:00
Matt Turnbull
e5489056ff less broken, please check sanity 2009-02-25 14:59:11 +00:00
Worros
a8a870548e Merge branch 'master' of git://git.assembla.com/free_poker_tools 2009-02-25 02:08:06 +09:00
Ray
c535dc7f24 Cleaner intermediate print during autoimport. 2009-02-24 10:54:02 -05:00
Worros
bca9fb15cf Note in code re: os.walk 2009-02-24 22:46:05 +09:00
Worros
3febe073cc Make FullTilt wrk with new importer 2009-02-22 12:42:16 +09:00
Matt Turnbull
483e177346 Explicit matching of player names 
About the only place where we are guaranteed to get all the players'
names correctly is in the seating plan in the hand header. We extract
the names and build a regex that matches only the names. We only do this
if the current regex is out of date, to avoid recompiling them too
often.
I also did away with self.rexx because it seemed unnecessary and was
difficult to work with.
 2009-02-20 16:29:52 +00:00
Worros
e5ab933659 Fix major bug in fpdb_import 
Hat tip to Eric
 2009-02-20 16:06:22 +09:00
Worros
98ea23cd49 Couple of additional fixes for import 2009-02-19 22:10:31 +09:00
eblade
d2159b6430 changes to make auto-import recursive, and to get the converter thingee kind of working 2009-02-19 05:38:51 -05:00
Worros
dcee1c6d28 Merge branch 'master' of git://git.assembla.com/free_poker_tools 
Conflicts:

	pyfpdb/fpdb_import.py
 2009-02-11 16:40:33 +09:00
Ray
18140defd8 Update bulk import, optionally drop indexes, improve gui 2009-02-07 10:06:48 -05:00
Worros
6214131713 Small change - note we should error check 2009-02-05 18:50:50 +09:00
Worros
315d010311 Make Everleaf converter actually function. 
Makes HandHistoryConverter actually write out a file to
$hhArchiveBase/sitename/

Adds code in importer call the functions in EverleafToFpdb
 2009-02-05 18:28:18 +09:00
eblade
e17c2eb64e fpdb_import: cleanup, and fix for FT bug that puts multiple seats on one line of summary 2009-01-28 03:46:43 -05:00
Worros
b2c135ee12 Possible fix for problem Eric is reporting in importer 2009-01-19 11:32:34 +09:00
Worros
ffb037b1fe Added output totals for bulk import 2009-01-17 02:24:00 +09:00
Worros
a3ff477395 Added comment/debug to damaged FTP file 
Doesn't fix issue, but documents what the bug is in the FTP software,
and prints the file name and line number that it occurs in so user can
fix.

Ideally the parser itself would be able to deal with this.
 2009-01-17 00:24:01 +09:00
eblade
ff3d6c75b8 Merge branch 'master' of git://git.assembla.com/mctfpdb.git 2009-01-07 12:36:09 -05:00
sqlcoder
257dd35c93 let fpdb_db routine work out which bits of config to use for connection 2009-01-03 17:59:48 +00:00
eblade
685abee8b1 re-indent function 2008-12-18 17:39:43 -05:00
eblade
2ccd00d3fd redo changes to checkPositions, and such 2008-12-18 15:42:46 -05:00
sqlcoder
d4e03424bf call routines to drop and recreate some indexes and foreign 
keys before and after bulk import
 2008-12-14 02:30:19 +00:00
sqlcoder
3148d0398c pulled from fpdboz 2008-12-13 03:35:31 +00:00
Worros
0e2f6bf4ab Fix default value when run from RegressionTest 2008-12-12 20:25:26 +09:00
sqlcoder
b62d1fb2a9 remove connection code from fpdb_import.py and use fpdb_db.py instead 2008-12-10 23:22:47 +00:00
sqlcoder
71a587faca mods to use insert_id() instead of repeated selects and get postgres working 2008-12-09 00:29:39 +00:00
sqlcoder
4dc15bfd94 pulled and merged from fpdboz 2008-12-07 23:38:33 +00:00
Mika Bostrom
90e348946f First step at fixing the bulk importer with postgres 2008-12-06 21:59:58 +02:00
Mika Bostrom
415eb4a3c6 Retab/whitespace demangly fpdb_import.py 2008-12-06 21:50:40 +02:00
sqlcoder
a824814c0a sqlcoder initial updates 2008-12-02 00:15:50 +00:00
eblade
1de34462e3 remove spurious window realize from popup_window as well 
printEmailErrorMessage also writes a traceback to the stderr, so we can more easily figure out -what part- it failed in
parsePositions will ignore a small blind if it appears to be coming from the same place as the big (does this mess with dead blinds? who knows)
 2008-11-27 08:35:17 -05:00
eblade
274cecdc58 stat windows no longer call realize() as that is done automatically by gtk when needed 
importer dumps bad hands to hand-errors.txt after printing the normal email message
parse_logic should ignore lines where small blind is found to be "$0", however, it's still causing errors
 2008-11-27 05:12:07 -05:00
Worros
6de1ac1c82 Merge branch 'master' of git://git.assembla.com/fpdb-eric 2008-11-10 22:55:24 +10:00
eblade
e713a9c654 fix weird error on windows? possibly just with p4e? trapping if we read 0 lines from history file 2008-11-10 03:46:37 -05:00
eblade
3325538c66 Merge branch 'master' of git://git.assembla.com/fpdb.git 
Conflicts:
	docs/default.conf
 2008-11-10 03:24:05 -05:00
eblade
f767ec2207 windows git is buggy 2008-11-10 03:22:21 -05:00
Worros
397cb9dd29 Merge branch 'master' of git://git.assembla.com/fpdb 2008-11-10 16:41:47 +10:00
steffen123
49151a92bf p142 - commented tourney summary parsing as it fails on me. this is alpha9 2008-11-10 02:02:12 +00:00
Worros
d4038c3f19 Lots of changes. Fixes the hud for auto-import - at least for me. 
Does configer Importer to use the Config class.
 2008-11-09 21:57:58 +10:00
Worros
bfa5137889 Merge branch 'master' of git://git.assembla.com/fpdb 2008-11-08 13:04:04 +10:00
Worros
8a2a851abc Merge branch 'master' of git://git.assembla.com/free_poker_tools 2008-11-08 13:03:52 +10:00
steffen123
5a045f944f Merge git://git.assembla.com/free_poker_tools 2008-11-07 19:58:11 +00:00
Ray
98d0305aa8 fixed psycopg2 connection parameters 2008-11-07 11:07:42 -05:00
Worros
1f3a39142a Change import_file_dict to take new args and fix all callers. 
Rename the main parser and make room for site converters.
 2008-11-07 21:18:12 +13:00
steffen123
c739395189 p140 - added CLI option to fpdb.py to control redirection of stderr 2008-11-07 00:14:25 +00:00
Worros
92381254e8 Change the filelist and dirlist type to be a hash of lists, delete postgres sql schema 
dirlist is in the form:
{'Site': [ "/path/to/dir", "filter" ] }

Where filter will be the plugin to convert to stars/fpdb hand history file.

filelist is in the form:
{'file': [ "site", "filter" ] }
 2008-11-06 00:36:24 +13:00
Worros
093302e76d Fix(?) problem with HUD only starting after 2 hands have been entered. 
Import code will now import_file_dict() on every file it initially finds that was modified in the last minute.

Not perfect, as a lot of hands can take longer than that.
 2008-11-05 10:39:27 +13:00
Worros
87c82df4b2 Merge branch 'master' of git://git.assembla.com/free_poker_tools 
Conflicts:

	pyfpdb/Hud.py
 2008-10-29 23:35:47 +09:00
Worros
9bb8e229e4 Fix TODO item - detect if directory passed in is a valid directory 2008-10-18 02:09:34 +08:00
Worros
5a085a6d23 Draft fix to mtime detection for files 2008-10-17 01:36:02 +08:00
Worros
10b0cc7bca Merge branch 'master' of git://git.assembla.com/fpdb 2008-10-16 21:21:55 +08:00
Worros
112f041a98 Start of changes to parse tournament hostory files 2008-10-16 21:21:11 +08:00
steffen123
bc15025be5 p135 - updated tv 2008-10-15 18:20:33 +01:00
Worros
3e9ef61fc8 Fix bug: incorrect use of set function to remove duplicates 
Now detects new tables without issues
 2008-10-13 23:34:18 +08:00
Worros
9a60cf84c8 Real fix this time, modify GuiAutoImport to actually tell importer to monitor the directory 2008-10-13 02:06:28 +08:00
Worros
8ca5a5c965 Should fix detection of new hh files. 2008-10-13 01:26:04 +08:00
Worros
c2c37dc5ee Fix mtime time detection 2008-10-12 18:21:42 +08:00
Worros
6c4e2f3eb9 Changes import_file_dict interface to Importer internal. 
Users of Importer should now addImportFile and addImportDirectory, set paramaters, then run either runImport - all files or runUpdated - modified files.

Comments out postgres regression test until fixed

Adds initial code for importing regression test files.
 2008-10-12 15:49:09 +08:00
Worros
638a6d6dab Move more "options" from calling class into settings has of fpdb_import and fix all callers 2008-10-12 02:14:06 +08:00
Worros
1e8333ec5d Move minPrint variable from callling class to setting in Importer class 2008-10-12 01:42:08 +08:00
Worros
ae9b70ea5d Change variable name to indicate what it actually is. 2008-10-12 01:19:57 +08:00
Worros
27ca82ca48 Change api so object and settings are passed in at creation time. 2008-10-12 01:12:30 +08:00
Worros
60b97582e5 Merge branch 'master' of git://git.assembla.com/free_poker_tools 2008-10-11 01:46:34 +08:00
Ray
576f80da25 use fh.tell() and fh.seek() to skip hh already seen in file 2008-10-10 13:42:09 -04:00