I'd like to parse about 6TB of logs from a verbose multi-threaded package, as the raw logs are a nightmare to decipher (and even then the results aren't stable).
In the process I'd like to apply some very simple logic to the parsing, but no matter how I try I can't seem to get the desired results:
I'm asking as language-agnostic because I'm sure the problem isn't tied to the one I'm using.
Any best practice, methodology or the like? Any suggestions at all really. Stupid logs.
EDIT: The log files are now 9TB. I'm keen to find a suitable answer so here are some example values and outputs that should help:
Logfile: yes Output: No Logfile: maybe Output: No Logfile: It's your decision Output: No Logfile: I'm not upset Output: I'm leaving you Logfile: Do you love me Output: Bad times divided by good times... divide by zero error
I'm adding a bounty too - if you can help me figure out the algorithm I'm made!