How to Error/Exception Handling in KDB

[Victor]

Hello Team,

I am working on the process to import the huge csv file data into kdb. I would like to add the error/exception handling logic to my script.

Please find the below code snippet that i am using. 

chunks:{.Q.en[`$dir] ("DIJSS*"; enlist ",") 0:x}

.Q.fs[{0N! .[path;();,; chunks x]} `$csvfile];

Here, in the above code how can i add the Error handling logic. How can i log the Error/Exception in to log.

Please help me on this.

Thanks in Advance.

[Connor Knox]

Hi Victor,

Q supports “try catch “ functionality using the . and @ operators in one of their many overloaded forms. Sample usage is given below, noting the distinction usage between functions monadic and polyadic functions.

q)f:{[x]x+1}

q)g:{[x;y]x+y}

q)

q)

q)@[f;1;{[x] show x;}]

2

q)@[f;`2;{[x] show x;}]

"type"

q)

q)

q)f[`2]

{[x]x+1}

'type

+

`2

1

q))

q))\

q)

q)

q)

q).[g;(1;2);{[x] show x;}]

3

q).[g;(1;`2);{[x] show x;}]

"type"

q)

q)

This simple example should extend to what you want.

You can find more information on try catch functionality here: http://code.kx.com/wiki/JB:QforMortals/execution_control

under "Protected Evaluation"

Regards,

Connor

[Victor]

Hi Connor,

Thank you for your response. However, i am still having confusion that how can i use . and @ operators with .Q.fs function. 

Do you have any idea ?

Regards,

Victor

[Connor]

Hi Victor,

To expand a little more:

.Q.fs takes two arguments. The first being a function to deal with chunks of a file and the second, the file name.

The file specified will be read in, as a string, and essentially each chunk passed as a string to the first argument, at which point you specify what logic is to be used to deal with the data. In your example I think you should do something like this (Ive chosen to insert data to a table in memory for the save of clarity) and I've created a counter so you can see exactly whats happening:

/Without error trapping initially.

/Note that the fancy logic is to account that my cvs contains a header row (which yours may or may not have.);

q)c

`time`sym`src`price`amount`side

q)p:();i:0;.Q.fs[{[x] `p insert $[i=0;1_;::] flip c!"TSSFFS"$flip "," vs' x;i+:1};`:trade.csv]

1494745

q)meta p

c     | t f a

------| -----

time  | t    

sym   | s    

src   | s    

price | f    

amount| f    

side  | s    

q)

q)count p

40000

q)\wc -l trade.csv

"   40001 trade.csv"

q)

/With error trapping (around the main function) that will exit the routine as soon as it sees an issue without attempting to carry on. 

q)p:();i:0;.[.Q.fs;({[x]if[i=10;'`oops]; `p insert $[i=0;1_;::] flip c!"TSSFFS"$flip "," vs' x;i+:1};`:trade.csv);{[x] show "error stopping"}]

"error stopping"

q)count p

346

q)i

10

q)/Note error trapping could be added to  the first argument of .Q.fs if you still wished to processes the rest of the file, assuming one chunk had issues. 

Regards,

Connor