1. Overview 2. String Functions 1. like 2. lower 3. ltrim 4. rtrim 5. ss 6. ssr 7. string 8. sv 9. trim 10. upper 11. vs 3. Mathematical Functions 1. acos 2. asin 3. atan 4. cor 5. cos 6. cov 7. cross 8. inv 9. lsq 10. mmu 11. sin 12. tan 13. var 14. wavg 15. wsum 4. Aggregate Functions 1. all 2. any 3. avg 4. dev 5. med 6. prd 7. sum 5. Uniform Functions 1. deltas 2. differ 3. fills 4. mavg 5. maxs 6. mcount 7. mdev 8. mins 9. mmax 10. mmin 11. msum 12. next 13. prds 14. prev 15. rank 16. ratios 17. rotate 18. sums 19. xbar 20. xprev 21. xrank 6. Miscellaneous Functions 1. Conditional Append (?) 2. asc 3. bin 4. count 5. cut 6. delete (_) 7. desc 8. distinct 9. drop (_) 10. eval 11. except 12. exit 13. fill (^) 14. find (?) 15. flip 16. getenv 17. group 18. iasc 19. identity 20. idesc 21. in 22. inter 23. join (,) 24. join-each (,') 25. list 26. null 27. parse 28. rand (?) 29. raze 30. reshape (#) 31. reverse 32. sublist 33. system 34. take (#) 35. til 36. ungroup 37. union 38. value 39. where 40. within Overview ¶ The collection of built-in functions in q is rich and powerful. In this chapter, we group functions by form. A string function takes a string and returns a string. An aggregate function takes a list and returns an atom. A uniform function takes a list and returns a list of the same count. A mathematical function takes numeric arguments and returns a numeric argument derives by some numerical calculation. Note that these categories are not mutually exclusive. For example, some mathematical functions are also aggregate functions. String Functions ¶ The basic string functions perform the usual string manipulations on a list of char. There are also powerful functions that are unique to q. like ¶ The dyadic like performs pattern matching on its first string argument (source) according to the pattern in its string second argument (pattern). It returns a boolean result indicating whether pattern is matched. The pattern is expressed as a mix of regular characters and special formatting characters. The special chars are "?", "*", the pair "[" and "]", and "^" enclosed in square brackets. The special char "?" represents an arbitrary single character in the pattern. "fan" like "f?n" 1b "fun" like "f?n" 1b "foP" like "f?p" 0b The special char "*" represents an arbitrary sequence of characters in the pattern. Note: As of this writing (Jan 2009), only a single occurance of * is allowed in the pattern. "how" like "h*" 1b "hercules" like "h*" 1b "wealth" like "*h" 1b "flight" like "*h*" 1b "Jones" like "J?ne*" 1b "Joynes" like "J?ne*" 0b "Joynes" like "J*ne*" 'nyi The special character pair "[" and "]" encloses a sequence of alternatives for a single character match. "flap" like "fl[ao]p" 1b "flip" like "fl[ao]p" 0b "459-0609" like "[09][09][09]-0[09][09][09]" 1b "459-0609" like "[09][09][09]-1[09][09][09]" 0b The special character "^" is used in conjunction with "[" and "]" to indicate that the enclosed sequence of characters is disallowed. For example, to test whether a string ends in a numeric character, "M26d" like "*[^09]" 1b "Joe999" like "*[^09]" 0b lower ¶ The monadic lower takes a char or string argument and returns the result of converting any alpha characters to lower case. lower "A" "a" lower "a Bc42De" "a bc42de" ltrim ¶ The monadic ltrim takes a string argument and returns the result of removing leading blanks. ltrim " abc " "abc " You can also apply ltrim to a non-blank char. ltrim "a" "a" rtrim ¶ The monadic rtrim takes a string argument and returns the result of removing trailing blanks. rtrim " abc " " abc" You can also apply rtrim to a non-blank char. rtrim "a" "a" ss ¶ The dyadic ss ("string search") performs the same pattern matching as like against its first string argument (source), looking for matches to its string second argument (pattern). However, the result of ss is a list containing the position(s) of the matches of the pattern in source. See above for a discussion of like. "Now is the time for all good men to come to" ss "me" 13 29 38 "fun" ss "f?n" ,0 If no matches are found, an empty int list is returned. "aa" ss "z" `int$() Note: You cannot use * to match with ss. ssr ¶ The triadic ssr ("string search and replace") extends the capability of ss with replacement. The result is a string based on the first string argument (source) in which all occurrences of the second string argument (pattern) are replaced with the third string argument. ssr["suffering succotash";"s";"th"] "thuffering thuccotathh" Note: You cannot use * to match with ssr. string ¶ The monadic string can be applied to any q entity to produce a textual representation of the entity. For scalars, lists and functions, the result of string is a list of char that does not contain any q formatting characters. Following are some examples. string 42 "42" string 6*7 "42" string 42422424242j "42422424242" string `Zaphod "Zaphod" f:{[x] x*x} string f "{[x] x*x}" The next example demonstrates that string is not atomic, because the result of applying it to an atom is a list of char. string "4" ,"4" The next example may be surprising. string 0x42 "42" To see why, recall from Creating Symbols from Strings that a string can be parsed into q data using $ with the appropriate upper-case type domain character. Now, converting to a string and parsing from a string should be inverse maps, in that their composite returns the original input value. That is, we should find, "X"$string 0x42 0x42 Thus, the behavior of string is determined by that of parse. "X"$"42" 0x42 Comparing these two results, we see that the result of string on a byte must not contain the format characterless. This reasoning works for other types as well. Although string is not atomic (it returns a list from an atom), it does act like an atomic function in that its application is extended item-wise to a list. string 42 98 "42" "98" string 1 2 3 ,"1" ,"2" ,"3" string "Beeblebrox" ,"B" ,"e" ,"e" ,"b" ,"l" ,"e" ,"b" ,"r" ,"o" ,"x" string(42; `life; ("the"; 0x42)) "42" "life" ((,"t";,"h";,"e");"42") Considering a list as a mapping, we see that string acts on the range of the mapping. Viewing a dictionary as a generalized list, we conclude that the action of string on a dictionary should also apply to its range. d:1 2 3!100 101 102 string d 1| "100" 2| "101" 3| "102" A table is the flip of a column dictionary, so we expect string to operate on the range of the column dictionary. t:([] a:1 2 3; b:`a`b`c) string t a b --------- ,"1" ,"a" ,"2" ,"b" ,"3" ,"c" Finally, a keyed table is a dictionary, so we expect string to operate on the value table. kt:([k:1 2 3] c:100 101 102) string kt k| c -| ----- 1| "100" 2| "101" 3| "102" sv ¶ The basic form of dyadic sv ("string from vector") takes a char as its left operand and a list of strings (source) as its right operand. It returns a string that is the concatenation of the strings in source, separated by the specified char. ";" sv("Now";"is";"the";"time";"") "Now;is;the;time;" When sv is used with an empty symbol as its left operand and a list of symbols as its right operand (source), the result is a symbol in which the items in source are concatenated with a separating dot. ` sv `qalib`stat `qalib.stat This is useful for q context names. When sv is used with an empty symbol as its left operand and a symbol right operand (source) whose first item is a file handle, the result is a symbol in which the items in source are concatenated with a separating forward-slash. This is useful for fully qualified q path names. ` sv `:`q`tutorial`draft1 `:/q/tutorial/draft1 When sv is used with an int left operand (base) that is greater than 1, together with a right operand of a simple list of place values expressed in base, the result is an int representing the converted base 10 value. 2 sv 101010b 42 10 sv 1 2 3 4 2 12342 256 sv 0x001092 4242 Advanced: More precisely, the last version of sv evaluates the polynomial, (d[n-1]*b exp n-1) + ... +d[0] where d is the list of digits, n is the count of d, and b is the base. Thus, we find, 10 sv 1 2 3 11 2 12412 -10 sv 2 1 5 195 trim ¶ The monadic trim takes a string argument and returns the result of removing leading and trailing blanks. trim " abc " " abc" Note: The function trim is equivalent to, {ltrim rtrim x} You can also apply trim to a non-blank char. trim "a" "a" upper ¶ The monadic upper takes a char, string or symbol argument and returns the result of converting any alpha characters to upper case. upper "a" "A" upper "a Bc42De" "A BC42DE" vs ¶ The dyadic vs ("vector from string") takes a char as its left operand and a string (source) as its right operand. It returns a list of strings containing the tokens of source as delimited by the specified char. " " vs "Now is the time " "Now" "is" "the" "time" "" When vs is used with an empty symbol as its left operand and a symbol right operand (source) containing separating dots, it returns a simple symbol list obtained by splitting source along the dots. ` vs `qalib.stat `qalib`stat When vs is used with an empty symbol as its left operand and a symbol representing a fully qualified file name as the right operand, it returns a simple list of symbols in which the first item is the path and the second item is the file name. ` vs `:/q/tutorial/draft `:/q/tutorial`draft Note that in the last usage, vs is not quite the inverse of sv. When vs is used with a null of binary type as the left operand and an value of integer type as the right operand (source), it returns a simple list whose items comprise the digits of the corresponding binary representation of source. 0x00 vs 4242 0x00001092 10h$0x00 vs 8151631268726338926j "q is fun" 0b vs 42 00000000000000000000000000101010b Advanced: The last form can be used to display the internal representation of special values. 0b vs 0W 01111111111111111111111111111111b 0b vs -0W 10000000000000000000000000000001b Mathematical Functions ¶ The mathematical functions perform the mathematical operations for basic calculations. Their implementations are efficient. acos ¶ The monadic acos is the mathematical inverse of cos. For a float argument between -1 and 1, acos returns the float between 0 and π whose cosine is the argument. sqrt 2:1.414213562373095 acos 1 0f acos sqrt2 0n acos -1 3.141592653589793 \ acos 0 1.570796326794897 asin ¶ The monadic asin is the mathematical inverse of sin. For a float argument between -1 and 1, asin returns the float between -π/2 and π/2 whose sine is the argument. sqrt2:1.414213562373095 asin 0 0f asin sqrt 2%2 0.7853982 asin 1 1.570796 asin -1 -1.570796326794897 atan ¶ The monadic atan is the mathematical inverse of tan. For a float argument, it returns the float between -π/2 and π/2 whose tangent is the argument. sqrt2:1.414213562373095 atan 0 0f atan sqrt 2 0.9553166181245093 atan 1 0.7853981633974483 cor ¶ The dyadic cor takes two numeric lists of the same count and returns a float equal to the mathematical correlation between the items of the two arguments. 23 -11 35 0 cor 42 21 73 39 0.9070229 Note: The function cor is equivalent to, {cov[x;y]%dev[x]*dev y} cos ¶ The monadic cos takes a float argument and returns the mathematical cosine of the argument. pi:3.141592653589793 cos 0 1f cos pi%3 0.5000000000000001 cos pi%2 6.123032e-017 cos pi -1f cov ¶ The dyadic cov takes a numeric atom or list in both arguments and returns a float equal to the mathematical covariance between the items of the two arguments. If both arguments are lists, they must have the same count. 98 cov 42 0f 23 -11 35 0 cov 42 21 73 39 308.4375 Note: The function cov is equivalent to, {avg[x*y]-avg[x]*avg y} cross ¶ The binary cross takes atoms or lists as arguments and returns their Cartesian product - that is, the set of all pairs drawn from the two arguments. 1 2 cross `a`b`c 1 `a 1 `b 1 `c 2 `a 2 `b 2 `c Note: The cross operator is equivalent to the function, {raze x,\:/:y} inv ¶ The monadic inv returns the inverse of a float matrix. m:(1.1 2.1 3.1; 2.3 3.4 4.5; 5.6 7.8 9.8) inv m -8.165138 16.51376 -5 12.20183 -30.18349 10 -5.045872 14.58716 -5 Note: An integer argument will cause an error, so cast it to float. lsq ¶ The dyadic matrix function lsq returns the matrix X that solves the following matrix equation, where A is the float matrix left operand, B is the float matrix right operand and · is matrix multiplication. A = X·B For example, A:(1.1 2.2 3.3;4.4 5.5 6.6;7.7 8.8 9.9) B:(1.1 2.1 3.1; 2.3 3.4 4.5; 5.6 7.8 9.8) A lsq B 1.211009 -0.1009174 2.993439e-12 -2.119266 2.926606 -3.996803e-12 -5.449541 5.954128 -1.758593e-11 Observe that the result of lsq can be obtained as, A mmu inv B 1.211009 -0.1009174 1.77991e-12 -2.119266 2.926606 -5.81224e-12 -5.449541 5.954128 -1.337952e-11 Note: Integer arguments will cause an error, so cast them to float. mmu ¶ The dyadic matrix multiplication function mmu returns the matrix product of its two float vector or matrix arguments, which must be of the correct shape. Note: Integer arguments will cause an error, so cast them to float. Here is an example of multiplying a matrix and its transpose. m1:(1.1 2.2 3.3;4.4 5.5 6.6;7.7 8.8 9.9) m2:flip m2 m1 mmu m2 36.3 43.56 50.82 79.86 98.01 116.16 123.42 152.46 181.5 The $ operator is overloaded to yield matrix multiplication when its arguments are float vectors or matrices. 1 2 3f mmu 1 2 3f 14f 1 2 3f$1 2 3f 14f sin ¶ The monadic sin takes a float argument and returns the mathematical sine of the argument. pi:3.141592653589793 sin 0 0f sin pi%4 0.7071068 sin pi%2 1f sin pi 1.224606e-016 tan ¶ The monadic tan takes a float argument and returns the mathematical tangent of the argument. Note: The value tan x is (sin x)%cos x pi:3.141592653589793 tan 0 0f tan pi%8 0.4142136 tan pi%4 1f tan pi%2 1.633178e+016 tan pi -1.224606e-016 var ¶ The monadic var takes a scalar or numeric list and returns a float equal to the mathematical variance of the items. var 42 0f var 42 45 37 38 10.25 Note: The function var is equivalent to {(avg[x*x]) - (avg[x])*(avg[x])} wavg ¶ The dyadic wavg takes two numeric lists of the same count and returns the average of the second argument weighted by the first argument. The result is always of type float. 1 2 3 4 wavg 500 400 300 200 300f Note: The expression w wavg b is equivalent to, (sum w*a)%sum w In our example, (sum (1 2 3 4)*500 400 300 200)%sum 1 2 3 4 300f It is possible to apply wavg to a nested list provided all sublists of both arguments conform. In this context, the result conforms to the sublists and the weighted average is calculated recursively across the sublists. (1 2;3 4) wavg (500 400; 300 200) 350 266.6667 ((1;2 3);(4;5 6)) wavg ((600;500 400);(300;200 100)) 360f 285.7143 200 wsum ¶ The dyadic wsum takes two numeric lists of the same count and returns the sum of the second argument weighted by the first argument. The result is always of type float. 1 2 3 4 wsum 500 400 300 200 3000f Note: The expression w wsum b is equivalent to, sum w*a In our example, sum (1 2 3 4)*500 400 300 200 3000 It is possible to apply wsum to a nested list provided all sublists of both arguments conform. In this context, the result conforms to the sublists and the weighted sum is calculated recursively across the sublists. (1 2;3 4) wsum (500 400;300 200) 1400 1600 ((1;2 3);(4;5 6)) wsum ((600;500 400);(300;200 100)) 1800 2000 1800 Aggregate Functions ¶ An aggregate function operates on a list and returns an atom. Aggregates are especially useful with grouping in select expressions. all ¶ The monadic all takes a scalar or list of numeric type and returns the result of & applied across the items. all 1b 1b all 100100b 0b all 10 20 30 10 any ¶ The monadic any takes a scalar or list of numeric type and returns the result of | applied across the items. any 1b 1b any 100100b 1b any 2001.01.01 2006.10.13 2006.10.13 avg ¶ The monadic avg takes a scalar, list, dictionary or table of numeric type and returns the arithmetic average. The result is always of type float. avg 42 42f avg 1 2 3 4 5 3f avg `a`b`c!10 20 40 23.33333 It is possible to apply avg to a nested list provided the sublists conform. In this context, the result conforms to the sublists and the average is calculated recursively on the sublists. avg (1 2; 100 200; 1000 2000) 367 734f avg ((1 2;3 4); (100 200;300 400)) 50.5 101 151.5 202 For tables, the result is a dictionary that maps each column name to the average of its column values. t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 avg t c1| 2.75 c2| 3.5 dev ¶ The monadic dev takes a scalar, list, or dictionary of numeric type and returns the standard deviation. For result is a float. dev 42 0f dev 42 45 37 38 3.201562 dev `a`b`c!10 20 40 12.47219 Note: The function dev is equivalent to {sqrt[var[x]]} med ¶ The monadic med takes a list, dictionary or table of numeric type and returns the statistical median. For lists and dictionaries, the result is a float. med 42 21 73 39 40.5 med `a`b`c!10 20 40 20f Note: The function med is equivalent to, {$[n:count x;.5*sum x[rank x]@floor .5*n-1 0;0n]} For tables, the result is a dictionary mapping the column names to their value medians. t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 med t c1| 2.75 c2| 3.5 prd ¶ The monadic prd takes a scalar, list, dictionary or table of numeric type and returns the arithmetic product. For scalars, lists and dictionaries the result has the type of its argument. prd 42 42 prd 1.1 2.2 3.3 4.4 5.5 193.2612 prd `a`b`c!10 20 40 8000 It is possible to apply prd to a nested list provided the sublists conform. In this case, the result conforms to the sublists and the product is calculated recursively on the sublists. prd (1 2; 100 200; 1000 2000) 100000 800000 prd ((1 2;3 4); (100 200;300 400)) 100 400 900 1600 For tables, the result is a dictionary that maps each column name to the product of its column values. t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 prd t c1| 35.1384 c2| 120 sum ¶ The monadic sum takes a scalar, list, dictionary or table of numeric type and returns the arithmetic sum. For scalars, lists and dictionaries the result has the type of its argument. sum 42 42 sum 1.1 2.2 3.3 4.4 5.5 16.5 sum `a`b`c!10 20 40 70 It is possible to apply sum to a nested list provided the sublists conform. In this case, the result conforms to the sublists and the sum is calculated recursively on the sublists. sum (1 2; 100 200; 1000 2000) 1101 2202 sum ((1 2;3 4); (100 200;300 400)) 101 202 303 404 For tables, the result is a dictionary that maps each column name to the sum of its column values. t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 sum t c1| 11 c2| 14 Uniform Functions ¶ Uniform functions operate on lists and return lists of the same shape. They are useful in select expressions. deltas ¶ The uniform deltas takes as its argument (source) a scalar, list, dictionary or table of numeric type and returns the difference of each item from its predecessor. deltas 42 42 deltas 1 2 3 4 5 1 1 1 1 1 deltas 96.25 93.25 58.25 73.25 89.50 84.00 84.25 96.25 -3 -35 15 16.25 -5.5 0.25 deltas `a`b`c!10 20 40 a| 10 b| 10 c| 20 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 deltas t c1 c2 ------ 1.1 5 1.1 -1 1.1 -1 1.1 -1 Important: As the third example shows, the result of deltas contains the initial item of source in its initial position. This may be inconsistent with the behavior of similar functions in other languages or libraries that return 0 in the initial position. The alternate behavior can be achieved with the expression 1_deltas (1#x),x In our example above, 1_deltas (1#x),x:96.25 93.25 58.25 73.25 89.50 84.00 84.25 0 -3 -35 15 16.25 -5.5 0.25 differ ¶ The uniform differ takes as its argument (source) a list and returns a boolean list whose item in position i is the result of match (~) applied to the item at position i and the item at position i-1. The result of differ on a scalar is 0b. Note: The item at position 0 in the result is always 1b. differ 1 1 2 101b differ 0N 0N 1 1 2 10101b differ "mississippi" 11101101101b differ (1 2; 1 2; 3 4 5) 101b One use of differ is to locate runs of repreated items in a list. L:0 1 1 2 3 2 2 2 4 1 1 3 4 4 4 4 5 L where nd|next nd:not differ L 1 1 2 2 2 1 1 4 4 4 4 fills ¶ The uniform fills takes as its argument (source) a scalar, list, dictionary or table of numeric type and returns a copy of the source in which non-null items are propagated forward to fill nulls. fills 42 42 fills 1 0N 3 0N 5 1 1 3 3 5 fills `a`b`c`d`e`f!10 0N 30 0N 0N 60 a| 10 b| 10 c| 30 d| 30 e| 30 f| 60 tt:([] c1:1 0N 3 0N; c2:`a`b``d) tt c1 c2 ----- 1 a b 3 d fills tt c1 c2 ----- 1 a 1 b 3 b 3 d Note: Initial nulls are not affected by fills. fills 0N 0N 3 0N 5 0N 0N 3 3 5 mavg ¶ The uniform dyadic mavg takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving average of source, obtained by applying avg over length consecutive items. For positions less than length-1, avg is applied only through that position. In the following example, the first item in the result is the average of itself only; the second result item is the average of the first two source items; all other items reflect the average of the item at the position along with its two predecessors. 3 mavg 10 20 30 40 50 10 15 20 30 40f For length 1, the result is the source converted to float. For length less than or equal to 0 the result is all nulls. Note: As of release 2.4, mavg ignores null values. 3 mavg 10 20 0N 40 50 60 0N 10 15 15 30 45 50 55f maxs ¶ The uniform maxs takes as its argument (source) a scalar, list, dictionary or table and returns the cumulative maximum of the source items. maxs 42 42 maxs 1 2 5 4 10 1 2 5 5 10 maxs "Beeblebrox" "Beeelllrrx" maxs `a`b`c`d!10 30 20 40 a| 10 b| 30 c| 30 d| 40 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 maxs t c1 c2 ------ 1.1 5 2.2 5 3.3 5 4.4 5 mcount ¶ The uniform dyadic mcount takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving count of source, obtained by applying count over length consecutive items. For positions less than length-1, count is applied only through that position. This function is useful in computing other moving quantities. For example, 3 mcount 10 20 30 40 50 1 2 3 3 3 For length less than or equal to 0 the result is all zeroes Note: As of release 2.4, mcount ignores null values. 3 mcount 10 20 0N 40 50 60 0N 1 2 2 2 2 3 2 mdev ¶ The uniform dyadic mdev takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving standard deviation of source, obtained by applying dev over length consecutive items. For positions less than length-1, dev is applied only through that position. In the following example, the first item in the result is the standard deviation of itself only; the second result item is the standard deviation of the first two source items; all other items reflect the standard deviation of the item at the position along with its two predecessors. 3 mdev 10 20 30 40 50 0 5 8.164966 8.164966 8.164966 For length less than or equal to 0 the result is all nulls. mins ¶ The uniform mins takes as its argument (source) a scalar, list, dictionary or table and returns the cumulative minimum of the source items. mins 42 42 mins 10 4 5 1 2 10 4 4 1 1 mins "Beeblebrox" "BBBBBBBBBB" mins `a`b`c`d!40 10 30 20 a| 40 b| 10 c| 10 d| 10 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 mins t c1 c2 ------ 1.1 5 1.1 4 1.1 3 1.1 2 mmax ¶ The uniform dyadic mmax takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving maximum of source, obtained by applying max over length consecutive items. For positions less than length-1, max is applied only through that position. In the following example, the first item in the result is the max of itself only; the second result item is the max of the first two source items; all other items reflect the max of the item at the position along with its two predecessors. 3 mmax 20 10 30 50 40 20 20 30 50 50 For length less than or equal to 0 the result is source. mmin ¶ The uniform dyadic mmin takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving minimum of source, obtained by applying min over length consecutive items. For positions less than length-1, min is applied only through that position. In the following example, the first item in the result is the min of itself only; the second result item is the min of the first two source items; all other items reflect the min of the item at the position along with its two predecessors. 3 mmin 20 10 30 50 40 20 10 10 10 30 For length less than or equal to 0 the result is source. msum ¶ The uniform dyadic msum takes as its first argument an int (length) and as its second argument (source) a numeric list. It returns the moving sum of source, obtained by applying sum over length consecutive items. For positions less than length-1, sum is applied only through that position. In the following example, the first item in the result is the sum of itself only; the second result item is the sum of the first two source items; all other items reflect the sum of the item at the position along with its two predecessors. 3 msum 10 20 30 40 50 10 30 60 90 120 For length less than or equal to 0 the result is all zeros. next ¶ The uniform next takes as its argument (source) a scalar, list or table of numeric type and returns the source shifted one position to the left with no wrapping. For lists and dictionaries, the last item of the result is a null matching the type of source. For tables, the last record of the result is a row of nulls. next 1 2 3 4 5 2 3 4 5 0N t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 next t c1 c2 ------ 2.2 4 3.3 3 4.4 2 prds ¶ The uniform sums takes as its argument (source) a scalar, list, dictionary or table of numeric type and returns the cumulative product of the source items. prds 42 42 prds 1 2 3 4 5 1 2 6 24 120 prds `a`b`c!10 20 40 a| 10 b| 200 c| 8000 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 prds t c1 c2 ----------- 1.1 5 2.42 20 7.986 60 35.1384 120 prev ¶ The uniform prev takes as its argument (source) a scalar, list, dictionary or table. It returns the source shifted one position forward with initial null filling. prev 42 42 prev 1 2 3 4 5 0N 1 2 3 4 prev `a`b`c!10 20 40 a| b| 10 c| 20 t:([]c1:`a`b`c;c2:10 20 40) t c1 c2 ----- a 10 b 20 c 40 prev t c1 c2 ----- a 10 b 20 rank ¶ The uniform rank takes as its argument (source) a list, dictionary or table whose values are sortable. It returns a list of int containing the order of each item in the source under an ascending sort. For dictionaries, the operation is against the range. rank 5 2 3 1 4 4 1 2 0 3 rank `a`b`c`e`f! 5 2 3 1 4 4 1 2 0 3 For tables and keyed tables, the result is a list with the rank of the records under ascending sort of the first column or the key column. ttt:([] c1:2.2 1.1 3.3 5.5 4.4; c2:1 2 3 4 5) ttt c1 c2 ------ 2.2 1 1.1 2 3.3 3 5.5 4 4.4 5 rank ttt 1 0 2 4 3 kt:([k:103 102 101 105 104] d:1 2 3 4 5) kt k | d ---| - 103| 1 102| 2 101| 3 105| 4 104| 5 rank kt 2 1 0 4 3 ratios ¶ The uniform ratios takes as its argument (source) a scalar, list, dictionary or table of numeric type and returns the float ratio of each item to its predecessor. ratios 42 42 ratios 1 2 3 4 5 1 2 1.5 1.333333 1.25 ratios 96.25 93.25 58.25 73.25 89.50 84.00 84.25 96.25 0.9688312 0.6246649 1.257511 1.221843 0.9385475 1.002976 deltas `a`b`c!10 20 40 a| 10 b| 10 c| 20 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 ratios t c1 c2 ------------------ 1.1 5 2 0.8 1.5 0.75 1.333333 0.6666667 Important: As the second example shows, the result of ratios contains the initial item of source in its initial position. This may be inconsistent with the behavior of similar functions in other languages or libraries that return 1 in the initial position. The alternate behavior can be achieved with the expression, 1,ratios 1_x In our example above, 1,ratios 1_x:96.25 93.25 58.25 73.25 89.50 84.00 84.25 1 93.25 0.6246649 1.257511 1.221843 0.9385475 1.002976 rotate ¶ The uniform dyadic rotate takes as its first argument an int (length) and as its second argument (source) a numeric list or table. It returns the source shifted length positions to the left with wrapping if length is positive, or length positions to the right with wrapping if length is negative. For length 0, it returns the source. 2 rotate 1 2 3 4 5 3 4 5 1 2 -2 rotate 1 2 3 4 5 4 5 1 2 3 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 --------- 1.1 5 2.2 4 3.3 3 4.4 2 2 rotate t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 sums ¶ The uniform sums takes as its argument (source) a scalar, list, dictionary or table of numeric type and returns the cumulative sum of the source items. sums 42 42 sums 1 2 3 4 5 1 3 6 10 15 sums `a`b`c!10 20 40 a| 10 b| 30 c| 70 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 sums t c1 c2 ------ 1.1 5 3.3 9 6.6 12 11 14 xbar ¶ The uniform dyadic xbar takes as its first argument a non-negative numeric atom (width) and a second argument (source) that is a numeric list, dictionary or table. It returns an entity that conforms to source, in which each item of source is mapped to the largest multiple of the width that is less than or equal to that item. The type of the result is that of the width parameter. 3 xbar 2 7 12 17 22 0 6 12 15 21 5.5 xbar 59.25 53.75 81.00 96.25 93.25 58.25 73.25 89.50 84.00 84.25 55 49.5 77 93.5 88 55 71.5 88 82.5 82.5 15 xbar `a`b`c!10 20 40 a| 0 b| 15 c| 30 t:([]c1:1.1 2.2 3.3 4.4; c2:5 4 3 2) t c1 c2 ------ 1.1 5 2.2 4 3.3 3 4.4 2 2 xbar t c1 c2 ----- 0 4 2 4 2 2 4 2 Since xbar is atomic in its second argument it can be applied to a nested list. 5 xbar ((11;21 31);201 301) 10 20 30 200 300 xprev ¶ The dyadic xprev takes an int as its first argument (shift) and is uniform in its second argument (source), which can be a list or a table. It returns a result that conforms to source. When shift is 0 or positive, each entity in source is shifted shift positions forward in the result, with the initial shift entries null filled. 2 xprev 10 20 30 40 0N 0N 10 20 t:([]c1:`a`b`c`d;c2:10 20 30 40) t c1 c2 ----- a 10 b 20 c 30 d 40 2 xprev t c1 c2 ----- a 10 b 20 When shift is negative, the result is a copy of source with the initial shift entries null filled. -2 xprev 10 20 30 40 30 40 0N 0N xrank ¶ The binary xrank is uniform in its right operand (source), which is a list, dictionary, table or keyed table whose values are sortable. The left operand is a positive int (quantile). It returns a list of int containing the quantile of the source distribution to which each item of source belongs. The analysis is applied to the range of a dictionary and the first column of a table. For example, by choosing quantile to be 4, xrank determines into which quartile each item of source falls. 4 xrank 30 10 40 20 90 1 0 2 0 3 4 xrank `a`b`c`d`e!30 10 40 20 90 1 0 2 0 3 t:([]c1:30 10 40 20 90;c1:`a`b`c`d`e) t c1 c11 ------ 30 a 10 b 40 c 20 d 90 e 4 xrank t 1 0 2 0 3 Choosing quantile to be 100 gives percentile ranking. Miscellaneous Functions ¶ We collect here the built-in functions that don't fit into any of the previously defined categories. Conditional Append (?) ¶ The left operand of conditional append ( ? ) is a symbol representing the name of a list of symbols (target) and the right operand is a symbol, the right operand is appended to target if and only if it is not in target. There is no effect when the right operand is already in target. The result is the enumeration of the right operand in target. v:`a`b`c `v?`z `v$`z v `a`b`c`z `v?`b `v$`b v `a`b`c`z Note: While conditional append is normally used with a target list of unique items, this is not a requirement. asc ¶ The monadic function asc operates on a list or a dictionary (source). The result of asc on a list is a list comprising the items of source sorted in increasing order with the s# attribute applied. The result of asc on a dictionary is an equivalent mapping with the range items sorted in increasing order and with the s# attribute applied. asc 3 7 2 8 1 9 `s#1 2 3 7 8 9 asc `b`c`a!3 2 1 a| 1 c| 2 b| 3 bin ¶ The dyadic bin takes a simple list of items (target) in strictly increasing order as its first argument and is atomic in its second argument (token). Loosely speaking, the result of bin is the position at which token would fall in target. More precisely, the result is -1 if token is less than the first item in target. Otherwise, the result is the position of the right-most item of target that is less than or equal to token; this reduces to the found position if the token is in target. If token is greater than the last item in target, the result is the count of target. Note: For large sorted lists, the binary search performed by bin is generally more efficient than the linear search algorithm used by in. Some examples with simple lists, 1 2 3 4 bin 3 2 "xyz" bin "a" -1 1.0 2.0 3.0 bin 0.0 2.0 2.5 3.0 -1 1 1 2 Observe that the type of token must strictly match that of target. 1 2 3 bin 1.5 `type We can apply bin to a dictionary to perform reverse lookup, provided the dictionary domain is in increasing order. When source is a dictionary, bin takes a token whose type matches that of the dictionary range. The result is null if token is less than every item of the range. Otherwise, the result is the right-most domain element whose corresponding range element is less than or equal to token. Loosely put, when token is not found, the result is the domain item after which you would make an insertion to place it into the dictionary in proper order. Note that the result reduces to the corresponding domain item if token is found in target, and is the last domain item if token is greater than every range item. d:10 20 30!`first`second`third d bin `second 20 d bin `missing 10 d bin `zero 30 d bin `aaa 0N Because a table is a list of records, we expect bin to return the row number of a record. t:([] a:1 2 3; b:`a`b`c) t a b --- 1 a 2 b 3 c t bin `a`b!(2;`b) 1 As always, the record can be abbreviated to the list of row values. t bin (1;`a) 0 t bin (0;`z) 0N Observe that a record that is not found results in a null result. Finally, since a keyed table is a dictionary, bin will perform a reverse lookup on a record of the value table, which can be abbreviated to a list of row values. kt:([k:1 2 3] c:100 101 102) kt k| c -| --- 1| 100 2| 101 3| 102 kt bin (enlist `c)!enlist 101 k| 2 kt bin 101 k| 2 Warning: While the items of the first argument of bin should be in strictly increasing order for the result to meaningful, this condition is not enforced. The results of bin when the first argument is not strictly increasing are predictable but not particularly useful. count ¶ The monadic count returns a non-negative int representing the number of entities in its argument. Its domain comprises scalars, lists, dictionaries, tables and keyed tables. count 3 1 count 10 20 30 3 count `a`b`c`d!10 20 30 40 4 count ([] a:10 20 30; b:1.1 2.2 3.3) 3 count ([k:10 20] c:`one`two) 2 Note: You cannot use count to determine whether an entity is a scalar or list since scalars and singletons both have count 1. count 3 1 count enlist 3 1 This test is accomplished instead by testing the sign of the type of the entity. 0>type 3 1b 0>type enlist 3 0b Aside: Do you know why they call it count? Because it loves to count!! Nyah, ha, ha, ha, ha. Vun, and two, and tree, and.... cut ¶ The binary operator cut is related to the _ operator. It is the same as _ when the right operand is a dictionary and the left operand is a list of items from the dictionary domain. d:1 2 3!`a`b`c (enlist 2) cut d 1| a 3| c However, for a list right operand source and an int left operand size, cut returns a new list created by collecting the items of source into sublists of count size. 5 cut til 13 0 1 2 3 4 5 6 7 8 9 10 11 12 Advanced: The cut function is equivalent to, {$[0>type x;x*til neg floor neg(count y)mod x;x]_y} delete (_) ¶ The symbol _ is overloaded to have several meanings depending on the signature of its operands. See also drop. Note: When _ is used as an operator, whitespace is required to the left if the left operand is a name. This is because _ is a valid non- initial name character. Whitespace is permitted but not required to the right. When the first argument of dyadic ( _ ) is a list of non-negative int and the second argument (source) is a list, it produces a new list obtained by breaking source into sublists at the positions indicated in the first argument. An example will make this clear. 0 3_100 200 300 400 500 100 200 300 400 500 Each sublist includes the items from the beginning cut position up to, but not including, the next cut position. The final cut includes the items to the end of source. Observe that if the left argument does not begin with 0, the initial items of source will not be included in the result. 2 4_2006.01 2006.02 2006.03 2006.04 2006.05 2006.06 2006.03 2006.04 2006.05 2006.06 When the right operand of _ is a dictionary (source) and the left operand is a list of key values whose type matches source, the result is a dictionary obtained by removing the specified key-value pairs from the target. For example, d:1 2 3!`a`b`c (enlist 42) _ d 1| a 2| b 3| c (enlist 2) _ d 1| a 3| c 1 3 _ d 2| b (enlist 32) _ d 1| a 2| b 3| c 1 2 3 _ d _ Note: The operand must be a list, so a single key value must be enlisted. When the first argument of dyadic delete ( _ ) is a list or a dictionary (source) and the second argument is a position in the list or an item in the domain of the dictionary, the result is a new entity obtained by deleting the specified item from the source. L: 101 102 103 104 105 L _2 101 102 104 105 d:`a`b`c`d!101 102 103 104 d _ `b a| 101 c| 103 d| 104 Since a table is a list, delete can be applied by row number. t:([]c1:1 2 3;c2:101 102 103;c3:`x`y`z) t c1 c2 c3 --------- 1 101 x 2 102 y 3 103 z t _ 1 c1 c2 c3 --------- 1 101 x 3 103 z Since a keyed table is a dictionary, delete can be applied by key value. kt:([k:101 102 103]c:`one`two`three) kt k | c ---| ----- 101| one 102| two 103| three kt _ 102 k | c ---| ----- 101| one 103| three desc ¶ The monadic function desc operates on a list or a dictionary (source). The result of desc on a list is a list comprising the items of source sorted in decreasing order with the s# attribute applied. The result of desc on a dictionary is an equivalent mapping with the range items sorted in decreasing order and with the s# attribute applied. desc 3 7 2 8 1 9 9 8 7 3 2 1 desc `b`c`a!3 2 1 b| 3 c| 2 a| 1 distinct ¶ The monadic function distinct returns the distinct entities in its argument. For a list, it returns the distinct items in the list, in order of first occurrence. distinct 1 2 3 2 3 4 6 4 3 5 6 1 2 3 4 6 5 For a table, distinct returns a table comprising the distinct records of the argument, in the order of first occurrence. tdup:([]a:1 2 3 2 1; b:`washington`adams`jefferson`adams`wasington) tdup a b ------------ 1 washington 2 adams 3 jefferson 2 adams 1 wasington distinct tdup a b ------------ 1 washington 2 adams 3 jefferson 1 wasington Observe that all fields of the records must be identical for the records to be considered identical. Otherwise put, if any field differs, the records are distinct. When applied to an int n, distinct produces a random int between 0 (inclusive) and n (exclusive). distinct 42 37 distinct 42 39 drop (_) ¶ The symbol _ is overloaded to have several meanings depending on the signature of its operands. See also delete. Note: When _ is used as an operator, whitespace is required to the left if the left operand is a name. This is because _ is a valid non- initial name character. Whitespace is permitted but not required to the right. When the first argument of the dyadic _ is an int and the second argument (source) is a list, the result is a new list created via removal from source. A positive int in the first argument indicates that the removal occurs from the beginning of the source, whereas a negative int in the first argument indicates that the removal occurs from the end of the source. The source can be a list, a dictionary, a table or a keyed table. 2_10 20 30 40 30 40 -3_`one`two`three`four`five `one`two 2_`a`b`c`d!10 20 30 40 c| 30 d| 40 -1_([] a:10 20 30 40; b:1.1 2.2 3.3 4.4) a b ------ 10 1.1 20 2.2 30 3.3 2_([k:10 20 30] c:`one`two`three) k | c --| ----- 30| three The result of drop is of the same type and shape as source and is never a scalar. 1_42 67 ,67 Observe that for nested lists, the deletion occurs at the top-most level. 1_(100 101 102;103 104 105) 103 104 105 In the degenerate case, the result is an empty entity derived from source. 4_10 20 30 40 `int$() 4_`a`b`c`d!10 20 30 40 4_([] a:10 20 30 40; b:1.1 2.2 3.3 4.4) a b -- 3_([k:10 20 30] c:`one`two`three) k| c -| - eval ¶ The monadic eval evaluates a list that represents a valid q parse tree, which can be produced by parse or by hand (if you know what you're doing). A discussion of parse trees is beyond the scope of this manual. show pt:parse "a: 6*7" : `a (*;6;7) eval pt 42 except ¶ The dyadic except takes a simple list or a dictionary whose range is a simple list as its first argument (target) and returns a list containing the items of target excluding those that are in its second argument, which can be a scalar or a list. The returned items are in the order of their first occurrence in target. 1 2 3 4 3 2 except 2 1 3 4 3 1 2 3 4 3 2 except 1 2 10 3 4 3 "Now is the time_" except "_" "Now is the time" d:`a`c`d`e!1 2 1 2 d except 1 2 2 The result of except is never a scalar. 1 2 except 1 ,2 1 2 except 2 1 `int$() d except 1 2 `int$() exit ¶ The monadic exit takes an int as its argument and a and executes the system command \\ with the specified parameter. Warning: Exit does not prompt for a confirmation. fill (^) ¶ The dyadic fill ( ^ ) takes an atom as its first argument and a list or dictionary (target) as its second argument. For a list, it returns a list obtained by substituting the first argument for every occurrence of null in target. It operates on the range of a dictionary. 42^1 2 3 0N 5 0N 1 2 3 42 5 42 ";"^"Now is the time" "Now;is;the;time" `NULL^`First`Second``Fourth `First`Second`NULL`Fourth d:`a`b`c`d!100 0N 200 0N 42^d a| 100 b| 42 c| 200 d| 42 Observe that the action of fill is recursive - i.e., it is applied to sublists of the target. 42^(1;0N;(100;200 0N)) 42^ a| 100 b| 42 c| 200 d| 42 find (?) ¶ When the first argument (target) of find ( ? ) is a simple list, find is atomic in the second argument (source) and returns the positions in target of the initial occurrence of each item of source. The simplest case is when source is a scalar. 100 99 98 87 96?98 2 "Now is the time"?"t" 7 If source is not found in target, find returns the count of target - i.e., the position one past the last element. `one`two`three?`four 3 In this context, find is atomic in its second argument, so it is extended item-wise to a source list. "Now is the time"?"the" 7 8 9 Note that find always returns the position of the first occurrence of each atom. "Now is the time"?"time" 7 4 13 9 When the first argument (target) of find is a general list, find considers both elements to be general lists and attempts to locate the second argument (source) in the target, returning the position where it is found or the count of target if not found. (1 2;3 4)?3 4 1 Observe that find only compares items at the top level of the two arguments and does not look for nested items, ((0;1 2);3 4;5 6)?1 2 3 ((0;1 2);3 4;5 6)?(1;(2;3 4)) 3 When the first argument (target) of find is a dictionary, find represents reverse lookup and is atomic in the second argument (source). In other words, find returns the domain item mapping to source if source is in the range, or a null appropriate to the domain type otherwise. d:1 2 3!100 101 102 d 1| 100 2| 101 3| 102 d?101 2 d?99 0N d?102 100 3 1 When the first argument (target) of find is a table and the second argument (source) is a record of the target, find returns the position of source if it is in target, or the count of target otherwise. t:([] a:1 2 3; b:`a`b`c) t a b --- 1 a 2 b 3 c t?`a`b!(2;`b) 1 As usual with records, you can abbreviate the record to its row values. t?(3;`c) 2 When the first argument of find is a keyed table, since a keyed table is a dictionary, find performs a reverse lookup on a record from the value table. kt:([k:1 2 3] c:100 101 102) kt k| c -| --- 1| 100 2| 101 3| 102 kt?`c!101 k| 2 Again, a record of the value table can be abbreviated to its row value (s). kt?102 k| 3 flip ¶ The monadic function flip takes a rectangular list, a column dictionary or a table as its argument (source). The result is the transpose of source. When source is a rectangular list, the items are rearranged, effectively reversing the first two indices in indexing at depth. For example, L:(1 2 3; (10 20; 100 200; 1000 2000)) L 1 2 3 10 20 100 200 1000 2000 L[1;0] 10 20 fL:flip L fL 1 10 20 2 100 200 3 1000 2000 fL[0;1] 10 20 When source is a singleton list whose item is a simple list, flip creates a vertical list. flip enlist 101 103 101 103 This idiom is used to index multiple key values into keyed tables. kt:([k:101 102 103] c:`one`two`three) kt flip enlist 101 103 c ----- one three When source is a column dictionary, the result is a table with the given column names and values. Row and column access are effectively reversed, but no data is rearranged. d:(`a`b`c!1 2 3;1.1 2.2 3.3;("one";"two";"three")) d `a`b`c!1 2 3 1.1 2.2 3.3 ("one";"two";"three") d[`b;0] 1.1 t:flip d t a b c ----------- 1 1.1 one 2 2.2 two 3 3.3 three t[0;`b] 1.1 When source is a table, the result is the underlying column dictionary. Row and column access are effectively reversed, but no data is rearranged. t:([]a:1 2 3;b:1.1 2.2 3.3;c:("one";"two";"three")) t a b c ------------- 1 1.1 "one" 2 2.2 "two" 3 3.3 "three" t[1;`c] "two" d:flip t d a| 1 2 3 b| 1.1 2.2 3.3 c| "one" "two" "three" d[`c;1] "two" getenv ¶ The monadic function getenv takes a symbol argument representing the name of an OS environment variable and returns the value (if any) of that environment variable. getenv `SHELL "/bin/bash" group ¶ The monadic function group operates on a list (source) and returns a dictionary in which each distinct item in source is mapped to a list of the indices of its occurrences in source. The items in the domain of the result are in the order of their first appearance in source. group "i miss mississippi" i| 0 3 8 11 14 17 | 1 6 m| 2 7 s| 4 5 9 10 12 13 p| 15 16 This can be used to extract specific information about the occurrences, such as, dm:group "i miss mississippi" count each dm i| 6 | 2 m| 2 s| 6 p| 2 first each dm i| 0 | 1 m| 2 s| 4 p| 15 iasc ¶ The monadic function iasc operates on a list or a dictionary (source). Considering source as a mapping, the result of iasc is a list comprising the domain items arranged in increasing order of their associated range items. Otherwise put, retrieving the items of source in the order specified by iasc sorts source in ascending order. L:3 7 2 8 1 9 iasc L 4 2 0 1 3 5 L[iasc L] 1 2 3 7 8 9 d:`b`c`a!3 2 1 iasc d `a`c`b d[iasc d] 1 2 3 identity ¶ The monadic function denoted by double colon ( :: ), is the identity function, meaning that the return value is the same as the argument. ::[42] 42 ::[`zaphod] `zaphod ::["Life the Universe and Everything"] "Life the Universe and Everything" Note: The identity function cannot be used with juxtaposition or @. Its argument must be enclosed in brackets. :: 42 ' idesc ¶ The monadic function idesc operates on a list or a dictionary (source). Considering source as a mapping, the result of idesc is a list comprising the domain items arranged in decreasing order of their associated range items. Otherwise put, retrieving the items of source in the order specified by idesc sorts source in descending order. L:3 7 2 8 1 9 idesc L 5 3 1 0 2 4 L[idesc L] 9 8 7 3 2 1 d:`b`c`a!3 2 1 idesc d `b`c`a d[idesc d] 3 2 1 in ¶ The dyadic function in is atomic in its first argument (source) and takes a second argument (target) that is an atom or list. It returns a boolean result that indicates whether source appears in target. The comparison is strict with regard to type. 3 in 8 0b 42 in 0 6 7 42 98 1b "cat" in "abcdefg" 110b `zap in `zaphod`beeblebrox 0b 2 in 0 2 4j 'type inter ¶ The dyadic inter can be applied to lists, dictionaries and tables. It returns an entity of the same type as its arguments, containing those elements of the first argument that appear in the second argument. 1 1 2 3 inter 1 2 3 4 1 1 2 3 "ab cd " inter " bc f" "b c " Note: Lists are not sets and the operation of inter on lists is not identical to intersection of sets. In particular, the result of inter does not comprise the distinct items common to the two arguments. One consequence is that the expression, (x inter y)~y inter x is not true in general. When applied to dictionaries, inter returns the set of common range items that are mapped from the the same domain items. d1:1 2 3!100 200 300 d2:2 4 6!200 400 600 d1 inter d2 ,200 Tables that have the same columns can participate in inter. The result is a table with the records that are common to the two tables. t1 a b -------- 1 first 2 second 3 third t2 a b -------- 2 second 4 fourth 6 sixth t1 inter t2 a b -------- 2 second join (,) ¶ The dyadic join ( , ) can take many different combinations of arguments. When both operands are either lists or atoms, the result is a list with the item(s) of the left operand followed by the item(s) of the right operand. 2,3 2 3 `a,`b`c `a`b`c "xy","yz" "xyyz" 1.1 2.2,3 4 1.1 2.2 3 4 Observe that the result is a general list unless all items are of a homogeneous type. When both operands are dictionaries, the result is the merge of the dictionaries using upsert semantics. The domain of the result is the (set theoretic) union of the two domains. Range assignment of the right operand prevails on common domain items. d1:1 2 3!`a`b`c d2:3 4 5!`cc`d`e d1,d2 1| a 2| b 3| cc 4| d 5| e When both operands are tables having the same column names and types, the result is a table in which the records of the right operand are appended to those of the left operand. t1:([]a:1 2 3;b:`x`y`z) t1 a b --- 1 x 2 y 3 z t2:([]a:3 4;b:`yy`z) t2 a b ---- 3 yy 4 z t1,t2 a b ---- 1 x 2 y 3 z 3 yy 4 z When both operands are keyed tables having the same key and value columns, the result is a keyed table in which the records of the left operand are upserted with those of the right operand. kt1:([k:1 2 3]v:`a`b`c) kt1 k| v -| - 1| a 2| b 3| c kt2:([k:3 4]v:`cc`d) kt2 k| v -| -- 3| cc 4| d kt1,kt2 k| v -| -- 1| a 2| b 3| cc 4| d join-each (,') ¶ The verb join ( , ) can be combined with the adverb monadic each ( ' ) to yield join-each ( ,' ), which can be used on lists, dictionaries or tables. List operands must have the same count. L1:1 2 3 L2:`a`b`c L1,'L2 1 `a 2 `b 3 `c As always with dictionaries, the operation occurs along the common domain items, with null extension elsewhere. d1:1 2 3!10 20 30 d2:2 3 4!`a`b`c d1,'d2 1| 10 ` 2| 20 `a 3| 30 `b 4| 0N `c For two tables with the same count of records, join-each results in a column join (Column Join), in which columns with non-common names are juxtaposed and overlapping columns are upserted. t1:([]c1:1 2 3;c2:1.1 2.2 3.3) t1 c1 c2 ------ 1 1.1 2 2.2 3 3.3 t2:([]c2:`a`b`c;c3:100 200 300) t2 c2 c3 ------ a 100 b 200 c 300 t1,'t2 c1 c2 c3 --------- 1 a 100 2 b 200 3 c 300 Note: When join-each is used in a select, it must be enclosed in parentheses to avoid the comma being interpreted as a separator. select j:(c1,'c2) from t1 j ----- 1 1.1 2 2.2 3 3.3 list ¶ The function list replaces plist. It XE "list (function)" takes a variable number of arguments and returns a list whose items are the arguments. It is useful for creating lists programmatically. Note: Unlike user-defined functions, the number of arguments to list is not restricted to eight. For example, list[6;7;42;`Life;"The Universe"] 6 7 42 `Life "The Universe" list[1;2;3;4;5;6;7;8;9;10] 1 2 3 4 5 6 7 8 9 10 null ¶ The atomic function null takes a list (source) and returns a binary list comprising the result of testing each item in source against null. null 1 2 3 0N 5 0N 000101b null `a`b``d```f 0010110b Since null is atomic, it is applied recursively to sublists. null (1 2;3 0N) 00b 01b It is useful to combine where with null to obtain the positions of the null items. where null 1 2 3 0N 5 0N 3 5 When applied to a dictionary (source), null returns a dictionary in which each item in the source range is replaced with the result of testing the item against null. null 1 2 3!100 0N 300 1| 0 2| 1 3| 0 The action of null on a table (source) is explained by recalling that the table is a flipped column dictionary. Based on the action of null on a dictionary, we expect the result of null on a table will be a new table in which each column value in the source is replaced with the result of testing the value against null. tnull:([]a:1 0N 3; b:0N 200 300) null tnull a b --- 0 1 1 0 0 0 Similarly, we expect null to operate on a keyed table by returning a result keyed table whose value table entries are the result of testing those of the argument against null. ktnull:([k:101 102 103];v:`first``third) null ktnull k | v ---| --- 101| 0 102| 1 103| 0 parse ¶ The monadic function parse takes a string argument containing a valid q expression and returns a list containing the corresponding parse tree. Applying the function eval to the result will evaluate it. A discussion of q parse trees is beyond the scope of this tutorial. .Q.s1 parse "a: 6*7" "(:;`a;(*;6;7))" eval parse "a: 6*7" 42 Note: It is useful to apply parse to a query template in order to discover its functional form. The result is not always exactly the functional form, especially for exec, but a little experimenting will lead to the correct form. t:([]c1:`a`b`a; c2:1 2 3) select c2 by c1 from t c1| c2 --| --- a | 1 3 b | ,2 parse "select c2 by c1 from t" ? `t () (,`c1)!,`c1 (,`c2)!,`c2 ?[t;();(enlist `c1)!enlist `c1;(enlist `c2)!enlist `c2] c1| c2 --| --- a | 1 3 b | ,2 exec c2 by c1 from t a| 1 3 b| ,2 parse "exec c2 by c1 from t" ? `t () ,`c1 ,`c2 ?[t; ();`c1;`c1] a| `a`a b| ,`b rand (?) ¶ The dyadic function rand ( ? ) is overloaded to have different meanings. In the case where both arguments are numeric scalars, ? returns a list of random numbers. More specifically, the first argument must be of integer type, and the second argument can by any numeric value. In this context, ? returns a list of pseudo-random numbers of count given by first argument. In case the second argument is a positive number of floating point type and the first argument is positive, the result is a list of random float selected with replacement from the range between 0 (inclusive) and the second argument (exclusive). 5?4.2 3.778553 1.230056 1.572286 0.517468 0.07107598 4?1.0 0.5274765 0.5435815 0.4611484 0.7493561 In case the second argument is of integer type and the first argument is positive, the result is a list of random integers selected with replacement from the range between 0 (inclusive) and the second argument (exclusive). 10?5 1 2 0 3 4 4 4 0 3 1 10?5 0 2 1 0 2 4 2 3 4 0 1+10?5 4 2 3 3 3 2 1 1 5 3 The last example shows how to select random integers between 1 and 5. More generally, for integers i and j, where i<j, and any integer n, the idiom, i+n?j+1-i selects n random integers between i and j inclusive. i:3 j:7 n:10 i+n?j+1-i 3 4 5 7 7 5 4 4 7 4 In case the second argument is of integer type and the first argument is negative, the result is a list of random integers selected without replacement from the range between 0 (inclusive) and the second argument (exclusive). Since the selected values are not replaced, the absolute value of the first argument cannot exceed the second argument, -3?5 2 3 0 -5?5 4 1 2 0 3 -6?5 'length raze ¶ The monadic raze takes a list or dictionary (source) and returns the entity derived from the source by eliminating the top-most level of nesting. raze (1 2;`a`b) 1 2 `a `b One way to envision the action of raze is to write the source list in general form, then remove the parentheses directly beneath the outer- most enclosing pair. raze ((1;2);(`a;`b)) 1 2 `a `b Observe that raze only removes the top-most level of nesting and does not apply recursively to sublists. raze ((1 2;3 4);(5;(6 7;8 9))) 1 2 3 4 5 (6 7;8 9) If source is not nested, the result is the source. raze 1 2 3 4 1 2 3 4 When raze is applied to an atom, the result is a list. raze 42 ,42 When raze is applied to a dictionary, the result is raze applied to the range. dd:`a`b`c!(1 2; 3 4 5;6) raze dd 1 2 3 4 5 6 reshape (#) ¶ When the first argument of the dyadic reshape ( # ) is a list (shape) of two positive int, the result reshapes the source into a rectangular list according to shape. Specifically, the count of the result in dimension i is given by the item in position i in shape. The elements are taken from the beginning of the source. A simple example makes this clear. 2 3#1 2 3 4 5 6 1 2 3 4 5 6 As in the case of take, if the number of elements in the source exceeds what is necessary to form the result, trailing elements are ignored. 2 2#`a`b`c`d`e`f`g`h a b c d Similarly, if the number of elements in the source is less than necessary to form the result, the extraction resumes from the initial item of the source; this process is repeated until the result is complete. 5 4#"Now is the time" "Now " "is t" "he t" "imeN" "ow i" It is possible create a ragged array of any number of columns by using 0N as the number of rows with the reshape operator ( # ). 0N 3#til 10 0 1 2 3 4 5 6 7 8 ,9 reverse ¶ The monadic reverse inverts the order of the constituents of its argument. In the case of an atom, it simply returns the argument. reverse 42 42 In the case of a list, the result is a list in which the items are in reverse order of the argument. reverse 1 2 3 4 5 5 4 3 2 1 For nested lists, the reversal takes place only at the topmost level. reverse (1 2 3; "abc"; `Four`Score`and`Seven) `Four`Score`and`Seven "abc" 1 2 3 In the case of an empty list, reverse returns the argument. reverse () () In the case of a dictionary, reverse inverts both the domain and range lists. reverse`a`b`c!1 2 3 c| 3 b| 2 a| 1 Since a table is a list of records, reverse inverts the order of the records. t:([] c1:`a`b`c; c2:1 2 3)
t c1 c2 ----- a 1 b 2 c 3 reverse t c1 c2 ----- c 3 b 2 a 1 Since a keyed table is a dictionary, reverse inverts both the domain and range tables, effectively inverting the row order. kt k| c -| --- 1| 100 2| 101 3| 102 reverse kt k| c -| --- 3| 102 2| 101 1| 100 sublist ¶ The dyadic function sublist retrieves a sublist of contiguous items from a list. The left operand is a simple list of two ints: the first item is the starting index (start); the second item is the number of items to retrieve (count). The right operand (target) is a list or dictionary. If target is a list, the result is a list comprising count items from target beginning at index start. L:1 2 3 4 5 1 3 sublist L 2 3 4 If target is a dictionary, the result is a dictionary whose domain comprises count items from the target domain beginning at index start, and whose range is the corresponding items in the target range. d:`a`b`c`d`e!1 2 3 4 5 1 3 sublist d b| 2 c| 3 d| 4 Since a table is a list of records, sublist applies to the rows of a table. t:([]c1:`a`b`c`d`e;c2:1 2 3 4 5) 1 3 sublist t c1 c2 ----- b 2 c 3 d 4 Since a keyed table is a dictionary, sublist is applied to the key table. kt:([k:`a`b`c`d`e]c1:1 2 3 4 5) 1 3 sublist kt k| c1 -| -- b| 2 c| 3 d| 4 system ¶ The monadic system takes a string argument and executes it is a q command, if recognized, or an OS command otherwise. The function system is equivalent to \\ but can be more convenient or readable in situations such as remote or programmatic execution in which the backslashes must be escaped. The following changes the current working directory to its sparent directory. system "cd .." take (#) ¶ When the left operand of take ( # ) is an int atom, it creates a new entity via extraction from its right operand (source) as specified by the first operand. A positive integer in the first operand indicates that the extraction occurs from the beginning of the source, whereas a negative integer in the first operand indicates that the extraction occurs from the end of the source. The source can be an atom, a list, a dictionary, a table or a keyed table. 2#3 3 3 -1#10 20 30 40 ,40 -2#`a`b`c`d!10 20 30 40 c| 30 d| 40 3#([] a:10 20 30 40; b:1.1 2.2 3.3 4.4) a b ------ 10 1.1 20 2.2 30 3.3 1#([k:10 20 30] c:`one`two`three) k | c --| --- 10| one The result of take is of the same type and shape as the source, except the result is never a scalar. 1#42 ,42 If the number of elements in source exceeds what is necessary to form the result, trailing elements are ignored. 4#`a`b`c`d`e`f`g`h `a`b`c`d If the number of elements in source is less than necessary to form the result, the extraction resumes from the starting point of the source list; this process is repeated until the result is filled. 5#98 99 98 99 98 99 98 -7#`a`b`c `c`a`b`c`a`b`c In the degenerate case, the result is an empty entity with the same type as the source. This is an effective way to obtain the schema of a q dictionary or list. 0#42 `int$() 0#10 20 30 40 `int$() 0#`a`b`c`d!10 20 30 40 _ 0#([] a:10 20 30 40; b:1.1 2.2 3.3 4.4) a b --- 0#([k:10 20 30] c:`one`two`three) k| c -| - Note: Since the result of 0# on a list is always a list, we can use this construct as shorthand to initialize an empty value column with a definite type in a table definition. This ensures that only values of the specified type can be inserted into the column. For example, ([] a:0#0; b:0#`) a b --- defines an empty table whose first column is of type int and whose second column is of type symbol. When the left operand of # is a list of symbol column names and the right operand is a table, the result is the table obtained by extracting the specified columns from t. t:([] c1:`a`b`c; c2:1 2 3; c3:1.1 2.2 3.3)
|
