M Language Tutorial - The cryptic style

In order to save typing at cost of readability, every command and built in function of M has abbreviation/shortcut.
The following examples are perfectly valid

f i=1:1:9 s sqtable(i)=i*i; 'f' stand for 'for' and 's' for 'set'

and

s i="" f  s i=$o(sqltable(i)) q:i=""  w i," ",sqltable(i),!;'q' stand for 'quit' '$o' for '$order', 'w' for 'write'

As we spend most of our time reading code written by others, modern programming style does care much about naming thing. The abbreviated style of most if M code is really pain especially for novices.

M Language Tutorial - for and Array

for command

GTM>for i=1:1:5 write i,!
1
2
3
4
5

The above for loop a variable i from 1 to 5, increasing by 1 in each iteration.
This one will loop over a list of arguments

GTM>for i="hello","world","bye","moon" write i,!
hello
world
bye
moon

array
In M array is stored as sparse B-tree structure, index or subscript can be anything and can be in any number

GTM> 
GTM>set a(1)="hello",a(2)="world" 
GTM>write a(1)," ",a(2)
hello world
GTM>
GTM>write a(3)         
%GTM-E-UNDEF, Undefined local variable: a(3)

the index of an Array can be any thing so it is like hash/dictionary in other language.

GTM>set a("hello")=1

GTM>set a("world")=2

GTM>write a("hello")
1
GTM>write a("world")
2
GTM>write a("moon") 
%GTM-E-UNDEF, Undefined local variable: a(moon)

We can use any number of subscripts

GTM>set a(1,2)="moon"

GTM>set a(1,1)="world"

GTM>write a(1,1)
world
GTM>write a(1,2)
moon
GTM>write a(1,0)
%GTM-E-UNDEF, Undefined local variable: a(1,0)

built-in function $order is used to get index/subscript of an element of an Array, that is particularly useful for traversal over an Array.

GTM>set b(3)="hello",b(5)="world",b("hello")=1,b("world")=2

GTM>write $order(b("")); with empty string we get a subscript of the first element of an Array
3
GTM>write $order(b(3)); passing one element we get a subscript of next element  
5
GTM>write $order(b(5))
hello
GTM>write $order(b("hello"))
world
GTM>write $order(b("world")); passing the last element we get empty string 

If array is multi dimensional, $order will return subscript of an element in one dimension

GTM>set a("h",1)="hello",a("w",1)="world"

GTM>write $order(a("")); return subscript in the first dimension of first element 
h
GTM>write $order(a("h")); return subscript in the first dimension of second element 
w
GTM>write $order(a("w"))

GTM>write $order(a("h","")); return subscript in the second dimension of first element  
1
GTM>write $order(a("h",1)) 

GTM>write $order(a("w","")); return subscript in the second dimension of second element 
1
GTM>write $order(a("w",1)) 

traversal over a array
for is typical command used to traversal an array

GTM>kill a for i=1:1:10 set a(i)=i*i

We use kill to erase content of the variable a if exists and a for to create 10 elements.

GTM>set i="" for  set i=$order(a(i)) quit:i=""  write i," ",a(i),!; two SPACE after for and two SPACE after quit:i=""
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
10 100

Here, the argument-less For repeats until stopped by a terminating quit. This line prints a table of i and a(i)
block of multi line of codes under for
M support execute multi line of code under for in the routine

GTM>zedit "nested"
finish
 for name="ivan","john","janes" do
 . if name]"jo"; the operator ] test if name follows string "jo"
 . else  write name,!
 quit

GTM>zlink "nested"

GTM>do finish^nested
ivan
janes
GTM>

M Language Tutorial - if, else and boolean condition

M language has no concept of structure that are familiar in other programming languages. In M all are commands including if,else,for.
if and else commands

GTM>set x=1

GTM>if x=1 write "boom"; if condition is satisfied
boom
GTM>if x=2 write "bang";  if condition is not satisfied

The if command support AND , OR, NOT condition

GTM>set x=1,y=2

GTM>if x=1,y=1 write "bang" ; we use , to denote AND

GTM>if x=1,y=2 write "bang"
bang

GTM> if (x=1)&(y=2) write "bang"; other way to express AND is to use &
bang

GTM>if x=1!y=1 write "boom"; we use ! to denote OR 
boom
GTM>if 1=0 write "oh" 

GTM>if '1=0 write "oh"; we use '  to denote a negation
oh

Internally, if set a special value $TEST to 1 (TRUE) or 0 (FALSE) depending on whether the condition is satisfied or not. $TEST is set to 1 at the start of gtm. The if command without condition will use $TEST as condition.

$gtm
GTM>write $TEST
1
GTM>if 1=0

GTM>write $TEST
0
GTM>if  write "hello"! note two SPACEs after if

GTM>if $TEST=1 write "hello"! equivalent to above

GTM>if 1=1

GTM>write $TEST
1
GTM>if  write "hello"; note two SPACEs after if
hello
GTM>if $TEST=1 write "hello"; equivalent to above
hello

else is command that perform the command followed it if the $TEST variable is 0

GTM>set x=1

GTM>if x>1 write "world"

GTM>else  write "moon"; note two SPACES after else
moon
GTM>if $TEST=0  write "moon"; equivalent to above
moon

command's postcondition
execution of almost all commands can be controlled by following it with a colon and a truthvalue expression.

GTM>set n=1
GTM>write:n>0 "hello"
hello
GTM>write:n>1 "world"
GTM> 

block of multi line of codes under if,else
M support execute multi line of code under if,else, for in the routine

GTM>zedit "nested"
start(x,who)
 if x=1 do  ;two SPACE after do
 . write "hello",!
 . write who
 else  do  ;two SPACE after else and do
 . write "bye",!
 . write who
 quit

GTM>zlink "nested"

GTM>do start^nested(1,"world")
hello
world
GTM>do start^nested(0,"world")
bye
world

operators for string' comparison
These operators are a little bit strange comparing to other languages

GTM>WRITE "A"="B"; compare if two string are equal
0
GTM>WRITE "C"="C"
1
GTM>WRITE "A"["B"; this is same as contains in other language
0
GTM>WRITE "ABC"["C"
1
GTM>WRITE "A"]"B"; this is same as > in other language
0
GTM>WRITE "B"]"A"
1

NOT operator can applied to either expression or other operator

GTM>write "A"="B" 
0
GTM>write '("A"="B")
1
GTM>write "A"'="B"
1

M Language Tutorial - routines

Divide and conquer is the technique used to counter complexity from the beginning of software development history, A complex system in M language comprise of many files called routines.
label, quit and function
In general each line of code in a routine contains of one or no label following by a SPACE or TAB and then M command, other M code can make a call to any line of the routine that has a label at start of the file. Let create a file display.m in one of these paths specified in env. variable gtmroutines using any editor.

start
 write "display",!
 quit
other 
 write "other",!
nothing
 write "no",!

Now start gtm and try to make some call

$gtm
GTM>zlink "display"
GTM>do ^display
display
GTM>do start^display
display
GTM>do start+1^display
display

The name after ^ is name of routine same as filename without extension .m. Without any label, GTM will execute the code starting from line 1 of the routine. It does exactly the same with label start. Making a call to a line 1 from the label start mean start the execution from line 2 of the file. The execution terminate at command quit.

GTM>do start+3^display
other
no
GTM>do other^display  
other
no
GTM>do nothing^display
no

Calling a line 3 from label start is the same as calling label other. Because there is no quit command, GTM continues the execution at the end of file.
There is good practice always structure a routines as series of sections starting with a label and ending with a quit command. That way we can consider each label as a function name when making a call. Using offset from a label is considered a bad practice as it decrease readability of the code.
When calling a function within the same routine, we can remove the ^filename

talkto(who)
 do say("Hello",who); call a function in the same routine
 write "bla bla",!
 do say("Byte",who)
 quit

say(what,who)
 write what," ",who,!
 quit
GTM>zlink "stuff"

GTM>do talkto^stuff("John")
Hello John
bla bla
Byte John

calling function, passing parameters, return value
We pass parameters when calling M routine in parentheses separated by comma, the leading period . is used to indicate a parameter being passed as reference that is used to store the output of function.

$gtm
GTM>zedit "calc"
calc(a,b,ret)
 set ret=(2*a)+(3*b)
 quit

Now make a call

 
GTM>zlink "calc"
GTM>do ^calc(1,2,.result)
GTM>write result
8

M provides a facility called Extrinsic Variable to create a function that return value so it can be used in a expression

$gtm
GTM>zedit "calc"
calc(a,b,ret)
 set ret=(2*a)+(3*b)
 quit

othercalc(a,b)
 quit (2*a)+(3*b); the function must put return value  after quit command

GTM>zlink "calc"
GTM>write $$othercalc^calc(1,2); put $$ before the name of function when calling
8
GTM>set x=$$calc^calc(2,3)     
%GTM-E-QUITARGREQD, Quit from an extrinsic must have an argument

M Language Tutorial - Getting started

I am learning MUMPS/M language to understand one of our systems, that has been implemented using M. I feel, there is a lack of a documentation in form of quick start and tutorial, so I put some notes here hoping that it will be useful for someone.
For a thorough listing of the M language commands, operators, functions and special variables, see MUMPS by Example
One of M implementations available as open source is GTM from FIS, we can download it from http://sourceforge.net/projects/fis-gtm/. The documentation is available from here
Setup a environment to start some test is fairly simple, just unzip the downloaded file, run configure and answer few questions, then we can start
setting env. for running GTM

$source ./gtmprofile

This shell will set various env, variables required by GTM also create a global directory and a default datafile, that is actually M database. M is a language with built in persistence. The global directory is kind of control file in Oracle.
start gtm - the interpreter

$gtm
GTM>
GTM>write "hello world"
hello world
GTM>halt

M use write command to output some thing to a console, in Ruby we would use puts in Python, this is print. The command halt is used to quit the gtm.
For a beginner of any language, the ability to write something out to see and to exit are the two most important commands.
string and numbers
String is enclosed by " as in C, Java, Python, Ruby, to concatenate two or more string we use operator _, not that common any more.

GTM>set x="hello"
GTM>set y="world"
GTM>set z=x_" "_y
GTM>write z
hello world

The command set set these local variables, these variables are created automatically if not exist.

GTM>set a=10
GTM>set b=20
GTM>set c=(a+b)*5/10
GTM>write c
15

For number, this is quite straightforward, no surprise.

create and run the first program

$export gtmroutines=/home/gtm/samples
$source ./gtmprofile
$gtm
GTM>write $ZROutines
/home/gtm/samples

this gtmroutines env. variable specifies where GTM is going to store and search for its routines, inside GTM however it is kept in variable $ZROutines. The concept routine in GTM is synonym for a single file containing M code.

GTM>zedit "hello"

This will popup a vi editor with opened file /home/gtm/samples/hello.m. Create the following content, save and quit vi editor.

hello(who)
  write "Hello ",who,!
  quit

In the routine hello.m we see a label hello(who), which is calling entry point. GTM's routine can have many labels and the general syntax of calling a section of code in the routine is label^routine. The label hello will take one parameters. The write above command takes 3 parameters separated by comma where ! means new line. It is possible to write the above code in a single line

hello(who) write "Hello ",who,! quit

The quit at the end is important in case other labels are added below if we want a label to behave as a function, which mean that the execution flow in the routine terminates at quit command .

GTM> zlink "hello"

The zlink compiles and link hello.m to GTM image so we can call this routine in GTM environment

GTM>do hello^hello("Moon")
Hello Moon

If we do not specify a label then the code will be executed from the first line.

GTM>do ^hello("World")
Hello World

If we look closely to gtm, we will see that gtm is shell script that call the binary mumps in direct mode (with parameter -direct).
We can run the routine hello.m from command line as follow

$export gtmroutines="/home/gtm/samples/ ."
$mumps -run %XCMD 'do ^hello("World")'

The %XCMD is in fact a routine _XCMD.m located in distribution directory of GTM, that is why we need add "." into gtmroutines env. variables, so mumps knows where to find the files being executed.
language syntax
M syntax is very strict, SPACE characters between M statements are significant. A single space separates a command from its argument, COMMA "," is used to separate one argument from other of these commands taking more than one arguments.
A SPACE, or newline, separates the command's arguments from others. Commands which take no arguments (e.g., ELSE) require two following spaces.
Character ; is used to indicate start of a comment, that runs until end of line.

GTM>write  "hello"; two SPACES after write
%GTM-E-EXPR, Expression expected but not found
 write  "hello"
       ^-----
GTM>write "hello"; single SPACE after write
hello
GTM>set x=1,y=2 write "x=",x,",y=",y !set and write commands are on the same line separated by SPACE
x=1,y=2
GTM>set x=1 + 2; SPACE surrounding + 
%GTM-E-CMD, Command expected but not found
 set x=1 + 2
         ^-----
GTM>set x=1+2; no SPACE in expression
GTM>write x
3
GTM>set x= 9 ; SPACE after =  
%GTM-E-EXPR, Expression expected but not found
 set x= 9
       ^-----
GTM>set x=9; no SPACE after=
GTM>write x
9

Math expression
Comparing to other language, math operator has no order precedences, a expression is evaluated from left to right

GTM>write 1+2*3
9
GTM>write (1+2)*3
9

Parentheses has to be used to make thing work as expecting.

GTM>write 1+(2*3)
7

Velocity conference 2010

Velocity conference 2010 Site
Videos

Automatically dump your Thread´s StackTrace whenever something get wrong

JVM either from SUN or IBM has features that allows us to do heap dump and thread stack trace dump. They can even do it automatically when there is OutOfMemory error.
But there is plenty of others situations that StackTrace of Threads can be useful e.g. when your application code enter into infinite loop or it forgets to free some resources or all threads of its thread pools are busy. In such cases Thread´s StackTrace are extremely valuable in root cause analysis.
Create StackTrace dump is also much cheaper and faster than Heap dump so that is why we should do it automatically in many situation when we suspect that something gets wrong.
The most easy way to do it to write a script that trigger a Thread´s StackTrace dump by invoking kill -3 jvm_pid (or jstack jvm_pid) when something happen( e.g. JVM use more then 90% CPU).

6 months in ING Direct Spain

Today marks 6 months of my tenure in ING Direct Spain. So it is helpful to have some reflections.
1. My job is primarily administrative. During the time, I was involved in a couple of troubleshooting issues, for which we are able to find out a root cause and proposed a conclusive solution.

2. The second thing is my effort to refactor our JBoss infrastructure and to use git to track and manage the change. One lesson I learned from that is not matter what sophisticated we are using, the key of success is ability a) to figure out what is specific and what is common and b) to reduce number of configuration items.
At the beginning I tried to track configuration items on each machine as one git branch but it turns out into a horror of hundreds of un-manageable branches. At the end I thrown it and start to divide the monolithic configuration into smaller parts, parametrize them for reuse. After that using git or any other VCS for tracking them is just as walk in a rose's garden.

3. The third thing I have started is documentation of our infrastructure, I decided to use wiki for doing it. It turned out to be a good choice. The document is very up to date and practical because I use the document during my daily work so whenever I find a gaps between the document and the reality, I correct it. Update documentation is not separated task but a part of my day-to-day activities.

4. An the last thing is my research to look for a central authentication solution for our unix farm of mix LINUX, AIX and Solaris. I have tested PAM & Open LDAP combination for a while but at the end I have selected and recommended Likewise with the existing Active Directory as authentication server due to a lower cost of installation and operation.

java.lang.OutOfMemoryError Out of swap space?

I have encountered this error in one of our Hotspot JVMs. Googling for a while without definitive result, I decided to look at source code of Hotspot VM (happily the source code is available in Sun website).
Within few minutes, with very little effort, I found that JVM print out this message when got NULL from calling malloc. So it turns out to find why malloc fails.
When OS does not have RAM to satisfy malloc, it may try to swap out unused real memory to disk and if the swap space is configured too small, the error appears. But this is just one cause.
Other more likely cause is that JVM run out of memory address space and this is our case. It can happen when we run 32 bit JVM and total memory used by the JVM exceeds magical limit of 3 GB (on 32 bit kernel LINUX).