Le Huy on software design, pattern, DSL and distributed system

Craft a solid chef-server

2011-08-10T05:22:00.010+09:00

The out of box chef server installation is enough to support large scale deployment . To handle hundreds of nodes we need to craft it a bit. I highlight here some configuration tweaks
1. Run multi instances with Apache or Nginx http server as front end.
chef-server-api and chef-server-webui is single process, the only way to scale them are run multi instances of them behind the front end reverse http proxy server.
Also the front end http server can be configured to perform ssl encryption making communication between nodes and chef-server-api secure. Note that chef-server-webui is also chef-client, so use URL of the reverse proxy server to communicate with chef-server-api.
2. Use ruby 1.9
chef server components are hungry to CPU, use ruby 1.9 can boost performance as much as twice comparing to ruby 1.8.
3. Couchdb and big hard disk
Avoid un necessary headache by installing latest version of Couchdb. The version that comes with OS is usually old, buggy, often crashes under high load with hundred GB database. Hard disk is cheap, keep space for Couchdb data files, log files always free around 30%.
Also learn Erlang a bit so you can look and understand crash dump file from Couchdb as well as change its configuration.
4. Solr server
Chef-solr is de facto Solr server for full text search, The default config is pretty small for any serious operation. As it is java application based on Lucene, we need increase java heap size. You also need modify the Solr config (see this tip) to make full text index run faster. Believe me , you will have to rebuild solr index more than you think and proper solr config will save days of frustrated waiting.
5. Rabbitmq-server
Follow the same advice as for Couchdb. Rabbitmq-server is also Erlang application. It need enough memory to work reliably. Give it few GB of RAM if you do not want it crash and then will have to rebuild solr index.
6. Not enough
Distribute load by running chef-client at different deterministic time avoiding run chef-client of all nodes at the same time.
Consider running each of chef-server component on separate box even one component in many boxes. But don't do it if you have not try all above options as the more complex configuration, the more time you have to spend on it and you end up to serve the chef instead of it serves your infrastructure.
7. Put chef-server configuration in a cookbook
Create your own cookbook to automate all the changes you make on chef-server so you can recreate exact new chef-server using chef-solo in matter of minute (when needed). At the end, who will believe that you will be able to automate the infrastructure if you can not automate your own stuff.

How a packet travels from 3COM NIC to udp socket

2011-07-31T23:55:00.006+09:00

Migrate configuration to Chef

2011-05-25T14:38:00.024+09:00

One of big headache in implementation Chef or any automate configuration management tool is migration. We usually have to face with a large legacy configuration that is poorly designed and implemented, which make thing worse.
I want to share some of our own experiences in doing migration of a relatively large configuration in term of both scale and complexity.
The tools and techniques
They are
1. a decent, fast and rock solid version control system: I recommend git.
2. semantic diff tools: diff of text file is OK, but not enough, depending on format of your configuration (XML, properties, yaml, JSON), you may need to write your own semantic diff, that is able to make intelligent diff of these formats.
3. source code of these applications that use these configuration: the migration involve legacy applications, access to source code is absolute need to order to do a refactoring.
4. collaboration: with the new tool developers need to change/cleanup the code that access configuration data, operation people need to change their practices, we all need to learn how to use, what to trust, where we need to pay attention.
Shadow configuration file
In order to minimize impact existing system, at the beginning we let chef to create shadow configurations, which then are verified with existing configuration using diff tools. The shadow configuration generated by chef will replace the actual one only after we make sure that they are semantically the same from the point of view of the program using it.
In shadow configuration we can have a configuration file with different prefix/postfix or different directory. e.g. if the actual configuration file is /etc/hosts then the shadow one will be /etc/hosts.chef or /etc-chef/hosts or /root-chef/etc/hosts. I personally prefer the complete shadow directory because it is easier to check, remove when needed.
The cookbook need to have one attribute (can be name of directory where we are going to generate configuration files) saying if we are going to generate shadow or actual configuration.
Conflicting changes
In ideal world, we just has one configuration management system and one administrator who modify the system. But reality is different. There are usually more actors that make the change. So there is potential conflict. E.g. Chef modify one file and then an administrator or scripts or other configuration system don't know and also modify it. The worst case is that everyone assume that the file has the content he expect and thing get broken. These scripts that work for many years suddenly fails.
How to minimize the conflict?. Communicate well with other team members what are managed by Chef, which they should modify through Chef and not manually or by other tools. Various methods can be used e.g. a) wiki pages, b) put a clear notice as comment at the beginning of all files managed by Chef such as "This file is generated by Chef, you shall not modify it as it will be overwritten" c) notify all people each time a file is modified by Chef.
Top-down vs. bottom-up
There is basically two approaches to start the migration project 1) top-down and 2) bottom-up.
1. In top-down approach, you start with create a cookbook/recipe for one configuration item (e.g. syslog-ng, the cookbook will be fairly generic so it is usable by all environments) . Then extract specific data (e.g. ip of syslog server, source of log) from configuration file from each server and put them as attributes into a role. After finishing one configuration item, you continue with other until you have all you need in Chef.
2. With bottom-up approach, we first create one generic cookbook/recipe per server role (e.g. middleware) then you add all configuration files from servers of all environments to it. The structure of cookbook files can be like

      middleware/files/default   # files that are same for all hosts of all env.
      middleware/files/default/dev   #  files that are same for all hosts of development env.
      middleware/files/default/dev/host1   # files that are specific for host1 of evelopment env. 
      middleware/files/default/dev/host2
      middleware/files/default/qa
      middleware/files/default/pro

The recipe can be easily developed to copy these configuration files to a server depending on environment and hostname (node[:hostname]). After everything are in Chef and work well, we will start refactor and split this big generic cookbook to many smaller cookbooks, adding more roles and make thing easily to reuse. This step is kind of continual improvement process aiming at making our chef repository better and better.

In our project we have taken the first approach because it seems more naturally at first sight, however later we change to the second one, which I think have many benefits.
With the second approach, after we put all things in the Chef (can be done pretty quickly), we can announce that from now all changes has to be done through Chef. The very first result is seen immediately: a) there is single point of making change b) change is well audited and communicated as Chef repository is under version control system c)conflicting changes is minimized

Chef - Different between method defined in recipes and libraries

2011-05-22T04:06:00.006+09:00

There is one thing that need an attention when developing a recipe. If we create a method in recipe then that method is not available inside resource block's parameter. I will show it in the following example using Shef, the Chef interactive Ruby console.

[root@localhost gems]# shef

Ohai2u root@localhost!
chef > recipe
chef:recipe >   # here we are inside of a recipe
chef:recipe > def account(path) # create a method
chef:recipe ?>    return 'nobody' if File.dirname(path) == '/tmp'
chef:recipe ?>    return 'root'
chef:recipe ?> end
 => nil 
chef:recipe > file "/tmp/file.test" do
chef:recipe >      action :create
chef:recipe ?>     owner account(name) # try to use this method inside the block and we got a error
chef:recipe ?> end
NoMethodError: undefined method `account' for Chef::Resource::File
 from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/lib/chef/resource.rb:84:in `method_missing'

One way to workaround is to call the method outside of resource block's parameter

chef:recipe > fname = "/tmp/file.test"
 => "/tmp/file.test" 
chef:recipe > account_val = account(fname)
 => "root" 
chef:recipe > file fname do
chef:recipe >   action :create
chef:recipe ?>  owner account_val
chef:recipe ?> end
chef:recipe >

Other way is to define it as library (put it inside the directory libraries), in shef we would do like that

chef:recipe > exit 
 => :recipe
chef >  # here we are in top context, in which libraries are loaded 
chef > def account(dirname)
chef ?>    return 'nobody' if File.dirname(path) == '/tmp'
chef ?>    return 'root'
chef ?> end
 => nil 
chef > recipe
chef:recipe >   # here we are inside of a recipe
chef:recipe > file "/tmp/file.test" do
chef:recipe >    action :create
chef:recipe ?>   owner account(name) # now it is OK
chef:recipe ?> end
chef:recipe >

Some experiences in using Opscode Chef

2011-04-22T23:05:00.053+09:00

I am reaching the end of our Chef's implementation project, in which Chef is deployed to manage hundreds of Apache and Jboss servers (Redhat) hosting complex and dynamic banking application.
So it is worth-wise to recap some issues, problems encountered and solutions employed during the course of implementation.
Chef vs other tools
The project actually started from around Aug/Sep last year with the decision to start with Chef instead of other tools Puppet, CF Engine. I choose Chef because CF Engine is just too old while Puppet is too complex, both require to learn new language for expressing configuration and they missed GUI. Other quite subjective reason is that I know Ruby better comparing to other.
The challenges
The technical challenges include the complexity of the configuration being migrated to Chef as well its ever changing nature during the course of implementation. As Chef represents paradigm's shift to what we called infrastructure as code, the most difficult part is to persuade the team to change their habit and to adopt software development practices in their work.
Refactoring
Our environment is complicated and legacy. It was created over years by different guys manually with a bundle of shell scripts, that are far from good in quality from software engineering perspective (consistency, less duplication, good naming). So the implementation also mean refactoring the environment to something more viable in order to automate.
Cookbooks, roles are developed in parallel with renaming of configuration files/directories, cleaning up obsolete stuffs while adding support for new developed applications, environments, at the same time making sure not to break any things.
Single vs multi instances
At the beginning of the project there is no environment supported in Chef so there is two options a) maintain separate Chef instance per environment b) embed environment's information into role. As we have too many environments (development, preproduction, quality assurance, training, laboratory, DRS, production), it would be less work to select second option.
Cookbooks are developed and shared between all environment while roles are created per environment. I use e.g. the following naming convention for role: dev_apache_internet_os to denote role containing attributes for OS related cookbooks of Apache server in Internet zone.
Roles as Ruby code
We have nearly hundred of roles, a server is assigned to at least 2 roles one of OS and other of application specific. That is for separate the concern. OS role will has attributes more less related to OS e.g. pam, ssh, hosts, dns, ntp, postfix, route while application specific are e.g. Apache, Java, Tomcat, Jboss, log4j, etc.
To make it easy to manage; a subdirectory is created for each environment and roles are placed in it. The role format is Plain Old Ruby instead of JSON. This is because comparing to JSON format, the Ruby one is easier to spot error saving a lot of time. When we have hundred lines in a Role's file, it is quite possible that we make some typing errors.
The other reason is that we can do some sort of programming in Role file e.g. it is more convenient using a loop to create mod_jk config that route request to dozen of servers.
One layer above roles
As the number if roles grows, I have seen that many roles of different environments share common attributes. At OS level e.g. we use same DNS and NTP for both production and non production just in case of DRS there is different. Such examples are endless. These attributes are mostly not same in all environments, they are just same in some so it is not wise to put them in attributes's file of relevant cookbooks. Therefore we have created additional layer to support reduce such kind of duplication. These attributes and their values are kept together with applicable environments e.g. in separate file

set_config_item(:item=>"dns",:env=>["dev","pre","qa"],:value=>["192.168.0.1","192.168.0.2"])

and in the Role' file, we fetch the relevant attributes value. e.g.

"dns"=>get_config_item(:item=>"dns",:env=>"dev")

Call Attribute#to_hash in recipe
Cookbooks are commonly used in all environments. Beside infrastructure, we also create cookbooks that maintain our application specific setting such as URL of external partner systems that we communicate with. The fundamental structure are instances and application modules. Many different application modules can be deployed in single instance and the same application module can be deployed in different instances to support of different versions of the same module and dedicate instance per customer delivery channel.
Recipe of a cookbook get data from a role to define resources (file, directory, permission, etc.). As we want make role simple, easy to create and modify, I put more logic in recipe. Some recipes are real programming stuff that need to iterate over structure of a role to get desire data. The autovivify feature of attribute sometime causes problem, in order to avoid it we have to call to_hash for attribute before doing any iteration over it.
Avoid putting big binary in a cookbook
We initially tried to put all files required by a recipe in its cookbook. It turn out to be a bad idea to keep e.g. 200 MB of Jboss installation in Couchdb of Chef. Both knife and Chef-client runs very slow and un reliably. At the end we decided to keep these big binary (more than 20 MB) outside Chef cookbook and setup an Apache for handling these files. The recipe then has to be written to handle checksum, downloading (we use wget) and installation.
We do not have to do it in case of software packages available as rpm, because chef package resource supports their installation.
Make sure that there is no non-ascii character in /etc/passwd,group
chef-client use ohai to parse certain OS configuration files e.g. /etc/password to JSON, if these files contain non-ascii characters, it can causes a problem because JSON lib can not cope with that. So it is better to remove non-ascii from OS configuration files when migrating existing machine to chef.
Disable SELinux
SELinux se store security context per files & directories. When chef-client modify configuration files , it may not retain that information making some daemon not working properly. Chef does not support SELinux so we need to disable it to avoid problem.
Start multi chef-server-api instances for better performance
A default chef-server can not support hundreds of clients, so we installed front end Apache that handles SSL encryption for chef-client as well as balances request to 4 instances of chef-server-api, which we start using option -c 4.
run chef-client as root in crontab
We run chef-client on our non production environments, this is because we want to avoid possible problem with long running ruby process and to reduce the chance that chef-client at many machines run at one moment causing high load to chef-server.
Using crontab we can easily specify time when they will run for each group of machines or even machine. Time to run chef-client can be further generated automatically using hash of machine name to achieve a fair distribution of load.
For user, group management chef-client uses unix command lines, so chef-client must run as root with appropriate path to locations of useradd, groupadd, … (e.g. /usr/sbin,/usr/bin,/bin, /sbin).
The flow of change
The chef-client does not run automatically in production one. This is because in production we want more control as well as due to our change management policy. It is also safer taking into consideration that modification of cookbook can potentially impact production.
The flow of change in production (after cookbook being modified and tested in other environments) involes 1) modify relevant production role, 2) take one production machine from service, 3) execute chef-client on that machine, 4) verify that it work as per expectation, 5) execute chef-client in all remain servers and put all into service.
To execute the chef-client command on a group of machines, we can use sort of Command Control system (e.g. rundesk, control tier, ..) or simple knife.
Hacking, Notification and improvement
To increase safety in the context of missing noop option in chef-client, I have create a monkey patch for certain providers so when chef-client modify a file it notify us by sending output of diff between new and old file by e-mail. The custom notification also send alert to certain group of people depending of nature of change and affected environment.
Beside that, we have create new resources and change behavior of fews existing one to suit more our need. The code base of our cookbook, roles is now reaching 20K line of Ruby code and we continually adding more thing as well improve it.

chroot - Change Root

2011-02-15T12:20:00.004+09:00

I have recently closed a year old ticket that has some thing to do with chroot. For security reason, we use chroot (in modsecurity) to restrict our Apache process to access only to a desired directory tree. As per chroot document, a program that is re-rooted to another directory cannot access or name files outside that directory, and the directory is called a "chroot jail".
We make sure that chroot is called after Apache process complete it' initialization in order to not break anything, Because otherwise Apache will not be able to access needed share lib, log files, pid file located in various system directories.
However there is a wrong access time the log produced by our Apache. It is always GMT not local time as we want. We have opened ticket with vendor, searching over internet, looking at source code but could not figure out why. The worse thing is that, for first few requests , the access time is correct (local time) but then it gets change to GMT.
Yesterday I have found the reason. I remember that when I ran the strace with the Apache process being chrooted, I saw Apache try to open some files but could not find it. We see a lot of file not found when running strace because various libraries intend to open some file and if it is not found then try others. But in that case the file is /etc/localtime. So it turns out that when logging Apache use apr lib, which call gmtime and mktime, which need to access /etc/localtime. So missing this file in "chroot jail" causes a problem with access time. Without file /etc/localtime gmtime and mktime consider that the machine is in GMT time zone.

Using debuginfo packages in Redhat

2011-01-01T17:15:00.013+09:00

Redhat provide debuginfo packages but not tell us so much how to use it. Here is a simple instruction to use debuginfo packages.
First this is important to know that a debuginfo package contains only debug symbol not the executable. Therefore we need to install both normal package and the corresponding debuginfo and of course their version must be matched.
Installation
Make sure that repository of debuginfo package is configured properly as follows

[root@localhost ~]# cat /etc/yum.repos.d/rhel-debuginfo.repo 
[rhel-debuginfo]
name=Red Hat Enterprise Linux $releasever - $basearch - Debug
baseurl=ftp://ftp.redhat.com/pub/redhat/linux/enterprise/$releasever/en/os/$basearch/Debuginfo/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

To install debuginfo package, we have to specify --enablerepo rhel-debuginfo as yum option

root@localhost ~]#yum install --enablerepo rhel-debuginfo httpd-debuginfo

root@localhost ~]#yum install httpd

[root@localhost ~]# rpm -q -a | grep httpd
httpd-debuginfo-2.2.3-43.el5_5.3
httpd-2.2.3-43.el5_5.3

Usage
To get use just start gdb session with a normal executable

[root@localhost ~]# gdb /usr/bin/ab
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5_5.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/bin/ab...Reading symbols from /usr/lib/debug/usr/bin/ab.debug...done.
done.
(gdb) break main
Breakpoint 1 at 0x7402: file /usr/src/debug/httpd-2.2.3/support/ab.c, line 1900.
(gdb) run
Starting program: /usr/bin/ab 
[Thread debugging using libthread_db enabled]

Breakpoint 1, main (argc=114424, argv=0x0) at /usr/src/debug/httpd-2.2.3/support/ab.c:1900
1900 {
(gdb) l
1895 
1896 /* ------------------------------------------------------- */
1897 
1898 /* sort out command-line args and call test */
1899 int main(int argc, const char * const argv[])
1900 {
1901     int r, l;
1902     char tmp[1024];
1903     apr_status_t status;
1904     apr_getopt_t *opt;
(gdb)

M Language Tutorial - The cryptic style

2010-12-12T17:46:00.013+09:00

In order to save typing at cost of readability, every command and built in function of M has abbreviation/shortcut.
The following examples are perfectly valid

f i=1:1:9 s sqtable(i)=i*i; 'f' stand for 'for' and 's' for 'set'

and

s i="" f  s i=$o(sqltable(i)) q:i=""  w i," ",sqltable(i),!;'q' stand for 'quit' '$o' for '$order', 'w' for 'write'

As we spend most of our time reading code written by others, modern programming style does care much about naming thing. The abbreviated style of most if M code is really pain especially for novices.

M Language Tutorial - for and Array

2010-11-20T17:00:00.020+09:00

for command

GTM>for i=1:1:5 write i,!
1
2
3
4
5

The above for loop a variable i from 1 to 5, increasing by 1 in each iteration.
This one will loop over a list of arguments

GTM>for i="hello","world","bye","moon" write i,!
hello
world
bye
moon

array
In M array is stored as sparse B-tree structure, index or subscript can be anything and can be in any number

GTM> 
GTM>set a(1)="hello",a(2)="world" 
GTM>write a(1)," ",a(2)
hello world
GTM>
GTM>write a(3)         
%GTM-E-UNDEF, Undefined local variable: a(3)

the index of an Array can be any thing so it is like hash/dictionary in other language.

GTM>set a("hello")=1

GTM>set a("world")=2

GTM>write a("hello")
1
GTM>write a("world")
2
GTM>write a("moon") 
%GTM-E-UNDEF, Undefined local variable: a(moon)

We can use any number of subscripts

GTM>set a(1,2)="moon"

GTM>set a(1,1)="world"

GTM>write a(1,1)
world
GTM>write a(1,2)
moon
GTM>write a(1,0)
%GTM-E-UNDEF, Undefined local variable: a(1,0)

built-in function $order is used to get index/subscript of an element of an Array, that is particularly useful for traversal over an Array.

GTM>set b(3)="hello",b(5)="world",b("hello")=1,b("world")=2

GTM>write $order(b("")); with empty string we get a subscript of the first element of an Array
3
GTM>write $order(b(3)); passing one element we get a subscript of next element  
5
GTM>write $order(b(5))
hello
GTM>write $order(b("hello"))
world
GTM>write $order(b("world")); passing the last element we get empty string

If array is multi dimensional, $order will return subscript of an element in one dimension

GTM>set a("h",1)="hello",a("w",1)="world"

GTM>write $order(a("")); return subscript in the first dimension of first element 
h
GTM>write $order(a("h")); return subscript in the first dimension of second element 
w
GTM>write $order(a("w"))

GTM>write $order(a("h","")); return subscript in the second dimension of first element  
1
GTM>write $order(a("h",1)) 

GTM>write $order(a("w","")); return subscript in the second dimension of second element 
1
GTM>write $order(a("w",1))

traversal over a array
for is typical command used to traversal an array

GTM>kill a for i=1:1:10 set a(i)=i*i

We use kill to erase content of the variable a if exists and a for to create 10 elements.

GTM>set i="" for  set i=$order(a(i)) quit:i=""  write i," ",a(i),!; two SPACE after for and two SPACE after quit:i=""
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
10 100

Here, the argument-less For repeats until stopped by a terminating quit. This line prints a table of i and a(i)
block of multi line of codes under for
M support execute multi line of code under for in the routine

GTM>zedit "nested"
finish
 for name="ivan","john","janes" do
 . if name]"jo"; the operator ] test if name follows string "jo"
 . else  write name,!
 quit

GTM>zlink "nested"

GTM>do finish^nested
ivan
janes
GTM>

M Language Tutorial - if, else and boolean condition

2010-11-15T01:14:00.040+09:00

M language has no concept of structure that are familiar in other programming languages. In M all are commands including if,else,for.
if and else commands

GTM>set x=1

GTM>if x=1 write "boom"; if condition is satisfied
boom
GTM>if x=2 write "bang";  if condition is not satisfied

The if command support AND , OR, NOT condition

GTM>set x=1,y=2

GTM>if x=1,y=1 write "bang" ; we use , to denote AND

GTM>if x=1,y=2 write "bang"
bang

GTM> if (x=1)&(y=2) write "bang"; other way to express AND is to use &
bang

GTM>if x=1!y=1 write "boom"; we use ! to denote OR 
boom
GTM>if 1=0 write "oh" 

GTM>if '1=0 write "oh"; we use '  to denote a negation
oh

Internally, if set a special value $TEST to 1 (TRUE) or 0 (FALSE) depending on whether the condition is satisfied or not. $TEST is set to 1 at the start of gtm. The if command without condition will use $TEST as condition.

$gtm
GTM>write $TEST
1
GTM>if 1=0

GTM>write $TEST
0
GTM>if  write "hello"! note two SPACEs after if

GTM>if $TEST=1 write "hello"! equivalent to above

GTM>if 1=1

GTM>write $TEST
1
GTM>if  write "hello"; note two SPACEs after if
hello
GTM>if $TEST=1 write "hello"; equivalent to above
hello

else is command that perform the command followed it if the $TEST variable is 0

GTM>set x=1

GTM>if x>1 write "world"

GTM>else  write "moon"; note two SPACES after else
moon
GTM>if $TEST=0  write "moon"; equivalent to above
moon

command's postcondition
execution of almost all commands can be controlled by following it with a colon and a truthvalue expression.

GTM>set n=1
GTM>write:n>0 "hello"
hello
GTM>write:n>1 "world"
GTM>

block of multi line of codes under if,else
M support execute multi line of code under if,else, for in the routine

GTM>zedit "nested"
start(x,who)
 if x=1 do  ;two SPACE after do
 . write "hello",!
 . write who
 else  do  ;two SPACE after else and do
 . write "bye",!
 . write who
 quit

GTM>zlink "nested"

GTM>do start^nested(1,"world")
hello
world
GTM>do start^nested(0,"world")
bye
world

operators for string' comparison
These operators are a little bit strange comparing to other languages

GTM>WRITE "A"="B"; compare if two string are equal
0
GTM>WRITE "C"="C"
1
GTM>WRITE "A"["B"; this is same as contains in other language
0
GTM>WRITE "ABC"["C"
1
GTM>WRITE "A"]"B"; this is same as > in other language
0
GTM>WRITE "B"]"A"
1

NOT operator can applied to either expression or other operator

GTM>write "A"="B" 
0
GTM>write '("A"="B")
1
GTM>write "A"'="B"
1

M Language Tutorial - routines

2010-11-05T05:58:00.031+09:00

Divide and conquer is the technique used to counter complexity from the beginning of software development history, A complex system in M language comprise of many files called routines.
label, quit and function
In general each line of code in a routine contains of one or no label following by a SPACE or TAB and then M command, other M code can make a call to any line of the routine that has a label at start of the file. Let create a file display.m in one of these paths specified in env. variable gtmroutines using any editor.

start
 write "display",!
 quit
other 
 write "other",!
nothing
 write "no",!

Now start gtm and try to make some call

$gtm
GTM>zlink "display"
GTM>do ^display
display
GTM>do start^display
display
GTM>do start+1^display
display

The name after ^ is name of routine same as filename without extension .m. Without any label, GTM will execute the code starting from line 1 of the routine. It does exactly the same with label start. Making a call to a line 1 from the label start mean start the execution from line 2 of the file. The execution terminate at command quit.

GTM>do start+3^display
other
no
GTM>do other^display  
other
no
GTM>do nothing^display
no

Calling a line 3 from label start is the same as calling label other. Because there is no quit command, GTM continues the execution at the end of file.
There is good practice always structure a routines as series of sections starting with a label and ending with a quit command. That way we can consider each label as a function name when making a call. Using offset from a label is considered a bad practice as it decrease readability of the code.
When calling a function within the same routine, we can remove the ^filename

talkto(who)
 do say("Hello",who); call a function in the same routine
 write "bla bla",!
 do say("Byte",who)
 quit

say(what,who)
 write what," ",who,!
 quit
GTM>zlink "stuff"

GTM>do talkto^stuff("John")
Hello John
bla bla
Byte John

calling function, passing parameters, return value
We pass parameters when calling M routine in parentheses separated by comma, the leading period . is used to indicate a parameter being passed as reference that is used to store the output of function.

$gtm
GTM>zedit "calc"
calc(a,b,ret)
 set ret=(2*a)+(3*b)
 quit

Now make a call

 
GTM>zlink "calc"
GTM>do ^calc(1,2,.result)
GTM>write result
8

M provides a facility called Extrinsic Variable to create a function that return value so it can be used in a expression

$gtm
GTM>zedit "calc"
calc(a,b,ret)
 set ret=(2*a)+(3*b)
 quit

othercalc(a,b)
 quit (2*a)+(3*b); the function must put return value  after quit command

GTM>zlink "calc"
GTM>write $$othercalc^calc(1,2); put $$ before the name of function when calling
8
GTM>set x=$$calc^calc(2,3)     
%GTM-E-QUITARGREQD, Quit from an extrinsic must have an argument

M Language Tutorial - Getting started

2010-11-02T02:28:00.053+09:00

I am learning MUMPS/M language to understand one of our systems, that has been implemented using M. I feel, there is a lack of a documentation in form of quick start and tutorial, so I put some notes here hoping that it will be useful for someone.
For a thorough listing of the M language commands, operators, functions and special variables, see MUMPS by Example
One of M implementations available as open source is GTM from FIS, we can download it from http://sourceforge.net/projects/fis-gtm/. The documentation is available from here
Setup a environment to start some test is fairly simple, just unzip the downloaded file, run configure and answer few questions, then we can start
setting env. for running GTM

$source ./gtmprofile

This shell will set various env, variables required by GTM also create a global directory and a default datafile, that is actually M database. M is a language with built in persistence. The global directory is kind of control file in Oracle.
start gtm - the interpreter

$gtm
GTM>
GTM>write "hello world"
hello world
GTM>halt

M use write command to output some thing to a console, in Ruby we would use puts in Python, this is print. The command halt is used to quit the gtm.
For a beginner of any language, the ability to write something out to see and to exit are the two most important commands.
string and numbers
String is enclosed by " as in C, Java, Python, Ruby, to concatenate two or more string we use operator _, not that common any more.

GTM>set x="hello"
GTM>set y="world"
GTM>set z=x_" "_y
GTM>write z
hello world

The command set set these local variables, these variables are created automatically if not exist.

GTM>set a=10
GTM>set b=20
GTM>set c=(a+b)*5/10
GTM>write c
15

For number, this is quite straightforward, no surprise.

create and run the first program

$export gtmroutines=/home/gtm/samples
$source ./gtmprofile
$gtm
GTM>write $ZROutines
/home/gtm/samples

this gtmroutines env. variable specifies where GTM is going to store and search for its routines, inside GTM however it is kept in variable $ZROutines. The concept routine in GTM is synonym for a single file containing M code.

GTM>zedit "hello"

This will popup a vi editor with opened file /home/gtm/samples/hello.m. Create the following content, save and quit vi editor.

hello(who)
  write "Hello ",who,!
  quit

In the routine hello.m we see a label hello(who), which is calling entry point. GTM's routine can have many labels and the general syntax of calling a section of code in the routine is label^routine. The label hello will take one parameters. The write above command takes 3 parameters separated by comma where ! means new line. It is possible to write the above code in a single line

hello(who) write "Hello ",who,! quit

The quit at the end is important in case other labels are added below if we want a label to behave as a function, which mean that the execution flow in the routine terminates at quit command .

GTM> zlink "hello"

The zlink compiles and link hello.m to GTM image so we can call this routine in GTM environment

GTM>do hello^hello("Moon")
Hello Moon

If we do not specify a label then the code will be executed from the first line.

GTM>do ^hello("World")
Hello World

If we look closely to gtm, we will see that gtm is shell script that call the binary mumps in direct mode (with parameter -direct).
We can run the routine hello.m from command line as follow

$export gtmroutines="/home/gtm/samples/ ."
$mumps -run %XCMD 'do ^hello("World")'

The %XCMD is in fact a routine _XCMD.m located in distribution directory of GTM, that is why we need add "." into gtmroutines env. variables, so mumps knows where to find the files being executed.
language syntax
M syntax is very strict, SPACE characters between M statements are significant. A single space separates a command from its argument, COMMA "," is used to separate one argument from other of these commands taking more than one arguments.
A SPACE, or newline, separates the command's arguments from others. Commands which take no arguments (e.g., ELSE) require two following spaces.
Character ; is used to indicate start of a comment, that runs until end of line.

GTM>write  "hello"; two SPACES after write
%GTM-E-EXPR, Expression expected but not found
 write  "hello"
       ^-----
GTM>write "hello"; single SPACE after write
hello
GTM>set x=1,y=2 write "x=",x,",y=",y !set and write commands are on the same line separated by SPACE
x=1,y=2
GTM>set x=1 + 2; SPACE surrounding + 
%GTM-E-CMD, Command expected but not found
 set x=1 + 2
         ^-----
GTM>set x=1+2; no SPACE in expression
GTM>write x
3
GTM>set x= 9 ; SPACE after =  
%GTM-E-EXPR, Expression expected but not found
 set x= 9
       ^-----
GTM>set x=9; no SPACE after=
GTM>write x
9

Math expression
Comparing to other language, math operator has no order precedences, a expression is evaluated from left to right

GTM>write 1+2*3
9
GTM>write (1+2)*3
9

Parentheses has to be used to make thing work as expecting.

GTM>write 1+(2*3)
7

Velocity conference 2010

2010-06-30T20:12:00.003+09:00

Velocity conference 2010 Site
Videos

Automatically dump your Thread´s StackTrace whenever something get wrong

2010-03-20T05:59:00.005+09:00

JVM either from SUN or IBM has features that allows us to do heap dump and thread stack trace dump. They can even do it automatically when there is OutOfMemory error.
But there is plenty of others situations that StackTrace of Threads can be useful e.g. when your application code enter into infinite loop or it forgets to free some resources or all threads of its thread pools are busy. In such cases Thread´s StackTrace are extremely valuable in root cause analysis.
Create StackTrace dump is also much cheaper and faster than Heap dump so that is why we should do it automatically in many situation when we suspect that something gets wrong.
The most easy way to do it to write a script that trigger a Thread´s StackTrace dump by invoking kill -3 jvm_pid (or jstack jvm_pid) when something happen( e.g. JVM use more then 90% CPU).

6 months in ING Direct Spain

2010-01-23T18:31:00.008+09:00

Today marks 6 months of my tenure in ING Direct Spain. So it is helpful to have some reflections.
1. My job is primarily administrative. During the time, I was involved in a couple of troubleshooting issues, for which we are able to find out a root cause and proposed a conclusive solution.

2. The second thing is my effort to refactor our JBoss infrastructure and to use git to track and manage the change. One lesson I learned from that is not matter what sophisticated we are using, the key of success is ability a) to figure out what is specific and what is common and b) to reduce number of configuration items.
At the beginning I tried to track configuration items on each machine as one git branch but it turns out into a horror of hundreds of un-manageable branches. At the end I thrown it and start to divide the monolithic configuration into smaller parts, parametrize them for reuse. After that using git or any other VCS for tracking them is just as walk in a rose's garden.

3. The third thing I have started is documentation of our infrastructure, I decided to use wiki for doing it. It turned out to be a good choice. The document is very up to date and practical because I use the document during my daily work so whenever I find a gaps between the document and the reality, I correct it. Update documentation is not separated task but a part of my day-to-day activities.

4. An the last thing is my research to look for a central authentication solution for our unix farm of mix LINUX, AIX and Solaris. I have tested PAM & Open LDAP combination for a while but at the end I have selected and recommended Likewise with the existing Active Directory as authentication server due to a lower cost of installation and operation.

java.lang.OutOfMemoryError Out of swap space?

2010-01-18T02:57:00.004+09:00

I have encountered this error in one of our Hotspot JVMs. Googling for a while without definitive result, I decided to look at source code of Hotspot VM (happily the source code is available in Sun website).
Within few minutes, with very little effort, I found that JVM print out this message when got NULL from calling malloc. So it turns out to find why malloc fails.
When OS does not have RAM to satisfy malloc, it may try to swap out unused real memory to disk and if the swap space is configured too small, the error appears. But this is just one cause.
Other more likely cause is that JVM run out of memory address space and this is our case. It can happen when we run 32 bit JVM and total memory used by the JVM exceeds magical limit of 3 GB (on 32 bit kernel LINUX).

Bringing more development practices to the world of administration

2009-11-23T04:33:00.004+09:00

In tradition software development and system administration tends to be separated. So useful practices in software development are not known by many administrators. I would say that the world of system administrations lack very much behind software development's world.
While using Version Control System (VCS) is obvious thing in any software project, not many administrator use this very useful tool during their daily work. The result is a dirty environment where a configuration's file is backup manually by renaming before making change. Also there is difficult to perform a rollback and reason and time of change is not well maintained.
Other often well encouraging software principles are to achieve re-usability, readability, explicitness as well as to avoid duplication. On the other hand an administrator cares much less about them. I have seen an environment where there are hundreds of scripts poorly written, repetitive, difficult to understand and maintained. I do thing that by using technique as re-factoring the situation can be improved significally.

Articles about deployment and operation are rare

2009-08-23T04:43:00.010+09:00

Deploy and run a software is a vital part of the full picture "make value from the software". However does not draw much attentions from community.I listed here links to few blogs and websites talking about this subject matter.
http://www.build-doctor.com
http://www.agileweboperations.com/
http://en.oreilly.com/velocity2009
http://reductivelabs.com/trac/puppet/wiki/FrequentlyAskedQuestions
http://ganglia.info/
http://www.cfengine.com/
http://trac.mcs.anl.gov/projects/bcfg2
http://automateit.org/
http://wiki.opscode.com/display/chef/Home

Joining to ING Direct Spain

2009-08-08T15:10:00.005+09:00

ING Group decided not to launch ING Direct Operation in Japan, as result I moved to ING DIRECT Spain. As for non European citizen, the visa process take long time for me. But finally I landed to Spain Office at the end of July.
This is logical step in my career, I want to stay with ING DIRECT and the job market has still very tough. I returned to Europe after 12 years, I moved from Prague to Hanoi in July, 1997.
The first two weeks in the Spain Office is exciting, people are friendly, HR is supportive, office is nice, lunch is free, job is interesting. But like you got marry and went to honeymoon, this period does not last forever. Back to reality, I have to be pragmatic, find the way to integrate to the team and the community, be helpful and continue the journey in my career path.
Weather is Spain in this season is good especially in the morning, drop me a call or a message whenever you go Madrid, I would love to meet you for a drink, tapas.

My Resume - SUMMARY of SKILLS

2009-07-12T17:58:00.007+09:00

System administration, support and maintenance

Responsible for large scale WebSphere Application Server ND and WebSphere MQ infrastructure used by iFlex (Oracle Flexcube) retail banking system for a leading consumer bank.
Led a team, that provide remote and onsite support 24/7 distributed critical J2EE application based on WebLogic and Oracle Database for immigration control.
Directly involved in identification and analysis critical problems, was interface point with vendors, customers, and management in incident handling, problems solving process
Well versed in using scripting language (RUBY, PYTHON) to automate all kinds of administrative activities including software installation, update; configuration management; application deployment, update; testing; monitoring; and log file management

Java and J2EE Application Server

More than 5 years experience with JAVA platform and J2EE framework particularly Websphere, WebLogic. Directly involved in capacity planning, application packaging, deployment, monitoring, troubleshooting that includes sizing memory, thread pools, setting transaction's timeout, maintaining resource pool, etc..
Well understood internal JVM and its assembler language (byte code). Hands-on experience in tools and technique for identifying JVM and J2EE problem such as analyzing heap dump, core dump, garbage collector, decompile byte code, injection tracing code into application and container code.

Messaging System

More then 7 years in design and implementation of software systems that communicate with each other using asynchronous messages. Understand message creation and processing patterns (Enterprise Integration Patterns) and their applications in real world systems.
Responsible for deployment IBM MQ as message backbone for many applications. The responsibility includes interfacing with development team for planning and configuration, testing, trial run, installation, monitoring and troubleshooting

Database System

More then 10 years extensive working experience on RDBMS Oracle in both administrator and developer role. Obtained certification on Oracle DBA in 1999, hand-on experiences in large site capacity planning, performance tuning, high availability using Real Application Cluster (RAC) and Data Guard.
Involved in database design for many enterprise projects, assisted implementation team in evolving database schema, data migration and cleansing.

Operating System and Hardware

More than 10 years experience in design, installation, administration and troubleshooting complex IT infrastructure including UNIX clustering system, IBM FC and iSCSI SAN, CISCO router and PIX firewall
Hand-on experience in AIX administration including HACMP. Obtained certification on AIX administration in 1999. Performed installation, configuration, tuning many AIX based systems(s), making them ready for deployment Oracle Database and J2EE application server.
Involved in technical pre-sale activities including sizing a system, proposal preparation, technical presentation and alike.

Software Development

More than 15 years experience in developing software products with deep knowledge of architecture / design of multi-tiered client-server systems based on middleware including Tuxedo, J2EE (WebSphere, WebLogic) and other emerging Web framework as Ruby on Rails.
Skillful in many programming languages including C/C++, JAVA, SQL, PL/SQL, RUBY. Passionate for the simple and easy to understand software with rich domain model. Well versed in OOP, design patterns (Design Patterns - GoF, Pattern for Enterprise Application Architecture - PEAA, Domain Driven Design).
Strong knowledge of algorithms and data structures and their applications in writing a high performance, time critical system for complex business domain.
Extensive experience with state of art software development practices such as pair programming, Test Driven Development (TDD), refactoring, continuous integration.

Business Analysis and Leadership

Able to attract, retain and lead people. Led team up to 25 developers, responsible for task planning, technical direction, risk management, code inspections, quality control, performance reviews, staff development. Understood how to create working environment to keep people happy, motivated, creative and productive.
Solid presentation skill, proven in communication with customers, understanding problems, capturing domain knowledge and turning it into high level requirements in suitable form for development team and other stakeholders

Context root of single war's web applicaton in JBoss

2009-06-05T17:49:00.011+09:00

Context root of web application is usually specifiedin META-INF/application.xml in ear file (or a vendor specific jboss-web.xml inside war ) e.g.

...
  <module>
    <web>
      <web-uri>app.war</web-uri>
      <context-root>/jsfejb3<context-root>
    </web>
  </module>
..

However if we drop a single war file without vendor specific descriptor (jboss-web.xml) into JBoss deploy directory then context root is implicitly the name of the war file without extension.
e.g.
Web application jmx-console residing in default/deploy/jmx-console.war has context root jmx-console
Web application groovy-scripts residing in default/deploy/groovy-script.war has context root groovy-script and can be accessible via http://localhost:7080/groovy-script

Centralized Management Tools

2009-05-31T08:54:00.007+09:00

Nagios
Big Brother
Hyperic HQ
JOPR
@ctiveFriend
OEM
OpenNMS
There is a great presentation of monitoring tools here

Testing JOPR 2.2.0 - Basic configuration

2009-05-16T09:11:00.002+09:00

Modify JBoss server startup options to include JVM as child resource of JBoss Server in JOPR

#$JBOSS_HOME/bin/run.conf
JAVA_OPTS="$JAVA_OPTS -Djboss.platform.mbeanserver"

Remove an uninterested resource from management a.k.a uninventory
Browse Resources->Servers, select uninterested one, hit uninventory button

Testing JOPR 2.2.0 - Installation

2009-05-02T10:23:00.057+09:00

What is JOPR, RHQ, JBoss ON
JOPR is an open source project from Redhat to develop complex management platform. JOPR is agent based JAVA/J2EE system, where agent is standalone JAVA running and server runs on top of JBOSS Application Server(AS). It can be used to manage JBOSS AS, Tomcat, Apache as well as Oracle, and any IT resources.

JBoss Operations Network (JBoss ON) is commercial offer of JOPR in form of support service by Redhat.

JOPR is developed on top of RHQ project. RHQ serves as an maven upstream project for JOPR. RHQ consists of GUI management console, agent container and a set of agent plugins for management of common IT resources.
JOPR extends RHQ by adding additional plugins for JBoss App Server, Tomcat Web Server, etc. . JOPR version 2.2.0 is corresponding with RHQ version 1.2.0.

The original of RHQ is Hyperic HQ. Redhat and Hyperic work together to extract common reusable parts of Hyperic HQ and maintain it as RHQ. The aim is to create a common infrastructure for developing management's application. That is why the project named RHQ, where R stand for Redhat.

Install JOPR Server
Download JOPR binary from JOPR website
Follow instruction in Installation Guide to prepare database server for JOPR server
Change working directory to jopr-server-2.2.0 and type

bin/rhq-server.sh start

then open web browser and point to http://localhost:7080
Follow instructions in screen to set database driver, url, username/password to correct values. After that verify database connection.
Hit button Install JOPR server to complete installation process

Modify JOPR server properties file for agent communication
Modify rhq-server.properties file

# jopr-server-2.2.0/bin/rhq-server.properties
rhq.communications.connector.transport=servlet
rhq.communications.connector.bind-address=<jopr-server-ip-address>
rhq.communications.connector.bind-port=7080

Restart JOPR server

bin/rhq-server.sh stop
bin/rhq-server.sh start

Login to JOPR console and watch its log file
Pointing Web browser to http://localhost:7080/Dashboard.do
Login use default rhqadmin/rhqadmin as initial username/password
JOPR log file is located in jopr-server-2.2.0/logs/rhq-server-log4j.log

Install JOPR Agent
This shall be done in all machines that we want JOPR to manage .
Download agent from http://<jopr-server-hostname>:7080/agentupdate/download
Install JOPR Agent by typing

java -jar rhq-enterprise-agent-1.2.0.jar --install

Modify JOPR Agent configuration file

#rhq-agent/conf/agent-configuration.xml

# ask jopr agent not to ask for configuration during startup
<entry key="rhq.agent.configuration-setup-flag" value="true" />

<entry key="rhq.agent.name" value="<jopr-agent-hostname>"/>

<entry key="rhq.agent.server.bind-address" value="<jopr-server-ip-address>" />

# ask jopr agent to listen to on particular ip
<entry key="rhq.communications.connector.bind-address" value="<jopr-agent-ip-address>" />

Start RHQ Agent and watch its log file
change working directory to rhq-agent and run

bin/rhq-agent.sh -c conf/agent-configuration.xml

If we reinstall JOPR server then agent's local data and server'data is not in sync. In that case we shall ask agent to clean it's local data when start by using

bin/rhq-agent.sh -l -c conf/agent-configuration.xml

The most recent agent's log file is located in rhq-agent/logs/agent.log

Develop an international career - Recruiters

2009-04-30T19:48:00.007+09:00

Recruiters (a.k.a Head Hunters) play a vital role in the job market. Many big companies use services of recruiters when they need to hire someone (they do not recruit directly). I myself and a lot of my colleagues got a job by a recruiter.
Below is a list of the most well known recruiters
- Robert Half
- Robert Walters
- Hays
- Wall Street
- Hudson
There are many smaller but also very good recruiters, who are constantly looking for a good candidate in all countries over the world. If you are somehow visible and your skill fit a opening position, they will contact you.
I got my first job via placement of a smaller recruiter named Juno located in New York.

JBoss resources

2009-04-29T17:45:00.010+09:00

JBOSS configuration and tunning articles from Master the Boss
Running Multi JBoss Instances
Good examples of EJB 2.1, EJB 3 from Learn Technology
JBoss Official Docs & Examples from Redhat
JBoss Operation Network (ON), a.k.a JOPR - Documents
JBoss ON - FAQ
RHQ project, upon it JOPR depends
JOPR project
A blog on RHQ, JOPR, JBoss ON

Develop an international career - Make yourself visible

2009-04-25T09:09:00.012+09:00

No matter how good and skilled you are,it is useless if no one know about it. So make yourself visible, post your resume to professional websites and social networks, create your own blog, subscribe to interested user groups, participate in an online discussion.
Keep in mind that all these techniques only work if you keep your data up to date. When a recruiter doing executive search and find your resume within dozens of others,he/she will easily exclude you if your resume is 2 years old.

Few years back I created my blog. It has only professional content and I decided to keep it in English because my intention is not to have many readers but the right one. Those are recruiters, potential employers and colleagues. I believe, that it helps me to get my first and also second job. Looking at site analytic of the blog, I don't see much traffic but I can identify those coming from area where I received a job offer or where my recruiter is located.

Linked In is great social networking site for professionals, so I would recommend to register and put your career information there.

Again, don't expect anything happened over night, it will take time.

Develop an international career - Resume

2009-04-22T13:50:00.017+09:00

This post is created (based on my own experiences) for people who want to work out of their home countries and therefore develop their career internationally. It can be used as an advice.
You have generally only one career, so don't let it goes based on some unexpected events. If you really want to work abroad in developed countries, be prepared that it will take time to get a job. It can be two years or more depending on many factors. During this time, you may need take many actions to move into that direction.

Resume (a.k.a CV) is your ticket for entering job market. Write your resume with care and update it gradually and of course it should be in English. During last few years, I spend many hours to re-write many time my resume as well as to ask friends, colleagues and career's adviser for my resume's correction. I have to admit that I am still not satisfied with my resume as there is a lot of things to improve.

I also spend time to read resume of others, who work in the same industries and/or share a similar knowledge and career path. It help me a lot to get know how others present themselves, so I can adjust my resume to highlight my strength.

Java Class Loader's links

2009-04-01T11:29:00.008+09:00

Impala blog post - Thread Context Class Loader
Java World - Class Loader article
On Java - Internals of Java Class Loading
Advanced JBoss Class Loading
Equinox OSGi Context Class Loader Enhancements

My resume - LANGUAGE

2009-03-03T14:05:00.004+09:00

Vietnamese, English, Czech

My resume - EDUCATION

2009-03-03T14:04:00.004+09:00

Charles University Prague, Czech Republic

MS in Computer Science (data engineering specialization) - 1997
BS in Computer Science - 1994

Using CINT - C/C++ Interpreter - An experiment with vector

2009-02-14T23:09:00.009+09:00

Let do something with vector

cint.exe> L vector
cint.exe> {vector<int> temps;}
cint.exe> {temps.push_back(6);}
cint.exe> {temps.push_back(4);}
cint.exe> {temps.push_back(5);}
cint.exe> {temps.size();}
(const unsigned int)3
cint.exe> {temps[0];}
(int)6
cint.exe> {temps[1];}
(int)4
cint.exe> {temps[2];}
(int)5
cint.exe>

So it works well, now there is time to try some algorithm

cint.exe> L algorithm
cint.exe> {sort(temps.begin(),temps.end());}
Error: Template Function value_type(first) is not defined in current scope  algo.h(735)
!!!Dictionary position rewound... !!!Error recovered!!!
cint.exe>

It seems that there is some problem with STL lib coming with CINT in Windows. Looking at STL header files, I found that Function value_type is declared in _iterator.h surrounded by a condition for whatever reason

#if (G__GNUC>=3)

#if (G__GNUC_VER>=3001) 
...
template  
inline T* value_type(const vector::iterator&) {return (T*)(0);}
..
#endif
#endif

To fix it is fairly simple

d:\cint-5.16.19\cint.exe
cint : C/C++ interpreter  (mailing list 'cint@root.cern.ch')
   Copyright(c) : 1995~2005 Masaharu Goto (gotom@hanno.jp)
   revision     : 5.16.19, March 16, 2007 by M.Goto

No main() function found in given source file. Interactive interface started.
'h':help, 'q':quit, '{statements;}' or 'p [expr]' to evaluate

cint.exe> {#define G__GNUC 4}
cint.exe> {#define G__GNUC_VER 4001}
cint.exe> L vector
cint.exe> L algorithm
cint.exe> {vector<int> temps;}
cint.exe> {temps.push_back(6);}
cint.exe> {temps.push_back(4);}
cint.exe> {temps.push_back(5);}
cint.exe> {temps.size();}
(const unsigned int)3
cint.exe> {temps[0];}
(int)6
cint.exe> {temps[1];}
(int)4
cint.exe> {temps[2];}
(int)5
cint.exe> {sort(temps.begin(),temps.end());}
cint.exe> {temps[0];}
(int)4
cint.exe> {temps[1];}
(int)5
cint.exe> {temps[2];}
(int)6
cint.exe>

Using CINT - C/C++ Interpreter - Basic Commands

2009-02-14T22:21:00.010+09:00

Load,Unload,files
L that load files into interpreter, and file that list these loaded files.

cint.exe> L iostream
cint.exe> files
  0 fp=0x781c1bd0 lines=13   file="iostream"
  1 fp=0x781c1bf0 lines=479  file="iostream.h"
  2 fp=0x781c1c10 lines=11  *file="iosenum.h"
  3 fp=0x781c1c30 lines=34   file="bool.h"
  4 fp=0x781c1c50 lines=173 *file="_iostream"

If we use #include in curly bracket, we will get the same result

cint.exe> {#include <iostream>}
cint.exe> files
  0 fp=0x781c1bf0 lines=13   file="iostream"
  1 fp=0x781c1c10 lines=479  file="iostream.h"
  2 fp=0x781c1c30 lines=11  *file="iosenum.h"
  3 fp=0x781c1c50 lines=34   file="bool.h"
  4 fp=0x781c1c70 lines=173 *file="_iostream"

U command unload a file, a C/C++ header file can include other header files, which in turn can include additional header files and so all. CINT is good enough and maintain this dependencies so if we unload one file, others referenced files may get unloaded also.

cint.exe> L string
cint.exe> files
  0 fp=0x781c1bd0 lines=11   file="string"
  1 fp=0x781c1bf0 lines=72   file="_string"
  2 fp=0x       0 lines=0    file="string.dll"
  3 fp=0x781c1c10 lines=13   file="iostream"
  4 fp=0x781c1c30 lines=479  file="iostream.h"
  5 fp=0x781c1c50 lines=11  *file="iosenum.h"
  6 fp=0x781c1c70 lines=34   file="bool.h"
  7 fp=0x781c1c90 lines=173 *file="_iostream"
cint.exe> U string
cint.exe> files

reset
reset reset interpreter environment and unload all files

cint.exe> {int a=1;}
cint.exe> {a;}
(int)1
cint.exe> reset
cint.exe> {a;}
Error: Symbol a is not defined in current scope  (tmpfile)(1)
!!!Dictionary position rewound... !!!Error recovered!!!
cint.exe> L vector
cint.exe> files
  0 fp=0x781c1bd0 lines=11   file="vector"
  1 fp=0x781c1bf0 lines=18   file="_vector"
  2 fp=0x       0 lines=0    file="vector.dll"
  3 fp=0x       0 lines=0    file="vectorbool.dll"
  4 fp=0x781c1c10 lines=313  file="_vector.h"
  5 fp=0x781c1c30 lines=282  file="function.h"
  6 fp=0x781c1c50 lines=34   file="bool.h"
  7 fp=0x781c1c70 lines=235  file="algobase.h"
  8 fp=0x781c1c90 lines=55   file="_pair.h"
  9 fp=0x781c1cb0 lines=17   file="_iterator"
 10 fp=0x781c1cd0 lines=668  file="_iterator.h"
 11 fp=0x781c1cf0 lines=6    file="stddef.h"
 12 fp=0x781c1d10 lines=479  file="iostream.h"
 13 fp=0x781c1d30 lines=11  *file="iosenum.h"
 14 fp=0x781c1d50 lines=173 *file="_iostream"
 15 fp=0x781c1d70 lines=177  file="defalloc.h"
 16 fp=0x781c1d90 lines=34   file="new.h"
 17 fp=0x781c1db0 lines=34   file="stdio.h"
 18 fp=0x       0 lines=0    file="stdfunc.dll"
 19 fp=0x781c1dd0 lines=14   file="stdlib.h"
 20 fp=0x  476800 lines=19   file="limits.h"

-- Press return for more -- (input [number] of lines, Cont,Step,More)
cint.exe> reset
cint.exe> files
cint.exe>

Using CINT - C/C++ Interpreter - Getting Started

2009-02-11T19:22:00.017+09:00

One of things I favor scripting languages (Ruby, Python) over none scripting (JAVA, C/C++) is there is no need to follow time consuming cycle of write-compile-run. Even with the best IDE, I still feels the pain of typing command, hit key combination, waiting for computer doing. This is especially true when I need to explore some API, test some ideas. So when I discovered CINT, a C/C++ Interpreter, I gave it a try. With CINT, we can type C/C++ code directly in it's interactive CINT console and get the result. CINT is available for Windows and many UNIX platform(s) some LINUX clone has CINT pre-installed. We start CINT console by type e.g. (on Windows)

d:\cint-5.16.19\cint.exe
cint.exe>{#include <iostream>
cint.exe>{cout << "Hello World\n";}  
Hello World (class ostream)2085223112

C/C++ code shall be type inside curly bracket {...}, CINT doesn't allow to define function in interactive console

> { int func999() {return 999;} }
Limitation: Function can not be defined in a command line or a tempfile
You need to write it in a source file (tmpfile)(1)
!!!Dictionary position rewound... !!!Error recovered!!!

So we have to create separate file e.g.

# func999.cpp
int func999() {
return 999;
}

,load it in the console and call a function inside

cint.exe>L func999.cpp
cint.exe>{func999();}

Some of the most frequently used commands includes

L  - load file in to the console
U  - unload the file
file - show which files are currently loaded
reset - reset memory of CINT console to initial state,unload all files
q - quit CINT console

Retrospective 2008

2009-01-12T15:32:00.009+09:00

Year 2008 was marked by an important event of my life and my career. I have joined ING Direct Japan (INGDJ) and moved to Tokyo in September. So far I feel OK. Within last 4 months, I have done a couple of things. Technically
1. Learned how to manage WebSphere Application Server and MQ infrastructure in a complex IT environment for banking business.
2. Write a bundle of ruby scripts to perform administrative activities related to WebSphere including system monitoring, log management and application deployment.
3. Learned JVM internal stuff like GC, heap dump, core dump analysis tools, JAVA decompiler and JVM assembler language
4. Use VIM (as primary editor) and GIT
5. Learned Python and it's Java implementation Jython primarily and use it for WebSphere Application Server scripting

Using VIM in troubleshooting JAVA application

2008-11-28T22:56:00.019+09:00

After trying many Text Editor available on Windows as TexPad, UltraEdit, Wordpad, Notepad, etc. , finally I decided to use vim, an enhanced version of unix vi. Vim is never been an easy choice but once you master it, it is wonderful, it give you high productivity. Also it is free.
Some basic vi commands can be found in http://www.cs.uiuc.edu/class/fa07/cs225/calendar/vim.pdf.
During my daily work, I often have to troubleshoot complex JAVA application. Sometime I get exception with full JAVA stack printout, but the application is provided by vendor without source code so there is very difficult to find a root cause.
There is a tool called JAD that can decompile JAVA classes. Because my primary editor is VIM so I decided to integrate the JAD into VIM. Googling a while, I found vim JAD plugin, that display decompiled java class whenever we open the class file. However JAVA application(s) are mostly provided in form of several jar, zip, ear package so it is not so convenient.
There is luckily zip plugin that is part of vim installation and can browse zip, jar, ear package and display content of selected file inside the package when we hit enter. Modify just few lines of this plugin, I am able to browse content of jar, zip file and view a decompiled inside JAVA class.
The steps is as follows (for version 7.2, that was installed in D:\Vim\)

1. open D:\Vim\vim72\autoload\zip.vim and change

fun! zip#Read(fname,mode)
...
  exe "silent r! ".g:zip_unzipcmd." -p -- ".s:Escape(zipfile,1)." ".s:Escape(fname,1)
...
endfun

fun! zip#Read(fname,mode)
...
 if fname =~ '.class$'
   exe "silent r! ".g:zip_unzipcmd." -o -- ".s:Escape(zipfile,1)." ".s:Escape(fname,1)
   exe "silent r! jad -lnc -p ".s:Escape(fname,1)
 else
   exe "silent r! ".g:zip_unzipcmd." -p -- ".s:Escape(zipfile,1)." ".s:Escape(fname,1)
 endif
...
endfun

2. download unzip.exe, jad.exe and put them in the PATH
3. start vim and open a jar file, a list of files inside the jar file will be displayed
4. select one class file and hit ENTER The '-lnc' option of JAD display line number of original source code as comment on the left side of decompiled code so it can be used to identify which part of code cause an exception.

Simple Python Syntax

2008-11-08T08:03:00.043+09:00

Start interpreter and print out something then exit

C:\python
ActivePython 2.5.2.2 (ActiveState Software Inc.) based on
Python 2.5.2 (r252:60911, Mar 27 2008, 17:57:18) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'hello world'
hello world
>>> print 1+2+3
6
>>> exit()

Run a script in the interpreter
To run a script within Python interpreter use execfile(path), it is useful for people who use Jython, because startup time of JVM is horrible . e.g

>>execfile('sample.py')

Create string from a template
This is one of my most frequently used statement, Python follow style of C printf function

>>> "my name is %s, my age is %d" % ("Goto",30)
'my name is Goto, my age is 30'
>>>

Ruby has the same function

irb(main):004:0> "my name is %s, my age is %d" % ["Goto",30]
=> "my name is Goto, my age is 30"
irb(main):005:0>

but there is nicer way to do it

irb(main):005:0> name,age = "Goto",31
=> ["Goto", 31]
irb(main):006:0> "my name is #{name}, my age is #{age}"
=> "my name is Goto, my age is 31"

Create a substring from a string
Python has nice methods get a substring from string

>>> s='hello world'
>>> s[0]
'h'
>>> s[0:10] #substring from a position(inclusive) until other (non inclusive)
'hello worl'
>>> s[2:] #substring to end of string
'llo world'
>>> s[:2] #substring from start until other (non inclusive)
'he' 
>>> s[-2:] #negative position indicates position from end of the string
'ld'

The equivalent in Ruby would be

$ irb
>> s="hello world"
=> "hello world"
>> s[0]
=> 104
>> s[0..0]
=> "h"
>> s[0..(10-1)] #unlike Python, Ruby includes the end position
=> "hello worl"
>> s[2..-1] # -1 indicate relative position from end
=> "llo world"
>> s[0..(2-1)]
=> "he"
>> s[-2..-1]
=> "ld"

Unlike Ruby, Python still throw out of range exception for single indice operation

>>> s[20]
Traceback (most recent call last):
  File "", line 1, in 
IndexError: string index out of range
>>> s[20:30] # this is OK as it consider as slice operation
''

String is immutable
Unlike Ruby, Python string can not be changed directly

>>> s[0:5]
'hello'
>>> s[0:5]="bye"
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'str' object does not support item assignment
>>> s1 = "bye" + s[5:] # to achieve the same goal we need create string from other string 
>>> print s1
bye world

In Ruby we can do

>> s[0..4]="bye"
=> "bye"
>> print s
bye world=> nil

As string is considered value object, create new string instead of changing an existing express that concept more clearly.
List and tuple
Python has tuple and list for representation of variable size array of items. Tuple is immutable array while list is mutable. People think that list is intended for homogeneous while tuple is for non-homogeneous, but this is convention only is not enforced by the language. However some api accept only tuple as argument e.g. String format %, that sometime lead to a confusion. In contras Ruby has only Array.
example of tuple

>>> y=('a',2)
>>> y[1]=1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> y
('a', 2)

example of list

>>> x=['a',2]
>>> x[1]=1
>>> x
['a', 1]

iterate over list or tuple

>>> items = ('a','b',2)
>>> for e in items:
...   print e
...
a
b
2

Hash aka dictionary
Python has hash structure called dictionary

>>> h = {1:'a',2:'b',3:'c'}
>>> h.__class__
<type 'dict'>
>>> h[3] = 'z'
>>> h
{1: 'a', 2: 'b', 3: 'z'}
>>>

iterate over dictionary

>>> for k,v in h.iteritems():
...   print "%d=>%s" % (k,v)
...
1=>a
2=>b
3=>z

h.items() also works well
Class, Object, method
Class is defined using class keyword, name of a class can start lower case or upper case character, which is different from Ruby. Ruby requires name of class start with upper case character.
However the convention is that, name of built-in class as string, unicode, list, tuple start with lower case character while user-defined class starts with upper case.
example of built-in class

>>> y = ('a',2)
>>> y.__class__
<type 'tuple'>
>>> y.__class__ == tuple
True
>>> z = tuple(y)
>>> z
('a', 2)

example of user defined class

>>> class Foo:
...   def m(self):
...     print self.__class__
...
>>>
>>> Foo().m()
__main__.Foo
>>>

Module
Every Python script store in a file is a module, the name of the file is module name.

$ cat hello.py
def say(whom):
  return "hello %s" % whom

To use method defined in a module, just import the module and call the function preceding by the module name plus '.'

>>> import hello
>>> hello.say('world')
'hello world'

It is also common to mix module methods into current name space, so we can call method without typing module name

>>> from hello import *
>>> say('moon')
'hello moon'
>>>

Parameters
Beside normal position based parameters, Python has two extra forms of passing parameters to a function *params and **params.
The first form is argument list, in which caller pass a list parameters and calling function receives them in form of an array.

 def foo(*numbers):
      return sum(numbers)
 
print foo(23, 42)        # prints: 65

The second form is name based, in which caller pass a list name,value pairs and calling function receives then as a hash map.

def bar(**options):
  if "verbose" in options and options["verbose"]:
     print "verbose is ON"
  else:
     print "verbose is OFF"

bar(verbose=True) # print: "verbose is ON"
bar(verbose=False) # print: "verbose is OFF"
bar(force=True) # print: "verbose is OFF"

Closures
Python closures

Where to download RPM package(s) for RedHat Linux

2008-10-30T20:36:00.008+09:00

If you are looking for RPM package(s) for LINUX, then you should look at this site http://linuxsoft.cern.ch/. The binary rpm package are built from source SRPM available in http://ftp.redhat.com/pub/redhat/linux/enterprise/. They are well organized, complete and updated. The slc4X directory is equivalent to RedHat ES/AS 4 while slc5X to RedHat ES/AS 5.
If you have not RedHat Linux media in hand for what ever reason, this is definitely a good help.

First few days in ING Direct Japan

2008-09-09T12:39:00.007+09:00

The first day was just for formality. I was given temporary access card (RFID card), seat and account to access the company computer. Then a lady from IT team took me to the local ward office for alien card registration and to local bank (Shinsei) to open a account to which the company can pay me.
The second day is workshop on kind corporate value and culture , they called it Orange value, where we listened to the CEO speed of ING group system of value, did self introduction, play game, conduct group discussion organized by external consultant.

Testing Oracle Real Application Cluster

2008-09-04T18:11:00.067+09:00

Installation and configuration of Oracle RAC is always challenging task even for an experienced dba.
I read a story saying that a database administrator will laugh at you if you ask him to create a workable RAC of 2 nodes within a week.
Oracle RAC is definitely complex environment but is it so difficult and time consuming to deploy ?
To find an answer, I decided to try install Oracle RAC 10.2 on my small computer with 2 GB RAM. I used VMWARE server to create the environment of 2 linux nodes, that concurrently access shared database on cluster file system OCFS2.
I took me not one week but nearly 4 day(s), during which, I have solved various encountered problems. Below are some notes that you need to consider before attempting similarly

Documentation
http://www.oracle-base.com/articles/rac/ArticlesRac.php
http://www.dbasupport.com/oracle/ora10g/index_RAC.shtml These two freely available resources are quite good, concise and practical beside metalink and official manual.

Summary of problems
Believe or not, the fact is that most of problems are rooted from
- mis-configuration of hardware and OS: without time synchronization between nodes, Clusterware misbehaves
- not install latest patch: many things suddenly work after installing latest patch
Oracle RAC document is generally not good
- it is huge and verbose: the Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide 10g Release 2 (P/N: B14197-08) is 400 pages thick
- knowledge is split across many metalink notes making it difficult in orientation: to understand why we need proper setting of default gateway for a public network interface and its role, we need search metalink more than once
- It is usually difficult and expensive to create environment for practical testing

The RAC software stack
RAC Software stack consists of 2 layers sitting on top of OS
a. RDBMS: Oracle RDBMS with RAC as Option
b. Clusterware: CRS – Cluster Ready Service
c. Operating System: Linux, Windows, AIX
By implementing CRS component to play a role of Clusterware, Oracle wants to remove dependency of RAC from Clusterware traditionally provided OS and third party vendor.
CRS need two device(s)/file(s) to store it's data; voting disk and oracle cluster registry - OCR . Because only one CRS is needed to support multi database(s), it is good to keep voting disk and OCR in a location independent from location for a database files.

VMWARE Server configuration
To experiment with Oracle Clusterware, also called Cluster Ready Service (CRS) and RAC, we need VMWARE Server. VMWARE Workstation does not work because it does not support concurrent access to a share disk.
The config file (*.vmx) of each node shall be specially modified to disable disk locking and cache. By doing it, a hard disk created as file on host system can be safely attached to 2 VMWARE nodes. An example of special setting is taken from Oracle RAC Installation on VMWARE article given below

...
...
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0" 
diskLib.dataCacheMaxReadAheadSize = "0" 
diskLib.dataCacheMinReadAheadSize = "0" 
diskLib.dataCachePageSize = "4096" 
diskLib.maxUnsyncedWrites = "0"

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic" 
scsi1.sharedBus = "VIRTUAL"
...
...

Network setting
RAC requires at least 2 NIC per node, one for private network and one for public network. Private network is used for communication between nodes while public network is for serving clients.
In case of public network RAC does not use the host IP address for client communication but what it calls service or virtual IP address (VIP). In the public network, the VIP address has the same netmask as the host IP address, it is managed (assigned/un-assigned to the NIC) by Cluster Ready Service - CRS so in case of failure of one node, the VIP address of the fail node will be move to surviving one by CRS.
It is good to follow some naming convention for host name. Example is given below

/etc/hosts
# public
190.2.4.100             rac1 rac1-en0
190.2.4.101             rac2 rac2-en0

# private
9.2.4.100               rac1-priv rac1-en1
9.2.4.101               rac2-priv rac2-en1

# virtual
190.2.4.200             rac1-vip
190.2.4.201             rac2-vip

CRS need a gateway assigned to a public network interface in order to reliably detect failure of NIC and node. It is important to use a IP of a live machine reachable from all nodes but it is not necessarily a gateway.

Enabling remote shell (RSH) and login (RLOGIN)
Oracle Universal Installer (OUI) uses RSH and RLOGIN to install software on multi nodes, which means we run OUI only on one node and it will automatically copy the software (CRS and RAC) and configure on other nodes. Other configuration programs as VIPCA, NETCA and DBCA also work similarly and depend on RSH and RLOGIN to perform remote operation.
Therefore we need to enable RSH and RLOGIN on all nodes before running installation program. The example below demonstrate how to enable RSH and RLOGIN.
Configure RSH

[root@rac1 ~]# rpm -q -a | grep rsh
rsh-0.17-25.4
rsh-server-0.17-25.4

[root@rac1 ~]# cat /etc/xinetd.d/rsh
# default: on
# description: The rshd server is the server for the rcmd(3) routine and, \
#       consequently, for the rsh(1) program.  The server provides \
#       remote execution facilities with authentication based on \
#       privileged port numbers from trusted hosts.
service shell
{
        disable = no
        socket_type             = stream
        wait                    = no
        user                    = root
        log_on_success          += USERID
        log_on_failure          += USERID
        server                  = /usr/sbin/in.rshd
}

Configure RLOGIN

[root@rac1 ~]# cat /etc/xinetd.d/rlogin
# default: on
# description: rlogind is the server for the rlogin(1) program.  The server \
#       provides a remote login facility with authentication based on \
#       privileged port numbers from trusted hosts.
service login
{
        disable = no
        socket_type             = stream
        wait                    = no
        user                    = root
        log_on_success          += USERID
        log_on_failure          += USERID
        server                  = /usr/sbin/in.rlogind
}

Start RSH and RLOGIN

[root@rac2 ~]# chkconfig rsh on
[root@rac2 ~]# chkconfig rlogin on
[root@rac2 ~]# service xinetd reload

Create oracle account and enable RSH and RLOGIN for oracle on node rac1 and rac2

#Create the /etc/hosts.equiv file as the root user.
[root@rac2 ~]# touch /etc/hosts.equiv
[root@rac2 ~]# useradd oracle
[root@rac1 ~]# useradd oracle

[root@rac[1,2] ~]# cat /etc/hosts.equiv
+rac1 oracle
+rac2 oracle
+rac1-priv oracle
+rac2-priv oracle
+rac1-vip oracle
+rac2-vip oracle

[root@rac[1,2] ~]# chmod 600 /etc/hosts.equiv
[root@rac[1,2] ~]# chown root:root /etc/hosts.equiv

Verify RSH and RLOGIN

[oracle@rac2 ~]$ rsh rac1 “uname –a”
[oracle@rac1 ~]$ rsh rac2 “uname –a”
[oracle@rac1 ~]$ rlogin rac2
[oracle@rac2 ~]$ rlogin rac1

Configure Network Time Protocol - NTP

Oracle clusterware CRS use time from nodes for it activities, so keeping time synchronized on all nodes is required. The clock synchronization method on unix is using NTP services, in which a machine contact other with preciser clock a reference clock, on regular interval to retrieve time and update its own clock.
In the following example, node rac1 is used as reference clock while rac2 use rac1 as it's reference clock. Sometime this configuration does work well due to the reason that VMWARE hypervisor provide very bad clock implementation, therefore it is recommended to use an external physical server as reference clock , if we can have connection to Internet then use one of available real reference clocks.

[root@rac1 ~]# more /etc/ntp.conf
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10 refid LCL

[root@rac1 ~]# /etc/init.d/ntpd restart
Shutting down ntpd:                                        [  OK  ]
Starting ntpd:                                             [  OK  ]
[root@rac1 ~]# chkconfig ntpd on

[root@rac2 ~]# more /etc/ntp.conf
server  rac1     # remote clock

[root@rac2 ~]# /etc/init.d/ntpd restart
Shutting down ntpd:                                        [  OK  ]
Starting ntpd:                                             [  OK  ]

[root@rac2 ~]# chkconfig ntpd on

[root@rac2 ~]# /usr/sbin/ntpdate -v -d rac1 # verify reference clock rac1

Turn off firewall

CRS communication between nodes happens via TCP and UDP, so it is important to turn off firewall in all nodes to allow inter nodes traffic.

[root@rac1 ~]# /etc/init.d/iptable stop
[root@rac1 ~]# chkconfig iptable off
[root@rac2 ~]# /etc/init.d/iptable stop
[root@rac2 ~]# chkconfig iptable off

Install, configure cluster file system - OCFS2

RAC use shared disk architecture, in which all nodes have to access (for both read and write) to a same collection of database and others files located in a external hard disk. There are few options, one is to keep these files on raw devices, which is usually quite painful due to it's complex administration ranging from creation to maintenance. Other option is store them on file system, because traditional unix file system don't allow access from multi nodes, we need a special one, a cluster file system. There are different cluster file system available for different OS, e.g. GFS on AIX, Oracle has implemented a cluster file system on LINUX called OCFS2.
Install OCFS2 is fairly simple and easy

[root@rac1 ~]# rpm -q -a | grep ocfs2
ocfs2-2.6.9-42.ELsmp-1.2.9-1.el4
ocfs2-tools-1.2.7-1.el4
ocfs2console-1.2.7-1.el4

[root@rac2 ~]# rpm -q -a | grep ocfs2
ocfs2-2.6.9-42.ELsmp-1.2.9-1.el4
ocfs2-tools-1.2.7-1.el4
ocfs2console-1.2.7-1.el4

To create a configuration run ocfs2console and follow GUI menu

[root@rac2 ~]# /usr/sbin/ocfs2console

Define a configuration of two nodes rac1, rac2 using IP on private NIC en1 and specified port
Propagate the configuration to other node
Create and format cluster file system, note that we shall create a cluster filesystem on a shared disk, that is accessible from both nodes.

[root@rac1 ~]#fdisk /dev/sdb
...  #create single partition sdb1
...

Use ocfs2console=>task=>format /dev/sdb1
Start OCFS2 services and mount cluster file system at startup

[root@rac[1,2] ~]#chkconfig --add o2cb
[root@rac[1,2] ~]#chkconfig --add ocfs2
[root@rac[1,2] ~]#/etc/init.d/o2cb configure

[root@rac[1,2] ~]#cat /etc/fstab:
...
/dev/sdb1  /u02  ocfs2    rw,_netdev,datavolume,nointr,heartbeat=local 0 0

[root@rac[1,2] ~]#mkdir /u02
[root@rac[1,2] ~]#mount /u02

Verify cluster file system by creating a directory on one node and check if it appears on other node.

[root@rac1~]#mkdir –p /u02/crsdata
[root@rac1~]#mkdir –p /u02/oradata
[root@rac2~]#ls –l /u02
crsdata
oradata

Modify LINUX kernel setting to meet OCRS,and ORACLE requirements
For the first node

[root@rac1]~]# vi /etc/sysctl.conf
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.rmem_max=262144
net.core.wmem_default=262144
net.core.wmem_max=262144
[root@rac1~]# sysctl -p
[root@rac1]~]# vi /etc/modprobe.conf
options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

For the second node

[root@rac2~]# vi /etc/sysctl.conf
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.rmem_max=262144
net.core.wmem_default=262144
net.core.wmem_max=262144
[root@rac2~]# sysctl -p
[root@rac2]~]# vi /etc/modprobe.conf
options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

Install Oracle Cluster Ready Service - CRS
Create the new groups and users.

[root@rac[1,2]~]#useradd oracle
[root@rac[1,2]~]#groupadd oinstall 
[root@rac[1,2]~]#groupadd dba 
[root@rac[1,2]~]#usermod -g oinstall -G dba oracle

Create Oracle Cluster Registry - OCR and votedisk on cluster file system

[root@rac1~]#mkdir –p /u02/crsdata
[root@rac1~]#touch /u02/crsdata/ocr.dbf
[root@rac1~]#touch /u02/crsdata/votedisk.dbf
[root@rac1~]#chown –R oracle /u02

Copy installation software

[root@rac1~]#mkdir –p /u02/setup/clusterware10.2.0.1
[root@rac1~]#mkdir –p /u02/setup/oracle10.2.0.1
[root@rac1~]#mkdir –p /u02/setup/patch10.2.0.4

Create home directory for crs and oracle

[root@rac1~]#mkdir –p /u01/crs
[root@rac1~]#mkdir –p /u01/oracle
[root@rac1~]#chown –R oracle /u01

Run Oracle Universal Installer - OUI as oracle

[oracle@rac1~]$/u02/setup/clusterware/runInstaller

Follow instructions and
- add node rac2 to make sure that there are two nodes in the cluster - for each node specify public node name (rac1, rac2), virtual host name (rac1-vip, rac2-vip) and private node name (rac1-priv, rac2-priv), this is also important to specify an IP of a reliable host of the same public network as gateway.
- specify public network for public and virtual IP and private network for private IP
- specify u02/crsdata/ocr.dbf as OCR and /u02/crsdata/votedisk.dbf as voting disk location and external redundancy OUI will perform installation on one node then using RSH to copy to other node, during final phase of the installation process, script root.sh shall be invoked manually as root on each node.
At the end, OUI will run virtual IP configuration assistant - VIPCA, that create and persist configuration of VIP, ONS and GSD etc. as resource in OCR.
If for any reason (e.g. mis typing, incorrect IP) VIPCA fails, we can re run it under root after correction.

Install Oracle Database Software and create a database
The installation of Oracle Database Software is similar as for non-RAC version, only different is after completion on one node, the OUI will copy the software to other node.
At the end of this process two configuration programs will be invoked
- a database configuration assistant - DBCA for database creation and
- a network configuration assistant - NETCA for listener and tnsname configuration

Difference between Remittance and Transfer

2008-08-28T09:50:00.019+09:00

In financial industries, 'Remittance' refers to an action of sending money from an individual to other not being resulted from either sale and transfer of good and funds. 'Transfer' generally refers to an action of sending money from a party to other as payment for a good or service.
E.g. CitiBank Japan have different charge for remittance and transfer

Calling RMI from EJB Stateless Session Bean

2008-08-08T19:54:00.008+09:00

I have to analyze one of our J2EE application(s), that call RMI from a Stateless Session Bean. I originally think that it could be a problem, because our Session Bean use remote object as singleton and the rule said that Session Bean shall be programed in thread safe manner because its methods run concurrently within container's threads. To confirm that I decided to do some experiments.
I create a sample RMI server

/* SampleServer.java */
import java.rmi.*;

public interface SampleServer extends Remote
{
  public int sum(int a,int b) throws RemoteException;
}

import java.rmi.*;
import java.rmi.server.*;
import java.rmi.registry.*;

/* SampleServerImpl.java */
public class SampleServerImpl extends UnicastRemoteObject
                             implements SampleServer
{
  SampleServerImpl() throws RemoteException
  {
     super();
  }

  public int sum(int a,int b) throws RemoteException
  {
     try {
       Thread.sleep(5000);
     } catch (InterruptedException e){
       e.printStackTrace();
     }

     return a + b;
  }

  public static void main(String args[])
  {
    try
      {
        SampleServerImpl sampleServer = new SampleServerImpl();

        Registry registry = LocateRegistry.createRegistry(8989);
        registry.rebind("Sample", sampleServer);

        System.out.println("Server waiting.....");
      }
    catch (RemoteException e)
      {
         e.printStackTrace();
      }
  }
}

I compile these classes compile and run using JDK1.5 with jmx enabled according jconsole article

$ javac SampleServer.java SampleServerImpl.java
$ rmic SampleServerImpl
$ java -Dcom.sun.management.jmxremote SampleServerImpl

I decide to use JRUBY to create and run a 20 thread(s), each issue a call RMI server

$jirb
> import java.rmi.Naming  
> sample = Naming.lookup("//127.0.0.1:8989/Sample")
> (1..20).each { Thread.new { puts sample.sum(3,7)} }
>

Then I run jconsole supplied in JDK1.5 and watch thread pool of the RMI server

$jconsole

The result is everything is OK, for each incoming call request, RMI server create a separate thread to handle it and the thread is dropped after it completion.

Joining ING Direct Japan

2008-07-09T09:00:00.007+09:00

I have given a resignation letter to the CEO of CSE, the company that I worked for 11 years. Officially I will join ING Direct Japan at the end of August and will move to Tokyo.
Taking such decision is not easy, CSE is not just my workplace, it is part of my life, where I put a lot of my desire and hope. These past 11 years was rewarding and I never regret of the decision to move from Prague to Hanoi in 1997. But it is time to change.
If anyone have chance to be in Tokyo, don't forget to drop me a e-mail, I will be pleased to go with you for a drink.

Faster jruby startup

2008-05-10T11:55:00.005+09:00

One of issue with JRuby and Java in general is long startup time. As the result, a lot of peoples including myself use MRI when developing application and then use JRuby to test and run the application.
Recent change is jruby shell script have improved startup time at significantly by putting JRuby library into java boot classpath (using -Xbootclasspath/a:) instead of normal classpath to bypass java class verification, however it still so slow compare to MRI.
Below is a simple test on machine
1. jruby lib in classpath

huy@huy-desktop:/u01/jruby-1.1.1/bin$ time jruby -e "puts 'hello world'"
hello world

real    0m2.934s
user    0m2.604s
sys     0m0.136s

2. jruby lib in boot classpath

huy@huy-desktop:/u01/jruby-1.1.1/bin$ time jruby -e "puts 'hello world'"
hello world

real    0m1.124s
user    0m0.896s
sys     0m0.072s

3. MRI

huy@huy-desktop:/u01/jruby-1.1.1/bin$ time ruby -e "puts 'hello world'"
hello world

real    0m0.060s
user    0m0.004s
sys     0m0.004s

Why Python

2008-04-15T17:10:00.006+09:00

I known Python during the time I read the book "Test-Driven Development By Example". It is dynamic programming language like Ruby. However I don't like that language as Ruby because of it's syntax, missing open classes, having difficult meta programming style, etc..
I get attention of Python after recent release of Google AppEngine, a combination of web framework written in Python with hosting service. Google AppEngine allows us to develop application in our machine then upload to Google hosting service and benefit from Google Infrastructure including identity management, Google datastore, scalability on demand, etc. . Google AppEngine is now available only for Python developers, they are many so within few hours from announcement of Google AppEngine, the first free 10,000 accounts has gone.
Python has additional good characteristics to be considered as language on new projects:
- Python can compile program to byte code so it has reputation of better performance then Ruby.
- In addition to C implementation, Python has already quite mature .Net and Java implementation, which can be very strong arguments in any integration projects.
- Python has proven track on scalable web site like Google or framework as Django , Zope or application as ERP5

Agile Reading List

2008-02-22T09:21:00.003+09:00

link to essential list
link to extended list

Using git for newbie

2008-01-20T15:17:00.019+09:00

I am doing some experiment with Rubinius code base, which uses distributed SCM git. Comparing with Subversion, Git is a little bit more difficult, so It takes me a while to understand its commands and find the proper way to work with it. Without commit right, here are typical commands I use in my work flow.

Clone a remote repository into the local disk - repository is create as a clone of the remote repository identified by a given url

git clone git://git.rubini.us/code 
cd code # goto directory containing local repository 
git config user.name "le huy"  #configure your name 
git config user.email "lehuy20@gmail.com" #and email that will be used when committing

Sample .git/config - git create a .git directory to store it's data, file .git/config contains configuration data including remote repository, which remote branch is link with which local one

cat .git/config 
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = git://github.com/evanphx/rubinius.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[gui]
        geometry = 885x450+5+65 399 192
[branch "master"]
        remote = origin
        merge = refs/heads/master
[branch "cpp"]
        remote = origin
        merge = refs/heads/cpp

Submodule - git submodule is used to link to external repository (equivalent to subversion extern)

cat .gitmodules
[submodule "mspec"]
        path = mspec
        url = git://github.com/brixen/mspec.git
[submodule "spec/frozen"]
        path = spec/frozen
        url = git://github.com/brixen/rubyspec.git

init and update submodule from external repository

git submodule init # init submodule repository based on .gitmodules
git submodule update

Show current branch - show which branch we are working on, switch between them

git branch # will show us local branches 
* cpp
  master

git branch -a # will show us all branches including local and remote 
* cpp
  master
  origin/cmakebuild
  origin/cpp
  origin/fusion-experiment
  origin/instance_eval
  origin/master
  origin/wilson64

git checkout master # switch to branch named 'master'
git checkout dev # switch to branch named 'dev'

We can sometime create new branches in addition to 'master' to work concurrently on many features
Create new branch

git branch working  # create branch 'working' 
git checkout working # switch to branch named 'working'
git branch dev --track origin/dev # create local branch 'dev' that track the remote branch origin/dev

The current working branch is now 'working'

Delete branch - we can remove unused branch if it is no longer needed

git branch -d working # delete 'working' branch

Update from remote repository - e.g. wait until someone commit the patch to the main trunk then update changes from git://git.rubini.us/code
git pull will perform merge change on remote branch to local branch

git pull

git rebase will save local changes away, update from remote branch, and then reapply them

git rebase master

Push change to remote repository

git push # push will update a --track branch in remote repository,

Edit source code - do some source code editing

gedit kernel/core/kernel.rb

Show modification - show what I have modify without committing change

git diff HEAD # HEAD is symbol for latest/head version of current branch

Revert back -when recognizing that there is something wrong, we can undo the modification, all changes are lost

git reset --hard HEAD

Edit source code - Do some source code editing again

gedit kernel/core/module.rb

Commit changes - commit change in current branch

git commit -a -s # vi editor will be popped up for entering committing message

Create patch - create patch containing changes between 2 branches local 'master' and remote 'origin/master'

git checkout master # switch to branch 'master'
git format-patch origin/master --stdout # to standard output
git format-patch origin/master -o ../patch # create formated patch as a file and store in directory ../patch

Create a ticket - goto in http://rubinius.lighthouseapp.com/m create a ticket and attach a patch file to it

Revert change - switch to branch named 'master' and revert back directory tree in original state, all changes are lost

git checkout master
git reset --hard HEAD

Resolve conflict - if there is a conflict on file, git stop doing and ask for manually resolve, to inform that the conflict has been resolved on e.g. kernel/core/module.rb, use

gedit kernel/core/module.rb # after manually resolve conflict
git add kernel/core/module.rb # inform git that conflict on file kernel/core/module.rb is resolved

Apply patch

git-apply ../patch/0001-Fixes-for-Class-include-and-Class-extend-to-handle-c.patch

Apply patch partly - if a patch contains modification of multi files and there are conflict with some of them during patching, then use --reject option to modify only non-conflicted files and create filename.rej for those conflicted

git-apply --reject ../patch/0001-Fixes-for-Class-include-and-Class-extend-to-handle-c.patch

Remove file from stage Making change into git repository actually happens in two steps, 1) add the new/modified file into stage area 2) commit stage area. If for some reason, we want to revert the step 1) we can do as follow

$git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached ..." to unstage)
#
# new file: dict_parser/tests/dict_tests.py
#
$ git rm --cached dict_parser/tests/dict_tests.py
rm 'dict_parser/tests/dict_tests.py'
$ git status
# On branch master
#
# Initial commit
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
# dict_parser/

Git getting popular now when many open source projects moved from subversion to git, Zack Rusin has done very nice git cheat sheet summarizing git usage scenarios below

Ruby Virtual Machine's Implementation

2007-12-21T18:06:00.001+09:00

link to The Great Ruby Shootout

rails-2.0 released summary of features

2007-12-09T13:21:00.000+09:00

link to rails-2.0 released summary of features

Martin Fowler DSL Book

2007-11-24T11:10:00.000+09:00

link to DSL Book work in progress

Nice test code sample using mocha

2007-11-17T19:38:00.002+09:00

link to Mock Object Injection
link to Mocking Jruby
link to Boost your Java Test with Ruby and JtestR

Useful unix tricks

2007-10-13T11:04:00.000+09:00

link touseful unix tricks from Paul Gross
part 1
part 2

Sea Fish

2007-10-02T09:22:00.000+09:00

This picture named 'Sea Fish' is from Le Ngoc Mai, my 8 year old daughter.

How to organize Rails model

2007-09-11T21:27:00.000+09:00

Link to How to organize Rails model

Calling Java from JRuby

2007-09-07T08:44:00.000+09:00

Current JRuby is lack of detail documentation with regard to Java integration, so in certain cases it take me and my colleague a while to figure out how to do.
String to/from Java byte's array
In our recent application, we need to pass image from Ruby to image processing method written in Java. The Java method accept byte array and has result as byte array also. So we need convert Ruby String to Java byte[] and back.
The

require 'java' add two methods to Ruby String class 
String#to_java_bytes # instance method return Java byte[] 
String::from_java_bytes # class method return Ruby string from Java byte[]

The below code snippet demonstrate it.
require 'java'
include_class 'net.csetech.sq.image.ImageHelper'

File.open('duongpl.jpg') do |f|
  f.binmode
  @original = f.read
end

@reduced_j = ImageHelper.reduce_to_fix_size(@original.to_java_bytes,15000)
@reduced_r = String.from_java_bytes(@reduced_j)

File.open('duongpl-15k.jpg','w') do |f|
  f.binmode
  f.write(@reduced_r)
end

Ruby Array to Java Array

Other example, that we may need is to convert Ruby Array to Java Array. The require 'java' add method Array::to_java(Symbol), that convert Ruby Array to Java Array, where Symbol represent Java Class of the Array's elements. If it is omitted, the result is Java Object[].
['a','b'].to_array(:String) # return Java String[]
['a','b'].to_array # return java Object[]

More about method to_array is on JRuby Cookbook



API design
2007-09-01T13:41:00.000+09:00
Link to  API: Design Matters

Link to 
How to Design a Good API & Why it Matters


ActiveRecord-JDBC 0.5 can not insert object with pre-assigned Id into Oracle Database
2007-09-01T13:01:00.001+09:00
If you encountered problem when try to insert object with pre-assigned Id into Oracle Database in JRuby 1.0.1 with ActiveRecord-JDBC 0.5, here is quick fix
module ::JdbcSpec
  module Oracle
     def insert(sql, name = nil, pk = nil, id_value =nil, sequence_name = nil) #:nodoc:
      if pk.nil? || id_value
        execute sql, name
      else # Assume the sql contains a bind-variable for the id
        id_value = select_one(
        "select #{sequence_name}.nextval id from dual")['id'].to_i 
        log(sql, name) {
          @connection.execute_id_insert(sql,id_value)
        }
      end
      id_value
    end
  end
end

The problem is on compatibility of Oracle JDBC driver and execute_insert method of java class JdbcAdapterInternalService. The fix simply remove the usage of execute_insert method. Verification code is below
require 'rubygems'
gem 'ActiveRecord-JDBC','0.5'
require 'jdbc_adapter'
require 'active_record'

ActiveRecord::Base.establish_connection(
 :adapter  => 'jdbc',
 :driver   => 'oracle.jdbc.driver.OracleDriver', 
 :url      => 'jdbc:oracle:thin:@localhost:1521:DEV',
 :username => "scott",
 :password => "tiger",
)

class Emp < ActiveRecord::Base
  set_table_name 'emp'
  set_primary_key 'empno'
end

emp = Emp.new
emp.id = 9999
emp.save



Ruby require idiom
2007-08-19T12:02:00.002+09:00
Ruby Kernel#require(filename) read and execute the specified file only once regardless how many time we call it as opposite to Kernel#load(filename) which reload the file each time.

With application having many files, each depend on others without this behavior of Kernel#require(filename), a single file will be read executed multi time, which may not desired from  functional and performance's perspective.


The Kernel#require(filename) however has an issue, after loading it puts path of a file required in global array $" and does not know if we refer to the same file using different path. e.g.

#file : lib/foo.rb
puts "loading #{__FILE__}"

#file : lib/bar.rb
require File.dirname(__FILE__) + '/foo'
puts "loading #{__FILE__}"

#file : app/runme.rb

require File.dirname(__FILE__) +'/../lib/foo'
require File.dirname(__FILE__) +'/../lib/bar'
puts $"

Then run
ruby app/runme.rb

loading ./app/../lib/foo.rb
loading ./app/../lib/foo.rb
loading ./app/../lib/bar.rb
["app/../lib/foo.rb", "./app/../lib/foo.rb", "app/../lib/bar.rb"]


There are basically few well known techniques to deal with this problem 


1. Using absolute path 


2. Modifying $LOAD_PATH 


3. Using defined? 





USING ABSOLUTE PATH



In this variant we always call  Kernel#require with  a absolute path, the File::expand_path will be used to remove '..' symbol representing parent directory  e.g
#file : lib/bar.rb
require File.expand_path(File.dirname(__FILE__)) + '/foo'
puts "loading #{__FILE__}"

#file : app/runme.rb
require File.expand_path(File.dirname(__FILE__)+'/../lib')+'/foo'
require File.expand_path(File.dirname(__FILE__)+'/../lib')+'/bar'
puts $"

run
ruby app/runme.rb

loading D:/huy/rubyapp/require_1/lib/foo.rb
loading D:/huy/rubyapp/require_1/lib/bar.rb
["D:/huy/rubyapp/require_1/lib/foo.rb", "D:/huy/rubyapp/require_1/lib/bar.rb"]

This method is described in post ruby require idiom



MODIFYING $LOAD_PATH 



The second quite popular technique is to modify $LOAD_PATH directly e.g.

#file : lib/bar.rb

libpath=File.expand_path(File.dirname(__FILE__)+'/lib')
$LOAD_PATH.unshift(libpath) unless $LOAD_PATH.first==libpath

require 'foo'
puts "loading #{__FILE__}"

#file : app/runme.rb

libpath=File.expand_path(File.dirname(__FILE__)+'/lib')
$LOAD_PATH.unshift(libpath) unless $LOAD_PATH.first==libpath

require 'foo'
require 'bar'
puts $"


In the previous mentioned techniques, the File#expand_path method is used to get absolute path of either file or directory, the same file or directory is kept only once in a relevant global variable. 



USING defined? 


This technique is very old and frequently used by C programmers to guard header file to include multiple e.g.
#file foo.h

#ifdef __FOO_H

#define __FOO_H

...

#endif
 
In Ruby, I have seen this technique being applied using Kernel#defined? e.g.

#file : lib/foo.rb
unless defined?(FooDefined)
   FooDefined=true
   puts "loading #{__FILE__}"
end

#file : lib/bar.rb
unless defined?(BarDefined)
  BarDefined=true
  require File.dirname(__FILE__) + '/foo'
  puts "loading #{__FILE__}"
end

#file : app/runme.rb

require File.dirname(__FILE__)+'/../lib/foo'
require File.dirname(__FILE__)+'/../lib/bar'
puts $"

UPDATE on 26-12-2007 

The new Kernel#require in Ruby 1.9 store full path in $" make this article obsolete.


Turn off color in output of ls command
2007-07-28T20:35:00.000+09:00
Output of ls command in color mode sometime causes a problem to my eyes. Turn it off is simple, just put 
unalias ls

in .bash_profile or .bashrc (in case of Ubuntu Feisty)


Fixture path in ActiveRecord Testing
2007-07-26T10:13:00.001+09:00
I am using fixture in ActiveRecord Testing. I have many test classes, each using fixtures located in different directories. Individual test work fine, but fails when running together in suite. The problem is fixture_path is class attribute/variable of Test::Unit::TestCase so all subclasses of it share the same value.
require 'rubygems'
require 'active_record'
require 'active_record/fixtures'

class TestFoo < Test::Unit::TestCase
   self.fixture_path='/u01/test/fixtures/foo'
end

TestFoo.fixture_path # "/u01/test/fixtures/foo"

class TestBar < Test::Unit::TestCase
   self.fixture_path='/u01/test/fixtures/bar'
end

TestBar.new().fixture_path # "/u01/test/fixtures/bar"
TestFoo.new().fixture_path # "/u01/test/fixtures/bar"

Because the Test Unit framework, load all test classes, then run them, so TestFoo.new().fixture_path get value of TestBar.new().fixture_path. The solution is simple, just define a method fixture_path in each test class (in upcoming version after ActiveRecord 1.15.3, the problem is fixed by change fixture_path to class instance variable see ticket 6672). 
class TestFoo < Test::Unit::TestCase
   def fixture_path
      '/u01/test/fixtures/foo'
   end
end

class TestBar < Test::Unit::TestCase
   def fixture_path
      '/u01/test/fixtures/bar'
   end
end

TestBar.new().fixture_path # "/u01/test/fixtures/bar"
TestFoo.new().fixture_path # "/u01/test/fixtures/foo"




Exception handling
2007-07-18T18:19:00.001+09:00
I have seen a lot of code, that handle exception in the following way
begin
  do_some_things()
rescue
  #... print message, closing file/socket etc. 
end

There is a problem with this style. If there is error, we lost information where it happens because it is quite common that do_some_things() again call methods of others classes located in different files and so all. 

I think that it should be a rule, that we alway print/save to logfile backtrace of exception in every exception handling code
begin
  do_some_things()
rescue => e
  require 'pp'
  pp e
  pp e.backtrace
  #... print message, closing file/socket etc. 
end

The backtrace information will help us to identify bug during development and to analyze root cause of exception during operation.


String concatenation
2007-07-18T09:39:00.000+09:00
One of common thing that any programs do is concatenate two or more String value, e.g.
first_name = 'huy'
sir_name = 'le'
name = first_name + ' ' + sir_name # huy le
 
But the above code suffers a problem, if one of concatenated variables is nil, then it will not work. Checking nil is tedious and error prone, a simple solution of this problem is putting concatenated variables into an array, removing nil using compact and using join method to perform concatenation.
first_name= 'huy'
middle_name=nil
sir_name = 'le'
name = [first_name, middle_name, sir_name].compact.join(' ') # huy le



Standard_params to improve readability of tests
2007-07-07T10:01:00.000+09:00
Link for standard_params to improve readability of tests


Ruby block and command/query separation
2007-07-01T15:50:00.000+09:00
The recent post Ruby block style do end versus {} on where to use do ... end versus  {...} to enclose Ruby block reminds me command/query separation OOP principle.


Easy verification of API behavior via irb
2007-06-29T09:07:00.001+09:00
Learning programming language is not just learning language syntax, programmers including myself spend lot of time learning various API. Irb is wonderful tool for learning Ruby API, when I do program in Ruby, I always have Irb open, and I try to run a method of API including core lib, that I am not sure how it works. Comparing to Java, it is much faster and have better productivity.


Open classes
2007-06-29T09:06:00.001+09:00
When I want to change or add new behavior to existing class, static typed language give me only two options create new class that either extend existing class or delegate to existing class (in Java world some peoples do aspect oriented programming AOP, but it is little bit complicated and is external to the language).

Ruby give me an ability to modify existing class directly. A cross concern or aspect then can be added easily by modifying existing class.


ADDING NEW FEATURE TO EXISTING CLASS

class Object
   def blank?
      return true if nil?
      return true if respond_to?(:empty?) && empty?
      return false
   end
end
nil.blank? #=> true
''.blank?  #=>true
[].blank?  #=>true
{}.blank?  #=>true
'hello'.blank? #false
[:a].blank? #false
{:a=>1}.blank? #false

The above code define method blank? in class Object, which is root of all Ruby classes. This will result in well behaving instances of NilClass, String, Array, Hash.


DECORATION A METHOD OF EXISTING CLASS 

To decorate a method of existing class is just easy as adding new one e.g.
require 'active_support'
class Hash
    alias original_symbolize_keys! symbolize_keys!

    def symbolize_keys!
        each_value do |value|
             value.symbolize_keys! if value.is_a?(Hash)
        end
        original_symbolize_keys!
    end
end

Rails ActiveSupport add method symbolize_keys!, above code decorate it to convert keys to symbol for a hash value if it is also hash.

When modifying an existing class, we shall make sure that the class is already loaded, otherwise we may get a strange behavior. This is specially important in rails, when classes are loaded dynamically using const_missing method. About how to handle this problem, there is good tip reopen with class/module eval on practical ruby blog. Other way to decorate method without using alias is available on replacing methods.


Using of keyword based parameter
2007-06-27T15:32:00.000+09:00
I plan to have presentation of Ruby Language in enterprise application development for developers of the company I worked for and I am thinking about which message should I pass to them. Instead of trying to give many reasons that others already talked about huge benefit of using Ruby, I decide to  give my own very personal reasons why I love Ruby based on my one and half year working with this language. 

In early day, when I programed in PL/SQL, I know that PL/SQL support keyword based parameter in procedure/function but I never used this feature, simply I have not recognized the benefit of using it that time.

Ruby does not support keyword based parameter but Ruby programmers fake it easily using  hash in combination with symbol. Using this style is very popular in Ruby core lib and Rails. The following example illustrates it, suppose we want to create method create_user that create database user, in traditional position based parameter, we will do like that
def create_user(username,password,ignore_error,force,verbose)
   #.. implementation detail is ignored
end
#calling it
create_user('scott','tiger',false,true,true)

on keyword based version, the method has simply one parameter params
def create_user(params)
   username = params[:username]
   password = params[:password]
   ignore_error = params[:ignore_error]
   verbose = params[:verbose]
   force= params[:force]
   #.. implementation detail is ignore
end
#calling it
create_user(:username=>'scott',:password=>'tiger',:ignore_error=>false,
  :verbose=>true, :force=>true)

The keyword based version is obviously more verbose, requires more typing, but offers several benefits.



MORE EXPRESSIVE AND LESS ERROR

The keyword based version is more expressive, just by looking at how the method is called, we know what is the intention, there is no need to look at the implementation file to figure out what it does.
The position based version suffers what Joshua Bloch mentioned in his How to Design a Good API and Why it Matters "Long lists of identically typed params harmful"



GOOD DEFAULT

In the position based version, we can only assign default value for those parameters that are at the end of parameter list.
def create_user(username,password=username,ignore_error=false,force=false,verbose=false)
   #.. implementation detail is ignored
end
#calling it
create_user('scott')

This will not give a flexibility of using default value just for few last one.
In the keyword based version, we can archive it easily as follow
def create_user(params)
   username = params[:username]
   password = params[:password] || username
   ignore_error = params[:ignore_error] 
   verbose = params[:verbose]
   force= params[:force]
   
   #implementation detail is ignored
   
   #note that Ruby consider nil as false in condition expression, so we should design 
   #boolean value keyword in such way that its default value is false
end

#calling it
create_user(:username=>'scott',:verbose=>true)
create_user(:username=>'scott',:force=>true)
create_user(:username=>'scott',:ignore_error=>true)

I also see people using Hash::merge to shorten assignment of default values and adding some assertion of accepted keywords e.g
#borrow from ActiveSupport
class Hash
  def assert_valid_keys(*valid_keys)
    unknown_keys = keys - [valid_keys].flatten
    raise(ArgumentError,
    "Unknown key(s): #{unknown_keys.join(", ")}\nValid key(s): #{valid_keys.join(',')}")   
      unless unknown_keys.empty?
  end    
end

class DatabaseSchemaBuilder
  
  attr_accessor :ignore_error,:verbose,:force

  def default_options
    {:ignore_error=>@ignore_error,:verbose=>@verbose,:force=>@force}
  end

  def create_user(params)
    params.assert_valid_keys(:ignore_error,:verbose,:force)
    params = default_options.merge(params)
   
    username = params[:username]
    password = params[:password] 
    ignore_error = params[:ignore_error]
    verbose = params[:verbose]
    force= params[:force]

   #implementation detail is ignored
  end
end

EASY TO CHANGE

When we need lets say adding new parameter to the method. In position based version, we end up with changing contract of the method, which may result in looking at every line of code that use this method and make change unless you put at the end of parameter list with default value.

In keyword based version, it is obvious less painful, just adding one more keyword, setting a default value, anyway change only the method itself, e.g. we want to add parameter :noop, meaning no operation, just for testing.
def default_options
   {:ignore_error=>false,:verbose=>false,:force=>false,:noop=>false}
end

def create_user(params)
   params.assert_valid_keys(:ignore_error,:verbose,:force,:noop)
   params = default_options.merge(params)
   
   username = params[:username]
   password = params[:password] || username
   ignore_error = params[:ignore_error]
   verbose = params[:verbose]
   force= params[:force]
   noop=params[:noop]

   #implementation detail is ignored
end

HIDING DESIGN DECISION

Using position based parameter with little bit long parameter list, I have to decide if put one parameter before other or not. I do not have this problem when using keyword based parameter, so it help me do program faster.


Ant depend task
2007-06-19T15:42:00.000+09:00
I am writing ruby helper class that will be used in Rakefile to build complex java application. Everything seem to be quite simple, I just copied what has been done by Matt Foemmel in his JRake. The ruby helper class has a method that check if java class is upto date by comparing access time of the class file and the java source file. I believe that Ant javac task does the same. 

Using the ruby helper file, my java application most of time get build correctly but sometime, few class files are not upto date. It take me a while to figure out what is going wrong. 

The problem is when a java class depends on other, if other class changes then the java file shall be recompiled even though it is not modified. Looking at Ant documentation, I found that Ant solve this problem using depend task. 

In order to solve this problem in ruby, I have to do the same as Ant depend task does, read our class file, extract all class references verify if these class references are changed.


Format of Date/Time when using ActiveRecord with Oracle Database
2007-06-19T07:36:00.001+09:00
I found no place in ActiveRecord Documentation, that describes how to enter Date/Time into Oracle Database. By looking at source code of OracleAdapter in ActiveRecord, I recognized that ActiveRecord uses Oracle alter session command to specify date format 
ActiveRecord::Base.connection.execute "alter session set NLS_DATE_FORMAT='YYYY-MM-DD HH24:MI:SS'"

As implication we shall use this format (e.g '2007-06-30' and '2007-06-30 23:01:45' String are accepted) when entering data (e.g in fixture file) to Date Field.


My resume - SUMMARY OF SKILLS
2007-06-18T10:38:00.007+09:00
Business Analysis & Leadership
Able to attract, retain and lead people. Led team up to 25 developers, responsible for task planning, technical direction, risk management, code inspections, quality control, performance reviews, staff development. Understood how to create working environment to keep people happy, motivated, creative and productive.
Solid presentation skill, proven in communication with customers, understanding problems, capturing domain knowledge and turning it into high level requirements in suitable form for development team and other stakeholders.
Domain Knowledge
Financial industry: retail and investment banking, automated trading systems, communication protocols including SWIFT, ISO8583 and FIX. Involved in proposal preparation, conceptual definition and prototyping automated trading system for brokerages based on Marketcetera platform.
Communication industry: Business processes, practices and standards in communication industry including eTom, SID, OSSJ, Mobile Network Architecture, IN, and TAP3. Hands-on experience in ERP implementation including Billing, Fulfillment, and Accounting.
Government sector: Hand-on experience in government sector including law enforcement, criminal justice, immigration. Good knowledge in biometrics technology (face and fingerprint) and it's application in government's projects including national ID Cards, and Immigration.
Software Development
More than 15 years experience in developing software products with deep knowledge of architecture / design of multi-tiered client-server systems based on middleware including Tuxedo, J2EE (WebSphere, WebLogic) and other emerging Web framework as Ruby on Rails.
Skillful in many programming languages including C/C++, JAVA, SQL, PL/SQL, RUBY. Passionate for the simple and easy to understand software with rich domain model. Well versed in OOP, design patterns (Design Patterns - GoF, Pattern for Enterprise Application Architecture - PEAA, Domain Driven Design).
Strong knowledge of algorithms and data structures and their applications in writing a high performance, time critical system for complex business domain.
Extensive experience with state of art software development practices such as pair programming, Test Driven Development (TDD), refactoring, continuous integration.
System Integration Using Messaging System
More then 7 years in design and implementation of software systems that communicate with each other using asynchronous messages. Understand message creation and processing patterns (Enterprise Integration Patterns) and their applications in real world systems.
Engaged as system architect in many projects that use Event-driven Architecture (EDA) and Service Oriented Architecture (SOA) integration style.
Infrastructure's Design, Implementation & Administration
More than 10 years experience in design, implementation and ongoing maintenance of complex IT infrastructure, that includes J2EE middleware (WebSphere AS, WebLogic), messaging system (MQ), database (Oracle), Unix servers, clustering system, IBM SAN DS series, IBM Tape libraries TS series, Tivoli Storage Manager, CISCO router, PIX firewall and VMWARE virtual technology.
Proven in writing script (SHELL, RUBY, PYTHON) to automate repetitive administrative activities including monitoring, deployment, configuration change, etc.



My resume - EMPLOYMENT HISTORY
2007-06-13T21:55:00.015+09:00

ING Direct Spain, Jul. 2009 - Present

Infrastructure's Software Engineer



Administer softwware infrastructure of WebSphere Application Server, JBOSS AS for direct banking system



ING Direct Japan, Sep. 2008 - Apr. 2009

Technical Analyst WebSphere


Maintain middleware infrastructure of WebSphere Application Server and WebSphere MQ for core banking system
Plan, install, configure and troubleshoot large scale WebSphere Application Server ND and MQ in AIX environment
Perform application software deployment, update, performance tuning and other house keeping activities
Provide assistant for application development team in problem identification and solving



Middleware Monitoring System: directly involved in design, develop and implement monitoring system for middleware infrastructure including exception, performance, capacity monitoring
Application Software Deployment Automation: directly involved in design and implementation system for automatic application software update that ensure fast, reliable, error free deployment with no downtime
Centre of Software Engineering - Vietnam, Dec. 1998 - Jul. 2008
Technical Director
Managed software development process and quality
Prepared technical proposal and system architecture
Provided training/mentoring on software design, pattern, refactoring, testing
Participated in various development/implementation projects by rotating between different roles according to the need at each project phase (team leader, project manager, business analyst, software developer)
Border Control System: managed the project to design, develop a J2EE application for immigration department and deploy under BEA WebLogic in major international airports across the country, integrate it with existing backend system using messaging middleware and perform data migration. Responsible for all technical aspects including overall architecture, software development, performance, quality control, server installation and production support.
Data cleansing for law enforcement agency: led a team to perform cleansing of a massive database of hundreds of million of records. Directly involved in design and development algorithm, data structure and rule engine in C/C++.
Utilities Billing - Hanoi Water Work Company: Led a team that implemented a distributed billing and customer management system based on ORACLE and JAVA. In charge of overall system design (including custom developed system, IT infrastructure component selection, data migration strategy) and development of data synchronization module using JMS.
Telecom Billing and Order Fulfillment System: Responsible for implementation of high performance (up to 20,000 event per second), nearly real time event rating engine using C/C++, Tuxedo and database in memory technology.
Personnel Management Information System - Ministry of Home Affair: Managed the development and implementation the Personnel Management Information System for public sector fund by Swedish Agency for Development Cooperation. The system was developed mainly using ORACLE PL/SQL and ORACLE Database.
Electronic Library - Institute of Mathematics - Vietnam: Participated in concept definition, architecture and technology selection (ORACLE and PERL).
Centre of Software Engineering - Vietnam, Jul. 1997 - Nov. 1998
Software Developer, Database Administrator, Team Leader
Performed installation, maintenance, support ORACLE Database System
Designed, developed and implemented framework/library for data synchronization
Utilities Billing - Hochiminh Cities Water Supply Company: managed a final phase of the project to implement of billing and customer's care for Hochiminh Cities Water Supply Company using ORACLE PL/SQL and ORACLE Database. Directly involved in development of module for data synchronization between branches and head quarter
Document management system: Installed, maintained, provided support ORACLE Database System ,Ministry of Science and Technology .
Notia Information System - Czech Republic, Apr. 1994 - May 1997
Software developer, Team leader
Performed installation, maintenance, support ORACLE Database System
Participated in various software development projects under different roles team leader, software developer, database administrator, technical support personnel
Custom developed ERP - Altron Czech Republic: Led development team in the project to develop and implement of ERP for a distribution and retail company in Czech Republic using ORACLE database design and GUI development in Borland Delphi.
Fixed Asset Management - Eurotel Prague: Participated in design a fixed asset management software for a mobile phone company based in Prague.
ERP Package Software: Responsible for development and support of C++ part A/R, A/P of ERP package software, that have been deployed to hundreds of customer in Czech Republic.
Metasoft - Czech Republic, Jun. 1993 - Jan. 1994
Software Developer
Developed, customized software, provided technical support
Implemented software supporting activities of an accommodation agency



irb and TAB autocompletion
2007-06-13T11:07:00.000+09:00
To enable autocompletion on irb, run
irb
require 'irb/completion'

Note to hit double TAB on your keyboard not single one.
One of most comprehensive description can be found here


My resume - TRAINING AND CERTIFICATION
2007-06-12T10:15:00.004+09:00
Oracle E*Business Suite Training (one week at IBM Singapore) - 2005
BEA Tuxedo 8 Administration Training (one week at BEA in Malaysia) - 2002
Project Management Training (3 weeks at Asia Institute of Technology (AIT) Thailand) - 2000
Software Engineering Conference (CASE University of Technology &Thomson CSF) - 1999
AIX 4.3 System Administration Certification (IBM Certified) - 1999
Oracle Certified Professional on Oracle8 DBA - 1999
RS/6000 Solution Sales Certification (IBM Certified) - 1999



Switch Off Convert line breaks
2007-06-12T08:05:00.000+09:00
I have problem when try to post some html stuff on my blog. The posted html contains simple table. However Blogger incorrectly displays it, alignment doesn't work, a lot of strange space. I finally recognized that, this is due to enabling "Convert line breaks" feature of Blogger, that encloses line break into html break tag, I switch it off, and everything work well.


My resume - PROGRAMMING LANGUAGE AND TOOLS
2007-06-12T06:15:00.007+09:00
Operating Systems
AIX 4.3/5L, HACMP 4.3, Solaris 7/8, SCO Unix, Windows NT/2000/XP, Linux (Redhat Debian, Ubuntu)
Middleware
Websphere ND 6, WebLogic Server 8.1, Websphere MQ 6, Tuxedo 8.0, Tomcat, Ruby on Rails
Database
Oracle8i/9i/10g, Oracle Real Application Cluster, MySQL, Microsoft SQL Server 6.5, Microsoft Access
Programming Languages
JAVA, C/C++, Ruby, Oracle PL/SQL, Visual Basic, PASCAL, PROLOG
Tools
IDEA IntelliJ, Microsoft Visual C++, Rational Rose, Oracle Designer/Developer, Borland Delphi, Toplink, iBATIS, Subversion,Git, Svn, Ant, Maven, Cruisecontrol , JIRA, Mingle, Microsoft Project & Office Suite



The secret behind Rails Integration Test
2007-06-10T22:07:00.000+09:00
Find what is behind Rails Integration Test on good blog's post named HeadlessApp
$ script/console
Loading development environment.

>> app.class
=> ActionController::Integration::Session

>> app.get "/home"
=> 302

>> app.controller.params
=> {"action"=>"home", "controller"=>"accounts"}

>> app.response.redirect_url
=> "http://www.example.com/login"



gem's environment
2007-06-09T16:00:00.000+09:00
When playing with gem on my Ubuntu machine, I found an useful command
huy@huy-desktop:~/.gem$ gem environment 
Rubygems Environment:
  - VERSION: 0.9.0 (0.9.0)
  - INSTALLATION DIRECTORY: /var/lib/gems/1.8
  - GEM PATH:
     - /var/lib/gems/1.8
  - REMOTE SOURCES:
     - http://gems.rubyforge.org

that shows default gem's environment. By setting the GEM_PATH environment variable to one desire directory, we can share single gem repository across many ruby installations (e.g C ruby and JRuby). The full descriptive information of all gem environments variables can be found in Gem Command References


Naming Exception Corba Comm Failure
2007-06-07T15:51:00.000+09:00

Java RMI, Corba, J2EE are sometime still nightmare for developers including these with more than 2 years working experiences with these technologies like myself. I have encountered a "Naming ExceptionCorba Comm Failure" when remote client try to create InitialContext to my Weblogic server. I did check all small details, everything seem to be OK. My Weblogic server is configured to listen on single host interface named "app-svr" that represents a valid IP address 192.168.2.79. My remote client is configured connect to exact same address 192.168.2.79. However it does not work. 
At some moment, I changed address that Weblogic server is listening from "app-svr" directly to IP address 192.168.2.79, and suddenly it starts work. 
It also works when I change back listening IP address of Weblogic server to "app-svr" and at the same time put "app-svr 192.168.2.79" in to hosts file of my remote client. Such behavior of the JVM RMI implementation can cause big trouble most JAVA developers including myself.


Cruisecontrol.rb is too slow
2007-06-01T20:20:00.000+09:00
I encountered a problem with Cruisecontrol.rb, the method link_to_code(log) is too slow to process a certain output log, sometime it takes 10 seconds making Cruisecontrol not workable. So for the time being, as workaround, I just comment out this code . 
#file: app/helpers/builds_helper.rb
  def link_to_code(log)
    log
=begin    
    @work_path ||= File.expand_path(@project.path + '/work')

    log.gsub(/((\#\{RAILS_ROOT\}\/)?([\w\.-]*\/[ \w\/\.-]+)\:(\d+))/) do
      path, line = File.expand_path($3, @work_path), $4
      
      if path.index(@work_path) == 0
        path = path[@work_path.size..-1]
        link_to ".#{path}:#{line}", "/projects/code/#{@project.name}#{path}?line=#{line}##{line}"
      else
        $1
      end
    end
=end
  end



ActiveRecord testing ouside of Rails
2007-05-25T19:39:00.000+09:00
There is many tips about how to use ActiveRecord outside of Rails but rarely about ActiveRecord testing ouside of Rails. I am working on project that use ActiveRecord ouside of Rails and need to create fixture for testing. It take me a little bit of time to figure out how to do. The source code below demonstrates the trick

require 'test/unit'
require "active_record"
require "active_record/fixtures"

ActiveRecord::Base.establish_connection(
 :adapter => "mysql",
 :host => "localhost",
 :username => "nightlybatch",
 :password => "secret",
 :database => "web_orders"
)

#this is because fixture verify existence of rails database configuration
ActiveRecord::Base.configurations[:test] = ActiveRecord::Base.connection
  
class Product < ActiveRecord::Base
end

class TestFoo < Test::Unit::TestCase
 #set fixture path explicitly
 self.fixture_path="c:/temp"
 
 fixtures :products

 def test_ruby_on_rails_should_exist
  assert products(:ruby_on_rails)
 end

end


In order to pass the test, the fixture file products.yml shall be created in directory c:/temp, and products table shall be created in MySQL database 
#c:/temp/products.yml
ruby_on_rails:
   id:   1
   name: ruby on rails
rails_recipes:
   id:   2
   name: rails recipes




Ruby adoption in Vietnam
2007-05-24T17:41:00.000+09:00
From the time, I know ruby, it is my favorite language, but it seems that no so many developer in Vietnam is using it. Today I have found Vietnam Ruby site. It is good news, that ruby got an attention in Vietnam and I am not alone, who see the huge benefit of wonderful and humane programming language.


Notia
2007-05-19T14:45:00.001+09:00
At my early time in Notia (in 1993 if my memory is correct), I has participated in a project, that develop a ERP package that includes General Ledges, Account Payable, Account Receivable, Warehouse Inventory modules. The technology used that time was Novell B*trieve as database engine, C++ and  for business logic, Borland Turbo Vision for C++ for GUI. At that time, Tran Duc Trung was leader, based on university compiler's knowledge, he developed a kind of Domain Specific Language, that allow end-users or implementors define posting rules using that DSL. The "posting rules" DSL is comprehensive featuring expression, AND, OR boolean condition, with Account, Money concept. Other team member, Martin Smid went so far and defined other DSL for reporting function. we called it FRT.  Until now, I have not seen better reporting tool with such flexibility, elegance, and easy to used.   Other other side, we also faced a lot problems. One of them is that MS*DOS can not run program with more than 1MB static data segment. C++ is also not good choice for GUI development, we encountered a lot of errors related to allocation/free memory, using pointer. And one of most ironical is in using of the DSL itself. The software developers didn't understand well accounting to be honest we was not willing to learn it on the other hand implementors and customers was not matured enough in using the DSL in effective manner. Despite these facts, Notia was able to sold the software and implement it in hundreds customers including few big names as ABB.


ruby module_function
2007-05-10T16:56:00.000+09:00
Ruby 'module' allows us to define methods then turn these methods into instance methods of a class by using key word 'include'. In ruby world, peoples call this feature 'mixin'. e.g.

module Foo
   def hello
      puts 'hello'
   end
end
class Bar
include Foo
end
Bar.new.hello #=> hello


Sometime we need to call method defined in a module without including the module into a class. We can do it by

module Foo
  def self.world
     puts 'world'
  end
end
Foo.world #=> world


There is a question how do we turn method 'hello' so it can be used in both mixin style and standalone . The answer is using kernel method 'module_function'

module Foo
   module_function :hello
end
Foo.hello + ' ' + Foo.world # => hello world




Install Ubuntu 7.04
2007-04-29T14:42:00.000+09:00

I took time during holiday to install Ubuntu 7.04 (code name Feisty) on my home computer. I thought for people with Unix background like myself, it should be simple exercise. It is not. My home computer is old Pentium IV 2.26 MHz, 768 MB RAM with installed Windows XP SP2 and I decided to install Ubuntu as guest OS on virtual machine using Vmware Workstation. First I used Vmware Workstation to create a machine with 6 GB Hard disk. Since Ubuntu is not on list of Vmware available guest OS, I just selected other Linux as guest OS for my virtual machine. I booted the virtual machine with fresh downloaded Ubuntu 7.04 and followed the instruction on the screen. The first installation got stuck at 69 %, I waited 10 minutes then reboot and try second attempt, that went well until completion. Now I can login using the account/password asked and created by installation program. My first impression is not bad, the screen looks nice on 1024x768 resolution, even sound card works. Pre-installed Firefox 2.0 work smoothly even on virtual machine, Ubuntu accepts an IP address assigned from my ADSL Internet modem automatically thus I got connection to Internet.

1. issue sudo: I want to install some additional software as root however what is password of my well known root?. After few minutes investigation, I recognized that Ubuntu using sudo as mechanism to allow normal account access to root and by default root is locked and no one can neither login as root nor su. To run program using root account, we have type e.g.
 
sudo shutdown -r +15 "quick reboot"

sudo will ask us to enter our password and then run shutdown using root account. I known that sudo mechanism is used also in Mac OSx 10.4 Tiger and it offers certain benefits against traditional root, however I can not change my habit so I unlock root account by enter
sudo passwd 

2. issue apt-get: I want to try ruby so I type in command line terminal
ruby

Ubuntu is intelligent enough to answer that ruby is not installed and offer me that I can install ruby by using
sudo apt-get install ruby

I tried and it works. The apt-get is Ubuntu package manager (now I know that it is from Debian and Ubuntu is rooted from Debian). It does all things: downloads required software including dependencies, installs and configures. If we does not know name of software package, then Synaptic Package Manager is application that offers features such as listing, installing and uninstalling software package.
apt-get use configuration on /etc/apt/sources.list, which contains list of ftp servers storing debian distribution package.   



After adding new repository into /etc/apt/sources.list, hit 'reload' button to get package information from the new added repository. 

3. issue add path: I download and install SUN JDK 1.5.0_11 and want to add JDK 1.5 to PATH so I can type java, javac, etc. I created .bash_profile in home directory and add two following line
export JDK_HOME=~/jdk1.5.0_11
export PATH=~/jdk1.5.0_11/bin:$PATH

and do logout/login, it doesn't work. I tried to put these line to .xssesion, after that I can't login to the system anymore and have to login using failsafe session to remove this .xsession file. Looking at documentation and even googling doesn't help. At the end I realized that I should add these line at the end of .bashrc. Then it works in both cases either I login using X-Desktop or terminal using Windows putty.

4. issue Vmware tools: we can not install Vmware tools using Vmware provided package, the Feisty kernel version simply does not match. However Ubuntu provides its own vmware tool package. Installation is pretty simple, just type
uname -a

to get kernel version, my kernel version is 2.6.20-15 and then invoke apt-get
 
sudo apt-get install vmware-tools-kernel-modules-2.6.20-15

Finally reboot to take effect. To be honest, I have not observed better response from any application as Vmware claimed.

5. issue update time: To keep your clock synchronized, install ntpdate package as follows
sudo apt-get install ntpdate
sudo /etc/network/if-up.d/ntpdate
sudo crontab -e 
@hourly /etc/network/if-up.d/ntpdate

6. issue missing c-header files: Ubuntu striped down in order to keep installation CD small and installation process fast. The result is default installation does not contain packages that developers normally expect. I got into trouble when I tried to compile a c program, gcc told me that file dlfcn.h is missing. I guested that some lib is missing but can not figure out which package contains the file. After spending time on Google, I found Ubuntu Package Web Site, where I can enter file name and search for package containing that file.


Create boot image in AIX
2007-04-16T10:19:00.001+09:00
To create boot image e.g. on hdisk2 just type
# bosboot -a -d hdisk2 --> update the boot image information
# bootlist -m normal -o hdisk0 hdisk2  --> create a new bootlist
# bosboot -a  -->  update the boot image information
# bootlist -m normal -o  --> verify the bootlist is correct



Oracle import
2007-04-14T16:58:00.002+09:00
I was asked by a colleague, how to import an oracle dump file, that was exported from customer production site without data. Someone may think that it is stupid to write about this trivial task. However it is not so simple. In order to provide support for production database, we sometime have to export customer database without data and then import the exported file containing only schema and store procedure into our developer sandbox for testing purpose. The problem is that when import, Oracle try to create table and allocate data segment on disk with the same size as in the production database. Our developer sandbox of couse does not have capacity as customer production machine, so the import process often fails. Solution to this problem is perform import process into 3 steps.
In Step 1, we create table and index creation file using e.g.
$imp userid=system/manager@test file=exp-code.dmp full=yes indexfile=create-schema.sql

This will create create-schema.sql that contains tables and index creation commands. With little effort, using any text editor, we can modify this file to remove segment size parameter of each command and save it as create-schema-small.sql.
In Step 2, we run modified create-schema-small.sql against test database to create tables and indexes
$sqlplus scott/tiger@test @create-schema-small.sql

In Step 3, we run normal import with ignore error option, so Oracle will create store procedures and others required metadata.
$imp userid=system/manager@test full=yes file=exp-code.dmp ignore=yes

After reading, other colleague points out that in Oracle 10g, Oracle provide other tools expdp and impdp that can handle the above mentioned problem.


Soft Coding
2007-04-12T17:42:00.000+09:00
I have read the soft coding article recently. Based on my own experience I have a sympathy to the hard coding approach. However I think the problem is not about whether soft coding or hard coding, but programmers including myself try rigidly to follow the soft coding approach believing that it will result in more flexible code, which is not true in most cases.


Using Regexp
2007-04-08T18:24:00.000+09:00
I got a task to parse file consisting of SQL commands and various noisy lines including empty, comment and echo. I decided to use ruby String::scan to extract SQL commands from files. The only problem is how to write an correct regexp. 
I started with some test
entire=<<-EOF
create table A(id number)
/
create table B(id number)
/
EOF

entire.scan(/create table.*\//) #=>[]

I passed to String::scan a regexp that represents a String starting with 'create table' following by any number of characters  and ending with '/'. But there is something wrong instead of 
=> ["create table A(id number)\n/", "create table B(id number)\n/"]

I got a empty array 
=>[]

I have looked at documentation, tried different alternatives without success. Then I picked the book Mastering Regular Expressions and read the chapter. It is immediately clear to me, instead of using '.' as any character, I should use [^\/] mean any character different from '/'. The correct usage of regexp is
entire.scan(/create table[^\/]*\//) 
#=> ["create table A(id number)\n/", "create table B(id number)\n/"]

In order to support multi-line and avoid case sensitive, I just add two options 'm' and 'i' to the regexp
entire.scan(/create table[^\/]*\//im) 



Using Ruby and Rails with Oracle database
2007-04-02T18:33:00.001+09:00
1. Download ruby-oci8-VERSION-mswin32.rb
2. Install oci8 adapter e.g. on MS Windows
c:\ruby ruby-oci8-VERSION-mswin32.rb

3. Using with with Rails
ActiveRecord::Base.establish_connection(
 :adapter => "oci",
 :host => "sql_net_connect_string",
 :username => "foo",
 :password => "bar"
)



AIX varyon volume group in multi nodes environment
2007-04-02T10:34:00.000+09:00
On environment with multi-nodes running AIX accessing single share fiber channel storage, I am facing a problem, if one node fails, I can not import volume group residing on that physical disk in an other surviving node, although physical volume owned by that node is accessible from surviving node. OS gives out a error can not access volume group descriptor. It make me crazy, because one reason, why I need expensive fiber channel storage is to have flexibility to access volume group of failure node from surviving node. It takes me a while to figure out how it works. When AIX activate volume group, it places a lock on corresponding physical volume, so other node can not activate this volume group. The varyon command always checks the physical volume and refuses to activate that volume group if it is locked by other. It does not matter if the locking node is still living or death. In order to release the lock, we should issue on locking node
varyon -b volume_group_name 

This '-b' flag breaks disk reservations on disks locked as a result of a normal varyonvg command. Because this flag only works on volume group that is already varied on, we should do it on a node that originally owns this volume group to make sure that if that node fails, other can activate the volume group.


Writing an Use  Story
2007-04-01T13:15:00.000+09:00
Having Use Story is first step of software development cycle. How to effectively write an good, concise Use Story is challenging task of business analyst. I have found quite good guideline on that topic on Dan North What is Story


Who will develop software in 10 years?
2007-03-18T18:13:00.000+09:00
This is topic of the panel moderated by Martin Fowler on recent JAOO Conference. But the discussion goes beyond it to other interesting questions "How is it going to be done ?" and "Where it is going to be done?". The video of this panel is available on InfoQ website here.
Opinions of panellists to the first question are software is still going to be developed by software developer in next 10 years. However tools, DSL will help non-software people i.e. domain experts more involving in the process such as configuring software or create domain model and using tool to generate software. This will make the distinction between application software developer and domain experts less obvious. As result of this, an idea to build a DSL rather than an application presented in Never Build an Application becomes more acceptable.


My second job
2007-03-17T14:16:00.001+09:00
In 1992, one friend introduced me to Notia, an finance consulting, auditor and software development company in Prague. Then I joined Notia as programmer. This was one of most important milestone in my professional career and Notia was my employer until 1996. I have worked for the company in many projects, playing different role, getting  a lot of wonderful friends, learned many new things. My first team leader was Tran Duc Trung, who is now Chief Architect for NetBean at Sun Microsystem.


An answer to PragDave Quick Ruby Quiz
2007-03-10T20:17:00.000+09:00
Dave Thomas in Quick Ruby Quiz asked how to call method say_hello in 
class Fred
  class << self
    def self.say_hello
      puts "Hi!"
    end
  end
end

I have figured out one of possible way is
klass = class << Fred; self; end
klass.singleton_methods #=> ["constants","nesting",
                        # "say_hello"]
klass.say_hello #Hi!



Using Ruby flatten and to_a
2007-03-08T11:33:00.000+09:00
class Processor
  def initialize arg
  @arg = arg
  end
 def process
  puts @arg
 end
end

def run_single(processor)
 processor.process
end

def run(processors)
 processors.each{|p| p.process}
end

run_single(Processor.new 'Hello')
run([Processor.new 'Hello',Processor.new 'World'])

In above example, we need to define two methods run_single taking a processor and run taking array of processors. To make it simple, we can remove run_single method just call 
run([Processor.new 'Hello']) 

in case of single processor. However passing a single element array seems to be not natural, by using flatten or to_a we can create a method that accept both element and array of element as parameter 
def run(processor)
 [processor].flatten.each{|p| p.process }
end

or other possible way is
def run(processor)
 processor.to_a.each{|p| p.process }
end

then we can do
run(Processor.new 'Hello')
run([Processor.new 'Hello',Processor.new 'World'])



Ruby's DSL implementation pattern
2007-02-20T20:25:00.000+09:00
DSLs are designed to solve a particular kind of problem. Implementation DSLs as internal DSLs in Ruby is getting popular. Jim Freeze in his article Creating DSLs with Ruby use "Interpret" pattern to implement DSL. According the article a problem will be solved by using instance of a Class Machine. The Machine has class method load that create an instance of the Class Machine and load a file containing machine's instructions writen in DSL. The Machine also has instance method process, that run loaded instructions with specified data. A Ruby implementation can be as follow
class Machine
  def self.load(filename)
    machine = Machine.new
    machine.instance_eval(File.read(filename), 
      filename)
    machine
  end

  def process(*arg)
  #...
  end
end

Using this pattern, in The ruby implementation of an example in Martin Fowler's Language Workbenches article  the Reader class can be modified as follows
class Reader
    #...

    def self.load dsl_file_name
        reader = Reader.new
        reader.instance_eval(File.read(dsl_file_name), 
          dsl_file_name)
        reader
    end

    def process data_file_name
        File.open(data_file_name,'r') do |file|
            while line = file.gets
                process_line line
            end
        end
    end

    #...

    def mapping type_code, type, &block
        @strategies[type_code] =
            ReaderStrategy.new type_code, type, &block
    end

end

#dsl.txt
mapping('SVCL', ServiceCall) do
  extract 4..18, 'customer_name'
  extract 19..23, 'customer_ID'
  extract 24..27, 'call_type_code'
  extract 28..35, 'date_of_call_string'
end
mapping('USGE', Usage) do
  extract 9..22, 'customer_name'
  extract 4..8, 'customer_ID'
  extract 30..30, 'cycle'
  extract 31..36, 'read_date'
end

#load_dsl.rb
require 'test/unit'
require File.join(File.dirname(__FILE__),
    '..', 'lib', 'data')
require File.join(File.dirname(__FILE__),
    '..', 'lib', 'data_reader')
require File.join(File.dirname(__FILE__),
    '..', 'lib', 'data_reader_strategy')
class TestLoadDsl < Test::Unit::TestCase
    def test_read_file_load_dsl
        dsl_file_name =  File.join(File.dirname(__FILE__),
            'dsl.txt')
        reader = Reader.load dsl_file_name

        data_file_name = File.join(File.dirname(__FILE__),
            'sample.txt')
        reader.process data_file_name

        assert_equal 4, reader.result.size
    end
end



Ruby and DSL
2007-02-19T20:00:00.000+09:00
Googling on Internet, I found few interested sources of information of how to implements DSL on top of Ruby language.

Creating DSLs with Ruby

The Way of Meta

Writing Domain Specific Languages


ERP Application Service Providers
2007-02-18T12:46:00.000+09:00
During Tet Holiday, I have time to read the Friedman's Book "The World Is Flat: A Brief History of the Twenty-first Century " that is given to me by a friend. One chaper of the book mention about Salesforce, a company that provides CRM software as service via Internet at very low cost. (Software as a Service) SaaS is not new, the idea rooted from 60, 70 year of last century under different name such as service bureau, utilities computing, etc. I think, using ERP as service is very attractive for small, mid-size companies. Searching on Google, I found the following companies (ASP) offering ERP sofware as service
24SevenOffice
NetSuite
SalesForce
TwinField
Oracle CRM On Demand
RightNow Technologies
Fee of service is different from company to company ranging from 40 to 100 USD per user per month.


White Paper về ngành CNTT Việt Nam của GS TS  John Vũ
2007-02-17T17:13:00.000+09:00
Một người bạn gửi cho Tôi tài liệu nhan đề "Ngành CNTT Việt Nam phải làm gì để thành công trên thị trường thế giới?" của John Vũ. Do đây là chủ đề được nhiều người từ các nhà hoạch định chính sách, các nhà đầu tư, các nhà quản lý, cho đến các kỹ sư phần mềm quan tâm, Tôi đưa nguyên văn dưới đây để mọi người tham khảo. 


Hội Tin Học TP.HCM (HCA) nhận được nội dung Bạch Thư (White Paper) về ngành công nghiệp CNTT Việt Nam của GS. TS John Vũ - Viện sĩ Viện Hàn Lâm Software Engineering Institute (SEI) thuộc Carnegie Mellon University (CMU); Kỹ sư Trưởng Trung Tâm CNTT của Tập Đoàn Boeing. Qua nội dung của Bạch Thư, người đọc không chỉ tiếp thu thông điệp từ Boeing về những khuyến nghị mà ngành công nghiệp CNTT Việt Nam cần thực hiện mà còn cảm nhận được tình cảm của một trong những chuyên gia Việt Kiều CNTT hàng đầu thề giới gởi đến cộng đồng CNTT Việt Nam, để cùng suy nghĩ cho một ước mơ sử dụng CNTT đưa đất nước vượt thoát đói nghèo, nắm bắt được thời cơ lịch sử của CNTT Việt nam trong bối cảnh cạnh tranh toàn cầu. Hội Tin Học TP.HCM (HCA) tạm dịch nội dung của Bạch Thư và mong nhận được những ý kiến trao đổi của các Anh Chị chuyên gia và doanh nghiệp trong cộng đồng CNTT Việt Nam:

Ngành CNTT Việt Nam phải làm gì để thành công trên thị trường thế giới?

Bạch Thư

Người thực hiện: John Vu – Viện sĩ Viện Hàn Lâm - Carnegie Mellon Univeristy-CMU;  Kỹ sư trưởng TT CNTT tập đoàn Boeing 09/2006
Dựa trên những thông tin được cung cấp bởi Hội Tin học TP.HCM (HCA), trang web của rất nhiều công ty cũng như những trao đổi với các lãnh đạo doanh nghiệp CNTT  trong suốt chuyến đi Việt Nam, tôi đã thực hiện báo cáo này, đưa ra một cái nhìn tổng quát về ngành CNTT Việt Nam. Dựa trên những kinh nghiệm mà tôi đã trải nghiệm tại Ấn Độ, Trung Quốc, Nam Triều Tiên và Châu Âu, tôi đã đúc kết được một vài vấn đề then chốt nhằm giúp Việt Nam thành công trên lĩnh vực CNTT toàn cầu. Tuy nhiên, báo cáo này cũng chỉ là những quan sát và nhận xét chủ quan của riêng cá nhân tôi, còn nhiều thiếu sót do không đủ thời gian và nỗ lực để thu thập đầy đủ dữ liệu và các thông tin quan trọng. Xét về khía cạnh nào đó, tôi tin chắc rằng báo cáo này có thể giúp Việt Nam tạo ra những lối đi riêng trong việc tập trung vào một vài hoạt động trong năm đầu tiên; tuy nhiên, những hành động có tính chất toàn diện của các khuyến nghị là cần thiết để làm thay đổi ngành công nghiệp CNTT Việt Nam và cho phép tham gia kinh doanh toàn cầu. 

1) Quy mô là một vấn đề:

Hầu hết các công ty CNTT tại Việt Nam đều là công ty nhỏ (số nhân sự dưới 50 người). Đây chính là nguyên nhân gây ra những khó khăn, cản trở trong việc hội nhập với thế giới. Nếu nhân sự của công ty bạn đủ mạnh, các đối tác nước ngoài sẽ tin rằng họ đang hợp tác với một công ty đủ sức đương đầu với những rủi ro, luôn có đội ngũ nhân viên chuyên nghiệp, có thể thực hiện được những mong muốn của họ và dĩ nhiên, họ sẽ giữ mối quan hệ hợp tác lâu dài với công ty bạn. Ngoài việc liên doanh và sát nhập, một giải pháp khác là thành lập một liên kết chặt chẽ giữa một số công ty CNTT nhưng vẫn giữ tính cách độc lập của các thành viên (consortium), bao gồm những công ty nhỏ để tập trung nguồn nhân lực cùng thực hiện các dự án chung. Lãnh đạo của consortium (có thể là Khu Công nghệ (CN) Cao sẽ đóng vai trò là nhà đại diện để giao dịch với khách hàng. Consortium có một thực thể pháp l‎ý riêng và tiếp cận thị trường qua một thương hiệu riêng. Dĩ nhiên, các công ty CNTT  trong consortium phải có khả năng làm việc độc lập và hiệu quả để tối thiểu hoá những rủi ro cho khách hàng. Để không ngừng phát triển và mở rộng thị trường quốc tế, doanh nghiệp CNTT tại Việt Nam phải chuyển đổi hình thức “kinh doanh gia đình” sang hình thức hoạt động chuyên nghiệp và cần tập trung vào những thị phần chuyên biệt. Mặt khác, các doanh nghiệp cần cải tiến quy trình quản l‎ý, đặc biệt là việc lập các báo cáo tài chính và tín dụng. Không có một hệ thống quản l‎ý và tài chính tốt, doanh nghiệp Việt Nam rất khó mà có được chỗ đứng trong thị trường quốc tế. Hơn nữa, doanh nghiệp Việt Nam cũng cần nắm rõ những thị trường tiềm năng, nắm bắt thị hiếu đối tác và đầu tư vào lĩnh vực nghiên cứu và phát triển (R&D) để đủ sức cạnh tranh với các đối thủ. 

Khuyến nghị:

• Nghiên cứu kỹ lưỡng về khả năng tạo ra một consortium CNTT thông qua Khu CN  cao. Khu CN cao  đóng vai trò như một thực thể pháp lý độc lập trong việc thương thảo với đối tác.

• Để làm việc quốc tế, ta nên tạo ra thương hiệu chung giữa 20-30 công ty nhỏ hơn là các công ty cá thể.

• Đầu tư vào lĩnh vực R&D để nhận biết những sở trường chủ yếu của ngành công nghiệp CNTT và phát triển các cơ hội tương lai.

• Tập trung vào việc cải tiến cơ cấu quản lý, tài chính, kế toán và cơ sở hạ tầng để nâng tầm kinh doanh toàn cầu.

• Nghiên cứu xu hướng thị trường để xác định và triển khai những thị phần chuyên biệt trước khi phát triển theo diện rộng.

• Tăng cường hoạt động tiếp thị để xây dựng hình ảnh một Việt Nam thực sự  là một địa điểm kinh doanh.

• Đề nghị sự hỗ trợ chuyên môn từ Boeing để nhận diện môi trường kinh doanh CNTT hiện nay và nắm bắt những cơ hội trong tương lai.

2) Dịch vụ là một vấn đề:

Dựa trên “Báo cáo Toàn cảnh CNTT-TT Việt Nam” của Hội Tin học TP.HCM (HCA), ngành công nghiêp CNTT Việt nam dường như tập trung nhiều vào các hoạt động như lắp ráp máy tính, kinh doanh phần cứng và cung cấp các dịch vụ internet.  Đây là những lĩnh vực có tính không ổn định rất cao do sự cạnh tranh từ các doanh nghiệp được tổ chức tốt như Lenovo, IBM, Dell, HP and AOL, đặc biệt là khi Việt Nam gia nhập Tổ chức Thương mại Thế giới (WTO). Sẽ tốt hơn và có nhiều lợi ích hơn nếu ngành công nghiệp CNTT  Việt Nam tập trung vào ngành công nghiệp dịch vụ; như gia công phần mềm, gia công dịch vụ trong qui trình quản lý (Business Process Outsourcing - BPO) và gia công lưu trữ dữ liệu thông tin (Media Storage Outsourcing - MSO) là những lĩnh vực có thể sinh lợi rất nhiều cũng như tạo ra nhiều cơ hội việc làm hơn là ngành công nghiệp sản phẩm hiện nay. 

Ngành công nghiệp gia công phần mềm Việt Nam vẫn còn rất non trẻ với nhiều công ty chỉ tập trung vào một phần nhỏ trong chu trình phát triển phần mềm như nhập liệu, testing và mã hóa, trong khi có rất ít doanh nghiệp thể hiện năng lực phát triển ứng dụng, quản trị dự án, kiến trúc hệ thống, tích hợp hệ thống, quản trị cơ sở dữ liệu hoặc phân tích quy trình kinh doanh. Nguyên nhân cũng có thể là do hệ thống giáo dục hiện nay vẫn còn ảnh hưởng từ truyền thống Đông Âu – chuyên về đào tạo lập trình hơn là phân tích, thiết kế và phát triển các ứng dụng, các kỹ năng quản l‎ý vốn là thế mạnh của truyền thống đào tạo giảng dạy tại Mỹ. 

Khuyến nghị:

• Tiếp thu cách tiếp cận mới về phát triển phần mềm và tập trung vào toàn bộ chu trình phát triển sản phẩm thay vì chỉ triển khai một phần nhỏ trong chu trình.

• Chuyển ngành CNTT Việt Nam từ khuynh hướng nghiêng về sản phẩm sang khuynh hướng nghiêng về dịch vụ.

• Xây dựng kế hoạch quôc gia, hoặt ít nhất là kế hoạch vùng để tập trung vào công nghiệp gia công.

• Hợp tác với các trường đại học nhằm đào tạo nhiều hơn nữa nguồn nhân lực tập trung vào quy trình phần mềm tổng thể và một số kỹ năng chuyên biệt hiện đang thiếu hụt trên thế giới hiện nay.

• Đề nghị sự hỗ trợ về chuyên môn của Boeing nhằm nhận thức đầy đủ về công nghiệp gia công CNTT và nắm bắt được các cơ hội thị trường.

3)Năng lực là một vấn đề:

Có một sự cường điệu khi giới thiệu ưu thế của ngành CNTT Việt Nam là chi phí nhân công thấp trong việc thay thế Ấn Độ và Trung Quốc trong thị trường CNTT thế giới. Ngay cả khi có một giá nhân công thấp hơn, thì nếu chỉ có giá thấp cũng không tạo ra một yếu tố chính cho sự lựa chọn của đối tác. Nhận được đơn hàng mới chỉ là giai đoạn ban đầu trong khi tạo được sự hài lòng cho khách hàng mới là tất cả công việc. Nếu khách hàng nhận ra việc thiếu năng lực của mình thì rất khó giữ được sự hợp tác lâu dài. Hầu hết các doanh nghiệp trên thế giới đều rất cẩn trọng trong việc chọn lựa nhà cung ứng thông qua hệ thống tiêu chuẩn chọn lựa khắt khe có được từ những kinh nghiệm tại những thị trường đã được định hình như Ấn Độ.

Tôi thiết tha khuyến cáo các doanh nghiệp CNTT tại Việt nam nên xác định vị thế của mình dựa trên năng lực chuyên môn hơn là chỉ dựa vào giá thấp. Năng lực chỉ được công nhận dựa trên một hệ thống giáo dục và đào tạo tiên tiến; điều này cần rất nhiều thời gian và công sức. Sự phóng đại năng lực chỉ làm tăng tỉ lệ thất bại hoặc đưa đến những dự án không đạt yêu cầu trong mắt của đối tác. Điều này làm cho chúng ta đánh mất sự tín nhiệm và mất cả những cơ hội hợp tác tiếp theo.  Tốt hơn cả là nên bắt đầu bằng năng lực thực sự và xây dựng sự tín nhiệm nơi khách hàng, khi đã tạo nên mối quan hệ tốt và năng lực đã vững vàng thì việc kinh doanh sẽ phát triển nhanh chóng.

Khuyến nghị:

• Đặt kế hoạch phát triển kỹ năng nghiệp vụ hơn là cách đưa ra giá thấp.

• Hợp tác với các trường đại học để đào tạo các kỹ năng như bảo mật, khai thác dữ liệu, quản l‎ý dự án, kiến trúc phần mềm, tích hợp hệ thống v.v…

• Tập trung vào việc cải tiến kỹ năng và năng lực cũng như xác định những kẽ hở thị trường mà Việt Nam có thể chiếm lĩnh trong hiện tại và tương lai.

• Xây dựng mối quan hệ tốt với khách hàng qua việc thực hiện công việc hơn là giá thấp.

• Phân tích thị trường nhằm nhận diện các khả năng phát triển.

• Đề nghị sự hỗ trợ của Boeing trong việc cải tiến giáo dục và đào tạo. 

4) Vấn đề về trình độ quản lý vững vàng và việc đạt được chính thức chứng chỉ chất lượng quản lý :

Các khách hàng quốc tế trông đợi nhiều vào trình độ quản lý và chất lượng công việc từ các nhà gia công CNTT. Ấn Độ là quốc gia gia công chiếm lĩnh được vị trí hàng đầu, Việt nam cũng nên đạt đến vị trí của các công ty Ấn độ hoặc sẽ trở thành nhà cung cấp hạng hai hay hạng ba, chỉ nhận được các công việc có lợi nhuận thấp như nhập liệu hoặc thử nghiệm phần mềm.

Chứng chỉ quản lý chất lượng CMMI (Capability Maturity Model Integrated) được phát triển bởi Viện Công Nghệ Phần Mềm (Software Engineering Institute - SEI) của Carnegie Mellon University (CMU), được chấp nhận bởi khách hàng trên toàn thế giới như một tiêu chuẩn bảo đảm cho chất lượng phát triển phần mềm. Các công ty Việt Nam cần đạt được các trình độ cao của CMMI (cấp 4 và 5).

Đối với các khách hàng quốc tế, việc tìm kiếm các nhà cung cấp mới đem đến nhiều rủi ro, nhiều người trong số họ không thích có thêm những nhà cung cấp mới. Một chứng chỉ quản lý chất lượng được công nhận bởi một cơ quan chứng thực độc lập giúp họ giảm bớt những rủi ro. Có được chứng chỉ chính thức CMMI là điểm tạo sự khác biệt chính trong hoạt động gia công phần mềm. Để cạnh tranh toàn cầu, tất cả các doanh nghiệp CNTT Việt Nam cần phải đạt được chứng chỉ CMMI từ các nguồn được công nhận.

Hiện nay, có nhiều chứng chỉ chất lượng “giống CMMI” được cấp bởi những nhà tư vấn ở một số quốc gia nhưng không được tổ chức quốc tế chính thức công nhận. Việc nhận các chứng chỉ như vậy có thể gây ra sự nhầm lẫn đem đến sự tổn hại về uy tín của doanh nghiệp Việt Nam. Chứng chỉ CMMI của Mỹ đòi hỏi sự chứng thực của cơ quan đánh giá được SEI uỷ nhiệm hợp pháp. Danh sách các đơn vị được cấp chứng chỉ CMMI được SEI công bố để bảo đảm tính hợp pháp của qui trình chứng thực. Hầu hết khách hàng quốc tế đều tham khảo danh sách cấp chứng chỉ CMMI của SEI để thẩm định tính xác thực về chứng chỉ CMMI của những ứng viên cung cấp dịch vụ gia công cho họ.

Khuyến nghị:

• Các công ty CNTT Việt Nam nên nhanh chóng đạt được chứng chỉ CMMI từ nguồn chính thức.

• Dự trù ngân sách cho việc đạt chứng chỉ CMMI.

• Tạo sự đồng thuận trong tất cả nhân viên xem việc cam kết chất lượng là phương cách kinh doanh.

• Tập trung vào chất lượng và hiệu quả để tạo nên điểm khác biệt của doanh nghiệp.

• Quảng bá kết quả của việc nhận chứng chỉ CMMI như là một phần việc marketing.

• Đề nghị sự hỗ trợ của Boeing trong việc đào tạo và cấp chứng chỉ CMMI.

5. Vấn đề thử thách mới, kỹ năng mới:
Là một thử thách rất lớn cho Việt Nam khi đối mặt với những đối thủ hàng đầu như Trung Quốc, Ấn Độ. Tuy vậy, với việc đào tạo đúng và tiếp cận tốt, CNTT Việt Nam có thể cạnh tranh trực tiếp với những đối thủ lớn này trong thị trường toàn cầu. Việt Nam có những ưu và nhược điểm riêng; nên tập trung vào những gì mình làm được hơn là theo những việc mà Trung Quốc và Ấn Độ đang làm. Để có thể chiến thắng nên thay đổi quan điểm như chi phí thấp và khả năng cao.

Hiện nay cả Trung Quốc và Ấn Đô đều bị ảnh hưởng của việc thiếu hụt trầm trọng nguồn nhân lực kỹ năng cao và họ cố gắng giải quyết vấn đề bằng cách đào tạo ồ ạt số lượng lớn người làm CNTT. Điều này dẫn đến việc giảm chất lượng của nguồn nhân lực. Đây là cơ hội tốt  cho Việt Nam để cạnh tranh bằng cách cung cấp nguồn nhân lực kỹ năng cao qua những chương trình giáo dục và dạy nghề nghiêm chỉnh.

Ngành công nghiệp CNTT thay đổi nhanh chóng, việc hiểu rõ tiến trình và xu hướng phát triển sẽ giúp Việt Nam điều chỉnh hướng đi và tái họach định để bảo đảm kẽ hở thị trường hơn là đi theo những việc người khác đã làm. Xu hướng hiện nay xem trọng giá trị công việc và mối quan hệ hơn là nguồn nhân lực giá rẻ. Hầu hết các phần mềm đang trở thành hệ thống hội tụ mạng lưới, ngày càng phức tạp; đòi hỏi những kỹ năng thành thạo về cấu trúc hệ thống, thiết kế, tích hợp hơn là lập trình hay testing. Khuynh hướng này không có nghĩa là sẽ không còn việc cung ứng nhân lực giá rẻ nhưng ngành công nghiệp ngày càng mở ra nhiều cơ hội cho những quốc gia nào có thể cung cấp được những kỹ năng mới cho khách hàng. 

Khuyến nghị:

• Tiến hành nghiên cứu thị trường để nhận diện những kỹ năng, những khả năng mới - đặc biệt tránh những kỹ năng thông thường như lập trình, testing mà nước nào cũng có …

• Thu thập và phát triển những chương trình đào tạo các kỹ năng chuyên sâu - đặc biệt trong quản lý dự án, phân tích yêu cầu kinh doanh, kiến trúc hệ thống, tích hợp hệ thống, quản trị toàn bộ, thiết kế, xây dựng hệ thống…

• Đào tạo những kỹ năng mới cho nhân viên.

• Đề nghị Boeing hỗ trợ các chương trình đào tạo kỹ năng mới từ các trường đại học Mỹ.

6. Vấn đề truyền thông:

Kinh doanh toàn cầu cần kỹ năng giao tiếp trong nhiều ngôn ngữ nhưng tiếng Anh vẫn quan trọng nhất. Doanh nghiệp CNTT Việt Nam cần đào tạo nhân viên về giao tiếp tiếng Anh và hiểu về văn hóa của khách hàng đối với từng nước.

Để giải quyết vấn đề này, cần đến sự trợ giúp của ngoại kiều hoặc của Việt kiều làm người đại diện. Trong môi trường kinh doanh toàn cầu, nên có bộ phận quản lý tại chỗ gồm những người am hiểu văn hóa và tập quán kinh doanh bản địa. Ước tính có khoảng một triệu người Việt Nam tại nước ngoài có khả năng như vậy. Cộng đồng này hầu hết có thể nói hai ngôn ngữ và thường hiểu văn hóa thị trường nơi mà họ lớn lên. 

Khuyến nghị:

• Thu thập những chương trình dạy Anh văn cho nhân viên CNTT.

• Thuê nhân viên địa phương hoặc Việt kiều làm đại diện ở mỗi thị trường.

• Tập trung vào việc hiểu rõ văn hóa kinh doanh của khách hàng.

• Thu thập các chương trình đào tạo Anh ngữ cho các khu công nghiệp kỹ thuật cao và các trường đại học.

• Đề nghị Boeing hỗ trợ việc hợp tác với các Học viện và trường đại học Mỹ. 

7) Vấn đề về đối tác và liên minh:

Tất cả những công ty CNTT thành công đều có đối tác tại những lĩnh vực nằm ngoài khả năng của họ. Các công ty CNTT Việt Nam cần liên kết với những đối tác khác và tạo ra những liên minh hoạt động trong những lĩnh vực nhất đinh nhằm xác định thị trường theo ngành dọc. Những doanh nghiệp Việt Nam nên giao cho các đơn vị khác cung ứng cho mình những dịch vụ bên ngoài dịch vụ chính và hình thành các mối quan hệ tin cậy với khách hàng. Khu công nghiệp cao có thể là nơi được cân nhắc trước tiên để tạo lập mối quan hệ đối tác hoặc liên minh lẫn nhau. Một cách cơ bản là một thực thể liên kết nhiều công ty sẽ tạo nên một sức mạnh lớn trong thị truờng cạnh tranh gay gắt hiện nay. 

Khuyến nghị:

• Thăm dò khả năng thành lập mối quan hệ đối tác và liên minh.

• Hình thành thực thể liên kết consortium lớn để quản lý toàn bộ chu trình hoạt động.

• Nghiên cứu các chủ trương chính sách nhà nước các cấp để xúc tiến thành lập consortium.

• Xác định, kết nối và đo lường những giá trị gia tăng của từng liên minh.

• Hợp thức hoá việc uỷ quyền cho Khu công nghiệp cao như một thực thể đại diện và thương hiệu duy nhất.

• Đề nghị Boeing hỗ trợ xây dựng thương hiệu cho Khu Công nghệ cao.

• Đề nghị Boeing hỗ trợ việc đào tạo cho các nhà quản lý Khu Công nghệ cao từ các trường Đại học và các trung tâm đào tạo ở Mỹ.

8) Vấn đề tiếp thị và quảng cáo:
Cho dù là ngành công nghiệp CNTT Việt nam phát triển rất nhanh trong vài năm qua, nhưng hầu như tình hình này chưa được biết đến trên bình diện kinh doanh toàn cầu. Nếu khách hàng không biết gì về Việt Nam thì tại sao họ phải làm kinh doanh tại đây?
Điều quan trọng bây giờ là Việt Nam cần nỗ lực sử dụng các chiến dịch kinh doanh để chào hàng cho được “CNTT Việt Nam”. Rất khó cạnh tranh với các đối thủ được tổ chức tốt như Ấn Độ, Trung Quốc . Nếu chỉ có giá thấp thì sẽ nhận được rất ít đề nghị có giá trị. Hiện nay hầu hết các doanh nghiệp phương tây giao gia công không coi vấn đề chi phí là chuyện chính yếu mà họ đang tìm kiếm những nguồn cung ứng các dịch vụ có giá trị như hệ chuyên gia và tiếp cận thị trường địa phương. 

Khuyến nghị:
Việt Nam cần xác định vị trí của ngành công nghiệp CNTT vào:

* chất lượng và hiệu quả (được chứng nhận bởi tiêu chuẩn CMMI).

* cơ sở hạ tầng ổn định để có thể chuyển giao các dịch vụ cho khách hàng nhanh hơn, tốt hơn và đáng tin cậy hơn.

* xây dựng các kỹ năng chuyên biệt.

* đầu tư phù hợp cho hạ tầng để có thể kinh doanh tốt.

* thân cận với các thị trường Châu Á Thái Bình Dương.

* bảo đảm sự ổn định chính trị và an ninh xã hội.

* thu hút doanh nhân và khách du lịch đến Việt nam.

Kết luận:

Những thông tin trong bạch thư này được tôi thu nhận trong chuyến công tác vừa qua ở Việt Nam, có tính chất chủ quan hơn là dựa trên số liệu nghiên cứu khách quan. Tôi tin chắc rằng ngành công nghiệp CNTT Việt Nam cần nghiên cứu rộng hơn và hành động. Những khuyến nghị của tôi giới hạn trong phạm vi quan sát cá nhân với nhận thức đây chỉ là những hỗ trợ ban đầu của Boeing giúp Việt Nam cải thiện hệ thống giáo dục để xây dựng ngành khoa học không gian và ngành CNTT - TT mạnh mẽ; được nhận biết trên thị trường quốc tế và có khả năng cạnh tranh toàn cầu.

(Nguồn: HCA)


Chúc mừng năm mới Đinh Hợi
2007-02-15T20:26:00.000+09:00

Bức tranh Hoa Đào (Peach Blossom) này là của họa sĩ đương đại Trung Quốc DU YINGQIANG.