Bringing more development practices to the world of administration

In tradition software development and system administration tends to be separated. So useful practices in software development are not known by many administrators. I would say that the world of system administrations lack very much behind software development's world.
While using Version Control System (VCS) is obvious thing in any software project, not many administrator use this very useful tool during their daily work. The result is a dirty environment where a configuration's file is backup manually by renaming before making change. Also there is difficult to perform a rollback and reason and time of change is not well maintained.
Other often well encouraging software principles are to achieve re-usability, readability, explicitness as well as to avoid duplication. On the other hand an administrator cares much less about them. I have seen an environment where there are hundreds of scripts poorly written, repetitive, difficult to understand and maintained. I do thing that by using technique as re-factoring the situation can be improved significally.

Articles about deployment and operation are rare

Deploy and run a software is a vital part of the full picture "make value from the software". However does not draw much attentions from community.I listed here links to few blogs and websites talking about this subject matter.
http://www.build-doctor.com
http://www.agileweboperations.com/
http://en.oreilly.com/velocity2009
http://reductivelabs.com/trac/puppet/wiki/FrequentlyAskedQuestions
http://ganglia.info/
http://www.cfengine.com/
http://trac.mcs.anl.gov/projects/bcfg2
http://automateit.org/
http://wiki.opscode.com/display/chef/Home

Joining to ING Direct Spain

ING Group decided not to launch ING Direct Operation in Japan, as result I moved to ING DIRECT Spain. As for non European citizen, the visa process take long time for me. But finally I landed to Spain Office at the end of July.
This is logical step in my career, I want to stay with ING DIRECT and the job market has still very tough. I returned to Europe after 12 years, I moved from Prague to Hanoi in July, 1997.
The first two weeks in the Spain Office is exciting, people are friendly, HR is supportive, office is nice, lunch is free, job is interesting. But like you got marry and went to honeymoon, this period does not last forever. Back to reality, I have to be pragmatic, find the way to integrate to the team and the community, be helpful and continue the journey in my career path.
Weather is Spain in this season is good especially in the morning, drop me a call or a message whenever you go Madrid, I would love to meet you for a drink, tapas.

My Resume - SUMMARY of SKILLS

---

System administration, support and maintenance

• Responsible for large scale WebSphere Application Server ND and WebSphere MQ infrastructure used by iFlex (Oracle Flexcube) retail banking system for a leading consumer bank.
• Led a team, that provide remote and onsite support 24/7 distributed critical J2EE application based on WebLogic and Oracle Database for immigration control.
• Directly involved in identification and analysis critical problems, was interface point with vendors, customers, and management in incident handling, problems solving process
• Well versed in using scripting language (RUBY, PYTHON) to automate all kinds of administrative activities including software installation, update; configuration management; application deployment, update; testing; monitoring; and log file management

Java and J2EE Application Server

• More than 5 years experience with JAVA platform and J2EE framework particularly Websphere, WebLogic. Directly involved in capacity planning, application packaging, deployment, monitoring, troubleshooting that includes sizing memory, thread pools, setting transaction's timeout, maintaining resource pool, etc..
• Well understood internal JVM and its assembler language (byte code). Hands-on experience in tools and technique for identifying JVM and J2EE problem such as analyzing heap dump, core dump, garbage collector, decompile byte code, injection tracing code into application and container code.

Messaging System

• More then 7 years in design and implementation of software systems that communicate with each other using asynchronous messages. Understand message creation and processing patterns (Enterprise Integration Patterns) and their applications in real world systems.
• Responsible for deployment IBM MQ as message backbone for many applications. The responsibility includes interfacing with development team for planning and configuration, testing, trial run, installation, monitoring and troubleshooting

Database System

• More then 10 years extensive working experience on RDBMS Oracle in both administrator and developer role. Obtained certification on Oracle DBA in 1999, hand-on experiences in large site capacity planning, performance tuning, high availability using Real Application Cluster (RAC) and Data Guard.
• Involved in database design for many enterprise projects, assisted implementation team in evolving database schema, data migration and cleansing.

Operating System and Hardware

• More than 10 years experience in design, installation, administration and troubleshooting complex IT infrastructure including UNIX clustering system, IBM FC and iSCSI SAN, CISCO router and PIX firewall
• Hand-on experience in AIX administration including HACMP. Obtained certification on AIX administration in 1999. Performed installation, configuration, tuning many AIX based systems(s), making them ready for deployment Oracle Database and J2EE application server.
• Involved in technical pre-sale activities including sizing a system, proposal preparation, technical presentation and alike.

Software Development

• More than 15 years experience in developing software products with deep knowledge of architecture / design of multi-tiered client-server systems based on middleware including Tuxedo, J2EE (WebSphere, WebLogic) and other emerging Web framework as Ruby on Rails.
• Skillful in many programming languages including C/C++, JAVA, SQL, PL/SQL, RUBY. Passionate for the simple and easy to understand software with rich domain model. Well versed in OOP, design patterns (Design Patterns - GoF, Pattern for Enterprise Application Architecture - PEAA, Domain Driven Design).
• Strong knowledge of algorithms and data structures and their applications in writing a high performance, time critical system for complex business domain.
• Extensive experience with state of art software development practices such as pair programming, Test Driven Development (TDD), refactoring, continuous integration.

Business Analysis and Leadership

• Able to attract, retain and lead people. Led team up to 25 developers, responsible for task planning, technical direction, risk management, code inspections, quality control, performance reviews, staff development. Understood how to create working environment to keep people happy, motivated, creative and productive.
• Solid presentation skill, proven in communication with customers, understanding problems, capturing domain knowledge and turning it into high level requirements in suitable form for development team and other stakeholders

Context root of single war's web applicaton in JBoss

Context root of web application is usually specifiedin META-INF/application.xml in ear file (or a vendor specific jboss-web.xml inside war ) e.g.

...
  <module>
    <web>
      <web-uri>app.war</web-uri>
      <context-root>/jsfejb3<context-root>
    </web>
  </module>
..

However if we drop a single war file without vendor specific descriptor (jboss-web.xml) into JBoss deploy directory then context root is implicitly the name of the war file without extension.
e.g.
Web application jmx-console residing in default/deploy/jmx-console.war has context root jmx-console
Web application groovy-scripts residing in default/deploy/groovy-script.war has context root groovy-script and can be accessible via http://localhost:7080/groovy-script

Centralized Management Tools

Nagios
Big Brother
Hyperic HQ
JOPR
@ctiveFriend
OEM
OpenNMS
There is a great presentation of monitoring tools here

Testing JOPR 2.2.0 - Basic configuration

Modify JBoss server startup options to include JVM as child resource of JBoss Server in JOPR

#$JBOSS_HOME/bin/run.conf
JAVA_OPTS="$JAVA_OPTS -Djboss.platform.mbeanserver"

Remove an uninterested resource from management a.k.a uninventory
Browse Resources->Servers, select uninterested one, hit uninventory button

Testing JOPR 2.2.0 - Installation

What is JOPR, RHQ, JBoss ON
JOPR is an open source project from Redhat to develop complex management platform. JOPR is agent based JAVA/J2EE system, where agent is standalone JAVA running and server runs on top of JBOSS Application Server(AS). It can be used to manage JBOSS AS, Tomcat, Apache as well as Oracle, and any IT resources.

JBoss Operations Network (JBoss ON) is commercial offer of JOPR in form of support service by Redhat.

JOPR is developed on top of RHQ project. RHQ serves as an maven upstream project for JOPR. RHQ consists of GUI management console, agent container and a set of agent plugins for management of common IT resources.
JOPR extends RHQ by adding additional plugins for JBoss App Server, Tomcat Web Server, etc. . JOPR version 2.2.0 is corresponding with RHQ version 1.2.0.

The original of RHQ is Hyperic HQ. Redhat and Hyperic work together to extract common reusable parts of Hyperic HQ and maintain it as RHQ. The aim is to create a common infrastructure for developing management's application. That is why the project named RHQ, where R stand for Redhat.

Install JOPR Server
Download JOPR binary from JOPR website
Follow instruction in Installation Guide to prepare database server for JOPR server
Change working directory to jopr-server-2.2.0 and type

bin/rhq-server.sh start

then open web browser and point to http://localhost:7080
Follow instructions in screen to set database driver, url, username/password to correct values. After that verify database connection.
Hit button Install JOPR server to complete installation process

Modify JOPR server properties file for agent communication
Modify rhq-server.properties file

# jopr-server-2.2.0/bin/rhq-server.properties
rhq.communications.connector.transport=servlet
rhq.communications.connector.bind-address=<jopr-server-ip-address>
rhq.communications.connector.bind-port=7080

Restart JOPR server

bin/rhq-server.sh stop
bin/rhq-server.sh start

Login to JOPR console and watch its log file
Pointing Web browser to http://localhost:7080/Dashboard.do
Login use default rhqadmin/rhqadmin as initial username/password
JOPR log file is located in jopr-server-2.2.0/logs/rhq-server-log4j.log

Install JOPR Agent
This shall be done in all machines that we want JOPR to manage .
Download agent from http://<jopr-server-hostname>:7080/agentupdate/download
Install JOPR Agent by typing

java -jar rhq-enterprise-agent-1.2.0.jar --install

Modify JOPR Agent configuration file

#rhq-agent/conf/agent-configuration.xml

# ask jopr agent not to ask for configuration during startup
<entry key="rhq.agent.configuration-setup-flag" value="true" />

<entry key="rhq.agent.name" value="<jopr-agent-hostname>"/>

<entry key="rhq.agent.server.bind-address" value="<jopr-server-ip-address>" />

# ask jopr agent to listen to on particular ip
<entry key="rhq.communications.connector.bind-address" value="<jopr-agent-ip-address>" />

Start RHQ Agent and watch its log file
change working directory to rhq-agent and run

bin/rhq-agent.sh -c conf/agent-configuration.xml 

If we reinstall JOPR server then agent's local data and server'data is not in sync. In that case we shall ask agent to clean it's local data when start by using

bin/rhq-agent.sh -l -c conf/agent-configuration.xml 

The most recent agent's log file is located in rhq-agent/logs/agent.log

Develop an international career - Recruiters

Recruiters (a.k.a Head Hunters) play a vital role in the job market. Many big companies use services of recruiters when they need to hire someone (they do not recruit directly). I myself and a lot of my colleagues got a job by a recruiter.
Below is a list of the most well known recruiters
- Robert Half
- Robert Walters
- Hays
- Wall Street
- Hudson
There are many smaller but also very good recruiters, who are constantly looking for a good candidate in all countries over the world. If you are somehow visible and your skill fit a opening position, they will contact you.
I got my first job via placement of a smaller recruiter named Juno located in New York.

JBoss resources

JBOSS configuration and tunning articles from Master the Boss
Running Multi JBoss Instances
Good examples of EJB 2.1, EJB 3 from Learn Technology
JBoss Official Docs & Examples from Redhat
JBoss Operation Network (ON), a.k.a JOPR - Documents
JBoss ON - FAQ
RHQ project, upon it JOPR depends
JOPR project
A blog on RHQ, JOPR, JBoss ON

Develop an international career - Make yourself visible

No matter how good and skilled you are,it is useless if no one know about it. So make yourself visible, post your resume to professional websites and social networks, create your own blog, subscribe to interested user groups, participate in an online discussion.
Keep in mind that all these techniques only work if you keep your data up to date. When a recruiter doing executive search and find your resume within dozens of others,he/she will easily exclude you if your resume is 2 years old.

Few years back I created my blog. It has only professional content and I decided to keep it in English because my intention is not to have many readers but the right one. Those are recruiters, potential employers and colleagues. I believe, that it helps me to get my first and also second job. Looking at site analytic of the blog, I don't see much traffic but I can identify those coming from area where I received a job offer or where my recruiter is located.

Linked In is great social networking site for professionals, so I would recommend to register and put your career information there.

Again, don't expect anything happened over night, it will take time.

Develop an international career - Resume

This post is created (based on my own experiences) for people who want to work out of their home countries and therefore develop their career internationally. It can be used as an advice.
You have generally only one career, so don't let it goes based on some unexpected events. If you really want to work abroad in developed countries, be prepared that it will take time to get a job. It can be two years or more depending on many factors. During this time, you may need take many actions to move into that direction.

Resume (a.k.a CV) is your ticket for entering job market. Write your resume with care and update it gradually and of course it should be in English. During last few years, I spend many hours to re-write many time my resume as well as to ask friends, colleagues and career's adviser for my resume's correction. I have to admit that I am still not satisfied with my resume as there is a lot of things to improve.

I also spend time to read resume of others, who work in the same industries and/or share a similar knowledge and career path. It help me a lot to get know how others present themselves, so I can adjust my resume to highlight my strength.

Java Class Loader's links

Impala blog post - Thread Context Class Loader
Java World - Class Loader article
On Java - Internals of Java Class Loading
Advanced JBoss Class Loading
Equinox OSGi Context Class Loader Enhancements

My resume - LANGUAGE

---

• Vietnamese, English, Czech

My resume - EDUCATION

---

Charles University Prague, Czech Republic

• MS in Computer Science (data engineering specialization) - 1997
• BS in Computer Science - 1994

Using CINT - C/C++ Interpreter - An experiment with vector

Let do something with vector

cint.exe> L vector
cint.exe> {vector<int> temps;}
cint.exe> {temps.push_back(6);}
cint.exe> {temps.push_back(4);}
cint.exe> {temps.push_back(5);}
cint.exe> {temps.size();}
(const unsigned int)3
cint.exe> {temps[0];}
(int)6
cint.exe> {temps[1];}
(int)4
cint.exe> {temps[2];}
(int)5
cint.exe>

So it works well, now there is time to try some algorithm

cint.exe> L algorithm
cint.exe> {sort(temps.begin(),temps.end());}
Error: Template Function value_type(first) is not defined in current scope  algo.h(735)
!!!Dictionary position rewound... !!!Error recovered!!!
cint.exe>

It seems that there is some problem with STL lib coming with CINT in Windows. Looking at STL header files, I found that Function value_type is declared in _iterator.h surrounded by a condition for whatever reason

#if (G__GNUC>=3)

#if (G__GNUC_VER>=3001) 
...
template  
inline T* value_type(const vector::iterator&) {return (T*)(0);}
..
#endif
#endif

To fix it is fairly simple

d:\cint-5.16.19\cint.exe
cint : C/C++ interpreter  (mailing list 'cint@root.cern.ch')
   Copyright(c) : 1995~2005 Masaharu Goto (gotom@hanno.jp)
   revision     : 5.16.19, March 16, 2007 by M.Goto

No main() function found in given source file. Interactive interface started.
'h':help, 'q':quit, '{statements;}' or 'p [expr]' to evaluate

cint.exe> {#define G__GNUC 4}
cint.exe> {#define G__GNUC_VER 4001}
cint.exe> L vector
cint.exe> L algorithm
cint.exe> {vector<int> temps;}
cint.exe> {temps.push_back(6);}
cint.exe> {temps.push_back(4);}
cint.exe> {temps.push_back(5);}
cint.exe> {temps.size();}
(const unsigned int)3
cint.exe> {temps[0];}
(int)6
cint.exe> {temps[1];}
(int)4
cint.exe> {temps[2];}
(int)5
cint.exe> {sort(temps.begin(),temps.end());}
cint.exe> {temps[0];}
(int)4
cint.exe> {temps[1];}
(int)5
cint.exe> {temps[2];}
(int)6
cint.exe>

Using CINT - C/C++ Interpreter - Basic Commands

Load,Unload,files
L that load files into interpreter, and file that list these loaded files.

cint.exe> L iostream
cint.exe> files
  0 fp=0x781c1bd0 lines=13   file="iostream"
  1 fp=0x781c1bf0 lines=479  file="iostream.h"
  2 fp=0x781c1c10 lines=11  *file="iosenum.h"
  3 fp=0x781c1c30 lines=34   file="bool.h"
  4 fp=0x781c1c50 lines=173 *file="_iostream"

If we use #include in curly bracket, we will get the same result

cint.exe> {#include <iostream>}
cint.exe> files
  0 fp=0x781c1bf0 lines=13   file="iostream"
  1 fp=0x781c1c10 lines=479  file="iostream.h"
  2 fp=0x781c1c30 lines=11  *file="iosenum.h"
  3 fp=0x781c1c50 lines=34   file="bool.h"
  4 fp=0x781c1c70 lines=173 *file="_iostream"

U command unload a file, a C/C++ header file can include other header files, which in turn can include additional header files and so all. CINT is good enough and maintain this dependencies so if we unload one file, others referenced files may get unloaded also.

cint.exe> L string
cint.exe> files
  0 fp=0x781c1bd0 lines=11   file="string"
  1 fp=0x781c1bf0 lines=72   file="_string"
  2 fp=0x       0 lines=0    file="string.dll"
  3 fp=0x781c1c10 lines=13   file="iostream"
  4 fp=0x781c1c30 lines=479  file="iostream.h"
  5 fp=0x781c1c50 lines=11  *file="iosenum.h"
  6 fp=0x781c1c70 lines=34   file="bool.h"
  7 fp=0x781c1c90 lines=173 *file="_iostream"
cint.exe> U string
cint.exe> files

reset
reset reset interpreter environment and unload all files

cint.exe> {int a=1;}
cint.exe> {a;}
(int)1
cint.exe> reset
cint.exe> {a;}
Error: Symbol a is not defined in current scope  (tmpfile)(1)
!!!Dictionary position rewound... !!!Error recovered!!!
cint.exe> L vector
cint.exe> files
  0 fp=0x781c1bd0 lines=11   file="vector"
  1 fp=0x781c1bf0 lines=18   file="_vector"
  2 fp=0x       0 lines=0    file="vector.dll"
  3 fp=0x       0 lines=0    file="vectorbool.dll"
  4 fp=0x781c1c10 lines=313  file="_vector.h"
  5 fp=0x781c1c30 lines=282  file="function.h"
  6 fp=0x781c1c50 lines=34   file="bool.h"
  7 fp=0x781c1c70 lines=235  file="algobase.h"
  8 fp=0x781c1c90 lines=55   file="_pair.h"
  9 fp=0x781c1cb0 lines=17   file="_iterator"
 10 fp=0x781c1cd0 lines=668  file="_iterator.h"
 11 fp=0x781c1cf0 lines=6    file="stddef.h"
 12 fp=0x781c1d10 lines=479  file="iostream.h"
 13 fp=0x781c1d30 lines=11  *file="iosenum.h"
 14 fp=0x781c1d50 lines=173 *file="_iostream"
 15 fp=0x781c1d70 lines=177  file="defalloc.h"
 16 fp=0x781c1d90 lines=34   file="new.h"
 17 fp=0x781c1db0 lines=34   file="stdio.h"
 18 fp=0x       0 lines=0    file="stdfunc.dll"
 19 fp=0x781c1dd0 lines=14   file="stdlib.h"
 20 fp=0x  476800 lines=19   file="limits.h"

-- Press return for more -- (input [number] of lines, Cont,Step,More)
cint.exe> reset
cint.exe> files
cint.exe>

Using CINT - C/C++ Interpreter - Getting Started

One of things I favor scripting languages (Ruby, Python) over none scripting (JAVA, C/C++) is there is no need to follow time consuming cycle of write-compile-run. Even with the best IDE, I still feels the pain of typing command, hit key combination, waiting for computer doing. This is especially true when I need to explore some API, test some ideas. So when I discovered CINT, a C/C++ Interpreter, I gave it a try. With CINT, we can type C/C++ code directly in it's interactive CINT console and get the result. CINT is available for Windows and many UNIX platform(s) some LINUX clone has CINT pre-installed. We start CINT console by type e.g. (on Windows)

d:\cint-5.16.19\cint.exe
cint.exe>{#include <iostream>
cint.exe>{cout << "Hello World\n";}  
Hello World (class ostream)2085223112 

C/C++ code shall be type inside curly bracket {...}, CINT doesn't allow to define function in interactive console

> { int func999() {return 999;} }
Limitation: Function can not be defined in a command line or a tempfile
You need to write it in a source file (tmpfile)(1)
!!!Dictionary position rewound... !!!Error recovered!!!

So we have to create separate file e.g.

# func999.cpp
int func999() {
return 999;
}

,load it in the console and call a function inside

cint.exe>L func999.cpp
cint.exe>{func999();}

Some of the most frequently used commands includes

L  - load file in to the console
U  - unload the file
file - show which files are currently loaded
reset - reset memory of CINT console to initial state,unload all files
q - quit CINT console

Retrospective 2008

Year 2008 was marked by an important event of my life and my career. I have joined ING Direct Japan (INGDJ) and moved to Tokyo in September. So far I feel OK. Within last 4 months, I have done a couple of things. Technically
1. Learned how to manage WebSphere Application Server and MQ infrastructure in a complex IT environment for banking business.
2. Write a bundle of ruby scripts to perform administrative activities related to WebSphere including system monitoring, log management and application deployment.
3. Learned JVM internal stuff like GC, heap dump, core dump analysis tools, JAVA decompiler and JVM assembler language
4. Use VIM (as primary editor) and GIT
5. Learned Python and it's Java implementation Jython primarily and use it for WebSphere Application Server scripting