Bringing more development practices to the world of administration

In tradition software development and system administration tends to be separated. So useful practices in software development are not known by many administrators. I would say that the world of system administrations lack very much behind software development's world.
While using Version Control System (VCS) is obvious thing in any software project, not many administrator use this very useful tool during their daily work. The result is a dirty environment where a configuration's file is backup manually by renaming before making change. Also there is difficult to perform a rollback and reason and time of change is not well maintained.
Other often well encouraging software principles are to achieve re-usability, readability, explicitness as well as to avoid duplication. On the other hand an administrator cares much less about them. I have seen an environment where there are hundreds of scripts poorly written, repetitive, difficult to understand and maintained. I do thing that by using technique as re-factoring the situation can be improved significally.

Articles about deployment and operation are rare

Deploy and run a software is a vital part of the full picture "make value from the software". However does not draw much attentions from community.I listed here links to few blogs and websites talking about this subject matter.
http://www.build-doctor.com
http://www.agileweboperations.com/
http://en.oreilly.com/velocity2009
http://reductivelabs.com/trac/puppet/wiki/FrequentlyAskedQuestions
http://ganglia.info/
http://www.cfengine.com/
http://trac.mcs.anl.gov/projects/bcfg2
http://automateit.org/
http://wiki.opscode.com/display/chef/Home

Joining to ING Direct Spain

ING Group decided not to launch ING Direct Operation in Japan, as result I moved to ING DIRECT Spain. As for non European citizen, the visa process take long time for me. But finally I landed to Spain Office at the end of July.
This is logical step in my career, I want to stay with ING DIRECT and the job market has still very tough. I returned to Europe after 12 years, I moved from Prague to Hanoi in July, 1997.
The first two weeks in the Spain Office is exciting, people are friendly, HR is supportive, office is nice, lunch is free, job is interesting. But like you got marry and went to honeymoon, this period does not last forever. Back to reality, I have to be pragmatic, find the way to integrate to the team and the community, be helpful and continue the journey in my career path.
Weather is Spain in this season is good especially in the morning, drop me a call or a message whenever you go Madrid, I would love to meet you for a drink, tapas.

My Resume - SUMMARY of SKILLS

---

System administration, support and maintenance

• Responsible for large scale WebSphere Application Server ND and WebSphere MQ infrastructure used by iFlex (Oracle Flexcube) retail banking system for a leading consumer bank.
• Led a team, that provide remote and onsite support 24/7 distributed critical J2EE application based on WebLogic and Oracle Database for immigration control.
• Directly involved in identification and analysis critical problems, was interface point with vendors, customers, and management in incident handling, problems solving process
• Well versed in using scripting language (RUBY, PYTHON) to automate all kinds of administrative activities including software installation, update; configuration management; application deployment, update; testing; monitoring; and log file management

Java and J2EE Application Server

• More than 5 years experience with JAVA platform and J2EE framework particularly Websphere, WebLogic. Directly involved in capacity planning, application packaging, deployment, monitoring, troubleshooting that includes sizing memory, thread pools, setting transaction's timeout, maintaining resource pool, etc..
• Well understood internal JVM and its assembler language (byte code). Hands-on experience in tools and technique for identifying JVM and J2EE problem such as analyzing heap dump, core dump, garbage collector, decompile byte code, injection tracing code into application and container code.

Messaging System

• More then 7 years in design and implementation of software systems that communicate with each other using asynchronous messages. Understand message creation and processing patterns (Enterprise Integration Patterns) and their applications in real world systems.
• Responsible for deployment IBM MQ as message backbone for many applications. The responsibility includes interfacing with development team for planning and configuration, testing, trial run, installation, monitoring and troubleshooting

Database System

• More then 10 years extensive working experience on RDBMS Oracle in both administrator and developer role. Obtained certification on Oracle DBA in 1999, hand-on experiences in large site capacity planning, performance tuning, high availability using Real Application Cluster (RAC) and Data Guard.
• Involved in database design for many enterprise projects, assisted implementation team in evolving database schema, data migration and cleansing.

Operating System and Hardware

• More than 10 years experience in design, installation, administration and troubleshooting complex IT infrastructure including UNIX clustering system, IBM FC and iSCSI SAN, CISCO router and PIX firewall
• Hand-on experience in AIX administration including HACMP. Obtained certification on AIX administration in 1999. Performed installation, configuration, tuning many AIX based systems(s), making them ready for deployment Oracle Database and J2EE application server.
• Involved in technical pre-sale activities including sizing a system, proposal preparation, technical presentation and alike.

Software Development

• More than 15 years experience in developing software products with deep knowledge of architecture / design of multi-tiered client-server systems based on middleware including Tuxedo, J2EE (WebSphere, WebLogic) and other emerging Web framework as Ruby on Rails.
• Skillful in many programming languages including C/C++, JAVA, SQL, PL/SQL, RUBY. Passionate for the simple and easy to understand software with rich domain model. Well versed in OOP, design patterns (Design Patterns - GoF, Pattern for Enterprise Application Architecture - PEAA, Domain Driven Design).
• Strong knowledge of algorithms and data structures and their applications in writing a high performance, time critical system for complex business domain.
• Extensive experience with state of art software development practices such as pair programming, Test Driven Development (TDD), refactoring, continuous integration.

Business Analysis and Leadership

• Able to attract, retain and lead people. Led team up to 25 developers, responsible for task planning, technical direction, risk management, code inspections, quality control, performance reviews, staff development. Understood how to create working environment to keep people happy, motivated, creative and productive.
• Solid presentation skill, proven in communication with customers, understanding problems, capturing domain knowledge and turning it into high level requirements in suitable form for development team and other stakeholders

Context root of single war's web applicaton in JBoss

Context root of web application is usually specifiedin META-INF/application.xml in ear file (or a vendor specific jboss-web.xml inside war ) e.g.

...
  <module>
    <web>
      <web-uri>app.war</web-uri>
      <context-root>/jsfejb3<context-root>
    </web>
  </module>
..

However if we drop a single war file without vendor specific descriptor (jboss-web.xml) into JBoss deploy directory then context root is implicitly the name of the war file without extension.
e.g.
Web application jmx-console residing in default/deploy/jmx-console.war has context root jmx-console
Web application groovy-scripts residing in default/deploy/groovy-script.war has context root groovy-script and can be accessible via http://localhost:7080/groovy-script

Centralized Management Tools

Nagios
Big Brother
Hyperic HQ
JOPR
@ctiveFriend
OEM
OpenNMS

Testing JOPR 2.2.0 - Basic configuration

Modify JBoss server startup options to include JVM as child resource of JBoss Server in JOPR

#$JBOSS_HOME/bin/run.conf
JAVA_OPTS="$JAVA_OPTS -Djboss.platform.mbeanserver"

Remove an uninterested resource from management a.k.a uninventory
Browse Resources->Servers, select uninterested one, hit uninventory button

Testing JOPR 2.2.0 - Installation

What is JOPR, RHQ, JBoss ON
JOPR is an open source project from Redhat to develop complex management platform. JOPR is agent based JAVA/J2EE system, where agent is standalone JAVA running and server runs on top of JBOSS Application Server(AS). It can be used to manage JBOSS AS, Tomcat, Apache as well as Oracle, and any IT resources.

JBoss Operations Network (JBoss ON) is commercial offer of JOPR in form of support service by Redhat.

JOPR is developed on top of RHQ project. RHQ serves as an maven upstream project for JOPR. RHQ consists of GUI management console, agent container and a set of agent plugins for management of common IT resources.
JOPR extends RHQ by adding additional plugins for JBoss App Server, Tomcat Web Server, etc. . JOPR version 2.2.0 is corresponding with RHQ version 1.2.0.

The original of RHQ is Hyperic HQ. Redhat and Hyperic work together to extract common reusable parts of Hyperic HQ and maintain it as RHQ. The aim is to create a common infrastructure for developing management's application. That is why the project named RHQ, where R stand for Redhat.

Install JOPR Server
Download JOPR binary from JOPR website
Follow instruction in Installation Guide to prepare database server for JOPR server
Change working directory to jopr-server-2.2.0 and type

bin/rhq-server.sh start

then open web browser and point to http://localhost:7080
Follow instructions in screen to set database driver, url, username/password to correct values. After that verify database connection.
Hit button Install JOPR server to complete installation process

Modify JOPR server properties file for agent communication
Modify rhq-server.properties file

# jopr-server-2.2.0/bin/rhq-server.properties
rhq.communications.connector.transport=servlet
rhq.communications.connector.bind-address=<jopr-server-ip-address>
rhq.communications.connector.bind-port=7080

Restart JOPR server

bin/rhq-server.sh stop
bin/rhq-server.sh start

Login to JOPR console and watch its log file
Pointing Web browser to http://localhost:7080/Dashboard.do
Login use default rhqadmin/rhqadmin as initial username/password
JOPR log file is located in jopr-server-2.2.0/logs/rhq-server-log4j.log

Install JOPR Agent
This shall be done in all machines that we want JOPR to manage .
Download agent from http://<jopr-server-hostname>:7080/agentupdate/download
Install JOPR Agent by typing

java -jar rhq-enterprise-agent-1.2.0.jar --install

Modify JOPR Agent configuration file

#rhq-agent/conf/agent-configuration.xml

# ask jopr agent not to ask for configuration during startup
<entry key="rhq.agent.configuration-setup-flag" value="true" />

<entry key="rhq.agent.name" value="<jopr-agent-hostname>"/>

<entry key="rhq.agent.server.bind-address" value="<jopr-server-ip-address>" />

# ask jopr agent to listen to on particular ip
<entry key="rhq.communications.connector.bind-address" value="<jopr-agent-ip-address>" />

Start RHQ Agent and watch its log file
change working directory to rhq-agent and run

bin/rhq-agent.sh -c conf/agent-configuration.xml 

If we reinstall JOPR server then agent's local data and server'data is not in sync. In that case we shall ask agent to clean it's local data when start by using

bin/rhq-agent.sh -l -c conf/agent-configuration.xml 

The most recent agent's log file is located in rhq-agent/logs/agent.log

Develop an international career - Recruiters

Recruiters (a.k.a Head Hunters) play a vital role in the job market. Many big companies use services of recruiters when they need to hire someone (they do not recruit directly). I myself and a lot of my colleagues got a job by a recruiter.
Below is a list of the most well known recruiters
- Robert Half
- Robert Walters
- Hays
- Wall Street
- Hudson
There are many smaller but also very good recruiters, who are constantly looking for a good candidate in all countries over the world. If you are somehow visible and your skill fit a opening position, they will contact you.
I got my first job via placement of a smaller recruiter named Juno located in New York.

JBoss resources

JBOSS configuration and tunning articles from Master the Boss
Running Multi JBoss Instances
Good examples of EJB 2.1, EJB 3 from Learn Technology
JBoss Official Docs & Examples from Redhat
JBoss Operation Network (ON), a.k.a JOPR - Documents
JBoss ON - FAQ
RHQ project, upon it JOPR depends
JOPR project
A blog on RHQ, JOPR, JBoss ON