HomeBlog 1 2 3 13 14

J2EE Security Model -JAAS

What is JAAS?

The Java Authentication and Authorization Service (JAAS) implements a Java version of the standard Pluggable Authentication Module (PAM) framework. JAAS simplifies Java security development by putting an abstraction layer between the application and the underlying authentication mechanisms, thereby enabling applications to be independent from the authentication mechanism.

 

 

 

New or updated authentication mechanisms can be plugged in without requiring modifications to the application. Applications initiate authentication by instantiating a LoginContext object, which in turn references a configuration that determines the authentication mechanisms or LoginModules to be used in performing the authentication. If authentication is successful, the subject is updated by a LoginModule with relevant principals and credentials.

Classes and interfaces

LoginModule (javax.security.auth.spi.LoginModule)

Login modules are written by implementing this interface; they contain the actual code for authentication. It can use various mechanisms to authenticate user credentials. The code could retrieve a password from a database and compare it to the password supplied to the module. It could also use a flat file, LDAP or any other means of storing user information for that purpose. Generally, in enterprise networks all authentication credentials are stored in one place, which might be accessed through LDAP.

LoginContext (javax.security.auth.login.LoginContext)

The login context is the core of the JAAS framework which kicks off the authentication process by creating a Subject. As the authentication process proceeds, the subject is populated with various principals and credentials for further processing.

Subject (javax.security.auth.Subject)

A subject represents a single user, entity or system –in other words, a client– requesting authentication.

Principal (java.security.Principal)

A principal represents the face of a subject. It encapsulates features or properties of a subject. A subject can contain multiple principals.

Credentials

Credentials are nothing but pieces of information regarding the subject in consideration. They might be account numbers, passwords, certificates etc. As the credential represents some important information, the further interfaces might be useful for creating a proper and secure credential – javax.security.auth.Destroyable and javax.security.auth.Refreshable. Suppose that after the successful authentication of the user you populate the subject with a secret ID (in the form of a credential) with which the subject can execute some critical services, but the credential should be removed after a specific time. In that case, one might want to implement the Destroyable interface. Refreshable might be useful if a credential has only a limited timespan in which it is valid.

Web Application URL-Based Authorization

In the J2EE security model, Web resources to be secured are identified by their URL patterns. This is specified in the web.xml file of the Web application. For example, the following excerpt is from the configuration to protect resources under the URL pattern “/resource” of an application.

<web-resource-collection>
    <web-resource-name>resource access</web-resource-name>
    <url-pattern>/resource</url-pattern>
  </web-resource-collection>

This is part of a security constraint in web.xml that also specifies the J2EE logical role that is allowed to access the resource. J2EE logical roles, discussed in the J2EE specification, include developers (application component providers), assemblers, deployers, and system administrators.

For example, assume the J2EE role sr_developers is declared in the web.xml file. The security constraint to allow this role to access the resource would look like this:

<security-constraint>
    <web-resource-collection>
      <web-resource-name>resource access</web-resource-name>
      <url-pattern>/resource</url-pattern>
    </web-resource-collection>
    <!-- authorization -->
    <auth-constraint>
      <role-name>sr_developers</role-name>
    </auth-constraint>
 </security-constraint>

The Java Authorization Contract for Containers (JACC) specification (JSR-115) defines new java.security.Permission classes to satisfy the Java EE 5 authorization model. JACC enables authorization decisions based on these permission classes.

This is a sample to use JAAS authentication with a windows active directory server. I use a Sun Java System Application Server, so the steps with other servers could be different.

Step 1: Defining LDAP realm

In this example you must define a LDAP realm named «ads-realm» with the following parameters:

Realm class:

com.sun.enterprise.security.auth.realm.ldap.LDAPReam

Properties:

directory            = ldap://ads.host.name:389
base-dn              = DC=ads,DC=domain,DC=com
search-bind-dn       = user
search-bind-password = password
search-filter        = (&(objectClass=user)(sAMAccountName=%s))
group-search-filter  = (&(objectClass=group)(member=%d))
jaas-context         = ldapRealm

You must change directory, base-dn, search-bind-dn and search-bind-password to your active directory configuration. The «search-bind-dn» and «search-bind-password» parameters are needed, because with default settings active directory doesn’t allow anonymous users to browse the directory.

Step 2: Setting the following JVM Switch for refferals

The following JVM switch is needed with active directory LDAP servers:

-Djava.naming.referral=follow

Add this switch to your server startup script or with the admin console.

Step 3a: Basic authentication

Add the following section to your web.xml or go to Step 3b for form
based authentication.

<login-config>
  <auth-method>BASIC</auth-method>
  <realm-name>ads-realm</realm-name>
</login-config>

Step 3b: Form based authentication

Add the following section to your web.xml:

<login-config>
  <auth-method>FORM</auth-method>
  <realm-name>ads-realm</realm-name>
  <form-login-config>
    <form-login-page>/login.html</form-login-page>
    <form-error-page>/login.html</form-error-page>
  </form-login-config>
</login-config>

Create the page /login.html with a least the following code:

<html>
  <head/>
  <body>
    <form action="j_security_check" method="post">
      Username: <input type="text" name="j_username"><br/>
      Password: <input type="password" name="j_password"><br/>
      <input type="submit" value="Login"/>
    </form>
  </body>
</html>

Step 4: Adding security role to web.xml

Add at least one security role to your web.xml, in this example «userRole».

<security-role>
  <role-name>userRole</role-name>
</security-role>

Step 5: Adding security constraint to web.xml

Now we must create a security constraint and the path to the pages we want to allow only authenticated access. In this sample the access to the folder /pages/ is resticted to authenticated users in role «userRole».

<security-constraint>
  <display-name>SecurityConstraint</display-name>
  <web-resource-colletion>
    <web-resource-name>SecuredFolder</web-resource-name>
      <url-pattern>/pages/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
      <role-name>userRole</role-name>
    </auth-constraint>
  <user-data-constraint>
    <transport-guarantee>NONE</transport-guarantee>
  </user-data-constraint>
</security-constraint>

Step 6: Create role mapping between active directory group and role

Role mappings are defined in sun-web.xml for the Sun Java System Application Server, so add the following section:

<security-role-mapping>
  <role-name>userRole</role-name>
  <group-name>users</group-name>
</security-role-mapping>

This maps the active directory group «users» to our role «userRole»,
so only users in the group «users» can access our secured folder.

Example JAAS and AD Integration Code

package com.example.authentication.activedirectory;

import javax.security.auth.callback.*;
import java.io.IOException;

/**
 * Copyright Alvin Alexander, http://devdaily.com.
 * This code is shared here under the Attribution 3.0 Unported License.
 * See this URL for details: http://creativecommons.org/licenses/by/3.0/
 *
 * This is an implementation of CallbackHandler to pass credentials to ActiveDirectoryValidator.java.
 * See JAAS documentation for more info.
 */
public class ADCallbackHandler implements CallbackHandler
{
  private String ADUserId;
  private char[] ADPassword;

  public void handle(Callback[] callbacks) throws java.io.IOException, UnsupportedCallbackException
  {
    for (int i = 0; i < callbacks.length; i++)
    {
      if (callbacks[i] instanceof NameCallback)
      {
        NameCallback cb = (NameCallback)callbacks[i];
        cb.setName(ADUserId);
      }
      else if (callbacks[i] instanceof PasswordCallback)
      {
        PasswordCallback cb = (PasswordCallback)callbacks[i];
        cb.setPassword(ADPassword);
      }
      else
      {
        throw new UnsupportedCallbackException(callbacks[i]);
      }
    }
  }

  public void setUserId(String userid)
  {
    ADUserId = userid;
  }

  public void setPassword(String password)
  {
    ADPassword = new char[password.length()];
    password.getChars(0, ADPassword.length, ADPassword, 0);
  }

  public static void main(String[] args) throws IOException, UnsupportedCallbackException
  {
    // Test handler
    ADCallbackHandler ch = new ADCallbackHandler();
    ch.setUserId("test");
    ch.setPassword("test");

    Callback[] callbacks = new Callback[] { new NameCallback("user id:"), new PasswordCallback("password:", true) };

    ch.handle(callbacks);
  }
}

If this Java source code works for you — or doesn’t work — please leave a comment below that may help other people who are in the same Java Active Directory boat.

Configure Tomcat

Copy Files

Copy waffle-jna.jar, jna.jar and platform.jar to Tomcat’s lib directory.

JAAS Realm

Add a JAAS realm to the application context. Modify _META-INF\context.xml _of your application.

<Context>
  <Realm className="org.apache.catalina.realm.JAASRealm"
         appName="Jaas"
         userClassNames="waffle.jaas.UserPrincipal"
         roleClassNames="waffle.jaas.RolePrincipal"
         useContextClassLoader="false"
         debug="true" />
</Context>

Authentication

Modify WEB-INF\web.xml of your application.

Enable BASIC, DIGEST or FORMS authentication for this realm.

<login-config>
  <auth-method>BASIC</auth-method>
  <realm-name>Jaas</realm-name>
</login-config>

Configure security roles. The Waffle login module adds all user’s security groups (including nested and domain groups) as roles during authentication.

<security-role>
  <role-name>Everyone</role-name>
</security-role>

Restrict access to website resources. For example, to restrict the entire website to locally authenticated users add the following.

<security-constraint>
  <display-name>Waffle Security Constraint</display-name>
  <web-resource-collection>
    <web-resource-name>Protected Area</web-resource-name>
    <url-pattern>/*</url-pattern>
  </web-resource-collection>
  <auth-constraint>
    <role-name>Everyone</role-name>
  </auth-constraint>
</security-constraint>

Login Configuration

Create a login configuration file, login.conf. This configuration file specifies how to plug the Waffle Windows Login Module.

Jaas {
    waffle.jaas.WindowsLoginModule sufficient;
};

The login.conf configuration file is passed to Java with -Djava.security.auth.login.config=<path-to-file>/login.conf.

JAAS Security Policy

Create JAAS policy configuration file, jaas.policy. This file specifies which identities are granted which permissions.

grant Principal * * {
  permission java.security.AllPermission "/*";
};

The policy file is passed to Java with -Djava.security.auth.policy=<path-to-file>/jaas.policy.

Start Tomcat

You must start Tomcat with Security Manager enabled (-security) and configure it with a login configuration and policy. For example, the following will start Tomcat using the demo login.conf and jaas.policy from the Waffle samples.

@echo off
setlocal
set JAVA_OPTS=-Djava.security.auth.login.config="C:/Program Files/Tomcat/webapps/waffle-jaas/login.conf" -Djava.security.auth.policy="C:/Program Files/Tomcat/webapps/waffle-jaas/jaas.policy"
call bin/catalina.bat run -security
endlocal

Demo Application

A demo application can be found in the Waffle distribution in the Samples\Tomcat\waffle-jaas directory. Copy the entire directory into Tomcat’s webapps directory, start Tomcat as explained above, and navigate to http://localhost:8080/waffle-jaas. You will be prompted for your Windows login, enter your Windows credentials and log-in.


Reference
What is JAAS or JACC?
https://www.ibm.com/developerworks/websphere/library/techarticles/0709_vamsi/0709_vamsi.html

AD Integration on J2EE
http://rundeck.org/docs/administration/authenticating-users.html

Oracle Leveraging JAAS
https://docs.oracle.com/cd/E16439_01/doc.1013/e13977/javaplat.htm

Declarative J2EE Authentication and Authorization with JAAS
http://www.oracle.com/technetwork/developer-tools/jdev/index-089689.html

JBOSS and JAAS
https://docs.jboss.org/jbossas/jboss4guide/r1/html/ch8.chapter.html

JAAS Example Code
http://alvinalexander.com/java/java-active-directory-jaas-example-source-code

JAAS and AD Integration Code Sample
http://code.dblock.org/2010/05/24/windowsactive-directory-authentication-tomcat-jaas-w-waffle.html

JAAS Authenication Model
http://www.javaranch.com/journal/2008/04/authentication-using-JAAS.html

JBOSS and JAAS
http://middlewaremagic.com/jboss/?p=378

J2EE and AD
https://www.gascoyne.de/archives/5

Hierarchical File System on Linux

The Linux File System Hierarchy

Image result for linux file system

Linux Directory Structure Overview

The Directory Structure in Unix & Linux are a unified Directory Structure where in all the directories are unified under the “/” Root file system. Irrespective of where the File System is physically mounted all the directories are arranged hierarchically under the Root file system.

The Linux Directory Structure follows the “Filesystem Hierarchy Structure (FHS)” maintained by the Free Standards Group although most of the distributions sometimes tend to deviate from the standards.

Lets have a quick stroll across the different directories under the Linux Filesystem Hierarchy

“/” Root

The Directory Structure starts with the Root file system “/” and is indeed the root directory for the whole structure.The partition where / (the root directory) will be located on a UNIX or UNIX-compatible system.

/boot

The /boot directory contains the Boot loader files including Grub or Lilo, the Kernel, initrd and system.map config files

/sys

This contains the Kernel, Firmware and system related files.

/sbin

Contains the essential System Binaries and System Administration tools essential for the system operation and performance

/bin

Contains the essential binaries for users and those utilities that are required in single user mode. Examples, include cat, ls, cp etc.

/lib

Contains the library files for all the binaries held in the /sbin & /bin directories

/dev

The /dev directory contains the essential system files and drivers.

/etc

The /etc/directory contain essential System configuration files including /etc/hosts, /etc/resolv.conf, nsswitch.conf, defaults and network configuration files. These are mostly host specific system and application configuration files.

/home

All the user home directories are held under this directory with the exception of the root home directory which is kept under /root directory. This directory holds users files, personal settings like .profile etc.

/media

A generic mount point for removable media like CD-ROM, USB, Floppies etc

/mnt

A generic mount point for temporary file systems. This comes handy particulary when troubleshooting from CDROM etc wherein you might have to mount the Root file system and edit configurations.

/opt

A rarely used directory in Linux for Optional Software Packages. This is extensively used in UNIX OS like Sun Solaris where the software packages are installed

/usr

A sub hierarchy to the root file system which is a User data directory. Contains user specific utilities and applications. You will again see a lot of important but not critical file systems are mounted. Here you would again find a bin, sbin & lib directory which contains non-critical user and system binaries and related libraries and a share directory. Also found here are the include directory with include files

/usr/sbin

Contains Non-essential Non-critical system binaries and network utilities

/usr/bin

Contains Non-Essential Non-critical command binaries for users.

/usr/lib

Library files for the binaries in /usr/bin & /usr/sbin directory.

/usr/share

A platform-independent shared data directory

/usr/local

A sub hierarchy under the /usr directory which has Local System specific data including user and system binaries and their libraries

/var

The /var directory is mostly mounted as a separate filesystem under the root where in all the variable content like logs, spool files for printers, crontab,at jobs, mail, running process, lock files etc. Care has to be taken in planning this file system and maintenance as this can fill up pretty quickly and when the FileSystem is full can cause system and application operational issues.

/tmp

A temporary file system which hold temporary files which are cleared at system reboot. There is also a /var/tmp directory which holds temporary files too. the only difference between the two is that /var/tmp directory holds files that are protected at system reboot. In other words, /var/tmp files are not flushed upon a reboot.

Then you have the virtual (psuedo) file system /proc which resides in the memory and is mounted under the Root holding kernel and process stats in text file formats.

Linux Directory Structure in Visual View

Note:-

This structure could vary from distro to distro and this is a very generic linux directory structure.The directory structure in Linux can be confusing for new users of Linux. Most distributions have the same general structure of the base directory system.

Reference
RHEL Linux File System Hierarchy Standard (FHS)
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Reference_Guide/s1-filesystem-fhs.html

 

How To Load Balance My SQL on Linux

How To Set Up A Load-Balanced MySQL Cluster

What is HA PROXY?

HAProxy is an open source software which can load balance HTTP and TCP servers. In the previous article on HAProxy we configured load balancing for HTTP and in this one we’ll do the same for MySQL. All your MySQL servers have to be configured to perform Master-Master replication as load balancing involves both reading and writing to all the backends.

The following three droplets will be used in this article:

Droplet 1 – Load Balancer
Hostname: haproxy
OS: Ubuntu
Private IP: 10.0.0.100

Droplet 2 – Node 1
Hostname: mysql-1
OS: Debian 7
Private IP: 10.0.0.1

Droplet 2 – Node 2
Hostname: mysql-2
OS: Debian 7
Private IP: 10.0.0.2

Before proceeding, make sure all MySQL servers are up, running and are properly replicating database writes.

Prepare MySQL Servers


We need to prepare the MySQL servers by creating two additional users for HAProxy. The first user will be used by HAProxy to check the status of a server.

root@mysql-1# mysql -u root -p -e "INSERT INTO mysql.user (Host,User) values ('10.0.0.100','haproxy_check'); FLUSH PRIVILEGES;"

A MySQL user is needed with root privileges when accessing the MySQL cluster from HAProxy. The default root user on all the servers are allowed to login only locally. While this can be fixed by granting additional privileges to the root user, it is better to have a separate user with root privileges.

root@mysql-1# mysql -u root -p -e "GRANT ALL PRIVILEGES ON *.* TO 'haproxy_root'@'10.0.0.100' IDENTIFIED BY 'password' WITH GRANT OPTION; FLUSH PRIVILEGES"

Replace haproxy_root and password with your own secure values. It is enough to execute these queries on one MySQL master as changes will replicate to others.

Install MySQL Client


MySQL client has to be installed on the HAProxy droplet to test connectivity.

root@haproxy# apt-get install mysql-client

Now try executing a query on one of the masters as the haproxy_root user.

root@haproxy# mysql -h 10.0.0.1 -u haproxy_root -p -e "SHOW DATABASES"

This should display a list of MySQL databases.

Installing HAProxy


On the HAProxy server install the package.

root@haproxy# apt-get install haproxy

Enable HAProxy to be started by the init script.

root@haproxy# sed -i "s/ENABLED=0/ENABLED=1/" /etc/default/haproxy

To check if this change is done properly execute the init script of HAProxy without any parameters.

root@haproxy:~# service haproxy
Usage: /etc/init.d/haproxy {start|stop|reload|restart|status}

Configuring HAProxy


Rename the original configuration file

mv /etc/haproxy/haproxy.cfg{,.original}

Create and edit a new one

nano /etc/haproxy/haproxy.cfg

The first block is the global and defaults configuration block.

global
    log 127.0.0.1 local0 notice
    user haproxy
    group haproxy

defaults
    log global
    retries 2
    timeout connect 3000
    timeout server 5000
    timeout client 5000

More information about each of these options are covered in this article. Since we’ve told HAProxy to send log messages to 127.0.0.1 we have to configure rsyslog to listen on it. This has too been covered in the same article under Configure Logging for HAProxy.

Moving to the main configuration part.

listen mysql-cluster
    bind 127.0.0.1:3306
    mode tcp
    option mysql-check user haproxy_check
    balance roundrobin
    server mysql-1 10.0.0.1:3306 check
    server mysql-2 10.0.0.2:3306 check

Unlike HTTP load balancing HAProxy doesn’t have a specific “mode” for MySQL so we use tcp. We’ve set HAProxy to listen only on the loopback address (assuming that application is on the same server) however if your application resides on a different droplet make it listen on 0.0.0.0 or the private IP address.

We need one more configuration block to see the statistics of load balancing. This is completely optional and can be omitted if you don’t want stats.

listen 0.0.0.0:8080
    mode http
    stats enable
    stats uri /
    stats realm Strictly\ Private
    stats auth A_Username:YourPassword
    stats auth Another_User:passwd

Replace the usernames and passwords in “stats auth”. This will make HAProxy listen on port 8080 for HTTP requests and the statistics will be protected with HTTP Basic Authentication. So you can access stats at

http://<Public IP of Load Balancer>:8080/

Once you’re done configuring start the HAProxy service.

service haproxy start

Use the mysql client to query HAProxy.

root@haproxy# mysql -h 127.0.0.1 -u haproxy_root -p -e "SHOW DATABASES"

The “-h” option has to be present with the loopback IP address. Omitting it or using localhost will make the MySQL client connect to the mysql.sock file which will fail.

Testing Load Balancing and Failover


To check if load balancing is working query the server_id variable twice or more.

root@haproxy# mysql -h 127.0.0.1 -u haproxy_root -p -e "show variables like 'server_id'"
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 1     |
+---------------+-------+
root@haproxy# mysql -h 127.0.0.1 -u haproxy_root -p -e "show variables like 'server_id'"
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 2     |
+---------------+-------+

This demonstrates roundrobin load balancing with equal weights, we’ll now change the weight for mysql-2 and see the results.

nano /etc/haproxy/haproxy.cfg

server mysql-2 10.0.0.2:3306 check weight 2

Reload to apply this change.

service haproxy reload

Query for the server_id multiple times.

root@haproxy:~# for i in `seq 1 6`
do
mysql -h 127.0.0.1 -u haproxy_root -ppassword -e "show variables like 'server_id'"
done

+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 1     |
+---------------+-------+
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 2     |
+---------------+-------+
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 2     |
+---------------+-------+
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 1     |
+---------------+-------+
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 2     |
+---------------+-------+
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 2     |
+---------------+-------+

Now load balancing works in the ratio of 1:2 with one-thirds of the requests going to mysql-1 and two-thirds going to mysql-2.

Fail a MySQL server either by stopping the service

root@mysql-1# service mysql stop

or bringing the interface down.

root@mysql-1# ifconfig eth1 down

Try the “show variables” query now to see the result. The following log entries will indicate when and how HAProxy detected the failure.

tail /var/log/haproxy/haproxy.log

Nov 15 00:08:51 localhost haproxy[1671]: Server mysql-cluster/mysql-1 is DOWN, reason: Layer4 timeout, check duration: 2002ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

Reducing Failover Interval


When a MySQL server goes down HAProxy takes some time to detect this failure and remove it from the cluster. In this section we’ll see how to control this time. First we’ll see how to measure this value. One way is to block the MySQL port using iptables for a certain amount of time, then remove the rule and check the log.

root@mysql-1:~# ifconfig eth1 down &&
date &&
sleep 20 &&
ifconfig eth1 up &&
date

Fri Nov 15 00:37:09 IST 2013
Fri Nov 15 00:37:29 IST 2013

The port 3306 was blocked for 20 seconds, we’ll look at the log file now.

root@haproxy:~# tail /var/log/haproxy.log
Nov 15 16:49:38 localhost haproxy[1275]: Server mysql-cluster/mysql-1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Nov 15 16:49:56 localhost haproxy[1275]: Server mysql-cluster/mysql-1 is UP, reason: Layer7 check passed, code: 0, info: "5.5.31-0+wheezy1-log", check duration: 1ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.

It took 6 seconds to detect a failure (difference between 16:49:38 and 16:49:32) and 4 seconds_to detect that the server can be reached (difference between 16:49:56 and 16:49:52). This is determined by the server parameters rise, fall and inter.

The rise parameter sets the number of checks a server must pass to be declared operational. Default is 2.

The fall parameter sets the number of checks a server must pass to be declared dead. Default is 3.

The inter parameter sets the interval between these checks. Default is 2000 milliseconds.

Putting this info together a server must fail 3 continuous checks which are performed at an interval of 2 seconds to be considered dead. So in our example above the following would’ve happened.

16:49:32 - Port 3306 on mysql-1 was blocked
16:49:34 - Check - Failed - Failure No. 1
16:49:36 - Check - Failed - Failure No. 2
16:49:38 - Check - Failed - Failure No. 3 (server removed and event logged)

And when the firewall rule was removed.

16:49:52 - Firewall rule removed port 3306 accessible
16:49:54 - Check - Passed - Success No. 1
16:49:56 - Check - Passed - Success No. 2 (server added to cluster and event logged)

The following settings will reduce the test interval to 1 second and also reduce the number of fall tests.

nano /etc/haproxy/haproxy.cfg

server mysql-1 10.0.0.1:3306 check fall 2 inter 1000
server mysql-2 10.0.0.2:3306 check fall 2 inter 1000

Sometimes you may not want to flood the private network with too many “test” packets especialy if you have a large amount of MySQL servers. In such cases the fastinter and downinter parameters will come handy.

The fastinter parameter sets the interval between checks while a server is transitioning UP or DOWN.

The downinter parameter sets the test interval when a server is DOWN.

That explanation might be confusing so we’ll see it with an example.

nano /etc/haproxy/haproxy.cfg

server mysql-1 10.0.0.1:3306 check fastinter 1000
server mysql-2 10.0.0.2:3306 check fastinter 1000

Since we haven’t specified the “inter” parameter it defaults to 2000ms. With this configuration we’ll restart HAProxy and do the test again.

root@mysql-1:~# iptables -A INPUT -p tcp --dport 3306 -j REJECT &&
date &&
sleep 20 &&
iptables -D INPUT -p tcp --dport 3306 -j REJECT &&
date
Fri Nov 15 17:18:48 IST 2013
Fri Nov 15 17:19:08 IST 2013

Check the HAProxy log file.

root@haproxy:~# tail /var/log/haproxy.log
Nov 15 17:18:52 localhost haproxy[1353]: Server mysql-cluster/mysql-1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Nov 15 17:19:11 localhost haproxy[1353]: Server mysql-cluster/mysql-1 is UP, reason: Layer7 check passed, code: 0, info: "5.5.31-0+wheezy1-log", check duration: 1ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.

Now it took only 4 seconds (compared to 6 earlier) to detect a failure and 3 seconds (compared to 4) to detect that the server was up. Behind the scenes this is what happened.

17:18:48 - Port 3306 blocked
17:18:50 - Check - Failed - Failure No. 1
17:18:51 - Check - Failed - Failure No. 2
17:18:52 - Check - Failed - Failure No. 3 (server removed and event logged)

And when the port was unblocked.

17:19:08 - Firewall rule removed
17:19:10 - Check - Passed - Success No. 1
17:19:11 - Check - Passed - Success No. 2 (server added to cluster and event logged)

First notice the interval between the port block event (17:18:48) and the first check (17:18:50), it is 2 seconds (the “inter” interval). Then notice the interval between Test 1 <-> Test 2 and Test 2 <-> Test 3, it is only 1 second (the “fastinter” interval). The same intervals can be noticed when the server moved from DOWN to UP. So “fastinter” controls the interval between these checks.

MYSQL HAPROXY Step by Step Installation and Configure
MY SQL HAPROXY LOAD BALANCING
HAPROXY Performance Tuning

How to Install and Configure MySQL Cluster on Linux

How to Install and Configure MySQL Cluster on Linux

MySQL Cluster is designed to provide a MySQL compatible database with high availability and low latency. The MySQL Cluster technology is implemented through the NDB (Network DataBase) and NDBCLUSTER storage engines and provides shared-nothing clustering and auto-sharding for MySQL database systems. In the shared-nothing architecture, each of nodes has its own memory and disk, the use of shared storage such as NFS, SANs is not recommended and supported.

To implement a MySQL Cluster, we have to install three types of nodes. Each node type will be installed on it’s own server. The components are:

1. Management Node – NDB_MGMD/MGM
The Cluster management server is used to manage the other node of the cluster. We can create and configure new nodes, restart, delete, or backup nodes on the cluster from the management node.

2. Data Nodes – NDBD/NDB
This is the layer where the process of synchronizing and data replication between nodes happens.

3. SQL Nodes – MySQLD/API
The interface servers that are used by the applications to connect to the database cluster.

In this tutorial, I will guide you trough the installation and configuration of a MySQL Cluster with centOS 7. We will configure the management node, two data nodes, and two SQL nodes.

Prerequisites

  • The OS is CentOS 7 – 64bit.
  • 5 CentOS servers or virtual machines. I will use the hostnames and IP addresses as shown below:
    • Management Node
      db1 = 192.168.1.120
    • Data Nodes
      db2 = 192.168.1.121
      db3 = 192.168.1.122
    • SQL Nodes
      db4 = 192.168.1.123
      db5 = 192.168.1.124

Step 1 – Setup Management Node

The first step is to create the “Management Node” with CentOS 7 db1 and IP 192.168.1.120. Make sure you are logged into the db1 server as root user.

A. Download the MySQL Cluster software

I’ll download it from the MySQL site with wget. I’m using the “Red Hat Enterprise Linux 7 / Oracle Linux 7 (x86, 64-bit), RPM Bundle ” here which is compatible with CentOS 7. Then extract the tar file.

cd ~
wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar
tar -xvf MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar

Install MySQL Cluster package rpm.

B. Install and Remove Packages

Before you install the rpm package for MySQL Cluster, you need to install perl-Data-Dumper that is required by the MySQL-Cluster server. And you need to remove mariadb-libs before we can install MySQL Cluster.

yum -y install perl-Data-Dumper
yum -y remove mariadb-libs

C. Install MySQL Cluster

Install MySQL Cluster package with these rpm commands:

cd ~
rpm -Uvh MySQL-Cluster-client-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-server-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-shared-gpl-7.4.10-1.el7.x86_64.rpm

Make sure there is no error.

D. Configure MySQL Cluster

Create a new directory for the configuration files. I will use the “/var/lib/mysql-cluster” directory.

mkdir -p /var/lib/mysql-cluster

Then create new configuration file for the cluster management named “config.ini” in the mysql-cluster directory.

cd /var/lib/mysql-cluster
vi config.ini

Paste the configuration below:

[ndb_mgmd default]
# Directory for MGM node log files
DataDir=/var/lib/mysql-cluster

[ndb_mgmd]
#Management Node db1
HostName=192.168.1.120

[ndbd default]
NoOfReplicas=2      # Number of replicas
DataMemory=256M     # Memory allocate for data storage
IndexMemory=128M    # Memory allocate for index storage
#Directory for Data Node
DataDir=/var/lib/mysql-cluster

[ndbd]
#Data Node db2
HostName=192.168.1.121

[ndbd]
#Data Node db3
HostName=192.168.1.122

[mysqld]
#SQL Node db4
HostName=192.168.1.123

[mysqld]
#SQL Node db5
HostName=192.168.1.124

Save the file and exit.

E. Start the Management Node

Next start the management node with the command below:

ndb_mgmd –config-file=/var/lib/mysql-cluster/config.ini

The result should be similar to this:

MySQL Cluster Management Server mysql-5.6.28 ndb-7.4.10
2016-03-22 19:26:08 [MgmtSrvr] INFO     — The default config directory ‘/usr/mysql-cluster’ does not exist. Trying to create it…
2016-03-22 19:26:08 [MgmtSrvr] INFO     — Successfully created config directory

The management node is started, now you can use command “ndb_mgm” to monitor the node:

ndb_mgm
show

Check cluster status.

You can see the management node has been started with: mysql-6.6 and ndb-7.4.

Step 2 – Setup the MySQL Cluster Data Nodes

We will use 2 CentOS servers for the Data Nodes.

  1. db2 = 192.168.1.121
  2. db3 = 192.168.1.122

A. Login as root user and download the MySQL Cluster software

Login to the db2 server with ssh:

ssh root@192.168.1.121

Then download the MySQL Cluster package and extract it:

cd ~
wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar
tar -xvf MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar

B. Install and Remove Packages

Install perl-Data-Dumper and remove the mariadb-libs:

yum -y install perl-Data-Dumper
yum -y remove mariadb-libs

C. Install MySQL Cluster

Now we can install the MySQL Cluster packages for the Data Nodes with these rpm commands:

cd ~
rpm -Uvh MySQL-Cluster-client-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-server-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-shared-gpl-7.4.10-1.el7.x86_64.rpm

Make sure there is no error.

D. Configure Data Node

Create a new configuration file in the /etc directory with the vi editor:

vi /etc/my.cnf

Paste configuration below:

[mysqld]
ndbcluster
ndb-connectstring=192.168.1.120     # IP address of Management Node

[mysql_cluster]
ndb-connectstring=192.168.1.120     # IP address of Management Node

Save the file and exit.

Then create the new directory for the database data that we defined in the management node config file “config.ini”.

mkdir -p /var/lib/mysql-cluster

Now start the data node/ndbd:

ndbd

results:

2016-03-22 19:35:56 [ndbd] INFO     — Angel connected to ‘192.168.1.120:1186’
2016-03-22 19:35:56 [ndbd] INFO     — Angel allocated nodeid: 2

MySQL cluster node is online.

Data Node db2 connected to the management node ip 192.168.1.120.

E. Redo step 2.A – 2.D on db3 server.

As we have 2 data nodes, please redo the steps 2.A – 2.D on our second data node.

Step 3 – Setup SQL Node

This is step contains the setup for the SQL Node that provides the application access to the database. We use 2 CentOS servers for the SQL Nodes:

  1. db4 = 192.168.1.123
  2. db5 = 192.168.1.124

A. Log in and Download MySQL Cluster

Login to the db4 server as root user:

ssh root@192.168.1.123

And download MySQL Cluster package:

cd ~
wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar
tar -xvf MySQL-Cluster-gpl-7.4.10-1.el7.x86_64.rpm-bundle.tar

B. Install and Remove Packages

Install perl-Data-Dumper and remove the mariadb-libs that conflict with MySQL Cluster.

yum -y install perl-Data-Dumper
yum -y remove mariadb-libs

C. Install MySQL Cluster

Install the MySQL Cluster server, client and shared package with the rpm commands below:

cd ~
rpm -Uvh MySQL-Cluster-client-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-server-gpl-7.4.10-1.el7.x86_64.rpm
rpm -Uvh MySQL-Cluster-shared-gpl-7.4.10-1.el7.x86_64.rpm

D. Configure the SQL Node

Create a new my.cnf file in the /etc directory:

vi /etc/my.cnf

And paste configuration below:

[mysqld]
ndbcluster
ndb-connectstring=192.168.1.120       # IP address for server management node
default_storage_engine=ndbcluster     # Define default Storage Engine used by MySQL

[mysql_cluster]
ndb-connectstring=192.168.1.120       # IP address for server management node

Save the file and exit the editor.

Start the SQL Node by starting the MySQL server:

service mysql start

E. Redo step 3.A – 3.D on db5 server.

Please redo the steps 3.A – 3.D on the second SQL server (db5).

Step 4 – Monitor the Cluster

To see the cluster status, we have to log into the management node db1.

ssh root@192.168.1.120

We can use the ndb_mgm command to see the cluster status:

ndb_mgm
ndb_mgm> show

Check the NDB clsuter state.

Another useful command is:

ndb_mgm -e “all status”
ndb_mgm -e “all report memory”

Step 5 – Testing the Cluster

To perform a test on our new MySQL Cluster, we have to login to the SQL Nodes db4 or db5 servers.

Login to the db4 server:

ssh root@192.168.1.123

Change the default MySQL password that stored in “.mysql_secret” file in root directory:

cd ~
cat .mysql_secret

this is my sample:

# The random password set for the root user at Tue Mar 22 19:44:07 2016 (local time): qna3AwbJMuOnw23T

Now change the password with command below:

mysql_secure_installation

Type your old mysql password and then type the new one, press enter to confirm all.

If all is done, you can login to the MySQL shell with your password:

mysql -u root -p

After you logged in, create a new root user with host “@“, so we will be able to access the MySQL from outside.

CREATE USER ‘root’@’%’ IDENTIFIED BY ‘aqwe123’;

Replace aqwe123 with your own secure password! Now you can see the new root user with host “@” on the MySQL user list:

select user, host, password from mysql.user;

And grant the new root user read and write access from the remote node:

GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY PASSWORD ‘*94CC7BF027327993D738E11…(Encrypted PASSWORD)’ WITH GRANT OPTION;

Grant priveliges.

Now try to create a new database from db4 server and you will see the database on db5 too.

This is just a sample result for testing the cluster data replication.

All nodes are online.

The MySQL Cluster has been setup successfully on CentOS 7 with 5 server nodes.

Conclusion

MySQL Cluster is a technology that provides High Availability and Redundancy for MySQL databases. It uses NDB or NDBCLUSTER as the storage engine and provides shared-nothing clustering and auto-sharding for MySQL databases.  To implement the cluster, we need 3 components: Management Node(MGM), Data Nodes (NDB) and SQL Nodes (API). Each of node must have its own memory and disk. It is not recommended to use network storage such as NFS. To install MySQL Cluster on a CentOS 7 minimal system, we have to remove the mariadb-libs package, mariadb-libs conflict with MySQL-Cluster-server and you have to install the perl-Data-Dumper package, it’s needed by MySQL-Cluster-server. A MySQL Cluster is easy to install and configure on multiple CentOS servers.

Reference
How To Install and Configure MySQL Cluster on Centos 7
https://www.howtoforge.com/tutorial/how-to-install-and-configure-mysql-cluster-on-centos-7/

How To Install and Configure MySQL on Centos 7
https://websetnet.com/install-configure-mysql-cluster-centos-7/

Install MySQL on RHEL 5 and 6
https://www.cyberciti.biz/faq/how-to-install-mysql-under-rhel/

Install Guide Centos
https://support.rackspace.com/how-to/configuring-mysql-server-on-centos/

Video Tutorial
My SQL server Installation and Configuration on Linux

MySQL on CentOS 7

MySQL/Mariadb on Centos 7

Install Docker and Run MySQL Image

How To Install a Kick Start Server on RHEL

What is Kick Start Server?

The Red Hat Kickstart installation method is used primarily (but not exclusively) by the Red Hat Enterprise Linux operating system to automatically perform unattended operating system installation and configuration. Red Hat publishes Cobbler as a tool to automate the Kick Start configuration process.

 

Installing Kickstart Server on RHEL

The first step in setting up a Kickstart server is to install a basic web server to hold the Kickstart configuration files and OS installation files. It is assumed that there is already a basic CentOS web server ready to go, with a static IP address and GUI loaded.

Due to testing in a private lab environment, we can disable SELINUX. Do this by manually editing /etc/selinux/config to set SELINUX=disabled.

We will also be disabling iptables in the environment with this command:

    chkconfig –level 35 iptables off

After this is done, you will need to reboot the host. Check that SELINUX is disabled by using the command ‘getenforce.’ It should report back as disabled.

Log in to the server as root and check to see if a web server is installed by opening a terminal session; use the following command to check if Apache web server is installed:

    yum info httpd

If the yum command shows the status as installed, skip the Apache web server installation and configuration steps below. If the server shows the Apache web server is not installed, install it using the command below as the root user in a terminal window:

    yum -y install httpd

Once the web server is installed, start it by using the command:

    service httpd start

There may be some warnings about being unable to resolve the hostname. In this instance it isn’t a problem and the warning can be ignored. To check the web server is working, use the command:

    service httpd status

Make the web server automatically start on bootup, use the command:

    chkconfig –level 35 httpd on

Now the web server configuration steps are complete. Below are some rudimentary troubleshooting steps if something wasn’t working correctly.

Troubleshooting Your Kickstart Installation

At this point the basic web server should be up and running, even if the server is rebooted. Check that it is working by using a web browser on a separate machine and try and load the web page of the server, using its IP address if needed. If you cannot browse to the web page I would suggest checking the following items:

  1. Check the basic connectivity between the two machines.
  2. Check the status of the Apache web server.
  3. Ensure that no firewalls are blocking port 80 traffic.

 

Assuming that everything is now working we can progress with setting up the Kickstart server.

Setting up Kickstart

To make the server work with Kickstart, we need to add some content to the web server in the form of installation files and Kickstart configuration files. Using the console or terminal session, create a folder in the web server document root. In CentOS using Apache, the default document root is

    /var/www/html/

It is advisable that rather than just placing the files in the root of the web folder, create a folder layout that lends itself to running multiple versions of software and architectures for various Kickstart installs. Each CentOS version and architecture should have its own folder structure. To create a basic folder setup adhering to the above recommendation use the command:

    mkdir -p /var/www/html/centos/6/x64

The easiest way to copy the contents across to the new web server is to copy the contents from the installation media used to the relevant folder. This is done by using the following commands:

    mount /dev/cdrom /media
    cp -r /media/* /var/www/html/centos/6/x64/

Note: if a different version or operating system is being used, the media command will need to be modified to reflect the name of the mounted media; for example, CentOS_6.5_Final may need to be changed.

Once this is complete, it is time to start creating our Kickstart file. The Kickstart file holds what could be termed an “answers” file to questions that the installer needs to know in order to perform a complete installation. It is possible to manually create a kickstart file using a text editor but RHEL and CentOS provide a GUI Kickstart maker program.

Install the Kickstart builder application using the command:

    yum -y install system-config-kickstart.noarch

After installation is complete, notice that you have a new application, as shown above. Filling in the details field by field allows you to configure the settings for each configuration option. One thing to be aware of is the location of the installation files on the web server, especially as there is a nice ordered folder structure with various version folders.

Once the Kickstart file is created, it needs to be saved to web servers’ root folder so that any new clients can read it. It isn’t required to be in the root, but a user will need to make adjustments when the guests are installed. As mentioned before, by default, the root folder is /var/www/html/ in a standard CentOS installation. When it comes to saving the file, it is advised to give each configuration created a simple but informative name.

Booting Your Kickstart

All the hard work is now done and all that is left to do is to boot off a minimal install CD (or any bootable install media).

Run the automated install by booting from the disk media. When the guests get to the boot screen press the tab key. This allows you to change or add options to the boot configuration.

Simply add the addition content, substituting the IP address for that of the web server in use and the ks.cfg file for the Kickstart file name when the test Kickstart file was created earlier.

    vmlinuz initrd=initrd.img ks=http://192.168.200.131/ks.cfg

Now the guest system will boot the initial kernel and load the Kickstart file. A user will however have to interact with the installer when it asks how the disks are to be configured. This is a safety measure to ensure a user really does want to wipe the disks in question; however, it can be overridden. If someone wants to do this, please, make sure they understand that removing this safety feature can be dangerous.

Reference
How To Install a Kick Start Server on RHEL
http://www.tomsitpro.com/articles/kickstart-linux-automation,2-798.html

How to Install KickStart on Red Hat
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Installation_Guide/ch-kickstart2.html

Deploy J2EE Web Application on Web Container-

Image result for war and ear files

Basic concepts of web applications, how they work and the HTTP protocol

Image result for war file j2ee what is it

What is EAR File?

EAR (Enterprise Application Archive) is a file format used by Java EE for packaging one or more modules into a single archive so that the deployment of the various modules onto an application server happens simultaneously and coherently.
It also contains XML files called deployment descriptors which describe how to deploy the modules.

Image result for ear file j2ee what is it

 

What is WAR File?
A WAR file is, Compressed package containing Java-based Web components and applications that are run on a Web server; formatted the same way as a .JAR file,
but includes additional information that tells the application server which Java servlet class to run.

 

Related image

Deploying EAR and WAR Files on CENTOS/RHEL

Summary

Deploying an application with JBoss AS is a straightforward task. If you have access to the host where JBoss is installed you can simply copy the application (.jar , .war, .ear etc) into the JBOSS_HOME/server/<your config>/deploy folder.

Simply substitute <your config> with your server configuration (default, all, minimal, web, standard)

An application can be deployed in two flavours:

  • Packed archive (.jar , .war, .ear)
  • Exploded directory (directory whose name ends with .jar, .war, .ear)

 It is advised to use exploded directory for development and packed archive for production since you can do partial redeployments of the application  with exploded directory.

For example if you are deploying a Web application and you want to update just one Servlet, with exploded directory you can do it without redeploying the application.

Also remember that simply “touching” a Java EE configuration file (web.xml , application.xml) causes the application to redeploy. In the same way, touching a JBoss specific configuration file (like jboss-web.xml) will cause the application to redeploy as well.

 

Newly-deployed content may not show up in the JBoss ON inventory for as long as 24 hours, even if it was successfully created. By default, discovery scans for services are only made every 24 hours.
To see it immediately, run an execute prompt command operation on the agent and enter the discovery command. This runs a discovery scan.
  1. Search for the JBoss server instance to which to deploy the EAR or WAR.
  2. On the details page for the selected JBoss server instance, open the Inventory tab.
  3. In the Create New menu at the bottom, select the item for – Web Application (WAR) or – Enterprise Application (EAR), as appropriate.
  4. Enter the version number.
    This is not used for the resource. The actual version number is calculated based on the spec version and implementation version in MANIFEST.MF, if any are given, or the calculated SHA-256 value for the package itself:
    SPEC(IMPLEMENTATION)[sha256=abcd1234]
    If no version numbers are defined in MANIFEST.MF, then the SHA value is used. The SHA value is always used to identify the package version internally.

    NOTE

    When the EAR or WAR file is exploded after it is deployed, the MANIFEST.MF file is updated to include the calculated SHA version number. For example:
    Manifest-Version: 1.0
    Created-By: Apache Maven
    RHQ-Sha256: 570f196c4a1025a717269d16d11d6f37 ...
    For more information on package versioning, see “Deploying Applications and Content”.
  5. Upload the EAR/WAR file.
  6. Enter the information for the application to be deployed.
    • Whether the file should be exploded (unzipped) when it is deployed.
    • The path to the directory to which to deploy the EAR or WAR package. The destination directory is relative to the JBoss server instance installation directory; this cannot contain an absolute path or go up a parent directory.
    • Whether to back up any existing file with the same name in the target directory.
Once the EAR/WAR file is confirmed, the new child resource is listed in the Child History subtab of the Inventory tab.
Basic concepts of web applications, how they work and the HTTP protocol
WAR Child Resource

Deploying applications on JBoss AS 7

Applications are deployed differentely depending on the type of server. If you are deploying to a domain of servers then you need the Command Line Interface because the application server needs to be informed on which server group/s  the deployment is targeted.

Ex. Deploy an application on all server groups:
 

deploy MyApp.war --all-server-groups

Ex. Deploy an application on one or more server groups (separated by a comma):

deploy application.ear --server-groups=main-server-group

If you are deploying to a standalone server then you can either use the CLI or drop the deployment unit into the server deployments folder.

 

 The deployments folder is the location in which users can place their deployment content (for example, WAR, EAR, JAR, SAR fi les) to have it automatically deployed into the server runtime. Users, particularly those running production systems, are encouraged to use the JBoss AS management APIs to upload and deploy deployment content instead of relying on the deployment scanner subsystem that periodically
scans this directory

 

As soon as the deployer HD scanner detects the application, the module is moved to the work folder of the application, leaving a placeholder Test.war.deployed file in the deployments folder.

jboss 7 tutorial

Note: With the default configuration, packaged archives (.ear, .war, .jar, .sar) are automatically deployed. Exploded archives need adding a .dodeploy empty file in the deployments folder to trigger deployment.

 

Reference

JBoss Web Web Application Deployment
https://docs.jboss.org/jbossweb/3.0.x/deployer-howto.html#Deployment%20on%20JBoss%20Web%20startup

Deploy  EAR and WAR File on RHEL JBOSS
https://access.redhat.com/documentation/en-US/JBoss_Operations_Network/3.1/html/How_to_Manage_JBoss_Servers/Child_Resource_types-EAR_and_WAR.html

Deploy Application on Jboss AS 7
http://www.mastertheboss.com/jboss-server/jboss-as-7/jboss-as-7-introduction

Video Tutorial

Basic concepts of web applications, how they work and the HTTP protocol

Web Server VS Web Container vs Application Server

How to Deploy your application On JBoss

 

Deploy Docker on Vmware

Docker Certification Program provides a way for technology partners to validate and certify their software or plugin as a container for use on the Docker Enterprise Edition platform.  Since the initial launch of the program in March, more Containers and Plugins have been certified and available for download.

VMware vSphere

Docker Certified containers

Certified Containers and Plugins are technologies that are built with best practices as Docker containers, tested and validated against the Docker Enterprise Edition platform and APIs, pass security requirements, reviewed by Docker partner engineering and cooperatively supported by both Docker and the partner. Docker Enterprise Edition and Certified Technology provide assurance and support to businesses for their critical application infrastructure.

Check out the latest Docker Certified technologies to the Docker Store:

Estimated reading time: 1 minute Creates machines on a VMware vSphere Virtual Infrastructure. The machine must have a working vSphere ESXi installation. You can use a paid license or free 60 day trial license. Your installation may also include an optional VCenter server.

Usage

$ docker-machine create --driver vmwarevsphere --vmwarevsphere-username=user --vmwarevsphere-password=SECRET vm

Options

  • --vmwarevsphere-username: required vSphere Username.
  • --vmwarevsphere-password: required vSphere Password.
  • --vmwarevsphere-cpu-count: CPU number for Docker VM.
  • --vmwarevsphere-memory-size: Size of memory for Docker VM (in MB).
  • --vmwarevsphere-disk-size: Size of disk for Docker VM (in MB).
  • --vmwarevsphere-boot2docker-url: URL for boot2docker image.
  • --vmwarevsphere-vcenter: IP/hostname for vCenter (or ESXi if connecting directly to a single host).
  • --vmwarevsphere-vcenter-port: vSphere Port for vCenter.
  • --vmwarevsphere-network: Network where the Docker VM will be attached.
  • --vmwarevsphere-datastore: Datastore for Docker VM.
  • --vmwarevsphere-datacenter: Datacenter for Docker VM (must be set to ha-datacenter when connecting to a single host).
  • --vmwarevsphere-pool: Resource pool for Docker VM.
  • --vmwarevsphere-hostsystem: vSphere compute resource where the docker VM will be instantiated (use /* or / if using a cluster).

The VMware vSphere driver uses the latest boot2docker image.

Environment variables and default values

CLI option Environment variable Default
--vmwarevsphere-username VSPHERE_USERNAME
--vmwarevsphere-password VSPHERE_PASSWORD
--vmwarevsphere-cpu-count VSPHERE_CPU_COUNT 2
--vmwarevsphere-memory-size VSPHERE_MEMORY_SIZE 2048
--vmwarevsphere-boot2docker-url VSPHERE_BOOT2DOCKER_URL Latest boot2docker url
--vmwarevsphere-vcenter VSPHERE_VCENTER
--vmwarevsphere-vcenter-port VSPHERE_VCENTER_PORT 443
--vmwarevsphere-disk-size VSPHERE_DISK_SIZE 20000
--vmwarevsphere-network VSPHERE_NETWORK
--vmwarevsphere-datastore VSPHERE_DATASTORE
--vmwarevsphere-datacenter VSPHERE_DATACENTER
--vmwarevsphere-pool VSPHERE_POOL
--vmwarevsphere-hostsystem VSPHERE_HOSTSYSTEM
Reference
Vmware Certified Docker
https://blog.docker.com/2017/05/latest-docker-certified-container-plugins-march-april-2017/
Vmware and Docker Blog
https://blog.docker.com/2014/08/docker-vmware-1-1-3/
Vmware Containers
https://octo.vmware.com/vmware-docker-better-together/

How To Install Docker On Linux

Get Docker for Red Hat Enterprise Linux

Image result for Docker

DOCKER ON RHELTo get started with Docker on Red Hat Enterprise Linux (RHEL), make sure you meet the prerequisites, then install Docker.

Prerequisites

Docker EE repository URL

To install Docker Enterprise Edition (Docker EE), you need to know the Docker EE repository URL associated with your trial or subscription. To get this information:

  • Go to https://store.docker.com/?overlay=subscriptions.
  • Choose Get Details / Setup Instructions within the Docker Enterprise Edition for Red Hat Enterprise Linux section.
  • Copy the URL from the field labeled Copy and paste this URL to download your Edition.

Use this URL when you see the placeholder text <DOCKER-EE-URL>.

To learn more about Docker EE, see Docker Enterprise Edition.

Docker Community Edition (Docker CE) is not supported on Red Hat Enterprise Linux.

OS requirements

To install Docker, you need the 64-bit version of RHEL 7, running on an x86 hardware platform.

In addition, you must use the devicemapper storage driver. On production systems, you must use direct-lvm mode, which requires one or more dedicated block devices. Fast storage such as solid-state media (SSD) is recommended.

Uninstall old versions

Older versions of Docker were called docker or docker-engine. If these are installed, uninstall them, along with associated dependencies.

$ sudo yum remove docker \
                  docker-common \
                  container-selinux \
                  docker-selinux \
                  docker-engine

It’s OK if yum reports that none of these packages are installed.

The contents of /var/lib/docker/, including images, containers, volumes, and networks, are preserved. The Docker EE package is now called docker-ee.

Install Docker EE

You can install Docker in different ways, depending on your needs:

  • Most users set up Docker’s repositories and install from them, for ease of installation and upgrade tasks. This is the recommended approach.
  • Some users download the RPM package and install it manually and manage upgrades completely manually. This is useful in situations such as installing Docker on air-gapped systems with no access to the internet.

Install using the repository

Before you install Docker for the first time on a new host machine, you need to set up the Docker repository. Afterward, you can install and update Docker from the repository.

Set up the repository

  1. Remove any existing Docker repositories from /etc/yum.repos.d/.
  2. Store two yum variables in /etc/yum/vars/.
    • Store your EE repository URL in /etc/yum/vars/dockerurl. Replace <DOCKER-EE-URL> with the URL you noted down in the prerequisites.
      $ sudo sh -c 'echo "<DOCKER-EE-URL>" > /etc/yum/vars/dockerurl'
      
    • Store your RHEL version string in /etc/yum/vars/dockerosversion. Use the appropriate value from the following table. Most users should use 7.
      Version string Description
      7 Unless you have specific requirements, you should use this version. Dependencies are not locked to specific versions but use the latest available version.
      7.3 Dependencies are locked to specific packages for RHEL 7.3.
      7.2 Dependencies are locked to specific packages for RHEL 7.2.
      $ sudo sh -c 'echo "<VERSION-STRING>" > /etc/yum/vars/dockerosversion'
      
  3. Install required packages. yum-utils provides the yum-config-manager utility, and device-mapper-persistent-data and lvm2 are required by the devicemapper storage driver.
    $ sudo yum install -y yum-utils device-mapper-persistent-data lvm2
    
  4. Use the following command to add the stable repository:
    $ sudo yum-config-manager \
        --add-repo \
        <DOCKER-EE-URL>/docker-ee.repo
    

Install Docker

  1. Update the yum package index.
    $ sudo yum makecache fast
    

    If this is the first time you have refreshed the package index since adding the Docker repositories, you will be prompted to accept the GPG key, and the key’s fingerprint will be shown. Verify that the fingerprint matches DD91 1E99 5A64 A202 E859 07D6 BC14 F10B 6D08 5F96 and if so, accept the key.

  2. Install the latest version of Docker EE, or go to the next step to install a specific version.
    $ sudo yum -y install docker-ee
    
  3. On production systems, you should install a specific version of Docker instead of always using the latest. List the available versions. This example uses the sort -r command to sort the results by version number, highest to lowest, and is truncated.

    Note: This yum list command only shows binary packages. To show source packages as well, omit the .x86_64 from the package name.

    $ yum list docker-ee.x86_64  --showduplicates |sort -r
    
    docker-ee.x86_64  17.03.0.el7                               docker-ee-stable   
    

    The contents of the list depend upon which repositories you have enabled, and will be specific to your version of RHEL (indicated by the .el7 suffix on the version, in this example). Choose a specific version to install. The second column is the version string. The third column is the repository name, which indicates which repository the package is from and by extension extension its stability level. To install a specific version, append the version string to the package name and separate them by a hyphen (-):

    $ sudo yum -y install docker-ee-<VERSION_STRING>
    
  4. Edit /etc/docker/daemon.json. If it does not yet exist, create it. Assuming that the file was empty, add the following contents.
    {
      "storage-driver": "devicemapper"
    }
    
  5. For production systems, you must use direct-lvm mode, which requires you to prepare the block devices. Follow the procedure in the devicemapper storage driver guidebefore starting Docker.
  6. Start Docker.
    $ sudo systemctl start docker
    
  7. Verify that Docker EE is installed correctly by running the hello-world image.
    $ sudo docker run hello-world
    

    This command downloads a test image and runs it in a container. When the container runs, it prints an informational message and exits.

Docker EE is installed and running. You need to use sudo to run Docker commands. Continue to Linux postinstall to allow non-privileged users to run Docker commands and for other optional configuration steps.

Upgrade Docker EE

To upgrade Docker EE, first run sudo yum makecache fast, then follow the installation instructions, choosing the new version you want to install.

Install from a package

If you cannot use the official Docker repository to install Docker, you can download the .rpm file for your release and install it manually. You will need to download a new file each time you want to upgrade Docker.

  1. Go to the Docker EE repository URL associated with your trial or subscription in your browser. Go to 7/x86_64/stable-17.03/Packages and download the .rpm file for the Docker version you want to install.

    Note: If you have trouble with selinux using the packages under the 7 directory, try choosing the version-specific directory instead, such as 7.3.

  2. Install Docker EE, changing the path below to the path where you downloaded the Docker package.
    $ sudo yum install /path/to/package.rpm
    
  3. Edit /etc/docker/daemon.json. If it does not yet exist, create it. Assuming that the file was empty, add the following contents.
    {
      "storage-driver": "devicemapper"
    }
    
  4. For production systems, you must use direct-lvm mode, which requires you to prepare the block devices. Follow the procedure in the devicemapper storage driver guidebefore starting Docker.
  5. Start Docker.
    $ sudo systemctl start docker
    
  6. Verify that docker is installed correctly by running the hello-world image.
    $ sudo docker run hello-world
    

    This command downloads a test image and runs it in a container. When the container runs, it prints an informational message and exits.

Docker EE is installed and running. You need to use sudo to run Docker commands. Continue to Post-installation steps for Linux to allow non-privileged users to run Docker commands and for other optional configuration steps.

Reference
Docker Docs
https://docs.docker.com/engine/installation/linux/rhel/#install-from-a-package

 Docker on RHEL
https://docs.docker.com/engine/installation/linux/rhel/

Docker on SUSE
https://docs.docker.com/engine/installation/linux/suse/

Docker on Ubuntu
https://docs.docker.com/engine/installation/linux/suse/

Docker Binaries Installation
https://docs.docker.com/engine/installation/binaries/#install-static-binaries

Docker on Azure
https://docs.docker.com/docker-for-azure/why/

Docker on Vmware
https://docs.docker.com/machine/drivers/vsphere/#options

Video Tutorial

What is Docker?

Install Docker on RHEL

Install Docker on Windows

Docker Beginner Tutorial – Install Docker on Linux -Step by Step

How to Install Apache HTTP Servecr on Linux

What is Apache HTTP?

Apache HTTP Server, colloquially called Apache (/əˈpætʃiː/ ə-PA-chee), is free and open-source cross-platform web server software,
released under the terms of Apache License 2.0. Apache is developed and maintained by an open community of developers under the auspices of the Apache Software Foundation.
commonly used on a Unix-like system (usually Linux), the program is available for Microsoft Windows as well.
Version 2.0 improved support for non-Unix, e.g. Windows and OS/2 (and eComStation). Old versions of Apache were ported to run on e.g. OpenVMS, and NetWare.

Apache HTTP Server can be compiled or downloaded from specific OS enviroments.

CenTOS
Install Apache and PHP on CentOS 6
Last updated on: 2016-06-21 Authored by: Rackspace Support
This article demonstrates how to install Apache and PHP on CentOS 6.
CentOS 6 comes with Apache 2.2.3 and PHP 5.1.6, and
you can install them by using the default CentOS Package Manager, yum. The advantages of using yum (as opposed to installing by using source code)
are that you get any security updates (when they are distributed) and dependencies are automatically handled.

Install Apache
Run the following command:
#sudo yum install httpd mod_ssl

Ubuntu
The Apache2 web server is available in Ubuntu Linux. To install Apache2:
At a terminal prompt enter the following command:

#sudo apt install apache2

SLES
The Apache web server can be installed by using zypper. Open a terminal and become root. Type the below command:
# zypper in apache2

Prerequisite
root account on the server machine.

Binaries Download Apache
Apache HTTP Server download site. Download the source files appropriate to your system.
Binary releases for some operating systems are available as well.
http://httpd.apache.org/download.cgi
Extract the Apache Files

uncompress file after complete download

#$gunzip -d httpd-2_0_NN.tar.gz
#tar xvf httpd-2_0_NN.tar

This creates a new directory under the current directory with the source files.

Configuring Your Server for Apache

Once you have the files, you need to tell your machine where to find everything by configuring the source files.
The easiest way is to accept all the defaults and just type:
./configure

Apache Standards Option is the prefix=PREFIX option.
This specifies the directory where the Apache files will be installed.

Specific environment variables and modules.

1.mod_alias – to map different parts of the URL tree
2.mod_include – to parse Server Side Includes
3.mod_mime – to associate file extensions with its MIME-type
4.mod_rewrite – to rewrite URLs on the fly

5.mod_speling (sic) – to help your readers who might misspell URLs
6.mod_ssl – to allow for strong cryptography using SSL
7.mod_userdir – to allow system users to have their own web page directories

For details about the modules go Apache homepage

Overview for the impatient

Download Download the latest release from http://httpd.apache.org/download.cgi
Extract $ gzip -d httpd-NN.tar.gz
$ tar xvf httpd-NN.tar
$ cd httpd-NN
Configure $ ./configure --prefix=PREFIX
Compile $ make
Install $ make install
Customize $ vi PREFIX/conf/httpd.conf
Test $ PREFIX/bin/apachectl -k start

Build Apache source installation, you’ll then need to build the installation:

Perform make Command

make install

Customize Apache

Assuming that there were no problems, you are ready to customize your Apache configuration. This really just amounts to editing the httpd.conf file.
This file is located in the PREFIX/conf directory. I generally edit it with text editor.
vi PREFIX/conf/httpd.conf

Note: you’ll need to be root to edit this file.
Follow the instructions in this file to edit your configuration the way you want it. More help is available on the Apache website.

Test Your Apache Server
Open a web browser on the same machine and type http://localhost/ in the address box.

You should see a page similar to the one in the partial screen shot above. It will say in big letters “Seeing this instead of the website you expected?”

This is good news, as it means your server is installed correctly.
Start Editing/Uploading Pages to Your Newly Installed Apache Web Server

Once your server is up and running you can start posting pages. Have fun building your website!

Reference
Apache HTTP Apache
http://httpd.apache.org/

Apache HTTP Server Version 2.4
http://httpd.apache.org/docs/2.4/install.html

Apache HTTP Serer Download Binaries
http://httpd.apache.org/docs/2.4/install.html#download

Apache on Ubuntu
https://help.ubuntu.com/lts/serverguide/httpd.html

Apache on Centos/RHEL
https://www.centos.org/docs/5/html/Cluster_Administration/s1-apache-inshttpd-CA.html

Apache on SLES
https://en.opensuse.org/SDB:Apache_installation

Video Tutorial
How to install and run Apache web server in Ubuntu Linux

How Install LAMP on Ubuntu Web Server

 

How To Install Jboss J2EE Application Server on Linux

jboss_logo

How To Install Jboss J2EE Application Server on Linux

Scope:
In this tutorial we will show how to install and configuration RHEL Jboss.

What is Jboss?

It is an open-source application server developed by RedHat based on the J2EE platform for developing and deploying enterprise Java applications, Web applications, Services, Portals and more. The J2EE allows applications to be standardized and modular allowing JAVA to handle many programming aspects when developing an application.

Jboss Directory Structure

Jboss Directory Structure Overview

Default Server Configuration File Set
An expanded view of the default server configuration file set conf and deploy directories

Prerequisites

Jboss binaries

*The most recent release of JBoss is available from the SourceForge JBoss project files page, http://sourceforge.net/projects/jboss. You will also find previous releases as well as beta and release candidate versions of upcoming releases.

Implementation Installation Steps

    Step 1: JDK installation and verification
    Step 2: Download JBoss and the installation procedure
    Step 3: Create the appropriate user
    Step 4: Start our new JBoss server and verify that the server has started properly
    Step 5: Stop the new JBoss server and verify that the server has shutdown properly

Step 1: JDK Installation and verification

The first step before installing JBoss AS 7, is to install a JDK. Any JDK can be used, such as Sun JDK, OpenJDK, IBM JDK, or JRocket etc. We chose Open JDK 6 for this tutorial, because it is the new Java reference implementation starting with Java 7.

NOTE: JDK 7 and above can also be used with JBoss. A JRE is also sufficient to run JBoss 7, however a JRE does not include some of the additional feature of a JDK.
Installing OpenJDK:
Issue the following command to install the JDK:

$ su -c “yum install java-1.8*”
Confirming the install:

Issue the following command to confirm that the proper version of the JDK is on your classpath:

$ java -version

NOTE: For our installation, we are not defining a explicit JAVA_HOME for JBoss AS 7. The default works in this situation, because we don’t have multiple java versions installed. For most production environments with multiple versions of Java, it is recommended to set the JAVA_HOME in the standalone.conf or domain.conf files.
Step 2: Download JBoss and the installation procedure

The next step is to download the appropriate version of JBoss AS 7. We will download the .zip version of JBoss AS 7, and install it using the unzip utility.
Downloading JBoss AS 7.1.1.Final:

Issue the following wget to download jboss-as-7.x Final.zip:

wget http://download.jboss.org/jbossas/7.1/jboss-as-7.1.1.Final/jboss-as-7.1.1.Final.zip

NOTE: jboss-as-7.1.1.Final.zip can also be downloaded with your favorite browser from the http://www.jboss.org/jbossas/downloads/ page.
Installing JBoss AS 7.1.1.Final:

Next, we issue the following unzip command to finally install jboss-as-7.1.1.Final in the /usr/share directory:

$ unzip jboss-as-7.1.1.Final.zip -d /usr/share

## chown jboss: -R /opt/jboss-as-7.1.1.Final

Ok, with all that in place we are somewhat ready to start the JBoss server. Before doing so, let’s fine-tune some stuff, like enable access to the JBoss web interface and set it up to run as a system service, so that it can automatically start on system start-up and you can use service jboss [start|stop] etc to manage the Jboss service.

– enable access to JBoss interface:

## vim /opt/jboss-as-7.1.1.Final/standalone/configuration/standalone.xml

<interface name=”management”>
<!–<inet-address value=”${jboss.bind.address.management:127.0.0.1}”/>–>
<any-ipv4-address/>
</interface>
<interface name=”public”>
<!–<inet-address value=”${jboss.bind.address:127.0.0.1}”/>–>
<any-ipv4-address/>
</interface>

This will make JBoss’ public and management frontend available from anywhere (using authentication of course) so you can manage it via a browser.

– set-up Jboss to run as a service

## cp /opt/jboss-as-7.1.1.Final/bin/init.d/jboss-as-standalone.sh /etc/init.d/jboss
## chmod +x /etc/init.d/jboss
## mkdir /etc/jboss-as

add the following configuration parameters

## vim /etc/jboss-as/jboss-as.conf

JBOSS_HOME=/opt/jboss-as-7.1.1.Final
JBOSS_CONSOLE_LOG=/var/log/jboss-console.log
JBOSS_USER=jboss

start and add the JBoss server to your CentOS VPS system start-up services using the following commands:

## service jboss start
## chkconfig jboss on
Alternatively, any directory can be chosen for the JBoss 7 installation.
Step 3: Create the appropriate user

Now that JBoss AS 7, is installed, we need to make sure that we create a user with the appropriate privileges. It is never a good idea to run JBoss as root for various reasons.
Create the new user:

We create a new user called jboss by issuing the following command:

n order to use the administration console you need to create new JBoss users using the add-user.sh script in the bin/ directory within your JBoss installation (which in this case is /opt/jboss-as-7.1.1.Final/bin).

example:

## cd /opt/jboss-as-7.1.1.Final/bin
## ./add-user.sh

What type of user do you wish to add?
 a) Management User (mgmt-users.properties)
 b) Application User (application-users.properties)
(a): a

Enter the details of the new user to add.
Realm (ManagementRealm) :
Username : testuser
Password :
Re-enter Password :
About to add user 'testuser' for realm 'ManagementRealm'
Is this correct yes/no? y
Added user 'testuser' to file '/opt/jboss-as-7.1.1.Final/standalone/configuration/mgmt-users.properties'
Added user 'testuser' to file '/opt/jboss-as-7.1.1.Final/domain/configuration/mgmt-users.properties'

You can now access your JBoss administration console at http://YOUR_IP:8080/console using the newly created testuser account.

Reference

Install Jboss Enterprise Application Platform on  RHEL 7.X
https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.0/html-single/installation_guide/

Installing and Building Jboss Server
https://docs.jboss.org/jbossas/jboss4guide/r2/html/ch01.html

How To Install Jboss 7.X ON Cent OS
http://www.opensourcearchitect.co/resources/tutorials/installing-jboss-7-1-on-centos-6

Jboss Binaries Files
http://sourceforge.net/projects/jboss

The JBoss directory structure
https://docs.jboss.org/jbossas/guides/installguide/r1/en/html/dirs.html

Video Tutorials

How To Install Jboss on RHEL 7

How Manage Operations, Adminstrations, Management of JBOSS AS7