Infinispan cluster setup between the Servers:

Infinispan Concept:

Infinispan Server is a standalone server which exposes any number of caches to clients over a variety of protocols, including HotRod, Memcached and REST. The server itself is built on top of the robust foundation provided by WildFly, therefore delegating services such as management, configuration, data sources, transactions, logging, security to the respective subsystems.

Objective: We are going to implement infinispan cluster setup between all 4 servers.

Prerequisites:

RHEL 6.x or 7 version must be installed.

Infinispan Server 9.1 must be set up for all the servers.

The database must be installed on the server (in this case we have used Cassandra DB and the DBS. are in the cluster which shares the same IP)

Modifications that needs to be done:

REQUIREMENT 1:

1.  Setup Infinispan server cluster between the nodes

• Following changes needs to be done in the file "clustered.xml" file of infinispan server for all the servers.

xx.xx.xx.xx, xx.xx.xx.xx, xx.xx.xx.xx and xx.xx.xx.xx

• Add the initial host tag under the protocol type TCP:

<stack name="tcp">

<transport type="TCP" socket-binding="jgroups-tcp"/>

<protocol type="TCPPING">

<property name="initial_hosts">

 xx.xx.xx.xx[7600],xx.xx.xx.xx[7600],xx.xx.xx.xx[7600],xx.xx.xx.xx[7600]

</property>

<property name="num_initial_members">

2

</property>

<property name="port_range">

0

</property>

<property name="timeout">

2000

</property>

</protocol>

Note: 7600 port is used to communicate tcp sync between the nodes. By default, port is 7600

Let’s discuss some points which are defined in the above tags:

The TCPPING protocol takes a set of known members and ping them for discovery.

Timeout: specifies the maximum number of milliseconds to wait for any responses. The default is 3000.

num_initial_members specify the maximum number of responses to wait for unless timeout has expired. The default is 2

initial_hosts are a comma-separated list of addresses for pinging.

•  change the value of default-stack from udp to tcp under the section <subsystem xmlns>

<subsystem xmlns="urn:infinispan:server:jgroups:7.0" default-stack="${jboss.default.jgroups.stack:tcp}">

• Changed the replicated-cache mode from ASYNC to SYNC as below under the section <replicated-cache name>

<replicated-cache name="uecmsConfigCache" mode="SYNC" start="EAGER" batching="true">

• Change the interface name i.e. management IP and public IP with the node ip as below

Changes on all the servers:

<interface name="management">

<inet-address value="${jboss.bind.address.management:”node_ip”}"/>

</interface>

<interface name="public">

<inet-address value="${jboss.bind.address:”node_ip”}"/>

</interface>

Note: Don’t use loopback IP’s for any of the servers. i.e. 127.0.0.1

Clustered.xml file is below for the reference:

2. Add the 4 IPs (xx.xx.xx.xx, xx.xx.xx.xx, xx.xx.xx.xx, xx.xx.xx.xx) with hotrod port “11222” as below in the hotrod-client.properties file.

infinispan.client.hotrod.server_list=xx.xx.xx.xx:11222;xx.xx.xx.xx:11222,xx.xx.xx.xx:11222,xx.xx.xx.xx:11222

infinispan.client.hotrod.socket_timeout=600

infinispan.client.hotrod.connect_timeout=20

infinispan.client.hotrod.ping_on_startup=true

infinispan.client.hotrod.max_retries=3

Follow the same steps as mentioned above to setup infinispan cluster for the rest of the nodes

REQUIREMENT 2:

To implement JBoss service with cache and without cache for all the 4 servers

With cache modified value will get populated and without cache modified value will not be populated

To further implement this done the following changes:

1. On Code level: if uecms.cache.load is true then the value will get populated otherwise not.
2. On Script level which calls jboss_cache.service: Defined the parameter as “-Duecms.cache.load=true”

ExecStart=/opt/uecms/jboss/bin/standalone.sh -Duecms.cache.load=true

Note: There are two scripts which calls the jboss, jboss_cache.service (which has the parameter -Duecms.cache.load=true) and jboss.service(without the parameter)

Path: /etc/system/system/jboss_cache.service

 / etc/system/system/jboss.service

REQUIREMENT 3:

If any modification in DB/config is done for any of the 4 servers, then the modified value should be reflecting on the other nodes.

REQUIREMENT 4:

One infinispan is stopped on any of the 4 servers and running on the other nodes then changes will reflect on all the 4 servers with the start of JBoss.

REQUIREMENT 5:

The value should be populated without restarting any of the application i.e. JBoss or infinispan

1. Change in the db and reload the cache with the API of reloadDb
2. Change in the config file and reload the cache with the API of reloadFiles

Attached is the test cases which we have gone through to get the above-mentioned requirements. You can refer to your servers.

Link

[ratings]