What is load balancing?
Load balancing is the practice of distributing a workload across multiple computers for improved performance. Load balancing distributes work among resources in such a way that no one resource should be overloaded and each resource can have improved performance, depending on the load balancing algorithm. Items such as network traffic, SSL requests, database queries, or even hardware resources such as memory can be load balanced. This practice is commonly used in server farms where multiple physical boxes are coordinated to fulfill the requests of many end users.
Archive for the ‘Infrastructure’ category
Loadbalancing and its benefits
March 17th, 2010Ubuntu Live Network Boot using PXE
February 16th, 2010Requirements
- Linux server with NFS (or compatible)
- TFTP server
- DHCP server
- syslinux / pxelinux files
To simplify these instructions we are going to make the following assumptions.
- DHCP server is 10.0.0.2
- TFTP server is 10.0.0.3
- NFS is a Ubuntu server at 10.0.0.4
In reality it’s likely your TFTP and NFS server are going to be the same server, however because we go by IP in this, it is hopefully easier to understand.
» Read more: Ubuntu Live Network Boot using PXE
Creating Services using SMF in OpenSolaris
January 22nd, 2010OpenSolaris has by far, one of the best service management interfaces that I have used. Below I am going to go over a simple way to turn in shell script into a service managed by the OS.
» Read more: Creating Services using SMF in OpenSolaris
Building Perl modules on OpenSolaris
January 11th, 2010Anyone that’s worked with Perl is probably familiar with CPAN.pm. CPAN.pm is the bundled module that handles downloading and installing modules from the CPAN repository. It usually works flawlessly but I’ve noticed that on OpenSolaris the process can be a bit more spotty.
» Read more: Building Perl modules on OpenSolaris
Who Bound Port 8080
January 6th, 2010“Port 8080 required by Tomcat v6.0 Server at localhost is already in use. The server may already be running in another process, or a system process may be using the port. To start this server you will need to stop the other process or change the port number(s).”
I have been seeing this error from Tomcat every time I reboot my Windows development box.
» Read more: Who Bound Port 8080
Unplugging an LVM partitioned USB drive
December 30th, 2009Recently I had the heartbreaking experience of having to reboot a Linux server. Normal usage should almost never require you to reboot the OS like you have to so frequently in Windows. In this case I had an external USB drive partitioned with LVM humming along on a Linux server. I needed to pull the drive, so like I’ve done with other drives I unmounted all partitions on the drive. Then proceeded to unplug it from the USB port. All well and good. But when I plugged it back in, the lvs command was showing error messages on the partitions and I was unable to mount them.
Some Google searches later I found that when it comes to LVM partitions the OS keeps references to it unless you explicitly tell it to unhook them. Only then can you tell the OS to hook the LVM partitions back up when you’ve plugged the drive back in. In my case I had to resort to rebooting the server in order for the OS to hook all the pieces together for the LVM partitions. Short of this I would have to manually delete certain files and move things around to get the LVM partitions to work again. So here are the magic incantations that will save you the headache.
Before you unplug an LVM partitioned USB drive, you must run the following commands:
#!/bin/bash lvchange -an /dev/your_volume_group_name vgexport -a
Use the man command to explore what these commands do.
Now you should be able to unplug the drive. When you are ready to plug it back in, stick it back in the USB port and run the following commands:
#!/bin/bash vgimport -a lvchange -ay /dev/your_volume_group_name
You should now be able to run lvs and see you LVM partitions on the USB drive without any errors and proceed to mount the partitions.
Hope you found this useful. Are there other or different ways of doing this? Please add your comments below and Happy Holidays!
Apache/Tomcat with Failover and Load Balancing in 20 minutes or less…
December 28th, 2009In order to get this done, you’ll need Apache, Tomcat, and the MOD_JK connector library. I’ve included the Windows binaries below, as I’m writing this blog post from a Windows machine. Feel free to swap out these downloads with whatever RPM, YAST, APT, etc. commands you want. Also, you’ll need to have a Java JDK installed, and your JAVA_HOME pointing to it. Make sure your path contains %JAVA_HOME%/bin.
Download and Apache from here:
http://apache.mirrors.tds.net/httpd/binaries/win32/apache_2.2.14-win32-x86-no_ssl.msi
Download and extract Tomcat from here:
http://mirrors.axint.net/apache/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip
Download MOD_JK from here:
http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/win32/jk-1.2.28/mod_jk-1.2.28-httpd-2.2.3.so
- Install Apache.
- Copy the mod_jk-1.2.28-httpd-2.2.3.so file to your apache/modules directory.
-
Add the following to your apache/conf/httpd.conf
LoadModule jk_module modules/mod_jk-1.2.28-httpd-2.2.3.so JkWorkersFile conf/workers.properties JkLogFile logs/jk.log JkLogLevel debug JkMount /* router JkMount /jk_status status -
Create a workers.properties file in your apache/conf directory. The file should contain the following:
worker.list=router,status worker.worker1.port=8109 worker.worker1.host=localhost worker.worker1.type=ajp13 worker.worker1.lbfactor=1 worker.worker1.local_worker=1 worker.worker1.sticky_session=0 worker.worker2.port=8209 worker.worker2.host=localhost worker.worker2.type=ajp13 worker.worker2.lbfactor=1 worker.worker2.local_worker=0 worker.worker2.sticky_session=0 worker.worker3.port=8309 worker.worker3.host=localhost worker.worker3.type=ajp13 worker.worker3.lbfactor=1 worker.worker3.local_worker=0 worker.worker3.sticky_session=0 worker.router.type=lb worker.router.balanced_workers=worker1,worker2,worker3 worker.router.local_worker_only=1 worker.status.type=status
-
Extract the Tomcat installation ZIP archive to three different directories, as we’re going to load balance three instances of Tomcat. You’ll be replacing the server.xml file in each of the Tomcat conf/ directories with the following:
<Server port="8100" shutdown="SHUTDOWN"> <GlobalNamingResources> <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" /> </GlobalNamingResources> <Service name="Catalina"> <Connector port="8180" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" /> <!-- Define an AJP 1.3 Connector --> <Connector port="8109" protocol="AJP/1.3" redirectPort="8443" /> <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1"> <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/> <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false"> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="228.0.0.4" port="45564" frequency="500" dropTime="3000"/> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/> </Sender> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/> <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer" tempDir="/tmp/war-temp/" deployDir="/tmp/war-deploy/" watchDir="/tmp/war-listen/" watchEnabled="false"/> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/> </Cluster> </Host> </Engine> </Service> </Server>
Looking into the XML above, there are three ports we’re concerned with: 8100, 8180, and 8109. These are the server, HTTP, and AJP13 ports, respectively. We’ll need each instance of Tomcat to run on it’s own ports. So, you can use this file as is in your first folder containing Tomcat, however, you’ll need to change the port numbers to: 8200, 8280, and 8209 for you 2nd installation. The third installation will use the ports, 8300, 8380, and 8309.
- Start (or Restart) Apache
- Start each instance of Tomcat (use the startup script in the Tomcat /bin directory) – you should see no errors.
- Verify each Tomcat is working by opening a browser window to each Tomcat instance – if you’ve followed my instructions, the links are: http://localhost:8180/examples/servlets/, http://localhost:8280/examples/servlets/, and http://localhost:8380/examples/servlets/.
- If Tomcat started correctly, start Apache. You should be able to access the Tomcat example pages via the following URL: http://localhost/examples/servlets/
- You’re done. Using my configuration, you can access a page to control the JK connector here: http://localhost/jk_status, I’d recommending hiding and protecting this should you want to put this configuration into production. Try experimenting with the configuration by stopping instance of Tomcat… as long as one instance of Tomcat is running, you should be able to see the examples.
-Jim
GlusterFS Replication for Clustering
December 23rd, 2009I recently was searching for a way to simulate shared physical storage in a VPS environment for clustering purposes. In an enterprise data center we can expect some type of SAN available to provide shared physical storage. GFS is a simple solution in this case to create a shared file system that can be used as a resource in a cluster. GlusterFS allows us to provide this type of functionality to multiple nodes when we have no means of providing access to the same physical storage.
The gluster community site will be a great resource for anyone wanting to implement the file system and is located at http://www.gluster.org.
For the remainder of this post I will be referring to an environment consisting of two CentOS VPS nodes.
Preparing Ext3 File System for Sharing
Gluster will not share raw devices but instead will use an already mounted file system. I will be assuming the use of a complete ext3 file system on the mount point /replicator. If you can’t provide a unique storage device for this purpose you can just use a directory on the root file system for testing.
Installing GlusterFS Server and Client
The following commands need to be executed on each node to grab and install the necessary RPMs.
wget -r -l 1 http://ftp.gluster.com/pub/gluster/glusterfs/3.0/3.0.0/CentOS/ cd ftp.gluster.com/pub/gluster/glusterfs/3.0/3.0.0/CentOS/ rpm -Uvh glusterfs-*-3.0.0-1.x86_64.rpm
Execute the following on either node to generate the necessary configuration files in the current working directory. This will create a client configuration along the lines of replicator-tcp.vol. A server configuration file will be created for each node and begin with the appropriate node hostname.
glusterfs-volgen --name replicator --raid 1 node1:/replicator node2:/replicator
Move the client file to /etc/glusterfs/glusterfs.vol on each node. Also move the appropriate server file to /etc/glusterfs/glusterfsd.vol for each node.
Mounting GlusterFS Volumes
The simplest way to configure mounting of the volumes is via /etc/fstab. Place a line in fstab on each node.
/etc/glusterfs/glusterfs.vol /data glusterfs defaults 0 0
This will mount the shared volumes to /data. Try writing a file to one node and watch it appear on the other!
cd /data dd if=/dev/zero of=/data/test bs=1M count=32
High Availability Implications
At this point I am still vetting gluster’s reliability as a HA solution. It will most definitely keep data intact during planned maintenance. If we properly stop the client/server on any node then changes can continue to occur on the other. Also we can join a node to active shared storage and synchronization is automatic.
The real test is whether gluster will hold up in the not so routine situations. Some crude tests involving yanking network connectivity from a node that is replicating changes seems to cause some issues. For example, if I start the dd operation above on node1 and kill the connection to node2, one way or another, before it finishes then node1 still completes the operation fine. When I reattach node2 even the active mount on /data seems to synchronize with node1 just fine. Where some differences start to appear is in the /replicator directory on node2. It seems that this does get out of whack and neither client pays attention to this server any longer.
If gluster can hold up to software and hardware failures without data corruption it can certainly be used as shared storage for clustering. I’ll continue to explore these options and report back later.
Online malware scanner
December 14th, 2009F-secure has the best antivirus/anti-malware software available. I have been using it since rootkits became a threat. F-Secure had a free app that would detect and remove rootkits which is a step above technet’s Rootkit Revealer. F-Secure licensing is inexpensive and user friendly. They also have an online scanner that detects and removes threats and also will send samples back to the vendor.
You can find the online scanner here
