Archive for December, 2009

Unplugging an LVM partitioned USB drive

December 30th, 2009

Recently I had the heartbreaking experience of having to reboot a Linux server. Normal usage should almost never require you to reboot the OS like you have to so frequently in Windows. In this case I had an external USB drive partitioned with LVM humming along on a Linux server. I needed to pull the drive, so like I’ve done with other drives I unmounted all partitions on the drive. Then proceeded to unplug it from the USB port. All well and good. But when I plugged it back in, the lvs command was showing error messages on the partitions and I was unable to mount them.

Some Google searches later I found that when it comes to LVM partitions the OS keeps references to it unless you explicitly tell it to unhook them. Only then can you tell the OS to hook the LVM partitions back up when you’ve plugged the drive back in. In my case I had to resort to rebooting the server in order for the OS to hook all the pieces together for the LVM partitions. Short of this I would have to manually delete certain files and move things around to get the LVM partitions to work again. So here are the magic incantations that will save you the headache.

Before you unplug an LVM partitioned USB drive, you must run the following commands:

#!/bin/bash
lvchange -an /dev/your_volume_group_name
vgexport -a

Use the man command to explore what these commands do.

Now you should be able to unplug the drive. When you are ready to plug it back in, stick it back in the USB port and run the following commands:

#!/bin/bash
vgimport -a
lvchange -ay /dev/your_volume_group_name

You should now be able to run lvs and see you LVM partitions on the USB drive without any errors and proceed to mount the partitions.

Hope you found this useful. Are there other or different ways of doing this? Please add your comments below and Happy Holidays!

Apache/Tomcat with Failover and Load Balancing in 20 minutes or less…

December 28th, 2009

In order to get this done, you’ll need Apache, Tomcat, and the MOD_JK connector library. I’ve included the Windows binaries below, as I’m writing this blog post from a Windows machine. Feel free to swap out these downloads with whatever RPM, YAST, APT, etc. commands you want. Also, you’ll need to have a Java JDK installed, and your JAVA_HOME pointing to it. Make sure your path contains %JAVA_HOME%/bin.

Download and Apache from here:
http://apache.mirrors.tds.net/httpd/binaries/win32/apache_2.2.14-win32-x86-no_ssl.msi

Download and extract Tomcat from here:
http://mirrors.axint.net/apache/tomcat/tomcat-6/v6.0.20/bin/apache-tomcat-6.0.20.zip

Download MOD_JK from here:
http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/win32/jk-1.2.28/mod_jk-1.2.28-httpd-2.2.3.so

  1. Install Apache.
  2. Copy the mod_jk-1.2.28-httpd-2.2.3.so file to your apache/modules directory.
  3. Add the following to your apache/conf/httpd.conf

    LoadModule jk_module modules/mod_jk-1.2.28-httpd-2.2.3.so
     
    JkWorkersFile          conf/workers.properties
    JkLogFile              logs/jk.log
    JkLogLevel             debug
     
    JkMount                /*            router
    JkMount                /jk_status    status
  4. Create a workers.properties file in your apache/conf directory. The file should contain the following:

    worker.list=router,status
     
    worker.worker1.port=8109
    worker.worker1.host=localhost
    worker.worker1.type=ajp13
    worker.worker1.lbfactor=1
    worker.worker1.local_worker=1
    worker.worker1.sticky_session=0
     
    worker.worker2.port=8209
    worker.worker2.host=localhost
    worker.worker2.type=ajp13
    worker.worker2.lbfactor=1
    worker.worker2.local_worker=0
    worker.worker2.sticky_session=0
     
    worker.worker3.port=8309
    worker.worker3.host=localhost
    worker.worker3.type=ajp13
    worker.worker3.lbfactor=1
    worker.worker3.local_worker=0
    worker.worker3.sticky_session=0
     
    worker.router.type=lb
    worker.router.balanced_workers=worker1,worker2,worker3
    worker.router.local_worker_only=1
     
    worker.status.type=status
  5. Extract the Tomcat installation ZIP archive to three different directories, as we’re going to load balance three instances of Tomcat. You’ll be replacing the server.xml file in each of the Tomcat conf/ directories with the following:

    <Server port="8100" shutdown="SHUTDOWN">
     
    <GlobalNamingResources>
       <Resource name="UserDatabase" auth="Container"
          type="org.apache.catalina.UserDatabase"
          description="User database that can be updated and saved"
          factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
          pathname="conf/tomcat-users.xml" />
    </GlobalNamingResources>
     
    <Service name="Catalina">
       <Connector port="8180" protocol="HTTP/1.1"
          connectionTimeout="20000"
          redirectPort="8443" />
     
       <!-- Define an AJP 1.3 Connector -->
       <Connector port="8109" protocol="AJP/1.3" redirectPort="8443" />
     
       <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
          <Realm className="org.apache.catalina.realm.UserDatabaseRealm" 
             resourceName="UserDatabase"/>
     
          <Host name="localhost"  appBase="webapps"
             unpackWARs="true" autoDeploy="true"
             xmlValidation="false" xmlNamespaceAware="false">
     
             <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                channelSendOptions="8">
     
                <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
     
                <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                   <Membership className="org.apache.catalina.tribes.membership.McastService"
                      address="228.0.0.4"
                      port="45564"
                      frequency="500"
                      dropTime="3000"/>
     
                   <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>
     
                   <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                      <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                   </Sender>
     
                   <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                   <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
                </Channel>
     
                <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
     
                <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                   tempDir="/tmp/war-temp/"
                   deployDir="/tmp/war-deploy/"
                   watchDir="/tmp/war-listen/"
                   watchEnabled="false"/>
     
                <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
                <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
             </Cluster>
          </Host>
       </Engine>
    </Service>
    </Server>

    Looking into the XML above, there are three ports we’re concerned with: 8100, 8180, and 8109. These are the server, HTTP, and AJP13 ports, respectively. We’ll need each instance of Tomcat to run on it’s own ports. So, you can use this file as is in your first folder containing Tomcat, however, you’ll need to change the port numbers to: 8200, 8280, and 8209 for you 2nd installation. The third installation will use the ports, 8300, 8380, and 8309.

  6. Start (or Restart) Apache
  7. Start each instance of Tomcat (use the startup script in the Tomcat /bin directory) – you should see no errors.
  8. Verify each Tomcat is working by opening a browser window to each Tomcat instance – if you’ve followed my instructions, the links are: http://localhost:8180/examples/servlets/, http://localhost:8280/examples/servlets/, and http://localhost:8380/examples/servlets/.
  9. If Tomcat started correctly, start Apache. You should be able to access the Tomcat example pages via the following URL: http://localhost/examples/servlets/
  10. You’re done. Using my configuration, you can access a page to control the JK connector here: http://localhost/jk_status, I’d recommending hiding and protecting this should you want to put this configuration into production. Try experimenting with the configuration by stopping instance of Tomcat… as long as one instance of Tomcat is running, you should be able to see the examples.

-Jim

GlusterFS Replication for Clustering

December 23rd, 2009

I recently was searching for a way to simulate shared physical storage in a VPS environment for clustering purposes.  In an enterprise data center we can expect some type of SAN available to provide shared physical storage.  GFS is a simple solution in this case to create a shared file system that can be used as a resource in a cluster.  GlusterFS allows us to provide this type of functionality to multiple nodes when we have no means of providing access to the same physical storage.

The gluster community site will be a great resource for anyone wanting to implement the file system and is located at http://www.gluster.org.

For the remainder of this post I will be referring to an environment consisting of two CentOS VPS nodes.

Preparing Ext3 File System for Sharing

Gluster will not share raw devices but instead will use an already mounted file system.  I will be assuming the use of a complete ext3 file system on the mount point /replicator.  If you can’t provide a unique storage device for this purpose you can just use a directory on the root file system for testing.

Installing GlusterFS Server and Client

The following commands need to be executed on each node to grab and install the necessary RPMs.

wget -r -l 1 http://ftp.gluster.com/pub/gluster/glusterfs/3.0/3.0.0/CentOS/
cd ftp.gluster.com/pub/gluster/glusterfs/3.0/3.0.0/CentOS/
rpm -Uvh glusterfs-*-3.0.0-1.x86_64.rpm

Execute the following on either node to generate the necessary configuration files in the current working directory.  This will create a client configuration along the lines of replicator-tcp.vol.  A server configuration file will be created for each node and begin with the appropriate node hostname.

glusterfs-volgen --name replicator --raid 1 node1:/replicator node2:/replicator

Move the client file to /etc/glusterfs/glusterfs.vol on each node.  Also move the appropriate server file to /etc/glusterfs/glusterfsd.vol for each node.

Mounting GlusterFS Volumes

The simplest way to configure mounting of the volumes is via /etc/fstab.  Place a line in fstab on each node.

/etc/glusterfs/glusterfs.vol    /data   glusterfs   defaults  0 0

This will mount the shared volumes to /data.  Try writing a file to one node and watch it appear on the other!

cd /data
dd if=/dev/zero of=/data/test bs=1M count=32

High Availability Implications

At this point I am still vetting gluster’s reliability as a HA solution.  It will most definitely keep data intact during planned maintenance.  If we properly stop the client/server on any node then changes can continue to occur on the other.  Also we can join a node to active shared storage and synchronization is automatic.

The real test is whether gluster will hold up in the not so routine situations.  Some crude tests involving yanking network connectivity from a node that is replicating changes seems to cause some issues.  For example, if I start the dd operation above on node1 and kill the connection to node2, one way or another, before it finishes then node1 still completes the operation fine.  When I reattach node2 even the active mount on /data seems to synchronize with node1 just fine.  Where some differences start to appear is in the /replicator directory on node2.  It seems that this does get out of whack and neither client pays attention to this server any longer.

If gluster can hold up to software and hardware failures without data corruption it can certainly be used as shared storage for clustering.  I’ll continue to explore these options and report back later.

Who is the client?

December 21st, 2009

A while back, I read blog post discussing who is the client. Software projects frequently have many different clients, many of whom are frequently underrepresented throughout the development process. Do you know who all of the clients of your application are? What can you do to make their lives easier or better? You might be excited that you’re project is converting an old green-screen application to a web application, but have you thought about the data entry staff? How long did it take for them to enter a widget through the old interface? How about with the new one? How about the people who consume the data that your application will be collecting? Will they be able to access everything they need? Don’t forget the IT staff. Will they be able to support your application?

When developing software, you need to be aware of all of the different customers you have. Even though they might not be represented in the meetings, whether or not your software satisfies their needs will determine your project’s success.

Signal To Noise

December 18th, 2009

The other evening I attended a technology industry event at a somewhat-trendy bar downtown.  The event was intended to foster networking between newer entrepreneurial startups and more established tech companies.  I eventually found myself comparing notes with a guy I’ll call “Sam.”  Sam is responsible for sales at his company and as with most sales professionals; the conversation with Sam wasn’t too difficult.  He clearly wasn’t trying to confuse anyone.  As I watched Sam interact with others I began to suspect that he was an expert at getting his message across with an optimal signal-to-noise ratio.

Shortly after Sam and I began speaking we were joined by another individual.  I’ll call this guy “Ted”.  We made introductions around and then I asked Ted what his company did.  Ted seemed to struggle with his description of his business.  After Ted circled the bulls eye for several minutes Sam asked him to boil his business model down to the basic value proposition. Ted seemed to struggle with this too and so Sam helped him through the short conversation with some prompting.

I found the conversation about a basic value statement interesting.  Once we’d arrived at Ted’s business the conversation became less interesting.  Ted began expounding the benefits of his favorite flavor of technology.  Before I wandered off I heard things like “no viruses”, “lower cost of ownership” and “a true Unix operating system…”.  Not much real information and not much in a message structure that I found entertaining.

Here are a few quick tips for the “Teds” out there:

  1. Always be ready to quickly describe your value proposition. You should practice that statement regardless of your position in the company.
  2. Keep the value proposition short and relatively non-technical. It doesn’t need to be so simple that Uncle Joe could understand it, unless Uncle Joe works in your industry, but it does need to be basic enough that others in your industry understand quickly, with minimum effort.
  3. If someone asks you what you do and you are able to respond with a concise answer then by all means also give them your contact information!  Hand them your business card; provide them with a URL or something else memorable.  Write that information on a matchbook, cocktail napkin or just scratch it into their forearm with your car keys.  Don’t assume that someone will remember your business contact information after a casual conversation, especially if that causal environment also includes libations.
  4. Read your audience.  Be cautious about providing too much detail unless you are confident the person you are speaking to understands your topic and wants to dive into it.  There are some people I just don’t get into elevators with.  Zealots are always near the top of that list.

Asking the hard questions

December 16th, 2009

There was a post a while back on TechRepublic about how leaders ask questions. When was the last time you asked yourself what could go wrong? What are you doing to prevent it, or minimize it’s damage?

I just got done reading Release It by Michael Nygard. I don’t remember the exact numbers he used, but he made the point that any system of sufficient size should expect to experience more than one “once in a million” situations. Assume that your system performs 1000 transactions a day, 365 days a year; after three years you will have processed over a million transactions. I don’t know about you, but saying something will happen about once every three years sounds a lot different than once in a million. So next time you’re in the car, waiting for traffic, start thinking about “what if.”

Online malware scanner

December 14th, 2009

F-secure has the best antivirus/anti-malware software available. I have been using it since rootkits became a threat. F-Secure had a free app that would detect and remove rootkits which is a step above technet’s Rootkit Revealer. F-Secure licensing is inexpensive and user friendly. They also have an online scanner that detects and removes threats and also will send samples back to the vendor.

You can find the online scanner here

ColdFusion for Rapid Application Development

December 14th, 2009

I’m helping to write a tool to edit simple database information in a browser. ColdFusion is meant for rapid application development and is perfect for this or any small scale data project because it’s very easy to quickly make pages and is designed for use with a database.

One of the reasons ColdFusion is so easy to pick up and run with is that its style is very similar to html with its own opening and ending tags(this probably makes it easier to learn than ASP or PHP). In fact, ColdFusion is commonly integrated with html directly meaning you will see a lot of html tags in coldfusion (.cfm) pages. The ColdFusion tags are denoted by the “cf” they include, for example:

<CFFORM name=“HelloWorld”>
...
</CFFORM >

Using the <CFQUERY> tag you can write your SQL directly into the page and display the results by using the <CFOUTPUT> tag and surrounding your variables with the ‘#’ symbol any time you desire to reference them.

<CFQUERY NAME=“findUser” DATASOURCE=“userDatabase”>
   SELECT *
   FROM userTable
   WHERE userName = ‘#Attributes.name#</CFQUERY>
 
<CFIF findUser.recordcount EQ 0>
   <textarea>
      User Name <CFOUTPUT>#findUser.userName#</CFOUTPUT> was not found
   </textarea>
</CFIF>

ColdFusion is now owned by Adobe Software who released ColdFusion 9 in October. ColdFusion 9 has integration with Adobe flash as well as Microsoft Office. It’s compatible with all databases. If you are new to scripting and markup languages and ColdFusion is available to you it is an excellent way to start out and learn.

Spring Roo Sample App Tutorial – Part 1

December 13th, 2009

In this blog, I will start creating a web application used to organize bookmarks. Because only certain bookmarks are of interest to specific groups of people, I will use groups in our LDAP server to control which users see which groups of bookmarks.

The entire blog will be released in posts staggered over time. Part 1 will focus on initial setup of Roo, the core web application and authentication with a directory server. Subsequent posts will refine the Spring Roo application.

What is Roo?

It’s a great rapid prototyping tool because prototypes don’t need to be scrapped to proceed with fleshing out the application if a prototype proves itself.

Roo gives you Spring best practices, Rails-like scaffolding, an interactive shell, no additional run-time dependencies, and a big productivity boost while not locking you into yet another framework. You can re-use your existing Spring/JPA/Hibernate knowledge, while getting the productivity gains from Roo.

Setting up Roo

  • wget http://s3.amazonaws.com/dist.springframework.org/milestone/ROO/spring-roo-1.0.0.RC3.zip
  • unzip spring-roo-1.0.0.RC3.zip
  • sudo ln -s ~/Frameworks/spring-roo-1.0.0.RC3/bin/roo.sh /usr/bin/roo
  • mkdir ~/Workspaces/intranetlinks; cd ~/Workspaces/intranetlinks

Starting our Project

Once in your new project directory, type ‘roo’. Then once in the Roo shell, execute these commands. See this guide for an explanation of what these commands do:

project --topLevelPackage com.sourceallies.links
persistence setup --provider HIBERNATE --database MYSQL
database properties set --key database.password --value password
database properties set --key database.username --value username
database properties set --key database.url --value jdbc:mysql://localhost:3306/intranetlinks
 
entity --name ~.domain.LinkCategory
field string name --notNull --sizeMin 1 --sizeMax 255
 
entity --name ~.domain.Link
field string name --notNull --sizeMin 1 --sizeMax 60
field string url --notNull --sizeMin 1 --sizeMax 255
field string ldapSecurityGroup --notNull --sizeMin 1 --sizeMax 60
field reference --class ~.domain.Link --fieldName category --type ~.domain.LinkCategory
 
logging setup --level DEBUG
 
controller scaffold --name ~.web.LinkCategoryController --entity ~.domain.LinkCategory
controller scaffold --name ~.web.LinkController --entity ~.domain.Link
 
finder list --class com.sourceallies.links.domain.Link
finder add --finderName findLinksByCategory --class ~.domain.Link
 
security setup
test integration
perform test
perform eclipse

Then of course, create your local database inside the MySQL shell:

create database intranetlinks;
create user 'username'@'localhost' IDENTIFIED BY 'password';
grant all privileges on intranetlinks.* to 'username'@'localhost' with grant option;

Next, unless you’re using Roo 1.0.0.RC4 (not available at the time of this blog post), you’ll need to add the following config to near the bottom of your pom.xml (to fix this bug).

<profiles>
       <profile>
 <id>jaxb</id>
       <activation>
            <jdk>1.5</jdk>
        </activation>
        <dependencies>
               <dependency>
                    <groupId>javax.xml.bind</groupId>
                    <artifactId>jaxb-api</artifactId>
                    <version>2.1</version>
                </dependency>
               <dependency>
                    <groupId>com.sun.xml.bind</groupId>
                    <artifactId>jaxb-impl</artifactId>
                    <version>2.1.3</version>
                </dependency>
         </dependencies>
  </profile>
</profiles>

Then pull the JAXB JAR into your build by executing this maven command (outside of the Roo shell):

mvn package clean

Finally, per a prior blog, replace the body of your src/main/resources/META-INF/spring/applicationContext-security.xml with this:

    <http>
    	<form-login login-processing-url="/static/j_spring_security_check" login-page="/login" authentication-failure-url="/login?login_error=t"/>
        <logout logout-url="/static/j_spring_security_logout"/>
        <intercept-url pattern="/admin/**" access="ROLE_ADMIN"/>
        <intercept-url pattern="/member/**" access="IS_AUTHENTICATED_REMEMBERED" />
        <intercept-url pattern="/resources/**" access="IS_AUTHENTICATED_ANONYMOUSLY" />
        <intercept-url pattern="/static/**" access="IS_AUTHENTICATED_ANONYMOUSLY" />
        <intercept-url pattern="/images/**" filters="none" />
        <intercept-url pattern="/styles/**" filters="none" />
	<intercept-url pattern="/link/form" access="ROLE_INTRANETLINKS-ADMINS" />
	<!-- We're doing REST, only allow GETs to normal users -->
    	<intercept-url pattern="/link/**" access="ROLE_INTRANETLINKS-ADMINS" method="DELETE"/>
    	<intercept-url pattern="/link/**" access="ROLE_INTRANETLINKS-ADMINS" method="POST"/>
    	<intercept-url pattern="/link/**" access="ROLE_INTRANETLINKS-ADMINS" method="PUT"/>
        <intercept-url pattern="/link/**" access="IS_AUTHENTICATED_REMEMBERED" />
        <intercept-url pattern="/login/**" filters="none" />
	<intercept-url pattern="/**" access="ROLE_USERS"  />
	 <anonymous /> 
    </http>
 
    <ldap-server id="ldapServer" url="ldap://yourdirectoryserver:338899/" />
 
   <authentication-manager>
    <ldap-authentication-provider server-ref="ldapServer"  
       user-search-base="ou=people,dc=sourceallies,dc=com" 
       user-search-filter="(uid={0})"
       group-role-attribute="cn"
       group-search-base="ou=groups,dc=sourceallies,dc=com"
       group-search-filter="(memberUid={1})"
       role-prefix="ROLE_" />
   </authentication-manager>

Note that in Spring Security 3.0, Authentication Providers must now be declared from within the authentication-manager element (more information here).

Then add a few more dependencies to your pom.xml

    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>org.springframework.security.ldap</artifactId>
        <version>3.0.0.RC1</version>
    </dependency>
 
    <dependency>
        <groupId>org.springframework.ldap</groupId>
        <artifactId>spring-ldap-core</artifactId>
        <version>1.3.0.RELEASE</version>
    </dependency>
 
    <dependency>
        <groupId>org.springframework.ldap</groupId>
        <artifactId>spring-ldap-core-tiger</artifactId>
        <version>1.3.0.RELEASE</version>
    </dependency>

This will allow you to use Spring LDAP and also conditionally render pieces of your application like this:

<security:authorize ifAllGranted="ROLE_SUPERVISOR">
    <li id="finder_findlinksbycategory_menu_item">
        <c:url value="/link/find/ByCategory/form" var="finder_findlinksbycategory_menu_item_url"/>
        <a href="${finder_findlinksbycategory_menu_item_url}">
            <spring:message arguments="Category" code="global.menu.find"/>
        </a>
    </li>
</security:authorize>

Finally, run the following command to startup Tomcat and start refining your UI.

mvn tomcat:run

Stay tuned for Part 2 of this series!

Continuous Learning and Career Development Goals

December 9th, 2009

Continuous learning is a critical puzzle piece to staying competitive in today’s business world.  In the IT world specifically, as we all know, the only constant is the fact that processes are changing and new processes are evolving all the time!  In order to keep up with the learning curve, we must make continuous learning a high priority.

At Source Allies, continuous learning takes place in many forms including:

  • Training – courses attended over the past year include: AFS & Kerberos Best Practices, No Fluff Just Stuff, Tuning and Improving your Agility and Spring One 2GX
  • Reading – Source Allies has an internal book club that meets on a weekly basis to review a specific book.  Books that have been and are currently being read and reviewed include TDD by Example and Effective Java, 2nd Edition. 
  • Weekly meetings – each Monday, after hours, the Source Allies team meets to discuss current projects and share techologies that are being used on these projects. 
  • Technical presentations – team members present at least yearly on the technologies they are involved closely with.
  • Internships
  • Mentoring

The examples above are just a small sample of the continuous learning opportunities available to our team at Source Allies.

Continuous learning should be an important part of your career development goals.  What do you want to learn?  How do you want to apply what you learn at your client site and to your projects?  During this time of year, many of us are reviewing our personal career goals.  In doing so, make sure you identify and include your learning goals… but don’t stop there!  Make sure you focus on these goals and review your progress often throughout the year.

You’ll notice that when you are focused on learning, it will not only grow your value but it also makes you more passionate about your projects and work in general.  It’s easy to get in a rut if you’re not focusing on staying ahead of the learning curve.

Specific knowledge and skills become obsolete with time but learning how to and having a passion for learning is a permanent skill that will carry you throughout your career and beyond!