« Archives in January, 2011

Manually override and launch quartz jobs…

Override quartz settings?

So you have a quartz job that’s chugging along nicely until you’re hit with the reality that the job details parameters change or the job needs to be suspended, or something happens that you end up having to recompile and redeploy your application just to update the packaged quartz job properties. This is no fun. You will undoubtedly have to take the updated code through the regular qa cycle, regress test, and then ultimately redeploy your code into the production environment. Surely there must be some way to address this problem when using Jboss…

One way I came up with was to divorce the job execution code from the job invocation, while making sure that the JobDetailsMap always checked an external resource before defaulting to loading the packaged resource within the deployed artifact. To allow for manual invocation, I also added a servlet that basically just wrapped the decoupled job invocation code in order to launch the quartz job. I also added a property to the JobDetailMap – “enable” which I used as a flag for whether the job should fire or not. Because it would try to load an external resource before defaulting, we were then able to have complete control over the quartz job’s properties. Note that you can’t change the cron fire date by using this method – the job itself is loaded in from the minute your application fires up – to reload the job you’d have to programatically find the existing job, destroy it and then create a new one based off the external properties. In my particular case we didn’t need to go that far but that option is available for those that need it.

The steps:

1) stick a copy of the quartz-config.xml file in the jboss.conf.dir location: maybe something like “/jboss/server/myInstance/conf/quartz-config.xml”. This conf directory is explored in depth in the related post Jboss System Properties.

2) Rig your quartz Job class so the execute(JobExecutionContext jobContext) method simply calls a plain launchJob() method. By doing this you end up separating the call that launches the job from the quartz specific entry point so any externally invoking code can call your launchJob() method directly without you having to figure out how to populate a JobExecutionContext object to pass into that execute() method:

public class QuartzJob implements Job {   
   
    public void execute(JobExecutionContext jobContext)   
        throws JobExecutionException {   
   
         log.info("launching regularly scheduled quartz job");   
        launch(); 
    }   
 
    public void launch() { 

         // your job code would go here

    }

}

3) Read in that quartz-config.xml file from the Jboss conf directory if one exists, and extract the properties from the xml file to populate your own JobDetailsMap object. Default it to read in the quartz-config.xml packaged in your war, jar or ear file:

public void launch() { 

  Document document = null; 
  SAXReader reader = new SAXReader(); 
  JobDataMap map = new JobDataMap(); 

  try { 

       // this section here extracts properties from the config file	    
       InputStream is = null; 
       String quartzConfig = "quartz-config.xml"; 


       try { 

	    String path = System.getProperty("jboss.server.config.url")
		 +quartzConfig;   
	    URL url = new URL(path); 

	    log.info("attempting to load " + quartzConfig + " file from: " + path); 
	    is = url.openStream(); 
	    log.info("loaded " + quartzConfig + " from URL: " + path);   

       } catch (Exception e) { 

	    is = this.getClass().getResourceAsStream("/" + quartzConfig); 
	    log.info("couldn't load " + quartzConfig + 
		 " from URL, loaded packaged from war: /"+quartzConfig); 

       } 

       document = reader.read(is); 

       String xPath =
            "/quartz/job/job-detail[name = 'myQuartzJob']/job-data-map/entry"; 
       List<Node> nodes = document.selectNodes(xPath); 
       for (Node node : nodes) { 
	    String key = ((Node) node.selectNodes("key").get(0)).getText(); 
	    String value = ((Node) node.selectNodes("value").get(0)).getText(); 

	    map.put(key, value); 
       }

    } catch (Exception e) { 
         e.printStackTrace(); 
    } 

    String enabled = map.getString("enabled");
    if(enabled  != null && enabled .equalsIgnoreCase("true") ) { 
  
  	    // your job code here...

    }
}

You could also just as well have hardcoded the location of your quartz-config.xml file into a java.net.URL object – and then grabbed that inputstream for xpath extraction.

4) Wrap your quartz Job class in an external servlet:

public class MyQuartzServlet extends GenericServlet { 
 
     private static final long serialVersionUID = 1L; 
     private static final Log log = LogFactory.getLog(MyQuartzServlet .class); 
 
     @Override 
     public void service(ServletRequest req, ServletResponse res)
          throws ServletException, IOException { 
 
          log.info("launching quartz job" from servlet); 
          QuartzJob  importJob = new QuartzJob (); 
          importJob.launch(); 
           
          // forward to some jsp, and/or add other job success/fail logic here
          getServletConfig().getServletContext()
                    .getRequestDispatcher("/servlet.jsp").forward(req,res); 
           
     } 
      
}

Of course you’d have to configure the servlet and servlet-mappings in your application’s web.xml, but that should be pretty straight forward.

Congrats, you now have a quartz job that loads an external configuration file, and that can also be invoked manually through a servlet. I’m not saying this is perfect and should be used in every possible quartz scenario, but this approach works well for quartz jobs where the properties might need overriding or temporary disabling. I can also understand the argument for why you would want to necessarily cycle every configuration change through qa. I hope at least this gives some folks idea outside the proverbial box. Now on to bigger fish to fry…

Configure ssh authorized keys for cvs access

Continous Integration

Lately I’ve been working on adding Hudson as the continuous integration (CI) server for projects at work. The whole notion of CI merits an entire discussion, but suffice to say it’s a very clean, approach that helps automate the build process particularly if you run manual builds that use prompted shell scripts.

After looking at a few solutions, Hudson seemed from many accounts to be the easiest to get running, and pretty flexible when integrating into an existing build system. Add to the resume that it could run in a servlet container, divorcing environmental configuration from its automated build functionality, and we suddenly have a winner.

I went to work setting up integration build scripts and projects and all kinds or cool plugins when I finally hit a wall when it came time to wire up Hudson with cvs access. As it turns out, in our particular setup we access cvs via ssh, and ssh will usually require a password in order to connect to a remote host. When automating builds, this can be quite problematic since it seems that part of the argument is to allow the builds to fire off without interactive human intervention. I noticed that prompted passwords are very capable of raining that parade out.

I dug around for what seemed like forever, until it seemed that the solution was to enable authorized key access via ssh, and configure the generated public key to not require a pass phrase. In a nutshell, you can set up a public and private key, and configure it to require a pass phrase or not when requesting access. You then copy that public key to the remote machines you want to enable access to into the correct location. The last step is to configure authorized key access via ssh on the remote machine. Only then will you be able to ssh to the remote machine with the public key and without a password or pass phrase – in essance that public key becomes trusted authentication.

Here are the steps, with more detail:

Configure your connect-from machine

Let’s assume you’re going to use an account called builder for this example. In your shell as builder, cd into ~/.ssh and run:

ssh-keygen -f identity -C ‘buildier identiy cvs key’ -N ” -t rsa -q

This will create the set of keys for you without a pass phrase. The -C flag sets the comment tagged at the end of the key. You want to end up with a file structure like this:

[builder@connectFrom.ssh]# ls -l iden*
-rw——- 1 jboss CodeDeploy 1675 Dec 5 09:54 identity
-rw-r–r– 1 jboss CodeDeploy 405 Dec 5 09:54 identity.pub

on your connect-from machine. You will need to chmod the user’s home and .ssh directories to permission 0700. It turns out that these folder permissions are very picky and these keys will not work if the group or others have read/write access to that .ssh directory or its contents.

Configure your connect-to machine

You will now want to again create a ~/.ssh directory, also with permissions set to 0700 on the connect-to machine. Then use your favorite text editor to create the file: ~/.ssh/authorization_keys. This one’s even more strict – ensure that ~/.ssh/authorized_keys permissions is set to 0600. Paste the contents of your connect-from machine’s file ~/.ssh/identity.pub into this authorized_keys file contents. This step essentially copies the public key over as an authorized key to the remote machine. The file authorized_keys should have only one key per line, or it will cause problems. Lastly, we’ll need to make sure that the flag PubkeyAuthentication is enabled on the connect-to machine and that it it reads in the correct authorized_keys file.

Edit the file /etc/ssh/sshd_config file and uncomment the following:


PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

Test
Now you should be able to test the ssh connection with debugging enabled by saying form the connect-from machine’s shell:

[builder@connectFrom.ssh]ssh -v builder@connectTo

You should see connection information useful for debugging – looking for something like this:

debug1: Next authentication method: publickey
debug1: Offering public key: /home/builder/.ssh/identity
debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Last login: Thu Dec 2 01:17:40 2010 from connectFrom
[builder@connectTo~]$

Configure Hudson to use the external ssh
Now that these authorized keys have been configured for use, you can go into Hudson and set up the cvs connection string. You will need to make sure that the cvs advanced configuration is set to :

$CVS_RSH: ssh

And you should be all set.

Your builder account should now be able to access the remote machine using the trusted authorized keys.

Resources:
How to allow SSH host keys on Linux (Fedora 10 & CentOS 5.2)
ssh – authorized_keys HOWTO
2.4.1 Connecting with rsh and ssh