Skip to content
Nov 6 12

Grails: Clustered Quartz Configs by Environment

by robin

I’ve been using the Quartz plugin for Grails on a recent project and ran into a minor annoyance after (partially) following Burt Beckwith’s clustering example.  Part of the problem was caused by not wanting to have Hibernate auto-run the DDL for the quartz tables, so I didn’t follow his steps for the Quartz.*.hbm.xml and hibernate.cfg.xml files.  Instead, I ran the appropriate DDL on my production and development environments.  Easy enough.

The annoying part was when it came time to run my integration tests against the h2 database, the quartz tables weren’t installed.  If I left quartz.properties on the class path even with autoStart=false and jdbcStore=false for the test environment, grails would exit with the error:

Error creating bean with name ‘quartzScheduler’: Invocation of init method failed; nested exception is org.quartz.SchedulerConfigException: DataSource name not set.

The problem was, the plugin’s doWithSpring closure configures the quartzScheduler bean based on the values in quartz.properties, which was setting the jobStore class to the jdbc implementation.   Luckily, further down in QuartzGrailsPlugin.groovy, we can see this:

// And now load quartz properties into main config
        def properties = new Properties()
        def resource = classLoader.getResourceAsStream("quartz.properties")
        if (resource != null) {
            properties.load(resource)
        }

        if (config.quartz.containsKey('props')) {
            properties << config.quartz.props.toProperties('org.quartz')
        }

Which shows that we can defined a ‘props’ block inside our config’s ‘quartz’ block rather than using quartz.properties. So, the final solution was to delete quartz.properties, and create a closure in QuartzConfig.groovy that I use in the dev and prod environments, but leave out of test. Here’s what that looks like:


quartz {
    waitForJobsToCompleteOnShutdown = true
    exposeSchedulerInRepository = false

    props {
        scheduler.skipUpdateCheck = true
    }
}

jdbcProps = {
        scheduler.instanceName = "reporting_quartz"
        scheduler.instanceId = "AUTO"

        threadPool.class = "org.quartz.simpl.SimpleThreadPool"
        threadPool.threadCount = 3
        threadPool.threadPriority = 5

        jobStore.misfireThreshold = 60000

        jobStore.class = "org.quartz.impl.jdbcjobstore.JobStoreTX"
        jobStore.driverDelegateClass = "org.quartz.impl.jdbcjobstore.PostgreSQLDelegate"

        jobStore.useProperties = false
        jobStore.tablePrefix = "qrtz_"
        jobStore.isClustered = true
        jobStore.clusterCheckinInterval = 5000

        plugin.shutdownhook.class = "org.quartz.plugins.management.ShutdownHookPlugin"
        plugin.shutdownhook.cleanShutdown = true
}

environments {
    test {
        quartz {
            autoStartup = false
            jdbcStore = false
        }
    }
    development {
        quartz {
            // Toggle scheduled tasks for local "run-app" environment

            autoStartup = true
            jdbcStore = true
            props(jdbcProps)
        }
    }
    production {
        quartz {
            jdbcStore = true
            autoStartup = true
            props(jdbcProps)
        }
    }
}

May 14 12

Java ActiveMQ Example

by robin

This type of example is surprisingly hard to find, so here’s a simple JMS client I use for pushing out messages to test one of our message driven services, that does _not_ require hooking into JNDI.

public static void main(String[] args) throws Exception {
    if (args.length &lt; 1) {
        JmsTool.printUsage();
        return;
    }
    ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(args[0]);
    QueueConnection queueConn = connectionFactory.createQueueConnection();
    Session session = queueConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
    ActiveMQQueue theQueue = new ActiveMQQueue("queue.theQueue");
    MessageProducer producer = session.createProducer(theQueue);
    MySerializableObject obj = new MySerializableObject();
    Message configMsg = session.createObjectMessage(obj);
    producer.send(configMsg);
    producer.close();
    queueConn.close();
}
Apr 29 12

Working with SVN and Heroku

by robin

We’ve decided to host our latest project on Heroku which uses Git as it’s deployment mechanism. Getting up and running with their toolchain was dead easy, especially since I’ve been using git-svn to work with our existing SVN repository. However, it raised the question of how to easily share changesets between Heroku and SVN. I didn’t want to use raw patches because it would either mean three steps for every commit (commit, patch, commit) , or losing granularity.

The solution is based on this Stack Overflow answer. The initial state involved two local repositories: one that I had initially cloned from SVN, the other had been cloned from Heroku.  The solution was to add a remote tracking branch for the heroku repository to the SVN repositry, then using gitk to cherry-pick from heroku to svn.

1. From the existing repository cloned from SVN, create a remote for the heroku repository:

git remote add heroku file:///path/to/heroku/repository

2. Fetch the remote info:

svn-proj rpercy$ git fetch heroku
warning: no common commits
remote: Counting objects: 202, done.
remote: Compressing objects: 100% (148/148), done.
remote: Total 202 (delta 74), reused 147 (delta 38)

3. Create a remote tracking branch to heroku:

svn-proj rpercy$ git branch --track heroku heroku/master
Branch heroku set up to track remote branch master from heroku.

4. From the svn master branch, use gitk to cherry-pick commits

svn-proj rpercy$ gitk heroku

5. GitK will open for the heroku branch, right click each commit you want on the top left and choose “Cherry Pick this commit”.

6. Verify that all commits were applied to the svn branch using ‘git log’ and then push them up to SVN with ‘git svn dcommit’

 

Feb 21 12

Happy Coding

by robin

I just finished reading The Happiness Advantage by Shawn Achor. Check out his TED talk if you haven’t yet seen it. In spite of its title, this is not a fluffy book on self-motivation.  (Though, I think it pretends to be). Its thesis is: the generally accepted belief in success leading to happiness is backwards.  Achor presents recent research and a number of studies in positive psychology as convincing evidence that happiness has a strong causal relationship to success.  As he says:

We become more successful when we are happier and more positive.  For example, doctors put in a positive mood before making a diagnosis show almost three times more intelligence and creativity than doctors in a neutral state, and they make accurate diagnoses 19 percent faster.  Optimistic salespeople outsell their pessimistic counterparts by 56 percent.  Students primed to feel happy before taking math achievement tests far outperform their neutral peers.

As a software developer, this resonated at a number of levels.  I’m an introverted nerd, so my default approach to a stressful problem is to lock myself away and work on the solution.   Impossible deadlines have meant evenings and weekends in the office.  Precisely the success-breeds-happiness mentality that Achor debunks.  Now, I genuinely love writing software but, usually after the fourth or fifth 80+ hour week, my positive feelings start to erode.

I’ve suspected there was something sub-optimal about by approach for a while.  Most, if not all, of the rockstars I’ve worked with seem to have two things in common: they are extremely productive at work, and they have very rich (and time-consuming) personal lives.  Of course, it’s common knowledge that a balanced life is better, but that’s optimizing for health or happiness, not productivity. Right?

The Advantage

As evidence that happiness leads to success, Achor cites a number of studies including a 2005 paper by  Lyubomirsky, King, and Diener: The Benefits of Frequent Positive Affect: Does Happiness Lead to Success?  The authors performed a meta-analysis of over 200 scientific happiness studies on 275,000 people worldwide.  From the abstract:

The results reveal that happiness is associated with and precedes numerous successful outcomes, as well as behaviors paralleling success. Furthermore, the evidence suggests that positive affect—the hallmark of well-being—may be the cause of many of the desirable characteristics, resources, and successes correlated with happiness

This is something I’ve always suspected.  It’s why I’m a firm believer in hiring people you can trust and giving them the tools and toys they need to stay happy and productive.  But, there is obviously a not-so-small percentage of managers (and managees) that believe in the power of fear or anger-based motivation.  What I’ve found challenging is quantifying the benefits of an enjoyable workplace to those with opposing views.

Self-fulfilling Prophecies

Oh yeah, and managers need to be happy too.  Obviously, its pretty hard to have happy developers if their manager is always pissed off, but there’s more to it.  Achor describes studies showing how the Pygmalion Effect (greater expectations leading to greater performance) manifests itself in teams.  The studies were based on two management theories: X and Y.  Theory X says employees won’t work if you don’t watch them, and Y says that employees work better if they aren’t micromanaged.

When researchers try to study what happens when X (or Y) workers are exposed to leaders with the opposing view, they run into a very telling snag.  Very few managers have employees with opposing theories.  Managers who believe Theory X turn out to have workers who need constant supervision, while managers who hold to Theory Y have employees Y have employees who work for the love of the job.

He goes on to state that this is indeed a causal relationship.  No source is given, but there was a related study by the University of Tel Aviv.

This alone is encouraging.  I hate mandating work hours and technologies.  My hope is: if my team is made up of smart, trustworthy people, I shouldn’t have to specify where, when, or how they work. We’re a Groovy/Java shop, but you prefer Ruby for automating functional tests?  Go for it.  If a Groovy developer can’t work with Ruby, then something’s wrong.  There is obviously some risk involved in this philosophy.  How do you really know that Steve’s project isn’t late because he’s been slacking off at home for the last two weeks?  How can you be sure someone can pick up another language or framework in a reasonable amount of time?  You can’t, but apparently chances are good that they’ll meet your expectations.  Plus, the added productivity of a happy team is worth the risk.

Jan 10 12

First Impressions of Stanford on iTunesU

by robin

Since it seems like Stanford’s online SaaS and NLP classes are going to be delayed even further, I decided to checkout their iPad and iPhone App Development (Fall 2011) course on iTunesU.  In the past, I’ve favoured written documentation over videos when ramping up on a technology, because the former tends to be more information-dense (or at least its easier to scan over the fluff).  In this case, I’ve been blown away by how effectively the material is presented.

Prof. Paul Hegarty does an excellent job of introducing the core concepts, each one building on the next, and pointing out common gotchas and best-practices along the way.  Coming from Stanford, you’d obviously expect a quality curriculum, but it’s much more than that.

Learning the syntax of a new programming language is easy.  Learning the idioms of that language and the paradigms its built on takes a little more time.  Learning the common frameworks and libraries that all the cool programmers use takes for-frigging-ever.  That’s why the best way to ramp up on a language is to a) use it a lot and b) read other people’s code.   Learning Objective-C/Cocoa Touch/iOS is no different -and that’s the beauty of these videos.  They provide the rare combination of having a competent programmer point out the why’s and how’s combined with a good number of not-so-trivial assignments to hack on.

I have to say, I’m really liking the new trend in open online courses.  From now on, I’ll be supplementing my reading with more of these sorts of videos.

- Robin

Dec 1 11

Creative, Self-Motivated, Geek

by robin

Those are the only qualifications I look for when interviewing for my team.  I don’t care which languages you’ve worked with before.  If you’re a geek, you’ll jump at the chance to learn our stack.  You know how to dig into a new technology and research solutions to common problems.  If you’re creative, you’ll find novel solutions to uncommon problems and identify new opportunities as well.  Finally, if you’re self-motivated, then I don’t have to babysit you.  You want to succeed and build something cool – so do I.  We’ll collaborate, brain storm, and leave each other alone when it’s time to crank out code..

A team built out of these qualities leads to productive, no bullshit cultures like what github seems to be cultivating .

Aug 17 10

Remote Profiling with JConsole

by robin

This is the third time I’ve wasted an hour hunting down the magical combination of JVM args to enable remote profiling.  Writing them down seems like a good idea.  This is how they appear in my Service Wrapper config, the -D’s would be added as cli options if you were calling java directly.

wrapper.java.additional.10=-Dcom.sun.management.jmxremote.port=8181 # port num is arbitrary, but one must be specified
wrapper.java.additional.11=-Dcom.sun.management.jmxremote.authenticate=false # if you're a slacker
wrapper.java.additional.12=-Dcom.sun.management.jmxremote.ssl=false # as prev comment
wrapper.java.additional.13=-Djava.rmi.server.hostname=192.168.1.100   # Required if your DNS is messed up

Connecting with JConsole:

$ $JAVA_HOME/bin/jconsole 192.168.1.100:8181  # TADA!

Useful references

Jul 3 10

Junior Rockstars

by robin

One of my primary filters when interviewing new hires is whether they hack in their spare time.  This seems to surprise many of my colleagues and I’m not sure why.  I want to work with Rockstars.  Rockstars are experts at their craft (I’m making a convenient distinction between Rockstars and Pop Stars).  They don’t just play a gig and leave their instrument in the case the rest of the time.  The only way to reach their level of skill is to practice like hell.

Studies have shown that good programmers are orders of magnitude more productive than bad programmers and that the relationship holds at all levels of experience.  So I apply the same filter regardless of a candidates seniority.   A junior rockstar is someone that may not have the depth or breadth of knowledge, but she’s well on her way. She’s scanning aggregators, reading books on new technologies, and hacking on side projects. This means three things for me:

  1. She’s well practiced at investigating and learning on her own; She’s not likely to need much hand-holding and will be able to quickly absorb the information we throw at her.
  2. She can write code – not necessarily good or clean code – but she knows how to get her hands dirty with it.  This is especially important at the Junior level.  Real-world development feels a lot different than what we did in school.
  3. I’m going to be a better programmer for working with her. Chances are good she’s looked into some technology, methodology, or concept that I haven’t come across yet. So the learning goes both ways, regardless of seniority.

That alone are why I value rockstar indicators over all other attributes, even familiarity with the technologies we use.  Technologies can be taught and, regardless of past experience, a new hire is going to have some learning to do.  I’ll trade a bit of ramp up time for those qualities any day.

Jun 24 10

Setting up Bamboo’s AJP Connector with Service Wrapper

by robin

I recently set this up for Bamboo and figure it’s worth writing down.  Atlassian and Apache have decent docs, so I’m not going to repeat them. If you don’t already have Apache setup with mod_jk, check out the HowTo.   I’ll assume that Bamboo standalone is already installed.

The first step is to ensure bamboo starts with the service wrapper.  Bamboo 2.6.1 for Linux has a bug (BAM-4197) in the wrapper.conf, all references to ../lib should be changed to ../wrapper/lib.  Confirm that it starts with:
$ BAMBOO/wrapper/start-bamboo start

Next, configure the service wrapper to load connector settings from jetty.xml so you can override the defaults.  Instructions can be found here, but the key step is to edit wrapper.conf:

  • You will need to replace the argument which specifies your port number “wrapper.app.parameter.2=8085″ with “wrapper.app.parameter.2=../webapp/WEB-INF/classes/jetty.xml”.
  • You will need to comment out the other arguments: “wrapper.app.parameter.3=../webapp” and “wrapper.app.parameter.4=/”

Then, follow these instructions to setup the AJP connector. Download the AJP connector for Jetty from http://repository.codehaus.org/org/mortbay/jetty/jetty-ajp/ to BAMBOO/webapp/WEB-INF/lib. The connector version needs to match the pre-installed Jetty libs. Next, add the connector config to BAMBOO/webapp/WEB-INF/classes/jetty.xml:

<Call name="addConnector">
  <Arg>
    <New class="org.mortbay.jetty.ajp.Ajp13SocketConnector">
      <Set name="port">8009</Set>
    </New>
  </Arg>
</Call>
There’s also a fix to make in jetty.xml.  Change the default attribute for the bamboo.webapp SystemProperty tag to ‘../webapp’ from ‘./webapp’.
Finally update your mod_jk configs.  Your worker properties should have something like this:


worker.bamboo1.type=ajp13
worker.bamboo1.host=bamboo.host.name
worker.bamboo1.port=8009

And the server config or host will need to contain


JkMount /bamboo* bamboo1

Oh yeah, and don’t forget to set the base URL on Bamboo: http://confluence.atlassian.com/display/BAMBOO/Specifying+Bamboo’s+URL

Jun 24 10

Less is More in Developer Resumes

by robin

We’re looking for an intermediate Java dev at the office so I’ve spent some time scanning resumes and interviewing recently. Being on this side of the table inevitably gets you thinking about how people present themselves as candidates. What surprises me is how out of touch many of them are with what an employer really wants to know.

People seem to have this need to cover every aspect of their previous job on a resume – no matter how boring. This usually leads to choosing the most general statements about what they did, since they’re also trying to be concise. You get bullet points of stuff like ‘Developed enterprise Java software on JBoss with ActiveMQ and MySQL’. Great, so does that mean you had a senior dev that setup the JBoss instance, wrote the EJB skeletons and asked you to fill in the code? Or were you the dev that Systems called in the middle of the night because a connection pool was exhausted and they needed your skillz?

If you’re applying for a greater-than-junior job, go ahead and list JBoss, Hibernate, and RabbitMQ in the Technologies section of your resume. I’ll just assume you can handle the mundane stuff. Now, what were the most painful, exciting, and brain-hemorrhaging 150 hours of your three years at that job?  That’s where I really get to see what you’ve learned.  Two weeks to get XA transactions working only to realize 3 months down the road that distributed transactions were developed in Satan’s private lab?  Cool.  Five hours, the night before your 2 week vacation started, troubleshooting a memory leak in the Biling System you just deployed?  Sounds like the kind of fun I want to know about.  Even if the best you can give me is, “configured DataSources for our Tomcat development server”, I’ll take it.

In short, don’t feel compelled to list every boring part of your last N jobs on your resume.  It just ends up diluting the good stuff.  Instead, list the extremes, good and bad.  In fact, the more painful the better – you’re more likely to have learned something from it.