HTTP Server for Integration testing on java

After another reimplementation of the HTTP server that is needed for integration testing I decided to create a simple library designed solely for integration testing: https://github.com/avshabanov/http-server-it

It is quite simple to use:

    ItHttpServer server = new DefaultItHttpServer(executorService);
    // Set response handler that returns 204 NO CONTENT for all the requests
    server.setResponseProducer(EmptyItResponseProducer.NO_CONTENT_INSTANCE);

    // request for someting
    HttpDelete httpDelete = new HttpDelete("http://127.0.0.1:" + server.getPort() + "/delete/something";
    HttpResponse response = httpClient.execute(httpDelete);
    // response.getStatusLine().getStatusCode() will return 204

Quartz+Spring scheduling-on-demand sample

Some time ago I stumbled at issues with Quartz 2 and Spring 3. All the standard samples illustrate fairly simple use cases and there were no samples dedicated to quite a frequent task: schedule certain job on demand, not to mention that some “standard” spring classes dedicated to ease work with Quartz are outdated and introduces some misterious errors in runtime (IncompatibleClassChangeError is one of these ones).

To illustrate this particular kind of task when you need scheduling-on-demand suppose you have a procedure that must be scheduled after the user entered some data and then executed after some time.

One of the best solution that is reliable and has fail-over on top of inner DB persistence, clustering capabilities is Quartz.

I’ve wrote a JUnit test that doesn’t test anything but actually illustrates the work with Quartz, it is quite self evident:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.quartz.Job;
import org.quartz.JobDetail;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.quartz.Scheduler;
import org.quartz.SchedulerContext;
import org.quartz.SchedulerException;
import org.quartz.SimpleTrigger;
import org.quartz.Trigger;
import org.quartz.impl.triggers.SimpleTriggerImpl;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.JobDetailFactoryBean;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import java.util.Date;

/**
 * Test scheduling-on-demand with spring and quartz
 */
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { SchedTest.Config.class })
public class SchedTest {

    private static final String SERVICE_KEY = "SERVICE_KEY";

    @Autowired
    private Scheduler scheduler;

    @Autowired
    private ApplicationContext context;

    @Test
    public void shouldExecuteSampleJob() throws Exception {
        {
            final String id = "1";
            scheduler.scheduleJob(createJobDetail(id, "task1"), createSingleshotTrigger(id, 100L));
        }
        {
            final String id = "2";
            scheduler.scheduleJob(createJobDetail(id, "task2"), createSingleshotTrigger(id, 150L));
        }
        Thread.sleep(1000L);
    }

    private Trigger createSingleshotTrigger(String triggerName, long delay) {
        final SimpleTriggerImpl simpleTrigger = new SimpleTriggerImpl();
        simpleTrigger.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_FIRE_NOW);
        simpleTrigger.setStartTime(new Date(System.currentTimeMillis() + delay));
        simpleTrigger.setRepeatCount(0);
        simpleTrigger.setName(triggerName);
        simpleTrigger.setRepeatCount(0);
        return simpleTrigger;
    }

    private JobDetail createJobDetail(String jobName, String url) {
        final JobDetailFactoryBean jobDetailFactoryBean = new JobDetailFactoryBean();
        jobDetailFactoryBean.setApplicationContext(context);
        jobDetailFactoryBean.setName(jobName);
        jobDetailFactoryBean.setJobClass(JobStarter.class);
        jobDetailFactoryBean.afterPropertiesSet();

        final JobDetail jobDetail = jobDetailFactoryBean.getObject();
        // data
        jobDetail.getJobDataMap().put("url", url);

        return jobDetail;
    }

    public static class JobStarter implements Job {

        @Override
        public void execute(JobExecutionContext context) throws JobExecutionException {
            try {
                final SchedulerContext schedulerContext = context.getScheduler().getContext();
                schedulerContext.get(SERVICE_KEY);

                final ApplicationContext applicationContext = (ApplicationContext) schedulerContext.get(
                        ApplicationContext.class.getName());

                final JobDetail jobDetail = context.getJobDetail();
                final String url = jobDetail.getJobDataMap().getString("url");
                System.out.println("url=" + url + ", applicationContext=" + applicationContext);
            } catch (SchedulerException e) {
                throw new JobExecutionException(e);
            }
        }
    }

    @Configuration
    public static class Config {

        @Bean
        public SchedulerFactoryBean schedulerFactoryBean() {
            final SchedulerFactoryBean bean = new SchedulerFactoryBean();
            bean.setApplicationContextSchedulerContextKey(ApplicationContext.class.getName());
            return bean;
        }
    }
}

Sun unsafe API

It is quite interesting that though Unsafe API is forbidden to be used in the “normal” java applications, it is still possible to access it, though it is strictly discouraged in the production code. I do it “just for fun” and don’t recommend for use in any “real” applications.

In short: unsafe API (sun.misc.Unsafe) allows access to native sun-specific API such as allocating chunk of native memory, get internal offsets for quick property sets and etc.
Though addressing Unsafe.getUnsafe generates compiler error it is possible to access it via reflection by reading static private properties from some classes, such as AtomicReference:

try {
    final Field unsafeField = AtomicReference.class.getDeclaredField("unsafe");
    unsafeField.setAccessible(true);
    final Unsafe u = (Unsafe) unsafeField.get(null);
    System.out.println("unsafe: " + u);
    // ...
catch (Exception e) { /* ... */ }

I was especially curious on quick, non-reflection driven property access.
It may be done via Unsafe.put* methods, such as Unsafe.putInt.

For example, if you have the following point object:

class Point2DV {
    int x;
    int y;
}

you may access x and y fields by addressing them directly.

The following snippet demonstrates it:

final long xOffset = u.objectFieldOffset(Point2DV.class.getDeclaredField("x"));
final long yOffset = u.objectFieldOffset(Point2DV.class.getDeclaredField("y"));
final Point2DV p = new Point2DV();
u.putInt(p, xOffset, 872);
u.putInt(p, yOffset, 1134);
System.out.println("offsetof(Point2DV.x) = " + xOffset + ", p.x = " + p.x);
System.out.println("offsetof(Point2DV.y) = " + yOffset + ", p.y = " + p.y);

It is quite curious to see that the offset of the very first property is 12 which tells us that *every* java object contains at least 12 bytes in memory which in its turn results that in reality even an object with no fields would occupy at least 16 bytes since jvm uses 8-bytes offsets).

Have fun (and never use it in real projects)!

Using clojure in maven projects + tuning Intellij IDEA for this kind of projects

This is a quick howto about mixing clojure/java in maven projects.

Common Part

  1. Add the following to your root pom.xml (I assume your project is large and you have a root pom where you define settings to the common plugins etc.):
        <properties>
            <!--  ... -->
            <clojure.version>1.4.0</clojure.version>
        </properties>
    
        <!--  ... -->
        <dependencyManagement>
    
            <dependencies>
    
                <!-- Clojure core -->
                <dependency>
                    <groupId>org.clojure</groupId>
                    <artifactId>clojure</artifactId>
                    <version>${clojure.version}</version>
                </dependency>
    
            </dependencies>
        </dependencyManagement>
    
        <build>
            <pluginManagement>
                <plugins>
                    <!-- Clean plugin -->
                    <plugin>
                        <artifactId>maven-clean-plugin</artifactId>
                        <groupId>org.apache.maven.plugins</groupId>
                        <version>2.2</version>
                    </plugin>
    
                    <!-- Dependencies plugin -->
                    <plugin>
                        <artifactId>maven-dependency-plugin</artifactId>
                        <groupId>org.apache.maven.plugins</groupId>
                        <version>2.2</version>
                    </plugin>
    
                    <!-- Clojure Maven Plugin -->
                    <plugin>
                        <groupId>com.theoryinpractise</groupId>
                        <artifactId>clojure-maven-plugin</artifactId>
                        <version>1.3.10</version>
    
                        <configuration>
                            <copyAllCompiledNamespaces>true</copyAllCompiledNamespaces>
                        </configuration>
    
                        <executions>
                            <execution>
                                <id>clojure-compile</id>
                                <phase>compile</phase>
                                <goals><goal>compile</goal></goals>
                            </execution>
                            <execution>
                                <id>clojure-test</id>
                                <phase>test</phase>
                                <goals><goal>test</goal></goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </pluginManagement>
        </build>
    

    Note, that this configuration set ups clojure plugin so that it expects clojure sources to be placed under $project/src/main/clojure dir and test clojure code – under $project/src/test/clojure

  2. Add the following to your pom.xml in the project where you want to mix clojure/java code:
    <dependencies>
            <!-- clojure dependencies -->
            <dependency>
                <groupId>org.clojure</groupId>
                <artifactId>clojure</artifactId>
            </dependency>
    
            <!-- test scope -->
            <dependency>
                <groupId>junit</groupId>
                <artifactId>junit</artifactId>
            </dependency>
        </dependencies>
    
        <build>
            <resources>
                <resource>
                    <directory>src/main/clojure</directory>
                </resource>
                <resource>
                    <directory>src/main/resources</directory>
                </resource>
            </resources>
            <testResources>
                <testResource>
                    <directory>src/test/clojure</directory>
                </testResource>
                <testResource>
                    <directory>src/test/resources</directory>
                </testResource>
            </testResources>
    
            <plugins>
                <plugin>
                    <groupId>com.theoryinpractise</groupId>
                    <artifactId>clojure-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    

IDE part

For Intellij users who use LaClojure plugin – just add clojure support by clicking on “Add Framework Support” context menu item for the project view.
Notice, that Intellij may suggest you to use builtin clojure library that is shipped with LaClojure plugin, since you use maven, pick the clojure library that is downloaded to your maven repository.
Then you may want to add classpath with your clojure source code to the REPL – this is tricky one.

Open your module settings view (“Open Module Settings” context menu item for project view, or just press F4), proceed to clojure REPL options and add the following to the JVM settings

-Xss1m -classpath $full_path_to_your_subproject/src/main/clojure/

Intellij IDEA Module Settings - Clojure REPL Facet

The latter will make your REPL aware of your project structure, so basically you’ll notice that the dependencies you specify in ns statement will be resolved when you’ll interactively evaluate (ns …) statement in your project. That’s basically because ns in clojure is built on top of the clojure.core/load function which in its turn uses JVM classpath to search your source code.

Adding custom Intellij Idea launcher on unity (Ubuntu 12.04)

As you know Intellij Idea comes in tar.gz distribution for linux OS, so it is only possible to launch it from terminal after unpacking.

But there is a possibility to add quick launcher button for unity.
First of all, create idea.desktop file with the following contents:

[Desktop Entry]
Name=Idea Community Edition
GenericName=Idea
Comment=Launch Idea Community Edition
Keywords=ide,programming,java
Exec=/home/alex/bin/idea-IC/bin/idea.sh
Terminal=false
Icon=/home/alex/bin/idea-IC/bin/idea.png
Type=Application
StartupNotify=true

Paste your location to the Idea distribution in the italicized text given above.
After that, open this file in nautilus and just drag it to your launcher bar.
Voila!

Functional programming patterns: Currying

In this post I would like to tell about the highly underrated functional programming pattern named “Currying“. I would emphasize that this pattern shines in the languages with the first class functions and you might find it tedious to use the pattern in the languages like java or C++.

In this article I’ll concentrate on the practical application of the currying.

Let’s start with the quick rough definition: F is a curried function with N arguments when an invokation of it can be transformed to N function invokations and each invokation takes exactly one argument and returns function for the next subsequent call. For those who interested in the formal definition of it here is the wikipedia article.

For instance in javascript a function that computes a sum of two arguments in a curried form can be written as follows:

function add(x) {
    return function (y) {
        return x + y
    }
}

// x + y ==> add(x)(y)

On the first glance the pattern may look disappointing and too artificial to be used in real world applications (at least for the novices in functional programming).

But in fact this powerful pattern can help you to deal with certain problems for what OOP is used before in more elegant way.
For instance you need a printing function that is capable to print something on the output stream.
The usual approach would be to introduce a Printer class that accepts output stream in the constructor and provides the prospective clients with the public print method.
The other functional approach that uses currying may help you to get rid of the excessive printer entity but still provide the prospective clients with the similar print function.

Here is how these approach may look in javascript:

// OOP-alike approach:
Printer = function(stream) {
    this._stream = stream
}

Printer.prototype = {
    _stream: null,

    print: function(message) {
        // implementation omitted for clarity
    }
}

// usage:
//   var p = new Printer(s);
//   p.print("Hello");
//   p.print("World");

and

// Currying-driven approach:
function print(stream) {
    return function (message) {
        // implementation omitted for clarity.
    }
}

// usage:
//   var p = print(stream);
//   p("Hello");
//   p("World");

You may find the latter approach much cleaner and more readable as I did :)

I would notice that currying was not emerged to replace OOP and this pattern is good when it fits the problem you want to solve (as any other design pattern does).

[To be continued]

Print assembly for Java

The modern openjdk’s JVM is able to print assembly for generated machine code – see Oracle’s blog post.

To start working with this option you need hsdis plugin, available here.

Then put the downloaded library to the location your OS aware of.
E.g. if you’re using linux all you need is to update LD_LIBRARY_PATH:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/alex/bin/hsdis"

Then you must be able to start java with the PrintAssembly flag:

/usr/lib/jvm/java-6-sun/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly -server -cp . App

The dissassembled output might look as follows:

0xb3a15f0f: jmp 0xb3a15f1b ;*invokespecial RangeCheck
; - java.util.ArrayList::get@2 (line 322)
; - App$Finder::findSolution@33 (line 243)
; - App$Finder::findSolution@182 (line 263)
; - App$Finder::findSolution@182 (line 263)
0xb3a15f11: mov %eax,%ecx
0xb3a15f13: jmp 0xb3a15f1b ;*invokespecial RangeCheck
Total: 13211984ns
; - java.util.ArrayList::get@2 (line 322)
; - App$Finder::findSolution@33 (line 243)
; - App$Finder::findSolution@182 (line 263)
0xb3a15f15: mov %eax,%ecx
0xb3a15f17: jmp 0xb3a15f1b ;*invokespecial RangeCheck
; - java.util.ArrayList::get@2 (line 322)
; - App$Finder::findSolution@33 (line 243)
0xb3a15f19: mov %eax,%ecx ;*synchronization entry
; - App$Finder::findSolution@-1 (line 236)
0xb3a15f1b: add $0x68,%esp
0xb3a15f1e: pop %ebp
0xb3a15f1f: jmp 0xb3a11da0 ;*return
; - App$Finder::findSolution@27 (line 240)
; {runtime_call}
0xb3a15f24: hlt
0xb3a15f25: hlt

Tuning Intellij Idea

First of all consider changing idea.vmoptions.
idea.vmoptions is a text file with a list of JVM settings, it’s usually found in the bin folder in the Idea installation folder. This file can be open in any text editor.

The following settings increase performance drastically on Sun JVM 1.6:

-server
-Xms128m
-Xmx512m
-XX:MaxPermSize=250m
-XX:ReservedCodeCacheSize=64m
-XX:+UseConcMarkSweepGC
-XX:+AggressiveOpts
-XX:+CMSClassUnloadingEnabled
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-XX:CMSIncrementalDutyCycleMin=0
-XX:-TraceClassUnloading

Notice, that I enabled concurrent mark-sweep GC (in theory it’d be faster than others on 1.6) and aggressive optimizations as well as server JVM and I don’t use assertions (no `-ea’ switch).

I tested these on x86 linux with Idea Community edition of build 111.69

As for appearance I’d suggest using Nimbus theme (Settings -> Appearance -> Look and Feel).

That’s how idea looks on my Ubuntu:

Idea 11 Community Edition on Ubuntu

Full screenshot is available here.

P.S.: For Mac OS X users:
There is no idea.vmoptions file on Mac OS X.
In order to modify the mentioned JVM settings, you’d go to the application’s package contents, then open the inner “Contents” folder and then open Info.plist (just click on it and the standard plist editor will be opened). In the opened plist editor consider changing Java > VMOptions key as given above.
You may copy-and-paste the following value I use on my Mac laptop:

-Xverify:none -XX:+UseConcMarkSweepGC -XX:+AggressiveOpts -XX:+CMSClassUnloadingEnabled -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=0 -XX:-TraceClassUnloading -Xbootclasspath/a:../lib/boot.jar

Note, that you may want to make Idea start in 32-bit mode to reduce the memory consumption and increase the overall IDE performance. Just ctrl-click on the application icon in the “Applications” folder and click on the checkbox “Open in 32-bit mode”.

After doing so Idea starts in 1 (ONE) second or so on my Mac Book Pro!

P.P.S:
You have to edit idea64.vmoptions if you’re running idea in 64-bit mode (thanks to Rudy for noticing this).

instanceof vs Visitor

I always was curious on how effective is visitor pattern vs instanceof when it comes to detect whether the given base implements certain interface or not.

I had 7 interface classes and visitor for all of these, and 7 concrete implementations, then I tested instanceof and visitor pattern on a large collection of randomly picked instances of these classes, the timing for 100000 objects was as follows on my machine (best value is taken for several interations of both benchmarks, the less is of course the better):

instanceof: 5.055 sec
visitor: 3.511 sec

The testing code snippets are as follows:

// for instanceof:
    public static int measureInstanceOf(Node[] nodes) {
        int litCount = 0;

        for (int i = 0; i < N; ++i) {
            if (nodes[i] instanceof Literal) {
                ++litCount;
            }
        }

        return litCount;
    }

// for visitor:
    public static final class LiteralCheckerVisitor implements NodeVisitor {
        boolean isLiteral = false;

        public void visitMarker(Marker marker) {}
        // other methods are similar to the given above and skipped

        public void visitLiteral(Literal literal) { isLiteral = true; }
    }

    public static int measureVisitor(Node[] nodes) {
        int litCount = 0;
        final LiteralCheckerVisitor visitor = new LiteralCheckerVisitor();

        for (int i = 0; i < N; ++i) {
            // equivalent of instanceof Literal
            visitor.isLiteral = false;
            nodes[i].accept(visitor);
            if (visitor.isLiteral) {
                ++litCount;
            }
        }

        return litCount;
    }

As for java version:

java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07-334-10M3326)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02-334, mixed mode)

The resolution is quite simple - try to avoid instanceof, at least in time-critical code. It's usage often indicates a misunderstanding of the OOP principles and often may result in bad design (and bad performance as well as higher maintenance costs).