How to make Ubuntu play music over bluetooth

After installing latest Ubuntu version 14.04 (Trusty Tahr) I found out by accident that Ubuntu is able to play music from bluetooth source. Here's how it works on my laptop (Sony Vaio VPCEA2S1E, perhaps there's differences in the bluetooth adapters?).

Check has PulseAudio already loaded module-bluetooth-discover.
$ pactl list | grep -i module-bluetooth-discover

If this command does not return anything, you should load the module with the following command.
$ pactl load-module module-bluetooth-discover

Now the first command shows that the module is loaded.
$ pactl list | grep -i module-bluetooth-discover
Name: module-bluetooth-discover

After executing the command, you should be able to add your for example your phone as the source of audio. In my Windows Phone Lumia 1020, I navigate to Settings -> Accessories -> add using bluetooth. If you have music playing on your phone and you connect to your Ubuntu via bluetooth, your phone will show text connected music. And the music is coming out of your computer's speakers!

I also discovered that in the latest Ubuntu version my bluetooth keyboard Logitech diNovo Edge settings have to be adjusted manually. The mouse pointer does not move fast enough when using keyboard's touchpad. In the previous version Ubuntu 12.04 I was able to set the speed and acceleration of pointer fast enough from System Settings UI. In 14.04, I cannot. However, it's easy to change the acceleration and speed of pointer from command line. Here's the command.
$ xset m 3 1

All in all, I'm very happy with the latest Ubuntu release!

Setting up Logback with nice default appender

I spent some time looking for a solution to make Logback log everything to console in development environment and still have easy setup for logging into file in production environment.

So, my goal was that if I execute my application without any extra Logback configuration, everything goes to console. And in case configuration file exists in the environment, Logback would use that and forget the default configuration.

The default console logging configuration is in the root of classpath and is named as logback.xml (the file Logback looks for by default). Here's the contents the file.

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="30 seconds">

  <define name="INCLUDED_FILE_EXISTS" class="ch.qos.logback.core.property.FileExistsPropertyDefiner">
    <path>${configpath}/included-logback.xml</path>
  </define>
  
  <if condition='property("INCLUDED_FILE_EXISTS").equals("true")'>
    <then>
      <!-- this configuration is used in other than development environments i.e. custom config per environment -->
      <include file="${configpath}/included-logback.xml"/>
    </then>
    <else>
      <!-- This configuration is used only in development enviroment -->
      <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
          <pattern>%d{HH:mm:ss.SSS} %-5level %logger{35} - %msg%n</pattern>
        </encoder>
      </appender>
      <root level="WARN">
        <appender-ref ref="STDOUT" />
      </root>
      <logger name="fi.foo.bar.package" level="DEBUG"/>
    </else>
  </if> 
</configuration>

This configuration has two important sections. First one is the use of FileExistsPropertyDefiner. It checks the existence of the given file in file system and sets value of property INCLUDED_FILE_EXISTS to true or false. If you want, you can also check existence of a file on classpath by using ResourceExistsPropertyDefiner. The variable configpath is defined in system properties.  Second interesting section is where the INCLUDED_FILE_EXISTS property is used to include the external file. If the file was found, its configuration is used and default configuration is discarded completely.

Here is example of the included file. The file has name included-logback.xml.

<included scan="true">
  <!-- This file is scanned so you can update it while application is running -->
  <appender name="FILELOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${configpath}/logs/application.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${configpath}/logs/application-%d{yyyy-MM-dd}.log.gz</fileNamePattern>
    </rollingPolicy>
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>

  <root level="INFO">
    <appender-ref ref="FILELOG" />
  </root>
</included>

Creating a bootable Xubuntu USB stick and installing Xubuntu on another USB stick

A friend of mine was complaining that his Asus Eee 1015BX is running too slow with Windows 7 Starter that came pre-installed with the laptop. It's no wonder that Windows is slow, because the laptop has only 1GB of memory. As a great (Ubuntu) Linux evangelist, I wanted to help the poor guy and give him a chance to test Ubuntu without any risks.

Because Ubuntu would probably be too heavy for such low end hardware, I thought that Xubuntu would serve better. I didn't want to change anything on the laptop. I just wanted to offer the Ubuntu experience without any risks. I decided that the best way to achieve this would be to install Xubuntu on USB disk.

To install Xubuntu on USB disk, you need installation media. Well, this laptop has only two USB ports (of course also LAN), so I cannot use DVD as in my previous installations. First problem was to create Xubuntu installation media. For some reason Ubuntu Startup Disk Creator doesn't allow user to create bootable USB stick from Xubuntu ISO image. Luckily there's an alternative, UNetbootin, which worked perfectly.

Next step was to run the installation software. You cannot set Eee's BIOS to boot from USB, but you can hit escape key during start-up to change booting options. After I got Eee to boot from USB, I inserted another USB stick, which would serve as the installation target media.

I'm not going through the details step-by-step, because installer is quite easy to use. I created /, /boot and /home partitions and no swap. It's important to notice that you shouldn't use USB stick to store a swap partition. Firstly, swap on USB is slow and secondly, it will - according to internet discussions - destroy your USB stick after some time. Installation took a lot of time, almost an hour. It would never take that long on a hard drive.

After the installation, there's some things you can do to optimise the speed of the system and the lifetime of the USB drive. Here's what I did.

Edit /etc/fstab. Add noatime flag to / and /home to avoid updating access times on files. This should speed things up  and help to avoid extra wearing of the USB stick. Here's an example.
# / was on /dev/sdb5 during installation
UUID=ae7eb15f-a65b-4e2b-a008-20a2b41ce72a /               ext4    noatime,errors=remount-ro 0       1
# /boot was on /dev/sdb1 during installation
UUID=190d4960-d773-451f-bbb3-b1a8bd39238b /boot           ext4    defaults        0       2
# /home was on /dev/sdb6 during installation
UUID=2fb3fc52-6155-4fdf-89f5-d991bca484fc /home           ext4    defaults,noatime        0       2

Disable mlocate so that it won't re-index the whole disk every day.
sudo chmod -x /etc/cron.daily/mlocate

Disable journaling from / and /home (you need to do these in single user mode or by starting Linux from some other media like the USB stick containing installation media, because the partitions cannot be mounted during the operation). By disabling journaling, you may end up with corrupted file system if the system crashes. On the other hand, you reduce the number of writes on the USB drive which should make things faster and prolong the lifetime of USB drive.
sudo tune2fs -O ^has_journal /dev/sdb5
sudo tune2fs -O ^has_journal /dev/sdb6

By the way, you can also prepare the drives already beforehand and format the partitions for ext4 without journaling with the following command. I did this after my first installation attempt, because I hoped it would speed up the slow installation process (I'm not sure did it help).
sudo mke2fs -t ext4 -O ^has_journal /dev/sdb6

After editing the file system parameters, it's a good idea to check that the file system is ok (-f for forcing the check)
sudo e2fsck -f /dev/sdb5
sudo e2fsck -f /dev/sdb6


Here's what I learned from all of this. Xubuntu is much much faster than Windows 7 on the Asus Eee. That was no surprise. All the hardware worked properly. After installing the proprietary AMD display drivers, battery consumption went down and battery life is at least as good as on Windows. Although it takes some time to install Xubuntu from USB to USB, it's a very good way to make a test drive with Xubuntu. You don't risk anything and user can always go back to Windows without any extra hassle.

Although Xubuntu worked well, my USB stick didn't. I bought a tiny Kingston 32GB Datatraveller Micro to serve as installation target media. The Datatraveller Micro is so small that you can carry it attached to the laptop all the time. Unfortunately, it's also slow making Xubuntu work very slowly during disk intensive operations. I compared the write speed of Datatraveller Micro to Datatraveller 100 on USB 2 port. Datatraveller Micro's write speed was only 6MB/s whereas Datatraveller 100's write speed was 22MB/s. Although the physical dimensions of Datatraveller Micro are perfect for running laptop Linux from it, the speed of the drive makes it questionable for this purpose.

This story may have a happy ending, because it seems that Ubuntu is getting yet another happy user in the very near future (after I install Xubuntu on Eee's hard drive)!

Easy JUnit testing with Elastic Search

It was quite difficult to find good examples on JUnit testing (this is more like integration testing than unit, but nevertheless JUnit is used) Elastic Search code. Here's my attempt to fix this issue.

The idea is to start a standalone Elastic Search instance in the test. This way we don't have to make sure that every developer has access to Elastic Search instance running somewhere out of build's control. The downside of starting an instance inside test is that it may get quite slow to run the test. However, that's another problem to tackle.

I'm using Jersey HTTP client in this example to connect to Elastic Search. Any other client works as well.

Maven POM 

Maven POM that's needed to execute my example.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>fi.elastic-search.test</groupId>
  <artifactId>elastic-junit</artifactId>
  <name>elastic-junit</name>
  <packaging>jar</packaging>
  <version>1.0.0-BUILD-SNAPSHOT</version>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>0.20.3</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-client</artifactId>
      <version>1.17</version>
</dependency> 
  </dependencies>
</project>

JUnit test code 

Here's the Java code starting standalone Elastic Search instance and creating an index.

import java.io.File;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;

import javax.ws.rs.core.MediaType;

import org.elasticsearch.common.io.FileSystemUtils;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import static org.elasticsearch.node.NodeBuilder.nodeBuilder;
import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.UniformInterfaceException;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.api.client.config.DefaultClientConfig;
import com.sun.jersey.api.client.filter.LoggingFilter;


public class ElasticSearchTest {
    private Client client;
    private static String testIndexName = "junitindexname";
    private static final String HTTP_BASE_URL = "http://localhost";
    private static final String HTTP_PORT = "9205";
    private static final String HTTP_TRANSPORT_PORT = "9305";
    private static final String elasticSearchBaseUrl = HTTP_BASE_URL + ":" + HTTP_PORT;

    private static Node node;

    @BeforeClass
    public static void startElasticSearch() throws Exception {
        final String nodeName = "junittestnode";

        Map settingsMap = new HashMap();
        // create all data directories under Maven build directory
        settingsMap.put("path.conf", "target");
        settingsMap.put("path.data", "target");
        settingsMap.put("path.work", "target");
        settingsMap.put("path.logs", "target");
        // set ports used by Elastic Search to something different than default
        settingsMap.put("http.port", HTTP_PORT);
        settingsMap.put("transport.tcp.port", HTTP_TRANSPORT_PORT);
        settingsMap.put("index.number_of_shards", "1");
        settingsMap.put("index.number_of_replicas", "0");
        // disable clustering
        settingsMap.put("discovery.zen.ping.multicast.enabled", "false");
        // disable automatic index creation
        settingsMap.put("action.auto_create_index", "false");
        // disable automatic type creation
        settingsMap.put("index.mapper.dynamic", "false");

        removeOldDataDir("target/" + nodeName);

        Settings settings = ImmutableSettings.settingsBuilder()
                .put(settingsMap).build();
        node = nodeBuilder().settings(settings).clusterName(nodeName)
                .client(false).node();
        node.start();
    }

    private static void removeOldDataDir(String datadir) throws Exception {
        File dataDir = new File(datadir);
        if (dataDir.exists()) {
            FileSystemUtils.deleteRecursively(dataDir, true);
        }
    }

    @AfterClass
    public static void stopElasticSearch() {
        node.close();
    }

    @Before
    public void initialize() {
        // create client for each test
        ClientConfig clientConfig = new DefaultClientConfig();
        client = Client.create(clientConfig);
        client.addFilter(new LoggingFilter(System.out));
    }
    
    @Test
    public void testCreateIndex() {
        WebResource service = client.resource(elasticSearchBaseUrl);
        // check first that does the index already exist
        ClientResponse clientResponse = service.path(testIndexName).head();
        if (clientResponse.getClientResponseStatus().equals(ClientResponse.Status.OK))
        {
            Assert.fail("Index exists already");
        }

        String indexJson = 
                "{\"settings\" : {\"index\" : {\"number_of_shards\" : 1,\"number_of_replicas\" : 0}}}";
        try {
            String response = 
                    service.path(testIndexName).
                    queryParam("refresh", "true").
                    queryParam("timeout","5m").
                    accept(MediaType.APPLICATION_JSON).
                    put(String.class, indexJson);
            if (!response.contains("ok")) {
                Assert.fail("Creating index failed. IndexName: " + testIndexName);
            }
        } catch (UniformInterfaceException e) {
            // failed due to Client side problem
            throw e;
        }

        // wait for Elastic Search to be ready for further processing
        HashMap statusParameters =
                new HashMap();
        final String timeout = "30s";
        statusParameters.put("timeout", timeout);
        statusParameters.put("wait_for_status", "green");
        String statusResponse = status(statusParameters);
        if (statusResponse.contains("red")) {
            Assert.fail("Failed to create index");
        }
    }
    
    public String status(final Map optionalParameters) {
        WebResource service = client.resource(elasticSearchBaseUrl);
        WebResource webResource = service.path("_cluster").path("/health");
        for (Entry entry : optionalParameters.entrySet()) {
            webResource = webResource.queryParam(entry.getKey(),entry.getValue());
        }
        return webResource.get(String.class);
    }

}

Some tomcat7-maven-plugin tricks

Here's some useful tomcat7-maven-plugin tricks that work also with other Maven plugins.

How to set java endorsed dir for tomcat7-maven-plugin

Sometimes you need to use Java endorsed standards override mechanism with Maven plugins. This can be accomplished simply by setting proper MAVEN_OPTS value. For example

export MAVEN_OPTS="-Djava.endorsed.dirs=/some/directory/where/endorsed/jars/are"

How to change tomcat7-maven-plugin classpath order

Very often you need to have the runtime classpath in certain order when running Tomcat or other Maven plugins. I found out by trial and error that the used classpath JAR ordering depends on the order of declared dependencies in pom.xml (most likely this is documented somewhere). So, if you need to get some libraries before others, just move them in the beginning of pom.xml. Of course this does not help, if you have many plugins that need a different order for each plugin.

How to list all loaded classes and their respective JAR files

This is something I had been looking for years and was extremely happy when I finally got to know it. At least Sun Java JRE provides a command line switch for showing all the classes that are loaded into JVM and the source JAR file of those classes. It's not always easy to know what classes are loaded, especially if your dependencies have dependencies to other JAR files that are somehow conflicting (for example different version) with JAR dependencies defined elsewhere. Anyway, to list the classes and their exact source, do the following.

export MAVEN_OPTS="-verbose:class"

The output is something like this.

[Loaded javax.xml.bind.annotation.XmlElement from /home/perttu/software/jdks/jdk1.6.0_31/jre/lib/rt.jar]
[Loaded javax.xml.bind.annotation.XmlElement$DEFAULT from /home/perttu/software/jdks/jdk1.6.0_31/jre/lib/rt.jar]
[Loaded javax.xml.ws.WebFault from /home/perttu/software/jdks/jdk1.6.0_31/jre/lib/rt.jar]

With these tools, any problem related to Tomcat Maven plugin and classpath should be easy to solve.

Encrypting SWAP partition and taking previously encrypted home partition into use

Due to my personal Ubuntu release update process, I have to configure encryption of my boot and home partition manually every now and then. Because I do this regularly, I'm documenting my process here in my blog...

Here's my Ubuntu release update process.

1. Make Ubuntu installation program to format the partition that held the previous Ubuntu version
2. Format and configure existing boot partition as the new boot partition
3. Configure existing swap partition as the new swap partition
4. Install Ubuntu normally without dedicated home partition

After installation, my encrypted home partition is not accessible anymore, because it was not configured as part of the installation process. I haven't been able to configure either normal installer or alternative installer to take previously encrypted partitions into use. So, it has to be done after installation.

After installation has finished, install cryptsetup.

$ sudo apt-get install cryptsetup

Then configure /etc/fstab file by adding configuration for swap and home. For example.

/dev/mapper/sda7_crypt /home           ext4    defaults        0       2
/dev/mapper/sda5_crypt none            swap    sw              0       0

The actual values depend on your hard disk partitioning. Here's example from my laptop.

$ sudo fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7250e0b9

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048    24588287    12293120   27  Hidden NTFS WinRE
/dev/sda2        24588288    24793087      102400    7  HPFS/NTFS/exFAT
/dev/sda3   *    24793088    26746879      976896   83  Linux
/dev/sda4        26748926   976773119   475012097    5  Extended
/dev/sda5        26748928    34559999     3905536   82  Linux swap / Solaris
/dev/sda6        34562048   132216831    48827392   83  Linux
/dev/sda7       132218880   976773119   422277120   83  Linux

The next step is to configure /etc/crypttab by adding appropriate encryption setup. In my case it looks like this.

sda5_crypt /dev/sda5 /dev/urandom cipher=aes-cbc-essiv:sha256,size=256,swap
sda7_crypt UUID=5a9c38c3-2aa9-433b-9efd-c0e9357d0811 none luks

The swap partition setup is "universal" and it should work on any computer (of course the correct partition may differ from this example). The swap is encrypted with a key that is randomly generated on each system startup.

For the home partition you have to know the UUID of the partition. Here's one way to find it.

$ sudo blkid 
/dev/sda1: LABEL="Recovery" UUID="78F82CACF82C6A98" TYPE="ntfs" 
/dev/sda2: LABEL="System Reserved" UUID="62ACA8E6ACA8B5C7" TYPE="ntfs" 
/dev/sda3: UUID="7d526d52-018a-4b0c-9e26-64f1143cf0da" TYPE="ext4" 
/dev/sda5: UUID="0d0178b1-ade5-416f-8535-82455a8febd5" TYPE="swap" 
/dev/sda6: UUID="b0d7a0b7-0bb8-4cf9-978a-f7c6ebb2126f" TYPE="ext4" 
/dev/sda7: UUID="5a9c38c3-2aa9-433b-9efd-c0e9357d0811" TYPE="crypto_LUKS" 
/dev/sdb1: LABEL="siirtoNTFS" UUID="494640E516B11A6B" TYPE="ntfs" 
/dev/sdb2: LABEL="siirto" UUID="570857a6-4ab3-4d4b-99a2-88d383d3e588" TYPE="ext4" 

With this configuration, the system should ask for home partition's encryption key during system startup and everything should work as before.

Jetty Maven Plugin with hot code replace

I'm often using Jetty Maven plugin to execute my web applications right from the build. It provides an easy and operating system independent way to execute the web app. You only have to check out the source code from version contol, execute Maven build and start Jetty with Maven command. What could be easier way to run your web application in development environment?

I want to point out the following benefits in using Maven Jetty plugin

• IDE independent way to execute web application. Works on Eclipse, IDEA, NetBeans and whatever
• Support hot code replace: you can change code inside methods without restarting the whole application
• Works on Windows, Linux and Mac or any other Java compatible operating system. No need to setup the application server instance
• Starts fast and it's easy to reload the application to the server after changes

Just add the XML snippet in the end of this post to your pom.xml and execute Maven with mvn jetty:run. As a result, you will get response from your web app in http://localhost:8180/example/. You can run the same goal from your favorite IDE and, thus, you get an IDE independent web app execution!

In case you want to try hot code replace, add the classpaths containing your code inside the extraclasspath-element. Then start the Maven build running Jetty in debug mode and connect to the debugging session with your favorite IDE. The easiest way to achieve this in Eclipse is to run the Maven build in debug mode and then connect to the process with Eclipse debugger. After Eclipse is properly connected to the Maven process running Jetty, all the code changes in method bodies are instantly visible in the running process  (of course the same code has to be modified by Eclipse so that the changes get to the classpath of Jetty).

If you prefer doing things in IDE, at least Eclipse has excellent Jetty plugin called Run Jetty Run (http://code.google.com/p/run-jetty-run/).

<build>
<plugins>
...
<plugin>
    <groupid>org.mortbay.jetty</groupid>
    <artifactid>jetty-maven-plugin</artifactid>
    <version>7.5.2.v20111006</version>
    <configuration>
        <stopport>9966</stopport>
        <stopkey>${project.artifactId}<stopkey>
        <!-- scanning is not used if reload is set to manual -->
        <scanintervalseconds>5</scanintervalseconds>
        <!-- application reloading by pressing enter in the console -->
        <reload>manual</reload>
        <webappconfig>
            <contextpath>/example</contextpath>
            <!-- Changes in these classes will be instantly applied to running Jetty process without restart -->
            <extraclasspath>target/classes;../dependant-project/target/classes;../another-dependant-project/target/classes</extraclasspath>
        </webappconfig>
        <!-- directories whose changes cause automated Jetty context reloading, not used if reload is manual -->
        <scantargets>
            <scantarget>../dependant-project/target/classes</scantarget>
        </scantargets>
        <connectors>
            <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
                <port>8180</port>
                <maxidletime>60000</maxidletime>
            </connector>
        </connectors>
        <systemproperties>
            <!-- system properties that are used for running Jetty -->
            <systemproperty>
                <name>some.system.property</name>
                <value>somevalue</value>
            </systemproperty>
        </systemproperties>
    </configuration>
    <dependencies>
        <!-- dependencies added to Jetty's classpath -->
        <dependency>
            <groupid>log4j</groupid>
            <artifactid>log4j</artifactid>
            <version>${log4j.version}</version>
            <type>jar</type>
        </dependency>
    </dependencies>
</plugin>
...
</plugins>
</build>