Remote VisualVM session through SSH tunneling

VisualVM is standard Java profiling&monitoring tool that comes with Oracle's SDK. Probably you knew this already, if you have read this far.

Remote VisualVM session through SSH seems to be quite difficult. I found several instructions on how to do it, but they didn't say what to do exactly, especially in my Amazon EC2 environment. The EC2 host we're monitoring here has public elastic IP as well as private IP address.

VisualVM requires SOCKS tunneling instead of standard SSH port forwarding for some reasons I didn't study too well. And anyway, you don't have to know either to get it working...

VisualVM connection setup

1. Create SOCKS tunnel to remote host (in this example it's an EC2 instance)

   % ssh -D 10000 user@remote.host.com

2. Prepare the process you want to monitor. In this case, we're monitoring a Tomcat process so we're adding the following JAVA_OPTS to the Tomcat process. Notice that the JMX port is set to 9000 and this is the port we're connecting to later.

   JAVA_OPTS="...all_the_required_java_opts_here... -Dcom.sun.management.jmxremote.port=9000 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
   

3. Start the Tomcat with the JAVA_OPTS given above
4. Create permissions.txt for jstatsd with the following content

   grant {
     permission java.security.AllPermission;
   };
   

5. Start jstatsd process

   jstatd -J-Djava.security.policy=permissions.txt
   

6. Start VisualVM on your localhost
7. Navigate to Tools -> Options -> Network and set there the following values
1. SOCKS Proxy: localhost
2. Port: 10000
8. Check the local IP of the remote host by executing ifconfig (assuming Linux as operating system) on the remote host. The output below is edited slightly to hide some details

   % ifconfig 
   eth0      Link encap:Ethernet  HWaddr XX:XX:XX:XX:XX:XX  
             inet addr:10.1.1.1  Bcast:10.1.205.255  Mask:255.255.255.0
   ...
   

9. In VisualVM, add new remote monitoring session from File -> Add Remote Host
1. Host name is the internal IP address of the remote host, in this case 10.1.1.1
10. Right click the created remote host and select Add JMX Connection
1. The correct value in this example is 10.1.1.1:9000
11. Now you're monitoring should start working! SSH was complaining about connection timeouts, but they don't seem to matter too much

What's important to notice is that public elastic IP does not work in VisualVM connection setup!