Mac OS X Lion HTTP Sniffer

Simple command-line sniffer:

sudo tcpdump -s 0 -A -i en1 port 80

Use ifconfig to lookup interface name (i.e. en1).

Running BIRT Reports in Tomcat

Context: You have database and you need to do data analysis: draw charts, build some tables, calculate totals, etc. You want it all to be available over the web and secured with a password.

Your database is any JDBC-supported database (I use MySQL 5.1.49).

Your server is running any OS where Java can run (I use Ubuntu Linux 10.10 available through SSH).

I will show how to implement this using Eclipse BIRT 3.7.

Developer Environment

1. Download "Eclipse IDE for Java and Report Developers" package here.
   Unzip to install.
2. Design new report (I created sales.rptdesign). This is really straightforward.

JDBC Driver. You will need MySQL JDBC driver to create Data Source for report. I got mysql-connector-java-5.1.17-bin.jar here.

Installing the driver in BIRT designer is easy using "Manage drivers..." dialog. But you will probably have problems deploying it to the runtime.

Fonts. You probably won't have any problems with fonts in BIRT designer. And again you will likely have problems with fonts in runtime.

Connection Profile Store. With BIRT 3.7 you can use Connection Profiles to hold database connections. After you've finished designing and testing your report, double click report Data Source to bring properties dialog and create new connection profile store in there. Save it to some file (I saved to planet33_v2.xml).
Here's what I have in there:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<DataTools.ServerProfiles version="1.0">
    <profile autoconnect="No" desc=""
        id="ecc3bc60-d4fd-11e0-957a-e0e31b9a34ee" name="planet33_v2"
        providerID="org.eclipse.datatools.enablement.mysql.connectionProfile">
        <baseproperties>
            <property
                name="org.eclipse.datatools.connectivity.db.connectionProperties"
                value="" />
            <property name="org.eclipse.datatools.connectivity.db.savePWD"
                value="true" />
            <property name="org.eclipse.datatools.connectivity.drivers.defnType"
                value="org.eclipse.datatools.enablement.mysql.5_1.driverTemplate" />
            <property name="jarList"
                value="/usr/local/share/mysql-connector-java-5.1.17-bin.jar" />
            <property name="org.eclipse.datatools.connectivity.db.username"
                value="your_username" />
            <property name="org.eclipse.datatools.connectivity.db.driverClass"
                value="com.mysql.jdbc.Driver" />
            <property name="org.eclipse.datatools.connectivity.db.databaseName"
                value="planet33_v2" />
            <property name="org.eclipse.datatools.connectivity.db.password"
                value="your_password" />
            <property name="org.eclipse.datatools.connectivity.db.version"
                value="5.1" />
            <property name="org.eclipse.datatools.connectivity.db.URL"
                value="jdbc:mysql://127.0.0.1:3306/planet33_v2" />
            <property name="org.eclipse.datatools.connectivity.db.vendor"
                value="MySql" />
        </baseproperties>
        <org.eclipse.datatools.connectivity.versionInfo>
            <property name="server.version" value="5.1.49" />
            <property name="technology.name.jdbc" value="JDBC" />
            <property name="server.name" value="MySQL" />
            <property name="technology.version.jdbc" value="4.0.0" />
        </org.eclipse.datatools.connectivity.versionInfo>
        <driverreference>
            <property name="driverName" value="MySQL JDBC Driver" />
            <property name="driverTypeID"
                value="org.eclipse.datatools.enablement.mysql.5_1.driverTemplate" />
        </driverreference>
    </profile>
</DataTools.ServerProfiles>

Note: I bet you can use JDNI data sources here (and I suppose this is even preferable because of connection pooling, etc.). Please, drop a few lines in comments below with instructions how you do this.

You can now edit XML source of you report and replace /report/data-sources with something like this:

<data-sources>
    <oda-data-source extensionID="org.eclipse.birt.report.data.oda.jdbc.dbprofile"
        name="Planet33 V2 Data Source" id="359">
        <property name="OdaConnProfileName">planet33_v2</property>
        <property name="OdaConnProfileStorePath">../conf/planet33_v2.xml</property>
    </oda-data-source>
</data-sources>

Several things to mention here:

• planet33_v2.xml (Connection Profile Store)
• Check all properties and change them according to your connection.
• Note the jarList property, there you should specify path(s) to where your JDBC drivers located (I copied driver that I've downloaded to /usr/local/share/mysql-connector-java-5.1.17-bin.jar).
• When you create connection profile store file from designer it places property with name="org.eclipse.datatools.connectivity.driverDefinitionID". You should remove this property because of this issue.
• sales.rptdesign (The Report)
• You should keep value of ida-data-source@id attribute the same that was in your design.
• Value of OdaConnProfileName should match value of DataTools.ServerProfiles/profile@name attribute from planet33_v2.xml.
• Note that OdaConnProfileStorePath is relative path (see below). But you can keep it absolute if you want. 

Server Environment

(Note: I recommend to configure Tomcat instance on your developer machine first to make it easier to verify report settings, and then transfer the entire $CATALINA_HOME to production server. Of course, you can do all these steps on production server directly.)

1. Download Apache Tomcat (any Java application server should be fine).
   Unzip to some folder (I used /usr/local/share/apache-tomcat-5.5.33/) -- this will be $CATALINA_HOME.
2. Download BIRT "Runtime" package here.
   Copy birt.war (BIRT Web Viewer application) to $CATALINA_HOME/webapps.
3. Edit $CATALINA_HOME/catalina.sh and paste these lines somewhere after JAVA_OPTS variable initialized (this prepares workspace for DTP plugin):
   

   java_io_tmpdir=$CATALINA_HOME/temp
   org_eclipse_datatools_workspacepath=$java_io_tmpdir/workspace_dtp
   mkdir -p $org_eclipse_datatools_workspacepath
   
   JAVA_OPTS="$JAVA_OPTS -Dorg.eclipse.datatools_workspacepath=$org_eclipse_datatools_workspacepath"
   
   

4. Start Tomcat by running $CATALINA_HOME/startup.sh. After this BIRT Report Viewer application should be available by http://localhost:8080/birt. Also birt.war should be now extracted to $CATALINA_HOME/webapps/birt -- this will be $BIRT_HOME. You can now delete $CATALINA_HOME/webapps/birt.war.
5. Copy planet33_v2.xml to $CATALINA_HOME/conf as (remember OdaConnProfileStorePath property in sales.rptdesign file?).
6. Copy your sales.rtpdesign file to $BIRT_HOME.

At this point you should be able to execute the report by simply following the address http://localhost:8080/birt/frameset?__report=sales.rptdesign&__dpi=600.

Note the __dpi URL parameter -- it controls DPI of chart images rendered in HTML/PDF. You will probably want to modify like this it to increase image quality. Also note that if you set chart output format to SVG you will get vector graphics quality in PDF output.

Security

There are obvious reasons why you may want to keep your reports secure.

Besides that keep in mind that in BIRT Web Viewer application all reports sources (*.rptdesign files) available to user by request. Try navigating to http://localhost:8080/birt/sales.rptdesign and you'll see what I mean. I think this is a good reason, why you should use connection profile store (or at least JNDI data sources), because that files not available through HTTP.

Implementing simple (HTTP BASIC AUTH) security with Tomcat is pretty simple.

First, modify $BIRT_HOME/WEB-INF/web.xml, by adding this (you can change role names as you want):

<!-- Define a security constraint on this application -->
<security-constraint>
  <web-resource-collection>
    <web-resource-name>Entire Application</web-resource-name>
    <url-pattern>/*</url-pattern>
  </web-resource-collection>
  <auth-constraint>
    <!-- This role is not in the default user directory -->
    <role-name>manager</role-name>
  </auth-constraint>
</security-constraint>             
<!-- Define the login configuration for this application -->
<login-config>
  <auth-method>BASIC</auth-method>
  <realm-name>BIRT Report Viewer</realm-name>
</login-config>
<!-- Security roles referenced by this web application -->
<security-role>
  <description>
    The role that is required to log in to the BIRT Report Viewer
  </description>
  <role-name>manager</role-name>
</security-role>

You may also do the same for $CATALINA_HOME/conf/web.xml to secure all applications in this Tomcat instance.
Second, you should edit $CATALINA_HOME/conf/tomcat-users.xml to define user login and password.

Thats all, you're secured :) This is should be fine for most cases, but I would recommend you to read about HTTPS if your data is extremely secure.

Deploy to server

1. Copy Tomcat to the server:
   Tip: Use scp command in terminal to transfer files from your machine to the server over SSH:
   scp /usr/local/share/apache-tomcat-5.5.33 dmitrygusev@planet33.ru:/usr/local/share/
2. Copy JDBC Driver to the server:
• Copy this driver to the same path as specified in the jarList property from planet33_v2.xml file.
• DO NOT COPY this driver to $BIRT_HOME/WEB-INF/lib, because it may lead to ClassNotFoundException.

Fix file permissions. When you copy files over scp you may need to chmod them to grant read/execute access. This should fix it:

chmod a+r /usr/local/share/mysql-connector-java-5.1.17-bin.jar
chmod -R a+r /usr/local/share/apache-tomcat-5.5.33/
chmod a+x /usr/local/share/apache-tomcat-5.5.33/bin/*.sh

Now you should be able to start tomcat and run reports on the server.

Fonts. BIRT PDF output doesn't work good for Russian fonts out-of-the-box, because of licensing issues with fonts. One simple solution to fix this is:

1. Get the *.ttf font files you need (you can copy them from any Windows installation, look in c:\Windows\Fonts). These 8 files should be enough in most cases (these are "Arial" and "Times New Roman" fonts):
   
   arialbd.ttf  arialbi.ttf  ariali.ttf  arial.ttf
   timesbd.ttf  timesbi.ttf  timesi.ttf  times.ttf
2. Copy these files to /usr/share/fonts/truetype (or any other place that is referenced from fontsConfig.xml).
3. Don't forget to fix file permissions:
   chmod a+r /usr/share/fonts/truetype/*.ttf
4. Reference the fonts from *.rptdesign (or configure font-aliases):
   /report/styles

   <style name="report" id="4">
       <property name="fontFamily">"Arial"</property>
       <property name="fontSize">9pt</property>
   </style>
   

5. Restart Tomcat:
• $CATALINA_HOME/bin/shutdown.sh
• $CATALINA_HOME/bin/startup.sh

Russian Localization. I used this method to do it. Although you may want to try BIRT Language Packs.

Troubleshooting.

• Neither the JAVA_HOME nor the JRE_HOME environment variable is defined
  At least one of these environment variable is needed to run this program
  
  You should define JAVA_HOME variable. Execute this command before running Tomcat's *.sh files in terminal:
  
  export JAVA_HOME=/usr/bin/java

• If you get OutOfMemoryError you may want to give JVM more memory. Edit $CATALINA_HOME/bin/catalina.sh to include this (see this thread on stackoverflow, and read more about JVM memory settings):
  
  JAVA_OPTS="$JAVA_OPTS -Xms512m -Xmx512m -XX:MaxPermSize=256m"

• If you got OutOfMemoryError you most likely couldn't restart Tomcat using $CATALINA_HOME/bin/shutdown.sh script.
  
  To kill Tomcat instance use htop command in terminal. In htop interface select Tomcat process (this is /usr/lib/java), press 'k', select 9 SIGKILL in "Send signal" area, and press Enter. To exit htop press 'q'.
• Executing report never stops. Tomcat process consumes all CPU resources.
  
  I've seen this situation when used charts in report and they were on page break. I fixed this by moving charts to other place (far from page break). Changing page size to avoid page breaks also fixes this issue.

Deploy SharePoint Designer 2010 Reusable Workflow As *.WSP File

SharePoint Designer 2010 makes workflow development really fast and simple. Much simpler than using Visual Studio. But unlike Visual Studio this tool has some limitations from developer's point of view.

This is due to SPD was created as a tool for end (SharePoint) users, but not for solution developers. As a result we have one serious limitation that prevents developers to use this tool: it doesn't allow deploy created workflows to another SharePoint servers. Which means you cannot develop and test workflows on a development SharePoint server and then move it to production. You forced to develop in production, which is not right.

In particular what I said is true for reusable workflows that work with custom list instances. The problem is once you reference some list in workflow, SPD will link workflow template (*.xoml) to this list using its ListId attribute which is unique identifier that is valid for that particular site. This ListId is a random value that SharePoint generates when the list deployed. Note that you deploy list instances not only when staging ready solution from development to production, but also repeatedly during development cycle.

There are two ways you can notice your workflow corrupted. The first is in SPD you will see GUIDs instead of list names, and if you click these GUIDs you'll see unbinded dialogs:




And if you deploy such workflow to another SharePoint site (with prior export to *.WSP) you will get the error like this (in SharePoint logs) when try to run it:

SOAP exception: System.Runtime.InteropServices.COMException (0x82000006): List does not exist. The page you selected contains a list that does not exist. It may have been deleted by another user.
at Microsoft.SharePoint.SoapServer.SPBaseImpl.GetSPListByTitle(SPWeb spWeb, String strListName)
at Microsoft.SharePoint.SoapServer.SPBaseImpl.GetSPList(SPWeb spWeb, String strListName, Boolean bGetMetaData, Boolean bGetSecurityData)
at Microsoft.SharePoint.SoapServer.SPBaseImpl.GetSPList(String strListName, Boolean bGetMetaData, Boolean bGetSecurityData)
at Microsoft.SharePoint.SoapServer.ListSchemaImpl.GetList(String strListName)
at Microsoft.SharePoint.SoapServer.ListSchemaValidatorImpl.GetList(String strListName)
at Microsoft.SharePoint.SoapServer.Lists.GetList(String listName)


The most common approach you may find on the Internet is to export SPD reusable workflow to *.WSP solution package, import that *.WSP to Visual Studio and continue development there. I don't like this approach for two reasons: first is once you do this you can't open that workflow in SharePoint designer again to made any changes (and you don't want to change it in Visual Studio because, like every auto-generated code, SPD generated *.xoml is not very human-friendly):


And the second (which I'm not sure, though)---you can't deploy such workflow as a sandboxed solution (correct me if I wrong).

Fortunately, there is nothing that prevents us from deploying SPD reusable workflows except ListIds. All we need to do is replace broken Ids with the new ones. Here's how you may do this:

1. Export reusable workflows to *.WSP files using SPD 2010
2. To fix ListIds change contents of the process*.xoml file contained in *.WSP file (which is *.CAB file that contain (inter alia) Feature.xml)
   To extract and package contents of *.WSP I recommend to use PowerShell + built-in expand command and WSPBuilder's CabLib.dll accordingly
3. *.xoml is an regular text/xml file so we can simply find and replace GUID strings
4. We know what GUIDs to be replaced by examining contents of the *.xoml file.
   Look for entries like this:
   

   <ns1:LookupActivity ListId="{}{909E9DFD-A30B-4E28-BF2E-5BA47095967D}"  
                       x:Name="ID10" FieldName="ID" LookupFunction="LookupInt"  
                       __Context="{ActivityBind ROOT,Path=__context}"  
                       ListItem="{ActivityBind ID11,Path=ReturnValue}" /> 
   

   
   
5. We know the replacement for old GUIDs by using PowerShell automation: get SPWeb object of site where we want to deploy the workflow, get list in that web by list title, get ID of that list and convert that ID to string. Note that GUID string should be in upper case, otherwize you won't be able to edit workflow in SPD, though it will run okay on site (thats probably SPD bug)
6. After replacing GUIDs we create new *.WSP with relevant ListIds which may be deployed to SharePoint

Below is sample PowerShell script that you can use as a reference to implement steps described above. To run it save contents to file, say Deploy-Workflows.ps1, change values of $siteUrl, $wspDir and $listIds to match your environment. You will also have to place CabLib.dll and AnjLab-SharePoint.ps1 files to the same folder as Deploy-Workflows.ps1. After that open SharePoint 2010 Management Shell, CD to directory with Deploy-Workflows.ps1 and run the script with command .\Deploy-Workflows.ps1.

# Allow running *.ps1 scripts from network shares 
# Set-ExecutionPolicy Unrestricted 

# Copy CabLib.dll to user's temp (to prevent security exceptions if your project files are on network share) 
# Note: $cablibFullName is a global variable used in AnjLab-SharePoint.ps1 
$cablibFullName = "$env:TEMP\CabLib.dll" 
if ((test-path $cablibFullName) -eq $false) 
{ 
    cp "CabLib.dll" $env:TEMP 
} 

# Import AnjLab-SharePoint functions 
. .\AnjLab-SharePoint.ps1 

# Replace with yours 

$siteUrl         = "http://dev-en/gls/"     # SharePoint site to deploy *.WSP workflows to 
$wspDir          = "bin\Debug\Workflows"    # Directory with *.WSP files  
                                            # (all files from this folder will be updated and deployed) 
$wspTempDir      = "$wspDir\temp"           # Temp directory 
$wspFinalDir     = "$wspDir\final"          # Directory where final *.WSP files will be placed

################################################################################################### 
# Define mapping for GUID replacement in the hashtable below. 
# All GUIDs that match keys from this hashtable will be replaced with corresponding GUIDs of lists
# taken from $siteUrl by specified list titles. 
################################################################################################### 

$listIds = @{ #  ListId in workflow's *.WSP              List Title on SharePoint Site
              #  -------------------------------------   ----------------------------- 
                "909E9DFD-A30B-4E28-BF2E-5BA47095967D" = "Consumers"; 
                "2F311150-7360-45DA-A4B1-C64339F3B931" = "Warehouses"; 
                "435E8D1B-FC3F-42A9-B761-1958A31D9BDE" = "Leads"; 
            } 

# Replace List Ids 

$wspFiles = (Get-ChildItem "$wspDir\*.wsp") 
Update-WspListIds $siteUrl $wspFiles $wspTempDir $wspFinalDir $listIds

# Deploy Packages 

$wspFiles = (Get-ChildItem "$wspFinalDir\*.wsp") 
Deploy-Wsp $siteUrl $wspFiles $wspTempDir

Write-Host "Done" 

Add Interactivity to ASP.NET ReportViewer

In one of our projects I had a task to add interactivity to charts located on (MS Report Server) reports that were viewed in ASP.NET web application using ReportViewer control.

Charts I worked with contained date series and user should had have the ability to add notes to data in series by clicking on that data on chart.

In the final solution I used jQuery to find images (charts) in rendered report HTML that had special value encoded in image's alt tag. Client javascript (which was on the same page as ReportViewer control) made AJAX requests to server to calibrate chart axes. Then calibration result was used on client side to build HTML MAP with AREAs to which jQuery onclick handlers were attached.

Here's how working example looked in browser:



Client side implementation was written in pure jQuery with help of jQuery qTip plugin and takes around 450 lines of code. Nothing interesting.

The most interesting part in this approach was getting chart image from generated report on server side.

To get image on server side I used the same mechanism that ReportViewer web control uses. I must to say that I couldn't do this without Red Gate's .Net Reflector (I used free version, it was enough here). This is really helpful tool that makes (nearly all) .Net libraries open sourced. I highly recommend it.

Browser obtains chart image by URL, so I had to pass that URL to server side to grab that image from there.

            function calibrate($image) {
                $.ajax({
                    type: "POST",
                    contentType: "application/json; charset=utf-8",
                    url: options.calibrateAxesUrl,
                    data: "{imageUrl: '" + $image.attr("src") + "'}",
                    dataType: "json",
                    success: function(msg) {
                        calibrationResult = msg.d;

                        // process calibration result...
                    }
                });
            }

As you see I used image's src tag to get image URL and invoked web method CalibrateAxes (which is a public static method with [WebMethod] annotation) of my ASP.NET page (the full URL looks like '<%= ResolveClientUrl("~/Reporting/ReportViewer.aspx") + "/CalibrateAxes" %>').

CalibrateAxes method gets chart image as bitmap and does some bitmap analysis to form the result:

        [WebMethod]
        public static CalibrationResult CalibrateAxes(string imageUrl)
        {
            byte[] image = RenderImage(imageUrl);

            using (var stream = new MemoryStream(image))
            using (var bmp = new Bitmap(stream))
            {
                var result = new CalibrationResult();

                // analyze bitmap...

                return result;
            }
        }

        private static byte[] RenderImage(string imageUrl)
        {
            var imageUri = new Uri(HttpContext.Current.Request.Url.GetLeftPart(UriPartial.Authority) + imageUrl);

            var parameters = HttpUtility.ParseQueryString(imageUri.Query);

            var reportSession = parameters["ReportSession"];
            var controlID = parameters["ControlID"];
            var culture = parameters["Culture"];
            var uiCulture = parameters["UICulture"];
            var reportStack = parameters["ReportStack"];
            var streamID = parameters["StreamID"];

            var rs = new ReportExecutionService
                         {
                             Url = ConfigurationManager.AppSettings["ReportExecutionServiceUrl"],
                             Credentials = new NetworkCredential(
                                 ConfigurationManager.AppSettings["ReportServerUserName"],
                                 ConfigurationManager.AppSettings["ReportServerUserPassword"],
                                 ConfigurationManager.AppSettings["ReportServerUserDomain"]),
                             ExecutionHeaderValue = new ExecutionHeader {ExecutionID = reportSession}
                         };

            string deviceInfo = GetDeviceInfo(reportSession, controlID, culture, uiCulture, reportStack);

            string encoding;
            string mimetype;

            return rs.RenderStream("HTML4.0", streamID, deviceInfo, out encoding, out mimetype);
        }

        private static string GetDeviceInfo(string reportSession, string controlID, string culture, string uiCulture, string reportStack)
        {
            var writer = new StringWriter();
            var xmlWriter = new XmlTextWriter(writer);
            xmlWriter.WriteStartElement("DeviceInfo");
            var url = CreateUrl(reportSession, controlID, culture, uiCulture, reportStack);
            xmlWriter.WriteElementString("StreamRoot", url);
            xmlWriter.WriteEndElement();
            return writer.ToString();
        }

        private static string CreateUrl(string reportSession,
                                        string controlID,
                                        string culture,
                                        string uiCulture,
                                        string reportStack)
        {
            var uriBuilder = new UriBuilder(HttpContext.Current.Request.Url.GetLeftPart(UriPartial.Authority));
            var applicationPath = HttpContext.Current.Request.ApplicationPath;
            if (!applicationPath.EndsWith("/", true, CultureInfo.InvariantCulture))
            {
                applicationPath = applicationPath + "/";
            }
            applicationPath = applicationPath + "Reserved.ReportViewerWebControl.axd";
            applicationPath = HttpContext.Current.Response.ApplyAppPathModifier(applicationPath);
            uriBuilder.Path = applicationPath;

            var builder = new StringBuilder();
            builder.AppendFormat("{0}={1}", "ReportSession", reportSession);
            builder.AppendFormat("&{0}={1}", "ControlID", HttpUtility.UrlEncode(controlID));
            builder.AppendFormat("&{0}={1}", "Culture", culture);
            builder.AppendFormat("&{0}={1}", "UICulture", uiCulture);
            builder.AppendFormat("&{0}={1}", "ReportStack", reportStack);
            builder.Append("&OpType=ReportImage&StreamID=");
            uriBuilder.Query = builder.ToString();

            return uriBuilder.Uri.PathAndQuery;
        }

This code of getting rendered image from report turned to be very generic, so you may also use it in your web applications.

Tapestry5: Caching Method Results

Assume you have methods that (almost) always return the same result for the same input arguments. If preparing method result is a heavy operation and/or it consumes time, it is reasonable to cache these results.

One way of building method cache in Tapestry5 is by implementing MethodAdvice interface like this:

public class CacheMethodResultAdvice implements MethodAdvice {

    private static final Logger logger = LoggerFactory.getLogger(CacheMethodResultAdvice.class);
    
    private final Cache cache;
    private final Class<?> advisedClass;
    private final Object nullObject = new Object();
    
    public CacheMethodResultAdvice(Class<?> advisedClass, Cache cache) {
        this.advisedClass = advisedClass;
        this.cache = cache;
    }
    
    @Override
    public void advise(Invocation invocation) {
        String invocationSignature = getInvocationSignature(invocation);
        
        String entityCacheKey = String.valueOf(invocationSignature.hashCode());

        Object result;
        
        if (cache.containsKey(entityCacheKey))
        {
            result = cache.get(entityCacheKey);

            logger.debug("Using invocation result ({}) from cache '{}'", invocationSignature, result);

            invocation.overrideResult(result);
        }
        else 
        {
            invocation.proceed();
            
            if (!invocation.isFail())
            {
                result = invocation.getResult();
                
                cache.put(entityCacheKey, result);
            }
        }
    }

    private String getInvocationSignature(Invocation invocation) {
        StringBuilder builder = new StringBuilder(150);
        builder.append(advisedClass.getName());
        builder.append('.');
        builder.append(invocation.getMethodName());
        builder.append('(');
        for (int i = 0; i < invocation.getParameterCount(); i++) {
            if (i > 0) {
                builder.append(',');
            }
            Class<?> type = invocation.getParameterType(i);
            builder.append(type.getName());
            builder.append(' ');

            Object param = invocation.getParameter(i);
            builder.append(param != null ? param : nullObject);
        }
        builder.append(')');
        
        return builder.toString();
    }
    
}

Implementation of getInvocationSignature(...) is not ideal, but you may improve it to match your requirements. One issue I see here is building invocation signature for null-value parameters in a clustered environment (which is GAE). In this implementation method nullObject.toString() will return something like java.lang.Object@33aa9b. And this value will vary in different instances of your application. You may replace nullObject with just "null" string. Just keep in mind that "null" != null.

To make this advice working you should declare it in your AppModule.java:

    @SuppressWarnings("unchecked")
    @Match("IPResolver")
    public static void adviseCacheIPResolverMethods(final MethodAdviceReceiver receiver, Logger logger, PerthreadManager perthreadManager) {
        try {
            Map props = new HashMap();

            //  IP address of URL may change, keep it in cache for one day
            props.put(GCacheFactory.EXPIRATION_DELTA, 60 * 60 * 24);
            
            CacheFactory cacheFactory = CacheManager.getInstance().getCacheFactory();
            Cache cache = cacheFactory.createCache(props);
            
            LocalMemorySoftCache cache2 = new LocalMemorySoftCache(cache);
            
            //  We don't want local memory cache live longer than memcache
            //  Since we don't have any mechanism to set local cache expiration
            //  we will just reset this cache after each request
            perthreadManager.addThreadCleanupListener(cache2);
            
            receiver.adviseAllMethods(new CacheMethodResultAdvice(IPResolver.class, cache2));
        } catch (CacheException e) {
            logger.error("Error instantiating cache", e);
        }
    }

    @Match("LocationResolver")
    public static void adviseCacheLocationResolverMethods(final MethodAdviceReceiver receiver, Cache cache) {
        //  Assume that location of IP address will never change, 
        //  so we don't have to set any custom cache expiration parameters
        receiver.adviseAllMethods(new CacheMethodResultAdvice(LocationResolver.class, cache));
    } 

These declarations tell Tapestry5 to add our advice to all methods of services that implement IPResolver and LocationResolver interfaces.

Note that we able to use caches with different settings for different methods/services like in example above (see comments in code).

See also:

• Tapestry5 Service Advisors


• Caching in Tapestry5 Data Access Layer


• How To Determine Client TimeZone In A Web Application

How To Determine Client TimeZone In A Web Application

In web applications when client and server located in different timezones we need a way to determine client timezone to display date/time sensitive information. This is almost always true for Google Appengine, where default server time zone is UTC.

There are several ways to determine client timezone.

One of them is resolving client IP address to location:

1. Get client IP


2. Get client location (latitude, longitude) by the IP-address


3. Get information about timezone by the location coordinates

Every web framework provides API to get client IP. For instance, in java there is a method ServletRequest.getRemoteAddr() for this purpose.

To resolve IP and location information you can use one of the numerous web services available online.

For instance, to resolve IP to location Ping Service uses IP-whois.net service.

Another service, Geonames.org provides web service API to get timezone information by latitude/longitude pair.

Here's an implementation of described approach in java:

    private TimeZone getTimeZoneByClientIP() {
        TimeZone timeZone = UTC_TIME_ZONE;
        
        try {
            String clientIP = globals.getHTTPServletRequest().getRemoteAddr();
            
            if (!Utils.isNullOrEmpty(clientIP)) {
                Location location = locationResolver.resolveLocation(clientIP);
                
                if (!location.isEmpty()) {
                    timeZone = timeZoneResolver.resolveTimeZone(location.getLatitude(), location.getLongitude());
                }
                
                if (timeZone == null) {
                    timeZone = UTC_TIME_ZONE;
                }
            }
            
            logger.debug("Resolved timeZoneId is {}", timeZone.getID());
        } catch (Exception e) {
            logger.error("Error resolving client timezone by ip " 
                    + globals.getHTTPServletRequest().getRemoteAddr(), e);
        }
        
        return timeZone;
    }

The disadvantages using this approach are:

• Your code becomes dependent on third party online services that are not 100% reliable


• Requesting third party services online will take time (up to several seconds) which may result in long response time

Note: according to Ping Service statistics IP-Whois.net availability is close to 100% with average response time ~270 ms, while Geonames.org availability is only around 80% with average response time ~1100 ms. Geonames.org low level availability is due to GAE hosting: Geonames.org restricts free access to its API to 3000 requests per IP per hour.

On the other hand you have really simple solution to implement that allows to determine client timezone at the very first client request so you can display all date/time sensitive data using client local time.

See also:

• DISCUSSION: Time zones and date selection


• Ping Service implementation of described approach




• IPWhoisNetLocationResolver.java


• GeonamesTimeZoneResolver.java

Update: GAE 1.6.5 introduces some request headers which already contains Lat/Lng pair for incoming request: https://developers.google.com/appengine/docs/java/runtime#Request_Headers

Profiling GAE API calls

While optimizing performance of GAE application its convenient to measure GAE API calls.

I'm using the following implementation of com.google.apphosting.api.ApiProxy.Delegate to do this:

public class ProfilingDelegate implements Delegate<Environment> {

    private static final Logger logger = LoggerFactory.getLogger(ProfilingDelegate.class);
    
    private final Delegate<Environment> parent;
    private final String appPackage;
    
    public ProfilingDelegate(Delegate<Environment> parent, String appPackage) {
      this.parent = parent;
      this.appPackage = appPackage;
    }
    
    public void log(Environment env, LogRecord logRec) {
        parent.log(env, logRec);
    }
    
    @Override
    public byte[] makeSyncCall(Environment env, String pkg, String method, byte[] request) throws ApiProxyException {
        long start = System.currentTimeMillis();
        byte[] result = parent.makeSyncCall(env, pkg, method, request);
        StringBuilder builder = buildStackTrace(appPackage);
        logger.info("GAE/S {}.{}: ->{} ms<-\n{}", new Object[] { pkg, method, System.currentTimeMillis() - start, builder });
        return result;
    }

    /**
     * 
     * @param appPackage
     *        Only classes from this package would be included in trace.
     * @return
     */
    public static StringBuilder buildStackTrace(String appPackage) {
        StackTraceElement[] traces = Thread.currentThread().getStackTrace();
        StringBuilder builder = new StringBuilder();
        int length = traces.length;
        StackTraceElement traceElement;
        String className;
        for (int i = 3; i < length; i++) {
            traceElement = traces[i];
            className = traceElement.getClassName();
            if (className.startsWith(appPackage)) {
                if (builder.length() > 0) {
                    builder.append('\n');
                }
                builder.append("..");
                builder.append(className.substring(className.lastIndexOf('.')));
                builder.append('.');
                builder.append(traceElement.getMethodName());
                builder.append(':');
                builder.append(traceElement.getLineNumber());
            }
        }
        if (builder.length() == 0) {
            for (int i = 1; i < length; i++) {
                traceElement = traces[i];
                className = traceElement.getClassName();
                if (builder.length() > 0) {
                    builder.append('\n');
                }
                builder.append(className);
                builder.append('.');
                builder.append(traceElement.getMethodName());
                builder.append(':');
                builder.append(traceElement.getLineNumber());
            }
        }
        return builder;
    }
    
    @Override
    public Future<byte[]> makeAsyncCall(Environment env, String pkg, String method, byte[] request, ApiConfig config) {
        long start = System.currentTimeMillis();
        Future<byte[]> result = parent.makeAsyncCall(env, pkg, method, request, config);
        StringBuilder builder = buildStackTrace(appPackage);
        logger.info("GAE/A {}.{}: ->{} ms<-\n{}", new Object[] { pkg, method, System.currentTimeMillis() - start, builder });
        return result;
    }
}

To register this delegate add the following code to prior to any API calls, i.e. to filter init() method:

    public void init(FilterConfig config) throws ServletException
    {
        this.config = config;
        //  Note: Comment this off to profile Google API requests
        ApiProxy.setDelegate(new ProfilingDelegate(ApiProxy.getDelegate(), "dmitrygusev"));
    }

Here's an example of log output:

02.09.2010 0:22:19 dmitrygusev.tapestry5.gae.ProfilingDelegate makeSyncCall
INFO: GAE/S datastore_v3.BeginTransaction: ->1076 ms<-
...LazyJPATransactionManager$1.assureTxBegin:48
...LazyJPATransactionManager$1.createQuery:137
...AccountDAOImpl.findByEmail:36
...AccountDAOImpl.getAccount:26
...AccountDAOImplCache.getAccount:36
...Application.getUserAccount:395
...Application.trackUserActivity:400
...AppModule$1.service:229
...AppModule$2.service:291
...LazyTapestryFilter.doFilter:62
02.09.2010 0:22:19 dmitrygusev.tapestry5.gae.LazyJPATransactionManager$1 assureTxBegin
INFO: Transaction created (1200 ms) for context ...AccountDAOImpl.findByEmail:36
...AccountDAOImpl.getAccount:26
...AccountDAOImplCache.getAccount:36
...Application.getUserAccount:395
...Application.trackUserActivity:400
...AppModule$1.service:229
...AppModule$2.service:291

See also GAE and Tapestry5 Data Access Layer