I found out today that MS Sql server seems to handle Unicode in a very special way. Instead of having some support a database or table level, each Unicode column have to be created as “national”. That is be either nchar, nvarchar or ntext.

Ms SQL Server 2005 seems to go one step further by announcing future deprecation for ntext, text and image types.

From Sql Server 2005 notes:

“ntext, text, and image data types will be removed in a future version of Microsoft SQL Server. Avoid using these data types in new development work, and plan to modify applications that currently use them. Use nvarchar(max), varchar(max), and varbinary(max) instead.”

When working with Hibernate it seems there is no dialect to handle Unicode integration properly. You have to get down and write a custom dialect that maps to the new data types.

/**
 * Unicode support in SQL Server
 *
 * @author icocan
 */
public class UnicodeSQLServerDialect extends SQLServerDialect {

    public UnicodeSQLServerDialect() {
        super();

        // Use Unicode Characters
        registerColumnType(Types.VARCHAR, 255, "nvarchar($l)");
        registerColumnType(Types.CHAR, "nchar(1)");
        registerColumnType(Types.CLOB, "nvarchar(max)");

        // Microsoft SQL Server 2000 supports bigint and bit
        registerColumnType(Types.BIGINT, "bigint");
        registerColumnType(Types.BIT, "bit");
    }
}

Share/Save

The natural sort order for String fields is 0-9 A-Z so it seems a custom sorter is needed. I guess this is a normal request for any application that does a little sorting so I am surprised nothing came up during a search. Here is the AZ09Comparator:

/**
 * Custom Lucene sorting A-Z 0-9
 *
 * @author icocan
 */
public class AZ09Comparator implements SortComparatorSource {

    private static final Logger logger = Logger.getLogger(AZ09Comparator.class);

    public ScoreDocComparator newComparator(final IndexReader indexReader, final String str) throws IOException {
        return new ScoreDocComparator() {
            public int compare(ScoreDoc scoreDoc1, ScoreDoc scoreDoc2) {
                try {
                    final Document doc1 = indexReader.document(scoreDoc1.doc);
                    final Document doc2 = indexReader.document(scoreDoc2.doc);
                    final String strVal1 = doc1.get(str);
                    final String strVal2 = doc2.get(str);

                    boolean c1 = Character.isDigit(strVal1.charAt(0));
                    boolean c2 = Character.isDigit(strVal2.charAt(0));

                    if (c1 && c2) {
                        return strVal1.compareTo(strVal2);
                    } else if (c1) {
                        return 1;
                    } else if (c2) {
                        return -1;
                    } else {
                        return strVal1.compareTo(strVal2);
                    }
                } catch (IOException e) {
                    logger.error("Cannot read doc", e);
                }
                return 0;
            }

            public Comparable sortValue(ScoreDoc scoreDoc) {
                return new Float(scoreDoc.doc);
            }

            public int sortType() {
                return SortField.CUSTOM;
            }
        };
    }
}

Share/Save

I thought of sharing with you this little problem that I encountered when using a TreeSet to display an ordered list of items:

I have some items stored in the database, each of them having a unique id.

Some of the items have the same values, only the ids are different.

I also have a business method that creates a list of items, sorted by some custom ordering of fields.

To sort them, I used the following lines of code:

SortedSet result = new TreeSet(itemsComparator);
for (Item item : someItems) {
    if (itemIsEligible(item)) {
        result.add(item);
    }
}

where itemsComparator is an instance of this class:

public class ItemsComparator implements Comparator {
    /**
     * Compare two items for order.
     * @param item1 the first item
     * @param item2 the second item
     * @return -1 if item1 should appear before item2,
               1 if item2 should appear before item1,
               or 0 if the order is not important
     */
    public int compare(Item item1, Item item2) {
        // Sort by field1 descending
        int compareField1 = item1.getField1().compareTo(item2.getField1());
        if (compareField1 != 0) {
            return -compareField1;
        }
        // For equal field1, sort by field2 ascending
        int compareField2 = item1.getField2().compareTo(item2.getField2());
        return compareField2;
    }
}

At some moment I discovered the following bug: if two items had the same values, but different ids, only the last one of them appeared in the result.

After a little debugging I discovered that the problem was this:

TreeSet uses internally a TreeMap to store the elements.

TreeMap uses compareTo to determine the order of the elements, to know where to place them in the internal tree.

And if two elements are different but compareTo returns 0, then TreeMap will consider them equal, and the newly added element replaces the old one.

My solution to this problem was the following:

public class ItemsComparator implements Comparator {
    /**
     * Compare two items for order.
     * To ensure that two different items are not treated as equal,
     * our method will never return 0 for different items!
     * @param item1 the first item
     * @param item2 the second item
     * @return -1 if item1 should appear before item2,
                    1 if item2 should appear before item1,
                    0 if item1 and item2 are the same
     */
    public int compare(Item item1, Item item2) {
        // Sort by field1 descending
        int compareField1 = item1.getField1().compareTo(item2.getField1());
        if (compareField1 != 0) {
            return -compareField1;
        }
        // For equal field1, sort by field2 ascending
        int compareField2 = item1.getField2().compareTo(item2.getField2());
        if (compareField2 != 0) {
            return compareField2;
        }
        // For equal field1 and field1, sort by item id
        return item1.getId().compareTo(item2.getId());
    }
}

technorati tags:TreeSet, Comparator, Problem

Share/Save

Changes in Hibernate Search are quite frequent, as it is in continued development. Today I found myself looking into a problem generated by migration from Hibernate-Lucene integration in 3.2.1.GA to the new Hibernate Search 3.0.0.Beta1.

The problem: deletion of an object in Hibernate will not trigger the removal from the Lucene index.

First, check the event listener: using a custom implementation I can see the onPostDelete method is called correctly. Still, no deletion from index and no error message either. Looking in the Hibernate Search sources I can see that the deletion uses the DocumentId marker in order to identify the field used for deletion. The old @Keyword(id=true) is deprecated and seems it no longer works.

The solution: Change the annotation from @Keyword(id=true) to @DocumentId. This will help Hibernate search do its magic.

Share/Save

I’ve been bugged by this problem for some time now, so I decided investigating a bit. The error shows up when tomcat is trying to reload the web application.


The full stacktrace is shown below:


SEVERE: Error reading tld listeners java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.log4j.Category.isEnabledFor(Category.java:746)
at org.apache.commons.logging.impl.Log4JLogger.isTraceEnabled(Log4JLogger.java:327)
at org.apache.catalina.startup.TldConfig.tldScanResourcePaths(TldConfig.java:582)
at org.apache.catalina.startup.TldConfig.execute(TldConfig.java:283)
at org.apache.catalina.core.StandardContext.processTlds(StandardContext.java:4307)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4144)
at org.apache.catalina.core.StandardContext.reload(StandardContext.java:3025)
at org.apache.catalina.loader.WebappLoader.backgroundProcess(WebappLoader.java:432)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1278)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1570)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1579)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1579)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1559)
at java.lang.Thread.run(Thread.java:595)


The solution is quite simple: replace commons-logging.jar from your 1.1 version to a previous version (1.0.4). That’s it. The problem has been discussed here.

technorati tags:java, tomcat, reload, application, commons-loggin

Share/Save

Seeing this thread on DWR mailing list, I though I’d post one way to deal with DWR response. The actual problem was initiated by someone asking what’s the best way to display status messages to users using DWR.

Lance Semmens describes a solution using a template like:

public class Result<T> {

    private boolean success;

    private T successObject;

    private Map<String,List<String,String>> messages;



    // getters and setters

}

The object obviously needs a DWR converter so that it can be converted to javascript. When an instance of this class gets returned to the js code, the function must check if the operation has been successful and display any errors included in the Result object.

The problem with this approach is that every javascript function used as a DWR callback needs to have the same piece of code:

if(!obj.success) {

    // display error messages

}

When maintaining a large number of DWR callbacks, maintaining the same code in all DWR callbacks is obviously not a choice. Wouldn’t it be useful to have some sort of AOP interceptors for every js callback, dealing with the error messages in an unified and transparent manner?

For this, I’d suggest using something similar to dojo’s event system to intercept all callbacks and handle success cases in an unified manner. If you don’t use dojo, here’s some sample code to intercept all dwr callbacks (more specifically - all methods prefixed with “dwr”).

<html>

<head>

   <script>

       dwrCallback1 = function(response) {

           alert(”fn1: further processing … (” + response.success + “)”);

       }



       dwrCallback2 = function(response) {

           alert(”fn2: further processing … (” + response.success + “)”);

       }



       dwrCallback3 = function(response) {

           alert(”fn3: further processing … (” + response.success + “)”);

       }



       init = function() {

           var m = getMethodsWithPrefixForObject(window, “dwr”);

           for (var i = 0; i < m.length; i++) {

               attachHandlerBefore(window, m[i]);

           }

       }



       attachHandlerBefore = function(obj, methodName) {

           obj["_" + methodName] = obj[methodName];

           obj[methodName] = function(response) {

               if (response.success) {

                   alert(response.msg);

               }

               obj["_" + methodName](response);

           }

       };



       getMethodsWithPrefixForObject = function(obj, prefix) {

           var methods = [];

           if (typeof obj == “object”) {

               for (var prop in obj) {

                   var tmpObj = obj[prop];

                   if (tmpObj == null || typeof tmpObj == “function”) {

                       if (prop.indexOf(prefix) == 0) {

                           methods[methods.length] = prop;

                       }

                   }

               }

           }



           return methods;

       };

   </script>

</head>

<body onload=”init()”>

<button type=”button” onclick=”dwrCallback1({success:true, msg:’User details saved successfully’})”>Sample response 1 - through DWR call</button>

<br/>

<button type=”button” onclick=”dwrCallback2({success:true, msg:’Password has been changed’})”>Sample response 2 - through DWR call</button>

<br/>

<button type=”button” onclick=”dwrCallback3({success:false, msg:’Username must contain only letters!’})”>Sample response 3 - through DWR call </button>

</body>

</html>

In addition to this mechanism, all exceptions can and should be caught using DWREngine.setErrorHandler.

Share/Save