The Time Zone Challenge

This entry is mostly just me rambling on about technical stuff and thinking out loud. If you want to read something really entertaining, read “Freedom, Justice and a Disturbingly Gaping Ass” instead.

Work continued on Buingo.com today. I’ve decided to lay down the user logic first to make room for more users and hence more content. That one of the goals of Buingo.com is world domination creates an interesting challenge:

How to handle time zone correctly?

The natural thing would be that every user had the times and dates associated with each Buingo converted to his or her local time zone. If a user in Florida (GMT-5) browsed the Buingo calendar of a user in Paris (GMT+1) posted at one in the night on Friday local time, it would appear as posted at 7pm in the evening on Thursday for the Florida user. Some people would probably argue that everything should be in the local times of the posters, but this way seems natural to me.

If this was a meeting calendar you would like to have all meetings scheduled displayed in your local time, wouldn’t you? Yes, you would.

So now I’m saving everything as GMT in the database regardless of the time zone of the poster. I’m also able to convert all the dates to the local time zone of the person browsing the content once the GMT value is extracted from the database, but I can’t get the calendar to display the Buingos correctly. The problem is that the time zones has to be converted in the database as the database query is executed, not just after the GMT value is extracted.

MySQL has a method for this, CONVERT_TZ, but I have not been able to get it to work properly yet and it’s not supported by the MySQL version my host is using.

Another day, another challenge.

In other, related news; the number of subscribers to the mailing list has actually doubled from January 1. That’s a good start. You an find out how to subscribe at Buingo.com.

Presenting: Buingo.com

In 2005 Yahoo! acquired Flickr and del.icio.us, both for undisclosed sums. In July the same year, News Corp. acquired MySpace.com for whopping $580 million USD in cash. In October 2006 Google bought YouTube for a respectable $1.6 billion USD in stock and Chad and Steve laughed all the way to the bank. They’re probably still waking up at night laughing until they pee themselves. And someone is probably negotiating for the acquisition of Digg as I’m writing this.

The common factor for all these sites is they are all boosting user-generated content. Hell, del.icio.us is basically just a large collection of tagged links. It seems like the secret to becoming filthy, filthy rich is to get a Web 2.0 community site with user-generated content up and running, get the rumor out there and hope that people are not tired of posting personal stuff about themselves on the interweb.

Enter stage left; Buingo.com. At the moment it’s very, very basic, but if time permits it will pretty soon be my ticket to MTV Cribs. I need another semi-dedicated PHP-programmer for this project. You game? I really need to get the user-handling and sign-up in place. Biggest challenge will probably be the damn time zones. Never liked those.

For all I know there might already be a site exactly like this online. But that I don’t know about it means that they haven’t got people talking about it yet so the race to the Google money is still on.

UTF-8 i18n Struts

As promised, here’s how you correctly get Struts and Tomcat to display UTF-8 characters correctly for i18n. It’s not that complicated, really, but it took quite a while to find out how:

You have to convert your UTF-8 properties-file with a program called natice2ascii. It comes with the SDK/JDK and is located in the bin folder of you Java-installation. This program converts the UTF-8 file to an ASCII file where the UTF-8 characters are written as Unicode Hex. The conversion is done like this:

native2ascii -encoding utf-8 c:\1.txt c:\2.txt

That’s about it, the file can now be used by Struts. Source: Support Eastern Languages in Your Struts Web Applications.

Join me later when I will write about something less nerdish.

UTF-8 MySQL Tomcat Struts

Yeah, I admit it. I have been neglecting you, faithful reader. But you know me, I always have a month or so each year when my updates are few and infrequent. And if you, based on the heading on this entry, assume that the first entry in a week will be full of strange three letter abbreviations, computer programming jargon and other information interesting to very few people but computer nerds like me, you’re absolutely right!

So this entry will not be very interesting for the majority of you, but some of you will love me for it. It will save you from hours of pulling your hair, swearing and coffee drinking at the office – time you should have used at home in front of your Xbox instead. Without further ado, I hereby bring you:

How to display UTF-8 characters in Tomcat using MySQL as the data source! There’s even a little something extra for you if you’re using Struts! I did this with MySQL 4.1, Tomcat 5.5 and Struts 1.2.

  1. Make sure your MySQL database is using the UTF-8 character set. For some reason, the default character set is Latin1. I think the argument for that is that databases using UTF-8 are larger because UTF-8 uses more bytes to save characters than Latin1. Screw that, use UTF-8 for all it’s worth. I used MySQL-Front to convert the database, change the character ser in the database, table and row properties to UTF-8. There probably is a much smoother way to do this, but it worked for me. As a side note I have to say that I recommend using MySQL-Front for very little else than converting the database. It’s too bug-ridden for that.
  2. Tell the JDBC connector that it has to talk to the database using UTF-8. To do this add useUnicode=true&characterEncoding=UTF-8 to the connection URL in the JDBC configuration. Your connection URL will look something like this: jdbc:mysql://localhost:3306/mydatabase?useUnicode=true&characterEncoding=UTF-8.
  3. The next step is to make sure the browser knows that what it’s receiving is actually UTF-8. In your JSP files add <%@ page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %> at the top. In your Servlets, make sure the right HTTP headers are sent back to the client by adding the line response.setContentType("text/html;charset=UTF-8");. Of course, you’ll have to use another MIME type than text/html if you’re not going to display HTML.
  4. After that, you’ll have to tell Java that you’re using UTF-8 by configuring the Java options. Add the parameter -Dfile.encoding=UTF-8 to your Java options, either in the catalina.bat file or by clicking on the Java tab in the Monitor Tomcat program.
  5. The fifth thing you’ll have to do – but as far as I know, you’ll only have to do this if you’re using Struts to handle web forms – is to make sure all input from the client is converted to UTF-8. This is done with a little bit of coding and a configuration change. First, create a class containing the following code:
  1. package filters;
  2. import java.io.IOException;
  3. import javax.servlet.*;
  4. public class UTF8Filter implements Filter
  5. {
  6. public void destroy() {}
  7. public void doFilter(ServletRequest request,
  8. ServletResponse response,
  9. FilterChain chain)
  10. throws IOException, ServletException
  11. {
  12. request.setCharacterEncoding("UTF8");
  13. chain.doFilter(request, response);
  14. }
  15. public void init(FilterConfig filterConfig)
  16. throws ServletException {}
  17. }

When that is done, you’ll have to configure the filter in your web.xml:

  1. <filter>
  2. <filter-name>UTF8Filter</filter-name>
  3. <filter-class>filters.UTF8Filter</filter-class>
  4. </filter>
  5. <filter-mapping>
  6. <filter-name>UTF8Filter</filter-name>
  7. <url-pattern>/*</url-pattern>
  8. </filter-mapping>

If I’ve remembered everything you now have a UTF-8 compliant web application. Yay!

Thanks a million to the following sources:
Struts, UTF-8 and form submissions : Jiploo.com
Google

Join me later, when I tell you the secret of how to internationalise your Struts application to display UTF-8 characters from the Application_xx.properties files. That was a true PITA to figure out, too.