During the first month of Forza Motorsport 2 online gaming, one of my new online friends I met in the public lobbies tuned one of my cars for me. Wow, what a difference that was compared to the not tuned car. It handled much better and was faster, too. Armed with the knowledge that tuned cars can have better handling and can be faster, I tried tuning some of my other cars myself. Hmm, was much more difficult than I had thought. Most of it was trial and error, and I got tired fast.

But I really wanted my cars to be and behave better. So I turned to the theory necessary to understand what was needed to be successful with that. It was complicated, and although I understood the interaction of the various settings better, the quality of the results was only slightly better than before.

And then, I found FEUERDOGs suspension calculator for Forza Motorsport 2. The settings it produced were much better than those of my own efforts. And to be honest, I wanted to race. You have to like tuning in order to like it – haha! It’s not my thing. But by trying to tune cars in Forza Motorsport 2 myself, I got a glimpse of what it means to become a successful tuner.

Then came Forza Motorsport 3, and I was eager to get it. I used the same tuning calculator for the new version of the game, and it still produced better tuning setups than I was able to come up with myself. FEUERDOG started working on a version for Forza Motorsport 3 and eventually he released it to the public.

I have no idea how many hours FEUERDOG spent to create the tuning calculators he did create. He might have spend more hours on it than I was spending on racing. Maybe he had as much fun doing the tuning calculator as I had during racing. But his work alone made it much more fun for me.

So today, I’m releasing ForzaDroid, the implementation of FEUERDOGs tuning calculator FM3-December Update-4 as an Android application on the Android market for free. There can be no other way, because I’m standing on the shoulders of giants here.

What does it offer? Well, here’s a list:

• based entirely on FEUERDOGs FM3-December Update-4 spreadsheet, you’ll get the same results
• added “Transmission” selection for RWD differential settings (see FEUERDOGs remark in the spreadsheet for details)
• available in English and German
• it’s available at the Android Market and free (as in beer)
• it’s simple and fast (hey, I’m not a designer, I’m a programmer)
• no gear spreader – yet

To download ForzaDroid, visit the Android Market and search for it (search term forzadroid, but forza might work as well), or just scan the following QR Code with your Android powered mobile phone. It will take you directly to the Android Market:

If you have never tuned a car in Forza Motorsport 3 before, you can head over to ForzaTune were you can find more information. They’re offering an iPhone application that does roughly the same thing – even more, it supports Forza Motorsport 2 – as ForzaDroid. It’s also based on FEUERDOGs work. But it has been around much longer.

If you like the Android application ForzaDroid, find a bug or have some suggestions, feel free to contact me. I won’t promise anything, but at least I’ll listen.

I’ll give an example: imagine we have rows describing some customers. Each customer is in exactly one customer category. What we now want to have is a list of all these customers, but grouped by customer category. And because we have a dimension for the customer category, we want a separate table for each category. The naive solution seems to put the customer category dimension as an iteration into the outer block of the pivot table. But this will not work because ROWNUM() – an thereby the LINE_DIMENSION – is the only dimension used for iteration.

But there’s at least one way to achive that. It’s probably not the most elegant way and I’m eager to hear of other solutions, but it works for reports that are not too big.

We create a pivot table and in the outer block where we’ve put the table, we iterate over the customer category dimension. Set the iteration property to {CUST_CAT::'CAT'}. To see the current iteration in the title of the pivot table, put a text into the title property of the table, e.g. "Customer category: " + {CUST_CAT::'CAT'}.

Now, put the fact that holds the information about the customer category in the table. I’ve put Fact:'CUSTOMER_CATEGORY' as the first column. We don’t need to see it, so we can set it invisible. And here’s the trick: for the iteration over ROWNUM(), set the property Visible like this: TONUMBER(CUST_CAT::'CAT') = TONUMBER(MATRIX(1)), where the argument into the MATRIX() function is the position of your fact for the customer category.

Because we have the dimension and level CUST_CAT::’CAT’ as an iteration in the outer block, we basically create as many identical tables as the dimensions level has keys. If there’re 10 keys in the level, we have 10 tables with all the rows from our result set. But only those rows are visible where our fact matches the current key of the dimension, which basically allows us to group the customer rows by the dimension.

The tool

Although I wasn’t involved in the selection process, there’re good reasons for TOS, some were decisive:

• it’s an Open Source solution
• there’s a branch office in our country
• based on the Java and Eclipse platform
• active community
• training and support offerings

I’ve had really good experiences with our main Talend contact over here. Based on a couple of workshops and support cases I have to say that he was of great help for our project.

Depending on where you come from and what you need, some of the points speaking in favor of TOS for us might speak against it for you. The good thing is, there’re enough solutions/products out there, maybe even one for you. But it might not be TOS. If you’re not sure if an ETL solution is right for you, maybe this article by J. Mundy helps.

The staging area

The database system used for the staging area is IBM DB2 Express-C. The staging area is cleared before every import and has two stages itself:

1. data is loaded into tables that reflect the schemas of the CSV files; all fields are VARCHAR fields
2. then, the data is converted into the proper types, e.g. DECIMAL(15,2), DATE, INTEGER, …

The table schemas for the tables in stage one and stage two have a couple more fields than the CSV schemas:

• ID_ROW identifies the data row from the CSV file
• STATUS holds information regarding the correctness of the data (and thus if it has been successfully written to the stage-two table)
• ERRMSG contains a detailed description on the problem (if any)

There’re many CSV files to import into the staging area, but the basic process is the same for all. The TOS job that does the work looks like this:

I’ll give you a short description of the process:

• first line imports the CSV data into the stage-one table
  • the CSV file is read in
  • because the CSV schema and the stage-one table schema differ (more fields, see above) we map the CSV schema to the table schema
  • each row gets the additional ID_ROW field filled
  • the data is written to the stage-one table
• the data now residing in the stage-one table is converted and written to the stage-two table
  • stage-one table data is read in
  • the conversion to the proper types takes place; the status and the error messages are generated here
  • all problem rows are sorted out, based on the content in STATUS
  • the rows without problems are written to the stage-two table
• error logging and reporting
  • all rows that didn’t make it to the stage-two table are updated in the stage-one table to have the correct STATUS and ERRMSG set
  • problematic rows are written to the console for this example

TOS components

TOS basically lets you work with components. You place them on the screen and connect them, enter some information into the properties of these components, and then TOS generates Java source code ready to compile and run. Once generated, you cannot alter the Java source code directly. You have to do it by altering the components properties. Some informations like schemas for CSV files and database tables, database connection info, etc. can be stored in a repository and you can connect them with components. That’s nice if you need the same information in more than one place.

For the type conversions, we use a component called tJavaRow. It has two main properties (called settings in TOS):

• the input and output schemas
• a text area that contains the Java source code executed by this component

I’ll explain later, why we use this really basic component instead of a more fancy one like tMap that seems to do the job more nicely. It doesn’t. Anyhow, the Java source code looks like this:

C_OUT.STATUS = "OK";
C_OUT.ERRMSG = null;
convert(C_IN, C_OUT, "ID_ROW", "toField", "ID_ROW");
convert(C_IN, C_OUT, "FIELD_ONE", "toNumber", "SignedNumber", ",");
convert(C_IN, C_OUT, "FIELD_TWO", "toNormString");
convert(...);

A central conversion routine written in Java is used for all conversions. It does the exception handling in case of a problem (e.g. sets the fields STATUS and ERRMSG) and controls the conversion. The input and output Java data objects are passed in as well as the field names, the name of the conversion routine, and additional parameters. The convert method does the work by using the Java Reflection API to look up fields and methods. The advantage is that it simplifies the exception handling, but it comes with a price: there’s no type safety anymore, i.e. problems are reported at run time, not compile time. We automatically generate the calls from a repository that contains all necessary information. The Java source code is simply copied, not written by hand.

The tMap component on the other hand is nice for ad-hoc mapping and conversion of a few fields, but isn’t really fit for the task otherwise. It might look good during a presentation/sale pitch, but it’s just too painful to work with. That’s because you have to select every field where you want to enter some functionality. It’s just too many clicks. And guess what, if you have to refactor you code, you’ll start all over again, i.e. go to every output field and do the refactoring manually. Imagine a schema with couple of hundred fields and you go nuts.

What I think so far

I’ve only written about a small number of things regarding TOS, but they were important for this part of the project. The import of data from CSV files into the DB2 database is possible within the requirements given above. I didn’t have performance issues related to TOS. The visual components are nice, but sometimes get in the way of working efficiently. What I really want to see is the possibility to edit the components settings with a text editor as a whole in addition to the form based entry.

This information is based on TOS 2.3.2.r12707. Things that were broken in this release were

• HTML documentation was not generated at all
• given exception locations weren’t in sync with generated source code
• export project didn’t work without refreshing project in navigator view (F5)

These things will be fixed in the next releases I’m sure.

Two last things I want to say:

• if you don’t have someone on the team who is sufficiently fluent in Java, you’ll have a hard time to implement non-trivial solutions; depends on your or my definition of ‘non-trivial’ of course ;-)
• after working with TOS for some time now, I have the feeling that it can solve nearly any problem that might occur in such a project

During the recent holidays I spent the entire time away from work and with my family. As most of the time when I’m away from work I made a list of things to look at. Wicket was not on that list. I seldom make it at least half way thru that list anyway. It was the same this time. But then I saw this article on TheServerSide about the release of Wicket 1.3. The big amount of comments caught my interest. Suddenly, Wicket was on that list.

Today, I downloaded and installed Wicket, and then looked for documentation and tutorials. Besides some interesting online resources, I found the combination of two books on Wicket the most helpful:

• The first ten available chapters of Wicket in Action (MEAP – Manning Early Access Program) by Martijn Dashorst and Eelco Hillenius, published by Manning Publications Co.
• Enjoying Web Development with Wicket by Ka Iok Kent Tong, published as PDF only, with freely available first three chapters

The first book seems a little more verbose than the second book and covers more background, while the second book (besides being more complete right now) has much more source code examples. They complement each other perfectly in my opinion. So I got them both.

After reading thru the first three freely available chapters of “Enjoying Web Development with Wicket”, I got interested in chapter 6 (Supporting Other Languages). I was really impressed to see how I18N is a first class citizen in Wicket.

I then turned to “Wicket in Action”. Right now it is not completed and contains many errors, but none of this is really a problem. I made it thru the chapters 1-4 and implemented the Cheesr sample application (from chapter 4) along the way.

I really like what I’ve seen so far from Wicket:

• It’s component based and encapsulates the request/response cycle
• Components seem easy to write
• Good internationalisation concept
• Separation from logic and presentation

Right, I’ve barely scratched the surface yet, but Wicket as a web framework looks promising.

Three days after GWT 1.4.60 came out, the issue was reported, approximately at the same time I was experiencing it. I was hoping for a quick fix, but it didn’t come.

Several month later, Google’s Scott Blum wrote in the GWT forum about an updated GWT 1.4 release. Although I read the forum quite regularly, I nearly missed his posting. He talks about the new release fixing the problem, and he also provides a download link for GWT 1.4.61. I’ve downloaded it and upgraded all my application without a problem so far.

Since it fixes a very serious problem, you should upgrade also. Problem is, you will not see the new version on the GWT download page. At least not yet, but I bet you will see it there shortly. In the meantime, just use this link.

Update: It took a little bit longer than I expected, but the GWT Team has just officially released GWT 1.4.61, which takes care of some minor bugs in the previous release of GWT 1.4.60. The two main issues that were addressed are the “operation aborted” startup error on IE and a potential XSS vulnerability in hosted mode.

There are no differences between the 1.4.61 version released on December 12th, 2007 and the 1.4.61 version released on November 3rd, 2007.

Even though small development teams use a version system in many cases, an issue tracking system is used to a lot lesser extend. But I believe that it’s quite as important as having something like Subversion, because it makes the developers life so much easier. I have just experienced it while working in a small team during my current project. And there are solutions out there, that are easy to install and maintain. One (and one of the easiest and best) is Trac: Trac is an integrated software configuration (and project) management system (SCM). It’s an issue tracking system for software development projects with an integrated wiki. Basically, it’s a project management suite which includes a wiki, a bug/issue tracking system, and a source control viewer that accesses a Subversion repository.

Since developers always tend to try out the newest and latest (I’m no exception to this rule, nor do I want to disappoint you), I’ve installed the Trac version currently under development (0.11dev). The integration with the Eclipse development environment also requires the latest development version of the Trac XML-RPC plugin. This is what you should have installed already:

• Eclipse development platform with Mylyn Trac Connector.
• Working Subversion (SVN) installation.
• Python is installed on your system (Trac is written in Python).
• easy_install is available.
• It helps if you have installed an older version of Trac (e.g. 0.10.4) once. If you don’t have any experience with Trac, you should go there and read their documentation.

Now install Trac and the Trac XML-RPC plugin:

• Checkout the latest development sources from Trac: http://svn.edgewall.org/repos/trac/trunk.
• Install/update Trac from the trunk directory: $ python setup.py install.
• Checkout the sources from the Trac XML-RPC plugin: http://trac-hacks.org/svn/xmlrpcplugin.
• Install/update the Trac XML-RPC plugin as decribed here: $ easy_install trunk.
• Edit trac.ini and add (under [components]) tracrpc.* = enabled.
• Start (or restart) Trac, e.g. via tracd.

You can now access and create your Trac tickets from within Eclipse with the help of the Mylyn Trac Connector.

If the application doesn’t load, please click here!

Before I was able to include a GWT application within a post in this WordPress powered blog, I had to create a WordPress plugin that generates the necessary meta tags. GWT applications only work, when those meta tags are present.

The actual calculation of the expression is done on the server with the help of my Java based framework. The expression is sent to the server asynchronously as a JSON data packet. The server logic evaluates the expression and sends back either the result, or an error message. If you want to see an error message, try a division by 0. The response is sent also in the JSON format back to the browser, where the GWT application takes it and displays it without refreshing the whole page.

The server side logic is built with the help of ANTLR and ANTLR Studio for Eclipse. Instead of writing a lexer and parser for evaluation of the expressions myself, I’ve used a grammar to let these tools do it for me. The following screenshot shows ANTLR Studio for Eclipse in action:

If you want to know what ANTLR is, here is a quote from Terence Parr, the primary author of ANTLR. He has been working on translation tools since the late 80s:

ANTLR, ANother Tool for Language Recognition, (formerly PCCTS) is a parser and translator generator tool, akin to the venerable lex/yacc duo, that lets you construct recognizers, compilers, and source-to-source translators from grammatical descriptions containing C++ or Java actions. You can build translators for database formats, graphical data files (e.g., PostScript, AutoCAD), text processing files (e.g., HTML, SGML), etc. ANTLR is designed to handle all of your translation tasks. ANTLR is recommended by the co-inventor of LL(k) parsers and by the inventor of SLR(k) and LALR(k) parser.

Another tool makes generation of GWT clients easier. It’s the GWT Designer from Instantiations. I’m having a few problems with it, but it still helps a lot. See the following screenshot for an impression:

If you’re interested in this example built with GWT, ANTLR, JSON, etc., and you want to look at the source code, just download it from here.

Do you remember the little problem from the Geometry for kids post earlier today? With the calculator, you can verify if the solution is correct: d = sqrt(3)*5.3.

Update 1: As of GWT 1.4, a new application-loading mechanism has been introduced, that doesn’t require a meta tag in the page header. It’s still supported for legacy applications. But now, the applications HTML code can directly reference the JavaScript file. I’ve changed the example above accordingly.

Update 2: The calculator has been updated to GWT 2.0. The project available for download has been updated as well.

First, I looked for help where all the WordPress plugins/meta are listed. None of them seems to solve my problem. To be honest, I only looked at a few, so I really don’t know if I’ve overlooked the one I need. One Meta plugin came quite close to what I want, but it was for description and keywords only. Except that, it would have been perfect. So I took its idea and modified it.

You have to use WordPress’ custom fields for storing the gwt:module meta tags. Just create a new custom field when you create or edit a post. The key should be set to gwt:module, and the value should be the path and name of your GWT module. The plugin then collects all gwt:module meta tags for all posts on the page and puts them in the header section of the page.

In case you want to use this plugin yourself, you can download the WordPress plugin for creating gwt:module meta tags from this web site.

Later today, I will write a post that actually has a GWT application included.

Epsilon is an Emacs clone, but unlike Emacs, which can be extended with Lisp code, Epsilon uses a C-like language called EEL (Epsilon extension language). As you can see from the list of programming languages I’m sufficient familiar with, Lisp isn’t on it, but C is. Although this short enumeration of programming languages isn’t complete, I haven’t used any of the others for a substantial amount of time. It’s important for me, that Epsilon is programmable in a language I’m fluent in.

There’s one exception to my otherwise consistent use of Epsilon. When it comes to Java programming, I’m using Eclipse most of the time. Not always, because some things I still do with Epsilon, but most of the things are a little easier within Eclipse.

During the time I worked with Epsilon, I observed a strange thing: some programmers (even brilliant ones) worked with editors that are not supporting them well with their work. Even when shown alternative editors that would support them much more, they gave up early on learning the features of the new tool. Instead, they spend hours writing some small utility programs for tasks that are possible within minutes for editors like Epsilon. Instead of investing the time in learning a new tool, they spend much more time with fixing the shortcomings of the editors they were used to.

Don’t get me wrong, Epsilon is not a panacea for everything, and I’m gone the minute I find something better. But in the past 10 years, I haven’t.

Fatal error:
Call to undefined function: some_function() in
/srv/domain.com/httpdocs/content/mytheme/header.php
on line 42

As a web user, what does this error message tell you? Maybe you can understand it. But can you do something about it? You can’t fix it, you’re left alone on finding your way out, it’s not helping you. Why then show it to you at all? The reason is, it wasn’t intended for you to see it. This error message is only useful for the developer. But he hasn’t planned for exactly this type of situation. Okay, it’s a programming error. But that you have to see this error in a form like this is lack of planning for this type of situation. Everybody knows or can understand that it’s not a good idea to have a web user look at a web page indicating an error like the one I showed you. But it’s still what happens in many cases. Why? Is it laziness on behalf of the developers, lack of good test cases, lack of imagination, lack of time or budget, or not enough knowledge? I think it’s a little of everything. The book Defensive Design for the Web from 37signals might tell you nothing a web developer should know already, but it contains a nice check list at the end to make sure he doesn’t forget to apply his knowledge.

To assist the developer in handling all not handled situations, synformation provides the mechanism of an error page that is called via forward() whenever a not handled error occurs. This error page is a normal synformation page with one exception: within the logic of the error page, you have access to an error object that holds all the information about the error. If you don’t define an error page within synformation, you’ll get something like the cryptic error shown above. It’s worse actually, because it might contain a Java stack trace because synformation runs on the Java platform.

I can only think of one reason to display a technical error message within the web page: the page is produced for the developer or tester. In all other cases, the user should see something more friendly, something that gives him some information on what to do next. He should not leave our web site in frustration. That’s why a typical synformation error page should contain logic like this:

<%%
$variable = getVariable("error");
if (null != $variable) {
    $msg = $error.toString();
    print S("X-ERROR-ERRORPAGE");
    // There's no need to send an email when an ACU can see the error
    if (isDevelopment() || getCurrentUser().isMemberOf("ACU")) {
%>
<div id="infobox" class="infobox-info">
<h1><%%= R("ERROR-PAGE-HEADING") %></h1>
<p><%%= $msg %></p>
</div>
<%%
    } else {
        print S("X-INFO-WHAT-NEXT");
        sendMail(R("ADDRESS"), R("SUBJECT"), R("TEXT") + $msg);
    }
} else {
    print S("PAGE-NOT-CALLABLE");
}
%>

I will explain this code snippet. First, we have to find out if an error object actually exists. If it does, the users gets a friendly hint in his language that an unexpected error has occurred and we’re really sorry about that. If we are in development mode, or the user has logged in and he is a developer, the actual error message is displayed, too. If we have a normal web user, a text with options on what to do next is presented. An email is then sent off to the development team with all the technical information needed to reproduce and fix the error. In case no error object exists, this page was called without having an error. Maybe somebody typed in the URL in his browser. We tell him in friendly language, that this page is not intended to be used like that. We can also display some information on what to do next – he is already on our page, so why not keep him.

We all know that things can go wrong. But we should all care more for having an actual plan installed, if that happens. I’m not perfect in this regard either, so this post might help me as a reminder.