Recently in Programming-fu Category

Remember CONGO, my convention registration software? And how over the last year I've been rewriting it from scratch?

This past week, the new rewrite, colloquially referred to as 'v2', went live for Arisia. Following is a bit of babble about the project, and the net steps therein.

I had my first decent coding stretch on CONGO tonight in quite a while. I put in 4-5 hours to work through some issues that have come up while running the v2 code for a 'live' event. This is the first time the new codebase has been exposed to 'real' customers, and so far things are going okay.

There haven't been any showstopper bugs yet (knock on every available source), but there have been a few rocks along the way. The fellow running the conference has been very supportive, knowing it's new code, and I'm looking forward to a post-event debriefing. "What should be done differently?"

Overall, I'm quite proud of the v2 code. It's still missing a lot of functionality from the first codebase, but I do enjoy having everything in Java in a sane build system and a rock solid database layer.

And now... bed.

Recently I was successfully marketed to by Woot.com and aquired an Asus EeePC 900 Linux netbook. For those who are not familiar with these puppies, they're hyper-small fully functional 'laptop' computers, scaled down to be the size of a hardcover book. The Netbook article on Wikipedia is a good summary of these devices.

The Asus EeePC 900 is an 'older' version (hence the reason I got it for only $149) with 512meg of RAM and a 4 gig SSD drive. It has all the basic features you'd expect for a laptop - wifi, decent screen, touchpad, USB ports, good battery life (about 3.5 hours), etc. In all respects, it should be a geeks dream. A fully functional Linux box that is only a few pounds, and can run for hours.

So why am I considering handing it off to my son?

The main problem is that in the current portable computing environment, the 'slot' that Netbooks like the EeePC can fill is narrowing rapidly. On the 'full laptop' side, there's a trend toward longer battery life, lighter designs, and stuffing all the functionality of a full desktop machine into a portable form. Many people don't even have desktop machines anymore, they use their laptops for all work (that's my situation). On the other side we have the emergency of smartphones like the iPhone (which I have). The iPhone is an enormously capable device. I can read my email, chat online, browse the web, play games - all the things I'd likely do on my laptop if it were small and light - the space that the EeePC and others are shooting for.

Even in the face of all this, I really did give the EeePC a try. I carried it around for a week, trying to see where I'd use it and where I wouldn't. I never 'clicked' into it in any particular fashion, due to a number of obstacles that were either filled by my iPhone or by my laptop:

• Very small keyboard
  The EeePC has a very small and somewhat wobbly keyboard. I have quite large hands, and though I could 'shrink' my hands down to type away, it took some serious concentration, and really only worked when the EeePC was flat on a desk and I was sitting in a proper chair. If I were in that situation, I'd just use my laptop.
• Wireless twitchy
  This is probably a fault of the Linux distribution the EeePC uses, but I had all sorts of problems with the machine waking up and not reassociating with any available wifi (it wouldn't even show networks available).
• No LEAP support
  The wireless also could not use LEAP authentication on wireless. This meant I could not use the EeePC anywhere at the office. Total loss there - I was hoping to be able to bring the machine with me to meetings so I didn't have to undock and haul my normal laptop along.
• Update failures from Asus
  ASUS has broken their updater. The EeePC will not software update properly from ASUS's servers. This is a real problem. There are workarounds, naturally, but it likely means there won't be OS updates from the manufacturer anytime soon. The answer seems to be to use Eeebuntu, a version of Ubuntu linux designed specifically for the EeePC netbooks.
• Touchpad
  I don't like the touchpad. I don't know why - I just can't get comfortable with it. The two-finger scrolling is cumbersome and prone to 'pausing' (this compared to the two-fingered scrolling on a macbook, which is smooth as silk).
• Yet Another Power Supply
  I have a problem with power supplies. If I'm going to carry another laptop, I have to have another power supply with me. So now I have 2 laptops, 2 power supplies. This is not saving me anything in weight in my backpack.

Given all these issues, I find myself either picking up my iPhone to twitter or check something on wikipedia, or get out my laptop if I'm going to do any real work.

So what to do? The current plan is to reload the EeePC with Eeebuntu and evaluate that. If it's stable, is able to browse youtube, run Python's IDLE environment, and play nethack, then it will be a perfect upgrade for my son, as he's outgrowing his XO laptop.

Some day some bright light will come forward and lay down the law about how Perl code should be formatted. And, just as likely, they will be roundly ignored, primarily out of sheer spite, but with a grain of "perl lets us do what we want. We don't need no steenkin standards" tossed in just to muddy the argument.

Today's annoyance is cleaning up some long forgotten programmer's way of of formatting blocks of code. Only slightly edited, here's a sub in one of the cgi scripts:

sub _getMediaList( $ $ $ ;$ )
{
  my ($carrierId, $contentType, $contentIds, $formatted) = @_; 

  return undef if (!defined($contentType)); # Charts can have no contents and thus no content type

  my $gpd = GPD::getDbh();

  my @mediaList;
  foreach my $contentId (@$contentIds)
  {
   my $resultList = mediaSearch(dbh => $gpd,
                                carrierId => $carrierId,
                                contentType => $contentType,
                                id => $contentId,
                                showdown => 1); 
   next if (!$resultList->[0]); # Shouldn't happen
   push(@mediaList, $resultList->[0]);
  }

  if ($formatted)
  {
    foreach my $media (@mediaList)
    {   
      $media->{title} = "$media->{artist} - $media->{name} - $media->{album}";
    }   
    return formatMediaListColored('[%s] [%s] [%s] %s [%s] [%s] [%s] [%s]', [ 'id', 'isrc', 'upc', 'title', 'genre', 'label', 'relate
d_content_isrc', 'release_date'], \@mediaList, 1);

  }

  return \@mediaList;
}

Now, I'm not harping on the code itself - this is pretty innocuous, if slightly braindead. But what annoys me is the overuse of newlines to stand off functional blocks. They're not necessary, the indenting is painful, and it makes the code hard to read. How hard is it to rework it like this:

sub _getMediaList( $ $ $ ;$ ) {
    my ($carrierId, $contentType, $contentIds, $formatted) = @_;
    my $gpd = GPD::getDbh();
    my @mediaList;

    return undef if (!defined($contentType)); # Charts can have no contents and thus no content type

    foreach my $contentId (@$contentIds) {
        my $resultList = mediaSearch(dbh => $gpd,
                                        carrierId => $carrierId,
                                        contentType => $contentType,
                                        id => $contentId,
                                        showdown => 1);
        next if (!$resultList->[0]); # Shouldn't happen
        push(@mediaList, $resultList->[0]);
    }

    if ($formatted) {
        foreach my $media (@mediaList) {
            $media->{title} = "$media->{artist} - $media->{name} - $media->{album}";
        }
        return formatMediaListColored('[%s] [%s] [%s] %s [%s] [%s] [%s] [%s]', 
            [ 'id', 'isrc', 'upc', 'title', 'genre', 'label', 'related_content_isrc', 'release_date'], 
            \@mediaList, 1);
    }

    return \@mediaList;
}

To me this is far easier to work with and understand where teh blocks are and how they work together. In my opinion, there's a couple basic rules about code layout:

• Consistent indenting - 4 spaces or 2, I don't really care which, just use the same thing EVERYWHERE.
• Curly braces to open a block go on the same line as the introductory keyword (if, foreach, sub, whatever).
• Lines longer than 80 characters should be broken into sane sub-lines. Just hit enter and indent, folks. Not that hard.
• Declarations at the top of the block when the variables are used across the entire block.
• Separate declarations from functional code by a newline.

Now, this particular piece of code has no comments in it, so I'll leave comments for another rant. But I think these basics sure would make perl code a lot more maintainable if everyone at least marginally followed them.

Dear Eclipse Developers... having a bug open for 3+ years, through a myriad number of releases is inexcusable. Pointing fingers at different development groups saying "it's their fault" and others saying "it's those peoples fault" is not appropriate.

Just fix the damned bug already.

(NB - yes, that bug is only 6 months old. It is a merge from several other bugs reporting the exact same thing, see )

On Sunday night I released CONGO v2 into alpha testing. The first client to be using the new platform wants to be up and running on January 1st, and this is on-schedule. The last couple weeks haven't quite been a death march, but there has been a helluva lot of code written, checked in, and tested. The log of messages posted to congo-dev tells the tale. While November wasn't as intense as October, we still did over 105 commits against the server. The CONGO v2 codebase is rivalling the original system in size, though now it is entirely java based (the old system was part PHP, part Java).

I'm pleased with how the system is coming along. An entirely new payment interface, refactored contact information, and a new Events module for managing activities at conventions is worked into the new model (though not all of it is complete yet). I'm satisfied with the choice to move to using Struts2 as my web framework, but my dissatisfaction with OGNL grows daily. Why OGNL was necessary for the struts tag library, when the expression language (EL) was around is beyond me. For most of my code I'm replacing struts tags with JSTL and assorted libraries, and using EL for referencing action, stack, and session based content.

The next 2 weeks should see the last pieces of functionality falling into place, at which point we go into beta testing. All coding should be done, and we'll be tuning the system for "go live" on January 1st.

After that? Well, due to the awesome new build structure, I'll be able to release CONGO v2 for use by other folks. The system is now self-installing and configuring, so setting up a new host to run a convention is an order of magnitude easier than it was in the past. Stay tuned for details on how you can download CONGO and install it for your convention!

Misc ramblings, just have to get them out there.

• KDE4 - Not Ready
  KDE4 is so not ready for prime time. Or maybe it's whatever version is currently bundled with Hardy. Ick ick ick. Not stable, unuseable, bleah.
• Struts2 - So far so good
  I'm getting into the proper mindset to go "Make that, wire it up, go." in S2. I'm missing a good powerful JPA layer though - too late to roll Hibernate into v2.0, but I so see it in the future.
• Laptop battery, not so good
  clipper has a dying battery. After 3.5 years, I'm not surprised, but it is sad. I'm only getting 15-20 minutes out of it now, the question is replace the battery or the laptop and the battery. Decisions decisions.
• Scratch is still cool
  Zach is still totally emfatuated with Scratch. I watched him 'program' for a bit today, and he clicks and moves components around with the best of them. He even figured out how to get it to generate an error and pop up a diagnostic window ("Cool, huh? What is that?" "That's smalltalk." "Neat!"), etc. Ahh, my little hacker.
• Work kicks my ass
  Tech Barbie says "Work is hard!". It is - it's taking up a huge portion of my brain and focus, which is something I'm not used to. I finish the day drained and wrung out, but still force myself to get code done for Congo before falling over for the night.
• Hudson is cool
  We have a hudson install going on one our servers. It's doing continuous integration builds and deployments. FAR better than the hacked up scripts of yor. Need a new build? *click* Build in progress!
• Perl made tolerable
  I'm still stuck with working on perl sometimes. But if I have to go total immersion, I'll likely use EPIC. It's a Perl IDE plugin for Eclipse, and actually seems to work. Very tasty.

Nuff rambling. Back to the grindstone.

So this weekend had me out to Ubercon down in NJ. All in all, things went pretty well. It was the second time I took Zach with me to an event, and he and blk's son Justin had a riproaring time gaming, socializing, and geeking.

On a personal level, this wasn't one of my banner events. It's been a while since I ran at at-con registration of a reasonable size, and a lot of things conspired together to fail so that, by today (the last day of the con), I felt pretty down about my showing. Let me esplain. No, there is too much, let me sum up.

• Mame - My MAME cabinet, which I'd been hauling down to Ubercon for now the third event, gave up the ghost last week. deathstar refused to boot, and I almost cancelled bringing the machine. The UC folks happily offered up some spare hardware, and I decided to bring the machine down. Early Friday morning I did an emergency load of Kubuntu 8.10 on a spare laptop, installed my MAME drive that has my roms in as an external drive, and configured up KXmame. The end result? Unstable, video modes not working right, and general bleah. I managed to keep it limping along through the weekend, but it was not the glorious, elegant machine of the last event. There needs be work here.
• CONGO - Hardware - The server I use for events, 'endor', has been running faithfully for almost 20 events now. I've done one full OS reload, and for the most part it has been dependable as all git out. This weekend however some hardware twitches started to come up. First, the CMOS battery died, which causes the 'things have reconfigured!' message on boot. What I didn't realize was it had also reset the dates, so that all the log entries for CONGO this weekend have a datestamp somewhere in 2006. This will require manual fixing. We also lost power twice due to a flaky outlet. The last bit was I attempted to cut back the amount of hardware I bring to events, and in doing so managed to arrive short 2 keyboards. Fortunately, Ubercon loaned me a pair so things were fine, but Grr.
• CONGO - Software - For the most part, CONGO behaved appropriately and did all we asked of it. I'm itching to get v2 up and running, because of all the deficiencies I keep seeing in v1. But, the old tried-and-true still chugs along, and we cranked out hundreds of badges over the weekend.
• Organization - I normally have a very competent reg manager running the event with me. This time, the normal Ubercon chap I work with was unavailable (due to health issues). While other folks helped man the desk (we were never short on people), not having an "In charge" Ubercon person with us really pointed out weaknesses in the process control in CONGO (things like cash drawer management).

Despite my grumblings, the convention was a success, and I think we did a bang-up job on keeping everything flowing nicely. I am utterly, 100% exhausted - the drives up and down take their toll, and caring for a 9yr old while running an event can be a bit taxing. Would I do it again? Absolutely, and will next year. For now, I'm going to go fall over.

Another good night of coding!

After finishing a hefty project at work, and a busy week of Life Fun, I took 4 or 5 hours last night to hack into CONGO and try and get some rough edges worked out. I really feel like I'm approaching a useable system, with workflow doing what it's supposed to, database tables updating the way they should, and crashes being few and far between.

I'm a visual person, so I wanted to come up with an up to date ERD for the v2 table structures. Ages ago I used DbDesigner4 to create an ERD for v1, but the tables have shifted a lot since then, and I wanted to see if I could auto-generate a diagram from table structures

Hunting around, it looks like DbDesigner has been absorbed into MySQL Workbench. I downloaded the latest version, and found myself bewildered by it's interface until I realized what this thing was. It's a tool for creating and designing databases, as opposed to a passive maintenance tool. Not... quite the approach I was shooting for - I was hoping for something like dbVisualizer that had a button that said "Make an ERD of this.". While dbVis does have that to a certain degree, it's pretty weak.

Eventually I figured out the incantations necessary to create an ERD with MySQL Workbench from existing table structures (it involves dumping the MySQL tables to a .sql file, and 'reverse engineering' the structures from that file). A few more mutterings and a dead chicken later, I had a visual representation of all my tables and their relationships to each other. Score!

Alas, the tool isn't really a 'reporting' system, so the resulting graph needed some fiddling in Gimp to add titles and other text. Naturally I realized after the fact that I didn't move a couple tables into the view before snapping the picture, but it's still a pretty good representation of where I am.

Coding wise, v2 is missing a vital component - the Properties system is not yet working, and this is critical for basic operation. I started into it last night, building up new DAOs and domain objects representing a Property for a registrant and an associated PropertyDefinition. What's left to do is write all the management tools around them to make them workable from Coconut, and start testing.

The last big project will be the public interface. In many ways, that's going to be the simplest and most rewarding - it'll be the outward facing side of CONGO. Essentially what registrants see when they want to attend a convention. Previous stabs at this worked reasonably well. Attendees at the various events were able to do the basics of what the event needed, but I'm really looking forward to going the 'next step' and offer a full service deployment.

My delivery date for CONGO v2 FC is December 1st. So far, I think I'm going to make it.

Have to share this one. I've been wracking my brain trying to figure out how to get table rows in a struts and JSTL supported page. I haven't found an easy example anywhere, but I finally sussed it out. So if you do JSTL and struts based stuff, and want to do those nice 'greenbar' like lines on a table, here's an example of how to do it:

<c:forEach var="rows" items="${registrantList}">
	<c:set var="cellColor">
		<c:choose>
			<c:when test="${cellColor == '#f0f0f0'}">#ffffcc</c:when>
		<c:otherwise>#f0f0f0</c:otherwise>
		</c:choose>
	</c:set>

	<s:url id="url" action="coconut/gotoUser" >
		<s:param name="rid">${rows.rid}</s:param>
	</s:url>

	<tr bgcolor="${cellColor}">
		<td>
			<s:a href="%{url}">${rows.rid}</s:a>
		</td>
		<td>${rows.lastName}</td>
		<td>${rows.firstName}</td>
		<td>${rows.company}</td>
		<td>${rows.badgeName}</td>
	</tr>
</c:forEach>

My pal Owen (who, incidentally, is doing some wonderful work on CONGO) has just written an excellent high level of of build system philosophy entitled Nobody Cares about your Build. If you're a developer or release engineer, or are just plain curious about build systems, check it out - here's a sample:

A reliable, comfortable build system has measurable benefits for software development. Being able to build a testable, deployable system at any point during development lets the team test more frequently. Frequent testing isolates bugs and integration problems earlier, reducing their impact. Simple, working builds allow new team members to ramp up more quickly on a project: once they understand how one piece of the system is constructed, they can apply that knowledge to the entire system and move on to doing useful work. If releases, the points where code is made available outside the development team, are done using the same build system that developers use in daily life, there will be fewer surprises during releases as the “release� build process will be well-understood from development.

Warning. I'm ranting. I'm annoyed. I'm frustrated. You might want to put on your rant-proof galoshes before proceeding.

Ready?

Perl should be considered a Write-Only Language.

Code that is written in it is generally incomprehensible by anyone but the person who wrote it. It's great for one-offs, quick simple jobs, and low-scope tools. But anyone who says it's appropriate for an enterprise level application (and I mean anything over about a thousand lines) should be strung up, draw, quartered, glued back together, and shot into space.

The main problem is Perl is so weakly typed, it should be considered not to have any. That means that parameters passed to methods are not type-checked, and therefore there's no way to tell if you're calling a method correctly, except to see if the app blows up when you run it. Editors and IDE's cannot magically determine that "such and such a method requires an integer, a string, and a hashmap containing such and such values. You're doing it wrong."

Because of this weak typing and lack of structure, Perl libraries become worthless. Let's leave aside the fact that Perl OOP implementations are complete and total trash (OOP is designed to organize your data into object form, and stabilize how components are used, defining a rigid structure so that when you use the component, you must use it correctly). In the case of libraries, it's impossible to use a library unless you understand how it works. Libraries are not self documenting, they don't even have a standard method of organization. They're just loose collections of Perl code, again, with very little defined structure.

Therefore, people who come along to maintain code after the first person has suffered the above fate has no clue how a method is supposed to work, unless the original coder documented it, or built in type checking in their code. I have seen NO Perl code that actively makes sure the parameters being passed in are of the correct type. At best, they make sure the parameter are not null. And if they're super-advanced, they even use prototypes.

And yes. I am in the unenviable position of maintaining thousands upon thousands of lines of undocumented, badly maintained, incomprehensible Perl code. Yes, the developers could have documented, formatted, and put the code into a form that's easier to maintain. But they didn't, and to me that's the languages fault, not the developers. A language should have some element of maintainability, and not require the programmer to make up for it's inherent ambiguity. If it does, it's doubling the burden on the programmer. They not only have to write the code, they have to document it, and make it maintainable.

In my opinion... Perl by its very nature encourages sloppy organization and unmaintainable code.

And now, back to the trenches.

I think I just made up for my lack of productivity toward the end of last week.

Database Fixes
I had been totally stymied by a problem in the CONGO rewrite - how to handle state of a registrant in a way that didn't involve scanning history logs and essentially replaying an audit each time. The real stumbling block was defining exactly what states a registrant could be in.

Going by a recent event, it appears there really is a situation where someone may have signed up to work up to attending an event, but hadn't actually completed the registration yet. In early version of CONGO, we called this 'subscribed'. It caused all sorts of havoc in the larger events, because there should never have been a situation where someone was subscribed, but not registered.

Cept, it kept happening. The counts of people subscribed compared to the counts of people registered would drift, but in weird ways, and also in ways that were hard to source, due to the funky way CONGO v1 was calculating state of the registrant.

All of this is going away now, with a directly updated and maintained reg_state table that underwent some major surgery tonight. I introduced reg_state to help with this 'what is the current situation of this person' question that arises constantly. Apparently, though, I never quite completed tuning it. To wit:

o The table was a MyISAM table, rather than InnoDB
o It had no foreign key indexes at all
o It was not tracking subscription status at all (the logic was 'if they were in the table, they were subscribed'

Etc etc. That's all fixed now, and logic was introduced to automatically toss the registrant into the reg_state table whenever they were looked up in the current event. So they're added, but NOT subscribed, that can be done later, but their state is no longer non-deterministic.

Layout Schmayout. Just show it
Well, not so much. The ShowRegistrants.jsp file may be my number-one edited file in the whole project. It's the one that determines what the 'registrant zoom' screen looks like - probably the most viewed page in CONGO. And I keep fiddling with it. I'm currently trying to cut down the number of pushbuttons on it, and organize them in a useful way. It's frustrating work, because I'm not really a UI designer. I'm sure I'll good solid feedback from the CONGO power users out there. I hope ya'll like it. *worry*.

The next step is attaching the reworked status display with functions (like 'subscribe' 'register' and 'print badge'). These new hooks really change the workflow of the application, but I think it'll be for the better.

Unfortunately, I've just gotten a poke that the next major event to use CONGO may be going live in about 10 weeks with pre-registration, so I have a hard deadline now for when v2 really needs to at least be in basic useful form. I really don't want to deploy the old version again, so maybe this is a good kick in the pants to work toward a functional version in a reasonable timeframe.

Yeah, but... Vacation?
So, I did mention vacation up there. This weekend I've been up at the house in Maine with the family and the water and the woods. I suppose it helped to get away from the code for a few days, because sitting down tonight definitely got things rolling again. It's the last 'official' day of the summer-vacation-house season, so we did things like pull the sailboat and power boat out of the water, put the docks away for the winter, and cleaned out the freezer. There's always some meloncholy associated with labor day because of this. The weekends spent up here during the summer are great, and we had a wonderful time this summer. On the flipside, we'll start having weekends back at home, which will primarily be taken up with packing for the move to Mosaic in a month or two, but I'll still miss the mornings waking up to loons calling and the calm still lake out our windows.

Another night of hax0ring, with not as much progress as I'd like, again. This time though it was mostly distraction as opposed to actual problems getting in the way. I did finish up the DAO for the Notes, and note creation and updating is working correctly. I'm having a little problem with auto-populating the TIMESTAMP columns, but I'll figure that out. I'm just glad Struts is playing nice with the DAO and Doing The Right Thing [tm]

Oddly, I'm sort of daunted by the next steps, which are going to really start impacting the workflow in CONGO - making direct changes to how Notes are handled, how registrations are managed, and how history is kept. Up until now, things have been pretty much duplicating functionality (or at least adding to existing limited functionality). But I'm looking at rewriting the Registration page and how history and status are being kept. This is at the core of how CONGO works - the very reason that, with something like 50,000 registrations having been run through the software since it's first rollout, we haven't corrupted a single table, or ended up with a mysterious situation where a users status is wrong or lost in a way that was not easily determined.

I'll get past this, I know, but it still makes me nervous.

On the "face of the future' front, though, we have the emerging view of what the database will look like. This first image is the old view of CONGO's tables. No foreign keys whatsoever, that means only programmatically enforced referential integrity.

Here's what my working v2 table structure looks like. I've only been working with address contacts and notes so far, as well as conference master data, but it's all starting to tie together as it should. By the way, these images are generated via dbVisualizer, an awesome generic SQL database tool that has a free mode that's extremely useful (and includes ERD diagrams like this :)

Bit by bit, I'm moving forward.

Today, durnit, I was going to get some work done. I specifically scheduled time after $dayjob to spend working on CONGO v2, and ya know what? I actually did it!

Tonight's joy was parking at the Panera in Marlborough, plugging in the iPhone for music, and getting down to coding. With the network being twitchy in the restaurant, I actually had about 2 hours of uninterrupted focus time to code, and I was productive!

I finished up much of the underlying work on the Notes system (the DAO and Data object are done), and was working on wiring up the Notes interface into Coconut (the web front end). One of my goals with this whole project is to upgrade how I'm managing database resources, and doing things 'properly'. For instance, CONGO v1 had absolutely no referential integrity mechanisms in place. No foreign keys of any sort. This meant I could easily have had orphaned data in many of the tables, there was just no way to tell.

The first step to setting up proper RI is to use foreign keys to link one table to another. The newly formed Notes table now has foreign keys linking to the reg_master table (which holds registrant master information). I did learn a couple basic concepts while doing this though. For instance, if the source of a key reference is null, then the foreign key constraint is not checked (handy!). If it has any other value, it must have a corresponding key in the target table. I found out quickly that specifying source columns as NOT NULL in the database scheme wasn't going to work, particularly on columns that get populated later.

The other thing I wanted to fix was date handling. I had a mishmash of DATE column types and raw text fields storing dates in the old version. MySQL (and most databases) have a handy column type called TIMESTAMP that can not only store time down to fractions of a second, but also support basic math operations ("Show me all rows that have been modified in 'now() - 3 HOURS' is a valid construct).

Here's the problem though. MySQL DATETIME columns cannot be null. So what if I don't have a date to put into the column yet? Well, the docs say the schema should say 'colname TIMESTAMP DEFAULT 0' - I did this, and it seemed to take. But when I tried to retrieve the row in my DAO, I was getting this error:

java.sql.SQLException: Cannot convert value '0000-00-00 00:00:00' from column 6 to TIMESTAMP.

The offending line causing this was ('where' is a Note object, postDate is a java.sql.Timestamp, and 'fromwhat' is a ResultSet):

where.postDate = fromwhat.getTimestamp("note_postdate");

I completely and totally don't understand this. I'm wondering if this is a bug in the JDBC driver I'm using, or what, but I should not be trying to load a date in the format 0000-00-00 into the java.sql.Timestamp object. It should be returning NULL, and therefore 'postDate' should be null, as it was when it was created.

I ended up having to code logic around it to say basically "Don't try to load the note_postdate value unless circumstances can guarantee there was a value there." I tried various approaches for 'Is this column going to be null?' - but it really appears as if the JDBC driver is delivering a Timestamp object, even though the source is null.

It gets a little better. In the MySQL docs, they say, as I mentioned above, that the declaration for a TIMESTAMP without a value should be 'default 0'. But after I create my tables from my schema (using that syntax), and then do a 'SHOW CREATE TABLE reg_notes', lo and behold, I see:

CREATE TABLE `reg_notes` (
  `note_id` int(8) NOT NULL auto_increment,
  `note_rid` int(8) NOT NULL default '0',
  `note_cid` int(8) default NULL,
  `note_postrid` int(8) default NULL,
  `note_ackrid` int(8) default NULL,
  `note_postdate` timestamp NOT NULL default '0000-00-00 00:00:00',
  `note_ackdate` timestamp NOT NULL default '0000-00-00 00:00:00',
  `note_message` varchar(100) default NULL,
  KEY `id` (`note_id`),
  KEY `rid` (`note_rid`),
  KEY `note_cid` (`note_cid`),
  KEY `note_postrid` (`note_postrid`),
  KEY `note_ackrid` (`note_ackrid`),
  CONSTRAINT `reg_notes_ibfk_1` FOREIGN KEY (`note_rid`) REFERENCES `reg_master` (`master_rid`) ON DELETE CASCADE,
  CONSTRAINT `reg_notes_ibfk_2` FOREIGN KEY (`note_cid`) REFERENCES `con_detail` (`con_cid`) ON DELETE CASCADE,
  CONSTRAINT `reg_notes_ibfk_3` FOREIGN KEY (`note_postrid`) REFERENCES `reg_master` (`master_rid`) ON DELETE CASCADE,
  CONSTRAINT `reg_notes_ibfk_4` FOREIGN KEY (`note_ackrid`) REFERENCES `reg_master` (`master_rid`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1

Observant folks may note that the note_postdate and note_ackdate columns now have default values of '0000-00-00 00:00:00'. Not what I specified in my creation SQL.

I'm past this now, and it's working, but if anyone has ideas about why this is behaving this way, I'd love to hear it. It's damned frustrating.

UPDATE 08/28/2008 - Through the magic of RTFM and some nudges from Imre, my table definition was wrong... Here's what it should be:

note_postdate TIMESTAMP null DEFAULT null,
note_ackdate TIMESTAMP null DEFAULT null,

This little bit of MySQL fun is in fact documented, it's just buried in the TIMESTAMP page on dev.mysql.com

As the title says, yesterday hardly went as planned, as evidenced by me just getting around to posting at 9:30 this morning.

The Geekery. It is everywhere!
First, the day was full of IT-related geekiness, trying to get SASL authentication working on the the mail gateway for the Greater Homeport Server Cluster. What should have been an hour long setup and fiddling turned into a 6 hour "Learn what SASL is, learn what TLS is, learn how Postfix manages it, learn how Cyrus implements SASL, learn how to set up standard authentication, and learn how to do this all on an extremely busy mail gateway without killing all inbound mail" process.

In the end, we were only partially successful. The main impetus for this setup was making it so Cat and I could send mail from our iPhones without using Googles SMTP gateway (if you have a gmail account, you can use SASL authentication and send mail from anywhere via smtp.google.com. The drawback? Any mail going through that gateway gets it's From: line rewritten to the gmail account ID.)

We got most bits working, except for finding out at the end that v2.0.2 firmware on the iPhone has a problem with self-signed certificates on TLS-enabled mail servers (but this is only for SMTP connections - Safari et al has no problem with self signed certs. Annoying). One assumes Apple will be fixing this soon, but it means we can't have encrypted inbound mail connections from the iPhone. Adam has suggested there is a list of certs that free and Apple certified, I may end up going that route.

Oh yeah, and about the project I'm supposed to be working on...
But that's not what I came here to talk about.

The point of this week's blogging was to get focused on CONGO v2 and make some progress on getting toward an Alpha release. To accomplish this, I need to get 2 hours or so a night of fairly focused work time.

Last night I failed.

The main culprit? Lack of sleep. I went out to dinner with my mom last night (mm, steak!), and ended up getting home around 8:30. Not too bad, thought I, I still have a couple hours before bed. Flopped down on the couch, had the laptop out, tossed a movie in, and started work and... fell asleep right on the couch. An hour and a half later, I woke up (now about 10;30), was immensely groggy, and decided "Right, that's it. Bed." and flopped right into bed, sleeping like the dead til 8am this morning. Total sleep time? About 10.5 hrs. How I feel today? Stiff, but caught up.

Unfortunately, this meant I got only a minimal amount of work done on CONGO last night. I fixed some of the Notes DAO setups, and started work on modifying the ShowRegistrant functions to enable the notes browsing. I'm trying to decide if the Notes listing should be attached directly to an instance of a Registrant, which means it gets populated whenever a Registrant is instantiated. This really only happens on single views (ala, getting a list of Registrants from a search query doesn't load the entire Registrant, just an index), so it shouldn't make things too burdensome. It has the advantage of making everything easily accessible if you have the Registrant at hand. Right now a registrant's attendance history is attached on instantiation (for Arisia, this is perhaps 20 rows, but that's somewhat unusual). If I don't embed the list in the Registrant object, I'll need a second set of lookups and queries when viewing a registrant, which is a little cumbersome.

For now I think I'll embed it, and hook up the DAO's appropriately. It'll make the display logic a lot easier (data is at hand), and if we end up beating the bejeezus out of the database for largish events, I'll look at optimizing it.

Conclusion
Onward and forward. Today I need to focus on the paying job and finish up an annoying script, hoping to get it audited and approved by the other engineer. Have I mentioned that Perl is not my favorite language? Oh there is a rant pending...

I didn't get as far as I had hoped to tonight, due to a variety of distractions - it was an impressive list of "things that get in the way of previous plans" (Work actually had work for me to do, then a stop by the Mosaic work site to check on some supplies, then dinner, then home to find the freezer has stopped freezing (due to a broken icemaker), then helping out Cat with some finance stuff so we can get our Mortgage application in. Ung!

Finally I sat down, and with The Time Machine on as background entertainment (not so good a movie - great for background noise - and what the heck is with Jeremy Irons? Is there no movie so bad he won't act in it? Sheesh) - I got down to work.

There were some great comments from yesterday on how to handle historical triggers on database updates. There's some challenges to using triggers (such as "who made this update?" - though using a 'last updated by' column on every record could fix this, I'm still not sure if it's a good idea). Jonah pointed out also that using database triggers limits the portability of the database layer. Given that MySQL isn't going anywhere, and that it's getting more and more attention as an enterprise-level database system, I'm not that worried about it, but I am taking that into consideration.

Tonight though I needed to find something short term to work through, so I focused on how 'notes' are applied to registrants. In the past, I would make a "NOTE" entry in the history table. These NOTEs could be anything from just an informational note by the operator ("This person was nice") to something important ("Paid by check"). Having NOTEs in history was "okay" but without filtering, it was hard to see operator comments in a meaningful way.

I also had a concept of 'NOTICE' records. These are essentially NOTEs to be displayed to the operator whenever a registrant logged in. They would show up as red messages on the registrant detail screen, and the operator would have to acknowledge the NOTICE before proceeding (that's the theory - in fact there was no logic to enforce this). These NOTICEs were very, er, noticeable to the operator, and were great for things like "DO NOT ADMIT THIS PERSON UNDER ANY CIRCUMSTANCE" (yes, we had a few of them, and I know of one instance where the CONGO 'NOTICE' record actually did prevent them from entering the event. Yay!). Generally though they were used for innocuous things like "STILL OWES $25" or the like.

For v2, I'm seperating out the NOTICE / NOTES into a separate table simple called 'Notes'. The option to making any note entry a 'notice' will be on the note creation, and it'll just be a flag on the record. Also the record will keep track of when the note was created, and by whom, as well as when it was acknowledged, and who acknowledged it.

The basic DAO structure is there, as is the data object, and I've created the schema file for it as well. Nothing is wired in yet, but I'm also still noodling around how I want the record to look, so I'm sure there'll be more work moving forward.

The other bit I did tonight was trying to squash a problem I'm having with the JDBC connection pooling. I'm using the C3PO connection pooling tool, and if CONGO has been idle for a while, I'll get a connection communications failure. Googling around hasn't showed me anything immediately obvious, though others have said they're having the same problem. What I'm trying now is adding some auto reconnection options to the JDBC connection string, so my c3po setup now looks like this:

private void configurePool() {
	logger.debug("Empty connection pool, recreating...");
	ComboPooledDataSource cpds = new ComboPooledDataSource(); 
	try {
		cpds.setDriverClass( "com.mysql.jdbc.Driver" );
	} catch (PropertyVetoException e) {
		e.printStackTrace();
	} 
	cpds.setJdbcUrl("jdbc:mysql://localhost/congo?autoReconnect=true&autoReconnectForPools=true");
	cpds.setUser("root"); 
	logger.debug("Initted.");
	applicationMap.put("cpds",cpds);
}

I won't know if it's working yet until I idle long enough to force a timeout. I'll let ya'll know.

Just quick update on this. Last night I did get some coding time in after blog-posting. The fix this time was a problem with Create Registrant that was throwing SQL errors all over the place. Turns out I never actually completed writing the DAO for the Registrant data object, and the create() method was broken in a ton of interesting ways.

There's still a twitch in restoring the state to the editor after creation - it's not reloading the new registrant into the sessionspace after creation, so you end up with a sort of 'blank' registrant screen. Exiting and coming back shows the new entry, so it is creating it, just isn't reloading.

I'm sort of worried about keeping history right now. The old code did everything in the history table programmatically - so whenever I made a change to a registrant's information, I had to make a call to logEntry() to create a new history record. So, naturally, there was some inconsistencies in the logging. It also didn't help that the history table was critical information. It was part informational, part auditing. In fact, some parts of CONGO used the history table to recalculate statistics and status on registrants. (Note - that is SO not going to be in the rewrite).

Anyway - regarding logging, I'm looking into learning MySQL triggers to have registrant history updated automatically whenever a change happens. That way I don't have to always remember to put in a logging call. We'll see if this is viable (for instance, how does the trigger know what the ID of the operator is? The one who is actually making the change?)

Stay tuned. :)

So this week I'm faced with a situation I haven't had in front of me in, well, as long as I've been married... at least as long as I've had kids in my life.

This week Cat is up in Maine with Zach. I have my normal work going on during the day, but no commitments for the evenings.

A unique opportunity to be sure.

The Plan [tm]
So here's what I'm going to do. A few months ago I started work on a complete rewrite of CONGO. The rewrite is underway, and has been getting attention fairly regularly over the last couple weeks, but I need to make the final push to an alpha release.

This week, I will dedicate 2 hours a night every night to continuing work on CONGO v2, with the goal of reaching an alpha-testable version by the end of the week. I will also make a post each night with an idea of where my progress is on the rewrite, and what I've accomplished. (I do reserve the right to post the next morning if I'm up until Oh-Dark-Thirty coding and fall asleep in the middle of writing an exception handler.)

Anyone wishing to follow the riveting details of my work, I subscribe to the Commit Early, Commit Often philosophy of source code control, so when I'm working, you'll see commits firing off pretty quickly. If this sounds interesting, you can sign onto the mailing list, or, if you're uber-hip, subscribe to the RSS feed.

Yeah? So? Why us?
So why bombard ya'll with my chattering? Well, I work better with encouragement, or if I know folks care about what I'm doing. Curious about bits of CONGO? Ask! Wanna help out? Give a "wow, kick butt, dude. Go for it" comment or two.

Hopefully I'll have some work to show for tonight, but if not, stay tuned for truly exciting blow by blow Java coding!

There are times when I want to take an axe to Eclipse. Today's particular grief is due to the XML editor. There are numerous faults with it (one of the biggest is that it 'captures' the control-pgup and control-pgdn keys into local tabs, so if you have an XML file open, and you use the keystrokes to navigate through your files, when you switch tabs to an XML file, the only way you can view a different file is to click on it), but todays stabbity-stab comes from the validator.

Eclipse shows you errors and warnings in realtime. Usually, they're fairly accurate. Today I was editing a struts.xml file, and was getting a validation warning. When Eclipse senses a problem with the XML (based on it's DTD), it tells you "this XML file fails the DTD validation".

But it doesn't tell you where.

This completely baffles me. "Somewhere in the following several hundred lines of XML code, there's a problem. But I won't tell you where or what it is."

Off to the W3.org Validation service. I fed the XML file into it, and lo! A detailed description of the problem, and where it was:

Line 26, Column 39:
document type does not allow element "include" here.

<include file="struts-editevent.xml"/>

Was that really that difficult?

Gnar.

I was gone all weekend on a wonderful camping trip, hence the radio silence. Today I'm back in the saddle, and faced with an annoying situation with my home setup.

I've come to the conclusion 'yawl' just can't cut it as a development host. I had no idea that Pentium 4 CPU's were such garbage. A 2.26 ghz machine isn't fast enough to do serious work in Eclipse.

Most of my development is done on 'clipper', but alas it's a Windows machine (it can't help it, it was born that way). I wanted to start fiddling with CONGO (v2.0) via Konqueror, and hit upon running up ye ole VNC viewer on 'yawl', and just 'moving' the Eclipse environment onto yawl's desktop.

Lo, it worked remarkably well. Clipper's monitor is powered down, and I have an Eclipse environment on my Linux KDE desktop, but actually powered by the Windows dual core laptop next to it. I could overlap a Konqueror browse window and flip between it normally, and keyboard interaction was snappy and useful.

It's not quite like having a local Eclipse install, but it's mighty close. It gives me a feel for what it's like for folks having big vmware server installations, and other less fortunate folks dealing with the variety of RDP based solutions, and being able to just pop into an existing running machine and work with it directly.

Really though, I should have a faster desktop machine. Alas, that'll have to wait, it's pretty low on the budget tree nowadays. And, in case anyone is curious, that is in fact a snapshot of CONGO v2 running in a pure Java environment - screens and back end all running within a Tomcat server. Sure it looks identical to CONGO v1, but that's by choice. Quick port to Struts + JSP, then enhance and refactor. Progress is being made.

Over 5 years ago, I began work on an application called CONGO, a registration and badging system, not only in the vein of "There's gotta be a better way", but also to help out a friend who was running registration at con in Philadelphia.

The first versions of CONGO were crude, but worked well - with a custom Swing interface, custom terminals and servers, it was an impressive setup. What wasn't apparent was this was my first foray into writing any decent sized Java application. I made what I thought at the time were good design decisions, and looking back on 30+ events run, and hundreds of thousands of badges printed, I'd say the design was, for what I knew at the time, solid.

But it's time to change. With everything I've done over the last 3 years in Java and enterprise-level applications, I feel I'm ready to rewrite CONGO into a proper Java based webapp. CONGO 1.0 was part Java, part PHP, part templating. CONGO 2.0 will be a pure java application, based on Struts, and will allow features that I could not shoehorn into the old model

I started work on CONGO2 about a week ago, and I have my first screens working properly and chaining together as they should. Part of this process is factoring in the old logic code, and linking it up with the new presentation layer I'm building using JSPs and Struts. This means making calls to 12,000+ lines of Java code that was written as I was learning the language.

But one thing I did consistently - I documented my methods. Every piece of the CONGO appserver has full Javadocs describing how to call the method, and what it returns

Because I'm doing CONGO2 using modern tools (like Eclipse and Tomcat, my GUI is showing me these docs whenever I try to link to my old code, and is reminding me how to use the old libraries. Why is this remarkable? Because other than generating static documentation, I've never actually seen my own comments and documentation popping in interactive windows in my IDE. Until now.

It gives me a little thrill each time I see my own docs pop up as helper windows in Eclipse, showing me comments I made half a decade ago on structures and calls, and reminding me how to use the system.

It wasn't easy to learn Java, it wasn't easy to learn Swing, it wasn't easy to learn JSPs, JSTL, and Struts. But now I do know them - some more than others, and I'm enjoying this massive refactoring of one of my proudest creations.

I don't talk about my day job much. For those outside of the Java Geeky circles, it's pretty dull stuff. But today has been particularly active, so I thought I'd share some of the bits I learned while teaching myself the wonders of Struts. This can best be summed up by the following revelations...

1) When changing a struts.xml file, within an Eclipse WTP environment, being serviced by Tomcat, WTP does not click that the Tomcat server needs to be restarted. The 'Dynamic' part of WTP does not take effect, and no matter how many times you save what you're working on, it won't go live in the server until you actually restart Tomcat.

2) Despite all the awesome advances in Eclipse in the last few years, there's twitches that still drive me absolutely gonzo. One is that the 'smart insert mode' (which helpfully closes XML tags and closes quotes and parentheses for you) cannot be turned off globally. You can do it on a per-editor basis, but not globally, so I'm always forgetting to turn it off when I open a new editor. Another is XML syntax validator is still quite twitchy, and occasionally will flag incomplete tags or non-well-formed XML when the file is just fine. A close and re-open fixes it, but ugh.

3) Last, but not least - that which almost got me up on the roof with a high powered rifle. I give you a quiz as my example. One of these two struts.xml configurations apparently tickles a bug in Struts2 and will cause an internal server error and stacktrace with a Null Pointer Exception. The other will not. Can you figure out which is which?

<action name="/*">
	<interceptor-ref name="mystack" />
	<result name="success">/WEB-INF/jsp/{1}.jsp</result>
	<result name="login">/WEB-INF/jsp/index.jsp</result>
</action>

<action name="*">
	<interceptor-ref name="mystack" />
	<result name="success">/WEB-INF/jsp/{1}.jsp</result>
	<result name="login">/WEB-INF/jsp/index.jsp</result>
</action>

1. INFO: Server startup in 954 ms 2. Jun 17, 2008 7:41:14 PM org.apache.catalina.core.StandardWrapperValve invoke 3. SEVERE: Servlet.service() for servlet default threw exception 4. java.lang.NullPointerException 5. at com.opensymphony.xwork2.config.impl.ActionConfigMatcher.convertActionConfig(ActionConfigMatcher.java:168) 6. at com.opensymphony.xwork2.config.impl.ActionConfigMatcher.match(ActionConfigMatcher.java:144) 7. at com.opensymphony.xwork2.config.impl.DefaultConfiguration$RuntimeConfigurationImpl.getActionConfig(DefaultConfiguration.java:316) 8. at com.opensymphony.xwork2.DefaultActionProxy.prepare(DefaultActionProxy.java:169) 9. at org.apache.struts2.impl.StrutsActionProxyFactory.createActionProxy(StrutsActionProxyFactory.java:41) 10. at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:494) 11. at org.apache.struts2.dispatcher.FilterDispatcher.doFilter(FilterDispatcher.java:419) 12. at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) 13. at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) 14. at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) 15. at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) 16. at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) 17. at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) 18. at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) 19. at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) 20. at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874) 21. at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) 22. at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) 23. at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) 24. at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) 25. at java.lang.Thread.run(Thread.java:619)

How the hell am I supposed to debug that?

I finally did, after a good 2 hours of hair-ripping, gnashing of teeth, and spewing my venom upon the #struts IRC support channel.

Ever since I first wrote Coconut, the web interface to CONGO, I've been using the same stylesheet. It was really my first foray into stylesheet-driven design and layout, and while it was a good first try, it was ugly as sin.

Last night I finally sat down to rewrite the stylesheet into something reasonably attractive. I got rid of the dark background and greytones, and the 'All Caps' font styles, and used more pastels.

I'm pretty happy with the result, but for the hundred or so folks who are used to the old look and feel, it'll be a dramatic change!

Somewhere, back in the mists of UI design, some bright light came up with the idea of making options in an application or desktop environment selectable by using a mouse to point at a menu along the top of a work area, clicking on the menu, and having it present more options.

This was called a 'Pull down menu'

Handy! Picks were readily available when needed, and stayed out of the way until requested.

I'm fairly sure the original designers of menuing systems never said to themselves, "Hey guys, I think it would be a good idea to SHOW A PULLDOWN MENU JUST BECAUSE YOUR MOUSE HAPPENED TO ROLL ACROSS IT ON ITS WAY TO PAUSE YOUR MUSIC!

I'm flabbergasted that the 'popup menu' has become de rigeur in web pages. It's a terrible UI design, and consistently gets in the way when trying to do other things. There's no other menuing system on the planet that does this. Menus should show up when you ASK for them, not because you happen to TOUCH it. That's what the little button the mouse is for. See? Clicky clicky. Now you try it.

Not that I expect this to change anytime in the near future. There's a revolution going on in UI design now, with AJAX services redefining what it means to write a UI. Unfortunately, I have yet to see any AJAX application that comes near the functionality of existing systems. It appears programmers are going back to 1985 and starting over, thus jettisoning all the lessons learned in the last 20 years.

Tonight, I am Da Man.

It's rare I get to sit down and see, so directly, a problem in the code I'm working on. In this case, while running at Ubercon the other weekend, I noticed that CONGO was showing every sign of a memory leak. After a while I'd get sluggish performance, and eventually I'd get an OutOfMemory error from the JVM. Didn't look good.

This had me in somewhat of a panic, since Arisia is right around the corner, and I need the system stable and useable for the event. With the help of the wonderous tool that is JConsole, I was able to actually see the memory usage going up in the JVM as I worked with the server. There was definately a problem.

It turns out my hastily set up code to handle changes to my JDBC driver was not being handled correctly. Apparently with the old mm.mysql driver, SQL handles were automatically closed at the end of use. With the new Connector/J driver from MySQL, you had to explicitely issue a close() statement on any Statement, or the handle would never be freed. This was probably always the case, it was just my sloppy code and a twitch of the old driver that made it never come up.

Unfortunately, this necessitated changing just about every SQL call in the server, all 10,000 lines of it. It was long, painful, boring work. I had some help from blk, but it still took a good week of off-hour fiddling to finish the changes and do some basic regression tests.

Tonight, I made the last set of changes, did some basic runthroughs, and didn't have a single crash. With someone trembling fingers, I started up JConsole again, connected to the application server, and started working with CONGO. Registrant lookups, reports, modifications, creations, subscriptions, convention detail editing - throughout it all, JConsole happily continued showing the normal 'sawtooth' pattern of memory usage. Things would get allocated, used, and then the garbage collector would come along and reap the memory.

Success! I had successfully searched for, found, and fixed a major memory leak that was persistent throughout the code.

For those insisting on the details, here's a properly structured, Prepared, and appropriately exception-managed call to get a number from the database. (And, before I get abusive comments all over the place, the Exception handling here is VERY POOR, and will result in a clean result coming back from the method, even though the call may have crashed. I know, thank you, now go away).

private static int calcRegistered(int cid, String typeName) {
        PreparedStatement p = null;
        ResultSet rset = null;
        String sql = "SELECT count(*) AS total FROM reg_state " +
            "WHERE state_cid=? AND state_regtype=? " +
            "AND state_registered=1";
        int value = 0;
        try {
            p = cserver.Conn.prepareStatement(sql);
            p.setInt(1,cid);
            p.setString(2,typeName);
            rset = p.executeQuery();
            if (rset.next()) {
                value = rset.getInt("total");
                logger.debug("\tcalcRegistered: count for conference '" + cid + "', type '" +
                   typeName + "' is --: " + value);
            }
        }
        catch (Exception e) {
            dumpException(e);
        }
        finally {
            try { p.close(); }
            catch (Exception e) { dumpException(e); }
        }
        return value;
    }

I feel all geeklystudly.

Next is to start seriously hammering away at the buglist, and get a test environment set up so other CONGO users can test out the new Merge function against 'real' data. If all goes well, we'll roll this version into production in time to run Arisia.

Aaaaand, we're back.

What an insane couple of days. There's a whole series of posts brewing in my head right now, but I'll just touch on probably the one that's most on my mind.

Last week, I was in mid-preparation for Ubercon, an awesome gaming convention I regularly work down in NJ. CONGO has been pretty idle for the last few months as the summer is not a big time for conventions. With a week to go until Ubercon started, it was time to pull out all the hardware and make sure everything was working.

Well, unsurprisingly, it wasn't.

The first major issue was coming to the conclusion that endor, the venerable server of dozens of conventions, really wasn't going to handle Yet Another Apt Upgrade. It was still running on a baseline Debian Sarge install that had been upgraded a dozen times over the years, and finally, after enough apt tweaking had gone on, a dependency just wouldn't resolve, and the machine would not take new upgrades. It was time to nuke from orbit and reinstall the OS from an Ubuntu baseline.

No problem, sez I. I whip out my Gutsy Gibbon Ubuntu CD, do a quick database dump and backup, install Ubuntu, and restore my home directory and databases. Reinstalled CONGO, loaded the working databases and...

stopped.

See, CONGO was my first big Java application. And as such, it has some... intriguing ways of doing database work. And by 'intriguing' I mean butt-ass stupid. In particular, not using PreparedStatements for SQL commands, not checking for failed transactions, etc etc. While this was okay for a fairly static application, once you replace the entire OS underneath it, things start to get a little squirrely. And, well, we had squirrels aplenty.

So over the space of 3 days I basically had to rewrite every SQL interraction in CONGO, resulting in some fairly major code changes, and all of this 3 days before a con.

Add on top of this the fact that I'm also using a brand new printer from Evolis for the first time under Linux. There's a whole nother post about this experience, but it did bring yet another variable into play. Oh, and did I mention that I rewrote the print routine in CONGO to generate PDFs on the fly and use them for badge rendering? Yep, also new.

Needless to say, things were a little panicy leading up to the event. Fortunately, by Wednesday afternoon, I had things fairly well stabilized, the code worked, endor was stable and functioning properly, and I could start packing for the event with a clear conscience.

The end result? It all worked. The convention went fine, the printer behaved wonderfully, running badges twice as fast as my old Fargo printers, and we had only minor glitches through the weekend. There's still some work to do to get endor ready for larger events, but for a week that started out with totally broken software, an unuseable server, and an untested printing system, things went mighty well.

Onwards.

... and the cast of thousands that made installing a Sun JDK onto Linux as simple as:

root@endor:~# apt-get install sun-java6-jdk
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
gcc-3.3-base java-common libstdc++5 odbcinst1debian1 sun-java6-bin sun-java6-jre unixodbc
Suggested packages:
equivs binfmt-support sun-java6-demo sun-java6-doc sun-java6-source sun-java6-plugin ia32-sun-java6-plugin sun-java6-fonts
ttf-baekmuk ttf-unfonts ttf-unfonts-core ttf-kochi-mincho ttf-sazanami-mincho ttf-arphic-uming libmyodbc odbc-postgresql
libct1
Recommended packages:
libxp6 libnss-mdns gsfonts-x11
The following NEW packages will be installed:
gcc-3.3-base java-common libstdc++5 odbcinst1debian1 sun-java6-bin sun-java6-jdk sun-java6-jre unixodbc
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 42.3MB/43.1MB of archives.
After unpacking 128MB of additional disk space will be used.
Do you want to continue [Y/n]?

Up until recently, getting Java onto a Linux box was, well, not difficult, but certainly not trivial. I'm happy to say it's gotten as easy as installing any other package, which, with current package managers, means it's a breeze.

Yay progress.

While working on my current Java project, I needed to better understand how the Preferences API stores data. My initial instinct was to dig around in my home directory looking for a .java/Preferences file or something similar. Perhaps under Linux, this would have been the case, but Preferences is platform agnostic, and under Windows, it uses the Windows Registry.

I've long since tried to drive out my knee-jerk reaction to the Registry. The initial implementations of it were awful, and prone to easy corruption. Navigating the registry has never been what one would call 'simple', but after a while things settled down a bit and stabilized.

In this case, I was using a simple Java command to store some values for a Webstart launched client:

prefs = Preferences.userRoot().node(nodeName);
prefs.put("ExpoLoginName",userField.getText());
prefs.put("ExpoPassword",new String(passwordField.getPassword()));

Pretty straightforward stuff. In general, the code was working as expected, but I was seeing doubling of some of my values. What I needed to be able to do was actually see the registry stored values themselves, without going through the interface, to see if I was actually storing the values properly, or things were going wrong on the way out.

This unfortunately brought me in contact with the Windows native tool 'REGEDIT'. This tool came out quite a while ago, and is the defacto standard way of editing, browsing, and searching the Windows Registry. Unfortunately, regedit hasn't seen any interface upgrades in approximately an eon. It is painfully difficult to work in, and while yes, it gets the job done, little interface quirks can flat out destroy productivity. For example. The FIND function (amazingly, bound to '^F'), does in fact let you search the registry for a key or value. But, it is a one-way search, from the current position forward. It does not wrap. It also does not RESET when you change to a new search term. So if your first search finds a match, say, at the 5th from the bottom key (by the way, the window does not scroll to show you where your match hit. You have to scroll it manually), and you decide to search for a second term, it will only search the last 5 rows in the registry.

I lost half an hour searching for keys I KNEW were in the registry, because my first search had set the pointer so low. GAARRRHHHH!

What I found most entertaining about this path was it gave me a chance to look under the thin veneer of civility that Windows brings to the operating system experience. Take for example the screen shot above. Note that the keys stored in the registry have been altered from their internal values. Some bright Windows developer back in the dark ages realized they should tell the difference between an upper case letter and a lower case letter. Rather than, say, writing that into the logic of the registry, they cleverly decided simply to preface upper case characters with a forward slash.

Ah, but that brought up a problem! You could now not store a forward slash into the registry! Not to worry, we'll just change any forward slashes in the registry to backslashes! Sounds like a great plan!

This sort of tortured logic is what you see happening in student built technology experiments, hacked together during all night jolt-driven marathons. It's not something you'd see in the largest software company in the world. Their programming practices, in theory vastly improved from the early days, wouldn't allow such awful hacks.

Would they?

Update - it appears, due to notes in the comments, that the registry values hack is due to Java's interpolation, not Windows. Java needs to specify slashed items because the backing store being used in this instance (Windows Registry) is case insensitive. If a different backing store were being used, that wouldn't be happening. My apologies to Redmond for assigning blame. On this one thing. :)

Recently I had to spend a fair amount of time working on CONGO on a Windows XP platform. Stonekeep is doing doing it's first event where CONGO will be running entirely on Windows XP, so all the environmental stuff I'm used to having under Linux doesn't work, naturally. Things needed to be updated. This presented many challenges...

Grump. I had no idea, but Jarindex was not working at all. Searches were coming up blank. Frustrating, when it's supposed to be a handy search tool for finding classes in the vast swampland of Jar files out on the net.

Anyway, I fixed the indexer, and loaded up Hibernate 3.2 and Jboss 4.2.0 into it. Probably should update other packages as well. Currently I have:

apache-ant-1.6.5 eclipse-SDK-3.1.2 jakarta-oro-2.0.8 jboss-4.0.2 jdk1.5.0_04 jetty-5.1.10 junit3.8.1 OROMatcher-1.1.0a velocity-1.4
axis2 hibernate-3.2 jakarta-slide-webdavclient-bin-2.1 jboss-4.2.0.GA jdom-1.0 jpcap-0.01.16-win32 lps-3.1.1 rss_utils_1.1 velocity-tools-1.2

If any Java geeks want to see other packages loaded into it, let me know.

This is a geek gripe. Particularly for Java programmers. If you're not doing JEE development, this probably won't make a lot of sense unless you like seeing open source companies being beaten up for version incompatibilities.

Hello, I'm Dave, and I'm a JBoss developer. (Hi Dave...)

Until recently, I was quite content with the series of tools available for JBoss4, and my chosen IDE, Eclipse. JBoss provided a lovely little plugin interface called JBoss IDE. I encourage people to click on that link, because you'll walk through a series of redirects until you land on a page that, amazingly, has no information on where to get JBoss IDE! How wonderful!

It turns out that JBoss (now wholly owned by Redhat has pulled JBoss IDE in favor of another Eclipse plugin called Exadel. Fantastic, a new IDE plugin that has a lot more functionality than JBoss-IDE ever had. Lets take a look!

An hour later, and Exadel is installed and running and grand. But. Wait a moment, the latest version of the JBoss application server is 4.2-GA. That's General Availability. Meaning the platform is released and is the recommended system for users.

Exadel has no configuration support for 4.2-GA. Only for 4.0.x releases.

And JBoss-IDE has been pulled completely (and even it's '2.0.0-beta' version, the most recent version they posted, did not support 4.2).

"Must be coming out shortly." So I mailed off to Exadel tech support asking if there was 4.2 support in the works, or when it will come out.

I get a direct, and undeniable response from them:

Dave Belfer-Shevett wrote:
> > Exadel Support Team wrote:
>> > > Exadel Studio Pro is going to be re branded as Red Hat Developer Studio in
>> > > later summer. Red Hat Developer Studio will support JBoss 4.2 .
> >
> > So the answer is "no, it does not support 4.2 now, and won't until late
> > summer" ?

That's correct.

-The Exadel Team

Excuse me, but WHAT THE F??? We've pulled the old IDE toolset. We've released a new product. But you CAN'T USE IT WITH OUR IDE TOOLS! Hahahahha! And you won't for a couple months. Sorry bout that, have a nice day.

There are workarounds. You can run the jboss server externally in a windows shell and deploy to it. This is a painful arrangement, but I guess I have no choice?

Thanks for leaving us all in the lurch, JBoss.

It feels good when you finally push a release of a product out the door.

The last 2 months I've been spending a chunk of time doing updates on Keystone. It's always fascinating working on code that is pushing 10 years old, seeing things I remember writing during the dotcom boom, and finding other tidbits that make me go "What the HECK was I thinking?"

This release is interesting in that much of the work was pay-for contract updates from a customer who wanted to see some bugs fixed and a few enhancements added in. It's nice working on something you enjoy, and getting paid for it as well.

The code is just about ready for it's next major overhaul though. I'm trying to get up the gumption to rip out the entire database back end code and replace it with ADODB. I wrote my own database abstraction layer back in, oh, 1998-ish, but really, the world has moved on a bit since then. It's time to move with the times.

About a year ago, I switched my primary source code control system from the venerable old CVS to the (relatively) new kid on the block, Subversion. On the whole, I've been ecstatically happy with the system. It patched many of the ridiculous problems with CVS, and added on things that opensource community has been asking for for ages (like 'rename'), but never made it into CVS.

Now I have all my projects stored in SVN, and my main client is using it as well for their code (they've chosen to go with SVN and are planning to End Of Life their VSS server - to the dismay of no one).

Subclipse
One of the best tools that made this switchover workable (aside from SVN's similarity CVS in many respects, particularly on the command line) is the Subclipse plugin for Eclipse. Subclipse provides a great easy to use interface into SVN servers, giving all the functionaly one would have on the command line via a very simple, tightly integrated GUI.

One thing that had been bugging me, however, was the access methodology I was using to get to my (remote) SVN server. It involved setting up a tunnel in SecureCRT (though Putty can do it as well), and then telling subclipse to use my 'svn://localhost/stonekeep' repository.

While doing some surfing, I found that Subclipse supports the svn+ssh syntax for specifying the repository. "Great!" says I, "I won't need to set up the tunnel each time!"

A few more fiddles, a pleasant discovery of a configuration screen in Subclipse, and I had an SVN over SSH connection to my repository, even using my ssh key pair.

Danger, Will Robinson!
But wait! All is not well. When I tried to browse the repository from Subclipse, I quickly hit this error:

Could not open file system at /var/lib/svn/stonekeep (13)Permission Denied: Berkley DB Error while opening environment for file system /var/lib/svn/stonekeep/db:

This vexed me, because I had been having no problems accessing the repository locally on the server, and over my ssh tunnel. Both used the locally running 'svnserve' on the repository host, so why wasn't the svn+ssh connection using it?

The answer comes in the SVN documentation, and via a little research:

What's happening here is that the Subversion client is invoking a local ssh process, connecting to host.example.com, authenticating as the user harry, then spawning a private svnserve process on the remote machine, running as the user harry. The svnserve command is being invoked in tunnel mode (-t) and all network protocol is being “tunneled� over the encrypted connection by ssh, the tunnel-agent. svnserve is aware that it's running as the user harry, and if the client performs a commit, the authenticated username will be attributed as the author of the new revision.

When running over a tunnel, authorization is primarily controlled by operating system permissions to the repository's database files; it's very much the same as if Harry were accessing the repository directly via a file:/// URL.

The Problem With This
I'm really unhappy with this model. The problem is that now the user must have read/write access to the entire repository tree. When using a local socket connection (or one over ssh via a normal tunnel), the Subclipse client connects directly to the svnserve process running on the repository box, and interactions with the server happen under that processes ownership.

The svn+ssh protocol does not use the svnserver on the target machine. It tunnels the command to a user-invoked svnserve process, and that process must have read-write access to the repository.

"Well gosh, that doesn't seem too bad. What's the issue?"

The issue is that to make this methodology work, I have to give the user read/write access to the repository tree. Meaning, they could happily type 'rm -rf /var/lib/svn' and destroy the entire repository. Even worse, the configuration files (including the password / access file, which has passwords in plaintext) must be made available to the general users.

Why svn+ssh doesn't simply make a local socket connection to the svnserve process already running, I don't know. But I can find no way to make that happen.

The fix?
As far as I can tell, there really is no direct fix for this. There are various workarounds, which the SVN documentation discusses, including setting up an 'svn user' for the svn+ssh logins, and the possibility of using unix groups for permissions, but I feel that if you have a listening socket server on your repository host, you should use it, not introduce a second methodology and have to jump through hoops to implement it.

For now, I have to abandon the svn+ssh possibility, and go back to my hand-configured socket tunnels. There's no real loss here - they work remarkably well, are very secure, and quite stable. The slight annoyance of having to open up a SecureCRT session before doing work in Eclipse is just that - a slight annoyance. I've dealt up until now, and I'll just continue to deal.

I ended up with a sort of special bonus afternoon today as Cat is off in town, and Zach is over his friends house. The basic idea was to make up for a very interrupted week of work and try to get some long-neglected work out of the way.

Mostly, it worked quite well. I got caught up on a couple things that had been nagging, and that's good. But what really got me grooving was working on... well, a project that a couple folks know about, but I'm not -quite- ready to release to the public. I've been referring to it simply as the "Secret Squirrel" app, or just 'The Squirrel' for short.

What's been good about this has been the basicness of the design and implementation. I've been spending SO much time delving into the deep complexities of EJB3, JBoss, Hibernate, and more recently Terracotta, I don't get much of a chance to just sit down and whack out simple code.

Today had 2-3 hours of basic grunt 'hamburger' coding, to borrow a phrase from MASH. No fancy libraries, environments, or oddities. Well, okay, one new library that handles OpenGL rendering, but everything else was basic data application coding and design.

It felt good! It sort of underlined where I've gotten with Java programming over the last 2 years or so, and now basic coding is second nature. Eclipse has become my environment of comfort, whether it's running on Windows or Linux (in fact I'm leaning toward moving back to Linux full time soon - with both machines up and running cleanly on my desk, I find myself doing more on the Linux box than on Windows... but I digress...) , and the Java OOP approach to program development Just Plain Makes Sense. It's nice.

Tomorrow it'll be back to EJB3 and Terracotta land, and all the mysteries therein, but for now, I can look at what I did today and do a "That felt good!" happy dance.

And the code even worked.

About 10 years ago, I started writing an application that would have a profound impact on my life. Keystone started out a simple problem tracker, grew into a mature product that was getting 3000+ downloads everytime I did an update (about once every 3 months), and was ultimately sold to a DotCom that basically killed it in its tracks. That sale let me have a few toys and was a high point of the dotcom bubble for me.

3 years ago I re-aquired the rights to the application from the failed dotcom, and set about upgrading the vastly outdated software. My user base had for the most part wandered away to other applications, but there was still interest and heck, it was my application, I wanted to do things with it.

But other projects were taking precedence, and Keystone languished.

On a recent trip down to DC, I had the opportunity to spend 7 uninterrupted hours on the train, each direction, with nothing but a laptop and a music library to keep me company. After trying to get my current projects working, I settled back into "well, maybe I'll work on Keystone some more."

In those 2 train rides, I did more upgrading, tinkering, and fixing in Keystone than I've done in the last 3 years. I revived the contact manager and fixed all the dependency problems. I continued the changes needed to bring a PHP application, written in 1995, up to 2006 standards. Keystone is over 12,000 lines of code - not a trivial application, but not so huge that it is an unassailable target.

The question is - why do this? Sure, part of it is 'this is my baby, I want to see it succeed', but in the back of my head, the question burbles... "Can lightning strike twice?" - can I make this a successful opensource application again?

I'm certainly not deluding myself into thinking "THIS WILL BE THE NEXT KILLER APP!" - that's a foolish and unrealistic mindset. But can I bring it back to where people are using it, they like it, they contribute suggestions and fixes, and the application continues to grow?

I'd like to think I can. But the code still needs a lot of work, and there are some design decisions that will most likely require huge chunks of code being ripped out at the roots (database connection methodology has advanced SIGNIFICANTLY since 1995).

It's a nice dream, I sort of miss my users. Maybe they'll come back.

In a previous post I admitted to the world that I, an avowed Linux weenie, was now using a Windows desktop for all my geeky endeavours. This continues to be true, but I've taken the steps necessary to make my environment comfortable to work in, without going the easy route of "I will do everything in my power to make Windows look and feel just like my Linux box." To me this defeats the purpose of potential learning experience of working with a new environment.

So about those tools...

Rain. Cold. Cloudy.

These to me are the harbingers of only one thing.

An excellent day of coding.

And an excellent day it was. Almost 12 hours, with a meeting and a good lunch to interrupt, had me rewriting an entire interface to the application, reworking a build script so it was more efficient, and implementing a 'Priority 1' change for the current programming sprint.

I'm coming to a happy medium with Eclipse and my forced migration. All in all, a very good couple days, with this one as a topper.

Tomorrow I have another bit of work, then drive back up to Boston. But tonight, it's time for food, then rest.

My ongoing conversations with Eclipse (see this older post for Pt 1)

dbs "Edit that."
eclipse "no."
dbs "but it's open in the editor."
eclipse "nuh uh."
dbs "*enter*enter*enter* CMON!"
eclipse "nope"
dbs "owait, i'm in the svn browser aren't i?"
eclipse "_I_ knew that. Don't know what your problem is.."
dbs "fine, java perspective, now edit."
eclipse "if you insist."

... can learn new tricks?

As part of my current contract, my employers are embracing new procedures and techniques for application development. While the normal buzzwords of "EXTREME PROGRAMMING" and "Agile Development" are being bandied about, they're not being whole-hog embraced, with the managers marching zombie-like into an undeliverable product schedule. We're trying new techniques such as scrum product development, and utlitizing sprints to structure short-term deliverables and milestones.

Since I got the core of my production programming experience with this same company almost 18 years ago, when there were only 3 employees, me being number 3, I find it fascinating to see the same programmers trying new tacks to take on the complexity of developing modern code.

In the Good Old days, our entire code base was perhaps 80,000 lines. One developer easily understood every aspect of the application, and could comfortably keep up with customer and internal needs. The current codebase is over a million lines and growing, and no one programmer understands it all. This is not a disaster story, it is one companies' normal evolution as they grow and expand. In this case, they have lucked out to have a person in charge who is not only a brilliant programmer, but also an open-minded manager and a good communicator. He recognized the pitfalls of 20 years of development on his product line, and is making careful, planned, and deliberate changes to the company's development methodology.

Of course, the panic element for this is... the changes being applied involve shifting their platform onto a framework I designed for them over the last 18 months. No pressure, really. It's only the future of this company and it's 20-some odd employees, not to mention the respect and 20 years good will of the owner. The same owner who gave a young programmer his first production development position.

Have to point this handy tool out to folks.

Because of the complexity of my desk, I can't really have a pile of keyboards and mouse lying around. I switch between machines constantly, and switching keyboards would just drive me batty. I had been using X2VNC for quite a while, which, while 'okay', had it's own quirks. One of which was it couldn't work in reverse (I could go from an X host to a Windows box, but not the other way around). It also wouldn't let me slide from an X display to another X display. Pretty limiting.

Enter Synergy2, a simple client-server tool set that lets you configure multiple displays to a single 'server' that controls the mouse and keyboard. With Synergy2, I'm able to configure all my machines in whatever configuration I want. At the moment I can slide my mouse off my primary Linux display onto my WindowsXP box, across it's display, and onto my laptop.

That's pretty neat, but.. the kicker? Synergy2 manages clipboard cut n paste operations across machines. If I slide over to hunter (the laptop running Ubuntu linux), highlight something and click 'Copy', then slide back to my primary desktop on yawl, I can just click 'paste' and it works. This is miraculous to my eyes!

If you run more than one machine on your desk, I highly recommend Synergy2. Available in apt repositories everywhere.

This week I'm on the road to scenic Pittsburgh for a few days. While there are various wonderful aspects of this trip, there is also the bonus of being to test out various geeky projects that have been percolating around.

Folks, family, and friends may have noticed I've been fairly absent from posting, chatting, and general socializing for the last few days. A chunk of this mysterious quietness can be attributed to digging my fingers into a coding project I've been thinking about for a while.

I've just released JRemoteforMyth. This is a webstart-enabled app for folks who tend to hang around on their laptops or desktop machines, with a MythTV box nearby. It allows for a small 'remote' application to sit on their desktop and control the Myth box remotely via normal "up" "down" "enter" buttons.

There were a number of challenges writing this. I haven't written Swing applications in a few months, and I'll be needing to get back in the saddle shortly for some consulting work I'm doing. This was a great 'simple' defined application I could whip up that had a fairly definite design, goal, and completion metric.

The second challenge was this was my first full webstart-enabled app written entirely inside Eclipse. For the most part, this didn't present any particular problems, and went quite smoothly. I'm still ecstatic about Eclipse' SVN and CVS repository integration. Using CVS to hold my working files, I was able to switch from my laptop to my desktop machine with only a Project->Team->Commit on one machine, and a Project->Team->Update on the other. Yay!

Last but not least, a grump. When writing a Webstart enabled app, the resulting 'jar' file that containst he app is published onto a webserver, and the JNLP descriptor file tells webstart well to download it. Webstart, however,will only install and run 'signed' jar files. Eclipse provides no mechanism for signing jar files internally. It can generate the jar file for the application, but several command line tools need to be run to actually sign the file. Very annoying.

I've announced the app to the mythtv-users list, and I'm seeing downloads, so SOMEONE is interested in it.

Next will be some basic enhancements to it, adding things like keyboard entry, an multiple tabs so the 'keypad' can be displayed along with normal navigation keys.

It's nice to be back into coding for a while. Of course, one of the drawbacks is coming across code written over half a decade ago that should never have seen the light of day. Take for example this gem I ran across while continuing with my updating Keystone project...

$pq = db_query("select dtable,dcolumn,dflags from dictionary $wclause");
while ($pd = db_fetch_array($pq)) {
        $varname = "op_$pd[dcolumn]";
        eval("\$oval=\"\$$varname\";");
        $varname = "sp_$pd[dcolumn]";
        eval("\$sval=\"\$$varname\";");
        if (empty($oval)) {
                        $oval=0;
        }
        $active = $sval ? "1" : "0";
        $subtable = ($pd[dtable] == $proptable) ? '' : $pd[dtable] ;

It's been said there's a special hell for people who write code that uses the eval statement in production code. Apparently I'm headed there already. This was a very bad coding decision, but I remember actually writing this particular snippet. It was around 1998, and I was flying to California to talk with the company that would eventually buy Keystone from me. I seem to do some of my best coding work while flying on airplanes, though this sample isn't exactly a sterling example of it. It did, however, enable one of the cooler features of the product - the ability to, using any of the various data sets, set up a custom browse view based on the structure of the table.

Using 'eval()' statements and depending on global variables was NOT the way to implement it though. Now that I'm converting all 12,000 some odd lines of Keystone code over to support running on a system that doesn't have register_globals enabled, it was time to update this particular code snippet. It took a good 1/2 hour to figure out exactly what it was doing, but once I did that, it was a simple change to:

            $pq = db_query("select dtable,dcolumn,dflags from dictionary $wclause");
            while ($pd = db_fetch_array($pq)) {
                $oval = $_POST["op_" . $pd[dcolumn]];
                $sval = $_POST["sp_" . $pd[dcolumn]];
                if (empty($oval)) {
                    $oval=0;
                }
                $active = $sval ? "1" : "0";
                $subtable = ($pd[dtable] == $proptable) ? '' : $pd[dtable] ;
              }

Don't see much of a difference? It's a big one from a code security and design standpoint. Don't sweat it too much, it means a lot to me at least.

The conversion is moving along nicely though. I think I can have eval versions ready for folks to test out within a day or three, if I keep this pace up. I'll be curious to see what sort of response I'll get on the net to the system. It's been a while.

Back in the dark ages, I wrote a very successful ticketing system called Keystone. This was a successor to another reasonably successful program called PTS, which I wrote while working at Fidelity using this new fangled thing called 'PHP/FI'. Online ticketing systems were still something of a novelty, and none existed in the opensource world (save for GNATS which, at the time, was an abysmal piece of code.

PTS, and later Keystone - flourished. In the dotcom runup, Keystone was a hot item. Whenever I posted a new version and announced it on Freshmeat, I would see thousands of downloads of the updated package within days. At conventions, I was well known. "YOU WROTE KEYSTONE? HOT DAMN!"

Alas, the dotcom era came along and an opportunity to sell Keystone dropped into my lap. I took it, along with a sizeable check, in exchange for an agreement with the new owners, that they would continue to support the product, as well as let me continue working on it.

Neither of these things actually happened. When the bubble burst, the owners found themselves holding a piece of software that they were not using, and still owed money on. The crunch came, I called in the debt, they couldn't pay, so the entire system reverted back to me, lock stock and copyright, in exchange for me foregoing the outstanding balance.

On the one hand, yay, I got my program back. Hard to argue with that. On the other hand, 2 1/2 years had passed. The new owners had not touched the code, the industry had advanced a LOT in that time, and the competitors, which had excellent products at the time of the buyout, now had a 2 1/2 year jump on me.

Keystone languished.

Over the last few months, several people have been poking me about the code. Some clients have been running versions for YEARS, and are asking if there are updates coming. Others would like to set up new installations. I've half-heartedly worked on updating Keystone off and on a few times, but never really finished the changes into something I could release again.

Tonight I spent 2-3 hours dusting off the old code and continuing the updating process. Files with revisions half a decade old are coming up in my editor, with me vaguely remembering even writing them. But, the system works. It has some interface designs that I still find intriguing and useful, as well as it's share of "WHAT WAS I THINKING?!?!?" elements. Like an old friend, though, the motions and patterns started up again - edit, save, flip, reload, flip, edit more... a dance step long disused, but not forgotten.

I fixed 4 bugs reported by the new users, committed and posted the changes, and updated the users' vhost. Fixed. Problems brought up and addressed. Happy users.

I remember this.

It's nice to be back with you, my old friend.

Friendzzzz, open our K&R to page 32, psalm 12. Today we shall recite from the Book of Why, wherein all manner of faults in life are exposed for cleansing...

Let us begin...

• Why... did my MythTV primary volume kick the bucket just at the point where I'm ready to start working on some code to interract with it? We thank the powers at Maxtor for not taking the half a terabyte of other storage with it during it's death throws. Amen.
• Why... does the Linux kernel decide to number ethernet ports, particularly wireless ethernet ports, in a totally arbitrary way? Booting up may provide us with the mysteries of eth1, or perhaps today it's eth2, or even something like eth1_someoddtext. Amen.
• Why... is the Eclipse WTP project, such an awesomely wonderful and fantastic environment, be occasionally revealing itself as 'not -quite- 100% stable', particularly when I'm in the middle of convincing a client to use it? Amen.
• Why... does the Bluetooth stack on the Treo 650 suck so bad? Simple requests for OBEX services cause the phone to crash and reboot. Connections to it are spotty at best, and it offers NO services up to a remote requestor. Makes it very hard to say "Please get my photos off my phone." It is safe to note that almost every other Bluetooth phone on the market today at least provides a Bluetooth FTP service. The Treo doesn't even have decency to say "no services", it simply drops the connection. Amen, dammit.
• Why... is there no easy way to hit the Tab key in Firefox in a textarea, and have it generate a Tab? Amen.

We shall ponder these life puzzles as we ommm around the coffeemaker and await enlightenment via it's gurgly goodness.

I was getting tired of having problems when building Java apps that required a certain library to be on the classpath or imported into the current app. Another site on the net had a simple lookup mechanism, but that site has gone to a paid subscription model. That just seems silly.

Enter JarIndex

The idea is if you get a compiler error like this:

Exception in thread "main" java.lang.NoClassDefFoundError: javax/wsdl/OperationType
at org.apache.axis.description.OperationDesc.(OperationDesc.java:59)
at org.apache.axis.client.Call.setOperationStyle(Call.java:650)

If you program in Java, check it out, let me know if there's anything missing!

Occasionally, an application does something unexpectedly right.

I've switched to using KMail as my primary mail client in my continued immersion into All That Is KDE. So far it's a capable, well performing, surprisingly robust IMAP mail client.

This morning it surprised me by doing something unexpectedly pleaseant. I was editing / writing a new mail message (new window open, everything) when I realized i hadn't set an identity for sending mail to this particular group. I use identities so a copy of my outbound mail is saved into the same folder I use for that list). So i flipped back to the main window, went to identities, set up a new one, and went back to my new message editor. Without really expecting it to be there, i clicked the down arrow on Identity, and lo, the new one was there.

I had expected needing to save a draft and re-run the editor to have it 'reload' the identities, but KMail just did the right thing and had it all set up for me on the fly.

Sometimes. Occasionally. Programmers do neat stuff.

Having now converted my development environment to Eclipse, I'm going through the normal growing pains associated with going from a total command line "edit, save, exit, compile, look at error, re-edit..." cycle to a totally integrated interactive IDE. The last immersion I did in this type of environment was using Turbo Pascal 7 somewhere back in the mid 90's.

Mostly it's going okay, but I'm sort of entertained at the 'compile on the fly' functionality that Eclipse has. Errors are shown immediately, not at compile-time, so you can see the state of your app at any moment. I just find some of my conversations with Eclipse amusing. The 'Problems' pane at the bottom of the screen shows the current state, and I feel like I see this sort of convo happening all the time:

dbs - typeitytypetyptyp type type. think. typetype

eclipse - Warning: variable foobar is never used.

dbs - yes yes, i know. I'm still working on it.

eclipse - Warning: you type too slow.

dbs - You're not helping.

I'm sure it's only a matter of time before the "Dave. Your code is absolute garbage." HAL-like plugin will be available.

In the midst of hyper-geekery today, Barb msgs me and asks what I'm working on.

So I answer...

publishing WSDL-driven servlets into an Axis environment to expose J2EE stateless session beans' methods via webservices - then i don't have to have the client apps deal with JNDI and RMI calls directly to the jboss server, they can just use SOAP calls, browsing via wsdl.

And just realized how much that looks like total gobbledygook. No wonder I sometimes feel like I'm in my own little world.

So one of my many hats involves helping out running the Arisia website. We changed the site over to being managed by Mambo several months ago, and have been 'mostly' happy with it, though it has its quirks.

One quirk raised its head today. We couldn't upload files via the media manager other than graphics and pdfs. One of the users wanted to punch up a preformatted HTML page, and the system was not allowing it.

After digging around the code, I found the culprit. The system was hardcoded to only accept certain extensions, and I have to admit, the code that did it was... er... painful:

if ((strcasecmp(substr($file['name'],-4),".gif")) && (strcasecmp(substr($file['name'],-4),".jpg")) && (strcasecmp(substr($file['name'],-4),".png")) && (strcasecmp(substr($file['name'],-4),".bmp")) &&(strcasecmp(substr($file['name'],-4),".doc")) && (strcasecmp(substr($file['name'],-4),".xls")) && (strcasecmp(substr($file['name'],-4),".ppt")) && (strcasecmp(substr($file['name'],-4),".swf")) && (strcasecmp(substr($file['name'],-4),".pdf"))) {

Ew. Just. Ew. Bleah. Ick. In the immortal words of Tim Allen... "So I rewired it!" :

# The valid file extension list is:
$validList="fluff|gif|jpg|png|bmp|doc|xls|ppt|swf|html|htm|rtf|";
$fileInfo = pathinfo($file['name']);
if (! strpos($validList,$fileInfo['extension'] . '|')) {


Muuuuch better. This patch goes around line 139 of admin.media.php, if folks are looking for an update.

Well, I'm stuck on something. If you or someone you know is a JBoss guru, they should take a look at this posting and see if you or they can offer any help. I'm stumped. :(

I'm not a computer scientist. I didn't go to school to get letters by my name to learn how languages are built and designed, and I never wrote a compiler in my life. I'm just a fairly high end programmer who writes in high level languages, and builds large applications.

I'm certainly a big user of PHP - less so recently, but since Claimit is written in it, I've been getting my wheels back on it.

But, occasionally I run across design decisions that just don't seem to make sense. Case in point, the 'asort()' function. It sorts an array, in this case backwards (as opposed to sort). One would assume you'd use it like this:

$sortedArray = asort($oldarray);

Not so! This function works, does not throw any errors, but the resulting value in $sortedArray is 'true' (or '1'). The asort function returns a BOOLEAN value (successful completion of a sort??) What actually happens is the asort() function reorders an existing array. So in order to have a new version of the array sorted, or to sort the result of a function call, you use this tortured syntax:

asort($sortedArray = $oldarray);

In my case, I'm actually sorting the returned array from a function call, so we have:

asort($sortedArray = getFiles())

I understand this is a Perl-ism. That doesn't mean it has to be perpetuated further on the masses. Ick.

I'm doing a lot of work in JBoss, and one of the bugs I've run into is that if an applications initial deployment fails, JBoss refuses to un-deploy it. It throws this error when you try to redeploy the app (in this case, the app is called 'app1' and is deployed as an EAR file) :

23:19:23,054 ERROR [MainDeployer] Could not initialise deployment: file:/home/dbs/jboss/jboss-4.0.1sp1/server/default/deploy/app1.ear
org.jboss.deployment.DeploymentException: Error in accessing application metadata: ; - nested throwable: (javax.management.InstanceAlreadyExistsException: jboss.j2ee:service=EARDeployment,url='app1.ear' already registered.)

Sometimes restarting JBoss appserver from scratch will fix it, but only if you've remembered to delete the .ear file before restart.

JBoss comes with a great little tool called 'twiddle'. It sends commands via JMX right into the appserver. You can force an app to unregister right from the command line:

twiddle.sh unregister jboss.j2ee:service=EARDeployment,url='app1.ear'

Voila :)

I've had a couple conversations with people regarding .NET vs J2EE as a choice for application development and deployment. From a purely technical standpoint, the two technologies are very similar. The tools are powerful and detailed in both environments, and the end result is a distributed, object-based application platform.

But to me, taking the 'C# is just like Java' argument doesn't actually address the underlying problem of a Microsoft vs Not-Microsoft solution. It took me a bit, but I think the best way to summarize it is thus:

The purpose of C# and .NET is to make money for Microsoft in tools, platform, and training sales. The purpose of J2EE is to get the job done. No one automatically gets rich when you decide to use J2EE as a platform. Which would you rather invest your time and money in, something where everything from the operating system on up to the deployment tools cost money, primarily to Microsoft, or a platform where the entire system can be designed, developed, distributed, and deployed on free software with the same level of stability and functionality? If support and training is important to you, then pay the people who know best to train you and support you, but you should not be charged just for your choice in platform.

So there.

I do a fair amount of 'web development', meaning I tend to write things that are viewable via a web browser. Whether they're posted on my blog, or on other sites I sometimes maintain, generally my preferred 'user interface' is a web browser.

I've been using a plugin for quite a while called "Web Developer Extension". It's a set of tools that integrates tightly with Firefox and lets you do all the things a web developer needs to do to make sure his or her application is displaying properly.

The most useful feature I've found is 'Outline block elements' and 'Display ID and Class Detail' - these functions change your displayed page and draw lines around all your block level elements (such as 'div' and 'table' and 'span'), and can also label them with what class and ID they are.

When dealing with multiple nested CSS elements, this sort of display can save HOURS trying to track down what element belongs to what container, particularly when working with content management systems like Movable Type.

The plugin is non-intrusive, and is only triggered when you select it off the menu. I can't imagine doing web development without it.

I've just arisen from my death-like existence for the past 2 days, and it appears I've just about shaken off the evil cold that has had me in it's grips since Sunday night. I wasn't even able to read email for more than 5 minutes without getting woozy. Talk about tragic.

So, in a burst of "I'M BACK!" I've done a bunch of LONG needed updates to Planet-Geek and the MT Comments Counter:

• Fixed the Comments preview function so it actually renders properly.
  
• Revamped the color scheme in the individual archive view - so comments and their authors are no longer in that weird green tint. Not sure what I was thinking there.
  
• CSS layout on individual archives and the main page were blocked wrong so it was very easy to have the 'links' sidebar disappear, relegated to the bottom of the page. This should be fixed so that sidebar will only move if you make your browser VERY narrow (comments on this please, I'm only evaluating with Firefox).
  
• We still don't have TypeKey support enabled, but we're still working on it! Anyone who has suggestions on how to get it working properly in MT 3.14 I'd love to hear it.
  
• On the MTC counter, I've changed from a single-image stream function to generating the graphic totally on the fly. This allows multiple-digit displays for very comment-heavy postings, as well as removing the need for a directory full of graphic images.
  

A good few hours of noodling. Check it out, let me know if anything needs tweaking, or if things just Look Terrible [tm].

Warning. This is a rant. This is the result of several weeks clawing up the learning cliff on this environment, and it's really wearing me down. You have been warned.

I'm have a Hard Time [tm] [reg us pat off] implementing stuff in this environment. The complexity level is enormous, and no one really has a good clue on how to do it properly. The level of knowledge needed to implement even a basic application is astronomical, and the number of ways it can possibly go wrong are mind boggling.

The simple thing is, this shouldn't be that hard. I have a JBoss J2EE server. That's an application server environment, you're suposed to be able to deploy your applications into it, and they're in a managed, hosted mechanism. Sounds simple, right?

Well Jboss (as well as other J2EE containers) provide various hooks and services for the applications they host. The problem is, those hooks and services are not well documented, and are so generic trying to fix a specific problem is maddening.

In this case, I'm using the persistent object engine that JBoss supports (Hibernate), and I can't even get a a simple "Create this row in that table" working. Why? Because here's what has to happen to run an application that creates One. Count 'em. One row in a SQL table managed by hibernate.

1. The sessionbean needs to be created
2. The Hibernate mappings definitions need to be written
3. The baseline object for the row needs to be written
4. The Hibernate configuration XML file needs to be put in place.
5. The MySQL JDBC connection has be implemented
6. Jboss needs to be told to provide that JDBC session as a service, and mapped into JNDI
7. The application archive (EAR) file needs to contain all components of the application, including the deployment descriptors, the servlet definitions for client access, and all the EJB pieces, which are plentiful in their own right
8. The database has to have the schema configured
9. The app has to be deployed and you PRAY that that all works when it's dropped into the appserver. Watch the logs for all the errors.

Once that's all deployed, you try it. The servlet is the 'easiest' way to access an EJB, so hit the servlet. The servlet has to call the EJB, which in this case is a Stateless Sessionbean, by far the EASIEST (COUGH) type of J2EE object to work with. The bean in turn is supposed to ask Hibernate for configuation information, session details, and talk to the managed datasource connection in JBoss.

But lo! There's a problem! This is a MANAGED environment. That means it must have transactions. JBoss has it's own transaction manager (JTA), which has to manage all commit/rollback actions. But, since JTA is acting as the manager between Hibernate and the database, certain things can't work. Like, oh, the basic "auto-increment the primary key" id that every database on the planet uses. Hibernate keys data based on a unique key. It comes iwht a simple 'hilo' mechanism that will generate those keys whenever you create a new instance of the persistent object, but THESE DONT WORK IN JBOSS. You have to MANUALLY write your own key generator for every type of object you need to instantiate.

This is INSANE. Now I have to deploy a whole new EJB just to manage the persistent object keys in Hibernate.

"Gosh Dave. That sounds silly. Why doesn't someone write that into the system?"

Why? Because every webpage I find either says "oh, this is easy, just whip up XYZ to manage it." or says "Use XDoclet to create this" (yet another system I'd have to learn just to generate the code to manage the objects i'm creating).

While I understand that huge flexibility in Java is a good thing, this level of abstraction and complexity really is driving me absolutely batshit. I just want to settle down and write my code, but finally, after getting Hibernate to actually CONNECT to the database and show that things are ready, I'm faced with the unhelpful error that looks like this:

12:36:50,522 INFO  [SessionFactoryObjectFactory] Bound factory to JNDI name: java:comp/env/hibernate/SessionFactory
12:36:50,522 WARN  [SessionFactoryObjectFactory] InitialContext did not implement EventContext
12:36:50,532 WARN  [JDBCExceptionReporter] SQL Error: 0, SQLState: null
12:36:50,533 ERROR [JDBCExceptionReporter] You cannot commit during a managed transaction!
12:36:50,534 ERROR [JDBCExceptionReporter] Could not save object
java.sql.SQLException: You cannot commit during a managed transaction!
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.jdbcCommit(BaseWrapperManagedConnection.java:499)
        at org.jboss.resource.adapter.jdbc.WrappedConnection.commit(WrappedConnection.java:451)
        at net.sf.hibernate.id.TableGenerator.generate(TableGenerator.java:126)

It's enough to make you cry.

Update, 1/2 hour later - there's a little-documented function in the Hibernate XML spec. Click through for details.

I'm trying to compile some sample classes to learn how to use Apache Axis the Java webservices system. Part of Java is that it needs to know where to load class libraries from. This is normally set by the CLASSPATH environment variable.

Java stores class libaries in 'jar' files, (java archive). Unfortunately, some apps need dozens of jar files to compile / run, and the CLASSPATH variable can only point to individual JAR files, not say "This directory full of jar files".

So a fellow on #java suggested this:

export CLASSPATH=`echo mydir/*.jar | tr " " ":"`

Cute. Loads up the CLASSPATH variable with all the jars in a directory.

Setting: headphones on, listening to Radio Paradise. The entire office is now empty, boss has left with a 'lock up' tossed back over his shoulder.

Said programmer is locked in a deep conversation with 'hunter', the laptop, which is at the moment doing many things, one of which is attempting to run a small program that asks the local webserver for a simple answer. P is the programmer, H is the anthropomorphized (only slightly) answers from the program.

P "Connect"
H "Error"
P "Connect"
H "Error!"
P "Why is that showing a null?"
H "Error."
P "But you should have gotten an answer."
H "Error."
P "Well, what was in the response?"
H "null"
P "What? But you should have thrown an error."
H ""
P "Owait, you were calling a servlet on a webserver. what was the HTTP response code?"
H "404"
P "Ah... HAH!"
H "Error."

Such is the mental processes after 10 hours of Java hacking. Liberal delivery of nummy food to said programmer has been scheduled.

I'm learning again. I mean, really learning stuff. Working with the folks down in NJ and learning things that I really want to learn. Things that not only make it easier for me to represent my ideas to my temporary employers, but also make me more marketable and give me a broader base.

In the last 2 weeks, I've increased my knowledge on the following topics a hundred fold:

J2EE, JSP, Hibernate, EJB, Object Persistence in Databases, Rich clients, DotNet, Eclipse, JBoss, WebSphere, Ant, Servlets, RMI, JNI, JMS, SOAP, WSDL, and general XML concepts.

Man this is cool

For quite a while I've been wanting to move into some of the more widely used methods for writing and deploying large-scale apps, particularly in Java. Sun developed a system called J2EE a while back that provides an environment where Java apps can scale to incredibly large installations. Up until now, I haven't had the opportunity to really explore it

I recently started a 4 month project with a company in NJ to explore the feasibility of porting their applications from a Visual Foxpro base into J2EE. This is really a fantastic opportunity. I'm not only helping a great project move into an exciting new environment, but I'm also getting the chance to learn something I've been interested in for ages

One drawback though is that the J2EE environment is huge and fairly complex, and therefore there's not 'one way' to do things. J2EE provides an object-based application server that's designed to let you design and implement virtually any system and do it big. The steps I'm taking now are determining what aspects of this system are appropriate for us to use, and how to use them

This process is not helped by the fact that I don't KNOW J2EE at all. I've never used these technologies myself for my own application development, and I've only brushed up against some of their technologies at a previous job. My work on CONGO used a hyper-simplified version of this concept, so there's a heck of a learning curve here.

I am making progress though. Part of this project really requires the environment to be workable from someone who has traditionally been using Microsoft Visual Studio applications. That means a clean IDE, object editor and browser, etc. Tonight I successfully configured Eclipse to use a plugin to manage the JBoss application server I'm running on my laptop. Following some tutorials, I built and deployed a servlet to the server, and, via an Apache module used to connect Java servlet containers to web servers, I successully ran the servlet, and got those wonderful words... "Hello World".

Seems like a lot of work to get 2 words on the screen, eh? But that's the joy of learning a whole new environment. It doesn't look like much, but it represents a big step down the road to understanding how I (and my client in NJ) may use this system to write and deploy applications. Personally, I'm okay as long as I don't get stuck, and continue moving forward.

This coming week I hope to have enough in place to get a full JSP->Servlet->database process working, so that I'm familiar enough with the environment that I can start looking at designing how things REALLY work inside the appserver

I don't talk much about what I do. Basically, I'm an application programmer. I write user-level apps that help folks do a specific task - that task generally being something non-geeky. (as opposed to system programmers who write things for other programs to use, or the like).

Today I was hung up on an annoying SQL problem where a task to update a 'state' table with all the registration, badging, checkedin information, etc - was acting wonky. It would look like it would work, then later it would show the wrong data.

It turns out a second instantiation of the call would -alter- the results set up by the first. So, one run would do the right thing. A second run on a different event would alter the data from the first run. The issue was I wasn't constraining the update enough to limit the scope of the changes, so it was running roughshod over previously existing data.

The corrected query:

153     sql = "UPDATE reg_state,reg_history SET " +
154             "reg_state.state_badged = 1 WHERE " +
155             "state_registered=1 AND state_rid=hist_rid AND hist_actcode='BADGED' AND " +
156             "hist_cid=" + cid + " AND " +
157             "hist_rid=" + rid ;

This problem has been banging around in my head for 2 days now, I'm ecstatic that I finally nailed it down. Now I can go to lunch.

This brief snapshot into Dave's Brain has been brought to you by...

Yesterday was one of the most productive days I've had in ages. I did some huge revisions to CONGO's appserver (the core application that holds all the business logic).

After being sick for a day, and having a tooth extracted earlier in the week, I was feeling edgy about the amount of stuff getting done, but yesterday's 4-6 hour burst of code updates really helped.

For those into the geekier details, read on!

Well you've probably noticed that Planet Geek is going through some changes. The original layout and design I had was aging, and my CSS-fu has improved greatly, so I'm taking a stab at building my own look and feel. The first couple changes are in place, and I'm going to continue tinkering over the next few days. Feedback is always welcome, but comments in particular regarding the wicky-cool tabs for the date headers will be given the most lavish attention. They were rendered in The Gimp and laid into the stylesheet. GEEK GEEK GEEK!

Wow, what a great coding session.

I've been continuously frustrated with the look and feel of the web client that goes with CONGO. The client is called "Coconut" (special thanks to Mister Privacy for coming up with that one), and has been slowly expanding into a system that lets you do all the functions that CONGO can do.

The problem? It was UGLY. I mean really ugly. I couldn't use it for advertising, and even though it ran beautifully in text mode for use at events, it just wasn't nice to look at.

I even got a poke in the side from a friend who said "Nice product, nice web site, but what's it look like? There's nothing for folks to see!"

Well gawrshdurnit she was right. So today and this evening I sat down and learned something that I've been avoiding, namely how to set up nice looking webpages using cascading stylesheets. My blog uses them extensively (course, I didn't -write- my blog), and I've edited other folks' code, so I had an idea what it was like, but it was still a challenge.

All you old HTML geeks. Remember learning all the tricks and tweaks to get a table element to line up right? That is -SO- 90's now. Go CSS. This stuff rocks.

I worked out a design, built my new stylesheet, and converted all of the Coconut pages over to use them.

Check them out here, here, here, and here.

I actually feel like I can take these and put them on the business site, and NOT die of embarrassment at how ugly they are.

Go me :)

Okay, maybe not quite nirvana, but I had a couple hours tonight to continue working on some code I had started yesterday. I've been so bogged down in dull aspects of the business, it was nice to actually -create- some stuff for a while.

Many new functions added to the CONGO appserver and client libraries that'll make things like Coconut (the PHP client class) a heckuva lot more useful.

Yesterday (and I mean Friday with that) - my chosen work environment was the Wendy's near where I picked up my car. Not too bad actually, foundan outlet, put the headphones on (music of choice was Pink Floyd Wish you were Here), and got the basic changes in place.

Tonight I went through my normal library, then settled into Beethoven's 9th symphony. I tell ya, that's one seriously powerful piece o tunage. Hard to NOT focus on the music and get the coding going - but somewhere through the third movement I totally clicked into Flow mode, and cranked out the last couple modules I needed. Committed them back into the repository, and started testing. Twuz great.

Denoument is accomplished with Delerium - Poem, another fantastic piece of music, and great for working - a nice wind down from Ludwig's bit o fun.

Anyway, I'm feeling pretty accomplished. Off to bed now.

With Cat in Chicago for the week, I'm taking care of Zach and doing lots of my own stuff. Part of this is taking him to school and back every day, which takes up some serious hours in the day, making it hard to get a good long set of work done.

So today I took my laptop with me on the morning drive. After I dropped him off I drove to Burlington, where I was going to meet a friend for lunch, and parked in the Barnes and Noble next door. I took along my new set of Koss Portapro headphones, (which was a replacement for a set I hard worn out) and sat down in the B&N coffeeshop.

After finding a power outlet and settling in, I jacked in the headphones, set up the laptop, and worked about 2 hours on some design documentation and other bits. I designed an entire new module for CONGO, and fixed a lingering logging bug in the code. Very nice. This sort of work groove is what I enjoy the most. The best situation is when I'm writing code - I can get into a sort of fugue state, being totally focussed on what I'm doing, ignoring everything around me. In this case it was design work and documentation, but it still gave me that sense of productivity and accomplishment.

By the way, there's a book written about this state, called "Flow - The Psychology of Ultimate Experience". I read it once a long time ago.

But here's the question for the masses. I was listening to Afrocelts' third album , 'Further in Time' at quite frightening volumes. It's a remarkable album, and I've found most of their music is -fantastic- for coding or concentrating on work.

So what do YOU like to listen to when working or thinking hard?