Subversion + SSH – Close but no banana

About a year ago, I switched my primary source code control system from the venerable old CVS to the (relatively) new kid on the block, Subversion. On the whole, I’ve been ecstatically happy with the system. It patched many of the ridiculous problems with CVS, and added on things that opensource community has been asking for for ages (like ‘rename’), but never made it into CVS.
Now I have all my projects stored in SVN, and my main client is using it as well for their code (they’ve chosen to go with SVN and are planning to End Of Life their VSS server – to the dismay of no one).
Subclipse
One of the best tools that made this switchover workable (aside from SVN’s similarity CVS in many respects, particularly on the command line) is the Subclipse plugin for Eclipse. Subclipse provides a great easy to use interface into SVN servers, giving all the functionaly one would have on the command line via a very simple, tightly integrated GUI.
One thing that had been bugging me, however, was the access methodology I was using to get to my (remote) SVN server. It involved setting up a tunnel in SecureCRT (though Putty can do it as well), and then telling subclipse to use my ‘svn://localhost/stonekeep’ repository.
SVN+SSH configuration under EclipseWhile doing some surfing, I found that Subclipse supports the svn+ssh syntax for specifying the repository. “Great!” says I, “I won’t need to set up the tunnel each time!”
A few more fiddles, a pleasant discovery of a configuration screen in Subclipse, and I had an SVN over SSH connection to my repository, even using my ssh key pair.
Danger, Will Robinson!
But wait! All is not well. When I tried to browse the repository from Subclipse, I quickly hit this error:

Could not open file system at /var/lib/svn/stonekeep
(13)Permission Denied: Berkley DB Error while opening environment for file
system /var/lib/svn/stonekeep/db:

This vexed me, because I had been having no problems accessing the repository locally on the server, and over my ssh tunnel. Both used the locally running ‘svnserve’ on the repository host, so why wasn’t the svn+ssh connection using it?
The answer comes in the SVN documentation, and via a little research:

What’s happening here is that the Subversion client is invoking a local ssh process, connecting to host.example.com, authenticating as the user harry, then spawning a private svnserve process on the remote machine, running as the user harry. The svnserve command is being invoked in tunnel mode (-t) and all network protocol is being “tunneledâ€? over the encrypted connection by ssh, the tunnel-agent. svnserve is aware that it’s running as the user harry, and if the client performs a commit, the authenticated username will be attributed as the author of the new revision.
When running over a tunnel, authorization is primarily controlled by operating system permissions to the repository’s database files; it’s very much the same as if Harry were accessing the repository directly via a file:/// URL.

The Problem With This
I’m really unhappy with this model. The problem is that now the user must have read/write access to the entire repository tree. When using a local socket connection (or one over ssh via a normal tunnel), the Subclipse client connects directly to the svnserve process running on the repository box, and interactions with the server happen under that processes ownership.
The svn+ssh protocol does not use the svnserver on the target machine. It tunnels the command to a user-invoked svnserve process, and that process must have read-write access to the repository.
“Well gosh, that doesn’t seem too bad. What’s the issue?”
The issue is that to make this methodology work, I have to give the user read/write access to the repository tree. Meaning, they could happily type ‘rm -rf /var/lib/svn’ and destroy the entire repository. Even worse, the configuration files (including the password / access file, which has passwords in plaintext) must be made available to the general users.
Why svn+ssh doesn’t simply make a local socket connection to the svnserve process already running, I don’t know. But I can find no way to make that happen.
The fix?
As far as I can tell, there really is no direct fix for this. There are various workarounds, which the SVN documentation discusses, including setting up an ‘svn user’ for the svn+ssh logins, and the possibility of using unix groups for permissions, but I feel that if you have a listening socket server on your repository host, you should use it, not introduce a second methodology and have to jump through hoops to implement it.
For now, I have to abandon the svn+ssh possibility, and go back to my hand-configured socket tunnels. There’s no real loss here – they work remarkably well, are very secure, and quite stable. The slight annoyance of having to open up a SecureCRT session before doing work in Eclipse is just that – a slight annoyance. I’ve dealt up until now, and I’ll just continue to deal.

Strictly Hamburger Coding

I ended up with a sort of special bonus afternoon today as Cat is off in town, and Zach is over his friends house. The basic idea was to make up for a very interrupted week of work and try to get some long-neglected work out of the way.
Mostly, it worked quite well. I got caught up on a couple things that had been nagging, and that’s good. But what really got me grooving was working on… well, a project that a couple folks know about, but I’m not -quite- ready to release to the public. I’ve been referring to it simply as the “Secret Squirrel” app, or just ‘The Squirrel’ for short.
What’s been good about this has been the basicness of the design and implementation. I’ve been spending SO much time delving into the deep complexities of EJB3, JBoss, Hibernate, and more recently Terracotta, I don’t get much of a chance to just sit down and whack out simple code.
Today had 2-3 hours of basic grunt ‘hamburger’ coding, to borrow a phrase from MASH. No fancy libraries, environments, or oddities. Well, okay, one new library that handles OpenGL rendering, but everything else was basic data application coding and design.
It felt good! It sort of underlined where I’ve gotten with Java programming over the last 2 years or so, and now basic coding is second nature. Eclipse has become my environment of comfort, whether it’s running on Windows or Linux (in fact I’m leaning toward moving back to Linux full time soon – with both machines up and running cleanly on my desk, I find myself doing more on the Linux box than on Windows… but I digress…) , and the Java OOP approach to program development Just Plain Makes Sense. It’s nice.
Tomorrow it’ll be back to EJB3 and Terracotta land, and all the mysteries therein, but for now, I can look at what I did today and do a “That felt good!” happy dance.
And the code even worked.

Can lightning strike twice?

About 10 years ago, I started writing an application that would have a profound impact on my life. Keystone started out a simple problem tracker, grew into a mature product that was getting 3000+ downloads everytime I did an update (about once every 3 months), and was ultimately sold to a DotCom that basically killed it in its tracks. That sale let me have a few toys and was a high point of the dotcom bubble for me.
3 years ago I re-aquired the rights to the application from the failed dotcom, and set about upgrading the vastly outdated software. My user base had for the most part wandered away to other applications, but there was still interest and heck, it was my application, I wanted to do things with it.
But other projects were taking precedence, and Keystone languished.
On a recent trip down to DC, I had the opportunity to spend 7 uninterrupted hours on the train, each direction, with nothing but a laptop and a music library to keep me company. After trying to get my current projects working, I settled back into “well, maybe I’ll work on Keystone some more.”
In those 2 train rides, I did more upgrading, tinkering, and fixing in Keystone than I’ve done in the last 3 years. I revived the contact manager and fixed all the dependency problems. I continued the changes needed to bring a PHP application, written in 1995, up to 2006 standards. Keystone is over 12,000 lines of code – not a trivial application, but not so huge that it is an unassailable target.
The question is – why do this? Sure, part of it is ‘this is my baby, I want to see it succeed’, but in the back of my head, the question burbles… “Can lightning strike twice?” – can I make this a successful opensource application again?
I’m certainly not deluding myself into thinking “THIS WILL BE THE NEXT KILLER APP!” – that’s a foolish and unrealistic mindset. But can I bring it back to where people are using it, they like it, they contribute suggestions and fixes, and the application continues to grow?
I’d like to think I can. But the code still needs a lot of work, and there are some design decisions that will most likely require huge chunks of code being ripped out at the roots (database connection methodology has advanced SIGNIFICANTLY since 1995).
It’s a nice dream, I sort of miss my users. Maybe they’ll come back.

It’s all about the tools.

In a previous post I admitted to the world that I, an avowed Linux weenie, was now using a Windows desktop for all my geeky endeavours. This continues to be true, but I’ve taken the steps necessary to make my environment comfortable to work in, without going the easy route of “I will do everything in my power to make Windows look and feel just like my Linux box.” To me this defeats the purpose of potential learning experience of working with a new environment.
So about those tools…

Continue reading “It’s all about the tools.”

The code, she is a flowin.

Rain. Cold. Cloudy.
These to me are the harbingers of only one thing.
An excellent day of coding.
And an excellent day it was. Almost 12 hours, with a meeting and a good lunch to interrupt, had me rewriting an entire interface to the application, reworking a build script so it was more efficient, and implementing a ‘Priority 1’ change for the current programming sprint.
I’m coming to a happy medium with Eclipse and my forced migration. All in all, a very good couple days, with this one as a topper.
Tomorrow I have another bit of work, then drive back up to Boston. But tonight, it’s time for food, then rest.

My Daily Conversations with Eclipse, Pt II

My ongoing conversations with Eclipse (see this older post for Pt 1)

dbs “Edit that.”
eclipse “no.”
dbs “but it’s open in the editor.”
eclipse “nuh uh.”
dbs “*enter*enter*enter* CMON!”
eclipse “nope”
dbs “owait, i’m in the svn browser aren’t i?”
eclipse “_I_ knew that. Don’t know what your problem is..”
dbs “fine, java perspective, now edit.”
eclipse “if you insist.”

An old programmer…

… can learn new tricks?
As part of my current contract, my employers are embracing new procedures and techniques for application development. While the normal buzzwords of “EXTREME PROGRAMMING” and “Agile Development” are being bandied about, they’re not being whole-hog embraced, with the managers marching zombie-like into an undeliverable product schedule. We’re trying new techniques such as scrum product development, and utlitizing sprints to structure short-term deliverables and milestones.
Since I got the core of my production programming experience with this same company almost 18 years ago, when there were only 3 employees, me being number 3, I find it fascinating to see the same programmers trying new tacks to take on the complexity of developing modern code.
In the Good Old days, our entire code base was perhaps 80,000 lines. One developer easily understood every aspect of the application, and could comfortably keep up with customer and internal needs. The current codebase is over a million lines and growing, and no one programmer understands it all. This is not a disaster story, it is one companies’ normal evolution as they grow and expand. In this case, they have lucked out to have a person in charge who is not only a brilliant programmer, but also an open-minded manager and a good communicator. He recognized the pitfalls of 20 years of development on his product line, and is making careful, planned, and deliberate changes to the company’s development methodology.
Of course, the panic element for this is… the changes being applied involve shifting their platform onto a framework I designed for them over the last 18 months. No pressure, really. It’s only the future of this company and it’s 20-some odd employees, not to mention the respect and 20 years good will of the owner. The same owner who gave a young programmer his first production development position.

QuickReview: Synergy2

Have to point this handy tool out to folks.
My busy deskBecause of the complexity of my desk, I can’t really have a pile of keyboards and mouse lying around. I switch between machines constantly, and switching keyboards would just drive me batty. I had been using X2VNC for quite a while, which, while ‘okay’, had it’s own quirks. One of which was it couldn’t work in reverse (I could go from an X host to a Windows box, but not the other way around). It also wouldn’t let me slide from an X display to another X display. Pretty limiting.
Enter Synergy2, a simple client-server tool set that lets you configure multiple displays to a single ‘server’ that controls the mouse and keyboard. With Synergy2, I’m able to configure all my machines in whatever configuration I want. At the moment I can slide my mouse off my primary Linux display onto my WindowsXP box, across it’s display, and onto my laptop.
That’s pretty neat, but.. the kicker? Synergy2 manages clipboard cut n paste operations across machines. If I slide over to hunter (the laptop running Ubuntu linux), highlight something and click ‘Copy’, then slide back to my primary desktop on yawl, I can just click ‘paste’ and it works. This is miraculous to my eyes!
If you run more than one machine on your desk, I highly recommend Synergy2. Available in apt repositories everywhere.

JRemoteforMyth Released

Folks, family, and friends may have noticed I’ve been fairly absent from posting, chatting, and general socializing for the last few days. A chunk of this mysterious quietness can be attributed to digging my fingers into a coding project I’ve been thinking about for a while.
I’ve just released JRemoteforMyth. This is a webstart-enabled app for folks who tend to hang around on their laptops or desktop machines, with a MythTV box nearby. It allows for a small ‘remote’ application to sit on their desktop and control the Myth box remotely via normal “up” “down” “enter” buttons.
There were a number of challenges writing this. I haven’t written Swing applications in a few months, and I’ll be needing to get back in the saddle shortly for some consulting work I’m doing. This was a great ‘simple’ defined application I could whip up that had a fairly definite design, goal, and completion metric.
The second challenge was this was my first full webstart-enabled app written entirely inside Eclipse. For the most part, this didn’t present any particular problems, and went quite smoothly. I’m still ecstatic about Eclipse’ SVN and CVS repository integration. Using CVS to hold my working files, I was able to switch from my laptop to my desktop machine with only a Project->Team->Commit on one machine, and a Project->Team->Update on the other. Yay!
Last but not least, a grump. When writing a Webstart enabled app, the resulting ‘jar’ file that containst he app is published onto a webserver, and the JNLP descriptor file tells webstart well to download it. Webstart, however,will only install and run ‘signed’ jar files. Eclipse provides no mechanism for signing jar files internally. It can generate the jar file for the application, but several command line tools need to be run to actually sign the file. Very annoying.
I’ve announced the app to the mythtv-users list, and I’m seeing downloads, so SOMEONE is interested in it.
Next will be some basic enhancements to it, adding things like keyboard entry, an multiple tabs so the ‘keypad’ can be displayed along with normal navigation keys.

Coding horrors from the past!

It’s nice to be back into coding for a while. Of course, one of the drawbacks is coming across code written over half a decade ago that should never have seen the light of day. Take for example this gem I ran across while continuing with my updating Keystone project…

$pq = db_query("select dtable,dcolumn,dflags from dictionary $wclause");
while ($pd = db_fetch_array($pq)) {
$varname = "op_$pd[dcolumn]";
eval("$oval="$$varname";");
$varname = "sp_$pd[dcolumn]";
eval("$sval="$$varname";");
if (empty($oval)) {
$oval=0;
}
$active = $sval ? "1" : "0";
$subtable = ($pd[dtable] == $proptable) ? '' : $pd[dtable] ;

It’s been said there’s a special hell for people who write code that uses the eval statement in production code. Apparently I’m headed there already. This was a very bad coding decision, but I remember actually writing this particular snippet. It was around 1998, and I was flying to California to talk with the company that would eventually buy Keystone from me. I seem to do some of my best coding work while flying on airplanes, though this sample isn’t exactly a sterling example of it. It did, however, enable one of the cooler features of the product – the ability to, using any of the various data sets, set up a custom browse view based on the structure of the table.

Using ‘eval()’ statements and depending on global variables was NOT the way to implement it though. Now that I’m converting all 12,000 some odd lines of Keystone code over to support running on a system that doesn’t have register_globals enabled, it was time to update this particular code snippet. It took a good 1/2 hour to figure out exactly what it was doing, but once I did that, it was a simple change to:

$pq = db_query("select dtable,dcolumn,dflags from dictionary $wclause");
while ($pd = db_fetch_array($pq)) {
$oval = $_POST["op_" . $pd[dcolumn]];
$sval = $_POST["sp_" . $pd[dcolumn]];
if (empty($oval)) {
$oval=0;
}
$active = $sval ? "1" : "0";
$subtable = ($pd[dtable] == $proptable) ? '' : $pd[dtable] ;
}

Don’t see much of a difference? It’s a big one from a code security and design standpoint. Don’t sweat it too much, it means a lot to me at least.

The conversion is moving along nicely though. I think I can have eval versions ready for folks to test out within a day or three, if I keep this pace up. I’ll be curious to see what sort of response I’ll get on the net to the system. It’s been a while.

Dusting off the Old

Back in the dark ages, I wrote a very successful ticketing system called Keystone. This was a successor to another reasonably successful program called PTS, which I wrote while working at Fidelity using this new fangled thing called ‘PHP/FI’. Online ticketing systems were still something of a novelty, and none existed in the opensource world (save for GNATS which, at the time, was an abysmal piece of code.
PTS, and later Keystone – flourished. In the dotcom runup, Keystone was a hot item. Whenever I posted a new version and announced it on Freshmeat, I would see thousands of downloads of the updated package within days. At conventions, I was well known. “YOU WROTE KEYSTONE? HOT DAMN!”
Alas, the dotcom era came along and an opportunity to sell Keystone dropped into my lap. I took it, along with a sizeable check, in exchange for an agreement with the new owners, that they would continue to support the product, as well as let me continue working on it.
Neither of these things actually happened. When the bubble burst, the owners found themselves holding a piece of software that they were not using, and still owed money on. The crunch came, I called in the debt, they couldn’t pay, so the entire system reverted back to me, lock stock and copyright, in exchange for me foregoing the outstanding balance.
On the one hand, yay, I got my program back. Hard to argue with that. On the other hand, 2 1/2 years had passed. The new owners had not touched the code, the industry had advanced a LOT in that time, and the competitors, which had excellent products at the time of the buyout, now had a 2 1/2 year jump on me.
Keystone languished.
Over the last few months, several people have been poking me about the code. Some clients have been running versions for YEARS, and are asking if there are updates coming. Others would like to set up new installations. I’ve half-heartedly worked on updating Keystone off and on a few times, but never really finished the changes into something I could release again.
Tonight I spent 2-3 hours dusting off the old code and continuing the updating process. Files with revisions half a decade old are coming up in my editor, with me vaguely remembering even writing them. But, the system works. It has some interface designs that I still find intriguing and useful, as well as it’s share of “WHAT WAS I THINKING?!?!?” elements. Like an old friend, though, the motions and patterns started up again – edit, save, flip, reload, flip, edit more… a dance step long disused, but not forgotten.
I fixed 4 bugs reported by the new users, committed and posted the changes, and updated the users’ vhost. Fixed. Problems brought up and addressed. Happy users.
I remember this.
It’s nice to be back with you, my old friend.

Today in the Book of Why

Friendzzzz, open our K&R to page 32, psalm 12. Today we shall recite from the Book of Why, wherein all manner of faults in life are exposed for cleansing…

Let us begin…

  • Why… did my MythTV primary volume kick the bucket just at the point where I’m ready to start working on some code to interract with it? We thank the powers at Maxtor for not taking the half a terabyte of other storage with it during it’s death throws. Amen.
  • Why… does the Linux kernel decide to number ethernet ports, particularly wireless ethernet ports, in a totally arbitrary way? Booting up may provide us with the mysteries of eth1, or perhaps today it’s eth2, or even something like eth1_someoddtext. Amen.
  • Why… is the Eclipse WTP project, such an awesomely wonderful and fantastic environment, be occasionally revealing itself as ‘not -quite- 100% stable’, particularly when I’m in the middle of convincing a client to use it? Amen.
  • Why… does the Bluetooth stack on the Treo 650 suck so bad? Simple requests for OBEX services cause the phone to crash and reboot. Connections to it are spotty at best, and it offers NO services up to a remote requestor. Makes it very hard to say “Please get my photos off my phone.” It is safe to note that almost every other Bluetooth phone on the market today at least provides a Bluetooth FTP service. The Treo doesn’t even have decency to say “no services”, it simply drops the connection. Amen, dammit.
  • Why… is there no easy way to hit the Tab key in Firefox in a textarea, and have it generate a Tab? Amen.

We shall ponder these life puzzles as we ommm around the coffeemaker and await enlightenment via it’s gurgly goodness.

New Java Tool – JarIndex

I was getting tired of having problems when building Java apps that required a certain library to be on the classpath or imported into the current app. Another site on the net had a simple lookup mechanism, but that site has gone to a paid subscription model. That just seems silly.
Enter JarIndex
The idea is if you get a compiler error like this:

Exception in thread “main” java.lang.NoClassDefFoundError: javax/wsdl/OperationType
at org.apache.axis.description.OperationDesc.(OperationDesc.java:59)
at org.apache.axis.client.Call.setOperationStyle(Call.java:650)

You can just go to JarIndex, enter ‘javax/wsdl/OperationType’, and JarIndex will tell you what library that class comes from. Add it to your classpath, and you’re back into happy compilation mode!
If you program in Java, check it out, let me know if there’s anything missing!

Good Programmer. Have a biscuit!

Occasionally, an application does something unexpectedly right.
I’ve switched to using KMail as my primary mail client in my continued immersion into All That Is KDE. So far it’s a capable, well performing, surprisingly robust IMAP mail client.
This morning it surprised me by doing something unexpectedly pleaseant. I was editing / writing a new mail message (new window open, everything) when I realized i hadn’t set an identity for sending mail to this particular group. I use identities so a copy of my outbound mail is saved into the same folder I use for that list). So i flipped back to the main window, went to identities, set up a new one, and went back to my new message editor. Without really expecting it to be there, i clicked the down arrow on Identity, and lo, the new one was there.
I had expected needing to save a draft and re-run the editor to have it ‘reload’ the identities, but KMail just did the right thing and had it all set up for me on the fly.
Sometimes. Occasionally. Programmers do neat stuff.