Category: Better Software Design

Do’s and Don’ts for designing software for reliability, maintainability and performance

CMake – a short history

In the Beginning, if you wanted to create an executable application in C, you’d run a program that translated C code to assembly language, run an assembler on the translated code, then run a linker to bundle it all up into an executable file. And it was Good.

But it was also annoying after a while, so someone (probably Dennis Ritchie or one of his friends) wrote a wrapper compiler – the “cc” program. Now with a single command, you could compile, assemble and link. Incidentally, linking for most other systems (which may or may not have compiled to assembler) was generally a separate program. And it was Good.

But as projects got larger, with a dozen or more source modules, something more was needed. Generally, when you’re working on a large project, you’re only changing a small number of files at a time, and back when a Raspberry Pi would have been welcomed as a virtual super-computer, you didn’t want to continually re-compile stuff that hadn’t changed. So Stuart Feldman created the “make” utility. Make could not only intelligently choose what to compile, it provided a central control point with user-designable goals like “make clean”, “make install” and “make all”. And it was Good.

But Unixes came in many flavors. And eventually, that Included Linux. With multiple distros of its own. And building for each of these disparate platforms often required slightly different rules. And more and more often, there were external dependencies. As a result, David Mackenzie of the Free Software Project developed automake, which extended to become the Gnu Build Systems, or as it’s often known, Autotools. With this framework, you could build Unix/Linux apps portable using 3 magic commands: “./configure; make; (sudo) make install”. And it was Good.

But that still didn’t cover everything. Although the Microsoft Windows™ Operating system (among others) is very different platform, many open-source app authors wanted to be able to use a common source system to be able to build both Windows™ and Unix/Linux versions of their apps. So a team of people developed CMake, which takes over the capabilities of Autotools and adds OS portability to the mix. With CMake, you can not only compile on many platforms, you can cross-compile between platforms. Plus, you can build to external target directories, which allows both support for multiple targets at once and an easy and safe way to blank out a build completely and start over. With CMake, you can do a build as simply as ‘cmake -S source_dir -B build-dir; cd build_dir; make; (sudo) make install)”. And if you script it right, even the “make install” can be automated. And it is Very Good.

It doesn’t stop there. A shop that does nightly builds might use something like Jenkins to further manage and track builds, but CMake is the place where you go if you want to build manually. If, for example, you’re the developer doing the code editing.

Gnome Evolution is an Abomination and gnome-keyring should die in a fire!

Really.

Between Evolution’s penchant for creating non-deletable – and defective – account associations and gnome-keyring’s useless pop-up dialogs, the whole thing almost makes Microsoft Windows seem attractive.

Then again, gnome is, by and large, a slavish attempt to imitate many of Windows’ more obnoxious features. Like the Windows Registry.

Honestly. People have been complaining about this stuff for years and it never gets fixed.

The popup for gnome-keyring is especially odious, since it blocks all other user interaction (including access to pwsafe) and it LIES. It says that the Google password incorrect when it isn’t.

There are no documented fixes to speak of, short of wiping the entire OS, no one on the respective gnome development teams does anything and users get angry.

Including me. So I’m going to go take a stress pill.

JPA and overlapping complex keys

If at all possible, you should avoid setting up databases with complex keys (or compound keys, as they are sometimes known). It makes life a whole lot more complex, and in a world where everyone already says you’re taking too long because “All You Have To Do Is…”, complex isn’t a good thing.

Sometimes, however, there’s just no choice.

I have one app that has a table with 2 complex-key relationships. The table’s primary key is (idA, idB) and there’s a foreign key reference to another table item with a key of (idA, idC). The table’s key is an embedded object of type “table1Id” and the foreign reference is a lazy-load “table2” object.

The full details are here.

org.hibernate.PropertyAccessException: could not get a field value by reflection getter of com.mypackage.MyEntity.entityId

There are reports that messages of this sort were due to bugs in one or more Hibernate releases. But this is also a legitimate error.

I wasted a lot of time before it hit me. The original code was:


@SuppressWarnings("unchecked")
public List<State> getStatesForCountry(String countryID) {
    final String SQLCOMMAND =
        "SELECT c " + "FROM "
            + State.class.getSimpleName() + " c "
            + " WHERE c.parentCountry = :countryID"
            + " ORDER BY c.StateName ASC";

    Query query = entityManager.createQuery(SQLCOMMAND);
    query.setParameter("countryID", countryID);
    List<State> results = query.getResultList();
    return results;
}

It should have been:


@SuppressWarnings("unchecked")
public List<State> getStatesForCountry(String countryID) {
final String SQLCOMMAND =
    "SELECT c " + "FROM "
         + State.class.getSimpleName() + " c "
        + " WHERE c.parentCountry = :country"
        + " ORDER BY c.StateName ASC";

    Query query = entityManager.createQuery(SQLCOMMAND);
    Country country = findCountry(countryID);
    query.setParameter("country", country);
    List<State> results = query.getResultList();
    return results;
}

Where findCountry is simply an entityManager.find(Country.class, countryID);

This is a sneaky one because when the database is laid out, you think in terms of foreign keys. But in ORM, you don’t see those keys directly – they translate to object references. So the original version was attempting to compare an object to the key of the object instead of an instance of the object.

I can only plead distraction. The object model in question had been designed by someone else. Instead of accessor methods, it was using direct field access (which I avoid for a number of reasons). Further aggravating the issue was that the fields were all given names starting with an upper-case letter as though they were independent classes instead of properties.

But even without distractions, it’s not too hard to make this mistake.

Writing files into a WAR – Another BAD IDEA

I’ve always recommended against this. For one thing, a WAR is a ZIP file and Java has no builtin support for updating ZIP files.

A lot of people abuse the fact that many JEE servers unpack (explode) WARs into a directory such as Tomcat’s webapps directory. They then proceed to use the ServletContext getRealPath() method to translate a subdirectory in the WAR into an absolute filename path.

There are 4 problems with that idea.

  1. If the server doesn’t explode the WAR, there won’t be a real path. So the pathname returned will be null and the code will probably throw an exception. This can be a problem when transporting the application to a different vendor’s server or when the configuration of the current server is changed.
  2. It’s generally good practice to keep executable code, forms, and other potentially-hackable constructs in a write-protected location. Plop down a writable directory in the middle of the WAR and you’ve opened up a potential exploit.
  3. If you write “permanent” files to the WAR directory, a redeployment may nuke the entire WAR substructure, losing the files forever. I’ve always preferred to explicitly erase a WAR before updating it anyway, since otherwise old stale stuff hangs around and pollutes the application. Sometimes with unfortunate results.
  4. If you hardcode the write directory relative to the WAR, what do you do when the disk fills up? Unix and Linux provide a special directory tree (/var) to hold things that may grow, but there’s no fixed relationship between the WAR directory and the /var directory. Coding an absolute path can work, but it’s not very flexible, either.

What to do? I normally get my writable directory location via JNDI lookup. For example: “java:comp/env/wardata”. The advantage of this is that I can relocate the directory any time I want. I put the default location in the web.xml resource definitions, but in Tomcat I can override this. Which is convenient when testing.

Why Do-it-Yourself Java Security is a BAD THING

One of my greatest peeves in working with Java in the Enterprise is the fact that everyone+dog seems to think they can do a better job on application security than the Java architects.

OK, so there’s some really awful stuff that’s part of the Java standards, but nevertheless, rolling your own security is still a BAD THING.

Here’s why:

  1. I’m so clever. You’re not as clever as you think you are. Even I’m not as clever as I think I am, and I, of course, am much cleverer than anyone else. Most DIY security systems I’ve encountered (or, alas, developed) have proven to contain at least one easily-findable hole that you could channel the Mississippi River through.

  2. Infrastructure lock-in. Your DIY system is almost certainly tied to your current infrastructure. If the infrastructure changes, you’ll probably have to recode (see below). And, of course, if you ever had any dreams of selling your work to other shops, they probably aren’t set up the same way.

  3. Documentation and procedures. You can’t go down to the local bookstore and buy a book on how to properly use a custom security system the way you can with the standard security system. In fact, any documentation you have is likely to be insufficient and out of date.

  4. Professional Design. The standard security frameworks were designed by security professionals. Yes, I know they’re not as clever as you. But they were trained specifically to work on security first and foremost and argue with other people looking for bulletproof general-purpose solutions, then the systems were exposed to legions of evil people for stress testing. If your primary job was security, if you have extensive training in mathematical cryptanalysis, your cleverness would outweigh the fact that these people are but pale shadows of your genius. But your primary job was to develop an application and most likely, the orders weren’t to make it all perfect, just to “Git-‘R-Dun!”.

  5. Declarative security. The standard Java security systems are mostly declarative. Studies have shown that declarative is less likely to contain unexpected bugs. You can write anything in code, but when all you have is fill-in-the-blank declarative options, your opportunities to make mistakes are far fewer.

  6. Minimal coding. The standard Java security systems are generally minimally invasive. You don’t have to turn the application code upside down every time the security infrastructure changes. You can test most application code with security switched off or using a local security alternative like the tomcat-users.xml file without having to establish some sort of heavyweight alternative to the production security system (or petition the security administrator for test security accounts and privileges).

  7. Maintenance costs. When you tie security code intimately into application code, anyone coming along later to do maintenance will probably be ignorant of the nuances (remember the local bookstore? and will end up punching a hole in the security. Security is like multi-threading and interrupt handling. It only takes ONE bug to bring everything down.

  8. Development costs. When you tie security code intimately into application code, you have to do twice the work, since you have to code both the business logic and the security logic. And, again, forget to do it in just one place and the whole thing turns into tissue paper. Additionally, in-application security code is not just another spanner in the works of setting up testing and debugging frameworks, you have to debug both the security code and the application code.

  9. Framework support. Standard frameworks like Struts and JSF have built-in support for the J2EE standard security system. They don’t have built-in support for one-off security. You’re paying for it regardless of whether you use it or not. You might as well reap the benefits.

  10. Mutability. If you want to rework a webapp into a portlet or web service, the standard J2EE security system will mostly port transparently, since it’s minimally invasive. Portlets virtually demand a Single Signon solution – forcing someone to enter login credentials into each and every portlet pane will not win you friends.

Ideology.

Sounds a lot like “idiot”. And with good reason. There is no such thing as a one-size-fits-all Silver Bullet solution. Sometimes the standard security framework isn’t a good fit. About 9 times out of 10, it is, however. For the 10th case, I generally prefer to augment the standard security framework. For example, when I need fine-grained security, I use role-based access control to fence off the major sections of the app, then use the identity provided by the standard framework to retrieve the fine-grained options. Where that doesn’t work or is insufficient, I try to use minimally-invasive approaches, such as Filters and AOP crosscuts. I’ll do anything it takes, in fact, but the more I can work within the supported system, the happier I am.

Why Software Projects Should Not Depend on an IDE

disclaimer: I’m about to show my age.

My first “IDE” was an IBM 029 keypunch machine. I keyed programs in FORTRAN, COBOL, Assembler and PL/1. When I messed up, I threw away the defective cards and punched new ones. After waiting in line for a free keypunch machine.

Over the years, things got better. I graduated to online terminals, then added debuggers, code assists and refactoring.

But I also learned something. IDEs are more fluid than programming languages. As projects got more complex, they took on more and more of the responsibilities of maintaining not only program code, but the entire build process. Including tracking the locations of the build components and running the associated build utilities such as resource compilers.

Eventually it reached a point where the IDE was less an aid than an addiction. A certain IDE which Shall Remain Nameless denegrated the ability to build from the command line using constructs like batch scripts and makefiles into virtual uselessness. It was too much trouble to learn how to create a build from the command prompt.

Then, the New, Improved version of the IDE came along. But guess what? The old projects had to be modified to build under the new IDE. Then came the late-night call for an emergency code fix. The project in question hadn’t been touched in 2 years. It wouldn’t compile without the old IDE, even though the actual fix was a 1-line code change. Installing the old IDE required installing old support programs. Taken to extremes, it would have ended up with taking an old junk computer out of the closet (at 2 a.m.), installing an old OS, installing the old IDE and supporting cast, all to resolve a minor problem.

But wait – there’s more!

Years passed, I moved onto other platforms. But now I worked in a shop where there were 2 different groups of developers. The other group had developed an entire ecology of their own – all tied to their IDEs and desktop configurations. We were supposed to be sharing standards. But their ecology had been designed specifically to to their needs, not ours. They could hand us code, but it wouldn’t build because we hadn’t invested our desktops in a lot of configuration that was scattered around inside of people’s IDEs.

Contrast this with a batch-based process such as Maven or Ant.

Maven, of course, tends to force a consistent organization, for better or worse. So while the choice of goals may be unclear, at least the build is consistent.

Ant is less standardized, but it’s no big deal to make an Ant script self-descriptive.

And in both cases, you can place project/platform-specific configuration in files that can be passed to build processes and stored in the project source code archives. Instead of being embedded on people’s desktops.

When an IDE Just Won’t Serve

There are cases where an IDE simply cannot do the trick. It’s not uncommon these days – especially in Agile development shops – to publich a Nightly Build. The Nightly Build is typically a collation of the daily commits done after hours in batch. “In Batch” and “IDE” don’t go together. An IDE is an interactive environment. Furthermore, the batch build machine may be a server, not a desktop machine. Not all servers have GUIs installed – or even Windowing systems. Ant and Maven won’t have a problem with that, but IDEs will.