Category: Uncategorized

Puppetserver in a can —Docker versus Podman

Things have been too exciting here lately. Attempting to set up Ceph, having a physical machine’s OS get broken, having a virtual machine get totally trashed and rebuilt from the ground up.

It was this last that gave me an unexpected challenge. When I built the new container VM, I moved it from CentOS 7 to AlamLinux 8. Should be no problem, right?

I sicced Ansible on it to provision it — creating support files, network share connections… and containers.

One thing that’s different between CentOS 7 and CentOS (IBM Linux) 8 systems is that the actual container manager changed from Docker to Podman.

For the most part, that change is transparent to the point that if you give a Docker command, it is passed more or less verbatim to Podman.

But one container proved difficult: the Puppetserver. It steadfastly refused to come up. It claimed that it couldn’t find the cert it was supposed to have constructed. Or that the cert didn’t match itself, if you prefer.

Tried it with Podman on my desktop according to instructions. No problem. Changed my Ansible provisioning from “docker_container” to “podman_container”. No luck. Did an extra evil trick that allowed be to freeze startup so I could dial into the container. Cert directories were incomplete or empty!

I hacked deeper and deeper into the initialization process, but no enlightenment came. So finally I tried manually running the container with minimal options from the VM’s command line.

It still failed. Just for giggles, I did one more thing. Since Docker requires a root environment and I was trying to keep changes to a minimal, I was running Podman as root. I tried launching the Puppetserver container from an ordinaty user account.

And it worked!

I’m not sure why, although I suspect a couple of things. I believe that Pupper moved loctions on some of its credential files and was possibly counting on references to the old locations to redirect. Maybe they didn’t because some of this stuff is apparently using internal network operations and networking works differently in userspace.

At any rate, simply running the container non-rooted was all it took. Happiness abounds and I’ve got my recalcitrant server back online!

Ceph – another mystery error

Cannot place on myhost: Unknown hosts

Ran into this when attempting to use ceph orch to manage mgr’s.

It’s actually quite simple and has nothing (directly) to do with /etc/hosts or DNS. Only with the list of hosts that ceph knows internally.

And the problem — yet again — is that Ceph doesn’t manage multiple hostnames!!!

The hostname was registered as “myhost.mousetech.com”, but I had requested like this:

ceph orch apply mgr --placement="ceph01 myhost"

“ceph01” was OK, since that’s how Ceph had been set up. But when I added “myhost”, I did so with the fully-qualifed domain name. The bloody names must match what Ceph has on file exactly. Grrr!

So the proper command was:

ceph orch apply mgr --placement="ceph01 myhost.mousetech.com"

Apache, Tomcat and SSL – with Pictures!

Or at least examples!

Apache SSL to non-SSL Tomcat:

<VirtualHost mytchost:80>
  ProxyPass / http://backend.tomcat.host:8080
  ProxyPassReverse / http://backend.tomcat.host:8080
<VirtualHost mytchost:80>

<VirtualHost mytchost:443>
  ProxyPass / http://backend.tomcat.host:8080
  ProxyPassReverse / http://backend.tomcat.host:8080
<VirtualHost mytchost:80>

Apache SSL to SSL Tomcat. This is what you’d normally use if the Tomcat webapp had secure transport specified in its web.xml:

<VirtualHost mytchost:80>
  ProxyPass / http://backend.tomcat.host:8080
  ProxyPassReverse / http://backend.tomcat.host:8080
<VirtualHost mytchost:80>

<VirtualHost mytchost:443>
  ProxyPass / https://backend.tomcat.host:8080
  ProxyPassReverse / https://backend.tomcat.host:8080
<VirtualHost mytchost:80>

HOWTO: get Docker Containers under Centos 5 with Xen

Centos5 is getting long in the tooth, but then again, many of my servers are antiques that would find native Centos6 to be problematic.

A recent adventure in disaster recovery led me to upgrade several of my Xen DomU’s from CentOS 5 to CentOS 6, but I was distressed to discover that about the minimum you can get by with on RAM for CentOS6 is nearly 400MB. I wanted to host several CentOS6 VMs, but the thought of getting dinged to the tune of half-a-GByte of RAM plus several gigs of disk image didn’t sit well for lightweight systems.

The “in” thing for this kind of stuff is Containers, which neatly fit in the space between a full VM and something less capable such as a chroot jail. The question was, could I get CentOS 6 containers to work in a CentOS5 Dom0?

As a matter of fact, yes, and it was considerably less painful than expected!

I cheated and did the real dirty work using my desktop machine, which is running Fedora 20, hence is better supported for all the bleeding-edge tools. Actually, Ubuntu would probably be even better, but I’m at home with what I’ve got and besides, the idea is to make it as little work as possible given my particular working environment.

Step 1: Vagrant.

Vagrant is one of those products that everyone says is wonderful (including themselves), but it was hard to tell what it’s good for. As it turns out, what it’s good for is disposable VM’s.

Specifically, Vagrant allows the creation of VM “boxes” and the management of repositories of boxes. A “box” is a VM image plus the meta-data needed for Vagrant to deploy and run the VM.

So I yum-installed vagrant on my Fedora X86_64 system.

My selected victim was a basic CentOS 6 box, since for the VirtualBox VM environment.

vagrant box add centos65-x86_64-20131205 https://github.com/2creatives/vagrant-centos/releases/download/v6.5.1/centos65-x86_64-20131205.box

Step 2. Docker

It would have been more convenient to get a ready-made Centos6 Docker box, but most Docker-ready boxes in the public repo are for Ubuntu. So I did a “vagrant up” to cause the box image to download and launch, connected to the Centos6 guest, and Docker-ized it using this handy resource: http://docs.docker.io/installation/rhel/

An alternative set of instructions:

http://cloudcounselor.com/2013/12/05/docker-0-7-redhat-centos-6-5/

The process is rather simple as long as you’re using the latest CentOS 6.5 release. Older kernels don’t have the necessary extensions, requiring a kernel upgrade first.

Step 3. Porting to Xen

Once docker was working, the challenge of getting the VM from VirtualBox to Xen presented itself. I was expecting a nightmare of fiddling with the disk image and generating a new initrd, but there was a pleasant surprise. All I had to do was convert the VM image from the “vmdk” format to a raw disk image, transfer the raw image to the Xen box, hack up a xen config and it booted right up!

The details:

On the Fedora desktop:

$ qemu-img convert -f vmdk virtualbox-centos6-containers.vmdk -O raw centos6-containers.img
$ rsync -avz --progress centos6-containers.img root@vmserver:/srv/xen/images/centos6-container/

File and directory names vary depending on your needs and preferences.

Now it’s time to connect to “vmserver”, which is my CentOS5 physical host.

I stole an existing XEN DomU pygrub-booting definition from another VM, changed the network and virtual disk definitions to provide a suitable environment. The disk definition itself looks like this:

disk = [ "tap:aio:/srv/xen/images/centos6-container/centos6-containers.img,xvda,w"]

xvda, incidentally is a standard Centos VM disk image, with a swap and LVM partition inside.

I launched the VM and behold! a Centos 6 Docker container DomU on a CentOS 5 Dom0 base.

Everything should be this easy.

A Utility Program for Cataloging Ebooks

My books are among my greatest assets. So much so that finding room to store them all has long been a problem. So the emergence of ebook readers has been a big help to me.

However, to be of true value, certain aspects of the “dead tree” format must carry over and one of the most important ones is that when I buy a book, I own the book. I don’t “buy a license” for a book. Especially when you consider that ebooks typically sell for about the same price as their physical counterparts.

In order for a book to be truly “buyable”, however, certain principles must apply. On my side, that means that I treat it just like I would a real book and don’t go giving away freebie copies to my friends. Ideally, I could sell “used” ebooks or give them away, but that’s an idea whose time has yet to come.

On the seller’s side, that means a minimum of intrusiveness. I have to be able to read the book without being in direct contact with its “Big Brother” server out on the Internet. I will not tolerate sellers who yank back or alter books without my express permission. No “1984”s down on the Animal Farm. And my library shouldn’t “burn down to the ground” if the vendor goes under a la Borders Books.

The Barnes&Noble system generally meets those criteria. There’s DRM, but it’s (presently) “locks for honest people”, which I can live with. If for any reason, B&N shuts down their servers and exits the business, I have local backups and the means to keep reading them. My means of obtaining local backups has been tampered with on the Nook Tablet, which “hides” the purchased books from the USB storage device, and that’s a feature I hope that they’ll reconsider, since I’ll be keeping it in mind as I make future purchases, but for now, I can deal with it, if not appreciate it.

However, once I’ve made my local backups, I needed a way to be able to browse the local library. The B&N ebook filenames are based on their ISBN numbers, which makes them unique, but hard to locate. So I wrote this little app to be able to scan epub-format books and list their names and authors in a format that’s convenient for spreadsheets and database to take in.

It also determines what encryption algorithm provides DRM. If I scan a library and see that someone’s using DRM that I can’t decode, that’s a red flag and WILL affect both my present-day opinion of the publisher (as in what I recommend to others) AND future purchases.

My sincere thanks to publishers such as O’Reilly and Associates, Baen Books, and others who have been brave enough to forgo DRM altogether. You have placed your trust in my honesty and I will uphold it. Late update: TOR has joined the no-DRM club according to an announcement I just heard today. I’ve helped them kill a few trees over the years!

Catalog application downloads

My ePub catalog utility is a Java application located at http://www.mousetech.com/ebooks-catalog-1.0.jar. To use it:


java -jar ebooks-catalog-1.0.jar f1.epub [f2.epub f3.epub ...]
http://www.mousetech.com/ebooks-catalog.zip.

Both source and executable may be freely redistributed as long as you don't charge for it or take away credit for my efforts.

The Horrors of OpenOffice Macro Programming

I love OpenOffice as a user. I have absolutely no envy for (ahem) That Other Product. In truth, just about every new MS-Office feature added since 1997 has been completely meaningless to me.

I have been manipulating OpenOffice/LibreOffice/StarOffice in various creative docs for some years now, but mostly by running XML processes against the ODF files. While XSLT is a royal pain, it beats the alternatives, and the ODT format is straightforward enough.

Recently, however, I’ve had a need to do some heavy reformatting of spreadsheets. They get generated as CSV files, imported, formatted, and various formulas plugged into them. I got tired of doing this by hand, so I decided it was time to automate the process.

¡Ay, Caramba! What a nightmare!

On the one hand, there are all sorts of different ways to script OpenOffice. Java, Python, beanshell, and Basic. Two different kinds of Basic.

If you use the macro recorder, it captures scripts using one programming interface. However, when you write scripts by hand, you would normally use a completely different, less-awkward interface. Even if you wanted to use the capture API, trying to figure out what the various functions and features are makes it a rough proposition, since almost everything is basically set up an array of parameters and passing it to an execute function.

The simpler, hand-coded BASIC API, however, isn’t much better. Unlike the capture API, which maps onto the document model objects directly, the BASIC API attempts to provide convenience methods and properties. Unfortunately, some of them are pretty darned inconvenient, and as awful as the access documentation is, at least the low-level stuff is documented. Lots of luck with the high-level stuff.

I’ve heard the excuse made that the reason MS has so much better Office programming documentation is that they have the force of a Fortune 50 company behind them, but that rings hollow. OpenOffice after all was owned by Sun, then Oracle, and the community of OOffice users is pretty large. There really isn’t an excuse. About the best I’ve seen for how to program OpenOffice is a book that’s out of print.

So, while I myself don’t have the resources to remedy this situation, I’m putting together something that will at least work as a “cheat sheet” for OOCalc macro programming. Here it is: OpenOffice Calc Macro CheatSheet

JPA and Fixed-length text fields in databases

Lucky me. I took over a system whose database is awash with compound keys. The more I work with this stuff, the more justification I find for always having a simple sequence key as the primary key, despite the apparent extra overhead.

I’ve had a long-running problem where parent-child (one-to-many) relationships weren’t able to fetch their children. At best, the collection came up empty. At worst, accessing the children collection property of the parent would throw a NullPointerException because the “bag” property of the PersistentBagCollection wasn’t initialized.

I spent a LOT of time with Hibernate’s trace-level logging turned on and the only thing I could determine was that the children were, in fact, being correctly loaded and formed into a collection, but the collection wasn’t the collection that was bound to the parent – instead the child collection was being silently lost.

One thing that did look suspicious, however was that the trace showed two collections found. I went round and round with this, added various bits of code that made objects more identifiable in the debugger, and finally – at long last ended up with a set of trace messages showing the owner primary keys of these collections.

Lo, and behold! One of them showed trailing spaces in the key field values, the other did not.

Trailing spaces in ORM fields have much the same effect and being sloppy with upper/lower case in Java filenames under Windows. Sometimes you get away with it, but not always. Because of the trailing spaces, the two primary keys didn’t compare equal, the “real” collection couldn’t be attached to its parent, and data was lost.

I suppose that this problem could be avoided by proper implementation of the equals() and hashCode() methods of the offending primary key objects, but I found it easier to simply force the key objects to be space-padded and to be more careful about what I passed to search functions, since the ultimate problem lies in the fact that sending a space-trimmed value to a Hibernate JPA query will match and return a space-padded object.

The ultimate cure for this would probably be to use fixed-size character arrays instead of String objects, but there are several problems here as well:

  1. Java’s static type checking doesn’t cover mismatches on array size.
  2. Individual characters in a character array can be null
  3. The ORM class-generating tools I use render fixed character fields as Strings, not arrays
  4. I’m not actually sure that either JPA or Hibernate support character array properties. I have enough grief trying to portably represent boolean and enum values.

Using the Apache OpenJPA command-line tools

Stuff that’s unfortunately not concentrated into a convenient ready-to-use example in the Apache OpenJPA docs. But that’s what this blog is all about!

When using the reverse engineering, schema, and other tools directly from a shell script (not Ant or Maven), the default place to get datasource definitions and related options is in META-INF/persistence.xml. This file is mandatory even in cases where you don’t actually connect to the database, such as generating Java source from an XML schema (reverse generation).

Because the tools are using a validating parser, a schema name is REQUIRED. Example, supplying the JPA schema via the xmlns attribute:

<?xml version="1.0"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence" version="1.0">
  <persistence-unit name="openjpa">
    <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
    <properties>
      <property name="openjpa.ConnectionURL" value="jdbc:hsqldb:tutorial_database"/>
      <property name="openjpa.ConnectionDriverName" value="org.hsqldb.jdbcDriver"/>
      <property name="openjpa.ConnectionUserName" value="sa"/>
      <property name="openjpa.ConnectionPassword" value=""/>
      <property name="openjpa.Log" value="DefaultLevel=WARN, Tool=INFO"/>
      <property name="openjpa.jdbc.DBDictionary" value="StoreCharsAsNumbers=false"/>
    </properties>
  </persistence-unit>
</persistence>


CAUTION: You should delete the orm.xml file when re-running these tools. Otherwise they will use the old copy, which may not be in sync with your current efforts.

JSF Submit-on-Enter forms

It’s not (so far as I know) officially required anywhere, but it’s a generally accepted principle that a form on a web page containing one text field and one submit button should submit itself when you type in text and press the ENTER key. A classic example of this is Google .

In actuality, it isn’t quite that simple. For totally unknown reasons, Microsoft Internet Explorer requires at least two text fields and this apparently goes all the way up to and including IE8 (at least when you upgrade from IE6). Fortunately, the “spare” text field can be invisible and ignored by the application.

JSF is more complicated, and AJAX-aware JSF such as RichFaces compounds the issue even more. Here’s a solution that seems to work for me:

  <h:panelGroup>
     <h:inputText style="visibility:hidden;display:none;"
        disabled="disabled" required="false" />
     <a4j:commandButton id="ask"
        type="submit" styleClass="button1"
        action="#{forgotPassword.doAskQuestion}"
        reRender="pnl1,fpMsg" value="Submit" />
 </h:panelGroup>

I am indebted to Lincoln Baxter for this particular hidden field definition (it’s quite touchy). He’s the person behind the JSF PrettyFaces product (ocpsoft.com).

Note that this is only a kludge, and may not always work. Additionally, it causes the RichFaces a4j:commandButton oncomplete action to misbehave under both IE and Firefox (though in different ways), as of RichFaces 3.4.