Wilmington Tree Lighting 2017

ice sculpture
Ice sculpture on the town green

The only thing to note from this year’s tree lighting: the girls went on a hay ride with boys.

hay ride
Kids on the hay ride. I apologize for the potato-like quality of the picture; my phone is old and does not do justice in low-light situations, never mind low-light with movement.

Perl’s Open3, Re-Explained

I recently went spelunking into a core Perl module that I previously knew nothing about, IPC::Open3.  After fifteen years of developing in Perl I finally had a reason to use it.

If you’re reading this, it’s probably because you went looking for information on how to use open3 because the module’s documentation is bad.  I mean it’s really, really terrible.

Not only will you not know how to use open3 after reading the docs, you may become convinced that maybe open3 isn’t the module that you need, or maybe it would work but you’d just be better off looking for something else because this is too damn hard to use.

Fear not, intrepid reader, because if I can figure it out so can you.  But I will try to save you some of the leg work I went through. There’s precious little information scattered online, because this isn’t a popular package.  My loss is your gain, hopefully this helps you.

Why IPC::Open3?

When Would I Use IPC::Open3?

open3 is used when you need to open three pipes to another process.  That might be obvious from the name as well as the package’s synopsis:

$pid = open3( \*CHLD_IN,
              \*CHLD_OUT,
              \*CHLD_ERR,
              'some cmd and args',
              'optarg', ...
            );

Why would you do that?  The most obvious situation is when you want to control STDIN, STDOUT, and STDERR simultaneously.  The example I provide below, which is not contrived by the way but adapted from real production code, does exactly that.

There Are Lots Of Modules To Make This Easier, Why Should I Use IPC::Open3?

IPC::Open3 is part of the Perl core.  There’s a lot to be said for using a library that’s already installed and doesn’t have external dependencies vs. pulling in someone’s write-once-read-never Summer of Code academic project.

In addition, the modules that I found only served to hide the complexity of Open3, but they did it badly and didn’t really remove much code compared to what I came up with.

What Else Do I Need?

One of the things that’s not obvious from the Open3 docs are that you’re not going to use IPC::Open3 by itself.  You need a couple of other packages (also part of core) in order to use it effectively.

How I Used IPC::Open3

In our example, we’re going to fork a separate process (using open3) to encrypt a file stream using gpg.  gpg will accept a stream of data, encrypt it, and output to a stream.  We also want to capture errors sent to STDERR.

In a terminal, using bash, this would be really easy: gpg --decrypt < some_file > some_file.pgp 2>err.out

We could do all of this in Perl by writing temporary files, passing special file handle references into gpg as arguments, and capturing STDERR the old fashioned way, all using a normal open().  But where’s the fun in that?

First, lets use the packages we’ll need:

use IO::Handle;
use IO::Select;
use IPC::Open3;

IO::Handle allows us to operate on handles using object methods.  I don’t typically use it, but this code really appreciates it.  IO::Select does the same for select, but it helps even more than IO::Handle here.

use constant INPUT_BUF_SZ  => 2**12;
use constant OUTPUT_BUF_SZ => 2**20;

You might want to experiment to find the best buffer sizes.  The input buffer should not be larger than the pipe buffer on your particular system, else you’ll block trying to put two pounds of bytes into a one pound buffer.

Now, using IO::Handle we’ll create file handles for the stdin, stdout, and stderr that our forked process will read and write to:

my ( $in,
     $out,
     $err,
   ) = ( IO::Handle->new,
         IO::Handle->new,
         IO::Handle->new
       );

Call open3, which (like fork) gives us the PID of our new process.

Note: If we don’t call waitpid later on we’ll create a zombie after we’re done.

my $pid = open3( $in, $out, $err, '/usr/bin/gpg', @gpg_options );

if ( !$pid ) {
    die "failed to open pipe to gpg";
}

One of the features of IO::Select is that it allows us to find out when a handle is blocked. This is important when the output stream is dependent on the input stream, and each stream depends on a pipe of limited size.

We’re going to repeatedly loop over the handles, looking for a stream that is active, and read/write a little bit before continuing to loop.  We do this until both our input and output is exhausted.  It’s pretty likely that they’ll be exhausted at different times, i.e. we’ll be done with the input sometime before we’re done with the output.

As we exhaust each handle we remove it from the selection of possible handles, so that the main loop terminates naturally.

The value passed to can_write and can_read is the number of seconds to wait for the handle to be ready.  Non-zero timeouts cause a noticeable delay, while not setting it at all will cause us to block until the handle is ready, so for now we’ll leave it at zero.

# $unencrypted_fh and $encrypted_fh should be defined as
# handles to real files

my $sel = IO::Select->new;

$sel->add( $in, $out, $err );

# loop until we don't have any handles left

while ( my @handles = ( $self->handles) ) {
    # read until there's nothing left
    #
    # write in small chunks so we don't overfill the buffer
    # and accidentally cause the pipe to block, which will
    # block us
    while ( my @ready = ( $sel->can_write(0) ) ) {
        for my $fh ( @ready ) {
            if ( $fh == $in ) {
                # read a small chunk from your source data
                my $read = read( $unencrypted_fh,
                                 my $bytes,
                                 INPUT_BUF_SZ,
                               );

                # and write it to our forked process
                #
                # if we're out of bytes to read, close the
                # handle
                if ( !$read ) {
                    $sel->remove( $fh );
                    $fh->close;
                }
                else {
                    syswrite( $fh, $bytes );
                }
            }
            else {
                die "unexpected filehandle for input";
            }
        }
    }

    while ( my @ready = ( $sel->can_read(0) ) ) {
        # fetch the contents of STDOUT and send it to the
        # destination
        for my $fh ( @ready ) {
            # this buffer can be much larger, though in the
            # case of gpg it will generally be much smaller
            # than the input was. The process will block if
            # the output pipe is full, so you want to pull as
            # much out as you can.

            my $read = sysread( $fh, my $bytes, OUTPUT_BUF_SZ );

            if ( !$read ) {
                $sel->remove( $fh );
                $fh->close;
            }
            elsif ( $fh == $out ) {
                # $encrypted_fh is whatever we're throwing output
                # into

                syswrite( $encrypted_fh, $bytes ) if $read;
            }
            elsif ( $fh == $err ) {
                print STDERR $bytes;
            }
            else {
                die "unexpected filehandle for output";
            }
        }
    }

    # IO::Handle won't complain if we close a handle that's
    # already closed
    $sel->remove( $in ); $in->close;
    $sel->remove( $out ); $out->close;
    $sel->remove( $err ); $err->close;

    waitpid( $pid, 0 );
}

That’s actually about it.

I keep my buffer for input small, as pipe buffers tend to be small.  If you overload your pipe your program will hang indefinitely (or until an alarm goes off, if you set one).  4096 bytes seems to be the limit, though your own limit may be different.  When in doubt, be conservative and go smaller.

The output buffer can afford to be bigger, up to the limit of available memory (but don’t do that).  In our example of encryption gpg will consume much more than it produces, so a larger buffer doesn’t really buy you anything but if we were decrypting it would be the reverse and a larger buffer would help immensely.

Road Trip 2017: Eclipse Edition

There’s a back-story to this trip.  My grandfather served on the USS Cobia during World War 2.  Through an accident of history, that very submarine is preserved as a floating museum in Manitowoc, Wisconsin.

Grandpa died over a decade ago.  I found out about the Cobia several years after he died.  Grandpa never mentioned that it still existed, though he had attended crew reunions there.  I think he considered his service to be very personal and never spoke of it much.  I’ve wanted to visit, to make a pilgrimage if you will, but I wanted to take the girls when they were old enough to remember and appreciate the history of the thing.

I took advantage of a confluence of events this year to do it: 1) Alpha is old enough, 2) an impending total eclipse would pass (relatively) close by, and 3) I wanted a vacation and a road trip.  Alpha was agreeable to going, and she too wanted a road trip, so boom it was on.

As a side note, Alpha handled this trip with aplomb.  She has a bladder of iron, remained agreeable even when things went sideways, and is generally helpful around the car.

Overall Trip Statistics: 5 days, 10 states, 2700 miles, 2900 feet at highest pass.

Day 0: Destination Erie, Pennsylvania

The first day of the trip started with overcast skies that swiftly turned to rain — but the day ended with sun

We chose to leave on a Friday.  I put in enough of a day at work to count it as a full day and avoid using another vacation day, which meant leaving around lunch time.  Alpha and I were packed and ready to go by 1 pm.  A final stop to see Meghan at work, and we hit the highway… and traffic.

I made the mistake of assuming that traffic would be moderate at mid day.  Friday afternoon rush hour tends to be nasty, especially so in summer, but it begins earlier than I know.

As we crawled our way down the turnpike we witnessed an almost-crash in front of us.  We mostly idled our way until Charleton.  I guess that house rentals in Maine must run from Saturday to Friday, since a large portion of the cars around us were from NY, NJ, and PA and they all peeled off at the I-84 exit.

Cruising through Schoharie, NY
We found lots and lots of construction, but we never slowed down in New York

The rain moved in as the traffic cleared.  We made fair time for a couple of hours as we skipped into New York state under showers.

Skies cleared by mid-state, with a few hours of sunlight left.  After dark we cruised through the Seneca reservation.  All of the road signs were translated into Seneca, which was kind of neat.  They also have a casino that, much like Foxwoods, rises out of the forest in a jumble of incongruence.

We made Erie at about 9:30 that night – a Motel 6 alongside Interstate 90.  It was a plain, but clean and serviceable hotel.  We turned in sans dinner and slept well.

Travel Report:

Central and western New York are very boring.  We didn’t get cell phone service anywhere and the highway just seems to go forever.  There are some peculiar town names in western New York, ‘Horseheads‘ was a particular favorite.

Day 2: Destination Chicago

Breakfast at Zodiac Dinor
Breakfast at the Zodiac Dinor (sic). It was delicious and huge; we never did care to eat lunch that day.

We had a fantastic breakfast at a local diner and set off on the scenic route for a little while.  We wanted to see Lake Erie.

Along the way we found some sites and some sights.  The day was warm but a little cloudy, which meant that it never got too hot.  I was looking forward to rolling with the top down on this trip and I was not disappointed.  We hit the sweet spot for a convertible: 50 mph @ 72 F, moderate humidity.

Before we left Conneaut, Ohio we stopped at a lemonade stand; the kids even served us in the car.  (I’ve made it a policy over the years to always stop for lemonade.)  The roads were beautiful, alternating between showing us the lake and just being green.

D-Day Re-enactment in Conneaut OH
We drove by a D-Day re-enactment in Conneaut, OH [click to enlarge]
D-Day Re-enactment, another view, Conneaut, OH
A better view of the D-Day Re-enactment. It appears to be a yearly event. [click to enlarge]
chicago skyline
Looking over the highway near our hotel. Chinatown is a little further left than is visible. [click to enlarge]

Lake Erie from Ashtabula, OH
The scenic byway made some very close passes to the coast and we saw some great views of the lake.

Lake Erie itself is gorgeous.  The towns that border the lake are a mix of quaint, middle-America, and gaudy: some reminded me of Cape Cod, some reminded me of the over-commercialized beach town of Misquamicut, RI, and some reminded me of any number of nondescript towns that I’ve encountered along my travels.

We switched over to the highway around Geneva-on-the-Lake, Ohio, as we became conscious of the time: I wanted to get into Chicago before sundown.  Being Saturday we didn’t have to worry about rush hour.  We rolled into town around 5:30, which was perfect.

Dinner at Tasty Place
Our meals at Tasty Place. As the name suggests the meals were, in fact, very tasty. [click to enlarge]
Our hotel in Chicago was right next to Chinatown, so we took a walk to find some dinner.  We found a hole-in-the-wall with some of the best Chinese food I’ve ever had.

We turned in no long after dinner; we had a full day ahead.  We spent a little time unwinding and calling home.  Earlier in the day we had found out that the USS Indianapolis had been located, and shared the news with Meghan and Beta.  They proceeded to watch Jaws (with the Indianapolis scene) and brag to us about doing so.

 

Travel Report:

Route 531 along the coast of Ohio is beautiful.

Indiana Rest Stop
We stopped to rest for a bit in Indiana. The turnpike was single-lane (for construction) and bordered by jersey barriers on both sides for miles, which does weird things to your brain after a while.

The Indiana turnpike is bumpy as hell, and boring, until you approach Elkhart.

Day 3: Destination Cobia

This was the main event.  We arrived around lunchtime.

USS Cobia
USS Cobia, positioned at the stern and looking towards the bow.

There are regular tours of the submarine, from forward torpedo room to aft torpedo room.  They’ve restore the submarine to about 80% working condition.

uss cobia torpedo room
The forward torpedo room

We found at least three, possibly four, photos with Grandpa in them.  The questionable photo is from a reunion photo, not everyone was looking at the camera and, sad but true, old men all start to look alike after a certain age.  One other photo that we saw him is from a reunion, the final two are from his time on the submarine.

Alpha, with the Cobia’s stern in the background [click to enlarge]
Our tour guide was new – his first day! – and he wasn’t familiar with any of the crew besides the captain.  Grandpa, being an officer but not the captain, wasn’t a name he knew.  The tour was about twenty minutes from bow to stern.

Being a real WW2 submarine the quarters were very cramped; I’m not sure how my grandfather, being a few inches taller than I am, was able to fit through the tiny intra-compartment doorways.

The submarine part of the museum is a little small; there’s a somewhat larger section devoted to boats on Lake Michigan and the various wrecks.  Alpha and I agreed that we’d like to come back another time to view the lake history in more detail.

We ate dinner in Chicago, a pizza place of course.  After we got back to our room Alpha wanted to veg out.  I felt cooped up by the car all day, so I went out for a walk around the neighborhood.  I took some photos but they were terrible; I need a new phone with a better camera.

Travel Report

The state of Wisconsin replaced a large section of the pavement on Interstate 43, both northbound and southbound, with a concrete washboard.

The car’s computer calculated our average MPG to be north of the 30 mark sometime this day.

Day 4: The Eclipse

Alpha and I got going a bit later than I had originally planned, and had a smashing breakfast at a market that’s a block from the hotel.

Clouds over Ashkum, IL
The clouds became progressively more forbidding as we raced south to meet the eclipse. The weather report for southern Illinois was clear skies, so we pressed on. Skies grew even darker shortly after this photo was taken, but we never got more than a sprinkle in the morning. [click for a very-hi-res version.]
Getting close to or under the eclipse path was a bonus, so we didn’t have any specific destination to reach, just “as close as we are able or choose to get.”  I picked Marion, Illinois as a target and we started driving.

We hit some traffic along the way, not due specifically to the eclipse but around construction along the highway.  I don’t know if traffic would have been lighter and we would have slipped right through if it hadn’t been for the eclipse.  We probably spent an hour and a half sitting in various stand-stills, and 30 minutes more finding detours, and were about an hour outside of Marion when we decided to stop to watch the eclipse in Effingham.  It was a fair place to stop, with fuel and food, and we weren’t the only ones peering up at the sun.  While it wasn’t directly in the path of totality the sun was reduced to a tiny hair-like sliver.

eclipse peak
The clouds proved to be a mixed blessing by obscuring the view but allowing us to take unfiltered photos of the eclipse. I was too busy taking in the moment to photograph the actual peak — this was from about four minutes before the peak. The eclipse was moving from left to right in this photo, and the top was the only bit visible at the peak.

From the stopping point, our next destination was Lexington, Kentucky.

Downpours over Newton, IL
We ran into thunder and lightning as we moved east through Illinois

We got into Lexington late, due in part to rain.  A large part of our drive was on country roads which, more than the interstates, are long, straight, and surrounded by corn.

We arrived in Lexington after 7 pm.  It’s a college town and we ate dinner at a joint that caters to the college crowd.  It was delicious.  Alpha had breakfast for dinner and I had a burger.  As we walked out the door Alpha remarked how good it was.  The person walking out behind us was, unbeknownst to us, one of the cooks heading out for a break; she flashed a huge smile and said “thanks!”

Alpha stayed in our room while I took a swim in the hotel pool.  We were in bed and lights-out before 9 pm, as we were leaving very early the next morning.

Traffic Report

Midwestern seasons seem to be like New England: almost-winter, winter, still-wintery, and road construction.  The side roads are much better for driving than the interstates, and almost as fast and direct.

Indiana doesn’t seem to understand the concept of interstates.  They have traffic lights and intersections on I-64.

Day 5: Destination Home

Alpha in the carThis was the longest, hardest drive of the trip: almost a thousand miles in one day.  Under the best of conditions it would be a fourteen-hour trip.  Alpha and I discussed doing it in one day or breaking it into two, and her answer was firm: lets go home today.  So we did, come hell or high water.

We didn’t take any scenic routes or make any unnecessary stops, it was just pounding the pavement (so to speak).  We left Lexington at 7:30 am and made it home around 11:30 pm.  We texted Meghan every time we crossed a state line, and she was reposting our updates to Facebook.

Travel Report

Somewhere in eastern Kentucky we passed a field that could have been straight out of a Bob Ross painting, with a shed or small barn, happy little trees, and a small mountain in the background.

West Virginia is beautiful, even from the highway, though cell phone reception is non-existent.  We suddenly had great coverage when we entered Maryland, even though the mountains didn’t stop, so it seems to be a political issue not a geographic one.

We reached a peak mileage of 31.8 MPG, according to the car’s computer.  My back-of-the-envelope math came up slightly lower at our last fill-up, but close enough.

The climate control system lost it’s mind somewhere in Kentucky and was alternately blowing cold and warm, regardless of actual setting, for the rest of the ride.  Things got a bit warm and sweaty, but not the worst I’ve ever had.

Pennsylvania doesn’t know how to manage traffic around construction, of which the was plenty, including some standstills well before rush hour.  Neither does Connecticut, where we were hitting standstills at 9 pm.  That shouldn’t happen on a Tuesday night.

Sixteen hours of straight driving is very, very tiring.

Suffolk Downs

We spent a day at the races.

Well, ok not a whole day, just a few hours.  We didn’t go to bet, though Meghan did bet $3 and is now up by $3.  We went to watch the horses.

Our destination was Suffolk Downs.  It is an eighty-something year old horse-racing track just outside Boston.  Its days are numbered but they’re still holding a few races per year in the meantime.

Our Alpha child loves horses.  She’s loved them since the first grade, when she abruptly gave up a love affair with dinosaurs to fall madly in love with horses instead.  When I found out about the races, I realized I had an opportunity to make one very happy child.

We bought a program as soon as we arrived and set Alpha loose.  She immediately headed over to the paddock, started matching the horses to the program, and figured out pretty much everything on her own.  She picked her favorites and apparently has a good eye because, of the two she picked, one was a winner and the other placed at #2.

We stayed for three races before heading back home.  The siren song of chores beckoned us home, plus Beta child was in a mood, one that didn’t include horses or racing.  There’s another race weekend this year, and I think Alpha and I will attend again.

Color Me Amused

A big shout-out to Notes from the Eastern Front!

I just found out that my brother’s family maintain a blog as well.  As usual with things that pertain to my brother, we found out by accident — he hasn’t mentioned it to anyone for the three months since it’s inception.

FYI, their blog overloads the Alpha and Beta identifier for their kids, who are known as Eta and Mu on this blog.

Camping @ Hammonasset

Our southern friends from Connecticut invited us to go camping at Hammonasset Beach State Park with them.  They go every year as a big party, with family and friends.

I only recently found out that this is a thing; our neighbors/friends from across the street go up to a campsite in New Hampshire every year to meet with other friends, some of whom they only know from camping.

tent
Our tent [click to see more of our campsite]
This was our first time going camping as a family, ever.  I’ve gone deep-woods camping by myself. (No facilities, no roads, no people, no nothing — I’ve never smelled worse than three nights of that.)  Meghan had been camping at Pennsic and Gulf Wars during college.  (She has stories that amaze.)  Beta has been overnight camping (in cabins) as part of Girl Scouts.  We’ve all been “camping” in the backyard.  This trip was a first for being away from home and trucking everything we would need.

Day 1

We arrived mid-afternoon and immediately set to pitching our tent, figuring that there could be nothing worse than setting up a tent in the dark when you’re exhausted.  Though the tent was new and this was our first time, it went up pretty quickly and cleanly.

Jones's and Gailey's
Jones’s and Gaileys. The Gailey’s custom campaign tent is in the background; behind are a couple of Hobie Cats that we sailed the next day.

Our friends provided dinner: our traditional Friday night get-together victuals, spaghetti with meat sauce and garlic bread, only on a Monday.  Being experienced campers with a lot more room to pack stuff, they graciously offered to provide dinner for the two nights we were there.  After dinner there were s’mores around a communal campfire, where we got to meet the rest of the party — more Gaileys and some family friends.

There were kids of all ages, including a few that were right around Alpha and Beta’s ages.  They played boffer swords until dark, then convinced a couple of us adults to play manhunt.

We didn’t tuck into bed until about 10 pm.  As the kids washed up some of us gathered around to chat and stargaze, and we were able to point out some satellites going by.  The other adults hadn’t known that you could see them so easily.  Schwing!

Sleep, for a variety of reasons, was somewhat elusive the first night, except for Alpha who can sleep through pretty much anything.

Day 2

We started the day with pancakes, eggs, and bacon.  The Gaileys possess a propane-fired flattop grill which made cooking a breeze.

sailing away
Quinn and Sam, sailing to Tahiti

Sam wanted to get the two Hobie Cats they brought along into the water.  We got to the beach and set to rigging them, then took ourselves and the kids out for some sails.

We were at the beach for about four hours, and as a group we only got slightly burned — I had planned ahead and coaxed the girls into going to the beach with me for a week leading up to the trip.  Meghan got a bit burned across the shoulders, and oddly enough Mu (the junior Gailey) got sunburned on the tops of his feet.

After packing the boats back onto their trailer, we headed back to camp (with a detour into town to get aloe) in order to make dinner.  Second night was a communal pot luck, with hot dogs, hamburgers, and a bunch of sides.  We got to know the other campers in the party and found that we have a lot in common.

rainbow
A passing shower gave us a rainbow… over the campsite bathrooms.

The rain, which had been holding off all day, finally came in the form of a few brief showers and a rumble of thunder.  I checked the weather radar back home and, wouldn’t you know it, heavy storms were moving through our town.  (I love thunderstorms but seem to have a repelling effect on them.  Even the strongest storms peter out as they reach our area.)

We took the opportunity to coax the girls into bed a bit earlier, though sleep was still hard to find the second night.  Besides possibly being over-tired, the temperature dropped to nearly 50° F so everyone (except me) was cold despite blankets.

Day 3

We all got up early to a beautiful morning.  Meghan and Joanne took a walk back out to the beach to look for some bald eagles we had noticed the day before.

Checkout is 12 pm, and we planned to swing through Noank on the way home to see the folks, so we packed up the campsite right after breakfast (more bacon, eggs, and pancakes, plus sausages) and hung out with the Gaileys until it was time to go.

We bid adieu and headed out right at noon, spent a few hours having lunch and visiting with Mom and Dad Jones, plus Katie Jones and Eta (my niece).  We made it home just in time to get Butter out of doggie daycare.

WordPress Error: cURL error 6: Couldn’t resolve host ‘dashboard.wordpress.com’

Background:

I maintain a WordPress blog that uses Jetpack’s Stats package.

Issue:

We started getting this error message when opening the ‘Stats’ page:

We were unable to get your stats just now. Please reload this page to try again. If this error persists, please contact support. In your report please include the information below.

User Agent: 'Mozilla/5.0 (X11; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0'
Page URL: 'https://blog.server.tld/wp-admin/admin.php?page=stats&noheader'
API URL: 'https://dashboard.wordpress.com/wp-admin/index.php?noheader=true&proxy&page=stats&blog=XXX&charset=UTF-8&color=fresh&ssl=1&j=1:5.0&main_chart_only'
http_request_failed: 'cURL error 6: Couldn't resolve host 'dashboard.wordpress.com''

The entire Stats block in the Dashboard was empty, and the little graph that shows up in the Admin bar on the site was empty as well.

Other errors noticed:

RSS Error: WP HTTP Error: cURL error 6: Couldn't resolve host 'wordpress.org'
RSS Error: WP HTTP Error: cURL error 6: Couldn't resolve host 'planet.wordpress.org'

These errors were in the WordPress Events and News section, which was also otherwise empty.

This whole thing was ridiculous on it’s face, as the hosts could all be pinged successfully from said server.

I checked with Jetpack’s support, per the instructions above, and got a non-response of “check with your host.”  Well, this isn’t being run on a hosting service so you’re telling me to ask myself.  Thanks for the help anyway.

Resolution:

The machine in question had just upgraded PHP, but Apache had not been restarted yet. The curl errors don’t make much sense, but since when does anything in PHP make sense?

It was kind of a “duh!” moment when I realized that could be the problem.  Restarting Apache seems to have solved it.