Author: H Walker Jones, Esq

A professional programmer with a sordid past involving sysadmin, tech support, and cooking.

Beautiful August Sunset

 

pink and orange clouds after sunset
The clouds caught the sunlight after the sun had dipped below the horizon. Taken from the Market Basket parking lot in Wilmington, MA, hence the street lights.

 

Presidential Politics 2016

This is probably the only political statement I’ll make on the internet this year, and it’s not telling you who to vote for or pushing a particular issue in your face. I want to introduce some perspective.

Everyone is worried about the presidential election this year — will it be Trump or Clinton, I can’t vote for him and I don’t like her, if X is elected it will be a disaster for the country, what about these very fine 3rd party candidates, etc. You’re all barking up the wrong tree, you’ve forgotten how the government really works, and that the occupant of the oval office has limited powers and doesn’t really matter. What matters is the Senate, and we should be talking about senatorial candidates.

The president doesn’t install justices on the Supreme Court, judges on the Court of Appeals, or more than 6000 people onto various agencies, and the president doesn’t decide who sits in their own cabinet. The president may nominate people for these positions, but the Senate confirms those appointments. If the Senate doesn’t consent to a nominee, that nominee will not be appointed.  These appointments have lasting effects long after a presidential term is up so the appointment really matters, but the field of unappointed nominees are irrelevant.

What about the House of Representatives? Their districts are fairly gerrymandered so individual representatives aren’t all that responsive to national politics, and the House’s power is somewhat limited by design. They do play a role in budgeting and legislation, though — along with the Senate.

The president doesn’t set the federal budget, Congress does. The president does proposes a budget (and only does so because Congress can’t be bothered — by law they are supposed to make it and they can always change anything proposed by the president) but the budget is voted on and set by Congress.

You may object that “my senators are fine, and they’re not even up for election this year!” This may be true, but it’s not a good objection. We all have friends and family in other states, you can discuss the issues and candidates across the nation, and persuade them to take an interest and vote. (But never brow-beat or fight, please, that’s neither respectful or respectable.)

So in this election cycle, lets talk about the things that matter. The president isn’t one of them.

Fenway 2016

fenway park
Fenway Park from the grandstands (section 30)

Baba took the whole clan to Fenway to see the Red Sox play the Chicago White Sox.  It was an afternoon game so that the kids could come, too.

del's lemonadeBaba generously bought us stadium food as well – no small sum for hot dogs, pretzels, pizza, and Del’s Lemonade.

The Sox stunk until the bottom of the 5th inning, at which point they came back from 4-0 lead by the White Sox to win 7-8.

Delta fell asleep by the 8th inning, which is surprising given the amount of noise every time the Sox got a run.  Kappa, who’s about a year and a half, stayed awake and in mostly good spirits through the entire game.  Beta was well behaved, and Alpha genuinely enjoyed herself.

We left as the 10th inning was starting so that we could avoid some of the crowds with the kids and headed down the street to get dinner at Wahlburgers.

QOTD

First they came for the verbs, and I said nothing because verbing weirds language. Then they arrival for the nouns, and I speech nothing because I no verbs.

Source: http://www.bash.org/?954654

“ERROR: … failed to process… ” in NiFi

I was greeted by a few cryptic things in NiFi this morning during my morning check-in.

  1. A PutSQL processor was reporting an error:
    "ERROR: PutSQL[id=$UUID>]failed to process due to java.lang.IndexOutOfBoundsException: Index: 1, Size: 1; rolling back session: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1"
  2. There were no recent errors counted in the LogAttribute counter we set up to record errors;
  3. The Tasks/Time count in the PutSQL processor was though the roof, despite the errors and lack of successes.

Needless to say, the processor was all bound up and a number of tasks were queued.  Not a good start to my day.

I checked the data provenance and didn’t see anything remarkable about the backed-up data.  The error message suggests (to me) that the first statement parameter is at fault, and that parameter happened to be a date (which has been problematic for me in NiFi with a MySQL backend).  Neither that value, nor the rest of the values, were remarkable or illegal for the fields they’re going into.

It wasn’t until I spent some time looking over the source data that I saw the problem: there is a duplicate key in the data.  This error is NiFi’s way of complaining about it.

In our case the underlying table doesn’t have good keys, or a good structure in general, and I’m planning to replace it soon anyway, but updating the primary keys to allow the duplicate data (because it IS valid data, despite the table design) has solved the issue.

Komodo IDE Headaches

I’m slowly coming around to the idea that an IDE might be useful for PHP/Symfony projects (still not convinced about other languages and frameworks) and I’m currently trying out ActiveState’s Komodo IDE 10 on Linux.

It looks great but it’s… buggy.  One day in and I’m already getting frustrated with it.

  • The preference file doesn’t appear to be saved until the application is closed, if the application crashes it’s not clear that your changes will be saved.  This might be a safety feature, but probably not, because…
  • At least some preferences don’t take effect until the application is closed.  Not the ones that you’re warned about like checking remote files for changes, but other ones like ‘Allow file contents to override Tab and Indentation settings’ (which itself is unreliable since at least 2011).
  • When changing preferences, there is more than one place to change: Edit / Preferences, Edit / Current File Preferences, and Project / Project Preferences (the last is not under the Edit menu).
  • The cursor blinks by default (which is super annoying when you’re moving the cursor around the screen) and there isn’t an explicit option to disable it.  You have to create a custom JavaScript script that executes at every file open.
  • The toolbar icons are heavily styled, making their use opaque and the tooltips mandatory reading.
  • It has already crashed while closing — which, per the above, I’m doing a lot.

It’s not all bad, there are some really nice features:

  • Vi keybindings, so things like ‘A’ to start appending to the current line, or ‘/’ to search the current file, are really nice to someone who uses vim every day.
  • I do appreciate the ability to script things
  • The syntax highlighting and coloring seems more reliable (i.e. harder to confuse) than average.
  • The installation to a local directory was painless, and an icon properly shows up in the applications menu (I use Mate).  The default installation dir is to your home directory instead of /usr/local (which is the right thing to do for trial software, imho).

I want to like this editor, I really do, but it’s just going downhill as I work with it more.  At $250 per license it’s hard to justify the expense to my boss unless I really like it.

Beautiful Code

The mark of a master programmer is someone who writes code that a novice can debug.
— attribution unknown

I read this quote, or something very similar to it, a long time time ago when I was just starting out.

I take the idea behind that quote to mean that master programmers have the experience to find the simplest solutions, which are easier to understand, but they also make their code easier to read so errors stand out.

It came back to me while reading a novice’s request for help in debugging something.  The example was a mess, with lots of extra activity, but it was also dense and poorly formatted.  The very simple bug was hard to see because of the sheer amount of code and the inconsistent formatting.

I strive to find simple solutions to the code I write, but I also strive to make my code neatly formatted and well-spaced.  I generally limit my lines to ~78 characters; I vertically align related operators; I leave space around operators.  This goes hand-in-hand with simple code: short functions that only do one thing; effective naming of things; do the least possible.  Together these generally make code that is both robust and easy to maintain.

I think of formatting to be like engineering a bridge.  Dense code is like big thick columns, steel plates, and stone architecture — it gets the job done, but it looks so heavy.  The best bridges are light and airy, full of empty space, yet they are stronger and more resilient.

PS: if you know this quote, and know who said it first, please drop me a line so that I can attribute it properly!

NiFi Build Error

I’m testing NiFi out on my local Gentoo installation to prepare for an implementation at work, and after a rather lengthy build/test process (“ten minutes” my fanny) ran into this error:

$ mvn clean install
[INFO] Scanning for projects...
...
'Script Engine' validated against 'ECMAScript' is invalid because Given value not found in allowed set 'Groovy, lua, python, ruby'

This error left me scratching my head.  Nothing related to JavaScript/ECMAScript dependencies were mentioned anywhere.  How would you get it, anyway?  Webkit, I suppose…

Sudden epiphany: this is a new Gentoo installation, and this program, including the build script, is running Java.  Gentoo doesn’t install Sun Oracle’s Java by default, but instead comes with IcedTea out of the box.  It’s acceptable for some simple uses, but is buggy for any complex. (Minecraft is a great example where it just doesn’t work.)  I haven’t used Java for anything yet, so I hadn’t installed the JDK yet.  The build instructions specify JDK 1.7 or higher, but I didn’t think anything of it because I’m used to just having it installed.

echo "dev-java/oracle-jdk-bin Oracle-BCLA-JavaSE" \
  >> /etc/portage/package.license/file
emerge -av dev-java/oracle-jdk-bin
...
$ mvn clean install
[INFO] Scanning for projects...
...
[INFO] BUILD SUCCESS

Finally!