{"id":1344,"date":"2016-06-03T13:04:55","date_gmt":"2016-06-03T17:04:55","guid":{"rendered":"http:\/\/blog.jonesling.us\/?p=1344"},"modified":"2016-06-03T13:04:55","modified_gmt":"2016-06-03T17:04:55","slug":"error-failed-to-process-in-nifi","status":"publish","type":"post","link":"https:\/\/blog.jonesling.us\/?p=1344","title":{"rendered":"“ERROR: … failed to process… ” in NiFi"},"content":{"rendered":"

I was greeted by a few cryptic things in NiFi this morning during my morning check-in.<\/p>\n

    \n
  1. A PutSQL processor was reporting an error:\n
    \"ERROR: PutSQL[id=$UUID>]failed to process due to java.lang.IndexOutOfBoundsException: Index: 1, Size: 1; rolling back session: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1\"<\/pre>\n<\/li>\n
  2. There were no recent errors counted in the LogAttribute counter we set up to record errors;<\/li>\n
  3. The Tasks\/Time count in the PutSQL processor was though the roof, despite the errors and lack of successes.<\/li>\n<\/ol>\n
    Needless to say, the processor was all bound up and a number of tasks were queued.\u00a0 Not a good start to my day.<\/p>\n
    I checked the data provenance and didn’t see anything remarkable about the backed-up data.\u00a0 The error message suggests (to me) that the first statement parameter is at fault, and that parameter happened to be a date (which has been problematic for me in NiFi with a MySQL backend).\u00a0 Neither that value, nor the rest of the values, were remarkable or illegal for the fields they’re going into.<\/p>\n
    It wasn’t until I spent some time looking over the source data that I saw the problem: there is a duplicate key in the data.\u00a0 This error is NiFi’s way of complaining about it.<\/p>\n
    In our case the underlying table doesn’t have good keys, or a good structure in general, and I’m planning to replace it soon anyway, but updating the primary keys to allow the duplicate data (because it IS valid data, despite the table design) has solved the issue.<\/p>\n","protected":false},"excerpt":{"rendered":"
    I was greeted by a few cryptic things in NiFi this morning during my morning check-in. A PutSQL processor was reporting an error: “ERROR: PutSQL[id=$UUID>]failed to process due to java.lang.IndexOutOfBoundsException: Index: 1, Size: 1; rolling back session: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1” There were no recent errors counted in the LogAttribute counter we set up … Continue reading ““ERROR: … failed to process… ” in NiFi”<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","wprm-recipe-roundup-name":"","wprm-recipe-roundup-description":"","advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9],"tags":[277,267],"class_list":["post-1344","post","type-post","status-publish","format-standard","hentry","category-linux","tag-errors","tag-nifi"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4o3FW-lG","jetpack-related-posts":[],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/posts\/1344","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1344"}],"version-history":[{"count":2,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/posts\/1344\/revisions"}],"predecessor-version":[{"id":1347,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=\/wp\/v2\/posts\/1344\/revisions\/1347"}],"wp:attachment":[{"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.jonesling.us\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}