HustleTime Development: Ignoring specific train lines (part 2)

So, I’m getting back to work on the next big feature, which is to allow the user to ignore specific train lines. I wasn’t 100% sure how to actually give the user an intuitive way to set this, but I’ve decided to build on my idea in my last post.

I’ve set the modal to have 2 ‘tabs’

My hope is that the user understands that each of the lines is a clickable object that will set that specific train to be ignored or not.

Each train line image is rendered as a <TouchableOpacity> which sets the state for each individual line to toggle a boolean. The opacity of the image rendered is influenced by the value of the bool:

I’ve clicked on the C and the F trains.

I’m pleased with this implementation so far. I still feel confused as to how to deal with the Franklin Avenue Shuttle and the Grand Central/Times Square Shuttle, but for now, I’ll leave that be.

The next steps:

  • save this information to UserDefaults so your preferences remain saved to your device
  • automatically load these preferences on open
  • include this information in the api call to the back end
  • set up the back end to be able to receive this array
  • change the redis search to disregard ignored lines
  • clean up some of the margins/padding etc.

I think this is a pretty exciting new feature and I’m sure people will use it. I personally find myself needing an F train in manhattan and when I open HustleTime and see two incoming M trains, I’m disappointed.

I’m actively working on this app, so if anyone has any suggestions for features that aren’t on my Trello Board please comment and let me know of a feature you’d like to see (and yes, I know people are hoping for buses).

You can get HustleTime on both Android and iPhone free!

HustleTime Development: Dealing with Wire Type Error (part 2)

In my last post, I talked a little about the occasional WireType error, and of course, over the past weekend, the BDFM feed was issuing corrupt data for over an hour.

I don’t have a real answer to how to deal with this sort of situation. The data I’m getting from the MTA lacks the necessary entity which contains arrivals and departures, so I don’t really have a choice but to not serve information for (in this case) the BDFM lines.

I also need to implement changes that allow for when a WireType error occurs. I started detailing this last week, but I think I finally have a temporary solution.

How I’m working this all out:

There are undoubtedly many ways to safely test code, especially when you have code in production. In my case, we’re starting to pick up a little steam:

just a little steam…

It’s just under 100 users! and that’s Android alone, on the AppStore, I’ve had about 60 installations in the last week.
At the very beginning stages of getting an app ‘out there,’ I feel it’s important not to alienate our early adopters by breaking the working code. If I was in a vacuum, I could of course shut the whole thing down and make fixes. Not right now – even with my small group of 100-200 users.

What I’m doing is spinning up a local version of my back end and seeding my own redis manually – the actual server seeds redis (which stores all the MTA data) every minute with an active crontask.

To test- I’ll simulate an api call from a device with Postman a super simple way to manually make api calls to wherever!

HustleTime is looking for a latitude and longitude among other things, so I’m sending that information to my local server with postman and checking the results.

I was asking myself how I would simulate a WireType error, and I decided to ping the MTA api, and then manually muddle up the data. Fortunately for me, When I called the MTA’s 12345 feed, I got a WireType error off the bat! Here’s what the response looks like from the MTA when a WireType error occurs:

"<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML+RDFa 1.0//EN\"\n  \"http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\" version=\"XHTML+RDFa 1.0\" dir=\"ltr\"\n  xmlns:content=\"http://purl.org/rss/1.0/modules/content/\"\n  xmlns:dc=\"http://purl.org/dc/terms/\"\n  xmlns:foaf=\"http://xmlns.com/foaf/0.1/\"\n  xmlns:og=\"http://ogp.me/ns#\"\n  xmlns:rdfs=\"http://www.w3.org/2000/01/rdf-schema#\"\n  xmlns:sioc=\"http://rdfs.org/sioc/ns#\"\n  xmlns:sioct=\"http://rdfs.org/sioc/types#\"\n  xmlns:skos=\"http://www.w3.org/2004/02/skos/core#\"\n  xmlns:xsd=\"http://www.w3.org/2001/XMLSchema#\">\n\n<head profile=\"http://www.w3.org/1999/xhtml/vocab\">\n  <meta http-equiv=\"X-UA-Compatible\" content=\"IE=EmulateIE7, IE=9/> \n  <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\n<meta name=\"Generator\" content=\"Drupal 7 (http://drupal.org)\" />\n<link rel=\"shortcut icon\" href=\"http://datamine.mta.info/misc/favicon.ico\" type=\"image/vnd.microsoft.icon\" />\n  <title>Page not found | MTA</title> \n  <link type=\"text/css\" rel=\"stylesheet\" href=\"http://datamine.mta.info/sites/all/files/css/css_xE-rWrJf-fncB6ztZfd2huxqgxu4WO-qwma6Xer30m4.css\" media=\"all\" />\n<link type=\"text/css\" rel=\"stylesheet\" href=\"http://datamine.mta.info/sites/all/files/css/css__eI-QBrkS7Fhq4ivYFedEXSif0P4eMiebr38a5RQ4bc.css\" media=\"all\" />\n<link type=\"text/css\" rel=\"stylesheet\" href=\"http://datamine.mta.info/sites/all/files/css/css_vixifsPSBw9iy7IyxjehjiO6IcmyZjymeKBz0Smvy5U.css\" media=\"all\" />\n<link type=\"text/css\" rel=\"stylesheet\" href=\"http://datamine.mta.info/sites/all/files/css/css_8myhrsfa-cTkPWW_TRh_AYx2G72aL21URIRqzgssVbo.css\" media=\"all\" />\n  <script type=\"text/javascript\" src=\"http://datamine.mta.info/sites/all/files/js/js_vDrW3Ry_4gtSYaLsh77lWhWjIC6ml2QNkcfvfP5CVFs.js\"></script>\n<script type=\"text/javascript\" src=\"http://datamine.mta.info/sites/all/files/js/js_f_6-OzlFXjayKlsU5sJKLwjz7LFQvdIZDm-Mt6jKwyM.js\"></script>\n<script type=\"text/javascript\" src=\"http://datamine.mta.info/sites/all/files/js/js_gKqQMEVMgeQjGClQWuyfgHaWcjlQLLazppG9WMNWTzg.js\"></script>\n<script type=\"text/javascript\">\n<!--//--><![CDATA[//><!--\n(function(i,s,o,g,r,a,m){i[\"GoogleAnalyticsObject\"]=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,\"script\",\"https://www.google-analytics.com/analytics.js\",\"ga\");ga(\"create\", \"UA-37680079-1\", {\"cookieDomain\":\"auto\"});ga(\"set\", \"page\", \"/404.html?page=\" + document.location.pathname + document.location.search + \"&from=\" + document.referrer);ga(\"send\", \"pageview\");\n//--><!]]>\n</script>\n<script type=\"text/javascript\">\n<!--//--><![CDATA[//><!--\njQuery(function(){\njQuery('#superfish-1').supersubs({minWidth: 12, maxWidth: 27, extraWidth: 1}).superfish({\nanimation: {opacity:'show'},\nspeed: 'fast',\nautoArrows: false,\ndropShadows: true});\n});\n//--><!]]>\n</script>\n<script type=\"text/javascript\" src=\"http://datamine.mta.info/sites/all/files/js/js_Tue3Uvub_P_YOin7KVW1e1Z1VntTQpJQiJF5B4Ec9tI.js\"></script>\n<script type=\"text/javascript\">\n<!--//--><![CDATA[//><!--\njQuery.extend(Drupal.settings, {\"basePath\":\"\\/\",\"pathPrefix\":\"\",\"ajaxPageState\":{\"theme\":\"mta\",\"theme_token\":\"m3bx1ipp_RmG3gn9MfuMPUKEvMOpaLsqpXVX7s-McVU\",\"js\":{\"misc\\/jquery.js\":1,\"misc\\/jquery.once.js\":1,\"misc\\/drupal.js\":1,\"sites\\/all\\/modules\\/google_cse\\/google_cse.js\":1,\"sites\\/all\\/libraries\\/superfish\\/jquery.hoverIntent.minified.js\":1,\"sites\\/all\\/libraries\\/superfish\\/jquery.bgiframe.min.js\":1,\"sites\\/all\\/libraries\\/superfish\\/superfish.js\":1,\"sites\\/all\\/libraries\\/superfish\\/supersubs.js\":1,\"sites\\/all\\/libraries\\/superfish\\/supposition.js\":1,\"sites\\/all\\/libraries\\/superfish\\/sftouchscreen.js\":1,\"sites\\/all\\/modules\\/google_analytics\\/googleanalytics.js\":1,\"0\":1,\"1\":1,\"sites\\/all\\/themes\\/mta\\/\\/js\\/jquery.maskedinput-1.3.min.js\":1},\"css\":{\"modules\\/system\\/system.base.css\":1,\"modules\\/system\\/system.menus.css\":1,\"modules\\/system\\/system.messages.css\":1,\"modules\\/system\\/system.theme.css\":1,\"modules\\/comment\\/comment.css\":1,\"modules\\/field\\/theme\\/field.css\":1,\"sites\\/all\\/modules\\/google_cse\\/google_cse.css\":1,\"modules\\/node\\/node.css\":1,\"modules\\/search\\/search.css\":1,\"modules\\/user\\/user.css\":1,\"sites\\/all\\/modules\\/views\\/css\\/views.css\":1,\"sites\\/all\\/modules\\/ctools\\/css\\/ctools.css\":1,\"sites\\/all\\/modules\\/panels\\/css\\/panels.css\":1,\"sites\\/all\\/libraries\\/superfish\\/css\\/superfish.css\":1,\"sites\\/all\\/libraries\\/superfish\\/css\\/superfish-vertical.css\":1,\"sites\\/all\\/libraries\\/superfish\\/css\\/superfish-navbar.css\":1,\"sites\\/all\\/libraries\\/superfish\\/style\\/default.css\":1,\"sites\\/all\\/themes\\/mta\\/base.css\":1,\"sites\\/all\\/themes\\/mta\\/grid.css\":1,\"sites\\/all\\/themes\\/mta\\/homepage.css\":1,\"sites\\/all\\/themes\\/mta\\/formalize.css\":1,\"sites\\/all\\/themes\\/mta\\/topbar.css\":1}},\"googleCSE\":{\"cx\":\"001206406127754148335:wfgv6ik0gky\",\"language\":\"\",\"resultsWidth\":600,\"domain\":\"www.google.com\",\"showWaterMark\":true},\"googleanalytics\":{\"trackOutbound\":1,\"trackMailto\":1,\"trackDownload\":1,\"trackDownloadExtensions\":\"7z|aac|arc|arj|asf|asx|avi|bin|csv|doc(x|m)?|dot(x|m)?|exe|flv|gif|gz|gzip|hqx|jar|jpe?g|js|mp(2|3|4|e?g)|mov(ie)?|msi|msp|pdf|phps|png|ppt(x|m)?|pot(x|m)?|pps(x|m)?|ppam|sld(x|m)?|thmx|qtm?|ra(m|r)?|sea|sit|tar|tgz|torrent|txt|wav|wma|wmv|wpd|xls(x|m|b)?|xlt(x|m)|xlam|xml|z|zip\"},\"urlIsAjaxTrusted\":{\"\\/files\\/k38dkwh992dk\\/gtfs\":true}});\n//--><!]]>\n</script>\n</head>\n<body class=\"html not-front not-logged-in no-sidebars page-files page-files-k38dkwh992dk page-files-k38dkwh992dk-gtfs\" >\n  <div id=\"skip-link\">\n    <a href=\"#main-content\" class=\"element-invisible element-focusable\">Skip to main content</a>\n  </div>\n    <!--[if lt IE 9]>\n\t\t\t<style>.roundCorners, #block-google-appliance-ga-related-searches {border: 1px solid #b4b4b4;-moz-border-radius: 5px;-webkit-border-radius: 5px;border-radius: 5px;background: #fff;behavior: url(/sites/all/themes/mta/border-radius.htc);}</style>\n\t\t<![endif]-->\n\n<div id=\"page\">\n\t\t<div id=\"mainbox\">\n\n\t\t\t<div id=\"topbar\">\n\t\t\t\t<div id=\"branding\">\n\t\t\t\t\t<a href=\"http://www.mta.info\"><img src=\"/sites/all/themes/mta/images/mta_info.gif\"></a>\n\t\t\t\t</div>\n\t\t\t\t<div id=\"middle-header\">\n\t\t\t\t\t\t\t\t</div>\n\t\t\t\t<div id=\"search\">\n\n\t\t\t  <div class=\"region region-header-right\">\n    <div id=\"block-search-form\" class=\"block block-search\">\n\n    \n  <div class=\"content\">\n    <form class=\"google-cse\" action=\"/files/k38dkwh992dk/gtfs\" method=\"post\" id=\"search-block-form\" accept-charset=\"UTF-8\"><div><div class=\"container-inline\">\n      <h2 class=\"element-invisible\">Search form</h2>\n    <div class=\"form-item form-type-textfield form-item-search-block-form\">\n  <label class=\"element-invisible\" for=\"edit-search-block-form--2\">Search </label>\n <input title=\"Enter the terms you wish to search for.\" type=\"text\" id=\"edit-search-block-form--2\" name=\"search_block_form\" value=\"\" size=\"15\" maxlength=\"128\" class=\"form-text\" />\n</div>\n<div class=\"form-actions form-wrapper\" id=\"edit-actions\"><input type=\"submit\" id=\"edit-submit\" name=\"op\" value=\"Search\" class=\"form-submit\" /></div><input type=\"hidden\" name=\"form_build_id\" value=\"form-DrCvzt-cKOStdN5ZIl3ZC47qdbvh7_q7C3gePjs-p-0\" />\n<input type=\"hidden\" name=\"form_id\" value=\"search_block_form\" />\n</div>\n</div></form>  </div>\n</div>\n  </div>\n\n\t\t\t\t</div>\n\t\t\t </div>\n\t\t\t<div id=\"navbar\">  <div class=\"region region-navbar\">\n    <div id=\"block-superfish-1\" class=\"block block-superfish\">\n\n    \n  <div class=\"content\">\n    <ul id=\"superfish-1\" class=\"sf-menu main-menu sf-horizontal sf-style-default sf-total-items-8 sf-parent-items-2 sf-single-items-6\"><li id=\"menu-218-1\" class=\"first odd sf-item-1 sf-depth-1 sf-total-children-6 sf-parent-children-0 sf-single-children-6 menuparent\"><a href=\"http://www.mta.info/\" title=\"\" class=\"sf-depth-1  menuparent\">Home</a><ul><li id=\"menu-592-1\" class=\"first odd sf-item-1 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/\" title=\"\" class=\"sf-depth-2 \">MTA Home</a></li><li id=\"menu-593-1\" class=\"middle even sf-item-2 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/nyct\" title=\"\" class=\"sf-depth-2 \">NYC Subways and Buses</a></li><li id=\"menu-594-1\" class=\"middle odd sf-item-3 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/lirr\" title=\"\" class=\"sf-depth-2 \">Long Island Rail Road</a></li><li id=\"menu-595-1\" class=\"middle even sf-item-4 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/mnr\" title=\"\" class=\"sf-depth-2 \">Metro-North Railroad</a></li><li id=\"menu-596-1\" class=\"middle odd sf-item-5 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/bandt\" title=\"\" class=\"sf-depth-2 \">Bridges and Tunnels</a></li><li id=\"menu-597-1\" class=\"last even sf-item-6 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/capital\" title=\"\" class=\"sf-depth-2 \">MTA Capital Program</a></li></ul></li><li id=\"menu-598-1\" class=\"middle even sf-item-2 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/schedules\" title=\"\" class=\"sf-depth-1 \">Schedules</a></li><li id=\"menu-599-1\" class=\"middle odd sf-item-3 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/fares\" title=\"\" class=\"sf-depth-1 \">Fares & Tolls</a></li><li id=\"menu-600-1\" class=\"middle even sf-item-4 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/maps\" title=\"\" class=\"sf-depth-1 \">Maps</a></li><li id=\"menu-601-1\" class=\"middle odd sf-item-5 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/service\" title=\"\" class=\"sf-depth-1 \">Planned Service Changes</a></li><li id=\"menu-602-1\" class=\"middle even sf-item-6 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/about\" title=\"\" class=\"sf-depth-1 \">MTA Info</a></li><li id=\"menu-603-1\" class=\"middle odd sf-item-7 sf-depth-1 sf-no-children\"><a href=\"http://web.mta.info/business\" title=\"\" class=\"sf-depth-1 \">Doing Business With Us</a></li><li id=\"menu-604-1\" class=\"last even sf-item-8 sf-depth-1 sf-total-children-10 sf-parent-children-0 sf-single-children-10 menuparent\"><a href=\"http://web.mta.info/accountability/\" title=\"\" class=\"sf-depth-1  menuparent\">Transparency</a><ul><li id=\"menu-605-1\" class=\"first odd sf-item-1 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/accountability\" title=\"\" class=\"sf-depth-2 \">Main Page</a></li><li id=\"menu-606-1\" class=\"middle even sf-item-2 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/mta/boardmaterials.html\" title=\"\" class=\"sf-depth-2 \">Board Materials</a></li><li id=\"menu-607-1\" class=\"middle odd sf-item-3 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/mta/budget/\" title=\"\" class=\"sf-depth-2 \">Budget Info</a></li><li id=\"menu-608-1\" class=\"middle even sf-item-4 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/capital\" title=\"\" class=\"sf-depth-2 \">Capital Program Info</a></li><li id=\"menu-609-1\" class=\"middle odd sf-item-5 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/capitaldashboard/CPDHome.html\" title=\"\" class=\"sf-depth-2 \">Capital Program Dashboard</a></li><li id=\"menu-4481-1\" class=\"middle even sf-item-6 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/mta/investor/\" title=\"\" class=\"sf-depth-2 \">Investor Information</a></li><li id=\"menu-610-1\" class=\"middle odd sf-item-7 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/mta/leadership/\" title=\"\" class=\"sf-depth-2 \">MTA Leadership</a></li><li id=\"menu-611-1\" class=\"middle even sf-item-8 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/persdashboard/performance14.html\" title=\"\" class=\"sf-depth-2 \">Performance Indicators</a></li><li id=\"menu-612-1\" class=\"middle odd sf-item-9 sf-depth-2 sf-no-children\"><a href=\"http://www.mta.info/mta-news\" title=\"\" class=\"sf-depth-2 \">Press Releases and News</a></li><li id=\"menu-613-1\" class=\"last even sf-item-10 sf-depth-2 sf-no-children\"><a href=\"http://web.mta.info/mta/news/hearings\" title=\"\" class=\"sf-depth-2 \">Public Hearings</a></li></ul></li></ul>  </div>\n</div>\n<div id=\"block-block-5\" class=\"block block-block\">\n\n    \n  <div class=\"content\">\n    <p style=\"text-align:right;\"><span class=\"text\">Already registered?</span><a href=\"/user\">Log In</a></p>\n  </div>\n</div>\n  </div>\n</div>\n\t\t\t<div id=\"breadcrumb\"> <!-- we leave the div and keep spacing -->\n\t\t\t\t\t\t\t\t\t\t\t</div>\n      <div class=\"messages-wrapper\"></div>\n\t\t\t<div id=\"contentbox\" class=\"roundCorners clearfix\">\n\t\t\t\t\t\t\t<div class=\"container\">\n\t\t\t\t\t\t\t\t<div class=\"span-90\">\n\t\t\t \t\t\t\t <div id=\"pageTitleArea\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h1 class=\"title\" id=\"page-title\">Page not found</h1>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t</div>\n\t\t\t\t\t\t\t\t\t\n\n\n\t\t\t\t\t\t\t\t<div id=\"main-message\" class=\"span-43 full-main\">\n\n\t\t\t\t\t\t\t\t\t<a id=\"main-content\"></a>\n\n\n\t\t\t\t\t\t\t\t\t<div class=\"tabs\"></div>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t  <div class=\"region region-content\">\n    <div id=\"block-system-main\" class=\"block block-system\">\n\n    \n  <div class=\"content\">\n    The requested page \"/files/k38dkwh992dk/gtfs\" could not be found.  </div>\n</div>\n  </div>\n\t\t\t\t\t\t\t\t</div><!-- close span-43 -->\n\n\n\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\n\n\n\t\t\t\t\t\t\t\t<!-- close span-21 last -->\n\t\t\t\t\t\t</div><!-- close container for grid -->\n\t\t\t</div>\n\n\t</div>\n\t\t  <div class=\"region region-footer\">\n    <div id=\"block-block-6\" class=\"block block-block\">\n\n    \n  <div class=\"content\">\n    <div id=\"translate\">\n<div id=\"google_translate_element\">\n<div id=\"temp_GT\">\n<ul class=\"social-links\" style=\"margin: 0;\"><li class=\"social-google\"><a href=\"#translate\">Google Translate</a></li>\n</ul></div>\n</div>\n<!-- close google_translate_element --></div>\n<!-- close translate --><script>\n<!--//--><![CDATA[// ><!--\n\nfunction googleTranslateElementInit() {\n  new google.translate.TranslateElement({\n  pageLanguage: 'en',\n  layout: google.translate.TranslateElement.InlineLayout.SIMPLE\n  }, 'google_translate_element');\n}\n\njQuery(\"#temp_GT\").click(function() {\n  var fileref = document.createElement('script');\n    fileref.setAttribute(\"type\", \"text/javascript\");\n    fileref.setAttribute(\"src\", \"http://translate.google.com/translate_a/element.js?cb=googleTranslateElementInit\");\n\n    document.getElementsByTagName(\"head\")[0].appendChild(fileref);\n  jQuery(\"#temp_GT\").hide();\n});\n\n//--><!]]>\n</script>  </div>\n</div>\n  </div>\n</div>\n  </body>\n</html>\n"

When I try to decode this message with Protobuf, it throws the WireType error that I’m so concerned about – so the first step will be to enact some error handling with WireType in mind.

For any given feed: the first beginnings of this look like this:

errors = []

.
.
.

begin
  feed123456S = Transit_realtime::FeedMessage.decode(data123456S)
rescue Protobuf::InvalidWireType
  errors = errors + 'skipped: 123456S (wireType) '
  # Do something
end

once that decoding is done, I need to run my getArrivals(lineFeed, timeNow, data_array) method on each feed, and if there’s that WireType error, ruby will throw yet another error. So I need to either add a conditional in that method, or add some more error handling. I’m going to add some more error handling:

begin
  getArrivals(feed123456S, time_now, data)
rescue
  puts "did not seed 123"
  # Do something
end

At this point I have the structure to handle the errors I’m seeing. Now I just actually need to do something about it. My DataHelper.rb file has some class methods including a retrieve_data function as such:

	def self.retrieve_data
		if @@data.length == 0
			@@data = JSON.parse(@@redis.get(@@redis.get(@@data_key)))
		end
			@@data = JSON.parse(@@redis.get(@@redis.get(@@data_key)))
	end

This basically grabs the data in redis and parses it. Since I’ve required DataHelper.rb into my CronHelper.rb file, I simply have to call the function and then sift through that data and grab all the data that pertains to the line that’s having the WireType error. It looks like this:

station_arrivals = []
skipped = []

begin
  feedBDFM = Transit_realtime::FeedMessage.decode(dataBDFM)
rescue Protobuf::InvalidWireType
  errors += ' skipped: BDFM (wireType)'
  old_data = DataHelper.retrieve_data
  skipped += old_data.find_all{ |dep| dep['train'] == 'B'}
  skipped += old_data.find_all{ |dep| dep['train'] == 'D'}
  skipped += old_data.find_all{ |dep| dep['train'] == 'F'}
  skipped += old_data.find_all{ |dep| dep['train'] == 'M'}
  skipped.each{ |entry| station_arrivals.push(entry)}
end

station_arrivals is the array that stores all the MTA and is what’s pushed into redis, so I’m basically calling all the old data, finding all the relevant arrivals and pushing it into station_arrivals to send back and save to redis.

Looking forward to further testing this and deploying to production in the next day or two!

HustleTime Development: Dealing with Wire Type Error

Occasionally the MTA sends corrupt or incomplete data.
It’s particularly frustrating for me because of the structure of how I’ve built HustleTime.

The MTA sends transit data through multiple GTFS feeds, the subway system (and SIRR) are covered by 9 individual feeds. My code looks a little like this:

require 'protobuf'
require 'google/transit/gtfs-realtime.pb'
require 'net/http'
require 'uri'

.
.
.

data123456S = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=1"))
dataACEHS = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=26"))
dataNQRW = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=16"))
dataBDFM = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=21"))
dataL = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=2"))
dataSIRR = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=11"))
dataG = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=31"))
dataJZ = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=36"))
data7 = Net::HTTP.get(URI.parse("http://datamine.mta.info/mta_esi.php?key=[KEY]&feed_id=51"))

As you can see, I’m touching each of these endpoints in one file, and assigning each feed to a variable which is then parsed by protobuf:

begin
  feed123456S = Transit_realtime::FeedMessage.decode(data123456S)
rescue Protobuf::InvalidWireType
  errors = errors + 'skipped: 123456S (wireType) '
end

What’s happening here is that a few times a week a feed starts sending corrupt data, and returns an InvalidWireType error. I’m catching this error and storing the error in an errors variable.
The problem with this method is that when all the various feeds are agglomerated, if one feed received an error, that entire feed will be omitted from the station_arrivals object that is being sent to redis.

/CronHelper.rb

.
.
.

DataHelper.save_data(station_arrivals)
/DataHelper.rb

class DataHelper

.
.
.

def self.save_data(array)
	new_key = DateTime.now
	old_key = @@redis.get(@@data_key)
	@@redis.set(new_key, array.to_json)
	@@data = array
	@@redis.set(@@data_key, new_key)
	@@redis.del(old_key)
end

This all means that if there’s an error, all the old data is rewritten in Redis without the offending line at all- and this isn’t a great way to approach this since there is actually data that I could use.

Imagine HustleTime is seeding normally. The MTA sends about 1 hour of future arrivals when you hit their api. So if I am pinging the BDFM feed, I’ll see predictions for then next hour of arrivals at all stations along the Orange line.
When you encounter a WireType error, all that old data would be rewritten, so I really need a way to go into the old data and grab those numbers, and splice them into the new data when an error is caught.

Remember, this only happens on occasion, but my good friend BanjoCat was good enough to set up some monitoring for me so I can see when it’s happening, and to be honest, it’s happening more than I would like. So I’ve shelved Ignoring trains to work on this issue next.
I’m developing a few methods now that do what I need and I’m currently testing. Hopefully I can avoid WireType blackouts in the future.

You can get HustleTime on both Android and iPhone free!

HustleTime Development: Ignoring specific train lines (part 1)

The next big feature I’ve been meaning to add to HustleTime is a way for users to ignore specific train lines.

Let’s say you’re on lower level at West 4th street station in manhattan, and you need to take the to Brighton Beach but all you’re seeing in the HustleTime Ui is other trains that are of no use to you. This feature will allow you to ignore trains so that you’ll only see the train you need.

How can I accomplish this?

At it’s most basic level the Front end will need to send something to the back end and the back end will need to send the requested information sans the ignored stations. It will also be necessary to save the ignored lines into both state and the phone memory, so that a user will not need to select ignored trains every time they open the app. We also need to make sure the back-end is backwards compatible with prior versions without the ignore feature. Finally, we need a way for the user to actually select the stations they hope to ignore.

How to get started is always tricky for me: I never know where to start. As HustleTime gets more complex, it will become more challenging to decide how to begin modifying all the various moving parts – and I’m sure as I become more experienced as a developer, this process will become easier and easier.

Real Estate

The screen on a mobile device has very limited real-estate. I have a modal built into HustleTime already, and that shows you your saved favorite places.

Where can I add ignored lines??

With the limited amount of space, I think it will probably be best to display the ignore options in a separate tab of the menu. Something like this…

There, I’ll put checkboxes for the various lines, each checkbox will toggle a boolean value in state for each of these lines.
I’ll then JSON.stringify that object when I save it to phone memory or send it to the back-end. In tomorrow’s post I’ll have something to show in regards to my progress on this feature.

HustleTime Development: Intro

As I push forward with new iterations of HustleTime, I wanted to create a blog to detail my progress, perhaps it will foster creativity, engagement with my small (but growing) audience, and / or simply serve to document my progress.

I am only a few days removed from my Android release, a painstakingly difficult process that led me down many dead ends, and eventually rebuilding the app in a completely different code repo.

The draw and selling points of React-Native eventually ended up failing me. Now that I’ve created the app, I can say that there are still a lot of advantages to building in React-Native, but it’s never as simple as it seems.

An example is in NSUserDefaults (Ios) and DefaultPreferences (android):

IOS

import UserDefaults from 'react-native-userdefaults-ios'

.
.

getSavedCenterPoints = () => {
  UserDefaults.arrayForKey('savedCenterPoints')
  .then(array => {
    array != null ? this.setState({savedCenterPoints: array}) : null        
  })
}

Android

import DefaultPreference from 'react-native-default-preference'

.
.

getSavedCenterPoints(){
  DefaultPreference.get('savedCenterPoints')
  .then(string => JSON.parse(string))
  .then(array => {
    array != null ? this.setState({savedCenterPoints: array}) : null        
  })
}

As you can see, the code is very similar, but one library can take an array, and the other forces me to use a string. It’s little things like this, coupled with various dependency issues – particularly how gradle dependency versioning works that really slowed me down.

I hope to blog more about how I improve the product in the future as well as a write a retrospective on my journey in future blog posts.

Thanks for reading!