MTA Technology

I’m on vacation, currently in Marrakech after a week in France, so that’s why posting has gone silent. I’ll be back in the States later this week and will attempt to catch up on transit news. There’s been a lot happening lately, all worth some of our attention.

In the meantime, the MTA is finally set to unveil a new website on Monday morning along with a new app that’s supposed to unify and streamline the agency’s various online offerings. I first caught wind of this effort a few years ago and saw an early prototype of the app a few months ago. The bones for something useful were in place then, but the MTA bureaucracy has a way of stymying the best of technological intentions. The proof will be in the pudding come tomorrow.

In the meantime, read James Barron’s preview of the new app and a behind-the-scenes look at the MTA’s social media team in this Times article. It’s worth your time. I’ll have an in-depth look at the new site when in return, but feel free to share your thoughts in the comments to this post once the new site and MYmta app are public come late Monday morning.

The B Division countdown clocks will be based on the current pilot underway in Manhattan.

No more promises of “three to five years.” After over half a decade of promises, the MTA Board is set to vote Wednesday on a plan that would finally bring countdown clocks to the B Division trains — the subway’s lettered lines — by the end of March of 2018. Based off the current pilot running at eight stations along the BMT Broadway line, the new system will be run by Transit Wireless infrastructure and will be a part of an initiative to bring wifi to the system’s outdoor stations. It’s not based on the same signal system upgrades as the A Division countdown clocks but should cost under $32 million to install.

Following Gov. Andrew Cuomo’s push to speed up the installation of the underground wireless and cell network so that all underground stations are wired by the end of this year, Transit Wireless has the capacity to implement additional technological upgrades, and the MTA and Transit Wireless are set to expand wifi capabilities at the 109 aboveground stations. The cellular carriers are on board — especially for popular stations where their networks can be overloaded — because of the ability to colocate cellular equipment within Transit Wireless base stations, and Transit Wireless is on board because this gives them an opportunity to expand their footprint in the subway system.

But the wifi is just a benefit. The main attraction are countdown clocks using commercially available off-the-shelf components that won’t run the MTA a bill in the nine-figure range. The technology will utilize Bluetooth, sensors, beacons and wifi to determine train arrivals times. Each of the B division stations with two LCD screens per platform and one outside of fare control. The data will run from the beacons to a cloud-based system that will determine arrival times, and all of the data will be available in the MTA’s Subwaytime app.

One way or another, Transit Wireless will bring these countdown clocks online, and if the MTA chooses to expand Transit Wireless’ wifi capabilities to the above-ground stations, the agency will save money on installation. The total installation costs for wifi and the countdown clock technology will be a little over $211,000 per station for the aboveground stations and around $54,000 per underground station (which are already wired for certain Transit Wireless capabilities). The total capital costs would run around $31.7 million with $5 million in annual operating fees, subject to a CPI multiplier each year.

The low cost is an extension of the wifi expansion plan I mentioned. If the MTA and Transit Wireless don’t agree on a wifi franchise license for the aboveground stations, the train arrival boards will move ahead but at a cost of about 50% more. Eventually, the wifi capabilities are going to be necessary for the MTA’s new fare payment system, but more on that soon. Meanwhile, the agency also let slip that it is in discussions with Transit Wireless for a plan to wire the tunnels as well, but let’s not get too far ahead of ourselves.

This is a pretty fast turnaround for a project we haven’t really been expecting. In spite of a funding request in the current capital plan, the MTA has given no indication that a systemwide plan was in the works. The BMT pilot came through via pressure from the governor’s office, and an ambitious 16-month rollout seems set to follow. It may not be the perfect system — but it’s far better than the current non-existent system. And after years of hearing that B Division countdown clocks are still three to five years away, we can say with some certainty that, if all goes according to plan, B Division countdown clocks are now just 16 months down the tracks.

The MTA launched a 90-day pilot bringing these countdown clocks to eight stations in Manhattan.

Among the transit cognoscenti, the promise of countdown clocks on the B Division has always seemed frustratingly out of reach. While the A division — the numbered lines — received countdown clocks as part of a signal upgrade a few years ago, the upgrades for the lettered lines are still decades away. Meanwhile, every few months, the MTA would promise some sort of interim solution in “three to five years.” Well, three to five years may finally have arrived as the MTA and Governor Cuomo announced a Bluetooth-based, eight-station 90-day pilot program for B Division countdown clocks.

The pilot — along the BMT Broadway line (N, Q, and R trains in Manhattan) — will feature to-the-minute countdown clocks with similar information as the ones on the A Division share but a different design. The information will be delivered from data transmitted by Bluetooth receivers on the trains to those on the platform and then fed into the digital displays. The countdown timers won’t be based off of data received while trains are in between stations so precise train location will still be an unknown, but the data should be reliable enough that passengers won’t know the difference.

Here’s how the press release described the technology:

The new clocks rely on technology that is straightforward, cost effective to deploy, and does not require large infrastructure. The system uses the existing wireless network in the stations and cloud computing, and involves four Bluetooth receivers placed in each station, two at each end of the platform. These receivers communicate with four Bluetooth devices that have been installed in the first and last cars of each train set running on the line. As the train enters and leaves a station, the system uses its arrival and departure time to estimate the time at which the train will reach the next stop in the line, and display the arrival times on the two LCD display screens that have been installed at each station.

The new displays, as you can see from the photo above, will feature the countdown timers but can also show PSAs and other contents (such as ads) concurrently, solving a major design flaw inherent in the current two-line displays. Now, when the MTA wants to issue a not-so-important message from the NYPD, it can do so on the portion of the screen that doesn’t include the countdown information.

“These actions,” Cuomo said in his press release today, “are the latest steps toward rebuilding and transforming the MTA into a unified, state-of-the-art transportation network that will meet the needs of current and future generations of New Yorkers. With this new and updated technology, we’ll help ensure riders have the information they need to get where they need to go.”

As part of the 90-day test, the system will be in use at the N/Q/R stations at 23rd Street; 28th Street; 34th Street; 42nd Street; 49th Street; 57th Street; 5th Avenue/59th Street; and Lexington Avenue/59th Street. During the evaluation period, the MTA says it will “identify and correct any issues with the new system. The goal is to evaluate the accuracy of location data, performance of Transit Wireless infrastructure, performance of the LCD displays, physical and network security of Bluetooth devices, security of data being transmitted, and internal access and use of data being generated.”

The governor says these clocks will ultimately be installed at all 269 stops along the lettered lines, but it’s not clear on what timeline these could be rolled out or at what cost. It is markedly cheaper than the CBTC upgrades, but unlike the CBTC upgrades, Bluetooth-based countdown clocks don’t increase service. They are a customer satisfaction measure through and through, one that both is welcomed and shouldn’t have taken so long to realize. But with Cuomo’s push to roll out Transit Wireless at all underground stations by the end of the year, this style of countdown clock became feasible. It is not yet clear how these could be deployed in stations that are above ground.

Still, the MTA appears committed to this way forward, and although I don’t always agree with the Governor’s transit priorities, he deserves praise for finally getting the MTA to move forward with technology projects that had been stalled for years. “Governor Cuomo challenged the MTA to develop an aggressive approach to putting countdown clocks on the lettered lines, and our technology team’s response has been phenomenal,” MTA Chairman and CEO Thomas F. Prendergast said in a statement. “In very short order they developed an easy to deploy, cost-effective system that we think will play a central role in bringing this essential service to more and more of our customers. We look forward to learning from this test, as well as to developing a roll out plan based on our findings.”

Gov. Cuomo unveiled a new MTA ticketing app to much fanfare earlier this week.

By many accounts Gov. Andrew Cuomo, like many politicians, thinks he’s the smartest person in the room. No matter the field, whether its in his wheelhouse of expertise (whatever that may be) or not, if he gets an idea in his head, the people who report to him have to execute. It’s not a dialogue between career public servants who know what they’re doing and the state’s chief executive; it’s a command from a boss to those who report in to him.

With that background, it’s easy to see how Cuomo’s recent transit steps, while earning him headlines and being a major part of his public relations push around the state, have been missteps in the eyes of the transit community. It’s why MTA staffers groan when they hear the governor is coming and just wish he would stay out of the agency’s hair. It’s why buses have USB ports, and it’s why these USB ports are trumpeted as some sort of savior at a time when New York City’s transit system is bursting at its seams with no relief in sight.

Earlier this week, Cuomo again seemed to think that a basic technological upgrade that’s standard operating procedure for most commuter rail systems the world over is a sign of strong leadership when he held a circus of a press conference to announce the long-delayed rollout of a mobile ticketing app for Metro-North and Long Island Rail Road. The app is powered by Masabi’s JustRide platform, and the details are rather mundane. The app will support a variety of time- or ride-based tickets and will require riders to “activate” tickets before use, as New Jersey Transit’s app does. It is Apple Pay-compatible and can support purchasing a ticket, according to promotional materials, in under ten seconds. It’s available now on the LIRR Port Washington Branch and the Metro-North Hudson Line, and it will be rolled out throughout the system by the end of August.

Cuomo’s words in a statement underscore his beliefs that technology is a transit savior. “Our administration remains committed to building a transit network that is responsive to the needs of travelers who rely on mass transportation, and the launch of the MTA eTix app is a major step forward in our efforts,” the governor said. “This new app puts riders first by eliminating the ticket line and helping New Yorkers and visitors get where they need to go with more freedom and convenience than ever before. We will continue to create a 21st century transit system that embraces innovation to ensure that we are building a stronger, more competitive New York.”

The most interesting part of the announcement came as part of the kicker paragraph in the press release which noted that “next year, the MTA expects to make the MTA eTix app even more user-friendly by allowing LIRR and Metro-North customers who transfer to or from the New York City Subway or New York City Buses to pay their fares using a single app and a single transit account.” As the Metrocard replacement project is still in the RFP stage, it’s not quite clear how this functionality will be available by mid-2017, and I’ll follow up on this aspect of the story.

Now, don’t get me wrong; mobile ticketing is a big step. But is it worth the brouhaha of a press release and official back-patting? The MTA is years late to game on mobile ticketing thanks to a combination of recalcitrant LIRR unions and an institutional aversion to major technological change. This announcement, and the fact that the governor had to prod the MTA to release this app, should have been slipped into a press release on a summer Friday, not trumpeted on the day back to work after the July 4th holiday.

But that’s not the weird part. The weird part was Cuomo’s rant at the press conference which you can find in this 20 minute video of the presser.

At around the eight-minute mark, Cuomo starts going off the rails on the current state of the transit system. “You can’t make a system that was designed and constructed for that [early 1900s] scale work for the current-day scale, and government’s tried in fits and starts to use bailing wire and duct tape and bubble gum and all sorts of ways to make it work. It’s not going to work. You have to build a new system,” he said. He also claimed that “our society is more litigious than ever” and thus government cannot make the changes to create a new system. In a concurrent tweet, Cuomo confused daily subway and bus ridership with monthly LIRR ridership and seemed to over-simplify when the subway system as we currently know it opened.

It’s hard to parse Cuomo’s words here. He seems to think that basic technological upgrades are a sign of a “new system.” He seems to think mandating buses that are “Ferrari-like” and faster rollouts of mobile ticketing apps will help with the crowds on Lexington Ave. or the lack of capacity in Queens. He doesn’t want to hear about it or take on the root causes of the MTA’s problems: high costs, an inability to deliver projects on time and on budget, and a lack of institutional support from the governor.

I don’t know where anyone involved goes from here. It’s not an intractable situation for transit advocates or the MTA or the governor, but it’s a situation careening from one band aid to another when real leadership is required. It’s not the collective fault of the millions of riders that the MTA can’t handle the crowds, and a magic app for Metro-North and LIRR riders won’t fix that problem. Unfortunately, Cuomo, the one man who thinks he can solve these problems, is blind to this reality, and we’ll have to live through it instead.

A few weeks ago, I first linked to The Atlantic’s lengthy piece on New York City Transit’s technological woes. At the time, I examined the trial and tribulations of bringing communications-based train control online and highlighted how the MTA’s current approach is both impossible to sustain and inefficient in its execution. It is the classic story of a large and conservative bureaucracy unable to adapt to technological change, let alone a fast pace of adaption.

Let’s dive back into the piece and explore the countdown clock conundrum. As you may recall, James Somers initially set out to write about why only the A Division subway lines — the numbered routes — have countdown clocks while the B Division trains — the lettered lines — do not and will not for the foreseeable future. He also wants to understand why the A Division countdown clocks arrived years late. It is, he writes, “the story of a large organization’s first encounter with a large software project.” As you can imagine, it hasn’t gone particularly well for the MTA.

First, Somers notes that Automatic Train Supervision, the project that allowed the MTA to introduce countdown clocks on the A Division, is a subset of CBTC, and had the agency better coordinated and understood technology, they wouldn’t have spent 14 years installing an interim solution. The story goes south from there:

A post-mortem by the Federal Highway Administration details how from the start, an agency which had had little experience with large “systems” projects tried to wing it. For instance, the consulting firm tasked with developing the project plan never made a list of requirements, didn’t talk to the workers who would be maintaining the system until after it was designed, and left vague instructions for large chunks of work—specifying, for instance, “similar functionality to what is currently available”—that later became the focus of drawn-out contract disputes.

The MTA thought that they could buy a software solution more or less off the shelf, when in fact the city’s vast signaling system demanded careful dissection and reams of custom code. But the two sides didn’t work together. The MTA thought the contractor should have the technical expertise to figure it out on their own. They didn’t. The contractor’s signal engineer gave their software developers a one-size-fits-all description of New York’s interlockings, and the software they wrote on the basis of that description—lacking, as it did, essential details about each interlocking—didn’t work.

Gaffes like this weren’t caught early in part because the MTA “remained unconvinced of the usefulness of what seemed to them an endless review process in the early requirements and design stages. They had the perception that this activity was holding up their job.” They avoided visiting the contractor’s office, which, to make things worse, was overseas. In all, they made one trip. “MTA did not feel it was necessary to closely monitor and audit the contractor’s software-development progress.”

The list goes on: Software prototypes were reviewed exclusively in PowerPoint, leading to interfaces that were hard to use. Instead of bringing on outside experts to oversee construction, the MTA tried to use its own people, who didn’t know how to work with the new equipment. Testing schedules kept falling apart, causing delays. The training documentation provided by the contractor was so vague as to be unusable.

The MTA’s attempts at bringing the ATS system to the larger B Division faltered three times during the first decade of the 21st Century, and instead of trying to speed up the pace of installation of the CBTC system which would, as an ancillary benefit, introduce countdown clocks systemwide, Transit is again looking at a piecemeal solution. Again, it’s not working.

As we now know, the MTA doesn’t anticipate completed the installation of the Integrated Service Information and Management system on the B Division before 2020. An original deadline of 2017 was deemed unrealistic, and the delay in capital funding pushed this project back to the next five-year plan. And here’s the rub:

The problem is that the project has slowly taken on a bigger and bigger scope. The minutes of a 2012 Capital Program Oversight Committee meeting reveal that initially, the project’s focus “was to provide Train Arrival Information in stations.” Several service incidents, including a winter storm, drove the MTA to “re-focus project priority to provide centralized service-monitoring and information… followed closely by customer information.”

It is growing to look more and more like ATS. A request for proposal as recent as six months ago—back when funding looked more secure—called for a 77-month software contract to build out a sophisticated Rail Traffic Management System as part of ISIM-B. That piece of the project is envisioned as a complex centralized “expert system” that would allow operators to quickly diagnose service problems and would intelligently suggest ways to work around the disruption. It is, in a word, ambitious. And ambition is the death knell for big software projects. It’s what made ATS such a quagmire in the first place. It is, one suspects, why funding for countdown clocks has been cut from the latest capital plan: The rest of ISIM-B costs too much. It costs too much because it is trying to do too much. The consequence being that for five or six years, customers will hardly see anything get done at all.

At this point, Somers comes up short on a solution. He properly cites to BusTime, the MTA’s greatest software success story, as an example that the agency has had a few people at various times with the ability and trust to do something in-house. But those involved in the original creation and implementation of BusTime have long left the MTA, and Jay Walder, the CEO who was willing to give BusTime a shot, was forced out over his apparent lack of political savviness in dealing with both Albany and the TWU.

So what comes next? Certainly not countdown clocks on the B Division trains any time soon, and certainly not much faith that the MTA can execute complex technology upgrades in a timely or efficient manner. The MetroCard replace is on tap and could suffer from the same fate. Meanwhile, everything is years late and millions of budget. When it’s going to take the better part of a century to bring CBTC to the entire subway line, in the end, do we have any real hope that, without a top-to-bottom organizational overhaul, the MTA can execute on projects that are standard throughout the world? I’m not sure anyone really likes the answer to that question.

Despite journalistic claims of objectivity, some of the best reporting happens when a writer pursues something personal. In this instance, James Somers wanted to know why his F train stop at Carroll St. didn’t have countdown clocks, and what he undercovered made for a massive piece in The Altantic on the dreadful state of the MTA’s technology and its efforts at modernizing. As you may imagine, what Somers found is an agency beset by institutional paralysis, on the one hand, and a fear of taking any risks, on the other.

What Somers uncovered is an open secret amongst the transit literati. The MTA admits it to those who ask, but it’s rarely publicized. The truth is that the F train — and all B division trains — do not have countdown clocks because the MTA doesn’t know where the trains are. The fixed-block signal system doesn’t allow for MTA operators or individual towers to identify which trains are where, and those non-stop signal problems we hear about can, the MTA says, be caused by something as innocuous as debris on the tracks.

I’d like to spend more time discussing some of the issues Somers’ piece raised, but for today, let’s delve into one section — the tale of bringing CBTC to the L train:

[The RPA’s Richard Barone] explained that the Canarsie pilot suffered from problems that weren’t unusual for big transit projects in New York. The first was outmoded work rules. CBTC is designed so that trains can run themselves. But the L still has two-person crews on board every train. They’re not very busy: An April 2007 article titled “Look, Ma—no hands!” in the trade magazine Railway Age featured a delighted train supervisor named Lance Parrish riding in a CBTC-equipped train on the Canarsie Line. “All Parrish has to do is scan the onboard displays and acknowledge a flashing/beeping alerter every 20 seconds.”

…The second was a fear of change. It costs $168,000 per track-mile per year to maintain trackside signals, 90 percent of which is spent on labor—much of it done overnight and on weekends, qualifying the workers for overtime. If those signals were eliminated, millions of dollars could be saved each year. But New York decided to run CBTC on top of a reduced form of the old fixed-block signaling system, requiring that both be expensively maintained, despite evidence from other cities that no backup was necessary. (In Vancouver, the SkyTrain has had no CBTC-related accidents in more than 26 years.) And the fact that the two systems had to work together—requiring the supplier to study the old signals in depth—became a major source of delays.

Barone says New York just wasn’t willing to rip the band-aid off. Cities like London deal with major transit upgrades by packing maintenance and line closures into as short a window as possible, however painful that might seem at the time. New York, by contrast, draws out its track maintenance. When I spoke to the president of Thales Transport & Security, one of two major CBTC suppliers to New York, he said that “getting time on the track is by far the biggest schedule driver.” Crucial test-runs get queued behind miscellaneous track maintenance, so that it takes months to validate even small changes. “In the New York mindset,” he said, “there just isn’t the concept of the trains ever stopping.”

All that waiting isn’t free. These are huge projects for a company like Thales; they’ll spin up a whole office, a whole mini workforce, just to work on it. And when they’re waiting for track time, that workforce doesn’t just spin down—it continues to get paid. Anticipating delays, contractors inflate their bids.

So what we see here in this one little excerpt from a much longer story is an insight into why the MTA can’t seem to bring technology innovation to our system in a timely fashion and, in part, why everything costs so much. If we are to reform MTA practices and get more (or perhaps any) bang for the massive amounts of capital bucks to which the agency now has access, we’re starting to understand the best places to start.

More coming later when we look at how Somers explains the countdown clock conundrum. In the meantime, be sure to check out the full article. It’s well worth the read.