Is it just me or does this article feel kind of shallow? Like there's a lot of fancy text boxes and nice big text, but it doesn't cover a whole lot of information. Like how did the MTA build this infrastructure? Who lead it? What went into their decision making process for OS/2? Were there other options? They briefly discuss how the MTA is moving to contactless payments and casually mention Andy Byford's hiring (incorrectly citing him as the former head of the Tube—unless the MTA hired another former Tube employee?), but they don't delve into that either.
It's nice to have a succinct article, but I'd much prefer a long form version. If this article ran in say, the New Yorker, it'd provide a level of depth and information that I'd find really valuable. I assumed from the design of the site, namely the very pretty interludes, that I'd be in for some really good long form journalism. Unfortunately not the case.
Edit: Here's a funny snippet from an article about Andy Byford in which the credit card processing for the Metrocard machines goes down.
> It seemed that only Miguel knew how to log in to the relevant subprocessor and do the reboot.
> “Where is Miguel?”
> He was in a car, apparently, on his way home. He wasn’t answering his cell. He lived in Port Jervis.
> Byford looked at Meyer and Nugent. They shook their heads. Port Jervis was upstate, three hours away.
> “Unbelievable.”
> More calls were made, more cages rattled. Was it really possible that hundreds of vital machines, the main revenue engines of the subways, could be repaired by only one person at the M.T.A.? It seemed so.
Yep, I quoted the article in my edit. He's incorrectly referenced as the "former head of London’s train system" and they claim the reason for his hiring was "ultimately eliminating MetroCards", which is flat out wrong. Byford was brought on to "fix" the inevitable trainwreck that is the MTA while also taking the fall for said trainwreck.
OS/2 was very stable compared to the NT of the same era.
I'm not sure why we laugh at old systems that are still doing their job thirty years later. If anything we should celebrate the resilience and longevity.
Would it be immediately better if we replaced it with a bunch of docker containers connecting to mongodb or something?
I think we'd be better off as a society if we became tech luddites a bit more. Maybe we could start by replacing all of the facial recognition advertising panels with OS/2.
I'm not sure why we laugh at
old systems that are still
doing their job
Because other countries are hacking our infrastructure, as we hack theirs, and those laughing are doing so nervously, because with an operating system that is decades out of date, it's painfully obvious that anything defended by a password protected user account on that operating system isn't really protected against a determined enemy at all, and any of the permissions and privileges conferred by user accounts on that system are likely a facade protected by an honor system, and mostly physical security.
In truth, even if the operating system were newer, we would not want it hooked up to the internet, but if the operating system were up to date, and not well isolated and became exposed to any public gateways, it would stand a fighting chance of not spilling its guts to the world.
With such a system disconnected from the internet, one can still bet that a determined attack is going to implant remote connectivity under their own control, and their job is much easier once the implant is connected. They don't need to sweat OS level password rotation, probably won't worry about strong encryption key changes or having to prove identity to policy enforcement tools with signing keys or tokens. Very little in the way of locking the integrity of the system is built into the concept of the OS. The application layer may offer better protection with proprietary controls, but the contents of disk and RAM are very probably possible to dump, inspect, bit twiddle and write back in place to just do anything you want, if your expertise runs deep enough.
The answer is: quite probably. A lot of old systems get retrofitted with additional features/terminals/addons on top to make them "more modern" and as a result end up being connected to the internet.
Is there something obvious I'm missing here? I can very easily build a computer without any sort of radio in it. Another person could come along and add one, of course, but that would be hardware.
I think the real question is “what is the physical security model of the OS/2?”
I would imagine a determined attacker could infiltrate the system in-person. The question then becomes - how easy is it to compromise the system given physical access? And, assuming total compromise, how much damage and loss of life can be caused? (Do the subways have physical fail-safes?)
Cameras, locks, and obfuscation, basically. Ti hack it, you need to be alone with it long enough to find the flaws, figure out how dispensor works, etc. People hardly ever try to bypass them.
Recently, hackers have been buying, studying, and hacking ATM's. They're not secure. You can see them in action searching for DEFCON ATM hacking on YouTube.
Statements like that ring a bit hollow considering the number of security patches that Microsoft has to provide on a ongoing basis for their most "modern" Windows OS, and the thousands to tens of thousands of potential holes still left to be patched there. Out of necessity that patching is just never going to end, in other words, so Windows will never really be secure. In fact, it may just be about time to bring something like OS/2 back from the grave.
It seems to be nothing more than an interesting observation, and nothing about it seems belittling or denigrating. An old system is still running a platform well. Story at 11.
The trains in Melbourne ran on a DEC PDP-11/84 based signalling system.
In 2003 they started to move the application to PC running Windows XP using a special processor card to virtualise the old CPUs although "Due to the core software limitation (no source code available) we were compelled to integrate some original PDP-11 computer cards into the final product and this resulted in a hybrid PC platform"
Mostly unrelated story: I did a security assessment for a small bank that had moved their core processing software from the original HP 3000 hardware platform to a hardware emulator environment running on windows (something like this https://www.stromasys.com/solutions/charon-hpa/)
We cordoned it off from the network assessment out of concerns that it would be fragile. The assessment went as expected, but as we wrapped that portion the bank manager wanted us to have a go at the HP off hours ‘just to see’.
Never did figure out which part died, emulator or emulatee, but it went down hard within about three seconds.
Australian lawyers from the top of Google Search results say that it can be if done careful:
> Generally, where a computer program is reverse engineered by copying the idea of the function presented in the program code, the original computer programmer’s copyright is not infringed. Where a computer program’s expressions in code are reproduced or adapted (including into a different computer programming language), the original computer program’s copyright is likely to be infringed. This means that where reverse engineering occurs by “clean room” design, Australian courts are unlikely to find that there has been an infringement of the original author’s copyright.
The system does not “run on” OS/2. That is used to glue the metrocard terminals to an IBM mainframe. The subway “runs on” relay logic. Trains occupy a block because they short the two rails together, which completes the necessary circuit. Only part of the NYC subway is computerized. That part uses off-the-shelf hardware and software that I believe runs at least partly on Solaris.
This seems needlessly argumentative and unwilling to allow for any editorial leeway. A subway is composed of many parts that all need to operate for the system to work.
But if we want to get pedantic, the subway wouldn't exist without revenue, and that revenue is indirectly managed through an OS/2 system. Ergo, the subway runs on OS/2.
Most subways don't pay for themselves; it's funded by the state. Really there's no reason to charge for usage except that everyone in government likes to steal money from them when nobody's looking.
> In the future, people will access New York’s subway like they’re queuing for a roller coaster at Disney World. The process will require users to have an internet connected device that gets you through the gates, whether that’s a phone or a smartwatch. If we’re lucky we get a new MetroCard option. But it’s no guarantee.
This is false:
- You do not need an "internet connected device." (There isn't even reliable cell service at all turnstiles.) You can use an offline device that's previously had a card added to mobile wallet. You can also just use an actual physical credit card that supports contactless payments.
- The MTA has indicated that they're replacing MetroCard with an OMNY-specific contactless payment card, which you can buy using cash if you want. "Although cash isn’t accepted for OMNY right now, cash will always be an option to access MTA services. You still have the option to use MetroCard until OMNY rolls out system-wide." https://new.mta.info/omny
"In 2020, we will enable mobile ticketing in the OMNY app. In 2021, we will introduce a new MTA-issued contactless transit card for OMNY that you will find at retail locations throughout the New York region. In 2022, we will install OMNY vending machines at subway and commuter rail stations. In 2023, we will say goodbye to MetroCard. You will always be able to pay with cash at retail locations and vending machines." https://omny.info
The OS/2 ecosystem helped to bring the Windows "start button" back to Windows 8/10.
After Microsoft removed the Start button, the former #1 OS/2 ISV (Stardock Systems) created the Start8 utility as a 3rd-party $5 replacement start button. They sold millions of copies, proving market demand and motivating the return of the button in future Windows releases.
> Over the years, we’ve developed and/or published a lot of software that was “ahead of its time”. Sometimes, too far ahead of its time. :) ... Broadly speaking, Stardock is a bunch of “techies” who have let their hobby get out of control ... most consumers would be shocked if they knew the actual dollars earned by reasonably “popular” iOS and Android games and apps ... Our poorest-selling DLC for PC games generates more income than nearly every iOS or Android developer app we've gotten numbers for ... Start8 will undoubtedly be obsoleted by a future version of Windows as its stunning popularity (over 30 million downloads and counting) makes it clear it was a mistake for Microsoft to get rid of the Start menu in Windows 8. In the meantime, Start8 has been an unexpected boon. If it makes Microsoft feel better, the revenue from Start8 has helped us hire a lot of new developers making new Windows software.
> ... In 1998, the OS/2 market died off. Stardock went from a multimillion dollar business to one that made less than $400,000 in one year. The only reason we survived was because our former OS/2 user base bought Object Desktop for Windows, sight unseen, a year before it would be released. While “Kickstarter” is now common, most of you reading this know that in 1998 just doing credit cards over the Internet was a new thing, let alone giving $50 to a company to make a product that didn’t exist.
Given what I have seen in some large corporations, I honestly believe that eBay has the data on which companies use very old computers in mission critical processes. The amount of "vintage" parts that were bought off sellers on eBay by a couple of corporations I was at was rather large. It amazes me what has to happen for some companies to realize that your logistic system needing parts from eBay is a bad thing.
It's the "whatever keeps it moving" mindset rather than the "take a hit now for a better return later" mindset. I used to work as a Colgate factory making liquid soap and it blew my mind how poor of a condition their machinery was in and how it was run. Machines help together literally with ductape, zipties, and miscellaneous objects that we could find around the plant to repurpose or materials we could find at Walmart (it's the only place around open 24hrs). Many times machines would be down and instead of fixing them we would just "throw people" at the issue and keep the lines running with manual labor, creating an insane amount of inefficiency and waste so that there's no "downtime".
That shows how little value most labor has, unfortunately, versus how much money can be make just keep the business running. It’s a simple computation for anyone upstairs.
In that kind of manufacturing environment, typically it's just middle management keeping things at a local maximum and hiding problems from plant managers. Middle management is incentivized to maintain said local maximum based on some arbitrary metric (ie uptime) which can be decoupled from productivity. This is the primary mistake. Plant managers usually have the bigger picture in mind, but aren't aware of the issues, as they trust the people below them. Operators eventually stop complaining when nothing gets done, as they assume that's how it works. This leads to money spent fixing operator culture, which is really just a more visible symptom of the actual issue.
Long story short, you get what you incentivize for. Incentivize carefully.
30 years ago I walked into a facility (in order to help them with some computer issues) and at the time their main production equipment was over 100 years old, but still going strong. (It had a "Made in 1875" nameplate on it.) They kept it in good working order, and were in no hurry to replace it, either, except that the EPA was forcing their hand because the old equipment couldn't meet modern environmental standards.
Incidentally, the mindset that you reference above is also the same mindset which causes places to not kept sufficient spare parts on hand so that fixes can be made quickly. It seems that they would rather experience extended downtime than pay to have spare parts on hand which could minimize that downtime.
What you're describing there is a somewhat perverted version of "Lean" manufacturing.
Example - there's a production line that's been running 20 years on a 1980s computer. For continuity reasons, there are two spares in the warehouse. There hasn't been a failure in years, the spares are kept because finding replacements would be near-impossible.
Some management consultant rides in screeching about how these two mission-critical systems are "sunk cost" and "need to be disposed of to make the place clean and tidy". So they are.
Three weeks later, the primary fails and there are no spares...
It really depends. A lot of old equipment around the world is kept running by finding and buying used parts or getting something custom made.
Sure, at a given company or plant, some equipment might be costing more to keep running than it's worth and/or maintenance is just being deferred. But pretty much any industrial location is keeping things running with a lot of used parts.
In the automotive world (and no doubt others), an awful lot of less-than-brand-new vehicles are kept running using salvaged parts, for the mechanical bits and the body and so on. And even in the computer world, in the old days it was nothing for someones's old, expensive-when-bought-new computer system to be retired and then become someone else's "new", far-less-expensive-when-bought-used computer system. Or for the old stuff to be raided for parts, and so on. So it's not at all a new concept, and places like eBay just help facilitate this process these days.
I grew up on OS/2 Warp on a PS/1 Consultant. I still have a couple of boxes with original copies of OS/2 Warp and the manuals and optional software disks in my back closet. I guess you could say I have a soft spot for it. It was really ahead of its time compared to its contemporaries, second only to BeOS. But people who weren't techies back in the early 90s didn't see the value in buying a different operating system when their system came with DOS and Windows 3.1. And, honestly, they were probably right! Computers didn't really have the oomph or the networking capabilities at the time to power all the multimedia applications IBM had in mind.
So I can't really judge the MTA too harshly because OS/2 is solid and it'll probably gladly do the job for another 30 years so long as the hardware lasts.
This got me wondering...if you were building out a system like this today, which OS would you choose that wouldn't seem ridiculous 30 year later? And why aren't we building OSes that are simple and feature-poor with the intention that they'll be used for 30 years? Or the airline problem: it must be annoying that the tech you put in seatbacks has a short shelf life and will look primitive compared to the phones in everyone's pockets.
Whichever open source system best fits the usecase. There are a number of FOSS systems specifically for embedded real time systems, for general purpose stuff I'd probably go with OpenBSD. The key is not being tied to the vendor of the software. You want to be able to hire a bunch of random programmers in forty years who can just look at the code and port it to whatever hardware is cheap at that time. Ideally you'll want to continuously have a small number of people working on keeping the system running on recent hardware and updating the dependencies.
Take it from the other direction: What could you have chosen in 1989 that wouldn't seem ridiculous today?
How many OS releases from 30 years ago still have commercial support? How many OS releases from today can still run a binary from 30 years ago? Will the 30 y/o toolchain still work? Will the most recent release of the toolchain be able to compile 30 y/o code? Do you still have all of that 30 y/o code? Are there other dependencies? Can it run on modern hardware or is it readily emulated / virtualized? Is talent with the necessary experience still available in the workforce?
Looking back is pretty bleak and I don't think it's any better looking ahead. Technology is a liability.
The seatback device is a consumer electronic that can (and should) be experimented with, updated, upgraded, and replaced regularly. I don't care what OS it is using today or how fancy it looks - that has nothing to do with the first part of your question, which should (and fortunately usually is) altogether separate.
The off-the-shelf answer today is a variant of QNX or VxWorks. No one gets fired for picking those.
pjmlp is right: life-critical software must be certified to high levels of DO-178C standard. That's always either custom software or some RTOS's that are certified already. Linux couldn't pass DO-178C's high levels, if at all. The INTEGRITY-178B page has the docs and processes listed in bottom right if you want to see them. That's for prior standard, though. New one looks similar.
They probably run Linux in the less-critical stuff on the plane. At one point, they were running some apps on Windows and Linux simultaneously as a reliability technique. I don't know if they still do that.
RHEL/Ubuntu/Debian LTS and pray enough people remember how to work on it.
Or don't let your software get 30 years old. Reduce the number of components required to run it (avoid complex toolchains) so that when upgrade/porting time comes, there are fewer things to break.
It's worth noting that Muni's train control software was OS/2 based for quite a while. Unsure if they've upgraded to the newer Windows(?) based version since then. It was an off-the-shelf setup sold by Alcatel/Thales. Garbage at that.
Despite being internet connected (sfmunicentral.com) I don't think it was effected by the downtime that the point-of-sale systems saw. That site came about after someone discovered a publicly accessible "admin" interface.
That would be reassuring if it wasn't a part of the biggest subway system in the world.
I think you'd need an obscure OS and an obscure organization in order to significantly decrease the risk of the internet hating you enough to infect you with ransomware.
Oops, a third requirement: your obscure OS can't be the same obscure OS used by the biggest subway system in the world. Because if someone did end up hitting NYC Subway why not hit the other instances that showed up in a scan? Is the hacker really going to think the extra 0.01% profit isn't worth the zero dollars it costs to infecting a few more machines?
"IBM had long given up on it, even allowing another company to maintain the software in 2001. (These days, a firm named Arca Noae sells an officially supported version of OS/2, ArcaOS, though most of its users are in similar situations to the MTA.)"
So I suppose there are still updates. As an aside, at $129 for the personal edition [0] I want to pick it up just for the novelty of having an OS/2 machine running on modern hardware.
That OS was pretty much a feature demo OS for the x86. All that task switching stuff is used to the max, and of course the segments are as well. Good luck porting anything, even if you did have the source code and a budget. Also, the popular language was PL/M, which is impressively awful. https://en.wikipedia.org/wiki/PL/M
Favorite quote from the article, "Despite the failure of OS/2 in the consumer market, it was hilariously robust..."
Would this make the NYC subway system more or less prone to hacking? Can't imagine there are a ton of exploits in OS/2 that modern hackers would know about.
Probably about the same. The OS/2 era was when IBM had maximum leverage on NY government as buying them was seen as saving jobs as IBM was laying off thousands.
Yes! Way back in the day, before my family had cable¹, by Windows 98 machine was networked to an OS/2 machine that actually had Internet. It also had some weird thing with SOCKS, though I don't fully remember the specifics, just that I had a rough idea of what settings to look for and what to fill them in with.
But there was definitely TCP/IP support. I remember the day I migrated my parents off Netscape to Firefox.
¹The cable didn't have anything to do with it. When when finally upgraded to cable, we also got an Ethernet router/switch, so my Windows machine then hooked into that.
We used it at work way back when. The version we used had a TCP/IP package that could be installed and removed as it was heavy on resources. Details are fuzzy.
I used OS/2 on a 486DX-66 back in the day at home. I ran also ran it on a my first SMP system - Dual PPro (custom IBM build that did SMP - I worked there at the time and even wrote some code for it). It was rock solid for the most part. I still miss parts of the GUI - the ability to rip off a template for printer or a Doc, etc. There are still things that the latest OS just do not do well. Biggest issue was alway the single input queue issue in the GUI that could lock up the GUI but the system would just keep running.
Also Galactic Civilizations was a damn good game and only on OS/2 at the time -
I think I played some sort of predecessor to that game, called Star Emperor?[1]
But from the first link in your link… they also made Trials of Battle! That it's listed under "Other Products" does not do it justice, IMO. And Stellar Frontier. I did not realize how much of their stuff I had played.
As late as 2003 (when I separated from them), Sterling Commerce, a Columbus based EDI network now owned by IBM, used an OS/2 system to convert zModem and FTP traffic to SNA, so that PCs could talk to a mainframe without needing to to buy special modems that could talk bisync. That was the only non-trivial use of OS/2 I've seen, and for all I know it's still running.
In the 90s, the CompuServe online service had a dedicated tech support team for people using OS2CIM to connect, and they were pretty evangelical about using OS/2 over Windows or pre-OSX Macs. It's a fun ecosystem, and its users were pretty passionate about it. I'm glad to see it's still being used for something interesting.
One thing that the IT world (being a field which is only a few decades old) is going to have to come to grips with is the fact that, in general, infrastructure doesn't necessarily get thrown out just because it has grown a bit long in the tooth. In the US we have physical infrastructure which is many decades to centuries old, and elsewhere in the world there's still infrastructure in active use which dates back to at least Roman times. Now that old stuff certainly may not be ideal by modern standards, of course, but to the extent that it gets replaced this is usually done relatively slowly, and some of it may yet end up sticking around indefinitely.
If I remember well, Trenitalia POS and many European banks still use OS/2. A lot of mainframes have OS/2 as a frontend to boot (IPL) the big iron. Got heartbroken when OS/2 got sold.
Trenitalia upgraded most of their machines a few years ago, I don't think there are still OS/2 machines around for that.
I was amazed the first time I saw one of these crashed, showing the OS/2 Logo (Warp 3 or 4 I think). From the GUI it was impossible to tell, they had relatively high resolution and even, at some point, videos embedded.
I wonder who developed and updated the GUI, and using what.
It is done in this way on almost any platform that is classified as mainframe. Originally one would IPL such platforms by manually toggling in either some simple code that would somehow DMA the OS bootloader from some external device. This was not exactly user friendly and replacing the front panel blinken lights with some sort of interactive monitor/debugger running on separate smaller CPU was common approach (for example many PDP-10s have PDP-11 in them used as "console processor").
For more recent systems (IBM mainframes, Sun/SGI large machines, etc.) the controller also handles things like power sequencing, hardware configuration (e.g. physical partitioning of interconnect buses, resource allocation...) and mainly also enforces the hardware licensing policies.
"In the future, people will access New York’s subway like they’re queuing for a roller coaster at Disney World. The process will require users to have an internet connected device that gets you through the gates, whether that’s a phone or a smartwatch. If we’re lucky we get a new MetroCard option. But it’s no guarantee."
Is that actually true? Seems like a terrible decision, if so. I fundamentally don't understand why the onus would have to be on the user to have internet connectivity.
London lets you use contactless payments with normal bank/credit cards, Apple Pay etc.
There's been a ton of FUD around this concept since the moment OMNY was announced as the MetroCard replacement. They've committed to supporting phones with NFC, contactless credit and debit cards, as well as a future vended card that can be paid for with cash, likely from any nearby convenience store or machines in the station. Yet despite the hundreds of options that enables, there are still people spreading fear that this change will somehow lock out hundreds of thousands of people from using the subway (the argument being that lower income riders don't have access to the contactless payment methods). I've seen nothing to indicate this will be the case and they've committed to supporting the additional card before retiring the existing MetroCard.
I remember IBM even handed out freebies of OS/2, I was quite a fan of it and its Scripting language REXX. Jokingly we pronounced Os/2 as "O - S - half"
Why so? Very likely they took every weekend ridership and averaged them together. If anything, "averaging the day" makes more sense than "averaging the people". The latter would be something like "the average person takes the train every other weekend".
I never said averaging the people; I said averaging the NUMBER of people, which is what's being done. "Averaging the day" makes no LOGICAL sense; what is being summed? Over what is that sum being divided?
I'm actually surprised it's this advanced. This is the system that still uses glass insulators and cloth wire. The unions keep any technology advancements out to prevent displacing jobs
I think Citibank did too, for the longest time. I had multiple occasions of visiting their ATMs and being treated to a space invaders style interface ...
I went with some other Microsoft folks on a tour of a ballistic missile sub 15 (?) years ago. The missile control room had a PC running Windows NT 3.51. The person in charge saw the nervous looks we were giving each other and understood our concern immediately. He assured us the PC was just for keeping logs, and the computer behind us, a thing the size of two large refrigerators that based on appearance could have contained vacuum tubes, was in control of the missiles. We all breathed a sigh of relief.
...and a lot of (physical) stores still use a DOS-based application for their POS systems, judging by the appearance, although they might not actually be running it on DOS but a later version of Windows.
there's a saying that if it ain't broken, don't fix it.
but the new york city government must be dumping tons of tax dollars maintaining this outdated technology, i'm speaking mostly about the hardware required to run this operating system and software.
It's nice to have a succinct article, but I'd much prefer a long form version. If this article ran in say, the New Yorker, it'd provide a level of depth and information that I'd find really valuable. I assumed from the design of the site, namely the very pretty interludes, that I'd be in for some really good long form journalism. Unfortunately not the case.
Edit: Here's a funny snippet from an article about Andy Byford in which the credit card processing for the Metrocard machines goes down.
> It seemed that only Miguel knew how to log in to the relevant subprocessor and do the reboot.
> “Where is Miguel?”
> He was in a car, apparently, on his way home. He wasn’t answering his cell. He lived in Port Jervis.
> Byford looked at Meyer and Nugent. They shook their heads. Port Jervis was upstate, three hours away.
> “Unbelievable.”
> More calls were made, more cages rattled. Was it really possible that hundreds of vital machines, the main revenue engines of the subways, could be repaired by only one person at the M.T.A.? It seemed so.
Perhaps an upgrade isn't such a bad idea.