Monday was supposed to be better for Delta than Sunday – and it was, just cancelled ‘only’ 1,153 flights or 29% of their operation, and delayed another 45% more. On Sunday it was 36% canceled and 44% delayed.
There’s no end in sight because even before 6 a.m. Tuesday they’ve already cancelled over 10% of the day’s flights. They have planes out of position for maintenance – perhaps as many as 40% of their aircraft – and they still don’t know where their crews are and can’t get them assigned to flights.
Atlanta airport is jam-packed. Mothers w/ small children on the floor in the terminal. Line for customer service is 1/2 mile long. No rental cars. Few hotel options. People’ve been stuck here 3 days.
If there weren’t bigger national news- @Delta CEO would be dragged to testify. pic.twitter.com/tGSRGyxYG5
— Cole T. Lyle (@ctlyle1) July 22, 2024
Delta’s CEO and Chief Information Officer on Monday were telling employees that the only systems left not working properly were crew tracking, that they were running several systems in parallel to catch up, but that got complicated once there were further changes that had to be synced across those systems.
However on Monday night they had another major system failure, as first reported by aviation watchdog JonNYC. That failure was, fortunately, temporary.
..functions, we are unable to continue to work through open trips, plan flights, delay flights, etc. – TCC is engaged; more to follow as information is known – Travelport acknowledges an issue and is working to learn more – Bridge ongoing – If this is an extended outage ..
— JonNYC (@xJonNYC) July 23, 2024
Delta told employees it would take until the weekend to recover their operation. They’ve been much more upbeat publicly. They continue blaming an IT outage (CrowdStrike) which affected ‘other airlines’ which is.. true.. but incredibly misleading as all their communications on this mess have been.
They were quoting wait times for an agent at 17 and even 20-hours. Now they’ve just stopped telling poeple how long the wait will be since it was bad publicity.
After 15+ hour waits, Delta has stopped estimating how long it takes to get an agent via chat.
They will, however, repeatedly remind you that a “vendor technology issue that has impacted several airlines.” Nevermind that no other airline has canceled 1/10 the flights as Delta pic.twitter.com/d63o9OCkqK
— Kyle Potter (@kpottermn) July 23, 2024
While Delta isn’t being nearly as generous as Southwest was during its Christmas 2022 meltdown – not reimbursing passengers for travel on other airlines, for instance – they’ve been insistent that they are providing hotel and meal vouchers but there are plenty of reports by passengers that this isn’t happening either.
The scenes at @mspairport are crazy due to the @Delta meltdown
People are sleeping on floors as gate agents are empty and unwilling to provide food or hotel vouchers to those affected@PeteButtigieg pic.twitter.com/rfDnxKIySu
— Seymour Aes (@AesSeymour) July 23, 2024
Delta Air Lines promotes itself as a premium airline, and far more reliable than competitors. But when meltdowns happen it’s historically performed worse at recovery. And it’s not able to simply own its failures – the hubris and years of promoting an image that outstrips reality are just too great a habit to overcome.
Do the cancellation numbers you cited include cancellations from Endeavor, which is wholly owned by Delta and only flies for Delta?
Regardless of the fact that Delta is struggling harder than either United, or American to recover, they are just in continuing to lay the blame on the original catalyst.
I will say that I did not agree with the extreme way that Southwest was attacked in 2022, but media has given Delta a lot more grace for a very similar, if not worse response.
Technology will fail us again, we need to expect, accept, and prepare for when it does.
I am caught in the middle of it now. I wasn’t able to cancel my business trip, and because they won;t start delaying the flights they don’t have crew for early, and won;t cancel until much later, people really can’t be pro-active. They know they don’t have the crew, they just pray they can get someone to show up. My 1st connection flight home has been canceled for the last 4 days. I am tempted to rebook early, but then it puts travel cost burdens on us. Delta needs to take the hit on this.
A LOT of leadership there needs to go. A LOT. They clearly did not have a Business Continuity Plan in place that adequately addressed their crew scheduling software being out of service. I mean, that is a BIG one. That is a CORE function. I had no pity for Southwest as they had been warned for years. My feelings for Delta are the same. Given how their attitude has been the last few years, where their product quality has been in decline, and they have been really upping spend demands, some humbling was in order.
Instead of focusing on Delta specialty shops to buy things at high prices to get status, they should have been focused on their hard product and getting passengers from point A to point B safely.
I mean for cripes sake, they didn’t even have enough room for the luggage for the people on the plane yesterday and were starting to threaten to remove people. So we sat another 30 minutes finding every corner possible. It was really pathetic.
I say this as a Delta fan by the way.
@WhereTo2Next
Southwest absolutely deserved to be hauled over the coals for their unwillingness to address a known weak point in their ability to meet their core function: Flying people safely from point A to point B with their baggage. They KNEW it for years, and had been warned again and again by their own staff about the risks. Customers having their entire holiday ruined and being stranded for a week meant they deserved everyrhing they got. It was nice to have an administration that actually cared to do something about it.
Delta has not reached the numbers yet and probably won’t, which is why it hasn’t been as loud yet. But the brand is going to suffer for it. Big time. There is a serious leadership problem in all of this.
There won’t be any long term effects or repercussions from this. All the whiners and complainers will jump right back on them just like they did Southwest. Delta knows this which is why they’re not sweating it. Ppl need to learn to vote with their wallets.
@ Gary — This is no surprise given Delta’s expertise at lying to the public.
What a mess in Paris . Everything is delayed and over booked . Isaw pax waiting for 4 days to get booking approached agents and said their flight is cancelled. They were crying . They asked For manager ,they begged him to rebook them on American . At the end he did it . I saw the family boarding flight to DFW on American . It was chaos at the gate ..screaming shouting where next door AA full flight to Dallas was boarding with no drama . Long story short …why delta manager denied to re book pax on American or other airlines when their journey disrupted again due to delta incompetence. Pax waited 4 freaking days as per delta request to get re accommodation on the flight . Ed Bastian is quick to make cheap comments on other airlines when things goes wrong . Hopefully he learned his lesson now …
So glad it’s American I’m flying this week. Over-reliance on vulnerable technology with no Plan B is a recipe for something going very wrong sooner or later. But equally concerning is their dissembling and lying to their customers and inability to figure anything out going forward when all the other airlines were essentially back to normal in a couple of days.
where2
the reason the media isn’t more engrossed in Delta’s issues is because they are all watching the unprecedented political meltdown that has taken place and continues to play out in Washington.
If I had a dollar for all of the internet-know-it-alls that are convinced they know what caused DL’s problems and could fix it, I would be a very rich man.
Better yet if all of those so-called experts were half as good as they claim they are, they could make a fortune fixing DL’s IT for them. But they, including Gary, can’t do a thing and DL will have to figure out its IT issues.
And let’s not forget that DL has been an incredibly loyal Microsoft client (and vice versa) and has undoubtedly engaged a number of people at MS In the architecture of DL’s systems. As much as people love to picture DL as a bunch of fumbling idiots trying to extricate itself, I strongly suspect that Microsoft is heavily engaged and DL will use its negotiating skills – which Gary accurately notes involves has a 50-50 plus the hyphen success rate – to hand the bill where it belongs.
Unlike Southwest which didn’t invest in it’s IT, DL has invested a quarter billion per year but has obviously managed with IT experts to build something so big and yet so difficult to control that they can’t get it all restarted.
Has anybody called the police to do a welfare check on Tim Dunn?
Deltas stubborness has finally caught up to them.
@ Tim — I find it very difficult to believe that loyalty is a 2-way street for Delta under any circumstances, including in its relationship with Microsoft.
Caesonia has it exactly right. Delta has obviously failed to invest sufficiently in Disaster Recover and Business Continuity Planning. Heads should roll!
This outage had nothing to do with Microsoft. So they aren’t going to be negotiating anything with Microsoft. The problem was caused by Crowdstrike. That’s who they are going to be dealing with.
With this mess still going on with Delta,I think it is high time that Delta admits openly to everyone that they are souly the ones whom have Seriously screwed up in this situation and they need to fix their mistakes before things get any worse because if they don’t, then I wouldn’t be surprised if the company tanks Big time
Delta has a long history of lousy IROPS performance.
They are basically Spirit with older planes and more arrogance.
Maybe American Express can help fly some planes?
This is going to end up costing them a lot of cash. We’re fortunate to not be a hub captive and everybody got to where they were going by switching carriers. Just out of curiosity an employee clicked on the options button in the app when their flight cancelled. It moved them from first to coach and wanted them to fly 3 1/2 hours in the opposite direction, a long layover and then fly back. Not today Ed. Guess I’ll start canceling next weeks tickets (refundable) and move on.
well, yes, Dave.
CrowdStrike was a trusted IT provider that MS allowed to access root systems.
CRWD has done this to other operating systems but for MS to allow CRWD to do this says alot about MS.
CRWD’s CEO will be testifying before Congress – after their stock has lost 30% of its value in 2 days after this meltdown started.
Someone will be accountable.
DL invested in its IT and created alot of complexity but they didn’t do it on their own. are their people at DL that signed off on all this complexity? absolutely
but the notion that DL is struggling because it didn’t invest in IT couldn’t be further from the truth.
@gary – Is DL interlining any of their pax to ease the burden? You would think that’s the one thing they could do. Completitors could upgauge or add special flights to ATL and other hubs to look like heroes… no?
@Tim Dunn
Please do not comment on things you don’t have any understanding of. MS is legally prohibiting from protecting it core kernel from outside vendors. Stick with being a Delta booster.
Seems we have finally found the limits of Tim Dunn’s knowledge. Today is a sad day.
Seek help, Tim. I mean that sincerely.
@Tim Dunn
They may have invested in IT but it was apparently the wrong IT. Do you wonder why millions weren’t plunged into darkness and had no water or gas from this outage? We have parallel systems running at once. If both go down we have a backup running a different operating system. Then even if all else fails there’s a offline backup computer that can be started in under 5 minutes (we call it our doomsday computer). Airlines could easily do this but let’s face it, US based airlines aren’t exactly considered the top of the heap.
Living in Iowa, I flew mainly United – loved seat 26C on the B727 🙂
I did like to fly on NorthCentral then Republic to MSP, but never got into NW let alone Delta
And of course not interested in SkyPesos so no dog in this fight
Now in DC fly either AA or UA so still just watching from the cheap seats Delta’s hubris and arrogance play havoc with flyers
Best Wishes
Tim is Ed’s cheerleader, since Ed lacks the capacity to be truthful with his top customers even after they’ve been stranded for days without compensation or accurate communications.
It’s appalling that Delta is still trying to blame CS instead of its incompetent IT (automatically allowing all CS updates without turning them off/testing as AA and United did) and disaster recovery plans. All parallel/backup systems had the exact same configuration!
3Q Revenue impact is ~$10B and $1B+operating income is gone, excluding the permanent damage to the brand with it’s top corporate clients and the billions of dollars it will take to update its horrible IT systems.
@Tim Dunn — CNN devoted a significant block of time multiple times this morning to DL’s meltdown and failure to recover — pointing out that while the Crowdstrike “implosion” affected many airlines, DL is the only one still affected…The video from ATL doesn’t look good.
CNN: Delta is still melting down. It could last all week.
https://www.cnn.com/2024/07/23/business/delta-flight-cancellations/index.html
Adam,
I get it completely.
MS architecture allows an outside vendor to take down the operating system and the operator cannot recover. that doesn’t happen on other systems including Apple.
Yes CrowdStrike is the one that delivered wrong code to the whole world rather than batch releasing which would have caught the error right away.
MS architecture is such that a fatal flaw can take down the operating system in an irrecoverable manner.
h2,
thank you for confirming that your business comes nowhere close to the scale of airlines or other industries. You can’t boot up a spare computer in the backroom in the middle of a meltdown when you are running IT on the scale that huge companies do.
and, airlines get the media attention – where they are getting it now – because they are so visible but there are still large-scale health care implications that won’t be sorted out for weeks which are STILL resulting in cancelled elective procedures and reduced ability for some of THEIR IT functions to work.
This isn’t just about aviation or Delta.
Other industry’s 60% operations just don’t create near as much buzz as DL’s 60% operations.
Kirby and Isom looking pretty good at this point.
@Tim Dunn
So you’re saying it’s impossible for an airline to have a redundant system? You would be surprised at the amount of money and time we spent on a multi-state system. Our systemic has back up in hardened locations in several states and we are audited annually for our cyber security by a federal branch that actually has some teeth. But then again, we don’t have a CEO making tens of millions of dollars or shareholders to answer to.
@Tim Dunn
And your whining about other industries not getting as much attention is laughable. Most other industries don’t strand families sleeping on a floor for days on end. I’m still waiting for my refunds from Monday. I’ve kept screenshots of the cancellations to file a chargeback if needed. And any interline agreement is a joke. They didn’t offer that to my employees. Only some ridiculous downgraded flight in the opposite direction.
And people thought the SWA Christmas meltdown was bad. SWA passengers were at least made “whole”. Getting reimbursements for hotels, cars and flights on other airlines. SWA even threw in miles on top of all that. 25,000 to be exact. And the reimbursement (in cash) only took a couple of weeks)
It will be interesting to see what, if anything Delta does, to try and make up for it.
@Tim Dunn-
Serious question and I’m not trying to bash DL. Thousands of companies, especially airlines, got screwed by CrowdStrike. I can’t think of one other company that has not either fully recovered or is not close to it. You can’t keep blaming this on CrowdStrike when every other company has figured out a way to get back on track.
@Tim
Willfully and painfully ignorant. Microsoft is not legally allowed to close its kernel in the same way as Apple per a ruling from the European Commission back in 2009. I guess Microsoft could just close off their kernel and not operate within the EU? The same EU entity that is trying to force Apple to take similar measures with iOS. I wish them best of luck in pushing back.
Every other major company dealt with the same issue and none did as bad of a job with dealing with it as Delta. Period. Now go and take several seats.
While Gary described the current problem well, the this problem is that the system keeping track of where crew is located has broken under the strain of a mass interruption. While CrowdStrike was the catalyst for the start of the problem, that problem is fixed. The crew scheduling software is still broken. A bad storm in Atlanta could have the same affect. This is the exact same kind of problem Southwest had this winter, so it’s not like Delta didn’t have a forewarning that this kind of event could happen. It’s also not like the US never had a countrywide shutdown of it’s airspace.
I understand that Delta was not the original trigger of the problem, but their inability to recover from the issue is most definitely Delta’s fault. It doesn’t matter how much work one may outsource, Delta is still responsible for their selection of the outsourcing company and the software work they choose to do. Delta also had to be the party who approved using CrowdStrike. Someone at Delta needs to lose their job for this, and not one of the low level people. They should do like the head of the Secret Service did, and resign. They might not be the one who didn’t fund improving crew scheduling, but they weren’t able to make the case for the funding.
@Tim Dunn
Ok, that tears it for me.
” You can’t boot up a spare computer in the backroom in the middle of a meltdown when you are running IT on the scale that huge companies do.”
Actually you can. That’s exactly what the Public Cloud is all about; booting up entire data centers running IT at the scale an organization the size of Delta – or larger – needs. Certainly if you have done your Business Continuity Planning and your Disaster Recovery Planning well, with routine table top exercises and practice, you will have those types processes in place, and be be able to kick them off fairly quickly, including data migration. It’s hours, not days.
You know, I kept my focus on a BCP and to keep it simple, because while the systems may be complex, and operations is hard, the failure is simple. Even if you don’t, Delta’s CISO Deborah Wheeler has a CISSP and will know exactly what a Business Continuity Plan is, and a Disaster Recovery plan is.
ALL the airlines, including Delta have a core mission:
Move people and their baggage from point A to point B safely and reliably.
That requires:
1) Planes
2) Crew and FAs
3) Ground crew and ops
4) Resources to supply everything
If any of those 4 items are missing, no airline can meet it’s core mission. A Business Continuity Plan (BCP) will address the risks to and loss of any of those 4 items, and plan what needs to be done to maintain the business, even if there is something that happens that supports one of those 4 items. The loss of the system that provides crew scheduling clearly impacts business continuity, so plans must be made to quickly restore that capability, even without the software.
In 2022 Southwest suffered exactly this loss, and every-single-airline should have reviewed what happened, analyzed their own weaknesses over the loss of their crew scheduling software, and come up with or improved their BCP along with Disaster Recovery. To not do so is frankly, absolute incompetence. Delta’s operational collapse suggest they did not learn these lessons.
As for those IT people you complain about not knowing so much, I can assure you, your comments show you know just about nothing about how the industry works or how such large and complex systems are maintained and developed. I have many decades of experience, and do know what it’s like to have a very well designed and resilient system get kicked over by a highly unusual use case. This includes mission critical systems for organizations every bit as large as Delta. It is not a matter of if, it’s a matter of when. That’s why you have BCPs, and DR, and Incident Management. I can also tell you that your kvetching over MS on this one is rot.
Yes, Crowdstrike blew it. So did everyone on the receiving side that didn’t have a testing pipeline set up before they pushed it to production on their side.
Oh, and before you scream about the airline industry, there is about a century of aviation experience in my family, from Executive to current active pilots.