Delta Air Lines Meltdown: Leaked Internal Comms Reveal Causes, Recovery Delayed Until Weekend

Delta Air Lines has been a mess since Friday. CrowdStrike brought down systems all over the world, but other airlines recovered while Delta continues to be an operational disaster. Airline leadership is explaining internally what happened.

So far today Delta has cancelled more than 21% of its flights. 34% of its flights are delayed, so over half of the airline’s operations remain encumbered. Delta subsidiary Endeavor Air is still cancelling about 10% of flights. In contrast, American Airlines has so far cancelled 1% of flights today and United less than 1%.

The airline’s public statements have been little but finger pointing and self-congratulations despite the misery they’re putting both employees and customers through. It’s nearly as bad as Southwest was in December 2022, yet they aren’t getting the same level of recriminations and aren’t as generous with customers as Southwest was.

What it’s like out there:

The big airlines were affected. Delta’s competitors recovered, Delta did not. They’re finally sharing details of what’s happened with their employees. Aviation watchdog JonNYC leaked a transcript.

CEO Ed Bastian begins by thanking employees for the toll this has taken on them. Remember, they are stranded like customers are. They also have to deal with customers. And Delta has cancelled more than twice as many flights since Friday as they did in all of 2019.

In the transcript, Bastian comes off as smug – blowing off Transportation Secretary Buttigieg’s imploring him to honor the airline’s obligations to customers: “I said,, you do not need to remind me, I know, because we do our very best, particularly in tough times taking care of customers.”

Bastian then concedes it is going to take days to recover from this still, “hopefully Tuesday and Wednesday will be that much better again and get ready for a real good weekend” even the cheerleading CEO would advise not to fly Delta this week.

Getting past the boilerplate, though, Bastian turned remarks over to the airline’s Chief Information Officer who started off saying ‘look, everybody uses CrowdStrike’ implying this is not their fault. But other airlines recovered, and Delta didn’t! Delta has a history of taking much longer to recover operations than peers, even though in normal times they operate somewhat more reliably.

  • They had to deal with “over 1,500…key systems…60% of our most critical applications that run the airline are Microsoft Windows-based, which means all of them were down”

  • But they got them back up in just “a few hours” in fact in time for 7:30 a.m. operations on Friday in Atlanta.

  • The problem is that their systems need to talk to each other, and the data transfer was completely overloaded. Most applications came back online, but the two key systems that continued to fail were “one that allows our Atlanta tower to really holistically manage our biggest hub in terms of gating arrivals, departures” (but that came back in a few hours) and “our crew tracking application.”

  • Delta couldn’t track crew or assign them to work flights.

  • Crew tracking has been running five parallel systems to catch up but “it’s a very dynamic environment.”

  • They haven’t known how to “resync” and “reset” to “bring synchronization between where you are and the schedule.” Southwest Airlines shut down their entire operation during Christmsa 2022, and rebuilt their schedules and assignments manually when their computers couldn’t do it (which is why they kept cancelling half their flights each day – it was too big of a task to accomplish manually with their staffing).

Their CIO expects Delta to “get to a better place by the end of the week.”

Here’s the full discussion:

Delta once said that only weather would ever cause Delta to cancel flights. One wonders whether the loss of their operations guru Gil West during the pandemic (now Hertz CEO), who said that, continues to have consequences and how much layoffs in IT this past fall may have come back to hurt them as well.

About Gary Leff

Gary Leff is one of the foremost experts in the field of miles, points, and frequent business travel - a topic he has covered since 2002. Co-founder of frequent flyer community InsideFlyer.com, emcee of the Freddie Awards, and named one of the "World's Top Travel Experts" by Conde' Nast Traveler (2010-Present) Gary has been a guest on most major news media, profiled in several top print publications, and published broadly on the topic of consumer loyalty. More About Gary »

More articles by Gary Leff »

Comments

  1. As a former CIO I hate to say this but Delta needs to fire their CIO. Accountability starts at the top and he shouldn’t be allowed to lay the blame on technical staff under his control.

    BTW, notice we haven’t heard from Tim Dunn. I always suspected he worked for DL (likely in marketing or communications) but maybe he works in IT and has had to deal with 24×7 issues since last Friday. If so that would make me smile given his constant DL cheerleader posts.

  2. Lol but Delta is such a premium airline. People pay more to fly them and accept pesos for rewards. Lol

  3. feel free, Gary, to post the cancellation rates for all airlines, year to date.

    Delta’s cancellation rate YTD is STILL BELOW AA and UA’s – thanks to the MAX grounding and Texas weather which cancelled and delays tens of thousands of flights this spring in a state where DL doesn’t happen to have a hub.

    UA itself cancelled 1500 flights Friday-Sunday.

    AA’s on-time percentage yesterday was worse than DL’s
    And right now, 27% of AA’s flights are delayed compared to 34% for DL. and AA has still cancelled 50 flights today.
    WN is operating more on-time with fewer cancellations than AA which operates a smaller mainline system than either DL or WN.

    If there is a “winner” in all of this, it is WN – which gave away far more passengers to competitors 20 months ago than DL will.

    I realize you generate page clicks but a little perspective would show that you, like Ben, are remotely committed to facts instead of simply stirring up controversy.

  4. I do enjoy the continuing comparison and contrast between WN (the reverse of NW, a predecessor to DL) and DL. This time WN came out smelling like a rose; whereas, DL stinks to high heaven…..

  5. It is very hard to watch a formerly proud company be humbled so badly and take so many millions of people’s plans and a lot of collateral money for vacations and associated travel expenses down the drain.

  6. AC,
    spare us. Please.

    The only person that should be fired is the CEO and everyone down the chain at CrowdStrike and the people at Microsoft that trusted CRWD to have as direct access to MS operating systems.
    For someone in IT, you seem woefully ignorant of how big of a deal this is far beyond the airline industry.

  7. Wow imagine being so bad at management you fail to realise your reliance on such a key vendor like this. Basic business resilience planning tells you not to have 60% of your systems reliant on one vendor… Bastian is going to have a lot of fun on his next earnings call.

    @Tim Dunn – the other airlines might have had plenty of cancellations when there was IT crash – the difference is they recovered when the crash was over… Delta’s underinvestment in IT is coming to haunt them now…

    Also now that they have employees stranded everywhere and have banned non-revenue flying, how do you think the employees are feeling about their employee value proposition?

  8. @Tim Dunn-
    You of all people should know that cancellations YTD compared to the last 5 days are meaningless. Canceling a small percentage per day compared with this weekend fiasco isn’t apples to oranges. Most days when a flight cancels the passengers can quickly be rerouted/rebooked. Not this weekend and of the thousands of businesses that use CRWD, DL has handled it very poorly. The part that makes it more shocking is DL is known for its service compared to its competitors. Not this time.

  9. @Tim Dunn – “AA’s on-time percentage yesterday was worse than DL’s”

    That is misleading af, Delta cancelled 36% of flights and delayed 44% of flights. AA cancelled 2% of flights and delayed 46% of flights.

    Delta took over 1/3 of its flights off the board. If they’d operated those their % of flights delayed would have been much worse!

    And American’s delays were driven by storms in Texas (61% of DFW flights were delayed). Whose delays were considered controllable?

    As far as facts, this post LAYS OUT WHAT DELTA’S CEO AND CIO ARE TELLING EMPLOYEES.

  10. Running a too mean and lean machine is a great way for company executives and shareholders at the time to see the money rain down upon them when things work. But then they also should have to pay the price to adversely impacted customers when the chickens come home to roost with things breaking and staying broken for longer because of all that history of “greed is good”.

  11. Heh, I love this gem.

    “I said,, you do not need to remind me, I know, because we do our very best, particularly in tough times taking care of customers.”

    The reality remains, over and over, we’ve seen Delta simply screw the pooch during IROPS. That’s when their deflections, their smugness, their arrogance, their past-Sell-By-Date equipment, their badly located hubs…. all come back to bite them.

    Reminder: When Delta’s own power systems at ATL failed? Who did they blame? Georgia Power!
    Georgia Power basically (both times. Delta pulled this nonsense twice.) did a press release to say “power’s fine. Delta’s lying. Again.”

    And, this is the core problem with Delta– they (and guys like TimD) want to believe their own BS so badly they’ll say about anything to deflect.

    Remember this: you absolutely cannot trust Delta to do the right thing in IROPS. They will strand you, starve and leave you to the wolves if that’s what it takes to keep the lies coming.

  12. Welcome to a great example of how the pursuit of “security” increases insecurity and of how reliance upon vendors with slick salespeople is a threat to operations and even more so when relying too much upon contractors to do what an organization could have previously done itself.

  13. Gary, pilit and Andy,
    You act as if the trauma to any single passenger is any less when AA or UA or WN cancel routinely compared to DL’s mass cancellation over a weekend followed by months of very high operational performance.

    You all act as if there weren’t tens of thousands of flights just a couple months ago when Texas was a weather disaster or in January when the MAX 9 was grounded.

    DL is taking its turn through the barrel but to somehow think that the rest of the world sees what DL is facing right now as worse than what AA, UA and WN have been through earlier this year is the height of bias – and ignorance.

    And if 27% of flights delayed in the early afternoon is a “normal” AA operation, its no wonder their revenue is falling so badly.

    and, yes, AA’s on-time has fallen far more during the past year than DL’s will when this is all factored into performance of the whole year.

  14. The ELT folks at WN must be just gloating all over the HQ building at Love Field today!

    WOW! I am utterly amazed that WN did not have any cancellations related to this software SNAFU.

    The project managers at Elliott are scratching their heads…”why did this not happen to WN?”
    Oh yeah – it’s that Commodore 64 computer and Windows 3.1/95 that saved WN!…lol

    As WN upgrades it’s IT infrastructure – the business case has been made to have redundant back-ups hiding out in that warehouse in Arlington – just for situations/failures such as this.

    CEO Bob and Exec. Chairman Gary get to stay another five years, poison pill – or not!

    Too bad for DL – but it can happen to any airline. Can’t wait to see if there is any fallout from the D.O.T. over this. Fines maybe? Buttigieg gets to toot his “horn” some more – for sure!

    SO_CAL_RETAIL_SLUT

  15. DL Slogans:

    We don’t get you there.

    We love to stay on the ground and it shows.

    Grounded

  16. Blaming the DL CIO for this? That’s a misdirection. Better to blame the people really responsible for these kind of debacles: the CEO, CFO, Board of Directors and the shareholders wanting more instant gratification from higher share prices regardless of what it means some years down the road.

  17. Gary,
    the DOT does not “excuse” any flight delays or cancellations.

    Weather is far more the responsibility of airlines to plan around than a first-ever vendor code contamination that took down more computers than had ever happened before.

    But, it is clear that you will justify anything you want to throw shade on someone else when it is clear that AA’s operation peaked for about 3 nanoseconds last year and is now the least reliable of the big 4.

    And when you factor in this CrowdStrike event and the MAX 9, UA will have cancelled a HIGHER PERCENTAGE of flights than DL – even considering that DL’s mainline operation involves about 30% more flights than UA.
    There simply is no distinction in any real and sane person’s eyes between weather, the MAX, or CrowdStrike.

  18. @Tim Dunn

    “””feel free, Gary, to post the cancellation rates for all airlines, year to date.”””

    You know this is nonsense, right? Delta’s FAMOUS for gaming the stats– changing flight numbers, pulling flights off the board completely to avoid hurting their scores.
    They also, famously, claimed to “never cancel flights”– Soon after than I saw a DL HNL-LAX flight sitting in Honolulu on a 18 hour delay. I”m sure those PAX felt great about “we will keep you hostage rather than cancel and let you re-book…”

    The point: The DL folks are great with the games to make themselves look better– despite some seriously rough pooch screwing in IROPS.

    I was once stranded by these Ehh-holes in BWI via Atlanta and started to pull up DL’s “claim” for ontime percentages (the site claimed “90-95%” for the flights in question)…
    FlightAware and FlightStats said only twice in the prior 60 days had any of those flights been less than 2hours late. There’s liars, damn liars and Delta performance stats.

    This OP post here reinforces you can’t believe a damn thing a DL exec (or mouthpiece like you) claims.

    I feel for the 200,000 people DL has stranded currently– having spent too many nights stranded on the floor at ATL myself. Delta is the Disgrace of The South.

  19. @Gary

    This one’s pure gold… Thanks for linking it.

    https://viewfromthewing.com/delta-tempts-gods-weather-will-ever-cause-cancellations/

    A gentle reminder of how Delta will almost pathologically deflect, and claim it’s got some sort of operational superiority– when the reality might be they “could” be 5% better on a good day, but it hardly makes up for the 500% worse they are in IROPS.

    I guarantee people die of malnutrition and dehydration at ATL every time DL takes two weeks to dig out of hole it dug for itself– a hole UA or AA seems able to navigate in hours, not weeks.

    You can’t strand 200,000 people without accommodations and not have adverse human consequences.

  20. Tom,
    if you haven’t been on a whole lot of Delta flights that arrived early, then you are exceptionally UNLUCKY in picking your flights.
    The data the DOT uses to calculate on-time is available and the statistics do show that DL runs a far better operation than any of its competitors over a year.

    And NO AIRPORT is any fun during a mass operational meltdown – as has happened at ALL OF THEM.

    Every part of the country has its “disgraces” which also happen to be the same places that pump billions of dollars into those region’s economies.

    To no surprise, anyone that myopically focuses on what they can see in an industry that serves a billion customers per year is going to come to the wrong conclusions.

  21. Everyone at the Delta leadership level has a hand in this mess. IT outages will happen, the impact and recovery need to be well thought out in advance. Having a razor thin Customer Support Staff will save you money until it costs you BIG, money not actually saved when things hit the fan. People standing in line for hours? That says it all, Delta is not capable to deal with the unexpected, time for them to re-think serving the customer properly in both good times and bad.

  22. The thing I don’t understand and by no means am I an IT guy. I work in critical infrastructure. If our systems go down 10,000s of people are affected. Our control room has dual PCs operating the system. If one of them goes down you simply use the other one. If both of them go down, there is a large touchscreen operating on a completely different operating system that can take over the operation. If something also happens to it, we have a doomsday laptop in the corner that is only manually updated. I simply don’t understand how something so critical can be brought down as easily as it was.

  23. Quoting a certain Tim Dunn, amongst hundreds of words of his comment barf on August 11, 2021: “Consumers, regardless of the price, buy their tickets with the expectation that their flights will be operated and reasonably close to on-time.” Based on Tim’s own words, Delta has failed. Anything else would be a #TimDunnDoubleStandard

    Which isn’t stopping him today from, in fact, imposing a double standard.

  24. @ Tim Dunn and other regarding Microsoft.

    It’s not clear how much MSFT is to blame since the EU required MSFT open up kernel level access. WSJ goes into the dynamics of this:

    “A Microsoft spokesman said it cannot legally wall off its operating system in the same way Apple does because of an understanding it reached with the European Commission following a complaint. In 2009, Microsoft agreed it would give makers of security software the same level of access to Windows that Microsoft gets.”

    Microsoft actually has a competing product to CloudStrike.

    https://www.wsj.com/tech/cybersecurity/microsoft-tech-outage-role-crowdstrike-50917b90?mod=tech_trendingnow_article_pos1

  25. @Tim Dunn
    Quit trying to compare an overall cancellation/delay rate to what has happened in Atlanta. Anybody that has traveled has been subject to IROPPS. There’s a major difference between 2- 3% of flights Canceled over a year’s time to canceling half of them at a large hub. People simply can’t get out of Atlanta like you would do if only a few flights were canceled.

  26. @ Tim Dunn — Always deflecting when precious, premium Delta fails. This has nothing to do with anytime besides NOW. Tell all of those people stuck at ATL right now how Delta WAS the most reliable airline year-to-date. I’m sure they will be filled with joy and pride at their selection of Premium Airways and will spend their spare five days of waiting at ATL booking more Delta flights!

  27. Adding on to Gary’s comment about yesterday, moving on to today, @Tim Dunn made a big deal that “AA has still cancelled 50 flights today”. The article stated that American canceled 1% of their flights today (thus far), which would have been the 50 flights. How much for Delta today ??? Try 21 %, roughly 20X WORSE !!! And Delta is talking about not fully recovering until this weekend ? Wow.

  28. Bastian needs to understand that the buck stops with him and resign. If Delta at least took care of passengers they screwed over then this would be different but they haven’t so Bastian needs to go. Now.

  29. Hmmmm, looks like the management at Delta has truelly Screwed the pooch this time with their Cocky attitudes and lying and this is the aftermath of there Screw ups ! Good luck fixing your mistakes Delta

  30. The story throughout business–everywhere, not just at airlines–is that IT is considered a cost to the business and is repeatedly cut. If you find an IT operation that isn’t hollowed out and running mostly with offshore contractors, it’s heading there.

    All of this is fine until something breaks, at which point there’s nobody left who knows how to do or fix anything (offshore contractors are trained to press buttons but rarely know how a computer actually works) and if there is, they probably aren’t local.

    Until IT operations are regulated like flight operations are, the incentive will always be to take unreasonable risks–either through underinvestment, ignoring cybersecurity requirements (it’s shocking how thinly staffed these roles are in airlines and how little they pay relative to other industries) and taking undue risks to the operation in order to save a few pennies.

  31. The market spoke today – Delta stock down nearly 4%, United outperforming it by nearly 5%.

    What a stain on Delta’s reputation right as its loyalty program is weaker than ever and high value customers make back half of the year status decisions.

  32. Something I did not see yesterday or today at ATL: upper management in the trenches with the calm, caring, heroic front-line workers. Ed Bastion could have been in terminal, wearing a name tag, and helping out.

    FWIW, I didn’t see any passengers having meltdowns. There was some tension in the air, but most people were patient and understood that the customer service and baggage staff were doing their best and working their tushes off, on ultra-long shifts.

  33. It is very hard to watch a formerly proud company be humbled so badly and take so many millions of people’s plans and a lot of collateral money for vacations and associated travel expenses down the drain.

  34. @Tim Dunn: It’s difficult to compare weather delays and the 737Max grounding to the Delta meltdown here. Weather and the 737Max were external factors that the airlines couldn’t fix themselves. Crowdstrike was an external factor as well, but the airlines had it in hand to fix it. Most airlines did by now, but Delta did not. So that’s more in Delta’s hand to fix it.
    That’s also why Delta said ‘only weather could stop them’ as it is an external factor that they can’t control. Flying out of Texas during storm season…no airline will have a stellar record. If you got a hub there… It will hit you harder.

    If airlines cancel a small percentage of flights everyday – that can be compensated (maybe it’s even overscheduling in the first place). But no airline can compensate 30% of a cancelation rate in one day. Once again you are confusing % vs absolute numbers. Or because you are invested into Delta – you try to show them in a better light as they currently are.
    We agree that Southwest and American have better recovered, followed by United. United is doing better than AA – but it’s also easier to fly in (for example) the Northeast, when one competitor cancels a lot of flights: Less Airplanes in the same amount of airspace.

  35. @Tom – Great idea about management helping actual customers but that would require management to care.

  36. Greg,
    and yet UAL as a company is still worth just 55% of what DAL as a company is worth.

    Bill,
    we get it that you want to brand DL with the scarlet letter M (for meltdown) while you hypocritically prostitute with others.
    There is no difference if DL takes its operational hits all it one time compared to UA’s doing in, so far, 3 major events this year – the MAX 9 grounding, May storms in Texas, and now this.

    And DL’s rate of cancellation so far this year is still lower than UA’s even with this event.

    and DL’s choice of crowdstrike is no different than UA’s choice of Boeing. If you want to argue that 3rd party events shouldn’t be counted then the same is true of both. UA just happened to make TWO errors this year.

    And I do wish all of the people that talk about how DL has underinvested in IT would tell us how much they have spent esp. compared to AA and UA (we know full well that WN is spending far more than all of them on IT after years of not spending on IT)
    What is far from known is how each of the big 3 configured their IT and what tech advise they followed.
    DL did say it was running FIVE parallel crew tracking systems but they all were built around Microsoft and CrowdStrike.
    Delta is undoubtedly entertaining what it takes to not have identical platforms or not allow updates to all systems happen at the same time.

    But to repeatedly state that DL has not invested in IT is as ignorant as excusing AS and WN for having all 737 fleets or United for placing hundreds of Boeing orders only to then go begging to Airbus for 321s to bail out Boeing after United became the last US Airbus operator to add the A321

    Everyone makes bad decisions.

    To try to argue that whatever led to this meltdown and recovery for DL is so much worse than everyone else is the height of hypocrisy and can only be done by selectively ignoring every other event at other which have cumulatively created more damage than this event has done to DL

  37. Tim: Do you often find people you meet in real life looking to end conversations with you as quickly as possible?

  38. I hope AA, SW and UA are charging Delta rates or more for flights out of ATL. Heck, bring in the big iron to help poor old Delta out. SWA has a big operation at ATL, that should help some of the deltoids pax get home.

  39. @ Tim Dunn, “And I do wish all of the people that talk about how DL has underinvested in IT would tell us how much they have spent esp. compared to AA and UA” — oh, the old “you do the research and prove my point for me” trick. A trick so lame that a decent first-year high school debater would know not to use it.

  40. @Tom
    “Something I did not see yesterday or today at ATL: upper management in the trenches with the calm, caring, heroic front-line workers.”

    And What a story that would’ve told!

    No sanitized state visits. No photo ops. No Tom Brady. Just senior mgmt. in places like the bagroom, and the counter and working the lines of customers.

    Worth noting that in some stations, local mgmt. has been out there. Shame the C-suite isn’t following their example(s).

  41. If I was Ed Bastian, I would probs talk down to Pete B. I mean what TF does he know about actually running an airline or taking care of guests? He has zero qualifications to be Secretary of Transportation. Great TV communicator/outwardly seems like a good person/wildly unqualified.

  42. @ Tim Dunn

    <>

    I mean WHO CARES if you arrive a few minutes early? Seriously. At LAX T2/3? The chaos of gate assignments over there virtually assures you’ll sit on the tarmac for awhile awaiting a gate to open. There’s ZERO symmetry to “arriving 10 minutes early” canceling out “arrived an hour late to ATL, missed the last connection (despite the fact that they COULD have dealt with a mis-connect at the originating station) and are facing another long night in America’s saddest late night airport”. An hour LATE is way, way worse than arriving an hour early.
    And, “CANCEL” is often better than leaving you hanging for a few hours as your options dwindle.

    The last time I flew LAX-OGG on DL? They were playing that Seat IOU game only Delta plays–along with a two hour delay– and I finally told the Service Agent “Look, assign me a seat or I’m going straight over to United in T7 that’s got a flight going to Maui in an hour… and I know THEY will assure me a seat”.

    <>

    And, AGAIN, Delta is been the Prime Mover in gaming that system by changing flight numbers, changing routing and times, to make problematic flights appear better. Back when DL flew those PoS Mad Dogs to ATL every hour? You could watch them for months on end– NOTHING departed or arrived when they claimed it did– and the “stats” claiming “90% ontime” were laugably false.

    I’ve got close to a million miles (and do have my million on both AA and UA) on Atlanta’s Disgrace, but after the THIRD time in a month Delta couldn’t get me to my destination even on the right day?
    I gave up on the pathological liars. They care much more about their credit card revenues than they do about actually serving passengers. Unlike Spirit and Frontier that are not on my lifetime ban list, but they are the carrier of last resort, because you simply can’t rely on them to perform.

  43. @Tim Dunn
    I can tell you exactly how much Delta should have spent on IT. Whatever it took to prevent people from sleeping on the floor of Atlanta airport for 4 days with no available hotels or rental cars. It can be done.

  44. @Joseph

    Well, Pothole Pete DOES have a degree in literature…

    Dunno, I guess his short stint at McKinsey was supposed to be a proxy for “understands businesses and logistics”, but now that I re-read his resume it’s pretty thin. Inspiring life story I guess, but I doubt he’d last a week in a job that required business process knowledge.

  45. @Tim Dunn Why do you care so much about what others think about Delta. Seriously, why does it get under your skin so bad that you fight in the comment section of blogs like this one and OMAAT as if your life depends on it. Are you getting paid to do so?

    Most people have a brand that they absolutely love, I’m no different but I’ve never defended any brand or company the way you do Delta. If you’re not a paid employee, then they damn sure need to hire you. You take a lot of verbal ass kicking’s in the name of Delta and I just don’t get it.

    You must know that you’re being mocked simply because others think of you as being smug, a smartass, or a Delta cheerleader. I’m not saying that it’s deserved or fair…but why do you care to engage as if you’re going to change their mind?

    Full transparency, I have no love or obsession with any airline. I just want whichever I choose to get me where I’m going, on time, and without accident or incident.

Comments are closed.