Apple Services Experiencing Outage

Posted by rock_artist 1 day ago

Counter129Comment63OpenOriginal

Comments

Comment by gnarlouse 1 day ago

I don’t think I’ve every heard of an Apple Services outage until today.

Comment by jonas21 1 day ago

Comment by rtkwe 1 day ago

Do they offer many services for *OS apps or are they mostly independent and only service their own first party apps?

Comment by ben_w 1 day ago

Theoretically they offer quite a few; in practice, I think nowhere or almost nowhere I've ever worked has gone for those options (except my own self-published games), instead preferring a custom solution that can be shared with web and android.

Comment by cr125rider 1 day ago

iCloud is the backend for basically any settings/file sync for normal apps.

Comment by paxys 1 day ago

But it is mostly used for periodic backup/sync. So you aren't going to really notice unless it is a very long outage.

Comment by amw-zero 1 day ago

Half of the world uses Apple products, how is that not the maximum possible impact?

Comment by Ferret7446 1 day ago

Because the consumer market is way less demanding than businesses. Hence why B2B and B2C are categorized separately

Comment by NicoJuicy 1 day ago

No website uses Apple

A sync issue in drive would barely be noticed.

Comment by x13 14 hours ago

Apple Pay, with 65M users, can rely on external content from Apple.

https://developer.apple.com/documentation/applepayontheweb/d...

Comment by jerlam 1 day ago

You'd probably be affected if you used Sign in with Apple, same as if you used third party sign in with Google, Facebook, Microsoft, or others and their authentication was down.

Comment by TiredOfLife 1 day ago

Apple market share is about 30%

Comment by fcsp 1 day ago

For mobile - For desktop computers, below 10% even. Given that only 73% of world population is estimated to have internet access in total, that makes it just a small fraction of world population overall

Comment by kstrauser 1 day ago

As a portion of Internet users who regularly pay for and depend on hosted services, I bet that percentage is much, much higher.

Comment by apparent 1 day ago

US marketshare is higher.

Comment by esseph 1 day ago

Half the world? Not even remotely close. Not for MacBooks or iPhones or anything else. They're a niche player.

Comment by dijit 1 day ago

To be perfectly honest with you, I wanted to come here to say that their services (on a good day) aren't very reliable anyway.

I routinely have email issues, file transfer issues (to icloud) and issues accessing their binary notarisation service.

The only thing that works routinely well is Apple Pay, however I think that it's refreshing a key lazily in the background and does not actually need a network connection to work. Good design at least.

So when I saw that they're having an outage, I thought. "All at once this time I guess".

I'll be really open here and say that I applied for an SRE job there out of hatred because whoever is in charge of SRE/Infrastructure Operations at Apple is doing a terrible job (or has terrible circumstances).

Comment by eek2121 1 day ago

I've never had issues at all. I use their consumer offerings (iCloud, Apple Music, Apple TV+, Apple Arcade, etc.) and I've never personally witnessed an outage or problem. I regularly transfer stuff to iCloud from my PC running Linux, and I use the email service with a custom domain that used to live on Google until Google canceled the free tier of small business and also raised prices on the paid tier.

Comment by dijit 1 day ago

It’s a bit hidden, for example if Apple Music isn’t working it will act as if my device is having an issue with the app playing; it doesn’t look like its a service problem because it hangs and freezes the UI for play- and pretty often I will force my mail to refresh and it will say the imap server is unreachable.

Regardless, these kinds of things tend to be somewhat regional. I’m based in Sweden.

Comment by 1970-01-01 1 day ago

Even downdetector.com has nothing about Apple Services being offline: https://downdetector.com/search/?q=apple

Comment by arprocter 1 day ago

Comment by 1970-01-01 1 day ago

They're in double-digit numbers for complaints. Must be a new record :)

Comment by crims0n 1 day ago

Same... to their credit, even during the major cloud outages I don't recall Apple services going down.

Comment by dabbz 1 day ago

If I remember right they primarily use Google Cloud with a small mix of multi-cloud stuff for iCloud

Comment by 8fingerlouie 1 day ago

depends on where in the world you live.

If you live in a region where they operate their own data centers, you will be running on Apple data centers. If not, you're running on a mix of Google Cloud and AWS (IIRC). They used to use Azure as well, but I think that's no longer the case.

In any case, your data is encrypted (by Apple) before being uploaded to Google or AWS, and only Apple has that key. Whatever E2EE encryption you use will be applied on top of that.

Comment by dabbz 16 hours ago

Last I checked they were phasing out their own DCs in favor of cloud-provided services. Though it's been a while since I have heard anything about it, so maybe those plans got canceled. It could have also been phasing out those DCs for only the specific services and not all services. My details on the whole thing are fuzzy at best.

Comment by 8fingerlouie 15 hours ago

As far as I know, everything iCloud and Apple Intelligence runs off of their own data centers if you happen to live "near" one, but you could still be using AWS and/or Google as well.

I live near the Danish Apple data center, and pretty much all my iCloud traffic goes there, with a small fraction (<10%) going to Stockholm, which has both AWS and Google data centers, so I assume they're using both for geographical redundancy (erasure coding)

It gets a bit more fuzzy once you start moving into Movies/Music/TV/Billing/whatever as well as their backend services for the store and monitoring.

Comment by bombcar 1 day ago

If the outages only happen to services that aren’t used much I understand why.

Comment by Etheryte 1 day ago

Apple is one of the largest service providers worldwide. Services are their second largest revenue stream after the iPhone.

Comment by justapassenger 1 day ago

1. About 25% of their service revenue is from charging commissions in app store and other 25% of the revenue is Google paying them for search default. Other services include things like insurance (applecare) That's not exactly same type services that most of the people would be thinking about.

2. A lot of their services have less criticality (and it's not a ding at them - it's often very explicit design choice).

3. App store having hiccups or iCloud backups being delayed it's not something that will usually gather enough attention of media.

Comment by ctime 1 day ago

You might be amazed to know how critical Services are to functioning Apple devices. While they mostly can run offline, there are dozens and dozens of services that Apple runs that modern ecosystems require (like certificate related stuff). Other oddball things related to iCloud, APNS and the private services like iCloud relay are all extremely critical to billions of devices. Thankfully the all mostly fail open (captive portal is particularly tricky). Not saying they are as critical or visible as, say, Google.com going down, but none the less would have a very very large and visible problem if they all did go down suddenly. Thankfully, due to Apple design philosophy, most are totally decentralized and teams are given almost complete autonomy on how services are ran, which makes them a huge confusing mess but also, kind of a feature as Apple generally expects them all to fail in odd ways and the software can generally handle it.

Comment by fsflover 1 day ago

macOS unable to open any non-Apple application (twitter.com/lapcatsoftware)

2603 points by mattsolle on Nov 12, 2020 | 1292 comments

> I am currently unable to work because macOS sends hashes of every opened executable to some server of theirs and [it's down]

https://news.ycombinator.com/item?id=25074959

Comment by amw-zero 1 day ago

that's a fantastic point.

Comment by 1 day ago

Comment by giancarlostoro 1 day ago

Seems both TV and Music are affected, I wonder if it's some storage service somewhere just totally degraded. I love CloudFlare for their total transparency reports about outages. I wish every other company would follow the same standard.

Comment by temp0826 1 day ago

I had to try a few times to download from the app store this morning. Thought it was a dns/blocking issue on my part.

Comment by thoughtpalette 1 day ago

Been using Apple Music all day today, streaming. Have yet to encounter an issue.

Comment by giancarlostoro 1 day ago

Must be a partial outage then? I was using Music earlier today as well. On my phone I download most of my music so I might never notice an outage though.

Comment by thoughtpalette 21 hours ago

Good point! I might not have noticed.

Comment by andersco 1 day ago

Also seeing outages relating to App Store Connect, currently unable to submit builds to TestFlight.

Comment by snarf21 1 day ago

The ASC website isn't loading correctly either.

Comment by apparent 1 day ago

Did this affect notifications/app updates? I found that podcasts from my third-party app were not showing up for me when they normally do. I would assume that when I manually refresh the list, that wouldn't rely on Apple servers, but given this outage maybe there's a linkage?

Comment by jordemort 1 day ago

Sorry, it's my fault, I migrated my playlists over to TIDAL earlier today

Comment by joecool1029 1 day ago

Curious to see the Testflight/Support apps updated just after. I’m betting they pushed an upgrade to something in their services and it was bumpy.

Comment by resonious 1 day ago

I wonder about APNs and Apple Business Manager. I've heard from people seeing weird stuff happening on those products but I don't see it on the report here.

Comment by niwtsol 1 day ago

This outage was affecting developer.apple.com and pushing iOS app versions to test flight/production. I thought I was going crazy, thanks for posting this.

Comment by impure 1 day ago

I'm getting a lot of 500 errors when trying to upload my app to App Store Connect. Also the website is loading slowly.

Comment by binsquare 1 day ago

Failed to install/download newest xcode after updating Mac.

Guess this explains it.

Does Apple internally do something like COE's like Amazon?

Comment by 1 day ago

Comment by JSR_FDED 1 day ago

Everything is working 100% for me

Comment by rvz 1 day ago

Looking forward to the post-mortem

Comment by bitpush 1 day ago

I'd be really surprised if Apple were forthcoming. Apple famously holds all cards close to the chest, so I dont expect anything from them. Happy to be wrong though.

Comment by seper8 1 day ago

Noticed this too earlier today. App Store apps not downloading.

Comment by sneak 1 day ago

Apple services are realtime keys to:

- activation: wiping an iOS/iPadOS/visionOS device, or reinstalling a mac after a full disk wipe

- apns: push notifications, used for realtime notifications for all apps on iOS/iPadOS, even things like Signal that do not use Apple's messaging infrastructure

- imessage: enough said

- boot ticket signing: required for any mac to do an OS update (it's serialized to the CPU's ECID)

any one of these going down for any significant period of time is going to cause widespread global economic disruption.

Comment by drnick1 1 day ago

And this is precisely why you need decentralized systems. It's absurd that text messages are routed through Apple's servers, or that you need Apple's permission to wipe a device.

Comment by NetMageSCW 1 day ago

You need Apple’s permission to wipe a device so when you steal a device, you can’t wipe it and sell it. It’s part of Apple’s ongoing efforts to prevent stolen devices providing any value when stolen.

Comment by fsflover 1 day ago

If this is mandatory, then it's not really your device.

Comment by Ferret7446 1 day ago

Well, the alternative is that it's not your device either when someone steals it, it's now their device. The average user cares far more about that

Comment by fsflover 1 day ago

If you can even opt-out of that, this is basically a nanny device.

Comment by FridgeSeal 1 day ago

> It's absurd that text messages are routed through Apple's servers

Well, that’s because…they’re not text messages.

Comment by gjsman-1000 1 day ago

If they were text messages, it's absurd that text messages are often routed through two servers even!

Comment by sosodev 1 day ago

Aren't all text messages routed through a server? I guess it's more decentralized when it comes to telecom though.

Comment by joecool1029 1 day ago

> Aren't all text messages routed through a server?

For SMS, no not a central server for all carriers. When SMS service originally launched it wasn’t even cross-carrier.

MMS acts like a mailserver at each carrier, sends a push notification and your device has like a day to download full message.

For RCS it’s supposed to be federated but carriers gave up and it’s now centralized through Google Jibe.

Comment by NetMageSCW 1 day ago

I don’t think it’s any more decentralized than Apple if you are talking about MMS (instead of SMS). Your carrier still provides the MMS servers.

Comment by 1 day ago