Apple Isn’t Caching Podcasts.
The guys at Dailysonic have an interesting theory about Apple caching podcasts. The way they have tested the theory is even more interesting. Very complicated, and interesting. Not sure why they just didn’t make a request and check their web server log files, but hey.
I don’t have access to the TPN logs, but I still thought I’d test the theory myself. I just used tcpdump.
First I tried The Gadget Show feed, and it definitely came from our hosting provider. I also did The Good Beer Show, and it came from their hosting provider. I tested this by grabbing a file while taking a dump from the network interface. Dailysonic’s file however came from an Akamai service. This might indicate that their hosting provider is using an Akamai service to help with bandwidth. If that’s the case I’d find it interesting in how that would effect their stats for checking show downloads.
It’s intriguing that their water mark shows a stamp from Apple. I wonder if this might be an artifact in how the request is sent. What if Apple makes the request via it’s web objects and then hands over the file so that they can track click throughs. I’d really need to know how their watermark software works, but that seems the most feasible.
The fact that iTunes show feeds and files coming from Phobos isn’t that conclusive. That is just a Webobject which is just a way of generating pages like PHP or CGI. Doesn’t mean that is there URL they are feeding the data from. It probably just redirects.
I’d find it very strange if Apple was caching all the shows. Imagine that. So far 1 million subscripts. Say that’s only 1 show per feed at an average of say 5MB (all very conservative estimates) That would work out at almost 5TB of data fed in only a few days. One hell of a bandwidth bill hey.
So I’d say that they’re a bit wrong about the caching. From my results the only possibility is that they cache some of the shows, but I can’t see why they’d bother. Sure, perhaps the ones they’ve initiated like Curry’s PodcastFinder, or ABC News Shuffle which they probably have control over, but certainly not just Joe Average’s feeds.
Maybe they should test again in combination with tcpdump or snoop tools and their server logs. That’d be a hell of a lot more definitive.
Wouldn’t worry. Someone will reverse engineer how it works in no time.
Update : Mick confirms that TPN is seeing hits in our logs.
July 3rd, 2005 at 11:00 pm
There is another way we can confirm Apple’s caching. Each one of our episodes is generated dynamically. So each time it is downloaded *from our server*, you will get different ads, even for the same episode. If you go to iTunes, subscribe to our podcast, download an episode, check the ads, delete the feed and repeat… you’ll find that every time you download the episode, the ads are the same.
That means that the show is not being downloaded from our servers, but rather cached somewhere.
Our theory is that Apple is only doing this for a subset of the podcasts in their directory, not all.
-Isaac, from Dailysonic
July 3rd, 2005 at 11:08 pm
[…] There’s a post over at The Podcast Network that questions whether Apple is caching podcasts at all. They think that our watermark may be displaying a hit from Apple because of the way the request is sent, not because there is any caching going on. […]
July 3rd, 2005 at 11:45 pm
Isaac
That could be the Akamai cache. Have you asked your hosting provider if they use the Akamai service? When I do a tcpdump when requesting your files I see it coming from an Akamai server, not Apples.
Rich
July 4th, 2005 at 12:09 am
The caching isn’t on our end. If it were, this problem would show up both when you download episodes from our website directly, and when using other podcatching software such as ipodder. We’ve tested our site on numerous software packages, and the problem only happens with iTunes.
I agree it sounds pretty far fetched that Apple would cache all the podcasts… that’s why I initially didn’t buy the theory myself. But it seems pretty clear, at least from our end, that at least some are being cached. That bizarre url to phobos.apple.com only appears for some of the podcasts. If it were simply apple’s redirect mechanism, why wouldn’t it show up on all of them?
Hopefully Apple can resolve the issue once and for all in the coming days. We love Apple. (We want to rock Apple slowly in our arms under the stars on a dark, romantic, summer evening…. mmmmm…) So we hope we can work with them to fix the problems that they seem to be having.
July 4th, 2005 at 12:25 am
We have a dedicated box with voxel.net… I’m pretty sure if they were using Akamai, we’d know about it. I can’t figure out how to filter what’s going on with tcpdump… too much info, can you gimme a hand? Hop on skype?
July 4th, 2005 at 2:24 am
As I read their description (and others) that they don’t cache all or everything but some.
Question is, what do they cache …
August 30th, 2005 at 9:22 pm
Two quick questions if you might indulge me:
1.) once a feed is submitted is there any way to delete it with iTunes? I mis-typed the location of my .xml file and it seems to be ‘stuck’ in iTunes for eternity? Can it be edited?
2.) Is there anyway to ‘tag’ .mp3 files with artwork, etc. through iTunes if they do not have to be converted to .mp3 (I’m recording straight to .mp3 and found myself longing for the ability to add the artwork and file specific tags that you noted in the tutorial, but don’t know how!
-jean@indepres.org
August 31st, 2005 at 10:07 am
Hi Jean
If I’m reading the questions right :
1. As far as I know the only way to make changes is to contact Apple. I’ve heard that in most cases they are very responsive.
2. You can tag you file in two ways. Firstly using the ID3 tags to add Album art. Secondly in the RSS for iTunes that Apple has created, there is a itunes:image tag. This will add images to the iTunes library.
Hope that helps.
Rich