Feedburner’s URL switch

[Update on the morning after (Dec 4) 2014: Feedburner has now rolled this change back. Nevertheless, it remains a valid thing for any feed to do, if unusual, so something podcast apps should be prepared to handle.]

Today I noticed Tim Pritlove tweet about an important Feedburner change in how episode URLs are published, which has caused playback errors with several podcast clients. Here’s an overview, technical detail to follow.

Player FM is back to normal now is all you really need to know.

Feedburner’s URL switch

It’s still not clear when they made the switch, but at some recent time, Feedburner started publishing scheme-less URLs. So instead of http://example.com, it would just be //example.com. Scheme-less URLs are perfectly valid for links on a web page, and simply mean “use the same scheme (e.g. http or https) as the current page”. Here’s an example – //google.com. I linked to //google.com, which the browser interprets as http://google.com since this blog is on a http URL. If you were instead reading this on Player FM’s website, which is now all SSL, the same link would go to https://google.com.

It’s possible other feeds use this standard too, but I’ve never come across it.

This works less well with feeds because many feed parsers don’t know about this standard. So they just save the URL as “//example.com”. Then when the episode is later downloaded or played by an app, the app also doesn’t know about the standard, and even if it did, might be detached from the original feed URL. So the app tries to download or play a nonsense URL, frustration ensues.

Impact on Player FM

Player FM is indexing approximately 9000 feedburner URLs. Of those, about 3000 were affected by this, judging by their URLs.

First step was to update TestData, the open-source project I use to publish configurable test feeds. I patched it to allow the scheme to be configurable. With that done, by passing in an empty media_scheme parameter, I could simulate Feedburner’s scheme-less URLs and get some test coverage for the subsequent fix. Example.

For the fix, I considered forking Feedjira, the Ruby feed parser, to deal with scheme-less URLs, but in the interests of a quick fix, I instead opted to just post-process its feed parsing. So after it parses the feed, some code will translate any //example.com URLs to the proper URL based on the scheme of the feed they’re contained in.

Once fixed, I’ve ensured feeds are re-fetched via Sidekiq (the message processor, so it will happen quickly and in parallel). The fetches were queued up with priority given to the most subscribed feeds, so for almost all users, it would only take about 10 minutes for feeds to be back to normal. The only delay after that is dependent on phone settings, ie how long until an update occurs. For the website, the re-fetches bust caches so that web pages were immediately fine again (as Player FM is SSL and these episode URLs weren’t, those episodes wouldn’t play until the re-fetches happened).

Although this was short notice, the good news is this URL format could theoretically be used by other feeds too. So the update today should help the feed crawler to be one notch more compatible with the universe of podcast feeds out there.

Player FM on the new Samsung Gear S

Here’s Player FM’s newest device integration. The Samsung Gear S smartwatch rolls out to the public today and the lucky new owners will be able to integrate with Player FM from the get-go.

If you have Player for Android installed, the connected watch will notify you of your favorite new episodes and let you play them immediately or add them to your Play Later list — screenshots below. As always, you can manage your notifications from Player FM side menu > Settings > Update Alerts. There you’ll be able to set the frequency of updates and rest assured you’ll only be notified about new episodes in shows you’re subscribed to.

Another way to stay informed while you’re on the move!

Player FM Website Updates

Here are some recent tweaks to make episode browsing easier and bring in functionality that’s been on Android for a while now. Channels now have a floating control bar to put all actions together in one place, with each section upgraded.

Episodes may now be sorted by newest, oldest, shortest, longest and random (shuffle), as on the Android app:

This applies to series too; reverse sort is especially useful for audiobooks and TV show recaps. As well as catching up on a series from the start.

Back to the channel view, series sorting is now clearer and sorted by default to show latest updated series.

The new channel info view shows is where you’ll find ownership details, stats, and links to the channel in various formats. RSS is useful if you want to subscribe to your Player FM subscriptions channel on another platform (e.g. iOS).

And it’s all available on mobile web of course:

I hope you find these useful, whichever direction you want to browse in!

Player FM 2.0 for Android

Just ahead of Player FM’s appearance at Google IO this week, we’ve gone 2.0! The app’s been brewing away in the beta community for some months now and I’m glad it’s finally out there in the hands and ears of all users. The app is a major user-interface upgrade as well as several important new features; read on for details.

Recapping Player FM 1.0 to 1.5

Before detailing the new 2.0 features, let’s quickly recap the journey from 1.0 to 1.5.

Player FM 1.0 launched a year ago with some new ideas that got people’s interest, particularly the focus on discovery and cloud-syncing between web and app. But it arguably launched a couple months too early, leaving a lot of core podcatcher features for later. We’ve been working hard to fill those holes over the past year, while simultaneously improving the user-interface and server quality.

Those features now in the app include: Search (with full episode search as well as series); OPML import; saving play position and marking-as-played; downloading older episodes; multi-selection; tablet and Chromecast support; and various settings to fine-tune the experience.

And now we’re 2.0: First, user-interface

Feedback for Player FM’s overall structure has been positive, so we’ve retained the core structure, but the appearance of each screen is substantially modified and in some cases completely redesigned, as you’ll see in the following screenshots. Plus, new logo.

(or view the album)

We’ve also emphasised animations much more – you’ll notice play button animating while it’s playing and paused, and 3D flipping between series and episode mode. We won’t do animation “just because we can”, but where it helps to convey meaning, we’ll take full advantage of it.

A few user-interface features are worth noting here:

  • Navigation Drawer (side menu). The dropdown is now exclusively for you to manage your own shows – that is, the shows among your subscriptions, the topics you’ve starred, and the episodes you’ve saved for later. Other sections, such as Downloads and Discover, are now in separate areas and reachable from the new side menu.
  • Sliding Tabs. The shows dropdown menu may optionally be replaced with a sliding tabs interface. That’s actually the default for users with only a handful of favorite channels, and also on tablets.
  • Full Screen Player. For easy control while moving around, there’s a new full screen player. It’s also where you’ll find the new playback-related features mentioned below. The full-screen player replaces the previous 2-row expansion mode for the mini player. It now expands to full height.
  • Mini Player. The Element Formerly Known As the Permaplayer now takes on the more familiar form of a typical app mini-player, with a big, convenient, play/pause toggle on the bottom right and also showing the current episode title. An important feature here is the slider, which is now full screen width and always present. Not something music players typically need, but important for podcasts which may be an hour or longer and sometimes in need of skipping around.
  • Episode Card View. Episodes may now be presented as graphical cards instead of the usual list view. We’re also planning a more text-based episode list view which will include some of the shownotes.
  • Chromecast Improvements Chromecast now resumes after a disconnect and supports multiple devices controlling the stream at the same time.

More than looks: Introducing a playback speed control

Player FM’s goal is already to save you time reading, and now you can save time listening too with the new speed control. It’s perfectly possible to play some podcasts back at fast speeds, even double-speed or more, and still understand it. You can also slow down speech, to 0.5x, for language learning and podcast announcers who may have drunk too much coffee. BTW since people have asked me, no, it doesn’t sound like high-pitched chipmunks :). Just regular speech, but fast. You’ll find the new speed control in the full-screen player.

Sleep timer

A common request in Player FM 1.x was a sleep timer and now it’s here. You can kick it off from the full-screen player and the mini player also includes the indication once it’s set. I never thought I’d be so happy to make an app that puts it’s users to sleep, but there it is.

What’s next

On the bottom of the full-screen player is a new and novel feature we’ve added to help you control the continuous-playback experience. What’s Next shows you, well, what episode is next, but you can also tap on it to get more options. There you can toggle continuous-play on and off; but more interestingly, you can switch the current playlist. So if you start an episode in Play Later, that would be the current playlist. You can now switch the playlist from What’s Next and decide whether to play right away or play when the current episode has finished. What’s Next is where you’ll start to see more intelligent recommendations over time.

Play history

A separate section in the navigation drawer is play history, showing you episodes you’ve previously played and including those that are mid-play.

Feedback is always welcome

I hope you find these updates useful. Please send feedback to mailto:android@player.fm and join the Google+ beta community for early updates as we begin the march to 3.0.

Feed Fetching Improvements

I’ve made several improvements to feed fetching lately, making podcasts update faster, more reliably, and more accurately. Read on for the details.

First and foremost, feeds are once again based on push notifications, meaning they will typically enter Player FM’s database within seconds of content providers hitting the Publish button. Or in the worst case, a few minutes later, capped at 15 minutes. Player FM’s new user-interface – soon landing with v2.0 – now supports pull-to-refresh, so this goes well together as you will frequently find fresh fodder when episodes are coming in every few seconds.

Second, the “plan B” polling solution is now more efficient, so if the aforementioned push process breaks down, feeds will still be updated within about 2 hours.

Third, if the publisher edits an already-published post, Player FM’s index will update with the new details. This was a pain point for some time, and at various times users mailed me about a missing episode or a title discrepancy. It took a while to fix mainly because there was a risk it would break caching, causing excessive bandwidth and battery use for mobile users. But it’s now been done, and efficiently.

Finally, no more duplicate episodes. Occasionally there were race conditions which caused the same feed to be indexed twice. This in itself was very rare, but the problem was that if it did happen, the duplicate would never be cleared. Now it should not happen at all, but even if the universe conspires for it to happen, the duplicate will be gone upon the next fetch a few hours later.

How feed fetching got slower and then faster

As further technical info, the performance of feed fetching regressed a couple of weeks ago. After some investigation, I found a couple of causes:

  • Push notifications from Superfeedr had failed because I made the site full-TSL (aka SSL, ie https://player.fm/* and http://player.fm no longer works). This point warrants a separate post later on, but suffice to say, it broke push notifications. I did actually have Superfeedr set up to push to the (valid) https address, but due to what appears weirdly to be a core Ruby library bug, it was not used and the original — now defunct — http URL was used instead. The http URLs do redirect to https, but the client wasn’t following redirects (it would not be common to follow redirects from a POST request anyway).
  • The “plan B” had its own problems due to a library that wasn’t thread-safe. It was causing background jobs (ie feed fetching) to fail out. Having pinpointed that library, I made a patch to use an alternative library instead and this is working smoothly now.
  • Lack of information. As a meta point, I didn’t have enough visibility on what was happening. I’ve now built some RESTful services to expose statistics and furthermore, services to verify how many episodes are being generated. If — in any given hour — that number falls short of a threshold, I get notified.

Website Update, May 2014

A website update today makes Player FM faster and easier to navigate. Let’s start with the desktop version …

Player FM Remix

The first thing eagle-eyed readers will notice is the new logo. Hope you like it :). Beyond that, what you can see is the new side menu, a simple list of categories and sub-categories. No more jumping through a cascading menu, cleaner to look at, and much simpler on touch devices.

The category in question is Science, and just like the old menu, you can dive into any specific topic. What’s new here is the episode list. Before, top-level categories were just categories; now they are much more like regular channels as they have their own playlist, an aggregation of all episodes in all channels. This new feature is possible for both top-level categories and sub-categories. I expect in the future it will become more intelligent and (at least, by default) filter episodes depending on popularity and relevance to the high-level topic. But for now, it’s one with the lot.

Got it? Now let’s go mobile …

Player FM Mobile Remix

The mobile website is now more app-y and easier to navigate. The previously – frankly buggy – top player is now fixed and the whole thing loads much faster.

There’s still work to do to incorporate logging in and search into the mobile app, and more performance improvements are coming.

Design, Develop and Distribute: Player FM at Google IO Developer Sandbox

It’s official: Player FM will be participating in the Developer Sandbox at Google IO 2014.

The IO Developer Sandbox showcases demos from a wide range of developers who have built applications based on technologies and products that are featured at I/O. We have been invited to demo our application and connect with I/O attendees developer-to-developer to answer questions and exchange ideas.

The conference theme is Design, Develop and Distribute, and we’re looking forward to sharing lessons and swapping war stories with developers about all of these activities. Here’s a quick overview on how we approach each of them — find us at IO or drop me a line to dive deeper on these.

Design

The most enlightening thing about IT in the past 10 years has been design getting the recognition it deserves. Certainly, it’s about visual design that delights users, but much more than that, it’s about interactions, features, and concepts that make apps “just work”.

Player FM plays podcasts. That’s hardly a new concept, so why make a YAAP (yet another podcast player). Yes, design! The fundamental problem a podcast app must solve is “How do I listen to topics I care about?”. New Player FM users immediately choose favourite topics, even if they don’t know any individual shows, and they’re immediately listening to them. Meanwhile, the app syncs recent episodes for offline use, saving them the hassle of download management.

Notice I didn’t mention anything about colors, gradients, or round corners above. Visual design matters more than ever, but it’s the price of entry for a consumer-facing app. On Player FM’s Android app, we do spend a lot of time iterating on visual designs and sweating each pixel, but ultimately the big challenge is moving forward while balancing simplicity and control.

Develop

As a small operation, developer productivity and server efficiency matter a lot. Not only does it keep costs low, but it lets us learn from feedback and iterate quickly.

Continuous integration is one of the ways we achieve this on the server. Automated testing gives enough confidence to release several times a day. Server configuration management via Ansible (similar to Chef and Puppet) lets us scale and load-balance for better uptime. Leveraging open source and contributing back helps us add new features quickly. As for the API consumed by the mobile app, we take advantage of REST and HTTP to keep bandwidth and battery usage as low as possible.

Developing’s all about the detail, so there’s not much more to say in this short summary, but I’ll be happy to talk code at IO and might even open up a Vim session or two :).

Distribute

Users experience Player FM on both Android and the web, and each has their own distribution strategy.

On Android, we make use of Play, receive valuable feedback, and are pleased to be able to comment and follow up on reviews. We also benefit from the private and staged rollouts which were announced at none other than Google IO last year. Oh and you should totally join our pioneers community for early access to features! Including the speed control that will be part of the imminent Player FM 2.0 release.

As for the web, that’s where Player FM initially launched. One of the goals was to make podcasting more accessible to a broader audience, people who have never considered installing a podcast app. Maybe people who’ve never installed any app at all. HTML5 provides a powerful platform for delivering an app-like experience without any installation process. And Chrome’s developer tools help to debug and improve the website. I’m currently working to improve the mobile web experience, and it’s amazing how much easier it is to do this now, compared to just a couple of years ago.

See you at IO

Google IO attendees work on many and varied applications, yet many face similar challenges and stand to benefit from similar solutions. Hopefully the above points give you a sense of how Player FM approaches design, development, and distribution, and we’re looking forward to sharing more about them at next month at IO. Visit the Sandbox page to learn more.

Feed quality improvements

After the recent upgrade (2 weekends ago), I’ve been able to start enhancing the server again. If I’ve done it right, you won’t notice the latest changes because they are more about Doing It Right than adding any new features.

If you’re interested in the specific changes:

  • Most importantly, old episodes are updated if the publisher makes a change. This sounds like it should have been easy to do and it was if I ignored efficiency. The reason it wasn’t done before was I was worried small changes would bust caching and cause a lot of unnecessary re-fetching. That’s been addressed here, so the whole process is efficient.
  • Changes to the main series data (such as author info) is immediately propagated to containing channels. Before it would only propagate when an episode update triggered it.
  • No more duplicate episodes! This bug previously happened in a condition when the same series is fetched twice, which in itself was rare and wouldn’t have been much of a problem in itself. The main problem was the old episodes would never go away after that one rare occurrence. Now duplicates are purged at the end of each fetch.
  • Images are now drawn from sources other than the iTunes image declaration. Some feeds include images in a separate tag that would be similar to how a normal blog would provide images. Those are now indexed in the event iTunes image isn’t included. Thanks to Lonely Bob for pointing out some feeds like this.

I’ve also begun work on an open-source project to help with processing grunt-work like this.

Get your cast on: Player FM 1.5 does Chromecast

Chromecast Podcasts

Bit exciting to announce Chromecast support for Player FM’s Android app.

Go yoink Player FM now from Play and get casting

As I mentioned the other day, I was skeptical about how much value TV playback has for audio podcasts, but there was substantial demand and after testing this for several weeks, it gets solid usage. We were going to make it happen after 2.0, but given demand, we decided to get this into a 1.5 release.

While Chromecast today works with TVs, it’s not hard to imagine a future of Chromecast-enabled speakers and other appliances. You can also imagine Chromecast being a convenient standard for airplane screens, treadmills, cars, and so on. I’m glad Player FM is able to take a first step in that direction.

Image via PlaceIt

New Channels on Player FM

If you visit Player FM today, you’ll find a number of enhancements to the channel organisation. As always, channel organisation updates are motivated by helping users find shows on their favorite topics. Better organisation also means a simpler out-of-the-box experience when the app suggests topics to new users. Not all shows are discovered via the catalogue of course. Many users will search and there are increasingly intelligent recommendations available. But for the times when users do peruse the catalogue, it should be the most logical organisation possible.

So here are the updates …

Goodbye Lifestyle. Hello everyone.

First up, I’m excited to say we’ve replaced the “Lifestyle” channel with three new – and more focused – channels:

  • True Stories Fans of shows like This American Life and The Moth will be familiar with the fascination of everyday stories presented the right way. Here you’ll find a range of shows in that ilk.
  • Factoids There’s more than a few podcast listeners who enjoy learning new things every day. And fortunately there’s more than a few podcasts catering to their needs with daily explainers and amazing trivia. As I write this, you can learn about chocolate chips, tattoos, and the electric car from the most recent episodes! Factoids is like Reddit’s Today I Learned for your ears.
  • Pop Culture Pop culture’s a hard topic to define neatly, but you know it when you hear it. Shows here cover the memes of the day and how people reacted to them. Whether they be movies, TV shows, or headline news items.

Lifestyle was always an ambiguous mash of these and it made a lot of sense to separate them, as you can tell from the descriptions. Behind the scenes, Lifestyle has simply been renamed to True Stories, which means people previously following Lifestyle are now following True Stories.

A community channel: LGBT

Next, another addition to Society and Culture is a new LGBT channel featuring podcasts for the Lesbian, Gay, Bisexual, and Transgender communities. Podcasts are a great medium for perspectives and news relevant to specific communities. I also recognise LGBT itself is a broad brush and we may well build channels for specific communities within it, and we’re likely to be considering channels for other communities too in the future.

Language Learners, Konnichiwa!

Several new language learning channels are now available:

These join the existing collection of English, French, Italian, Spanish, Portuguese, Chinese, and Korean. With special thanks to Jon for prodding me to publish Japanese, which had been in draft mode.

Some departures

We’ve removed the various “Featured in [Category]” channels as they were an experiment that hasn’t panned out so far. I’m certainly interested in more effective ways to recommend the best shows in each category, but I think we can do it in a better way than just picking a few popular series. To be continued.

Business channel is also gone from the catalogue. There’s already Business News, MBA, Trading, Entrepreneur and a bunch of other business related channels. Business was too general, and people tend not to be interested in “general business” in the same way they would be interested in “general science” for example. Followers of Business are still following Business and we’ll keep it updated for now, but longer term it will be flagged as discontinued.

Category Renames

Some categories and subcategories have been renamed to avoid confusion. For example, we previously had Tech channel (general tech shows) in Tech category (which also contains Gadgets, Android, Apple, etc). It’s now Technology. Several other changes like that should hopefully make it clearer for users.

Separating Food from Drink

Cooking is renamed to Food and moved to Health and Well-Being. Wine somehow didn’t seem right in there, so it’s moved to Recreation, where it may soon be joined by some similar channels …

Brace yourselves: The channels are coming

It doesn’t stop here. A number of long-term planned channels are still coming, and we intend to shuffle the categories a bit more too. Furthermore, the next major development project on the server is to update how catalogues are organised, which will in turn lead to a much better experience for browsing shows on both web and mobile. Indeed, it’s been far too long since I last posted here and I’ll be rectifying that next week with a broader update about these – and other – development activities happening.