Part four of our music series builds on the previous three articles by giving examples of music-related mobile phone content and services. It also outlines the audio support available for Sony Ericsson's Java ME and Symbian OS mobile phones.

In case you missed them, part one in the series presents the key stakeholders within the music business, part two introduces data compression, bandwidth management and a technical overview of audio codecs and part three outlines licensing, copyright ownership and DRM:

• Part 1: Understanding the players, market trends and service definitions>>
• Part 2: Understanding codecs and bandwidth issues>>
• Part 3: Understanding licensing and the importance of DRM>>

Introduction

Evidence suggests that twice as many mobile phone users listen to music on their mobile phones than on dedicated digital music players, but that's only part of the story. Because Sony Ericsson mobile phones are flexible devices that support added functionality through Java ME and Symbian OS-based applications, developers can offer rich, personalized audio and music-related experiences to end-users. This can range from games that allow players to create customized background music from their personal music libraries to pronunciation guides in mobile dictionaries or language courses, and audio instructions for mobile navigation applications.

Music-related services, content and applications are fundamental parts of Sony Ericsson's overall content strategy. Even though the Walkman portfolio is the main focus of Sony Ericsson's music activities, music solutions for our other series of mobile phones are just as relevant. Network limitations, product market reach, end-user choice and model-version exclusivity to certain operators mean that we need to make sure that the music experience is as good as possible across the entire product portfolio.

Let's take a look at some recent market developments for music-related mobile content.

Sony Ericsson music service partnerships

Sony Ericsson has partnered with Gracenote to provide a music recognition service free to Walkman phone users. The TrackID feature is available on the W850, W710 and W950 models. The TrackID application records audio from the microphone or the built-in FM radio and then submits that audio sample to the TrackID service. The Gracenote service compares a "musical fingerprint" generated from the audio sample against an enormous database of fingerprints containing 2.5 million songs which covers the global music market. The fingerprint is matched and the listener is sent the song name and artist info. Ty Roberts, the Co-founder and CTO of Gracenote says the service was developed in part because, "we see the mobile phone becoming a superior music playback device and our overall goal is to design technology that makes it just as easy to manage, enjoy and discover music on a phone as on a PC." TrackID is a prime example of the importance of partners in the creation of innovative mobile music applications. The core technology, an audio waveform signature analysis and recognition technology, was originally developed by Philips Research and later acquired by Gracenote. It was incorporated into their offering with the search database system of music fingerprints. Roberts says that it was the partnership with Sony Ericsson which really took the technology to a new level. "Sony Ericsson connected us with Teleca who was the application developer on this project. Teleca built the supporting infrastructure, while Sony Ericsson developed the handset application. We altogether did the fantastic implementation you see today on Sony Ericsson's W850 and W710."

W850 with TrackID.

M-BUZZ is a new promotional space on the web launched in October 2006 which allows Sony Ericsson to showcase emerging artists and their music so people can get to know them before they have been signed by a major label. Through an agreement with Sony /ATV Music Publishing to source music, video and other relevant artist-related content for M-BUZZ, Sony Ericsson distributes selected content to mobile phones via its PlayNow over-the-air (OTA) download service, giving unknown artists access to a mass audience through Sony Ericsson mobile phones and the web.

Mikael Nerde, Head of Developer Program & Content Planning, commented, "M-BUZZ is important for the traditional music industry, but it also shows developers that Sony Ericsson is serious about bringing mobile music to the masses. Not only through joint operator promotions of major mainstream artists like Christina Aguilera and Robbie Williams but now also by promoting unknown bands and up-and-coming artists. Developers should ride on the wave and create music and audio related apps that can be used by consumers to take the mobile music experience beyond just listening to music tracks."

Making these applications music agnostic, i.e. not base them on a specific music content property, will prevent you from limiting your application to specific music tastes or user groups and avoid licensing issues of music properties. These music and sound related applications may eventually be distributed through a number of different channels; Sony Ericsson Fun & Downloads, PlayNow or the Application Shop, operator channels or independent software sites.
 
Innovative applications available now for Sony Ericsson phones

A number of applications that incorporate audio- and music-related features in innovative ways are already available for Sony Ericsson phones. Two embedded applications provided by Sony Ericsson are Music Mate, a program that teaches guitar chords, found on the W810 Walkman phone, and QuadraPop, a Tetris-like game with colorful musical notes, available for many Sony Ericsson mobile phones.

A few notable examples available for Sony Ericsson mobile phones from third-party developers are described below:

The Audible Player is currently available for Sony Ericsson's UIQ 3-based P990 smartphone, M600 messaging device and W950 Walkman phone, and delivers digital audio books and programs from Audible, the Internet's leading provider of spoken audio entertainment, information, and educational programming. It allows you to listen to books (e.g. a chapter of the latest best seller like the Da Vinci Code), news readings from magazines, newspapers (like The Wall Street Journal, Financial Times or Die Zeit) and more from your phone. More information>> 

Bokilur is Swedish for "book in phone" and the company offers exactly that: streaming audiobooks for mobile phones. Customers can download and install a piece of Java-based software that's compatible with over both 3G and GPRS mobile phones. End-users can then use the software's interface to browse available titles, and listen to five minutes of each book for free, before deciding to purchase and download. Besides audiobooks, Bokilur has recently started offering comic books. A smart move, considering the natural fit between a comic's image panel and a mobile phone screen. Find out more here (in Swedish)>>

Developed by Smart Mobile Solutions, RadioBee is an internet radio player for Java ME mobile phones that allows end-users to listen to any MP3 internet radio through the phone's speaker. This project is still under development with the alpha version available for download from RadioBee's website>> 

Another radio application is VirtualRadio, a streaming MP3 web radio player available for Sony Ericsson's P800, P900 and P910 UIQ 2.x-based smartphones. VirtualRadio plays a given set of Web radio stations from all over the world: just start up the player and it immediately starts playing audio, with no need to define play lists or search for URLs. More information here>>

Audio support in Java ME and C++

Developers have the option of working in two programming languages for Sony Ericsson mobile phones: Java and C++.  Java ME is supported on most mass-market phones and both Java ME and C++ are supported on Symbian OS-based phones.

For development of media applications, the C++ development platform provides the MMF (Multi Media Framework). The MMF client API implements several interfaces that encapsulate a lightweight plugin framework for manipulation of audio and video features.

Java ME can be used to create applications handling various media-oriented tasks, for example, connection to networks (global or local), downloading and playing audio and video. The Sony Ericsson Java ME Developers' Guidelines gives a number of examples of these features and how to implement them in Java applications for Sony Ericsson phones.

Below you'll find a list of audio features supported on Sony Ericsson phones, which may help stimulate some ideas among software developers to come up with new music and sound related applications.
 
Audio support in Sony Ericsson Java ME mobile phones

Sony Ericsson facilitates porting of applications to many different phone series using its successful Java Platform strategy. Each backwards compatible platform version is used for a number of phone models, delivering consistent Java implementations which help developers effectively create applications for a range of phones at the same time.

Java Platform 7 (JP-7) is the most recent and includes the W850 and W710 Walkman phones. Examples of Walkman phones that support JP-6 include the W810 and W550 and JP-5 includes the W800. An overview of JP-7 can be found here>>

Here's an overview of the audio controls supported by Java ME phones in the context of Java Platforms:

• VolumeControl
• ToneControl
• StopControl
• MIDIControl (JP-5, JP-6 and JP-7 only)
• PitchControl (JP-6 and JP-7 only)
• TempoControl (JP-7 only)
• RateControl (JP-7 only)

Included as of:
JP-5: Play two audio files at the same time (one midi and one wav).
JP-6: Progressive playback from file system.
JP-7: Progressive playback from the web.
 
Streaming support:
JP-4: Real-time Streaming (RTS) protocol is supported which will start streaming sound within seconds.
JP-6: The playback from the file system will start within a second or two.
JP-7: When streaming from the web, a buffer is first created before the playback is started.
 
Base level of audio support in all Sony Ericsson phones:

• audio/midi: MIDI (GM, GML and SP-MIDI)
• audio/x-wav: WAV (PCM)
• audio/x-tone-seq: JSR 135  Mobile Media API tone sequence
• audio/mpeg: MP3 (MPEG-1 layer 3, MPEG-2 layer 3, MPEG 2.5 layer 3)
• audio/imelody: iMelody
• audio/amr:AMR
• audio/mp4a-latm: 3GP (MPEG-4 AAC LC) 

For some descriptions and explanations of these formats and codecs, read "Getting the hang of the music industry Part 2: Understanding codecs and bandwidth issues">>

Audio support using C++ in Sony Ericsson Symbian OS phones

There are a number of different classes available for playing, recording and conversion of audio in C++ applications for Sony Ericsson's UIQ 3-based phones (P990, M600 and W950).

Playing: CMdaAudioPlayerUtility provides a simple interface to open, play and obtain information from sampled audio data. The audio data can be supplied either in a file, a descriptor or a URL. One significant difference between this class and CMdaAudioRecorderUtility (that also provides play methods) is that multiple audio files can be played within a single instance with this class.

The Audio Tone Player provides a simple interface for tone generation (synthesized sounds), enabling the playing of tone sequences and DTMF (Dual-Tone Multi-Frequency) strings. The tone player functionality is provided by the CMdaAudioToneUtility class and when using this class you can play:

• Single tones of a specified duration and frequency
• DTMF strings
• Sequences of tones held in files or descriptors
• Predefined (fixed) sequences of tones stored on themobile phone

MIDI playing: CMIDIClientUtility provides an interface to open, play and obtain information in MIDI format. The MIDI data can be supplied either in a file, a descriptor or a URL.

Recording: CMdaAudioRecorderUtility provides features to record audio clips to files, descriptors or URLs. It also provides methods for creating and manipulating meta data entries in the newly recorded clips.

Methods are provided in CMdaAudioRecorderUtility for both the recording and playing of audio data. This class is intended primarily for recording audio data and the playback methods have been included for convenience sake. If you just need to play back an audio clip, you should use CMdaAudioPlayerUtility instead as it provides more control over playing and does not include the overheads that are necessary for recording.

Conversion: CMdaAudioConvertUtility provides features to convert audio clips between different formats.

Base level of audio support in all UIQ 3 phones
All of the audio classes are plug-in based, leaving the list of supported audio formats for input and output open ended. The audio file formats supported as standard by the MMF are: AU, WAV and raw audio data. Each file format may support one or more compression algorithms (codecs). The standard file formats and corresponding codecs are shown below:

Audio file format	Codecs supported
AU	A-Law, mu-law, signed 16 bit PCM big-endian, signed 8 bit PCM
WAV	IMA-ADPCM, A-Law, mu-law, unsigned 8 bit PCM, GSM 6.10, signed 16 bit PCM
Raw data	Signed big-endian 16 bit PCM, signed 16 bit PCM, unsigned 16 bit PCM

Music inspired animations and graphics

The extremely popular Themes Creator tool provided for free by Sony Ericsson Developer World can be used to create music-inspired themes, also incorporating sound, for both Java ME and UIQ 3-based phones. Most Sony Ericsson phones also support Scalable Vector Graphics (SVG Tiny), which can be used to create animated graphics to support various music genres and consumer segments etc. Things like this are very popular among the youth generation.

More information:

• Getting the hang of the music industry:
  • Part 1: Understanding the players, market trends and service definitions>>
  • Part 2: Understanding codecs and bandwidth issues>>
  • Part 3: Understanding licensing and the importance of DRM>>
• Mobile Music applications developer resources>>
• Sony Ericsson Music and Video Developers' Guidelines>>
• Themes Creator tool>>