Apps for Making Music

Transforming musical expression
Founder and Chief Product Officer of Sonation

Can we create an app that makes you feel like a god when you play or sing by transforming recorded music into a living, breathing, intelligent partner that senses your musical expression?

iOS App

Cadenza

As a founder of Harvard Innovation Lab startup, Sonation, I brought together a team to create apps for musicians to live their dream: become the star of a musical experience revolving around their own, real-time performance, accompanied by some of the greatest music ever created.

Cadenza, our first app, listened to musicians and, using predictive algorithms, accompanied them with a real orchestra, band, or piano recording synced with their own expressive performances as they happen. Then, using machine learning, Cadenza learned each user’s style to be better able to predict their performance nuances over time.

Watch Cadenza adapt to these very different musicians in real time.

This montage shows different musicians each playing the same piece of music. Each one was originally recorded without accompaniment, but here Cadenza listens to them, accompanies, and even detects and recovers from their inevitable mistakes.

My roles
  • Product owner—manage design, development
  • Create user flows, wireframes
  • Build and test prototypes
  • Oversee testing, app store rollout, support
  • Oversee content production, including music recording
  • Establish and maintain artist relationships
  • Create pitch decks
  • Orchestrate, arrange music
  • Founder
Accomplishments
  • Created apps loved by musicians and singers from the highest level to beginner
  • Harvard University Entrepreneurship Challenge Winner, 2013
  • MassChallenge Finalist, 2014
  • Raised over $900,000 seed funding
  • Best New App, Apple, 2015
  • Over 9,000 monthly active users, 2016
A centuries-old problem:

How can musicians turn pages
without interrupting their playing?

Problem

When musicians read music they can’t interact with apps. Their hands are already full, but they have to turn pages, too. For centuries this need was partly addressed partly by clever printing and layout tricks. More ambitious solutions have been tried, included pedal-powered contraptions and even employing a real person just to turn pages for another musician (a practice still in use now).

What’s the Ideal Experience?

See the right music at the right time, without requiring any interaction (foot pedals, taps, gestures) and without making musicians bend to fit into pre-recorded page-turns. Free the musician to focus on making music. Eliminate page-turning anxiety.

Competitive Analysis

Most solutions replicate those of the pre-digital era. Instead of reaching out to turn a page, they have you merely reach out to tap a screen. That’s not an improvement. Slightly better: press a BlueTooth pedal to turn the page onscreen. These solutions are not ideal.

Solution We Created

  1. Use the microphone to know where the musician is at all times. Are they coming to the end of the page?
  2. Musicians need to see what’s coming up, not just what notes are being played now—without obscuring current content. So, let them see what’s coming up without obscuring what they’re playing now.
  3. Use a “peek-and-slide” pattern. Show some of the upcoming content before removing what’s currently onscreen. Then, at the right moment, unobtrusively replace the current screen with the next chunk (page) without drawing any attention away from reading and playing music.
  4. The timing of page turns is determined by the user’s real-time speed of playing.
  5. Even better: Dynamically form page boundaries according to display size, orientation and zoom level to optimize legibility.
  6. Preserve the artful and information-rich musical notation graphics at the highest level. Don’t sacrifice that quality, elegance and clarity for functionality. Use real musical scores from printed editions.
  7. Prevent any of the notated music from being cut-off in any view, at any size or rotation, automatically. Musicians can’t stop to re-fit a page of music to their screen.