feat: first? iteration of the piano tutorial

packages/audiodocs/docs/fundamentals/lets-make-some-noise.mdx

@@ -108,3 +108,17 @@ import LetsMakeSomeNoiseSrc from '!!raw-loader!@site/src/examples/LetsMakeSomeNo
 In web environment you can use `decodeAudioDataSource` directly on the asset url, without the need to download it first.
 
 :::
+
+## Summary
+
+In this guide, we have learned how to create a simple audio player using `AudioContext` and `AudioBufferSourceNode` as well as how we can load audio data from a remote source. To sum up:
+
+- `AudioContext` is the main object that controls the audio graph.
+- `decodeAudioDataSource` method can be used to load audio data from a local audio source in form of `AudioBuffer`.
+- `AudioBufferSourceNode` can be used to any `AudioBuffer`.
+- In order to hear the sounds, we need to connect the source node to the destination node exposed by `AudioContext`.
+- We can control the playback of the sound using `start` and `stop` methods of the `AudioBufferSourceNode` (And other source nodes which we show later).
+
+## What's next?
+
+In [the next section](/fundamentals/making-a-piano-keyboard) we will learn more about how the audio graph works, what are audio params and how we can use them to create a simple piano keyboard.

packages/audiodocs/docs/fundamentals/making-a-piano-keyboard.mdx

@@ -0,0 +1,362 @@
+---
+sidebar_position: 3
+---
+
+# Making a piano keyboard
+
+In this section, we will use some of the core audio api interfaces to create a simple piano keyboard. We will learn what is an `AudioParam`, how to use it to change the pitch of the sound.
+
+## Base application
+
+Like in previous example, we will start with simple app with couple of buttons so we just don't need to worry about the UI later.
+You can just copy and paste the code below to your project.
+
+```tsx
+import React from 'react';
+import { View, Text, Pressable } from 'react-native';
+
+type KeyName = 'A' | 'B' | 'C' | 'D' | 'E';
+
+interface ButtonProps {
+  keyName: KeyName;
+  onPressIn: (key: KeyName) => void;
+  onPressOut: (key: KeyName) => void;
+}
+
+const Button = ({ onPressIn, onPressOut, keyName }: ButtonProps) => (
+  <Pressable
+    onPressIn={() => onPressIn(keyName)}
+    onPressOut={() => onPressOut(keyName)}
+    style={({ pressed }) => ({
+      margin: 4,
+      padding: 12,
+      borderRadius: 2,
+      backgroundColor: pressed ? '#d2e6ff' : '#abcdef',
+    })}
+  >
+    <Text style={{ color: 'white' }}>{`${keyName}`}</Text>
+  </Pressable>
+);
+
+export default function SimplePiano() {
+  const onKeyPressIn = (which: KeyName) => {};
+  const onKeyPressOut = (which: KeyName) => {};
+
+  return (
+    <View
+      style={{
+        flex: 1,
+        justifyContent: 'center',
+        alignItems: 'center',
+        flexDirection: 'row',
+      }}
+    >
+      {Keys.map((key) => (
+        <Button
+          onPressIn={onKeyPressIn}
+          onPressOut={onKeyPressOut}
+          keyName={key}
+          key={key}
+        />
+      ))}
+    </View>
+  );
+}
+```
+
+## Create audio context and preload the data
+
+Like previously, we will need to preload the audio files in order to be able to play them. Using the interfaces we already know, we will download them and store in the memory using good old `useRef` hook.
+
+First comes the import section and list of the sources we will use, also lets help ourselves with type shorthand for partial record:
+
+```tsx
+import {
+  GainNode,
+  AudioBuffer,
+  AudioContext,
+  AudioBufferSourceNode,
+} from 'react-native-audio-api';
+import * as FileSystem from 'expo-file-system';
+
+/* ... */
+
+type PR<T> = Partial<Record<KeyName, T>>;
+
+const sourceList: PR<string> = {
+  A: 'https://software-mansion-labs.github.io/react-native-audio-api/audio/sounds/C4.mp3',
+  C: 'https://software-mansion-labs.github.io/react-native-audio-api/audio/sounds/Ds4.mp3',
+  E: 'https://software-mansion-labs.github.io/react-native-audio-api/audio/sounds/Fs4.mp3',
+};
+```
+
+then we will want to fetch the audio files and store them. We want the audio data to be available to play as soon as possible,
+so we will use the `useEffect` hook to download them and store in the useRef hook for simplicity.
+
+```tsx
+export default function SimplePiano() {
+  const audioContextRef = useRef<AudioContext | null>(null);
+  const bufferMapRef = useRef<PR<AudioBuffer>>({});
+
+  useEffect(() => {
+    if (!audioContextRef.current) {
+      audioContextRef.current = new AudioContext();
+    }
+
+    Object.entries(sourceList).forEach(async ([key, source]) => {
+      bufferListRef.current[key as KeyName] = await FileSystem.downloadAsync(
+        url,
+        `${FileSystem.documentDirectory}/${key}.mp3`
+      ).then(({ uri }) => audioContextRef.current!.decodeAudioDataSource(uri));
+    });
+  }, []);
+}
+```
+
+## Playing the sounds
+
+Now it is finally time to play the sounds, but still nothing new here. We will use the `AudioBufferSourceNode` and simply play the buffers.
+
+```tsx
+export default function SimplePiano() {
+  const onKeyPressIn = (which: KeyName) => {
+    const audioContext = audioContextRef.current!;
+    const buffer = bufferMapRef.current[which];
+
+    const source = new AudioBufferSourceNode(audioContext, {
+      buffer,
+    });
+
+    source.connect(audioContext.destination);
+    source.start();
+  };
+}
+```
+
+Great! But a lot of things are a bit off here:
+
+- We are not stopping the sound when the button is released, which is kind of the way piano should work, right? 🙃
+- You have probably noticed in the previous section, but we are missing sounds for keys 'B' and 'D'.
+
+Let's see how we can tackle that using the audio api. We will take them down one by one, ready?
+
+## Key release
+
+To stop the sound when keys are released, we will need to store somewhere source nodes, in order to be able to call `stop` on them later. Like for audio context lets use `useRef` hook.
+
+```tsx
+const playingNotesRef = useRef<PR<AudioBufferSourceNode>>({});
+```
+
+Now we need to modify the `onKeyPressIn` function a bit
+
+```tsx
+const onKeyPressIn = (which: KeyName) => {
+  const audioContext = audioContextRef.current!;
+  const buffer = bufferMapRef.current[which];
+
+  const source = new AudioBufferSourceNode(audioContext, {
+    buffer,
+  });
+
+  source.connect(audioContext.destination);
+  source.start();
+
+  playingNotesRef.current[which] = source;
+};
+```
+
+And finally we can implement the `onKeyPressOut` function
+
+```tsx
+const onKeyPressOut = (which: KeyName) => {
+  const source = playingNotesRef.current[which];
+  if (source) {
+    source.stop();
+  }
+};
+```
+
+And they stop on release, just as we wanted. But if we hold the keys for a short time, it sounds a bit strange. Also have You noticed that the sound is simply cut off when we release the key? 🤔
+It leave a bit unpleasant feeling, right? So let's try to make it a bit more smooth.
+
+## Envelopes ✉️
+
+We will start from the end this time, and finally we will use new type of audio node - `GainNode` :tada: <br />
+`GainNode` is a simple node that can change the volume of any node (or nodes) connected to it. `GainNode` has a single element called `AudioParam` which name is also `gain`..., a bit lame but we have to live with that.
+
+## What is an AudioParam?
+
+An `AudioParam` is an interface, that controls various aspects of most audio nodes, like volume (in `GainNode` described above), pan or frequency. It allows to control them over time, so we can make smooth transitions and complex audio effects.
+For our use case, we are interested in two methods of an audio param: `setValueAtTime` and `exponentialRampToValueAtTime`.
+
+- `setValueAtTime(value: number, time: number)` - method is as simple as it sounds, it sets the given value at the time specified in the second argument.
+- `exponentialRampToValueAtTime(value: number, time: number)` - method is a bit more complex, it changes the value using exponential curve to the value passed as first argument at the time specified in the second one. It starts as soon as preceding change is finished. (f.e. `setValueAtTime`).
+
+So, how we can use that for our piano? <br />
+The title of one of the previous sections might ring a bell 🔔. <br /><br />
+Envelope is term widely used in music and sound engineering, it describes how a sound changes over time (sounds similar to `AudioParam`, doesn't it?).
+The most common way of describing an envelope is ADSR (please don't mistake it with ASMR 🙂). This acronym stands for: **attack**, **decay**, **sustain** and **release**.
+
+- **Attack** - time it takes for the sound to reach its peak volume from the beginning.
+- **Decay** - time it takes for the sound to reach the sustain level after the peak volume.
+- **Sustain** - volume level that the sound will stay at until the key is released.
+- **Release** - time it takes for the sound to fade out after the key is released.
+
+![ADSR Example](/img/ADSR.svg)
+
+You can read more about envelopes and ADSR on [Wikipedia](<https://en.wikipedia.org/wiki/Envelope_(music)>).
+
+## Implementing the envelope
+
+With all the knowledge we have gathered, let's get back to the code. In our `onKeyPressIn` function, besides creating the source node, we will create a `GainNode` which will stand in the middle between the source and destination nodes, which will be our envelope. <br /> <br />
+We want to implement the **attack** in `onKeyPressIn` function, and **release** in `onKeyPressOut`. In order to be able to access the envelope in both functions we will have to store it somewhere, so let's modify the `playingNotesRef` introduced earlier. <br /> <br />
+Also lets not forget about the issue with short key presses, we will address that, by enforcing minimal duration of the sound to one second (As it works nicely with the samples we have 😉).
+
+First comes the types:
+
+```tsx
+interface PlayingNote {
+  source: AudioBufferSourceNode;
+  envelope: GainNode;
+  startedAt: number;
+}
+```
+
+and the `useRef` hook:
+
+```tsx
+const playingNotesRef = useRef<PR<PlayingNote>>({});
+```
+
+Now we can modify the `onKeyPressIn` function:
+
+```tsx
+const onKeyPressIn = (which: KeyName) => {
+  const audioContext = audioContextRef.current!;
+  const buffer = bufferMapRef.current[which];
+  const tNow = audioContext.currentTime;
+
+  const source = aCtx.createBufferSource();
+  source.buffer = buffer;
+
+  const envelope = aCtx.createGain();
+
+  source.connect(envelope);
+  envelope.connect(audioContext.destination);
+
+  envelope.gain.setValueAtTime(0.001, tNow);
+  envelope.gain.exponentialRampToValueAtTime(1, tNow + 0.1);
+
+  source.start(tNow);
+  playingNotesRef.current[which] = { source, envelope, startedAt: tNow };
+};
+```
+
+and the `onKeyPressOut` function:
+
+```tsx
+const onKeyPressOut = (which: KeyName) => {
+  const audioContext = audioContextRef.current!;
+  const { source, envelope, startedAt } = playingNotesRef.current[which];
+
+  const tStop = Math.max(audioContext.currentTime, startedAt + 1);
+
+  envelope.gain.exponentialRampToValueAtTime(0.0001, tStop + 0.08);
+  envelope.gain.setValueAtTime(0, tStop + 0.09);
+  source.stop(tStop + 0.1);
+
+  playingNotesRef.current[which] = undefined;
+};
+```
+
+And it finally sounds smooth and nice. But what about decay and sustain phases? Both are done by the audio samples themselves, so we don't need to worry about them. To be honest, same goes for the attack phase, but we have implemented it for the sake of this guide. 🙂 <br /><br />
+So the only missing piece left is doing something about the missing sample files for 'B' and 'D' keys. What we can do about that?
+
+## Tampering with the playback rate
+
+`AudioBufferSourceNode` also has its own `AudioParam`, it is called `playbackRate` as the title suggests. It allows to change the speed of the playback of the audio buffer. <br /> <br />
+Yay! nice. But how can we use that to make the missing keys sound? I will keep this one short, as this guide is already quite long so lets wrap up!<br /><br />
+
+When we change the speed of some sound, it will also change it's pitch (frequency), so we can use that to make the missing keys sound.<br /><br />
+Each piano key has it's own dominant frequency (f.e. the frequency of the `A4` key is `440Hz`), we can check frequency behind each key, calculate the ratio between them and use it to change the playback rate of the buffers we have. <br /> <br />
+
+![Piano keys frequencies](/img/frequencies-on-piano.jpg)
+
+for our example, lets use those frequencies as base of our calculations:
+
+```tsx
+const noteToFrequency = {
+  A: 261.626, // real piano middle C
+  B: 277.193, // Db
+  C: 311.127, // Eb
+  D: 329.628, // E
+  E: 369.994, // Gb
+};
+```
+
+First we need to find closest key to the missing one, we can do it in simple for loop:
+
+```tsx
+function getClosest(key: KeyName) {
+  let closestKey = 'A';
+  let minDiff = noteToFrequency.A - noteToFrequency[key];
+
+  for (const sourcedKey of Object.keys(sourceList)) {
+    const diff = noteToFrequency[sourcedKey] - noteToFrequency[key];
+
+    if (Math.abs(diff) < Math.abs(minDiff)) {
+      minDiff = diff;
+      closestKey = sourcedKey;
+    }
+  }
+
+  return closestKey;
+}
+```
+
+Now we just use the function in `onKeyPressIn` when the buffer is not found and change the playback rate for the source node:
+
+```tsx
+const onKeyPressIn = (which: KeyName) => {
+  let buffer = bufferListRef.current[which];
+  const aCtx = audioContextRef.current;
+  let playbackRate = 1;
+
+  if (!buffer) {
+    const closestKey = getClosest(which);
+    const closestBuffer = bufferMapRef.current[closestKey];
+    playbackRate = noteToFrequency[closestKey] / noteToFrequency[which];
+  }
+
+  const source = aCtx.createBufferSource();
+  const envelope = aCtx.createGain();
+  source.buffer = buffer;
+
+  // rest of the code remains the same
+};
+```
+
+## Final results
+
+As previously, you can see the final results in the live example below with full source code.
+
+import InteractiveExample from '@site/src/components/InteractiveExample';
+import SimplePiano from '@site/src/examples/SimplePiano/Component';
+import SimplePianoSrc from '!!raw-loader!@site/src/examples/SimplePiano/Source';
+
+<InteractiveExample component={SimplePiano} src={SimplePianoSrc} />
+
+## Summary
+
+In this guide we have learned how to create a simple piano keyboard with help of the gain node and audio params. To sum up:
+
+- `AudioParam` is an interface that provides ways to control various aspects of audio nodes over time.
+- `GainNode` is a simple node that can change the volume of any node connected to it.
+- `AudioBufferSourceNode` has param called `playbackRate` that allows to change the speed of the playback of the audio buffer and change pitch of the sounds.
+- We can use `GainNode` to create envelopes and make the sound transitions more smooth and pleasant.
+- We have learned how we can use the audio api in react environment in more production environment like scenario.
+
+## What's next?
+
+I don't know, give yourself a pat on the back, You've earned it! We will come with more guides soon, stay tunned! 🎉

packages/audiodocs/docusaurus.config.js

@@ -60,6 +60,7 @@ const config = {
   markdown: {
     mermaid: true,
   },
+  themes: ['@docusaurus/theme-mermaid'],
 
   themeConfig: {
     // Replace with your project's social card

packages/audiodocs/package.json

@@ -24,6 +24,7 @@
     "@docusaurus/core": "^2.4.3",
     "@docusaurus/plugin-debug": "^2.4.3",
     "@docusaurus/preset-classic": "^2.4.3",
+    "@docusaurus/theme-mermaid": "^2.4.3",
     "@emotion/react": "^11.10.6",
     "@emotion/styled": "^11.10.6",
     "@mdx-js/react": "^1.6.22",