diff --git a/CHANGELOG.md b/CHANGELOG.md index 0bab210f..441f6cf3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,8 @@ This change log follows the conventions of [plausible simulated data is produced](https://github.com/Deep-Symmetry/beat-link-trigger/issues/160) for more convenience bindings. +- A new integration example shows how to send phrase information to + TouchDesigner. ### Changed diff --git a/doc/modules/ROOT/assets/attachments/TouchDesigner.bls b/doc/modules/ROOT/assets/attachments/TouchDesigner.bls new file mode 100644 index 00000000..4ccdf069 Binary files /dev/null and b/doc/modules/ROOT/assets/attachments/TouchDesigner.bls differ diff --git a/doc/modules/ROOT/assets/images/BeatsToTouchDesigner.png b/doc/modules/ROOT/assets/images/BeatsToTouchDesigner.png new file mode 100644 index 00000000..6c4fd718 Binary files /dev/null and b/doc/modules/ROOT/assets/images/BeatsToTouchDesigner.png differ diff --git a/doc/modules/ROOT/assets/images/TableFromJSON.png b/doc/modules/ROOT/assets/images/TableFromJSON.png new file mode 100644 index 00000000..263c78c6 Binary files /dev/null and b/doc/modules/ROOT/assets/images/TableFromJSON.png differ diff --git a/doc/modules/ROOT/nav.adoc b/doc/modules/ROOT/nav.adoc index 068280c9..3b7dc69d 100644 --- a/doc/modules/ROOT/nav.adoc +++ b/doc/modules/ROOT/nav.adoc @@ -30,6 +30,7 @@ ** xref:Integration_BeyondEssentials.adoc[Pangolin BEYOND Essentials] ** xref:Integration_QLC.adoc[QLC+ lighting cues] ** xref:Integration_SMPTE.adoc[SMPTE Linear Timecode] +** xref:Integration_TouchDesigner.adoc[TouchDesigner Phrase Information] ** xref:Integration_XoneOnAir.adoc[Xone:96 Channels On Air] * xref:Older.adoc[Older Approaches] ** xref:Matching.adoc[Matching Tracks Manually] diff --git a/doc/modules/ROOT/pages/Integration_TouchDesigner.adoc b/doc/modules/ROOT/pages/Integration_TouchDesigner.adoc new file mode 100644 index 00000000..039ca8c3 --- /dev/null +++ b/doc/modules/ROOT/pages/Integration_TouchDesigner.adoc @@ -0,0 +1,140 @@ += TouchDesigner Phrase Information + +I received an interesting query from a lighting designer who uses +https://derivative.ca[TouchDesigner] to generate dynamic lighting +looks, and who wanted a way to be able to feed song structure (phrase +analysis) information into a TouchDesigner table so that the lights +could respond to the nature of the music being currently played. He +wanted the table to update on each beat from a master player with the +master player device number, current tempo, the track bank and current +phrase type from the song structure analysis, the track title, the +beat within bar that was beginning, and whether the phrase was +currently in a “fill” section, vamping until the start of the next +phrase. + +While most of that information could be obtained in the Beat +Expression of a Phrase Trigger, some of it would require a bit of +complex code. This was kind of an “inside out” way of using song +structure information in a phrase trigger than I had originally +anticipated. But it was clearly a very useful approach, so I decided +to add a few features and new convenience values in the expression to +make it easier. + +With those in place, a little back and fort discussion about how to +convey the information led us to send it as UDP packets containing +JSON-formatted data to TouchDesigner. I'll list all the required code +here, but you can also download and open a +link:{attachmentsdir}/TouchDesigner.bls[show file] that has them all +in place for you; whenever it is open, on any beat from a master +player that is playing a track with song structure analysis, JSON data +describing that beat will be sent to TouchDesigner on the port +configured in the show's Global Setup Expression. + +Speaking of that, here is how we configure where to send the data: + +[[global-setup-expression]] +.Global Setup Expression +```clojure +;; Create a socket for sending UDP to TouchDesigner, and record the +;; address and port to which such UDP messages should be sent. +(swap! globals assoc :td-socket (java.net.DatagramSocket.)) +(swap! globals assoc :td-address (java.net.InetAddress/getLocalHost)) +(swap! globals assoc :td-port 7000) +``` + +If TouchDesigner is running on a different machine than Beat Link +Trigger is, you would change the `:td-address` value to something like +`(java.net.InetAddress/getByName "192.1.2.3")`, replacing the IP +address string with the address of that machine. + +And if the UDPIn DAT node in TouchDesigner is configured to listen on +a port other than 7000, change the `:td-port` value in this expression +to match your UDPIn configuration. + +The next step is to write a helper function in the Show's Shared +Functions to format the desired song structure information as JSON, +write it into a UDP packet, and send that to TouchDesigner: + +[[shared-functions]] +.Shared Functions +```clojure +(defn send-json-to-touchdesigner + "Encodes a map as JSON and sends it in a UDP packet + to TouchDesigner." + [globals m] + (let [message (str (cheshire.core/encode m) "\n") ; Encode as JSON line. + {:keys [td-address td-port td-socket]} @globals ; Find where to send. + data (.getBytes message) ; Get JSON as raw byte array. + packet (java.net.DatagramPacket. data (count data) td-address td-port)] + (.send td-socket packet))) +``` + +Then we need to create a Phrase Trigger that is enabled for the Master +player, for all Phrase Types and Track Banks. I called this one Beats +to TouchDesigner: + +image:BeatsToTouchDesigner.png[Beats to TouchDesigner Phrase Trigger,1236,198] + +The final piece is to set up a Beat Expression in the Phrase Trigger +to send information on each beat. (This is why I call this an “inside +out” approach to a Phrase Trigger; where normally we would be painting +cues within the Phrase Trigger canvas to make lights do things, +instead on each beat the Beat Expression gives TouchDesigner the +information it needs to decide what cues it wants to run.) + +Here is a Beat Expression that sends the information that was useful +to the lighting designer who inspired this integration: + +.Beat Expression +```clojure +(let [payload {"masterPlayerNumber" device-number # <1> + "bpm" effective-tempo + "trackBank" track-bank + "phraseType" phrase-type + "trackTitle" track-title + "beat" beat-within-bar + "fill" (= section :fill)}] # <2> + (send-json-to-touchdesigner globals payload)) # <3> +``` + +<1> This sets up a map of the keys and values that we want to send to +TouchDesigner. + +<2> The new `section` convenience binding (added to Beat Link Trigger +to support this integration) will have the value `:start`, `:loop`, +`:end`, or `:fill` depending on which of the four sections of the +Phrase Trigger is currently playing. By comparing it to `:fill` we can +send a boolean flag that will be `true` only when we are in the +Fill-In section of a phrase. + +<3> We pass the expression globals (so the values we +<<#global-setup-expression,set up>> in the Global Setup Expression are +available) and the payload we want to send to the +`send-json-to-touchdesigner` function we added to the Shared Functions +<<#shared-functions,above>>. + +On the TouchDesigner side, we create a UDPIn DAT node with a an +`onReceive` callback that parses the JSON and puts it into a Table DAT +for the use of the TouchDesigner show: + +.UDPIn DAT callbacks +```python +import json +def onReceive(dat, rowIndex, message, bytes, peer): + data = json.loads(message) + table = op('table1') # <1> + table.clear() + for key, value in data.items(): + table.appendCol((key,value)) + return +``` + +<1> In this example, our target Table DAT is named `table1`. Change +this string to match the name of the table that you actually want to +be affected by these JSON UDP packets. + +This leads to table contents like the following, updated on each beat +from a track for phrase analysis information is available playing on +the current master player: + +image:TableFromJSON.png[Phrase information table,988,181]