Middle Layer SDK (aka patching with Lua)

[odevices]

(Here is a first draft post introducing the Middle Layer SDK. I will be expanding it over the next few days.)

The Middle Layer SDK allows you to construct Units using the Lua scripting language in the same way that the builtin units are constructed. In a nutshell, you construct a unit by:

• Required: Give your unit a title. (i.e. Override and implement the init method)
• Required: Building the unit’ internal DSP graph by instantiating DSP objects and connecting them together in useful ways. (i.e. Implement the onLoadGraph method of your unit.)
• Required: Building the unit’s GUI by creating controls and connecting them to the appropriate ports and parameters of your DSP graph. (i.e. Implement the onLoadViews method of your unit.)
• Optional: Adding any special-purpose serialization/deserialization code (i.e. stuff that you want to save in a preset that is not already handled automatically for you). (i.e. Override and implement the serialize and deserialize methods)
• Optional: Adding any special-purpose menu commands. (i.e. Implement the onLoadMenu method)

Once you have one (or more) units coded. Then you need to collect them into a library. This just means:

• Placing all necessary files in a folder.
• Creating a library description file called init.lua in the same folder. This init.lua file is where you name your library and list all of the units it contains as well as some other meta data.

Finally, you would copy this library folder (containing your unit files and the init.lua) to the ER-301/libs of your front SD card. The ER-301 will automatically detect the new library and load it. Now whenever you insert a unit, you will see your units added to the unit selection menu.

Here is an example library with just 1 unit (FM Operator) to get people started:
testlib-v0.4.25.zip (1.7 KB)

Unzipping this file and dropping the testlib folder into ER-301/libs on your SD card will give you access to a nifty little FM Operator unit which I was able to make in 10mins.

For your reference, here is the entire (Lua only) source code for the builtin library of units:
builtins-v0.4.25.zip (109.9 KB)

Doxygen of ER-301 Application Layer (App):
er-301-app-doxygen-v0.4.25.zip (5.0 MB)

TODO:

• Very soon: Create some simple documented examples for people to follow.
• Soon: Create an SDK reference listing all of the available DSP objects and what they do.
• Further down the line: create a USB driver that lets you upload changes to a unit library while the ER-301 is running so that you don’t have keep swapping the SD card in and out.

[2disbetter]

Most excellent!

[vcoadsr]

Hi Brian, this looks amazing - as a non-coder is your hope that with the examples building units will be attainable?

[odevices]

I hope so but I would definitely recommend that the non-coders wait a bit. The need to go back-and-forth between your computer and the ER-301 with an SD card is definitely not fun if you are using trial-and-error to write your units.

[odevices]

Here is the contents of the testlib example for easier reference.

init.lua

local units = {
  {category="Oscillators"},
  {title="FM Operator",moduleName="FMOperator",keywords="source, pitch, modulate"},
}

return {
  title = "Test Units",
  name = "testlib",
  contact = "john.doe@test.com",
  keyword = "test",
  units = units
}

FMOperator.lua

-- GLOBALS: app, connect
local Class = require "Base.Class"
local Unit = require "Unit"
local PitchTranspose = require "Unit.ViewControl.PitchTranspose"
local GainBias = require "Unit.ViewControl.GainBias"
local Trigger = require "Unit.ViewControl.Trigger"
local Fader = require "Unit.ViewControl.Fader"
local Encoder = require "Encoder"
local ply = app.SECTION_PLY

local FMOperator = Class{}
FMOperator:include(Unit)

function FMOperator:init(args)
  args.title = "FM Op"
  args.mnemonic = "FM"
  args.version = 2
  Unit.init(self,args)
end

function FMOperator:onLoadGraph(pUnit,channelCount)
  local carrier = self:createObject("SineOscillator","carrier")
  local modulator = self:createObject("SineOscillator","modulator")
  local rational = self:createObject("RationalMultiply","rational",true)
  local multiply = self:createObject("Multiply","multiply")
  local tune = self:createObject("ConstantOffset","tune")
  local tuneRange = self:createObject("MinMax","tuneRange")
  local f0 = self:createObject("GainBias","f0")
  local f0Range = self:createObject("MinMax","f0Range")
  local vca = self:createObject("Multiply","vca")
  local level = self:createObject("GainBias","level")
  local levelRange = self:createObject("MinMax","levelRange")
  local num = self:createObject("GainBias","num")
  local numRange = self:createObject("MinMax","numRange")
  local den = self:createObject("GainBias","den")
  local denRange = self:createObject("MinMax","denRange")
  local index = self:createObject("GainBias","index")
  local indexRange = self:createObject("MinMax","indexRange")

  connect(tune,"Out",tuneRange,"In")
  connect(tune,"Out",carrier,"V/Oct")

  connect(f0,"Out",carrier,"Fundamental")
  connect(f0,"Out",f0Range,"In")

  connect(f0,"Out",rational,"In")
  connect(rational,"Out",modulator,"Fundamental")
  connect(num,"Out",rational,"Numerator")
  connect(num,"Out",numRange,"In")
  connect(den,"Out",rational,"Divisor")
  connect(den,"Out",denRange,"In")

  connect(index,"Out",multiply,"Left")
  connect(index,"Out",indexRange,"In")
  connect(modulator,"Out",multiply,"Right")
  connect(multiply,"Out",carrier,"Phase")

  connect(level,"Out",levelRange,"In")
  connect(level,"Out",vca,"Left")

  connect(carrier,"Out",vca,"Right")
  connect(vca,"Out",pUnit,"Out1")

  self:addBranch("level","Level",level,"In")
  self:addBranch("V/oct","V/Oct",tune,"In")
  self:addBranch("index","Index",index,"In")
  self:addBranch("f0","Fundamental",f0,"In")
  self:addBranch("num","Numerator",num,"In")
  self:addBranch("den","Denominator",den,"In")

  if channelCount > 1 then
    connect(self.objects.vca,"Out",pUnit,"Out2")
  end
end

local views = {
  expanded = {"tune","freq","num","den","index","level"},
  collapsed = {},
}

function FMOperator:onLoadViews(objects,controls)
  controls.tune = PitchTranspose {
    button = "V/oct",
    branch = self:getBranch("V/Oct"),
    description = "V/oct",
    offset = objects.tune,
    range = objects.tuneRange
  }

  controls.freq = GainBias {
    button = "f0",
    description = "Fundamental",
    branch = self:getBranch("Fundamental"),
    gainbias = objects.f0,
    range = objects.f0Range,
    biasMap = Encoder.getMap("oscFreq"),
    biasUnits = app.unitHertz,
    initialBias = 27.5,
    gainMap = Encoder.getMap("gain5000"),
    scaling = app.octaveScaling
  }

  controls.num = GainBias {
    button = "num",
    description = "Numerator",
    branch = self:getBranch("Numerator"),
    gainbias = objects.num,
    range = objects.numRange,
    biasMap = Encoder.getMap("int[1,32]"),
    biasUnits = app.unitInteger,
    initialBias = 1
  }

  controls.den = GainBias {
    button = "den",
    description = "Denominator",
    branch = self:getBranch("Denominator"),
    gainbias = objects.den,
    range = objects.denRange,
    biasMap = Encoder.getMap("int[1,32]"),
    biasUnits = app.unitInteger,
    initialBias = 1
  }

  controls.index = GainBias {
    button = "index",
    description = "Index",
    branch = self:getBranch("Index"),
    gainbias = objects.index,
    range = objects.indexRange,
    initialBias = 0.1,
  }

  controls.level = GainBias {
    button = "level",
    description = "Level",
    branch = self:getBranch("Level"),
    gainbias = objects.level,
    range = objects.levelRange,
    initialBias = 0.5,
  }

  return views
end

return FMOperator

[freqout]

very cool… wondering if there is any plan to expose any drawing APIs so that we can add cool custom animated graphics a-la the scale quantizer unit

[Joe]

Super cool, Brian! I was able to load the test lib successfully. Nice little bonus new unit you have snuck in there for those paying attention.

I was able to change the order of the num and den controls, since they seemed backward to me. No problems, just inserting the unit with the new modified LUA file on the card picked up the change.

Generally speaking the code seems very readable to me. I don’t think I could write anything useful at this point but I assume the first couple items in your TODO list will remedy that.

Cheers! Glad to see this new arrival!

[Joe]

What software do you write your LUA in, Brian? I made my modification in the heavy hitting notepad.exe. :)

Looks like Visual Studio has a LUA extension with Intellisense, syntax checker and color coding.

[odevices]

I can highly recommend this editor:

A hackable text editor for the 21st Century

[auxren]

The source code is gnarly for .lua files created on the 301.
Can’t wait to get some more details on this process.

Thanks for the FMOperator

[trickyflemming]

I popped the “testlib” folder into the “libs” folder. I tried to load the FM Operator unit from the chooser screen, but it dumped this error log:

1:/ER-301/libs/testlib/FMOperator.lua:4: module 'Unit.ViewControl.PitchTranspose' not found:
	no field package.preload['Unit.ViewControl.PitchTranspose']
	no file 'X:/Unit/ViewControl/PitchTranspose.lua'
	no file 'X:/Unit/ViewControl/PitchTranspose/init.lua'
	no file '1:/ER-301/libs/Unit/ViewControl/PitchTranspose.lua'
	no file '1:/ER-301/libs/Unit/ViewControl/PitchTranspose/init.lua'
stack traceback:
	[C]: in function 'require'
	1:/ER-301/libs/testlib/FMOperator.lua:4: in main chunk
	[C]: in function 'require'
	X:/Unit/Factory/init.lua:89: in function <X:/Unit/Factory/init.lua:89>
	[C]: in function 'xpcall'
	X:/Unit/Factory/init.lua:90: in function 'Unit.Factory.instantiate'
	X:/Chain/ChainBase.lua:287: in function 'GlobalChains.GlobalChain.loadUnit'
	(...tail calls...)
	X:/Chain/EmptySection.lua:42: in local 'x'
	X:/Base/Observable.lua:50: in function 'SourceChooser.SourceGroup.emitSignal'
	X:/Unit/Chooser/init.lua:79: in function 'Unit.Chooser.choose'
	(...tail calls...)
	...
	X:/Base/Widget.lua:151: in function 'SourceChooser.SourceGroup.sendUpHelper'
	X:/Base/Widget.lua:141: in function 'SourceChooser.SourceGroup.sendUp'
	X:/Context.lua:192: in function 'Context.notify'
	X:/Application.lua:142: in upvalue 'notify'
	X:/Application.lua:241: in upvalue 'dispatch'
	X:/Application.lua:317: in function 'Application.loop'
	x:/startup/start.lua:54: in function <x:/startup/start.lua:47>
	[C]: in function 'xpcall'
	x:/startup/start.lua:57: in main chunk
	[C]: in function 'dofile'
	[string "dofile('x:/startup/start.lua')"]:1: in main chunk
---ERROR REPORT END

I’m on 0.3.15

[odevices]

The testlib that you downloaded was for v0.3.12 I’ll put the v0.3.15 version up in an hour so.

I’m constantly refactoring and renaming classes (I seldom get the naming right the first time), hence the unstable adjective.

[odevices]

However, if you want to quickly fix it yourself, just rename PitchTranspose to PitchControl everywhere.

[odevices]

1:/ER-301/libs/testlib/FMOperator.lua:4: module 'Unit.ViewControl.PitchTranspose' not found:
	no field package.preload['Unit.ViewControl.PitchTranspose']
	no file 'X:/Unit/ViewControl/PitchTranspose.lua'
	no file 'X:/Unit/ViewControl/PitchTranspose/init.lua'
	no file '1:/ER-301/libs/Unit/ViewControl/PitchTranspose.lua'
	no file '1:/ER-301/libs/Unit/ViewControl/PitchTranspose/init.lua'

I would like to explain this kind of an error in a little detail because I expect it to crop up every once in awhile.

In lua, you organize your code into modules and then bring them into other code using the require statement. Like this:

local PitchTranspose = require "Unit.ViewControl.PitchTranspose"

The error message is showing how lua was unable to find this particular module by displaying the various paths that it tried. Upon failing to find the module at any of these paths, it generates the error in question and halts.

[sixnon]

Thanks for that explanation Brian, it makes something which looks quite complicated at the oustset, even for me quite clear to understand.

[udbhav]

Spent some time reading through a few of the builtins. Trying to get a grasp of what kinds of things make sense to build using this SDK.

I’ve been wanting to make a sampler that lets you select different samples to play back using modulation. I was looking at BasePlayer:setSample and MonoPlayHead and StereoPlayHead. Is this possible within the SDK? I thought the right approach would be to make something similar to MonoPlayHead that held multiple samples in memory, but it looks like objects like that live outside the SDK.

Would the suggested approach be to create multiple instances of PlayerUnit controlled by a single set of controls and switch between them?

[anon83620728]

Sounds very sensible to me @udbhav

The middle layer was described elsewhere as a convenient method to write custom units so this, if I understand correctly, would be an ideal application.

I might be wrong though

[odevices]

Unfortunately, not really. No Lua code is executed in the DSP thread which means that the only way you can get your Lua code to do something based on audio or modulation signals is to have it poll every so often which would be pretty hokey. And it would be very slow to do anything but simple math in Lua. The purpose of the middle layer is to patch existing DSP objects together.

In this case, we are missing an implementation of a multi-sample object which would allow a list of samples to be treated as one big sample.

By the way, I guess this means that creating sample chains outside of the ER-301 is not a viable solution? May I ask why?

[anon83620728]

That’s interesting, because I can think of ways to patch this up using existing units… e.g. the switching is routing the trigger and cv values to different sample players… so, I am a little confused now!

[odevices]

You are thinking of cross-fading amongst a bunch of sample players that are running in parallel.

I thought the original request was for actually switching a play head from one sample to another.