Thursday, June 12, 2014

 

 

Ruby Sound and Music Project


I've decided to found the Ruby Sound and Music Project, or RSMP for short.  It's aim, in short, is to provide audio tools for Ruby developers.

Here are some notes I jotted down this morning:

Mission Statement:

  To create and provide tools for Ruby developers to work with music, audio, and sound in .general on any platform for any application.

  For example: to open and play an mp3 file using only Ruby in a Linux, Windows, or iOS environment.
Or perhaps to create audio conversion wrappers.  Want to convert that ogg to wav?  Nor problem.

To provide a central hub for discussion and development of the project.

Also.. mobile development?  Raspberry Pi tools?  Rails-specific audio helpers?  Want to write music?  Want your computer to write music for you?  Love Ruby? Look no further!

______________

Anyhow, these are some rough-and-tumble ideas to get the project going, and with my Juicy gem, this can get off the ground in no time!

Saturday, March 8, 2014

Holy crap I've done it

I've done it.  I can now play two arbitrary frequencies through FFI using native library calls.  And all in 155 lines of code. Also, it takes 76 milliseconds to calculate 1 second of PCM audio from scratch on my system.


require 'ffi'

module Win32

  WAVE_FORMAT_PCM = 1
  WAVE_MAPPER = -1
  
  class HWAVEOUT < FFI::Struct
  
    layout :i, :int
    
  end

  class WAVEHDR < FFI::Struct
  
    def initialize(dwBufferLength, dwBytesRecorded, dwLoops, dwFlags, lpData)
      self[:dwBufferLength] = dwBufferLength
      self[:dwBytesRecorded] = dwBytesRecorded
      self[:dwLoops] = dwLoops
      self[:dwFlags] = dwFlags
      self[:lpData] = lpData
    end
  
    layout :lpData,          :pointer,
           :dwBufferLength,  :ulong,
           :dwBytesRecorded, :ulong,
           :dwUser,          :ulong,
           :dwFlags,         :ulong,
           :dwLoops,         :ulong,
           :lpNext,          :pointer,
           :reserved,        :ulong
      
  end

  class WAVEFORMATEX < FFI::Struct
  
    def initialize(nSamplesPerSec, wBitsPerSample, nChannels, cbSize)
      self[:nSamplesPerSec] = nSamplesPerSec
      self[:wBitsPerSample] = wBitsPerSample
      self[:nChannels] = nChannels
      self[:cbSize] = cbSize
      self[:wFormatTag] = WAVE_FORMAT_PCM
      self[:nBlockAlign] = (self[:wBitsPerSample] >> 3) * self[:nChannels]
      self[:nAvgBytesPerSec] = self[:nBlockAlign] * self[:nSamplesPerSec]
    end
  
    layout :wFormatTag,      :ushort,
           :nChannels,       :ushort,
           :nSamplesPerSec,  :ulong, 
           :nAvgBytesPerSec, :ulong, 
           :nBlockAlign,     :ushort,
           :wBitsPerSample,  :ushort,
           :cbSize,          :ushort
    
  end
  
  class Sound
    extend FFI::Library
    
    private
    
    typedef :uintptr_t, :hwaveout
    typedef :uint, :mmresult
    typedef :ulong, :dword
    
    ffi_lib :winmm
    
    attach_function :waveOutOpen, [:pointer, :uint, :pointer, :dword, :dword, :dword], :mmresult
    attach_function :waveOutPrepareHeader, [:hwaveout, :pointer, :uint], :mmresult
    attach_function :waveOutWrite, [:hwaveout, :pointer, :uint], :mmresult
    attach_function :waveOutUnprepareHeader, [:hwaveout, :pointer, :uint], :mmresult
    attach_function :waveOutClose, [:hwaveout], :mmresult
    
    ffi_lib FFI::Library::LIBC
    
    attach_function :malloc, [:size_t], :pointer
    attach_function :calloc, [:size_t], :pointer
    attach_function :realloc, [:pointer, :size_t], :pointer
    attach_function :free, [:pointer], :void
    attach_function :memcpy, [:pointer, :pointer, :size_t], :pointer
    
    def self.fill_data
    
      data = []
      period = 1.0/440.0
      
      ramp = 1500.0
      
      time = Time.now
      
      44100.times do |sample|
        t = sample/44100.0
        angle = (2.0*Math::PI/period) * t
        angle2 = 1.5*angle
        factor = 0.5*Math.sin(angle) + 0.5
        factor2 = 0.5*Math.sin(angle2) + 0.5
        x = 0.5*32768.0*(factor + factor2)
        if sample < ramp
          x *= sample/ramp
        end
        if 44100 - sample < ramp
          x *= (44100 - sample)/ramp
        end
        data << x.floor
      end
      
      puts "This took #{Time.now - time} milliseconds to compute"
      
      data
      
    end
    
    public
    
    def self.play
    
      hWaveOut = HWAVEOUT.new
      wfx = WAVEFORMATEX.new(44100, 16, 2, 0)
      
      if ((error_code = waveOutOpen(hWaveOut.pointer, WAVE_MAPPER, wfx.pointer, 0, 0, 0)) != 0)
        raise SystemCallError, FFI.errno, "waveOutOpen: #{error_code}"
      end
      
      data = fill_data
      
      data_buffer = malloc(data.first.size * data.size)
      data_buffer.write_array_of_int data
      
      header = WAVEHDR.new(4*44100, 0, 1, (4 | 8), data_buffer)
      
      if ((error_code = waveOutPrepareHeader(hWaveOut[:i], header.pointer, header.size)) != 0)
        raise SystemCallError, FFI.errno, "waveOutPrepareHeader: #{error_code}"
      end
      
      if ((error_code = waveOutWrite(hWaveOut[:i], header.pointer, header.size)) != 0)
        raise SystemCallError, FFI.errno, "waveOutWrite: #{error_code}"
      end
        
      while (waveOutUnprepareHeader(hWaveOut[:i], header.pointer, header.size) == 33)
        sleep 0.1
      end
      
      if ((error_code = waveOutClose(hWaveOut[:i])) != 0)
        raise SystemCallError, FFI.errno, "waveOutClose: #{error_code}"
      end
      
      self
    end
  
  end
  
end

Win32::Sound.play

waveOutOpen

I have successfully made a library call to waveOutOpen!


require 'ffi'

module WaveOutWrapper

  WAVE_FORMAT_PCM = 1
  WAVE_MAPPER = -1

  class WAVEFORMATEX < FFI::Struct

    def initialize(nSamplesPerSec, wBitsPerSample, nChannels, cbSize)
      self[:nSamplesPerSec] = nSamplesPerSec
      self[:wBitsPerSample] = wBitsPerSample
      self[:nChannels] = nChannels
      self[:cbSize] = cbSize
      self[:wFormatTag] = WAVE_FORMAT_PCM
      self[:nBlockAlign] = (self[:wBitsPerSample] >> 3) * self[:nChannels]
      self[:nAvgBytesPerSec] = self[:nBlockAlign] * self[:nSamplesPerSec]
    end

    layout :wFormatTag,      :ushort,
           :nChannels,       :ushort,
           :nSamplesPerSec,  :ulong, 
           :nAvgBytesPerSec, :ulong, 
           :nBlockAlign,     :ushort,
           :wBitsPerSample,  :ushort,
           :cbSize,          :ushort
    
  end

  class Sound
    extend FFI::Library
    
    private
    
    typedef :uint, :mmresult
    typedef :ulong, :dword
    
    ffi_lib :winmm
    
    attach_function :waveOutOpen, [:pointer, :uint, :pointer, :dword, :dword, :dword], :mmresult
    attach_function :waveOutClose, [:pointer], :mmresult
    
    public
    
    def self.play
    
      hWaveOut = FFI::MemoryPointer.new(:int)
      wfx = WAVEFORMATEX.new(44100, 16, 2, 0)
      waveOutOpenResult = waveOutOpen(hWaveOut, WAVE_MAPPER, wfx.pointer, 0, 0, 0)
      raise SystemCallError, FFI.errno, "waveOutOpen didn't work: #{waveOutOpenResult}" if waveOutOpenResult != 0
      waveOutClose(hWaveOut)
      
      self
    end

  end

end

WaveOutWrapper::Sound.play # this won't actually play anything, but it doesn't return an error!

Audio and Ruby

I'm working on a gem which will make songs for me.  This may be a pie-in-the-sky dream, but I'm still working on it.  One thing I need is to be able to play arbitrary digital signals to a sound device.  A cursory google search will show that there is a gem to do just this!

gem install win32-audio

And then all you have to do is load up the gem, and make a native system call to make a beep.

require 'win32/sound'
include Win32
Sound.beep(440, 500)

This code will play a 440Hz tone for 500 millisecond.  But there's a problem.  It's synchronous. I need to be able to play multiple tones at once, or even be able to mix some signals and stream the result to a sound device.

The win32-audio uses the ffi gem to do its dirty work, so I've decided to jump in and do the same.

My plan is to code up a gem (or an extension to win32-audio) which makes native calls to the waveOut multimedia library in windows.  Hopefully then I'll be able to shove in arbitrary PCM signal to a sound device of my choosing, and even with as many channels as I'd like!

It's basically this:

1. waveOutOpen to open up a device for streaming
2. waveOutPrepareHeader to prepare a block of audio for playback
3. waveOutWrite to submit the prepared audio to the device for playing
4. waveOutClose to close up the stream when I'm done

As much as I wish it was simple to just define these four functions in ffi, it gets a little more hairy.  Of course, I have to go in and define and set up memory management for the underlying code which relies on various structs like WAVEFORMATEX and WAVEHDR.

Wish me luck!