Thursday, May 30, 2013

What's your Millionaire picture all about?

So, to make a tie-in to game shows on this blog, I thought I'd write a quick blurb about what my profile picture is.  (Just in case my picture ever changes, the picture I'm referring to can be seen here at http://www.proteinpower.com/drmike/low-carb-diets/atkins-diet-and-who-wants-to-be-a-millionaire/)   No, that's not me -- I'm not a lady, I sport a manly beard, I was never on Who Wants To Be a Millionaire?, and I'm not that big.  (Yes, people have asked me these questions. :-P)  However, it's just a humorous picture from a relatively recent game show (recent, at least compared to the 1/20/1961 episode of You're In the Picture I had intended to write about tonight, but I felt like these people did a far better job!).

The hapless contestant is being asked a question about the Atkins diet, popular back in the early-mid 2000s.  Having probably ignored all advice about dieting, she proceeded to use the 50/50 lifeline in order to take away two of the incorrect answer choices.  However, she still appears to be stumped.  (The answer is, of course, that the Atkins diet encourages weight watchers to eliminate carbohydrates from their diet.)  Now I can't say I could answer any question about any trendy subject myself (my knowledge of pop music ends after about 1969), but even in my circles, it was pretty hard to avoid constantly hearing about the Atkins diet back then.

So, that picture has very little to do with me except that I follow game shows, and in all honesty, Millionaire doesn't really rank among my favorites.  In short, I picked that picture because it is humorous.  They're asking a fat lady about a diet, she sought help, and is still stumped!

Oh, and if you want to see what game shows I really enjoy, head on over to www.stev-o.us!

Thursday, May 23, 2013

JavaScript Checkboxes - click & drag to highlight 'em all!

With jQuery, it becomes possible for people to make much simpler and more intuitive user interactions with controls on the Web.  I'm sure many people have had the problem where they want to select, say, half of the checkboxes on a page, or maybe all the ones within just a particular value or range.  It's time-consuming and error-prone to click them all, and the "Select All" button just won't buy much time savings.

Thus far, I haven't really seen a good implementation that solves this problem, so I rolled my own.  Please use it in your own designs!  This is something that needs to be shared freely for the good of humanity. :-P  You can do this completely in JavaScript with dynamically-generated checkboxes or even ones you have pre-existing.  Here goes:



<script src="http://code.jquery.com/jquery-1.7.2.min.js" type="text/javascript"></script>

<script language="javascript" type="text/javascript">
var mouseDown = 0;

$(document).ready(function() {
    $("body").mousedown(function() {
        mouseDown = 1;
    });
    $("body").mouseup(function() {
        mouseDown = 0;
    });

    // If you had a function to dynamically make checkboxes, it would start here {
    // Then you would also indent this next jQuery mouseenter handler

    $('.messageCheckParent').mouseenter(function() {
        if (mouseDown) {
            checkbox = $(this).find("input").first();
            checkbox.attr('checked', !( checkbox.attr('checked') ));
        }
    });
    // } Now your special dynamic generator function ends

});

</script>

<html><etc ...>
<!-- Or if you prefer to statically generate your HTML... -->
<table>
  <tbody>
    <tr>
      <td class="messageCheckParent" id="cell1"><input class="messageCheck" id="checkBox1" type="checkbox" />
      </td>
    </tr><tr>
      <td class="messageCheckParent" id="cell2"><input class="messageCheck" id="checkBox2" type="checkbox" />
      </td>
    </tr><tr>
      <td class="messageCheckParent" id="cell3"><input class="messageCheck" id="checkBox3" type="checkbox" />
      </td>
    </tr>
  </tbody>
</table>
</etc></html>

The best way to tie this all together is with the jQuery tablesorter plugin.  With tablesorter, you can allow users to click on column headings, and the table will be sorted by all the values within.  So if, for some reason, you ever had to check off all the names of the players whose teams went to the playoffs, you could sort the table by team name, then click & drag over all the checkboxes next to the right names.

The way the code is implemented here leaves a little to be desired, though.  For instance, the first checkbox you click on won't actually get checked because mouseDown was 0 (false) when you entered it, and dragging out of a checkbox with the mouse down causes it not to become checked.  The other undesirable action is if you drag the mouse down and then back up, the lowest checkbox that became checked won't get unchecked, but everything else as you drag back up will become unchecked.  You're welcome to fix these flaws though, and post your fixes in the comments!

Nevertheless, this ought to save time, frustration, and repetitive motion syndrome in your users' fingers if you ever need to make a panel with lots of JavaScript checkboxes.

Thursday, May 9, 2013

Futures & Options on Bitcoin: Why It Won't Work...

Just to clarify: it won't work now without significant attention from the right people.

Bitcoin would stand to benefit from some type of options or futures market, as these instruments help people mitigate their risk in other assets when used properly (yes, I'm looking at you, Dick Fuld).  Currently the only way you can protect yourself from losses in BTC is to sell at the right time, and everyone knows timing any market is very hard to do.  (Well, not so hard---just sell next time someone hacks the system, as it starts falling, then buy again in about 24 hours.  It's tricky to actually pull off, though, when currently one exchange handles most of the transactions and basically falls apart during high volume.)

Apparently, due to the denial of service attacks on Mt. Gox, the most popular Bitcoin exchange, within the past few weeks, folks have been looking to create an open-source trading platform that will allow anyone to make their own BTC exchange with all the bells & whistles they care to provide.  This not only gives users choices for where to keep and/or how to store BTCs, but also makes it more difficult for hackers to propagate an effective DoS attack because they'll have to pour more resources into bringing down more sites.  The open-source platform is called Buttercoin, and I'm sure someone could adapt it to do all sorts of futures & derivatives, at least in the software.  Several people have asked me about what it would take to do this, given some of my past exploits, but I've advised them to just stick with a straight-up bitcoin exchange with no such frills.

I'm sure the capability to trade futures and even options on BTC will exist eventually.  To do it right will require people with much deeper pockets and many more connections than your average college student or DIY enthusiast so the exchange can be properly bankrolled and publicized.  Time to market is everything; the first player to offer these kinds of products will probably gain a lot of notoriety among active Bitcoin fanatics and market watchers alike, and perhaps even the SEC if they try to set up an exchange without following their rules, obtaining licenses, collecting user info to comply with the USA PATRIOT Act, and all sorts of other things.  Existing Bitcoin exchanges have to follow the same rules, much less anyone trading options and/or futures.  However, besides making a futures exchange itself, it might be smarter to start up the harder part that most folks will probably forget about, yet will be very important to their success -- the clearing house.

One big reason deep pockets are required is the need for any BTC exchange dealing with contracts to manage exposure to risk, such as market risk or counterparty risk, brought on by the wild fluctuations of BTC itself or simply the avarice of any nefarious traders.  A good exchange will mitigate risk for itself and its participants; a sketchy exchange will leave every man to himself.  Commodity exchanges manage risks by balancing the buying power of people's accounts based on movements in their holdings on a daily basis.  Margin calls can be imposed on accounts that fall short of minimum requirements, forcing the deposit of more funds or liquidation.  Exchanges also use clearing houses to manage risks.  Most clearing houses employ sophisticated mathematical models such as Monte Carlo simulations to help figure out various things such as the right level of funding in trading accounts.  In most American markets, the Options Clearing Corporation, backed by the SEC & CFTC, is employed as a clearing house, or some exchanges use their own.  Since there are no clearing houses on BTC transactions that I know of, an exchange founder would have to start one themselves to enforce good behavior and proper account maintenance.  The exchange's compliance department would have to be powerful enough to enforce actions on users whose positions completely backfire, and be able to unwind the positions of people who can't satisfy their margin calls by liquidating part or all of the existing positions and just eating the rest of the losses.  There are still a lot of shady people using Bitcoin for illegitimate purposes, too.  Trying to get them to play by legitimate rules before they deceive exchanges and honest traders, while leaving the rest of us holding the bag for their bad bets, could be near impossible.

Margin accounts require a lot of maintenance, and a careful eye on exactly how much leverage all trading accounts are using.

For a futures exchange to guarantee maximum security for all its participants:
  • Anyone looking to buy BTC futures must have sufficient funds in their account, held in "escrow" until time of delivery or the futures position is removed.
  • Anyone selling  BTC futures must have that number of BTCs held in "escrow" until delivery time occurs or until they remove their futures position.
  • Never allow any kind of naked shorting due to the insane volatility of BTCs.
However, traders would have little reason to trust that exchanges or "bitcoin banks/brokerages" would truly hold their valuables in escrow and not just steal them outright.  There's an untrusting mentality with many of the folks who seriously back BTCs.  Plus, having to hold property in escrow isn't the way a margin account usually works; most times, the account's buying power is reduced by the maximum amount the positions would possibly cost to cover if they end up going bad.  The problem lies in that Bitcoin is so volatile, plus some of Bitcoin's biggest proponents are, say, just a little too good with computers.

One final thing: to do options on BTC, which would be the ultimate way to hedge against downside risk, whoever starts the market would need substantially deep pockets to create enough liquidity for them to really work.  Unfortunately, it would be enormously expensive to try and hedge against all that risk because there are just no other assets like BTC at all.  You can't just buy some other options contracts whose deltas add up to the opposite of the position you just took on, since all the deltas of BTC options will be correlated with each other.  And if the market becomes extremely lopsided and everyone wants to buy or sell at once, there's no way anyone looking to simply inject liquidity could ever manage to stay alive.  The options contracts would need to be treated like futures, i.e. not there until someone really wants them.  There may have to be non-standard arrangements made "over-the-counter" instead of in an organized fashion with set strikes, expiration dates, etc.  But even if you left the control in the hands of individuals, you'd still need a clearinghouse and an observant margin-watching department with some sharp teeth to make sure people behave out there, make honest deals, and don't just run away leaving someone holding the bag.

There are too many things that could go wrong if a bunch of small-time, inexperienced people try to go on building exchanges to trade Bitcoin futures and derivatives.  Unfortunately, this pretty much leaves the creation of such exchanges to those who rigged certain "casinos" in New York City and elsewhere in the first place.  It would be interesting, though, to try and build a futures & derivatives trading platform based on Buttercoin that could be used on actual licensed products.  This could promote even more transparency in markets and nurture a DIY culture among trading enthusiasts.

Thursday, May 2, 2013

Decorrelab -- Real-time Audio Decorrelation Engine for SuperCollider

The first project I'd like to share is DecorreLab, a script I wrote back in college as a project for a couple of my classes, particularly 3D Sound and Spatial Audio.  While there are apparently some VST plugins that do decorrelation, I'll provide my source so you can run with it and improve it if you wish.  Plus, you'll know exactly what algorithm I use. ;)


What's SuperCollider?

It's an IDE and language (heavily based on Smalltalk) for carrying out real-time audio synthesis and effects processing.  It works best on Mac OSX, and has built-in filters, sequencers, frequency generators, impulse generators, noise gates, and all sorts of goodies to let you do pretty much anything you can imagine with computer sound.  You might be thinking that a purely object-oriented language such as Smalltalk would incur substantial performance penalties (i.e. how the heck is it doing real-time audio synthesis without choking all the time?), but surprisingly, it's not easy to get SuperCollider to hiccup unless you have bugs in complicated code, have memory leaks, etc.


Decorre-what?

Correlation describes the similarity which two things fluctuate with respect to each other.  See the examples in the table below about the three possible relationships:

Relationship Correlation Value When Object 1 Is... When Object 2 Is... Under these conditions
Correlated 0 < x < 1 Length of time of a smoker's habit Chance they will get cancer Any time
Inversely correlated -1 < x < 0 Price of corn Price of beef When there's a drought
Decorrelated 0 The number of children you have How long you live Any time


Basically, things that are correlated or even inversely correlated can exhibit insight into each other because they behave in similar or opposite ways.  Things that are decorrelated bear little resemblance to each other, so such statistics don't really give much insight or meaning into each other.  (See Notes 1 & 2 at the bottom.)


Great.  What's so special about audio decorrelation?

Our brains are wired to interpret sound in particular ways, and can be fooled when we present sound in a different way than it's expecting.  This is where audio decorrelation fits in.  Naturally, there is a predictable (and usually minuscule) phase shift as sound travels past one ear and then the next.  Decorrelation, as described here, takes one channel of audio (a mono signal) and completely alters the phase of a copy of it so you get two channels of audio (a stereo signal) that have massively different characteristics, even though they sound very similar.  The phase (time at which various frequency waves experience maximum amplitude) becomes drastically different between the two channels, and this does not tend to affect the way it sounds to the human ear.

This decorrelation process can be obtained in several ways, to varying degrees, and produces at least five perceptual effects on what you hear:
  • Reduces perceived distortions from constructive & destructive interference between similar sound sources.
  • Causes sound in headphones to sound as if it's coming from right by your ears instead of inside your head.  (This only happens if you listen to mono tracks anyway.)
  • Produces diffuse sound fields, similar to the rich reverberant experience in concert halls.
  • Defeats the precedence effect, a psychoacoustic phenomenon where our auditory system naturally filters out echos shorter than a particular time (50 ms or so).  (You can also defeat the precedence effect by playing your recorded audio backwards.)
  • Prevents sound coming from separate loudspeakers from seeming like it's coming from one speaker as you sit closer to that particular speaker (i.e. image shift).

The Program

The method I use to produce decorrelation is derived from (Kendall 1995), a fascinating paper by the class professor on the effects of and mechanisms to produce decorrelation.  You can read more about how it works by downloading the paper.  DecorreLab also produces several other interesting psychoacoustic effects, including presenting interaural time & intensity delays between the two sound signals.  By changing the time delay and/or volume of the two signals as they reach your ears, you can fool your brain into thinking the sound is coming from different places.  Combine this with the effect of enriching the sound field in general, and you can make your mono recordings sound much more realistic.

Click here to download DecorreLab.  You need the latest version of the SuperCollider audio processing language to run it.

SuperCollider makes it very easy to adjust the parameters such as intensity and delay of audio signals by using its wide array of unit generators.  They can be applied to both channels because the original mono waveform gets copied into another buffer, then the filtering is applied to either or both buffers as prescribed.

As part of this project, I have included two robust convolution kernels with 256 coefficients.  Convolution is the act of applying a filter to a signal, and a kernel is the description of that filter.  They are shown to alter the phase shifts of specific frequencies in vastly different ways from each other while minimally coloring or altering the way it sounds to our ears.  Specifically, their correlation coefficient with respect to each other is approximately 0.034, so the filters bear little resemblance to each other.  As you slide the Correlation slider from right to left, the two channels change from correlated to decorrelated to inversely correlated, and the filters are applied to the two audio channels by the means described in the paper.  On an actual audio sample I tested, the correlation of the two channels with respect to each other after the filters were fully applied was approximately 0.3.  This means that when you slide the "Correlation" slider to the middle (to get 0, fully decorrelated), the audio signals are, in practice, not completely decorrelated.  However, the effect is definitely strong enough to produce all the characteristics mentioned above.  Plus, the amount of correlation you truly get at 0 is likely to change depending on the audio sample.

There are a number of tradeoffs to be made when producing the convolution kernels.  For one, there are not more than 256 kernel coefficients because longer convolution kernels tend to smudge the audio and produce undesirable echo effects.  In order for the precedence effect to be altered to the maximum degree, an impulse (instantaneous burst of energy at all frequencies) convolved with the filter must last less than 20 milliseconds.  However, the number of coefficients must be maximized in order to obtain the greatest decorrelation effect, and has to be constrained to a power of 2 because this greatly optimizes computation time.

There is also a tradeoff to be made between timbral neutrality and impact at low frequencies.  Unfortunately, to have the maximum amount of decorrelation at low frequencies, the filters will have to be built in such a way that will introduce perceptible timbral differences from the original signal (i.e. the filters will make it sound noticeably different).  Changing the sound's timbre may not be desirable.

Finally, when designing the filters, one could consider applying phase shifts linearly across frequency or grouping them into critical band spacing.  (Critical bands dictate the way our inner ear and brain handles various frequencies, and are quite complex to describe.)  Using a linear spacing of phase shifts eliminates interference better, but using a logarithmic spacing (which is how critical bands are spaced) does a better job at creating diffuse sound fields.

The MATLAB code snippet below can be used to create 100 different no-frills filters quickly and provide the lowest correlation coefficient found between any of them:

% The real part of the filter is always 1 in order to keep the sound as close as possible to the same timbre; the imaginary part applies the phase shift.
Hs = 1 + i*(2*pi*rand(256, 100)-pi);  % real (1) + imaginary (i)
lowestConv = 10 + 10i;  % i is the imaginary number, sqrt(-1)
for filter1 = 1:100
    % Calculate 1/2 the matrix to reduce redundancy
for filter2 = filter1 + 1:100
        % Find the cross-correlation vector between the two filters
        c_h = xcorr(ifft(Hs(:,filter1)), ifft(Hs:,filter2)), 'coeff');
        % Find the middle value of the result vector, as this is the correlation value for when the two filters were directly overlapping each other
        c((filter1-1)*100 + filter2) = abs(norm(c_h(256)));
        % Find out if this is lower than what we've seen before
        if (c((filter1-1)*100 + filter2) < lowestConv
            a = filter1; b = filter2;  % a & b are the index of the best filters
            lowestConv = c((filter1-1)*100 + filter2);
        end
    end
end

Side Notes

1. Another key point, especially when thinking of asset correlation, is that correlation of assets changes over time.  The 2007 credit crunch & financial crisis made bundled assets (e.g. loans packaged in big groups & sold to Fannie & Freddie) fall steeply in value.  Originally, the loans inside them were supposed to be decorrelated and thus less risky because usually some people default and some don't.  When the market turned, the loans became correlated because more people were unable to pay them on time.  It is thus unsafe to assume most assets will keep their exact rate of correlation for long.

2. Correlation does not prove causation.  Over time, research has linked smoking to cancer, but many people make preposterous claims simply because they see two things that are related and think one must cause the other.