Activation Function – A Cubic Hermite Spline ? This is no Sigmoid!

In the midst of writing my semi-neural net compressor I have introduced a scaling function based off the premise of an Activation function for a neural net – stretch and squash. Because I always work within a normalized domain of 0.0 to 1.0 I was looking for a logarithmic type function that would work similar to a sigmoid to bring my probabilities closer to a best guess or at least a better prediction.

Below is a graph showing a hermite basis function of a Cubic Hermit Spline , which acts as a sigmoid in the domain of 0.0 to 1.0. The function is very simple and results are phenomenal. In comparison I show a linear curve, of course with no scaling and a hyperbolic tangent that obviously is skewed to fit within the domain 0.0 to 1.0. I have also tested other functions such as log sigmoid and double exponential sigmoid with decent results. This simple function results in a logarithmic curve that seems to produce amazing results. Of course outside the domain I use, the function is definitely a fake sigmoid.

The formula is very basic: p(x) = -2x^3+3x^2

* ^ = exponent

Activation Functions


Posted in Articles | Leave a comment

Generic Compression Workflow

I’ve been developing compression techniques for 17 years now and have many unpublished items. One great thing about compression is it never gets old. Data never changes and Moores law does not apply to the field of digital compression. Data compression is bound by speed, memory, compression ratio very similar to the business triangle of quality, speed, and price.

My most recent soon to be published experimental work follows a generic workflow for compression which I believe can be useful for future rapid development and testing.

The below workflow allows for a base accumulator or multiple accumulators, multiple predictors using different accumulators (models). Those predictors would be put through a mixer or number of mixers. After the mixers produce a final refined prediction of the best guess to the encoder or decoder (depending on the input being a compressed or uncompressed file) and outputs the code or original information and continues along the process back to the accumulator.

What are the benefits of this type of model ? So far it allows for rapid testing of different mixing strategies, different models for predicting, and different rates of change for accumulating statistics. This workflow also allows for the creation of a neural network mixed with traditional compression techniques.

An example of this beneficial strategy is that adding a predictor has no impact or change on the accumulator, multiple predictors can be swapped out or tested at anytime. Changing the accumulator will however have an impact on the predictor using that accumulator as well as the mixer.


Posted in Articles | Leave a comment

X-Hex Production Re-started

X-Hex is moving along. I have updated the graphics and started production on my new IMAC. Unfortunately there is a bug with the production software that is causing a delay. I would like to get everything ready to post another video soon (It’s been two years). Let me know what you think.

Posted in Articles | Leave a comment

X-Hex Grid Preview

A little preview of the Grid used in X-Hex. With over 40 3D universes, dynamic HD lighting, and physically based rendering, X-hex has an amazing visual and varied atmosphere.

Click on the images below for a full HD preview.







Posted in Articles | Leave a comment

Why I gave up programming a long time ago as my day job …

Will this get me a job, help my professional network, or make me look smart? Nope, not even close – it’s my blog and a rant about innuendo I get to listen to, so I hope I can answer it gracefully. It will explain a couple of things and also remind me that my life is mine to live, I have one programming boss, it’s me.

I’ve been asked why I gave up programming as my day job for many years, from employers, head hunters, prospective employers, friends, family and people in general who think a day job is a day job regardless. The bottom line and reality is that I have my own shit to program and I like to do it my way, but these are some of the reasons that brought me to the conclusion many years ago.

Employer: I’ve got the best idea in the world
Me: Ideas are a dime a dozen

“I have a great idea, can we (which really means you) make a website where people connect, socialize, post pictures and have likes that I can make billions off of it? Have this done by the end of next week! … Is that website done yet ?”

Are you kidding me?? Talk about over-shooting, I’ve heard this many times over the years and if I could speak my mind, I would have been fired. Expectations of programmers are grossly exaggerated … All non programming employers end up making requests that programming becomes a task of impossibility and a waste of talent, time and cause burnout.

My sarcastic delusional response: “Hold on, let me give my good friend Mark Zuckerberg a call, and then get back to you …”

But your really smart and talented, your skills would be better utilized elsewhere (ie. programming)

This has been a hindrance for me for years, I’ve had to give up three careers, and one potential opportunity, so far in my educational field because I’m ‘super talented’. Employers believe they can choose my Career path, by indirectly providing me with a choice of ‘switch to programming’ or get out mentality. Unless an employer is paying for me and my time to get a University Degree in comp science, my suggestion is to drop it. For employers, I’m a network admin, not a programmer (Even though I kick ass at it). This website has a ring of truth attached to it and I need to buy one of their shirts to support the idea:

Having a learning disability, diagnosed when I was a teenager, this is what resonates with me the most from the websites manifesto (I wish I could just wear this on the front of my shirt):

“We are tired of being told we’re autistic idiots who need to be manipulated to work in a Forced Pair Programming chain gang without any time to be creative because none of the 10 managers on the project can do… Programming, Motherfucker.”

The Cluster F@#k

Last but not least, I don’t like living other peoples drama daily work lives or messed up un-organized routines. The internet is filled with stories of the same caliber and I have worked them. One blog in particular I have to agree with wholeheartedly which describes what I call The Cluster F@#k (every programmer knows what I am talking about):

I don’t have much to say about the above, it is self explanatory. The Cluster F@#k, for me is one of the top reasons I don’t program for other people.

Posted in Articles | Leave a comment

Q Submit – Helpdesk Ticket Submission

I wrote this piece of software to compliment my article about IT Metrics ( This software allows the process of gathering statistics to be enhanced through the efficiency of automation to quickly submit a ticket without having to login to a ticketing system. The program interfaces with a backend (currently spiceworks), and allows the end user to submit multiple tickets one after the other or simply shrink the software to be left out of sight until another ticket comes up.  The current user is the logged in user, email is parsed from Active Directory.

Tickets can be written down throughout the day and entered at a later time, entered as an email,  entered as a phone call comes in, or if an individual walks in requesting service. One of the challenges in IT is to maintain a consistent workflow while also being able to document the work without interruptions. This software has so far been tested successfully in a production environment.


Posted in Articles | Leave a comment

Frequency Coding Draft Paper

I have attached a draft paper explaining the math behind frequency coding, a method of lossless data compression I came up with. The algorithm has been tested as I have freely available computer source code for anyone who wants to test it. The paper still needs to be complete and is missing the following:

  • Math Cleanup – my terminology needs to be re-worked, its vague and hard to understand 🙂
  • Decoding Process – important for paper proof – shown in source code on my website.
  • Probability comparison – chart of actual p(x), frequency encoding output, neural network stretch = ln(x / (1 – x))
  • Calgary Corpus Comparison – compare to other arithmetic encoders on this standard data set

This algorithm is fairly intuitive once you understand the concept behind it.

Here is the PDF document:
Posted in Compression | Leave a comment

X-Hex re-ignited!

X-hex is on for it’s second revision. This is in very early stages but is looking very promising. Currently I am developing it in c# mono develop IDE.  Graphics, Assets, Audio, Music will all be outsourced some how or another. The goal is to provide a simple game to the majority of platforms including windows, mac, Linux, ios, windows store, xbox , ps3, android and ios. The professional development platform allows for the output to many devices with a single project.  The game is currently rendered in isometric 3d using a top down perspective but taking full advantage of awesome graphical effects.



Posted in New Projects | 3 Comments

Voice Assistant

*Updated Version Added – June 8, 2013* – designed for windows 7

Voice Assistant is a voice dictation program I decided to write in order to correct some of the difficulties using existing voice dictation software. When investigating voice dictation and given feedback about the same subject, I noticed a direct correlation to compression and also writing grammars for programming languages. It was then that I realized dictation from Microsoft and Dragon were missing some key elements and were also structuring their dictation wrong. Escape codes happen to be the common ground or link to compression. Dictation and commands require context and with context we can have escape codes to seperate the context properly. With current dictation software there are many escape codes which in turn causes confidence of a correct guess to be reduced substantially. Also visual queues are very weak and the brain is too quick to wait for feedback that is not obvious.  I have taken a logical approach following the footsteps of Star Trek.  The First escape code is “Computer” any other word or sentence is completely ignored.  Also the “computer” talks to you with transparent visual text that show up on the screen.  The Commands are as follows:

  • Computer Dictate
  • Computer Time
  • Computer Date
  • Computer Lock
  • Computer Exit
  • Computer StarDate (Star Trek humour)
  • Computer About
  • Computer Help

When you go into Dictation mode – the voice assistant switches commands –

  • Computer Stop (Stops Dictation, and back to Command Mode)
  • Insert Period
  • Insert Comma
  • Insert Space
  • Insert New-Line
  • Insert Tab
  • Insert Insert
  • Insert Undo
  • Insert Backspace

Dictation mode is somewhat limited but it does function very well, and the reason is that it only uses two escape codes {computer and insert} as opposed to other dictation engines, which tend to mix a large range of different sounding escape codes into the mix.

Voice Assistant (1135) <-(Free Download)

The program requires dot net 4.0 and you must train your computer to recognize your voice and test your microphone.

** Important: Train your computer to recognize your voice first (Microsoft Wizard Built-in)**



Posted in New Projects, Projects | Leave a comment

Frequency Coding

The source code below is a bit coder class that can be used for reading or writing a compressed file. The source code provides access to a new type of entropy encoder. This new type of coder uses an algorithm I have created which I call frequency coding. The class I provide is  currently using two symbols, but is not limited to two. A frequency coder is a variant of a range coder with some minor similarities with huffman coding. Without getting into details, the main difference between a range coder and a frequency coder is that a frequency coder does not require the use of multiplication or division by only using the frequencies of the symbols but still retaining fractions, where a range coder or arithmetic coder use the probability to output a codeword. Huffman coding does not use fractions or at least retain fractions as the code is being created, where a frequency coder does, but also requires a sorted list of frequencies from lowest to highest to properly handle fractions. I will be putting out a paper in the near future that will explain how a frequency encoder can work for any number base.


download -> bitcoder.h (1405)


Copyright (c) 2013 Chris Chunick,
All rights reserved.

Redistribution and use in source and binary forms are permitted
provided that the above copyright notice and this paragraph are
duplicated in all such forms and that any documentation,
advertising materials, and other materials related to such
distribution and use acknowledge that the software was developed
by Chris Chunick  The name of Chris Chunick may not be used to 
endorse or promote products derived from this software without 
specific prior written permission.


class bitcoder

    FILE *fin;
    FILE *fout;    
    unsigned int top,bot,range;
    unsigned int code;
    unsigned int m[32];

    void init()
        bot=0, top=-1;
        for(unsigned int i=0;i<32;i++)m[i]=(0x7FFFFFFF>>(31-i));

    unsigned int passed;
    unsigned int fsize;

    void initcoder(FILE* fin, FILE* fout)

    void startdecoder()
        for(unsigned int i=0;i<4;i++)fsize=(fsize<<8)|getc(fin);
        for(unsigned int i=0;i<4;i++)code=(code<<8)|getc(fin);

    void startencoder()

    void flushencoder()

    void fencode(const unsigned int bit, const unsigned int c0, const unsigned int c1)
        unsigned int lbit=(c0>c1),w=31,lo=c0,tc=c0+c1,tr=top-bot;



            if (lbit^bit)bot=(((bot>>w)+(lo)+1)<<w)|(bot&m[w]);
            else top=(((bot>>w)+(lo))<<w)|(top&m[w]);

        while((top^bot)<=0xFFFFFF || top<bot)

    unsigned int fdecode(const unsigned int c0, const unsigned int c1)
        unsigned int lbit=(c0>c1),w=31,lo=c0,bit,tc=c0+c1,tr=top-bot;


            if (lbit^bit)bot=(((bot>>w)+(lo)+1)<<w)|(bot&m[w]);
            else top=(((bot>>w)+(lo))<<w)|(top&m[w]);
        else top=bot=bot+(bit=code!=bot);

        while((top^bot)<=0xFFFFFF || top<bot)

        return bit;

Posted in Articles, Compression | Leave a comment