Introduction ~ Created Instagram Filter with JavaScript

Introduction ~ Created Instagram Filter with JavaScript

One day, I was learning canvas of HTML5 protocol just for fun. Suddenly, an interesting idea came upon to me. "How about creating instagram or twitte's image-enhancing filters with canvas and js?". Soon after that I began creating such filters with HTML5/CSS3/JS, with a bunch of swings and roundabouts. This post is to introduce how I created the filter and whay I have learnt through the fun work on the weekend.

Personally, the filter is not perfectly imitated yet, so I am welcoming your advice or comments.

Github

<a href="https://github.com/KENJU/instagram_js_filter"></a>

Extract pixels from images with canvas

With getContext('2d') and getImageDate(), you can extranc image pixels like below codes. This is the first step for creating filters.

• HTMLCanvasElement.getContext()
• CanvasRenderingContext2D.getImageData()

Filter.canvas.getPixels = function(img) {
  var canvas,
      context
  ;
  canvas = Filter.canvas.getCanvas(img.width, img.height);
  context = canvas.getContext('2d');
  context.drawImage(img, 0, 0);
  return context.getImageData(0, 0, canvas.width, canvas.height);
};

Grayscale, Sepia, and Mirroring Effects were Easy

As a return of context.getImageData(), you can get an array that is composed of RGB + Alpha data of each pixels.

var pixels = 
  [ 15, 16, 28, 255, // Red, Green, Blue, Alpha of 1st pixel
    22, 51, 59, 255, // Red, Green, Blue, Alpha of 2nd pixel
    33, 55, 59, 255, // Red, Green, Blue, Alpha of 3rd pixel
    ...
  ];

Therefore, if you convert those RGB + Alpha data, you can create a new image with interesting effects as follows.

Example 1: Grayscale

// grayscale
Filter.grayscale = function(pix){
  for (var i = 0, n = pix.length; i < n; i += 4){
    // calculated from NTSC
    var grayscale = pix[i] * .29 + pix[i+1] * .58 + pix[i+2] * .11;
    pix[i]    = grayscale;
    pix[i+1]  = grayscale;
    pix[i+2]  = grayscale;
  };
};

Example 2: Sepia

Filter.sepia = function(pix){
  for (var i = 0, n = pix.length; i < n; i += 4){
    pix[i]    = pix[i]    * 1.07;
    pix[i+1]  = pix[i+1]  * .74;
    pix[i+2]  = pix[i+2]  * .43;
  }
};

Example 3: Horizontal Mirroring

Filter.horizontalMirror = function(pix, width, height) {
  for(var i = 0; i < height; i++) {
    for(var j = 0; j < width; j++) {
      var off = (i * width + j) * 4;
      var dstOff = (i * width + (width - j - 1)) * 4;
      pix[dstOff]   = pix[off];
      pix[dstOff+1] = pix[off+1];
      pix[dstOff+2] = pix[off+2];
      pix[dstOff+3] = pix[off+3];
    }
  }
};

How about other effects???

So far, it was a piece of cake. There are many sample codes on the Web, and the algorhithm is not so difficult at all.

"Okay, let's go on to creating the Instagram filters!"

...

And, you can get RGB + Alpha value
Then, what should I do next?

Amaro Effects, Toaster Effects,
In Photoshop, what you have to do is to manipulate
tone curve...
but can I manipulate tone curve on the web??

...

I am not the professional of image processing,
so at first I did not know how to create
effects which you can get on Photoshop.

So, I did research.

Instagram Filters with CSS3

I found this article, Instagram filters recreated using CSS3 filter effects. In the article, the author is trying to create Instagram filters like me, but only with CSS3 properties. This artcile was interesting and full of insights, but I realized that there is limitation if you use only CSS3 properties.

After that, I found this post, How to make Instagram Filters in Photoshop. In the post, the author is introducing how to create Instagram filters by manipulating tone curve on Photoshop.

In the post, each RGB value was plotted at some particular point, so I realized if I can realise something like this with JS, then I might have be able to create instagram-like filters.

The next step. How to create tone-curve manipulatin with JS?

Lagrange

With the help of this gist, I implementekd lagrange's formula into the filter.

For example, below code is to realize Toaster-like effects onto images.

Filter.toaster()

Filter.toaster = function(pix){

  var lag_r  = new Lagrange(0, 0, 1, 1);
  var lag_g  = new Lagrange(0, 0, 1, 1);
  var lag_b  = new Lagrange(0, 0, 1, 1);

  // red
  var r = [
    [ 0,   0, 120 ], // [index, before, after]
    [ 1,  50, 160 ],
    [ 2, 105, 198 ],
    [ 3, 145, 215 ],
    [ 4, 190, 230 ],
    [ 5, 255, 255 ]
  ];
  // green
  var g = [
    [ 0,   0,   0 ],
    [ 1,  22,  60 ],
    [ 2, 125, 180 ],
    [ 3, 255, 255 ]
  ];
  // blue
  var b = [
    [ 0,   0,  50],
    [ 1,  40,  60],
    [ 2,  80, 102],
    [ 3, 122, 148],
    [ 4, 185, 185],
    [ 5, 255, 210]
  ];

  lag_r.addMultiPoints(r);
  lag_g.addMultiPoints(g);
  lag_b.addMultiPoints(b);

  for (var i = 0, n = pix.length; i < n; i += 4){
    pix[i]    = lag_r.valueOf(pix[i]);
    pix[i+1]  = lag_b.valueOf(pix[i+1]);
    pix[i+2]  = lag_g.valueOf(pix[i+2]);
  }
};

Each Lagrange points are based on rgb values introduced in How to make Instagram Filters in Photoshop.

With below method, each lagrange points are added for creating a smooth line.

addMultiPoints()

Lagrange.prototype.addMultiPoints = function(arr){
  for(var i = 0, n = arr.length; i < n; i++){
    if(arr[i][0] !== 0 && arr[i][0] !== 1){
      this.addPoint(arr[i][1], arr[i][2]);
    }
  }
};

Web Worker API for Optimization

Okay, almost done.
Still, there is one big problem...

This process is too heavy because
this algorithm calculate and convert each pixels' data.

Before

After

Therefore, I made full use of Web Worker API to calculate in the background thread (or multi thred) for non-blocking user experience.

Filter.process = function(img){

  ... 

  // extract pixels data
  pixels = Filter.canvas.getPixels(img);

  // send the pixels to a worker thread
  worker = new Worker('js/worker.js');
  obj = {
    pixels: pixels,
    effects: effect
  };
  worker.postMessage(obj); // send pixels data to worker thread for calculating

  ...
};

Conclusion

At last, I may have be able to create Instagram-like filters with JS.

This might not be the best practice, still it would be great if you give me advice or comments to refactor this filter, or tell me what kind of algorithm is in the real twitter/instagram image-processing.

Thanks.