boostworthyisryantaylor

FP10: Pixel Bender DisplacementMapFilter

June 01st, 2008 | Category: Dirty South, Code

The Flash API already includes a filter for displacement maps; however I created my own Pixel Bender equivalent for the sake of testing and experimentation. Below is the source, followed by a few examples of how it can be implemented in Flash.

CODE:
  1. // *****************************************************************************************
  2. // DisplacementMapFilter.pbk
  3. //
  4. // Copyright (c) 2008 Ryan Taylor | http://www.boostworthy.com
  5. //
  6. // Permission is hereby granted, free of charge, to any person
  7. // obtaining a copy of this software and associated documentation
  8. // files (the "Software"), to deal in the Software without
  9. // restriction, including without limitation the rights to use,
  10. // copy, modify, merge, publish, distribute, sublicense, and/or sell
  11. // copies of the Software, and to permit persons to whom the
  12. // Software is furnished to do so, subject to the following
  13. // conditions:
  14. //
  15. // The above copyright notice and this permission notice shall be
  16. // included in all copies or substantial portions of the Software.
  17. //
  18. // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  19. // EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
  20. // OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  21. // NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
  22. // HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
  23. // WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
  24. // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
  25. // OTHER DEALINGS IN THE SOFTWARE.
  26. // *****************************************************************************************
  27. //
  28. // +          +          +          +          +          +          +          +          +
  29. //
  30. // *****************************************************************************************
  31.  
  32. <languageVersion : 1.0;>
  33.  
  34. kernel DisplacementMapFilter
  35. <   
  36.     namespace   : "Boostworthy::Filters";
  37.     vendor      : "Ryan Taylor";
  38.     version     : 1;
  39.     description : "Standard filter for displacing source pixels based on map color values.";
  40. >
  41. {
  42.     parameter   int2        component
  43.     <
  44.         minValue        : int2(0, 0);
  45.         maxValue        : int2(3, 3);
  46.         defaultValue    : int2(1, 2);
  47.     >;
  48.  
  49.     parameter   float2      offset
  50.     <
  51.         minValue        : float2(-256.0, -256.0);
  52.         maxValue        : float2(256.0, 256.0);
  53.         defaultValue    : float2(0.0, 0.0);
  54.     >;
  55.  
  56.     parameter   float       scale
  57.     <
  58.         minValue        : -1.0;
  59.         maxValue        :  1.0;
  60.         defaultValue    :  1.0;
  61.     >;
  62.  
  63.     input       image4      source;
  64.     input       image4      map;
  65.     output      pixel4      result;
  66.    
  67.     void evaluatePixel()
  68.     {
  69.         float2  coord       = outCoord();
  70.         pixel4  mapPixel    = sampleLinear(map, coord);
  71.        
  72.         // Note: The following statements are used for the sake of
  73.         // Flash compatibility. mapPixel[component[0]], for instance,
  74.         // is not allowed because the index must be a constant.
  75.        
  76.         float   componentX;
  77.        
  78.         if(component[0] == 0)
  79.             componentX = mapPixel[0];
  80.         else if(component[0] == 1)
  81.             componentX = mapPixel[1];
  82.         else if(component[0] == 2)
  83.             componentX = mapPixel[2];
  84.         else if(component[0] == 3)
  85.             componentX = mapPixel[3];
  86.            
  87.         float   componentY;
  88.        
  89.         if(component[1] == 0)
  90.             componentY = mapPixel[0];
  91.         else if(component[1] == 1)
  92.             componentY = mapPixel[1];
  93.         else if(component[1] == 2)
  94.             componentY = mapPixel[2];
  95.         else if(component[1] == 3)
  96.             componentY = mapPixel[3];
  97.        
  98.         float   x           = coord.x + (componentX - 0.5) * offset[0] * scale;
  99.         float   y           = coord.y + (componentY - 0.5) * offset[1] * scale;
  100.        
  101.         result  = sampleLinear(source, float2(x, y));
  102.     }
  103. }

NOTE: It is important to make sure that the main source image is the first input. Flash's ShaderFilter class makes this assumption and automatically passes it's input to the input at index 0.

In Flash, you can embed the 'DisplacementMapFilter.pbj' file at compiletime or keep it external and load it at runtime using a Loader instance. Regardless of the approach, you then feed the filter's byte code to a Shader object as seen below.

ActionScript:
  1. var sourceImage:Bitmap = Bitmap(new SourceImage());
  2. var mapImage:Bitmap = Bitmap(new MapImage());
  3.            
  4. var shader:Shader = new Shader(new DisplacementMapFilter());
  5. shader.data.source.input = sourceImage.bitmapData;
  6. shader.data.map.input = mapImage.bitmapData;
  7. shader.data.component.value = [1, 2];
  8. shader.data.offset.value = [50, 190];
  9. shader.data.scale.value = [1];

Once you have set the inputs and tweaked any parameters, one way to render the shader is to draw it as a fill using the Flash drawing API.

ActionScript:
  1. var container:Sprite = new Sprite();
  2. container.graphics.beginShaderFill(shader);
  3. container.graphics.drawRect(0, 0, sourceImage.width, sourceImage.height);
  4. container.graphics.endFill();
  5. addChild(container);

Alternatively, you can also create a ShaderFilter and apply it to a BitmapData object.

ActionScript:
  1. var containerData:BitmapData = new BitmapData(sourceImage.width, sourceImage.height);
  2. var container:Bitmap = new Bitmap(containerData);
  3.            
  4. var displacementMapFilter:ShaderFilter = new ShaderFilter(shader);
  5. var filterRect:Rectangle = new Rectangle(0, 0, sourceImage.width, sourceImage.height);
  6. var filterPoint:Point = new Point(0, 0);
  7.            
  8. containerData.applyFilter(sourceImage.bitmapData, filterRect, filterPoint, displacementMapFilter);
  9.            
  10. addChild(container);

Unlike the other filter classes, the ShaderFilter class is not final - you can create subclasses for interfacing with your custom Pixel Bender filters. Performance-wise, the fill approach seems to beat out the use of a filter slightly, though that may change by the final release of Flash Player 10 as they continue to optimize things.

In terms of usage, the filter accepts two image inputs; one for a source image and one for the displacement map. The map's colors dictate the displacement of the pixels. By default, the component used for X is the green channel and for Y the blue channel. A channel value of 128 translates to 0 movement along the corresponding component's axis, where as 0 is -1, 255 is 1, and everything in between is smooth steps. This value is then multiplied against an offset and scale value before being applied as the final displacement. Here is an example:

Source image:

Pixel Bender DisplacementMapFilter

Map image:

Pixel Bender DisplacementMapFilter

Source image with displacement applied {component:[1, 2], offset:[50, 190], scale:[1]}:

Pixel Bender DisplacementMapFilter

Map image overlaid 50% over displaced source image:

Pixel Bender DisplacementMapFilter

Also available on Adobe Labs.

6 Comments so far

  1. satya July 28th, 2008 7:14 am

    very nice

    I need an image to be applied on a spherical object. the spherical object is in an image but need another image to create effect like the second image is applied only to the spherical object in the first image

    Please guide me how can I achieve it

    Thanks in advance
    Satya

  2. gabriel October 15th, 2008 2:00 pm

    Hi your work is amazing, I was wondering if you could make something like this,
    http://img374.imageshack.us/my.php?image=distortdr6.jpg
    it's done with a Photoshop filter called "Lens Correction"
    Please contact me

    Cheers.

  3. Jake February 13th, 2009 4:31 pm

    Hey, just wanted to let you know that this posting was fantastic - if nothing else, the displacement code makes it a lot easier to develop my displacement bitmaps, cutting down on the hours of trial and error it usually takes me to get it 'just right'. Thanks for the examples!

  4. Philip Thonbo April 2nd, 2009 11:37 am

    to start of with im gonna title my self as displacementMapFilter expert - i have been spending alot of time the last years putting my mind into how the maps should be perfectly designed to work perfect

    i did some labs:

    http://www.thonbo.com/tempShowCase/lensLab.html
    http://www.thonbo.com/filtersOnTheRun/
    http://thonbo.com/tempShowCase/loader6_mc.html
    http://www.thonbo.com/tempShowCase/push2Pinch.html
    http://thonbo.com/tempShowCase/displaceTest4.html (dynamic map)
    http://thonbo.com/tempShowCase/sliceisnice.html
    http://www.thonbo.com/tempShowCase/sphear3.html

    This is amazing news - when i was at max at milan where i think kevin goldsmith talked about pixelbender i told him that after his session i planned to do a displacementmap shader with 2 input images a map and an image - and then take the values from the map and so on but the kernal language was abit tricky so i quickly abandondoned the project because of time issues and such - but to see its finally came to life is amazing - i hope i can write you later on if any special requests come to my mind - but for now i will test some of my filters on your shader and post some links when im done - GREAT

  5. Philip Thonbo April 2nd, 2009 12:25 pm

    i believe an important parameter thats missing is the point value and maybe the mode - if the map is smaller than the image the pixels outside should not be displaced so the color value outside the map should be read as 0.5 (neutral) by default - and the scale should have a larger magin but that easy to fix :D

  6. marc June 14th, 2009 2:21 am

    how does the performance of your shader displacement filter compare with the built in player filter?

    im thinking it shouldnt be any faster, but is it??

    sorry... i know this is an old post.... nice work by the way...
    =)

Leave a reply

*
To prove you are human (not an imperial spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Click to hear an audio file of the anti-spam word