GIMP - Out of Bounds (OOB) Tutorial

Have you ever noticed that when you get to a tutorial on how to do some cool trick with graphics software, the test image it uses is always selected for being an easy example? The tutorial works well for the demo image. The demo image is usually a clean, simple composition that's easy to manipulate. But in real life it isn't always that way. In real life, you don't always get to pick the photo; sometimes the photo picks you.

before

So, today we will look at taking this photo and creating an out-of-bounds effect, sometimes also called an "out of frame" or "pop-up".

Open the image in Gimp and use the magnify tool to click once to zoom it in.

Right-click, choose layer->transparency->add alpha channel. This makes it so the image can move freely though the layer stack.

Now off the right-click image menu pick dialog->layers. We'll need that layers dialog open through the tutorial.

In the layers dialog, choose the farthest-left button for a new layer. Pick 'black' for the layer fill color and click OK.

In the layers dialog, click the down arrow to sink the white layer under the image. Then click on the new layer button again, and this time select 'transparency'. This time click the up arrow in the layers dialog to move the transparent layer on top of the background image.

In the layer dialog, look to the left where the eye icons are. Click the eye icon next to the background image, so it's not shown for now. Lastly, click the top transparent layer to select it as your work area.

We're now going to create a distorted white border to serve as our picture frame. In the Gimp main dialog, select the paint bucket tool. Notice in the lower-left corner of the Gimp main dialog whether the white color is in the far left box (and hence the fill color) and change it if it isn't by clicking the little arrow in that corner to switch white to the foreground fill color. Click in the image to fill it with white.

Here's how you should be set up so far...

shot1

Now hit 'Control-I' to select the whole field. In the Gimp main dialog, select the 'change perspective' tool. Click this in the image and you'll be greeted with the perspective dialog. In the image, grab and drag the selected box until the corners of the frame are where you want it. Click 'transform' in the transform dialog to complete the operation.

In the Gimp main dialog, click the top-left dotted box, then in the image window click to set the selection down into the top (transparent) layer.

We're going to have to re-select the area to turn our layer into a floating selection for the next step. So in the Gimp main dialog select the 'magic wand' tool in the top row and in the image window click inside the white oblong to select it.

I'd like to know how it works for you. See the contact us page and send me an email.

Now right-click where you are and choose select->shrink. A dialog will pop up asking how much you want to shirk it by. Pick '20' pixels and click 'OK'. You should now have a selected area inside your white oblong. Hit 'Control-K' to clear the smaller area of white. Click anywhere outside the selection in the image window to set it down.

In the layers dialog, click the eyeball next to the background image to turn it back on. You now have the frame on top of the image, and in this case we're going to do it so the horses are drawing the rider out of the frame.

shot2

Now in the layers dialog, click the eyeball next to the frame to turn it off for now, and click the background image to select it.

Our next step is to create the image mask layer, and it's going to be simpler if we use a separate layer to mask the background from the horse. In the image, right-click->image->mode->decompose. The decompose dialog will show up, and 'rgb' should be selected by default. Click OK and you will get a new image window with the decomposed image.

The decomposed image is three black-and-white renderings of the source image, with each layer drawn from the red, green, and blue channels. You will be able to select and de-select each of these channels in your still-open layers dialog. Try viewing each layer individually to find the clearest version of the image. In this case, green seems the easiest to work with, so we're going to select the red and blue channels in the layers dialog and click the trash can icon in the lower right of the layers dialog to delete each of them, leaving just the green channel.

With the green channel of the black-and-white image selected, hit 'Control-C' to copy it. You can close this image now; you don't need to save it.

Back in our main image, use the layers dialog to make another new transparency layer. In the image window hit 'Control-V' to paste the black-and-white image into our new transparency layer. Hit 'Control-H' to anchor it down.

You should be set up like this...

shot3

Now comes the tedious part. Because we have no method to break the horses image out from the background by color contrast, clear line definition, or other tricks, we're going to build our layer mask from scratch by using the eraser to clear the black-and-white off of the horses. Use the magnify tool in the Gimp main dialog to zoom in over the part you're ready to work on, and also in the main dialog click the eraser tool.

The eraser tool is subject to the same brush sizes as the paintbrush and other drawing tools. So in the main dialog, click the lower-right brush pattern and bring up the dialog. Here, you can select different sizes of brush. I recommend the large hard circle for the general area, and a small circle with a fuzzy border for use around the edges. Switching between various brush sizes, try to fill in the horses in color while leaving their background black-and-white.

shot4

This part is entirely up to you and your mouse skills. Remember to use short brush strokes, so that if you make a mistake, you can hit 'Control-Z' and undo it. Occasionally, you can use the layers dialog eyeball buttons to switch on and off different parts of the image, to get different views of which area you need to cover. Parts of this image have shadowed areas that are going to be hard to trace exactly. Remember that in a later step, we can fix a great deal more detail, and repair minor mistakes. Do as much as you can with larger brushes before switching to smaller ones.

When you get done masking in (more or less) the rest of the target areas, move over by the frame. In the layers dialog, select the frame (top) layer and click in the slider at the top to adjust the opacity to about 40-50%, enough so you can clearly see both the frame and the image under it. Click back to the black-and-white later and continue erasing inside the frame. You don't have to worry about anything past the extreme inside edge of the frame, since our source image will fill that.

shot5

Our next step will be to convert the black-and-white image layer into a solid mask. In the layers dialog, switch off the frame layer and the color image, leaving only the black-and-white layer and black layer active. Be sure the black-and-white layer is selected. In the main Gimp dialog, select the magic wand tool. In the main image, click in our black-and-white outline of the horses.

shot6

Now hit 'Control-I' to invert the selection. In the main Gimp dialog select the bucket-fill tool. At the color selection in the lower left, click the arrow again so the top fill color is black. Use the bucket to click in the main window outside of the horses outline. You will have a black-on-black image with a dotted line around the horses. Hit 'Control-C'.

In the layer dialog, select the original image layer and click the duplicate layer again. Right-click on this background copy and choose 'add layer mask'. Leave it at 'white full opacity' and click OK. You will now have the mask part of the background copy selected. In the image window, hit 'Control-V, Control-H' to paste and anchor the mask into the mask layer.

Click the eyeball icon next to the masked layer to make it visible. You should now see just your horses...

shot7

If all looks well, we can go back to the layers dialog and click the old black-and-white layer and click the trash can to delete it. We are going to use a similar method to get the rest of the source photo within the frame. So, click the eyeball icons in the layers dialog so that just the frame and the original (unmasked) source image layer is visible. Now select the top frame layer and move the slider back all the way to 100% opacity.

In the layers dialog, click the eyeball next to the source image layer to turn it off temporarily. In the main dialog, select the magic wand tool and click in the image inside the white frame. Now in the layer dialog select the source image layer and in the image window hit 'Control-C' to copy that section of the source image inside the frame. In the layers dialog select the frame again and then in the image window hit 'Control-V, Control-H' to paste it into the frame. In the layers dialog with the frame layer still selected, click the down arrow to sink it below the masked horses layer.

You should now be able to click the eyeball icons so that everything but the original image layer is visible and see this:

shot8

From here on out, we're going to touch up the horses outline, fixing any mistakes, using the more common method of filling in the mask with the paintbrush tool and erasing more of the mask with the eraser tool. In the layers dialog, select the frame layer and move the slider so its opacity is about 70%, so you can see what you need to bring in front of the frame. Select the original background copy and move its slider to opacity 30%, so the image part outside the frame and horses fades and you can more easily see what's masked and not. Finally, select the top, masked layer and be sure you've selected the right half of it, to draw in the mask.

Once again, we use the magnifier to zoom in and use the eraser tool to brush in any parts of the image we need to add. The reins here, for instance:

shot9

Go around the outlines of each part and fix as much as you can. For seeing what you need to include, keep the original layer visible and use the eraser to brush away more of the mask. For seeing what you need to take away from the outline, turn off the original background so the horses are plain against the black background and use the paintbrush (keeping the black color selected) to fill in more mask over it. For getting rid of the pixelated edge, use the 'fuzzy circle' brushes for both eraser and brush, gently blurring the edge so that it's softer. For a low-resolution image you're going to have some digital distortion in some places, and there isn't much you can do about it.

When you are confident that you have the last of the touch-up done, go to the layers dialog and select the top layer. Right-click and select 'apply later mask'. Remember that you cannot edit the layer mask after this step, so if you want to save it for later you can save the whole process up til now as an .xcf image (before applying the mask) and you will be able to re-open it and tweak it some more if need be.

Also in the layers dialog, delete the original source image layer, turn all eyeball icons so they're switched on to visible, and return all layer opacities to 100%. While selecting the bottom layer, you may now want to turn it to some other color except black. A neutral gray will help the image feel more settled.

You should now be here:

shot10

Now for an easy, optional after-affect: add a drop shadow to the horses layer. Select it in the layers dialog, then right-click->script-fu->shadow->drop shadow. In the drop-shadow dialog, uncheck 'allow resizing' (it's always on by default), because we don't want to break our image frame. The rest of the shadow preferences don't matter because we're going to squash it down anyway. Click 'OK' on the drop-shadow dialog and let it generate the new shadow layer.

In the layers dialog, select the new shadow layer. In the main Gimp dialog, pick the scale tool and click the shadow in the image window. Similar to the tool we used to distort the frame, the scale tool will give us a box which we can click and drag. Squash the shadow down until it's flat. Then in the Gimp main dialog select the shear tool right next to it. Use it in the same manner, to tilt the shadow so it matches the angle of the shadow under the rider.

In the Gimp main dialog, select the 'move' tool and in the image window, use the arrow keys to position the shadow so it matches up with the horses hooves. And finally, in the layers dialog select the shadow layer and adjust the opacity until it matches the shadow tones within the frame.

If it's all to your liking, in the image window right-click->image->flatten image and save it. You're done!

after

I'd like to know how it works for you. You can find my email address on the contact us page