The Selection Preview section along the bottom of the screen displays how the edges of the current selection line up when repeating the thumbnail contained in the selection, and serves to assist when creating the moulding/mat texture pattern.  

Rotate the selection to match the frame/texture/art angle.  

Click on the selection to put it in focus.

Click on the rotation control point located off of the right side of the selection.

Hold and drag the control point up or down to rotate the selection. 

Release the mouse when the selection is rotated to the desired position.