On the Components tab in Setup Editor right click the .DLL you want to isolate
From the right click menu, select New / Isolated Component. The Isolated Component Details dialog appears where you pick a file for isolation from the feature that contains the component.
If the key path for the current component is not an .EXE, then the drop-down list shows all .EXEs in the containing feature that are key paths of .DLL files.
If the key path for the current component is an .EXE, then the drop-down list shows all files from the containing feature that are key paths other than the current component.
From the Associated File drop-down list, select the .EXE you want to assign to this .DLL.