We present a new parallelization technique that significantly improves performance of certain data-parallel algorithms on heterogeneous clusters of workstations. The two main goals of our technique are to improve execution times (compared to traditional parallelization techniques) and to efficiently use the computing resources available in the cluster. The technique is based on a pre-processing phase where information about the cluster capacity is obtained, a load balanced data decomposition is derived, and information is generated to guide the cluster node utilization during the execution of the parallel algorithm. We applied our technique to Gaussian Elimination and Pair Wise Interaction problems, the experiments show speedup improvements up to 133% and 275% respectively and the cluster utilization efficiency improves up to 180% and 300% when compared with traditional parallelization techniques.