Develop Your Algorithm

Start by prototyping your algorithm on your local machine. The example uses integer factorization as a sample problem. It is a computationally intensive problem, where the complexity of the factorization increases with the magnitude of the number. You use a simple algorithm to factorize a sequence of integer numbers.

Create a vector of prime numbers in 64-bit precision, and multiply pairs of prime numbers randomly to obtain large composite numbers. Create an array to store the results of each factorization. The code in each of the following sections in this example can take more than 20 min. To make it faster, reduce the workload by using fewer prime numbers, such as2^19. Run with2^21to see the optimum final plots.

primeNumbers = primes(uint64(2^21)); compositeNumbers = primeNumbers.*primeNumbers(randperm(numel(primeNumbers))); factors = zeros(numel(primeNumbers),2);

Use a loop to factor each composite number, and measure the time that the computation takes.

tic;foridx = 1:numel(compositeNumbers) factors(idx,:) = factor(compositeNumbers(idx));endtoc

Elapsed time is 684.464556 seconds.

Run Your Code on a Local Parallel Pool

Parallel Computing Toolbox™ enables you to scale up your workflow by running on multiple workers in a parallel pool. The iterations in the previousforloop are independent, and so you can use aparforloop to distribute iterations to multiple workers. Simply transform yourforloop into aparforloop. Then, run the code and measure the overall computation time. The code runs in a parallel pool with no further changes, and the workers send your computations back to the local workspace. Because the workload is distributed across several workers, the computation time is lower.

tic;parforidx = 1:numel(compositeNumbers) factors(idx,:) = factor(compositeNumbers(idx));endtoc

Elapsed time is 144.550358 seconds.

When you useparforand you have Parallel Computing Toolbox, MATLAB automatically starts a parallel pool of workers. The parallel pool takes some time to start. This example shows a second run with the pool already started.

The default profile is'Processes'. You can check that this profile is set as default on the MATLABHometab, inParallel>Select Parallel Environment. With this profile enabled, MATLAB creates workers on your machine for the parallel pool. When you use the'Processes'profile, MATLAB, by default, starts as many workers as physical cores in your machine, up to your preferred number of workers. You can control parallel behavior using the parallel preferences. On the MATLABHometab, selectParallel>Parallel Preferences.

To measure the speedup with the number of workers, run the same code several times, limiting the maximum number of workers. First, define the number of workers for each run, up to the number of workers in the pool, and create an array to store the result of each test.

numWorkers = [1 2 4 6]; tLocal = zeros(size(numWorkers));

Use a loop to iterate through the maximum number of workers, and run the previous code. To limit the number of workers, use the second input argument ofparfor.

forw = 1:numel(numWorkers) tic;parfor(idx = 1:numel(compositeNumbers), numWorkers(w)) factors(idx,:) = factor(compositeNumbers(idx));endtLocal(w) = toc;end

计算加速比是通过计算en the computation time of a single worker and the computation time of each maximum number of workers. To visualize how the computations scale up with the number of workers, plot the speedup against the number of workers. Observe that the speedup increases with the number of workers. However, the scaling is not perfect due to overhead associated with parallelization.

f = figure; speedup = tLocal(1)./tLocal; plot(numWorkers, speedup); title('Speedup with the number of workers'); xlabel('Number of workers'); xticks(numWorkers); ylabel('Speedup');

When you are done with your computation, delete the current parallel pool so you can create a new one for your cluster. You can obtain the current parallel pool with thegcpfunction.

delete(gcp);

Set up Your Cluster

If your computing task is too big or too slow for your local computer, you can offload your calculation to a cluster onsite or in the cloud. Before you can run the next sections, you must get access to a cluster. On the MATLABHometab, go toParallel>Discover Clustersto find out if you already have access to a cluster with MATLAB Parallel Server™. For more information, seeDiscover Clusters.

If you do not have access to a cluster, you must configure access to one before you can run the next sections. In MATLAB, you can create clusters in a cloud service, such as Amazon AWS, directly from the MATLAB Desktop. On theHometab, in theParallelmenu, selectCreate and Manage Clusters. In the Cluster Profile Manager, clickCreate Cloud Cluster. To learn more about scaling up to the cloud, seeGetting Started with Cloud Center. To learn more about your options for scaling to a cluster in your network, seeGet Started with MATLAB Parallel Server(MATLAB Parallel Server).

After you set up a cluster profile, you can modify its properties inParallel>Create and Manage Clusters. For more information, seeDiscover Clusters and Use Cluster Profiles. The following image shows a cluster profile in theCluster Profile Manager:

Run Your Code on a Cluster Parallel Pool

If you want to run parallel functions in the cluster by default, set your cluster profile as default inParallel>Select Parallel Environment:

哟u can also use a programmatic approach to specify your cluster. To do so, start a parallel pool in the cluster by specifying the name of your cluster profile in theparpoolcommand. In the following code, replaceMyClusterwith the name of your cluster profile. Also specify the number of workers with the second input argument.

parpool('MyCluster',64);

Starting parallel pool (parpool) using the 'MyCluster' profile ... connected to 64 workers.

As before, measure the speedup with the number of workers by running the same code several times, and limiting the maximum number of workers. Because the cluster in this example allows for more workers than the local setup,numWorkerscan hold more values. If you run this code, theparforloop now runs in the cluster.

numWorkers = [1 2 4 6 16 32 64]; tCluster = zeros(size(numWorkers));forw = 1:numel(numWorkers) tic;parfor(idx = 1:numel(compositeNumbers), numWorkers(w)) factors(idx,:) = factor(compositeNumbers(idx));endtCluster(w) = toc;end

Calculate the speedup, and plot it against the number of workers to visualize how the computations scale up with the number of workers. Compare the results with those of the local setup. Observe that the speedup increases with the number of workers. However, the scaling is not perfect due to overhead associated with parallelization.

figure(f); holdonspeedup = tCluster(1)./tCluster; plot(numWorkers, speedup); title('Speedup with the number of workers'); xlabel('Number of workers'); xticks(numWorkers(2:end)); ylabel('Speedup');

When you are done with your computations, delete the current parallel pool.

delete(gcp);

Offload and Scale Your Computations withbatch

After you are done prototyping and running interactively, you can use batch jobs to offload the execution of long-running computations in the background with batch processing. The computation happens in the cluster, and you can close MATLAB and retrieve the results later.

Use thebatch函数来提交您的集群的批处理作业。哟u can place the contents of your algorithm in a script, and use thebatchfunction to submit it. For example, the scriptmyParallelAlgorithmperforms a simple benchmark based on the integer factorization problem shown in this example. The script measures the computation time of several problem complexities with different number of workers.

Note that if you send a script file usingbatch, MATLAB transfers all the workspace variables to the cluster, even if your script does not use them. If you have a large workspace, it impacts negatively the data transfer time. As a best practice, convert your script to a function file to avoid this communication overhead. You can do this by simply adding a function line at the beginning of your script. To learn how to convertmyParallelAlgorithmto a function, seemyParallelAlgorithmFcn.

The following code submitsmyParallelAlgorithmFcnas a batch job.myParallelAlgorithmFcnreturns two output arguments,numWorkersandtime, and you must specify2as the number of outputs input argument. Because the code needs a parallel pool for theparforloop, use the'Pool'name-value pair inbatchto specify the number of workers. The cluster uses an additional worker to run the function itself. By default,batchchanges the current folder of the workers in the cluster to the current folder of the MATLAB client. It can be useful to control the current folder. For example, if your cluster uses a different filesystem, and therefore the paths are different, such as when you submit from a Windows client machine to a Linux cluster. Set the name-value pair'CurrentFolder'to a folder of your choice, or to'.'to avoid changing the folder of the workers.

totalNumberOfWorkers = 65; cluster = parcluster('MyCluster'); job = batch(cluster,'myParallelAlgorithmFcn',2,'Pool',totalNumberOfWorkers-1,'CurrentFolder','.');

To monitor the state of your job after it is submitted, open the Job Monitor inParallel>Monitor Jobs. When computations start in the cluster, the state of the job changes torunning:

哟u can close MATLAB after the job has been submitted. When you open MATLAB again, the Job Monitor keeps track of the job for you, and you can interact with it if you right-click it. For example, to retrieve the job object, selectShow Details, and to transfer the outputs of the batch job into the workspace, selectFetch Outputs.

Alternatively, if you want to block MATLAB until the job completes, use thewaitfunction on the job object.

wait(job);

转移函数的输出从cluster, use thefetchOutputsfunction.

outputs = fetchOutputs(job); numWorkers = outputs{1}; time = outputs{2};

After retrieving the results, you can use them for calculations on your local machine. Calculate the speedup, and plot it against the number of workers. Because the code runs factorizations for different problem complexities, you get a plot for each level. You can see that, for each problem complexity, the speedup increases with the number of workers, until the overhead for additional workers is greater than the performance gain from parallelization. As you increase the problem complexity, you achieve better speedup at large numbers of workers, because overhead associated with parallelization is less significant.

figure speedup = time(1,:)./time; plot(numWorkers,speedup); legend('Problem complexity 1','Problem complexity 2','Problem complexity 3','Problem complexity 4','Location','northwest'); title('Speedup vs complexity'); xlabel('Number of workers'); xticks(numWorkers(2:end)); ylabel('Speedup');