Run the sample app

Make sure you finished the Download and config the sample step, run the app from Bash as follow:

cd src
python batch_python_experiment.py

output

Sample start: 2020-12-05 12:35:50

Container [input-1607168150] created.
Container [output-1607168150] created.
Uploading file /Users/sczho/code/batch-python-experiment/src/inputFiles/job4.txt to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/inputFiles/job1.txt to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/inputFiles/job2.txt to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/inputFiles/job3.txt to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/requirements.txt to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/mlp.py to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/linear_regression.py to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/svm.py to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/random_forest.py to container [input-1607168150]...
Uploading file /Users/sczho/code/batch-python-experiment/src/sourceFiles/boston_house_price.py to container [input-1607168150]...
Creating pool [BatchExperimentPool_1607168150]...
Creating job [BatchExperimentJob_1607168150]...
Adding 4 tasks to job [BatchExperimentJob_1607168150]...
Monitoring all tasks for 'Completed' state, timeout in 0:30:00...........

The job normally will finish in about 8 minutes, most of the time is spent on VM initialization and install dependencies (environment). Create your own custom VM image can accelerate this step. While waiting for the tasks to be completed, we can go to Azure Portal and check all the resources created by the program:

• Storage account

  • input-timestamp - the container for upload inputs and app

  • output-timestamp - the container for store task output

• Batch account

  • BatchExperimentPool_timestamp - the VM instance pool

  • BatchExperimentJob_timestamp

    • 4 tasks - to execute our Python program

After tasks complete, you see output similar to the following:

Success! All tasks reached the 'Completed' state within the specified timeout period.
Printing task output...
Task: Task0
Node: tvmps_d458be0321e514a85a1967f4eb9c370bf56c2771e0dc30dd99bc48666b67f612_p
Standard output:
...........
Model Evaluation on training data
R^2: 0.6419097248941195
Adjusted R^2: 0.628218037904777
MAE: 2.9361501059460293
MSE: 26.953752101332935
RMSE: 5.191700309275655

Model Evaluation on test data
R^2: 0.5900158460478174
Adjusted R^2: 0.5513941503856553
MAE: 3.7561453553021686
MSE: 42.81057499010247
RMSE: 6.542979060802691
The accuracy is 59.00158460478174

Error output:
.............

Output file job4output.txt:
svm
R-squared Score: 59.00158460478174

...........

Evaluation and comparision of all the models:
Model                    R-squared Score
random forest            82.86251408071476
multi-layer perceptron   78.65179140389007
linear regression        71.21818377409195
svm                      59.00158460478174

Sample end: 2020-12-05 12:42:46
Elapsed time: 0:06:56

And clean up resources:

Delete input container? [Y/n] 
Deleting container [input-1607168150]...
Delete job? [Y/n] 
Delete pool? [Y/n] 
Delete output container? [Y/n] 
Deleting container [output-1607168150]...

Press ENTER to exit...

If you choose to keep the storage container and pool to observe the config and data, don't forget to delete them later, Azure will charge you for the pool and storages.

When no longer needed, delete the resource group, Batch account, and storage account. To do so in the Azure portal, select the resource group for the Batch account and select Delete resource group.